How packages will work

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

How packages will work

gokr
Hi!

I was dead tired last night and had to do some company book keeping, but
let me reiterate a bit about how packages will work "to begin with" (we
can always go further later on):

- Amber has *Packages* (there are no class categories anymore). A
package is a group of classes + loose method extensions in other
classes, like Monticello.

- Packages are today stored in three different files:
    js/Foo.js        - this is compiled js + embedded st source.
    js/Foo.deploy.js - like above, but without embedded st source.
    st/Foo.st        - only st source, in chunk format.

    NOTES:
      - Foo.deploy.js is totally redundant. Just nice for deployment.
      - Foo.js is "all you need". If loaded into Amber and recommitted -
the other two files will be reconstructed.
      - Foo.st is also "all you need". The js file can be produced by
compiling Foo.st using amberc from the command line.
        Or you can copy/paste the contents of Foo.st into a Workspace,
select and push "File in" button.

- When Amber is loaded from an index.html it will only load the Amber
packages. Extra packages need to be listed explicitly in index.html, see
examples/presentation/index.html etc.

- A class knows its Package instance.

------------------------------
Now, upcoming stuff:

- A Package instance will have a dictionary that holds meta data, and
the most important meta data is probably "comment" and "dependencies".
This information is saved first in the js file and in the st file. Code
for this is almost ready to be committed.

- A PackageResolver will be implemented that *finds and downloads*
packages by name by looking in different places, in order. So given
"Foo" it will look in place A, B, C etc until it finds Foo.js. There
will be no version aspect in this to begin with.

- A PackageLoader will be implemented that *loads packages and their
dependencies*. When we execute "Smalltalk current loadPackage: 'Foo'" it
will use PackageResolver to find and download Foo.js. Then PackageLoader
will evaluate the package information (but nothing more) in Foo.js so
that it creates the Package instance for Foo. Then we look at its
dependencies - a list of Package names, say Bar and Baz. We then proceed
recursively to find, download and scan those too.

- PackageLoader will, for all scanned packages from above:
        - Select those that have all dependencies loaded (or the dependencies
are in our install queue) and add them to our install queue.
        - Rinse and repeat the above until all scanned packages are in the queue.
        - Present queue to the user and ask for "Go ahead" etc.
        - Process queue. This loads the packages one by one.



Ok, so that is the "sketch" for our Package system. Does it look descent?

regards, Göran
Reply | Threaded
Open this post in threaded view
|

Re: How packages will work

drush66
How do you see, what form of package gets stored and versioned into the git?

I see that some people have started storing .js files, which in one way is natural choice since that is what amber development needs to load. Problem with this is that git diff gets pretty meaningless since it displays a lot of javascript diffs. Also merge conficts could appear in double size, once in compiled javascript, once in smalltalk that is stored inside compiled javascript.

So, I think .st form of a package would be most natural to be stored in git. Is it possible to import directly .st file from html? Or to have import facility within the IDE?

Davorin Rusevljan
http://www.cloud208.com

Reply | Threaded
Open this post in threaded view
|

Re: How packages will work

gokr
On 09/29/2011 12:04 PM, Davorin Ruševljan wrote:
> How do you see, what form of package gets stored and versioned into the git?
>
> I see that some people have started storing .js files, which in one way
> is natural choice since that is what amber development needs to load.
> Problem with this is that git diff gets pretty meaningless since it
> displays a lot of javascript diffs. Also merge conficts could appear in
> double size, once in compiled javascript, once in smalltalk that is
> stored inside compiled javascript.

Yes, it is a tad cumbersome at the moment. But we *need* to have js
files in the clone because we have no way of "boot strapping" since the
Compiler etc is written in Amber itself. ;)

But the st files are of course "King" so if the js files gets "messy" to
merge then I tend to nuke them, fix merge in the st file, then recreate
js files using amberc - and then commit them all.

> So, I think .st form of a package would be most natural to be stored in
> git.

Yes. Especially for "non core Amber" packages. Amber itself is a tad
more tricksy :)

And also - yes - PackageResolver/PackageLoader could load st files too,
but then it needs to first load Compiler package etc (which of course is
already in the Amber IDE, but you could be using PR and PL from a
deployed app *without* the Compiler package loaded).

> Is it possible to import directly .st file from html? Or to have
> import facility within the IDE?

Currently there are two ways of getting an st file into Amber:

1. Use amberc from command line and compile the st file into a js file,
which then can be loaded using the index.html file etc. If you are
hacking in Amber itself you can use the "make clean && make && make
install" to compile Amber's st files into js files etc.

2. Copy/paste the contents of an st file, paste into a Workspace, then
select it all with mouse and press "File in" button. :)

We could of course add some "upload" mechanism etc, but the copy/paste
works pretty easily.

regards, Göran
Reply | Threaded
Open this post in threaded view
|

Re: How packages will work

Nicolas Petton
In reply to this post by gokr
On Thu, 2011-09-29 at 10:57 +0200, Göran Krampe wrote:
> Ok, so that is the "sketch" for our Package system. Does it look descent?

Göran,

As usual, that's quite a read, and it looks great!

Nico


> regards, Göran


Reply | Threaded
Open this post in threaded view
|

Re: How packages will work

Nicolas Petton
In reply to this post by drush66
On Thu, 2011-09-29 at 03:04 -0700, Davorin Ruševljan wrote:
> So, I think .st form of a package would be most natural to be stored
> in git. Is it possible to import directly .st file from html? Or to
> have import facility within the IDE?

For speed reasons, I don't want .st files to be compiled on the fly.
That would be way too slow for a web application.

Now you can always "file in" a st file in the workspace.

Cheers,
Nico

Reply | Threaded
Open this post in threaded view
|

Re: How packages will work

drush66
On Thursday, September 29, 2011 1:10:13 PM UTC+2, Nicolas Petton wrote:

That would be way too slow for a web application.

For speed reasons, I don't want .st files to be compiled on the fly.

Now you can always "file in" a st file in the workspace.


that is understood, deployed web app would always include .js file. But on the development side life would be simpler if ide could load .st just as well.

As I said main motivation is here that we need to put the smalltalk source of the package into the git not compiled javascript since otherwise reading and understanding history of such package is a mess. In order to ease/motivate storing of .st files into the git, it would be nice that IDE could load also load .st, and compile it on the fly. Invoking command line would work, but it is a bit of a bore, and copy and pasting all .st files into the workspace after the git pull is a chore

Not a big deal, but I think it would ease things a bit.

Davorin Rusevljan
http://www.cloud208.com/

Reply | Threaded
Open this post in threaded view
|

Re: How packages will work

gokr
On 09/29/2011 01:39 PM, Davorin Ruševljan wrote:

> On Thursday, September 29, 2011 1:10:13 PM UTC+2, Nicolas Petton wrote:
>
>     That would be way too slow for a web application.
>
>     For speed reasons, I don't want .st files to be compiled on the fly.
>
>     Now you can always "file in" a st file in the workspace.
>
>
> that is understood, deployed web app would always include .js file. But
> on the development side life would be simpler if ide could load .st just
> as well.
>
> As I said main motivation is here that we need to put the smalltalk
> source of the package into the git not compiled javascript since
> otherwise reading and understanding history of such package is a mess.
> In order to ease/motivate storing of .st files into the git, it would be
> nice that IDE could load also load .st, and compile it on the fly.
> Invoking command line would work, but it is a bit of a bore, and copy
> and pasting all .st files into the workspace after the git pull is a chore
>
> Not a big deal, but I think it would ease things a bit.

I agree and it should be trivial to implement. We do have some
interesting pull requests to go through also that might affect these
things too.

regards, Göran
Reply | Threaded
Open this post in threaded view
|

Re: How packages will work

drush66
In reply to this post by gokr
On Thursday, September 29, 2011 12:52:15 PM UTC+2, Göran Krampe wrote:

but then it needs to first load Compiler package etc (which of course is
already in the Amber IDE, but you could be using PR and PL from a
deployed app *without* the Compiler package loaded).

Hm do you need package dependecy resolver on the deployed web app? or it would just have precompiled .js files there to load without fuss. It seems to me this is mostly development stuff.

Regarding package dependency, and all that stuff. I know you guys like to take an opportunity to make fresh approach to all things, but have you considered Metacello or something close to it?  Metacello seems to be quite good at handling configurations and dependencies, and in spite the name does not seem to be too deeply married to Monticello, it seems to me that it could be made to wo with git based package as well?

Lastly, have you considered to change .st format so that package ends in directory, with each class beeing represented by a separate file, and one file for intialization, meta info etc, and maybe one file for loose methods? That might be nicer to look in the git repo.

Davorin

And also - yes - PackageResolver/PackageLoader could load st files too,

> Is it possible to import directly .st file from html? Or to have
> import facility within the IDE?

Currently there are two ways of getting an st file into Amber:

1. Use amberc from command line and compile the st file into a js file,
which then can be loaded using the index.html file etc. If you are
hacking in Amber itself you can use the "make clean && make && make
install" to compile Amber's st files into js files etc.

2. Copy/paste the contents of an st file, paste into a Workspace, then
select it all with mouse and press "File in" button. :)

We could of course add some "upload" mechanism etc, but the copy/paste
works pretty easily.

regards, Göran

Reply | Threaded
Open this post in threaded view
|

Re: How packages will work

gokr
Hey!

On 09/29/2011 01:48 PM, Davorin Ruševljan wrote:
> On Thursday, September 29, 2011 12:52:15 PM UTC+2, Göran Krampe wrote:
>     but then it needs to first load Compiler package etc (which of
>     course is
>     already in the Amber IDE, but you could be using PR and PL from a
>     deployed app *without* the Compiler package loaded).
>
> Hm do you need package dependecy resolver on the deployed web app? or it
> would just have precompiled .js files there to load without fuss. It
> seems to me this is mostly development stuff.

Yes, it is mostly dev stuff. And I think that the "normal" approach
would be to "link" together all used js libs when deploying an app, so
then the problem is "moot" at deployment.

I agree that using PR/PL without having Compiler is a far fetched scenario.

> Regarding package dependency, and all that stuff. I know you guys like
> to take an opportunity to make fresh approach to all things, but have
> you considered Metacello or something close to it? Metacello seems to be
> quite good at handling configurations and dependencies, and in spite the
> name does not seem to be too deeply married to Monticello, it seems to
> me that it could be made to wo with git based package as well?

*Personally* I think Metacello is a fat overkill for Amber. I want to
keep things much simpler.

> Lastly, have you considered to change .st format so that package ends in
> directory, with each class beeing represented by a separate file, and
> one file for intialization, meta info etc, and maybe one file for loose
> methods? That might be nicer to look in the git repo.

I haven't been thinking in these directions, I am fine with one file per
package. But I am also not against other approaches.

One thing I do plan to work on is porting Delta from DeltaStreams into
Amber. That is not a "snapshot of source" - it is instead a model of
changes. I want to use that together with local storage etc to give us
real undo, stash etc etc.

regards, Göran
Reply | Threaded
Open this post in threaded view
|

Re: How packages will work

laurent laffont
In reply to this post by gokr
- A PackageResolver will be implemented that *finds and downloads* packages by name by looking in different places, in order. So given "Foo" it will look in place A, B, C etc until it finds Foo.js. There will be no version aspect in this to begin with.

I have not found the time to dig into this but what I need (I think maybe better idea will come) would be to specify prefix per group of packages in loadAmber. For example:

loadAmber( {  packages: { 'some/path/' =>  ['package1', 'package2'] ,   'other/path' => ['package3', 'package4'] },
                     deploy: false, .... } );

and that commit always PUT at the right place.

Laurent

 

- A PackageLoader will be implemented that *loads packages and their dependencies*. When we execute "Smalltalk current loadPackage: 'Foo'" it will use PackageResolver to find and download Foo.js. Then PackageLoader will evaluate the package information (but nothing more) in Foo.js so that it creates the Package instance for Foo. Then we look at its dependencies - a list of Package names, say Bar and Baz. We then proceed recursively to find, download and scan those too.

- PackageLoader will, for all scanned packages from above:
       - Select those that have all dependencies loaded (or the dependencies are in our install queue) and add them to our install queue.
       - Rinse and repeat the above until all scanned packages are in the queue.
       - Present queue to the user and ask for "Go ahead" etc.
       - Process queue. This loads the packages one by one.



Ok, so that is the "sketch" for our Package system. Does it look descent?

regards, Göran

Reply | Threaded
Open this post in threaded view
|

Re: How packages will work

Nicolas Petton
Laurent,

using prefixes is possible for loading, but yes, not yet for PUT
requests. But now that we have packages this will change :)

Cheers,
Nico

On Thu, 2011-09-29 at 15:39 +0200, laurent laffont wrote:

>         - A PackageResolver will be implemented that *finds and
>         downloads* packages by name by looking in different places, in
>         order. So given "Foo" it will look in place A, B, C etc until
>         it finds Foo.js. There will be no version aspect in this to
>         begin with.
>
>
> I have not found the time to dig into this but what I need (I think
> maybe better idea will come) would be to specify prefix per group of
> packages in loadAmber. For example:
>
>
> loadAmber( {  packages: { 'some/path/' =>  ['package1', 'package2'] ,
> 'other/path' => ['package3', 'package4'] },
>                      deploy: false, .... } );
>
>
> and that commit always PUT at the right place.
>
>
> Laurent
>
>
>  
>        
>         - A PackageLoader will be implemented that *loads packages and
>         their dependencies*. When we execute "Smalltalk current
>         loadPackage: 'Foo'" it will use PackageResolver to find and
>         download Foo.js. Then PackageLoader will evaluate the package
>         information (but nothing more) in Foo.js so that it creates
>         the Package instance for Foo. Then we look at its dependencies
>         - a list of Package names, say Bar and Baz. We then proceed
>         recursively to find, download and scan those too.
>        
>         - PackageLoader will, for all scanned packages from above:
>                - Select those that have all dependencies loaded (or
>         the dependencies are in our install queue) and add them to our
>         install queue.
>                - Rinse and repeat the above until all scanned packages
>         are in the queue.
>                - Present queue to the user and ask for "Go ahead" etc.
>                - Process queue. This loads the packages one by one.
>        
>        
>        
>         Ok, so that is the "sketch" for our Package system. Does it
>         look descent?
>        
>         regards, Göran
>