CogVM binaries as per VMMaker.oscog-eem.302/r2749

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
33 messages Options
12
Reply | Threaded
Open this post in threaded view
|

CogVM binaries as per VMMaker.oscog-eem.302/r2749

Eliot Miranda-2

CogVM binaries as per VMMaker.oscog-eem.302/r2749

Fix bad performance regression that on certain platforms (linux) results in all
send misses causing a discarded PIC creation followed by a slow hash lookup.

Fix bug when assigning to some context inst vars from generated methods.
Add accessors to document the context inst var access scheme.
Fix a compiler warning comparing an error code in cog:selector:

Update the BitBltPlugin to include the fast Arm ASM option.

Fix type errors in the Cogit that prevent the Cogit working when compiled with
clang.  Specifically void * pointers are not comparable.  Make sure that
fetchPointer:ofObject: & isIntegerValue: are declared in cointerp.h.

Update the Newspeak version of the VMProfileLinuxSupportPlugin.
Improve robustness of the nscogbuild/unixbuild mvm scripts.

Add the linux policy change dance to sqUnixVMProfile.c (cf sqUnixHeartbeat.c).
Fix a bail_out typo.

Change the VMProfileLinuxSupportPlugin to follow symlinks,
answering pairs of module name, dereferenced symlink or nil.

Fix 3 (!!) bugs in primitiveDLSymInLibrary.


--
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

Re: CogVM binaries as per VMMaker.oscog-eem.302/r2749

David T. Lewis
On Mon, Jul 15, 2013 at 05:35:36PM -0700, Eliot Miranda wrote:
> are in http://www.mirandabanda.org/files/Cog/VM/VM.r2749/.
>

Thanks Eliot!

Dave


> CogVM binaries as per VMMaker.oscog-eem.302/r2749
>
> Fix bad performance regression that on certain platforms (linux) results in
> all
> send misses causing a discarded PIC creation followed by a slow hash lookup.
>
> Fix bug when assigning to some context inst vars from generated methods.
> Add accessors to document the context inst var access scheme.
> Fix a compiler warning comparing an error code in cog:selector:
>
> Update the BitBltPlugin to include the fast Arm ASM option.
>
> Fix type errors in the Cogit that prevent the Cogit working when compiled
> with
> clang.  Specifically void * pointers are not comparable.  Make sure that
> fetchPointer:ofObject: & isIntegerValue: are declared in cointerp.h.
>
> Update the Newspeak version of the VMProfileLinuxSupportPlugin.
> Improve robustness of the nscogbuild/unixbuild mvm scripts.
>
> Add the linux policy change dance to sqUnixVMProfile.c (cf
> sqUnixHeartbeat.c).
> Fix a bail_out typo.
>
> Change the VMProfileLinuxSupportPlugin to follow symlinks,
> answering pairs of module name, dereferenced symlink or nil.
>
> Fix 3 (!!) bugs in primitiveDLSymInLibrary.
>
>
> --
> best,
> Eliot

>


Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] CogVM binaries as per VMMaker.oscog-eem.302/r2749

Frank Shearar-3
In reply to this post by Eliot Miranda-2
On 16 July 2013 01:35, Eliot Miranda <[hidden email]> wrote:
>
> are in http://www.mirandabanda.org/files/Cog/VM/VM.r2749/.

I've upgraded the build scripts for build.squeak.org accordingly. Thanks!

frank

Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] CogVM binaries as per VMMaker.oscog-eem.302/r2749

Paul DeBruicker
Frank Shearar-3 wrote
On 16 July 2013 01:35, Eliot Miranda <[hidden email]> wrote:
>
> are in http://www.mirandabanda.org/files/Cog/VM/VM.r2749/.

I've upgraded the build scripts for build.squeak.org accordingly. Thanks!

frank

Should I make a new all-in-one?
Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] CogVM binaries as per VMMaker.oscog-eem.302/r2749

Frank Shearar-3
On 17 July 2013 02:15, Paul DeBruicker <[hidden email]> wrote:

> Frank Shearar-3 wrote
>> On 16 July 2013 01:35, Eliot Miranda &lt;
>
>> eliot.miranda@
>
>> &gt; wrote:
>>>
>>> are in http://www.mirandabanda.org/files/Cog/VM/VM.r2749/.
>>
>> I've upgraded the build scripts for build.squeak.org accordingly. Thanks!
>>
>> frank
>
>
> Should I make a new all-in-one?


That sounds like a good idea. If it's an automated process, we can set
up a CI build for it.

frank

Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] CogVM binaries as per VMMaker.oscog-eem.302/r2749

Paul DeBruicker
Hi Frank,

Frank Shearar-3 wrote
On 17 July 2013 02:15, Paul DeBruicker <[hidden email]> wrote:
> Frank Shearar-3 wrote
>> On 16 July 2013 01:35, Eliot Miranda <
>
>> eliot.miranda@
>
>> > wrote:
>>>
>>> are in http://www.mirandabanda.org/files/Cog/VM/VM.r2749/.
>>
>> I've upgraded the build scripts for build.squeak.org accordingly. Thanks!
>>
>> frank
>
>
> Should I make a new all-in-one?


That sounds like a good idea. If it's an automated process, we can set
up a CI build for it.

frank

Sorry for the delay to get back to you.  I think it could be run by the CI system.  Its just a bunch of bash scripts now.  They are in this github repo:

https://github.com/pdebruic/Squeak-All-In-One

It does download the updated VMs but does not yet download new images.  

To make a new all-in-one just

1. clone that repo
2. download and extract the image you want to use
3. edit the create4.4AllInOne script to reflect the new VM version info
4. run the create4.4AllInOne script.


If you want to automate the build I'd be happy to help.  It may also be good to make 'bleeding-edge', latest-stable, and latest-release  all-in-one builds.  To do that we'd just need to get the proper images downloaded prior to making the all-in-one.  My reason for opening bug 7752 was to make this easier on me.  But there's probably another way to do it.

Let me know how I can help.

Paul
 

Reply | Threaded
Open this post in threaded view
|

Re: CogVM binaries as per VMMaker.oscog-eem.302/r2749

Herbert König
In reply to this post by Eliot Miranda-2
Hi Eliot,

under Win7 64 Bit the upgrading from r2701 of March 11th I got a speedup from 575 to 231 seconds for my "CSV file reading using symbols to reduce memory consumption and speedup #select" job.

The average full gc came down from  811 to 450 ms (averaged over 19 GC's)

So the performance regression may have been in Windows VM's too.

Thanks a lot!

What I don't understand:
I do a "purge undo records" then a GC.
In both cases I get around 5MB less used Memory but Squeak uses some additional 68 MB (old VM) resp 73 MB (new VM). Total Memory after GC is around 445 MB in both cases.
Is this to be expected and wouldn't happen if the image size were closer to the 512 MB Limit of Windows?

Cheers

Herbert

Am 16.07.2013 02:35, schrieb Eliot Miranda:

CogVM binaries as per VMMaker.oscog-eem.302/r2749

Fix bad performance regression that on certain platforms (linux) results in all
send misses causing a discarded PIC creation followed by a slow hash lookup.

Fix bug when assigning to some context inst vars from generated methods.
Add accessors to document the context inst var access scheme.
Fix a compiler warning comparing an error code in cog:selector:

Update the BitBltPlugin to include the fast Arm ASM option.

Fix type errors in the Cogit that prevent the Cogit working when compiled with
clang.  Specifically void * pointers are not comparable.  Make sure that
fetchPointer:ofObject: & isIntegerValue: are declared in cointerp.h.

Update the Newspeak version of the VMProfileLinuxSupportPlugin.
Improve robustness of the nscogbuild/unixbuild mvm scripts.

Add the linux policy change dance to sqUnixVMProfile.c (cf sqUnixHeartbeat.c).
Fix a bail_out typo.

Change the VMProfileLinuxSupportPlugin to follow symlinks,
answering pairs of module name, dereferenced symlink or nil.

Fix 3 (!!) bugs in primitiveDLSymInLibrary.


--
best,
Eliot



    



Reply | Threaded
Open this post in threaded view
|

Re: CogVM binaries as per VMMaker.oscog-eem.302/r2749

Eliot Miranda-2


On Thu, Jul 25, 2013 at 12:18 PM, Herbert König <[hidden email]> wrote:
Hi Eliot,

under Win7 64 Bit the upgrading from r2701 of March 11th I got a speedup from 575 to 231 seconds for my "CSV file reading using symbols to reduce memory consumption and speedup #select" job.

The average full gc came down from  811 to 450 ms (averaged over 19 GC's)

So the performance regression may have been in Windows VM's too.

Thanks a lot!

What I don't understand:
I do a "purge undo records" then a GC.
In both cases I get around 5MB less used Memory but Squeak uses some additional 68 MB (old VM) resp 73 MB (new VM). Total Memory after GC is around 445 MB in both cases.
Is this to be expected and wouldn't happen if the image size were closer to the 512 MB Limit of Windows?

The Cog VM keeps a lot more memory around in reserve.  It has a machine code zone, a larger new space and a reserve for flushing the stack zone to the heap as contexts that the standard VM doesn't use. (all of these extras are part of the standard linear heap segment).  So the disparity is to be expected.


Cheers

Herbert

cheers!
 

Am 16.07.2013 02:35, schrieb Eliot Miranda:

CogVM binaries as per VMMaker.oscog-eem.302/r2749

Fix bad performance regression that on certain platforms (linux) results in all
send misses causing a discarded PIC creation followed by a slow hash lookup.

Fix bug when assigning to some context inst vars from generated methods.
Add accessors to document the context inst var access scheme.
Fix a compiler warning comparing an error code in cog:selector:

Update the BitBltPlugin to include the fast Arm ASM option.

Fix type errors in the Cogit that prevent the Cogit working when compiled with
clang.  Specifically void * pointers are not comparable.  Make sure that
fetchPointer:ofObject: & isIntegerValue: are declared in cointerp.h.

Update the Newspeak version of the VMProfileLinuxSupportPlugin.
Improve robustness of the nscogbuild/unixbuild mvm scripts.

Add the linux policy change dance to sqUnixVMProfile.c (cf sqUnixHeartbeat.c).
Fix a bail_out typo.

Change the VMProfileLinuxSupportPlugin to follow symlinks,
answering pairs of module name, dereferenced symlink or nil.

Fix 3 (!!) bugs in primitiveDLSymInLibrary.


--
best,
Eliot



    







--
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

SqueakTrunk image on build.squeak.org broken?

David T. Lewis
I noticed that the VM tarball jobs on build.squeak.org (InterpreterVM and
CogVM jobs) have been failing for some time. These jobs use the latest trunk
image from the SqueakTrunk job, which is supposed to be a base Squeak image
updated from the trunk stream (see http://build.squeak.org/job/SqueakTrunk/).
However, that image is missing the ST80 package entirely (which indirectly
causes the VM tarball job failures).

I tried to update the image (world menu -> help... -> update code from
server) in hopes that this would load the missing packages, but this fails
due to some other problem.

The project comment for the SqueakTrunk job says:

 * Take a base image (currently 4.5-12565), update it, archive the result.
 * Run the entire suite of in-image tests.

I think that I had mistakenly assumed that the "SqueakTrunk" job was a release
image updated from the trunk stream, but actually it must be a stripped "base"
image with packages reloaded, and maybe the reloading part has forgotten to
install ST80. Is that right?

Dave



Reply | Threaded
Open this post in threaded view
|

Re: SqueakTrunk image on build.squeak.org broken?

Frank Shearar-3
On 28 July 2013 07:24, David T. Lewis <[hidden email]> wrote:

> I noticed that the VM tarball jobs on build.squeak.org (InterpreterVM and
> CogVM jobs) have been failing for some time. These jobs use the latest trunk
> image from the SqueakTrunk job, which is supposed to be a base Squeak image
> updated from the trunk stream (see http://build.squeak.org/job/SqueakTrunk/).
> However, that image is missing the ST80 package entirely (which indirectly
> causes the VM tarball job failures).
>
> I tried to update the image (world menu -> help... -> update code from
> server) in hopes that this would load the missing packages, but this fails
> due to some other problem.
>
> The project comment for the SqueakTrunk job says:
>
>  * Take a base image (currently 4.5-12565), update it, archive the result.
>  * Run the entire suite of in-image tests.
>
> I think that I had mistakenly assumed that the "SqueakTrunk" job was a release
> image updated from the trunk stream, but actually it must be a stripped "base"
> image with packages reloaded, and maybe the reloading part has forgotten to
> install ST80. Is that right?

Yes. ReleaseSqueakTrunk contains a rehydrated/full fat Squeak image
_with_ ST80 and friends loaded.

Sorry! I should have noticed the failing builds and connected that
with the recent stripping of ST80.

frank

> Dave
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: SqueakTrunk image on build.squeak.org broken?

David T. Lewis
On Sun, Jul 28, 2013 at 08:26:18AM +0100, Frank Shearar wrote:

> On 28 July 2013 07:24, David T. Lewis <[hidden email]> wrote:
> > I noticed that the VM tarball jobs on build.squeak.org (InterpreterVM and
> > CogVM jobs) have been failing for some time. These jobs use the latest trunk
> > image from the SqueakTrunk job, which is supposed to be a base Squeak image
> > updated from the trunk stream (see http://build.squeak.org/job/SqueakTrunk/).
> > However, that image is missing the ST80 package entirely (which indirectly
> > causes the VM tarball job failures).
> >
> > I tried to update the image (world menu -> help... -> update code from
> > server) in hopes that this would load the missing packages, but this fails
> > due to some other problem.
> >
> > The project comment for the SqueakTrunk job says:
> >
> >  * Take a base image (currently 4.5-12565), update it, archive the result.
> >  * Run the entire suite of in-image tests.
> >
> > I think that I had mistakenly assumed that the "SqueakTrunk" job was a release
> > image updated from the trunk stream, but actually it must be a stripped "base"
> > image with packages reloaded, and maybe the reloading part has forgotten to
> > install ST80. Is that right?
>
> Yes. ReleaseSqueakTrunk contains a rehydrated/full fat Squeak image
> _with_ ST80 and friends loaded.
>
> Sorry! I should have noticed the failing builds and connected that
> with the recent stripping of ST80.
>

Not at all, it was not obvious that this was connected to the problem.

I guess that once the package reorganizing settles down, it would be
good to have some kind of sanity-check test to ensure that a rehydrated
image contains the expected set of packages.

Dave
 

Reply | Threaded
Open this post in threaded view
|

Re: SqueakTrunk image on build.squeak.org broken?

Hannes Hirzel
Question of clarification:

What do you mean by a 'rehydrated image'?

--HH

On 7/28/13, David T. Lewis <[hidden email]> wrote:

> On Sun, Jul 28, 2013 at 08:26:18AM +0100, Frank Shearar wrote:
>> On 28 July 2013 07:24, David T. Lewis <[hidden email]> wrote:
>> > I noticed that the VM tarball jobs on build.squeak.org (InterpreterVM
>> > and
>> > CogVM jobs) have been failing for some time. These jobs use the latest
>> > trunk
>> > image from the SqueakTrunk job, which is supposed to be a base Squeak
>> > image
>> > updated from the trunk stream (see
>> > http://build.squeak.org/job/SqueakTrunk/).
>> > However, that image is missing the ST80 package entirely (which
>> > indirectly
>> > causes the VM tarball job failures).
>> >
>> > I tried to update the image (world menu -> help... -> update code from
>> > server) in hopes that this would load the missing packages, but this
>> > fails
>> > due to some other problem.
>> >
>> > The project comment for the SqueakTrunk job says:
>> >
>> >  * Take a base image (currently 4.5-12565), update it, archive the
>> > result.
>> >  * Run the entire suite of in-image tests.
>> >
>> > I think that I had mistakenly assumed that the "SqueakTrunk" job was a
>> > release
>> > image updated from the trunk stream, but actually it must be a stripped
>> > "base"
>> > image with packages reloaded, and maybe the reloading part has forgotten
>> > to
>> > install ST80. Is that right?
>>
>> Yes. ReleaseSqueakTrunk contains a rehydrated/full fat Squeak image
>> _with_ ST80 and friends loaded.
>>
>> Sorry! I should have noticed the failing builds and connected that
>> with the recent stripping of ST80.
>>
>
> Not at all, it was not obvious that this was connected to the problem.
>
> I guess that once the package reorganizing settles down, it would be
> good to have some kind of sanity-check test to ensure that a rehydrated
> image contains the expected set of packages.
>
> Dave
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: SqueakTrunk image on build.squeak.org broken?

Frank Shearar-3
It's a term I picked up from work: SqueakTrunk is like a dessicated,
dried out thing that's quite small, like a dessicated pea. But
ReleaseSqueakTrunk is like the rehydrated pea, useful for cooking.

As the package layering work proceeds, and more packages become
unloadable, I unload them from SqueakTrunk. ReleaseSqueakTrunk takes
that small SqueakTrunk artifact and reloads all those unloadable
packages.

The idea is that people just keep on using the ReleaseSqueakTrunk
image, and without even realising it, are using a _constructed_ image,
built up from some small core.

frank

On 28 July 2013 18:58, H. Hirzel <[hidden email]> wrote:

> Question of clarification:
>
> What do you mean by a 'rehydrated image'?
>
> --HH
>
> On 7/28/13, David T. Lewis <[hidden email]> wrote:
>> On Sun, Jul 28, 2013 at 08:26:18AM +0100, Frank Shearar wrote:
>>> On 28 July 2013 07:24, David T. Lewis <[hidden email]> wrote:
>>> > I noticed that the VM tarball jobs on build.squeak.org (InterpreterVM
>>> > and
>>> > CogVM jobs) have been failing for some time. These jobs use the latest
>>> > trunk
>>> > image from the SqueakTrunk job, which is supposed to be a base Squeak
>>> > image
>>> > updated from the trunk stream (see
>>> > http://build.squeak.org/job/SqueakTrunk/).
>>> > However, that image is missing the ST80 package entirely (which
>>> > indirectly
>>> > causes the VM tarball job failures).
>>> >
>>> > I tried to update the image (world menu -> help... -> update code from
>>> > server) in hopes that this would load the missing packages, but this
>>> > fails
>>> > due to some other problem.
>>> >
>>> > The project comment for the SqueakTrunk job says:
>>> >
>>> >  * Take a base image (currently 4.5-12565), update it, archive the
>>> > result.
>>> >  * Run the entire suite of in-image tests.
>>> >
>>> > I think that I had mistakenly assumed that the "SqueakTrunk" job was a
>>> > release
>>> > image updated from the trunk stream, but actually it must be a stripped
>>> > "base"
>>> > image with packages reloaded, and maybe the reloading part has forgotten
>>> > to
>>> > install ST80. Is that right?
>>>
>>> Yes. ReleaseSqueakTrunk contains a rehydrated/full fat Squeak image
>>> _with_ ST80 and friends loaded.
>>>
>>> Sorry! I should have noticed the failing builds and connected that
>>> with the recent stripping of ST80.
>>>
>>
>> Not at all, it was not obvious that this was connected to the problem.
>>
>> I guess that once the package reorganizing settles down, it would be
>> good to have some kind of sanity-check test to ensure that a rehydrated
>> image contains the expected set of packages.
>>
>> Dave
>>
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: SqueakTrunk image on build.squeak.org broken?

Hannes Hirzel
Thank you Frank for the explanation, I consider it to be a useful metaphor.

"rehydrated" image vs. "dehydrated" image

--Hannes

On 7/28/13, Frank Shearar <[hidden email]> wrote:
> It's a term I picked up from work: SqueakTrunk is like a dessicated,
> dried out thing that's quite small, like a dessicated pea. But
> ReleaseSqueakTrunk is like the rehydrated pea, useful for cooking.

"rehydrated" = ReleaseSqueakTrunk
http://build.squeak.org/job/ReleaseSqueakTrunk/


> As the package layering work proceeds, and more packages become
> unloadable, I unload them from SqueakTrunk.

"dehydrated"  = SqueakTrunk
http://build.squeak.org/job/SqueakTrunk/

> ReleaseSqueakTrunk takes
> that small SqueakTrunk artifact and reloads all those unloadable
> packages.



> The idea is that people just keep on using the ReleaseSqueakTrunk
> image, and without even realising it, are using a _constructed_ image,
> built up from some small core.
>
> frank
>
> On 28 July 2013 18:58, H. Hirzel <[hidden email]> wrote:
>> Question of clarification:
>>
>> What do you mean by a 'rehydrated image'?
>>
>> --HH
>>
>> On 7/28/13, David T. Lewis <[hidden email]> wrote:
>>> On Sun, Jul 28, 2013 at 08:26:18AM +0100, Frank Shearar wrote:
>>>> On 28 July 2013 07:24, David T. Lewis <[hidden email]> wrote:
>>>> > I noticed that the VM tarball jobs on build.squeak.org (InterpreterVM
>>>> > and
>>>> > CogVM jobs) have been failing for some time. These jobs use the
>>>> > latest
>>>> > trunk
>>>> > image from the SqueakTrunk job, which is supposed to be a base Squeak
>>>> > image
>>>> > updated from the trunk stream (see
>>>> > http://build.squeak.org/job/SqueakTrunk/).
>>>> > However, that image is missing the ST80 package entirely (which
>>>> > indirectly
>>>> > causes the VM tarball job failures).
>>>> >
>>>> > I tried to update the image (world menu -> help... -> update code
>>>> > from
>>>> > server) in hopes that this would load the missing packages, but this
>>>> > fails
>>>> > due to some other problem.
>>>> >
>>>> > The project comment for the SqueakTrunk job says:
>>>> >
>>>> >  * Take a base image (currently 4.5-12565), update it, archive the
>>>> > result.
>>>> >  * Run the entire suite of in-image tests.
>>>> >
>>>> > I think that I had mistakenly assumed that the "SqueakTrunk" job was
>>>> > a
>>>> > release
>>>> > image updated from the trunk stream, but actually it must be a
>>>> > stripped
>>>> > "base"
>>>> > image with packages reloaded, and maybe the reloading part has
>>>> > forgotten
>>>> > to
>>>> > install ST80. Is that right?
>>>>
>>>> Yes. ReleaseSqueakTrunk contains a rehydrated/full fat Squeak image
>>>> _with_ ST80 and friends loaded.
>>>>
>>>> Sorry! I should have noticed the failing builds and connected that
>>>> with the recent stripping of ST80.
>>>>
>>>
>>> Not at all, it was not obvious that this was connected to the problem.
>>>
>>> I guess that once the package reorganizing settles down, it would be
>>> good to have some kind of sanity-check test to ensure that a rehydrated
>>> image contains the expected set of packages.
>>>
>>> Dave
>>>
>>>
>>>
>>
>
>

cbc
Reply | Threaded
Open this post in threaded view
|

Re: SqueakTrunk image on build.squeak.org broken?

cbc
Is there a location to pull from that would keep my image up to date with the ReleaseSqueakTrunk version of Squeak?  Or if I update my image, will parts start dissappearing?

Is the only way to continue to use the ReleaseSqueakTrunk to download occasional snapshots and move all the code into it?

Or is the trunk system still working off of the release version, but the SqueakTrunk job strips out the unloadable packages as part of its job?  and then the ReleaseSqueakTrunk loads them back in?

I guess I could figure this our myself, but just in case anyone else is wondering, I'll ask.

-Chris


On Tue, Jul 30, 2013 at 7:13 AM, H. Hirzel <[hidden email]> wrote:
Thank you Frank for the explanation, I consider it to be a useful metaphor.

"rehydrated" image vs. "dehydrated" image

--Hannes

On 7/28/13, Frank Shearar <[hidden email]> wrote:
> It's a term I picked up from work: SqueakTrunk is like a dessicated,
> dried out thing that's quite small, like a dessicated pea. But
> ReleaseSqueakTrunk is like the rehydrated pea, useful for cooking.

"rehydrated" = ReleaseSqueakTrunk
http://build.squeak.org/job/ReleaseSqueakTrunk/


> As the package layering work proceeds, and more packages become
> unloadable, I unload them from SqueakTrunk.

"dehydrated"  = SqueakTrunk
http://build.squeak.org/job/SqueakTrunk/

> ReleaseSqueakTrunk takes
> that small SqueakTrunk artifact and reloads all those unloadable
> packages.



> The idea is that people just keep on using the ReleaseSqueakTrunk
> image, and without even realising it, are using a _constructed_ image,
> built up from some small core.
>
> frank
>
> On 28 July 2013 18:58, H. Hirzel <[hidden email]> wrote:
>> Question of clarification:
>>
>> What do you mean by a 'rehydrated image'?
>>
>> --HH
>>
>> On 7/28/13, David T. Lewis <[hidden email]> wrote:
>>> On Sun, Jul 28, 2013 at 08:26:18AM +0100, Frank Shearar wrote:
>>>> On 28 July 2013 07:24, David T. Lewis <[hidden email]> wrote:
>>>> > I noticed that the VM tarball jobs on build.squeak.org (InterpreterVM
>>>> > and
>>>> > CogVM jobs) have been failing for some time. These jobs use the
>>>> > latest
>>>> > trunk
>>>> > image from the SqueakTrunk job, which is supposed to be a base Squeak
>>>> > image
>>>> > updated from the trunk stream (see
>>>> > http://build.squeak.org/job/SqueakTrunk/).
>>>> > However, that image is missing the ST80 package entirely (which
>>>> > indirectly
>>>> > causes the VM tarball job failures).
>>>> >
>>>> > I tried to update the image (world menu -> help... -> update code
>>>> > from
>>>> > server) in hopes that this would load the missing packages, but this
>>>> > fails
>>>> > due to some other problem.
>>>> >
>>>> > The project comment for the SqueakTrunk job says:
>>>> >
>>>> >  * Take a base image (currently 4.5-12565), update it, archive the
>>>> > result.
>>>> >  * Run the entire suite of in-image tests.
>>>> >
>>>> > I think that I had mistakenly assumed that the "SqueakTrunk" job was
>>>> > a
>>>> > release
>>>> > image updated from the trunk stream, but actually it must be a
>>>> > stripped
>>>> > "base"
>>>> > image with packages reloaded, and maybe the reloading part has
>>>> > forgotten
>>>> > to
>>>> > install ST80. Is that right?
>>>>
>>>> Yes. ReleaseSqueakTrunk contains a rehydrated/full fat Squeak image
>>>> _with_ ST80 and friends loaded.
>>>>
>>>> Sorry! I should have noticed the failing builds and connected that
>>>> with the recent stripping of ST80.
>>>>
>>>
>>> Not at all, it was not obvious that this was connected to the problem.
>>>
>>> I guess that once the package reorganizing settles down, it would be
>>> good to have some kind of sanity-check test to ensure that a rehydrated
>>> image contains the expected set of packages.
>>>
>>> Dave
>>>
>>>
>>>
>>
>
>




Reply | Threaded
Open this post in threaded view
|

Re: SqueakTrunk image on build.squeak.org broken?

Frank Shearar-3
On 30 July 2013 18:13, Chris Cunningham <[hidden email]> wrote:
> Is there a location to pull from that would keep my image up to date with
> the ReleaseSqueakTrunk version of Squeak?  Or if I update my image, will
> parts start dissappearing?

Updating still works exactly as normal. Nothing will disappear from your image.

SqueakTrunk and ReleaseSqueakTrunk are the names for jobs on the
continuous integration server, build.squeak.org. The update stream
continues entirely unchanged.

> Is the only way to continue to use the ReleaseSqueakTrunk to download
> occasional snapshots and move all the code into it?
>
> Or is the trunk system still working off of the release version, but the
> SqueakTrunk job strips out the unloadable packages as part of its job?  and
> then the ReleaseSqueakTrunk loads them back in?

Yes, but the stripping out part is manual. Well, it's scripted [1],
and when a new package becomes unloadable I add it to the script, run
the script against the "master" artifact for the SqueakTrunk job and
publish it. The SqueakTrunk job then runs a bunch of tests against a
copy of that image. After that, build.squeak.org fires off the
ReleaseSqueakTrunk job. This takes the SqueakTrunk job's output, adds
the packages unloaded in [1] through ReleaseBuilder class >>
#prepareNewBuild (see #loadWellKnownPackages for the inverse of [1]'s
list), and publishes an artifact. Periodically, someone pushes that
artifact to ftp.squeak.org for public consumption. (This step still
needs work.)

> I guess I could figure this our myself, but just in case anyone else is
> wondering, I'll ask.

Sure! Questions are always welcome!

frank

[1] https://github.com/squeak-smalltalk/squeak-ci/blob/master/shrink-trunk.st

> -Chris
>
>
> On Tue, Jul 30, 2013 at 7:13 AM, H. Hirzel <[hidden email]> wrote:
>>
>> Thank you Frank for the explanation, I consider it to be a useful
>> metaphor.
>>
>> "rehydrated" image vs. "dehydrated" image
>>
>> --Hannes
>>
>> On 7/28/13, Frank Shearar <[hidden email]> wrote:
>> > It's a term I picked up from work: SqueakTrunk is like a dessicated,
>> > dried out thing that's quite small, like a dessicated pea. But
>> > ReleaseSqueakTrunk is like the rehydrated pea, useful for cooking.
>>
>> "rehydrated" = ReleaseSqueakTrunk
>> http://build.squeak.org/job/ReleaseSqueakTrunk/
>>
>>
>> > As the package layering work proceeds, and more packages become
>> > unloadable, I unload them from SqueakTrunk.
>>
>> "dehydrated"  = SqueakTrunk
>> http://build.squeak.org/job/SqueakTrunk/
>>
>> > ReleaseSqueakTrunk takes
>> > that small SqueakTrunk artifact and reloads all those unloadable
>> > packages.
>>
>>
>>
>> > The idea is that people just keep on using the ReleaseSqueakTrunk
>> > image, and without even realising it, are using a _constructed_ image,
>> > built up from some small core.
>> >
>> > frank
>> >
>> > On 28 July 2013 18:58, H. Hirzel <[hidden email]> wrote:
>> >> Question of clarification:
>> >>
>> >> What do you mean by a 'rehydrated image'?
>> >>
>> >> --HH
>> >>
>> >> On 7/28/13, David T. Lewis <[hidden email]> wrote:
>> >>> On Sun, Jul 28, 2013 at 08:26:18AM +0100, Frank Shearar wrote:
>> >>>> On 28 July 2013 07:24, David T. Lewis <[hidden email]> wrote:
>> >>>> > I noticed that the VM tarball jobs on build.squeak.org
>> >>>> > (InterpreterVM
>> >>>> > and
>> >>>> > CogVM jobs) have been failing for some time. These jobs use the
>> >>>> > latest
>> >>>> > trunk
>> >>>> > image from the SqueakTrunk job, which is supposed to be a base
>> >>>> > Squeak
>> >>>> > image
>> >>>> > updated from the trunk stream (see
>> >>>> > http://build.squeak.org/job/SqueakTrunk/).
>> >>>> > However, that image is missing the ST80 package entirely (which
>> >>>> > indirectly
>> >>>> > causes the VM tarball job failures).
>> >>>> >
>> >>>> > I tried to update the image (world menu -> help... -> update code
>> >>>> > from
>> >>>> > server) in hopes that this would load the missing packages, but
>> >>>> > this
>> >>>> > fails
>> >>>> > due to some other problem.
>> >>>> >
>> >>>> > The project comment for the SqueakTrunk job says:
>> >>>> >
>> >>>> >  * Take a base image (currently 4.5-12565), update it, archive the
>> >>>> > result.
>> >>>> >  * Run the entire suite of in-image tests.
>> >>>> >
>> >>>> > I think that I had mistakenly assumed that the "SqueakTrunk" job
>> >>>> > was
>> >>>> > a
>> >>>> > release
>> >>>> > image updated from the trunk stream, but actually it must be a
>> >>>> > stripped
>> >>>> > "base"
>> >>>> > image with packages reloaded, and maybe the reloading part has
>> >>>> > forgotten
>> >>>> > to
>> >>>> > install ST80. Is that right?
>> >>>>
>> >>>> Yes. ReleaseSqueakTrunk contains a rehydrated/full fat Squeak image
>> >>>> _with_ ST80 and friends loaded.
>> >>>>
>> >>>> Sorry! I should have noticed the failing builds and connected that
>> >>>> with the recent stripping of ST80.
>> >>>>
>> >>>
>> >>> Not at all, it was not obvious that this was connected to the problem.
>> >>>
>> >>> I guess that once the package reorganizing settles down, it would be
>> >>> good to have some kind of sanity-check test to ensure that a
>> >>> rehydrated
>> >>> image contains the expected set of packages.
>> >>>
>> >>> Dave
>> >>>
>> >>>
>> >>>
>> >>
>> >
>> >
>>
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: SqueakTrunk image on build.squeak.org broken?

Edgar De Cleene
In reply to this post by Frank Shearar-3



On 7/28/13 3:22 PM, "Frank Shearar" <[hidden email]> wrote:

> It's a term I picked up from work: SqueakTrunk is like a dessicated,
> dried out thing that's quite small, like a dessicated pea. But
> ReleaseSqueakTrunk is like the rehydrated pea, useful for cooking.
>
> As the package layering work proceeds, and more packages become
> unloadable, I unload them from SqueakTrunk. ReleaseSqueakTrunk takes
> that small SqueakTrunk artifact and reloads all those unloadable
> packages.
>
> The idea is that people just keep on using the ReleaseSqueakTrunk
> image, and without even realising it, are using a _constructed_ image,
> built up from some small core.
>
> frank


And when we go bold and use a Core.image?
Our cousins Pharoers build a PharoKernel from sources , great work of Pavel
and Guillermo (and others maybe).

Edgar



Reply | Threaded
Open this post in threaded view
|

Re: SqueakTrunk image on build.squeak.org broken?

Frank Shearar-3
On 24 October 2013 22:02, Edgar J. De Cleene <[hidden email]> wrote:

>
>
>
> On 7/28/13 3:22 PM, "Frank Shearar" <[hidden email]> wrote:
>
>> It's a term I picked up from work: SqueakTrunk is like a dessicated,
>> dried out thing that's quite small, like a dessicated pea. But
>> ReleaseSqueakTrunk is like the rehydrated pea, useful for cooking.
>>
>> As the package layering work proceeds, and more packages become
>> unloadable, I unload them from SqueakTrunk. ReleaseSqueakTrunk takes
>> that small SqueakTrunk artifact and reloads all those unloadable
>> packages.
>>
>> The idea is that people just keep on using the ReleaseSqueakTrunk
>> image, and without even realising it, are using a _constructed_ image,
>> built up from some small core.
>>
>> frank
>
>
> And when we go bold and use a Core.image?
> Our cousins Pharoers build a PharoKernel from sources , great work of Pavel
> and Guillermo (and others maybe).

When I have time, basically. I reconstructed the base image from
scratch recently, and haven't had a chance to re-rip out Nebraska,
Universes, and all the other bits I'd previously removed.

I'm very happy to see Pharo do this work. Kudos to Pavel & Guille & friends!

frank

> Edgar
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: SqueakTrunk image on build.squeak.org broken?

Nicolas Cellier
Yes, Pharo is doing a great work of simplification.
On the other hand, it deliberately has zero requirements to make removed parts reloadable, so the task is a bit easier...


2013/10/25 Frank Shearar <[hidden email]>
On 24 October 2013 22:02, Edgar J. De Cleene <[hidden email]> wrote:
>
>
>
> On 7/28/13 3:22 PM, "Frank Shearar" <[hidden email]> wrote:
>
>> It's a term I picked up from work: SqueakTrunk is like a dessicated,
>> dried out thing that's quite small, like a dessicated pea. But
>> ReleaseSqueakTrunk is like the rehydrated pea, useful for cooking.
>>
>> As the package layering work proceeds, and more packages become
>> unloadable, I unload them from SqueakTrunk. ReleaseSqueakTrunk takes
>> that small SqueakTrunk artifact and reloads all those unloadable
>> packages.
>>
>> The idea is that people just keep on using the ReleaseSqueakTrunk
>> image, and without even realising it, are using a _constructed_ image,
>> built up from some small core.
>>
>> frank
>
>
> And when we go bold and use a Core.image?
> Our cousins Pharoers build a PharoKernel from sources , great work of Pavel
> and Guillermo (and others maybe).

When I have time, basically. I reconstructed the base image from
scratch recently, and haven't had a chance to re-rip out Nebraska,
Universes, and all the other bits I'd previously removed.

I'm very happy to see Pharo do this work. Kudos to Pavel & Guille & friends!

frank

> Edgar
>
>
>




Reply | Threaded
Open this post in threaded view
|

Re: SqueakTrunk image on build.squeak.org broken?

Edgar De Cleene


De: Nicolas Cellier <[hidden email]>
Responder a: The general-purpose Squeak developers list <[hidden email]>
Fecha: Fri, 25 Oct 2013 14:06:43 +0200
Para: The general-purpose Squeak developers list <[hidden email]>
Asunto: Re: [squeak-dev] SqueakTrunk image on build.squeak.org broken?

Yes, Pharo is doing a great work of simplification.
On the other hand, it deliberately has zero requirements to make removed parts reloadable, so the task is a bit easier...


Still exploring and understanding his system, but reporting ReferenceStream to Pharo 2.0 and having DependencyBrowser of Squeak working on it, a long time work could be put our view of Morphic on top of his kernel.

Or Cuis Morph hierarchy.

Edgar


12