Smalltalk › Squeak › Squeak - Dev

Compressed sources

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

9 messages Options

Bert Freudenberg

Compressed sources

I seem to remember a hack that allowed the sources file to be compressed on disk. It would be uncompressed on-the-fly. Does someone remember that hack? I couldn't find it.

Background: The new OLPC XO does not use a compressing filesystem anymore. So the Etoys sources file size hurts more than before.

- Bert -

Eliot Miranda-2

Re: Compressed sources

On Wed, Jan 13, 2010 at 2:35 PM, Bert Freudenberg <[hidden email]> wrote:

I seem to remember a hack that allowed the sources file to be compressed on disk. It would be uncompressed on-the-fly. Does someone remember that hack? I couldn't find it.

CompressedSourceStream & SystemDictionary>>compressSources is in our 3.8 derived Teleplace images.

There's also the scheme that discards the sources altogether by using a compressed array of temp names at the end of each method. See SystemDictionary>>abandonSources.

best

Eliot

Background: The new OLPC XO does not use a compressing filesystem anymore. So the Etoys sources file size hurts more than before.

- Bert -

Bert Freudenberg

Re: Compressed sources

On 13.01.2010, at 23:51, Eliot Miranda wrote:

On Wed, Jan 13, 2010 at 2:35 PM, Bert Freudenberg <[hidden email]> wrote:
I seem to remember a hack that allowed the sources file to be compressed on disk. It would be uncompressed on-the-fly. Does someone remember that hack? I couldn't find it.

CompressedSourceStream & SystemDictionary>>compressSources is in our 3.8 derived Teleplace images.

Ah, great, thanks! Scott also pointed me to this:

http://wiki.squeak.org/squeak/3510

- Bert -

There's also the scheme that discards the sources altogether by using a compressed array of temp names at the end of each method. See SystemDictionary>>abandonSources.

best
Eliot

Background: The new OLPC XO does not use a compressing filesystem anymore. So the Etoys sources file size hurts more than before.

- Bert -

David T. Lewis

Re: Compressed sources

On Wed, Jan 13, 2010 at 11:59:01PM +0100, Bert Freudenberg wrote:

> On 13.01.2010, at 23:51, Eliot Miranda wrote:
> >
> >
> > On Wed, Jan 13, 2010 at 2:35 PM, Bert Freudenberg <[hidden email]> wrote:
> > I seem to remember a hack that allowed the sources file to be compressed on disk. It would be uncompressed on-the-fly. Does someone remember that hack? I couldn't find it.
> >
> > CompressedSourceStream & SystemDictionary>>compressSources is in our 3.8 derived Teleplace images.
>
> Ah, great, thanks! Scott also pointed me to this:
>
> http://wiki.squeak.org/squeak/3510

The necessary support has somehow survived in Squeak trunk as well,
see also FileDirectory class>>openSources:forImage:

#compressSources encounters an error in trunk, but a little bit of fiddling
around would probably get it working.

It looks like you just need to write a compressed EtoysV3.sources.stc
file using #compressSources, and it might just work :)

Dave

David T. Lewis

Re: Compressed sources

On Wed, Jan 13, 2010 at 09:54:34PM -0500, David T. Lewis wrote:

> On Wed, Jan 13, 2010 at 11:59:01PM +0100, Bert Freudenberg wrote:
> > On 13.01.2010, at 23:51, Eliot Miranda wrote:
> > >
> > > On Wed, Jan 13, 2010 at 2:35 PM, Bert Freudenberg <[hidden email]> wrote:
> > > > I seem to remember a hack that allowed the sources file to be compressed on disk.
> > > > It would be uncompressed on-the-fly. Does someone remember that hack? I couldn't find it.
> > >
> > > CompressedSourceStream & SystemDictionary>>compressSources is in our 3.8 derived Teleplace images.
> >
> > Ah, great, thanks! Scott also pointed me to this:
> >
> > http://wiki.squeak.org/squeak/3510
>
> The necessary support has somehow survived in Squeak trunk as well,
> see also FileDirectory class>>openSources:forImage:
>
> #compressSources encounters an error in trunk, but a little bit of fiddling
> around would probably get it working.
>
> It looks like you just need to write a compressed EtoysV3.sources.stc
> file using #compressSources, and it might just work :)

Attached .cs is all that is required to produce a compressed sources
file that seems to work correctly on a 3.8 image. It does *not* produce
a usable compressed sources file on a Squeak trunk image, but that can
be a problem for another day.

I would spend a few minutes browsing source code to make sure nothing
is broken, but at first blush it seems fine.

All I had to do is 'Smalltalk compressSources', quit the image, delete
the normal sources file, restart the image, and viola it was using
the compressed SqueakV3.stc file instead of SqueakV3.sources.

Dave

SystemDictionary-compressSources-patch-dtl.1.cs (1K) Download Attachment

Edgar J. De Cleene

Re: Compressed sources

In reply to this post by Bert Freudenberg

On 1/13/10 8:35 PM, "Bert Freudenberg" <[hidden email]> wrote:

> I seem to remember a hack that allowed the sources file to be compressed on
> disk. It would be uncompressed on-the-fly. Does someone remember that hack? I
> couldn't find it.
>
> Background: The new OLPC XO does not use a compressing filesystem anymore. So
> the Etoys sources file size hurts more than before.
>
> - Bert -
Yes, was a Dan proposal many moons ago and I test and have some old images
with it.
I could look in old backups if nobody have it at hand.

And I several times proposal the attached and have a "class repository"

Edgar

ReleaseBuilderFor3dot11-sources managment.st (3K) Download Attachment

Bert Freudenberg

Re: Compressed sources

In reply to this post by David T. Lewis

On 14.01.2010, at 04:40, David T. Lewis wrote:

>
> On Wed, Jan 13, 2010 at 09:54:34PM -0500, David T. Lewis wrote:
>> On Wed, Jan 13, 2010 at 11:59:01PM +0100, Bert Freudenberg wrote:
>>> On 13.01.2010, at 23:51, Eliot Miranda wrote:
>>>>
>>>> On Wed, Jan 13, 2010 at 2:35 PM, Bert Freudenberg <[hidden email]> wrote:
>>>>> I seem to remember a hack that allowed the sources file to be compressed on disk.
>>>>> It would be uncompressed on-the-fly. Does someone remember that hack? I couldn't find it.
>>>>
>>>> CompressedSourceStream & SystemDictionary>>compressSources is in our 3.8 derived Teleplace images.
>>>
>>> Ah, great, thanks! Scott also pointed me to this:
>>>
>>> http://wiki.squeak.org/squeak/3510
>>
>> The necessary support has somehow survived in Squeak trunk as well,
>> see also FileDirectory class>>openSources:forImage:
>>
>> #compressSources encounters an error in trunk, but a little bit of fiddling
>> around would probably get it working.
>>
>> It looks like you just need to write a compressed EtoysV3.sources.stc
>> file using #compressSources, and it might just work :)

The compression appears to work fine but it actually does not correctly deal with Unicode characters. I added a fix at

http://tracker.squeakland.org/browse/SQ-647

> Attached .cs is all that is required to produce a compressed sources
> file that seems to work correctly on a 3.8 image. It does *not* produce
> a usable compressed sources file on a Squeak trunk image, but that can
> be a problem for another day.

Is the symptom similar to what I see in Etoys?

> I would spend a few minutes browsing source code to make sure nothing
> is broken, but at first blush it seems fine.
>
> All I had to do is 'Smalltalk compressSources', quit the image, delete
> the normal sources file, restart the image, and viola it was using
> the compressed SqueakV3.stc file instead of SqueakV3.sources.

You don't even need to delete the old sources file, the compressed one is searched for first. Just put it next to the image.

> <SystemDictionary-compressSources-patch-dtl.1.cs>

Maybe my fix for Etoys works in trunk too?

- Bert -

Igor Stasenko

Re: Compressed sources

Alternatively, with new method trailers you could save the comressed
method's source code method inside method's trailer.
Loosing the history track, however.

2010/1/14 Bert Freudenberg <[hidden email]>:

> On 14.01.2010, at 04:40, David T. Lewis wrote:
>>
>> On Wed, Jan 13, 2010 at 09:54:34PM -0500, David T. Lewis wrote:
>>> On Wed, Jan 13, 2010 at 11:59:01PM +0100, Bert Freudenberg wrote:
>>>> On 13.01.2010, at 23:51, Eliot Miranda wrote:
>>>>>
>>>>> On Wed, Jan 13, 2010 at 2:35 PM, Bert Freudenberg <[hidden email]> wrote:
>>>>>> I seem to remember a hack that allowed the sources file to be compressed on disk.
>>>>>> It would be uncompressed on-the-fly. Does someone remember that hack? I couldn't find it.
>>>>>
>>>>> CompressedSourceStream & SystemDictionary>>compressSources is in our 3.8 derived Teleplace images.
>>>>
>>>> Ah, great, thanks! Scott also pointed me to this:
>>>>
>>>> http://wiki.squeak.org/squeak/3510
>>>
>>> The necessary support has somehow survived in Squeak trunk as well,
>>> see also FileDirectory class>>openSources:forImage:
>>>
>>> #compressSources encounters an error in trunk, but a little bit of fiddling
>>> around would probably get it working.
>>>
>>> It looks like you just need to write a compressed EtoysV3.sources.stc
>>> file using #compressSources, and it might just work :)
>
> The compression appears to work fine but it actually does not correctly deal with Unicode characters. I added a fix at
>
> http://tracker.squeakland.org/browse/SQ-647
>
>> Attached .cs is all that is required to produce a compressed sources
>> file that seems to work correctly on a 3.8 image. It does *not* produce
>> a usable compressed sources file on a Squeak trunk image, but that can
>> be a problem for another day.
>
> Is the symptom similar to what I see in Etoys?
>
>> I would spend a few minutes browsing source code to make sure nothing
>> is broken, but at first blush it seems fine.
>>
>> All I had to do is 'Smalltalk compressSources', quit the image, delete
>> the normal sources file, restart the image, and viola it was using
>> the compressed SqueakV3.stc file instead of SqueakV3.sources.
>
> You don't even need to delete the old sources file, the compressed one is searched for first. Just put it next to the image.
>
>> <SystemDictionary-compressSources-patch-dtl.1.cs>
>
> Maybe my fix for Etoys works in trunk too?
>
> - Bert -
>
>
>

--
Best regards,
Igor Stasenko AKA sig.

David T. Lewis

Re: Compressed sources

In reply to this post by Bert Freudenberg

On Thu, Jan 14, 2010 at 03:30:48PM +0100, Bert Freudenberg wrote:
>
> The compression appears to work fine but it actually does not correctly deal with Unicode characters. I added a fix at
>
> http://tracker.squeakland.org/browse/SQ-647

> Maybe my fix for Etoys works in trunk too?

Bert,

It works fine. I just added your change set to Squeak trunk.

Thanks,
Dave