4.6/5.0 -- how to help users manage different formats?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

4.6/5.0 -- how to help users manage different formats?

Chris Muller-4
Hi Eliot and all, for this release I wonder whether we should provide
alternate binary's in the bin directory of the VM to help users manage
the different image formats.

What if, in addition to bin/squeak, Cog would include bin/cog and Spur
would include bin/spur.  In that case, although bin/squeak would be
whoever was installed last, at least users could be explicit about
which VM they want to launch which image.

I know we also talked about some universal bin/squeak which could id
the image format and launch the appropriate binary, which would be
great.

We should plan on doing _something_ in this area for the release.

Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

Trygve
What's happened to the "write once, run everywhere" that used to be the hallmark of Smalltalk?

On 11.03.2015 21:34, Chris Muller wrote:
Hi Eliot and all, for this release I wonder whether we should provide
alternate binary's in the bin directory of the VM to help users manage
the different image formats.

What if, in addition to bin/squeak, Cog would include bin/cog and Spur
would include bin/spur.  In that case, although bin/squeak would be
whoever was installed last, at least users could be explicit about
which VM they want to launch which image.

I know we also talked about some universal bin/squeak which could id
the image format and launch the appropriate binary, which would be
great.

We should plan on doing _something_ in this area for the release.





Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

Chris Muller-3
On Wed, Mar 11, 2015 at 3:45 PM, Trygve Reenskaug <[hidden email]> wrote:
> What's happened to the "write once, run everywhere" that used to be the
> hallmark of Smalltalk?

LOL, nothing, all the code which has been written once can still be
ran everywhere.

This thread is about helping users manage the transition to the new
image-format of the future for Squeak, Spur.

Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

Frank Shearar-3
In reply to this post by Trygve
It's right where we left it. Spur introduces a new object memory
format, so we're doing the right thing: we're breaking the format and
telling people.

When you load packages into a Spur image, things will just work. When
you load changesets, things will just work. As far as I know, we've
taken care that ImageSegments will also load into Spur, and just work.

But sometimes, if you want to actually make progress, you're going to
have to bite the bullet and make breaking changes.

frank

On 11 March 2015 at 20:45, Trygve Reenskaug <[hidden email]> wrote:

> What's happened to the "write once, run everywhere" that used to be the
> hallmark of Smalltalk?
>
>
> On 11.03.2015 21:34, Chris Muller wrote:
>
> Hi Eliot and all, for this release I wonder whether we should provide
> alternate binary's in the bin directory of the VM to help users manage
> the different image formats.
>
> What if, in addition to bin/squeak, Cog would include bin/cog and Spur
> would include bin/spur.  In that case, although bin/squeak would be
> whoever was installed last, at least users could be explicit about
> which VM they want to launch which image.
>
> I know we also talked about some universal bin/squeak which could id
> the image format and launch the appropriate binary, which would be
> great.
>
> We should plan on doing _something_ in this area for the release.
>
>
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

Eliot Miranda-2


On Thu, Mar 12, 2015 at 7:24 AM, Frank Shearar <[hidden email]> wrote:
It's right where we left it. Spur introduces a new object memory
format, so we're doing the right thing: we're breaking the format and
telling people.

When you load packages into a Spur image, things will just work. When
you load changesets, things will just work. As far as I know, we've
taken care that ImageSegments will also load into Spur, and just work.

Ah, not quite.  Image segments saved in 32-bit Spur will load in 32-bit Spur.  Image segments saved in 64-bit Spur will load in 64-bit Spur.  Image segments saved in Cog will load in Cog.  But no interoperability is possible (yet!).  And indeed I believe that image segments saved in the 64-bit Interpreter VM will only load in that 64-bit VM.

The problem with image segments is that they are in the format of the VM's heap objects.  However, Bert and I want co co-mentor a project to implement image-level support for loading segments from "foreign" object memories.  Alas GSoC did not approve our organization this year.  Maybe soon.  Or maybe some student at one of the teaching institutions using Squeak or Pharo would be interested in having a go.

Note that e.g. Fuel does not have this problem.

But sometimes, if you want to actually make progress, you're going to
have to bite the bullet and make breaking changes.

+1.  Spur has lots of advantages over the existing system but these can't be introduced without forcing a change to a new image and a new VM.

frank

On 11 March 2015 at 20:45, Trygve Reenskaug <[hidden email]> wrote:
> What's happened to the "write once, run everywhere" that used to be the
> hallmark of Smalltalk?
>
>
> On 11.03.2015 21:34, Chris Muller wrote:
>
> Hi Eliot and all, for this release I wonder whether we should provide
> alternate binary's in the bin directory of the VM to help users manage
> the different image formats.
>
> What if, in addition to bin/squeak, Cog would include bin/cog and Spur
> would include bin/spur.  In that case, although bin/squeak would be
> whoever was installed last, at least users could be explicit about
> which VM they want to launch which image.
>
> I know we also talked about some universal bin/squeak which could id
> the image format and launch the appropriate binary, which would be
> great.
>
> We should plan on doing _something_ in this area for the release.
>
>
>
>
>
>




--
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

David T. Lewis
In reply to this post by Frank Shearar-3
At the risk of giving myself an assignment ...

One thing we should do for the upcoming release is provide something in a
welcome workspace that gives a clear explanation of Spur from an end user
perspective. Something along the lines of what it is, why we are doing it
now, and what you (the end user) will gain from it now and in the future.
This should be at the "elevator speech" level of detail, as opposed to
deep technical details (for which we just give a pointer to Eliot's blog).

Dave


> It's right where we left it. Spur introduces a new object memory
> format, so we're doing the right thing: we're breaking the format and
> telling people.
>
> When you load packages into a Spur image, things will just work. When
> you load changesets, things will just work. As far as I know, we've
> taken care that ImageSegments will also load into Spur, and just work.
>
> But sometimes, if you want to actually make progress, you're going to
> have to bite the bullet and make breaking changes.
>
> frank
>
> On 11 March 2015 at 20:45, Trygve Reenskaug <[hidden email]> wrote:
>> What's happened to the "write once, run everywhere" that used to be the
>> hallmark of Smalltalk?
>>
>>
>> On 11.03.2015 21:34, Chris Muller wrote:
>>
>> Hi Eliot and all, for this release I wonder whether we should provide
>> alternate binary's in the bin directory of the VM to help users manage
>> the different image formats.
>>
>> What if, in addition to bin/squeak, Cog would include bin/cog and Spur
>> would include bin/spur.  In that case, although bin/squeak would be
>> whoever was installed last, at least users could be explicit about
>> which VM they want to launch which image.
>>
>> I know we also talked about some universal bin/squeak which could id
>> the image format and launch the appropriate binary, which would be
>> great.
>>
>> We should plan on doing _something_ in this area for the release.
>>
>>
>>
>>
>>
>>
>



Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

Chris Muller-3
In reply to this post by Eliot Miranda-2
>> It's right where we left it. Spur introduces a new object memory
>> format, so we're doing the right thing: we're breaking the format and
>> telling people.
>>
>> When you load packages into a Spur image, things will just work. When
>> you load changesets, things will just work. As far as I know, we've
>> taken care that ImageSegments will also load into Spur, and just work.
>
>
> Ah, not quite.  Image segments saved in 32-bit Spur will load in 32-bit
> Spur.  Image segments saved in 64-bit Spur will load in 64-bit Spur.  Image
> segments saved in Cog will load in Cog.  But no interoperability is possible
> (yet!).  And indeed I believe that image segments saved in the 64-bit
> Interpreter VM will only load in that 64-bit VM.

64-bit will not be ready in time for 5.0 on 4/30 right?

> The problem with image segments is that they are in the format of the VM's
> heap objects.  However, Bert and I want co co-mentor a project to implement
> image-level support for loading segments from "foreign" object memories.
> Alas GSoC did not approve our organization this year.  Maybe soon.  Or maybe
> some student at one of the teaching institutions using Squeak or Pharo would
> be interested in having a go.
>
> Note that e.g. Fuel does not have this problem.

Nor does Ma Object Serializer.  Ma Object Serializer has been updated
to be transparently Spur compatible, so for 32-bit systems anyway, one
could load ImageSegment file into a Cog image, save it with Ma, go
into spur and load it with Ma, and then finally save it as
ImageSegment.  You could even stay as Ma for a while because I intend
to do the same for 64-bit.

> +1.  Spur has lots of advantages over the existing system but these can't be
> introduced without forcing a change to a new image and a new VM.

Now that we've answered the troll-style question extensively, may we
please talk about ideas to help users manage multiple formats?

Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

Tobias Pape
In reply to this post by Eliot Miranda-2
Hey Eliot,

On 12.03.2015, at 17:44, Eliot Miranda <[hidden email]> wrote:

>
> The problem with image segments is that they are in the format of the VM's heap objects.  However, Bert and I want co co-mentor a project to implement image-level support for loading segments from "foreign" object memories.  Alas GSoC did not approve our organization this year.  Maybe soon.  Or maybe some student at one of the teaching institutions using Squeak or Pharo would be interested in having a go.

Can you or Bert give me an outline or a heads up?
(maybe off-list…)

Best
        -Tobias
Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

Eliot Miranda-2
In reply to this post by Chris Muller-3


On Thu, Mar 12, 2015 at 3:51 PM, Chris Muller <[hidden email]> wrote:
>> It's right where we left it. Spur introduces a new object memory
>> format, so we're doing the right thing: we're breaking the format and
>> telling people.
>>
>> When you load packages into a Spur image, things will just work. When
>> you load changesets, things will just work. As far as I know, we've
>> taken care that ImageSegments will also load into Spur, and just work.
>
>
> Ah, not quite.  Image segments saved in 32-bit Spur will load in 32-bit
> Spur.  Image segments saved in 64-bit Spur will load in 64-bit Spur.  Image
> segments saved in Cog will load in Cog.  But no interoperability is possible
> (yet!).  And indeed I believe that image segments saved in the 64-bit
> Interpreter VM will only load in that 64-bit VM.

64-bit will not be ready in time for 5.0 on 4/30 right?

Right.  I don't even have the optimized 64-bit Spur StackInterpreter working on Mac.  The assert VM works fine, but optimized crashes on start-up.  Let alone having a working 64-bit Spur StackInterpreter on Windows and Mac.  And let alone having a 64-bit Spur Cog JIT.


> The problem with image segments is that they are in the format of the VM's
> heap objects.  However, Bert and I want co co-mentor a project to implement
> image-level support for loading segments from "foreign" object memories.
> Alas GSoC did not approve our organization this year.  Maybe soon.  Or maybe
> some student at one of the teaching institutions using Squeak or Pharo would
> be interested in having a go.
>
> Note that e.g. Fuel does not have this problem.

Nor does Ma Object Serializer.  Ma Object Serializer has been updated
to be transparently Spur compatible, so for 32-bit systems anyway, one
could load ImageSegment file into a Cog image, save it with Ma, go
into spur and load it with Ma, and then finally save it as
ImageSegment.  You could even stay as Ma for a while because I intend
to do the same for 64-bit.

> +1.  Spur has lots of advantages over the existing system but these can't be
> introduced without forcing a change to a new image and a new VM.

Now that we've answered the troll-style question extensively, may we
please talk about ideas to help users manage multiple formats?




--
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

Eliot Miranda-2
In reply to this post by Tobias Pape


On Thu, Mar 12, 2015 at 3:56 PM, Tobias Pape <[hidden email]> wrote:
Hey Eliot,

On 12.03.2015, at 17:44, Eliot Miranda <[hidden email]> wrote:

>
> The problem with image segments is that they are in the format of the VM's heap objects.  However, Bert and I want co co-mentor a project to implement image-level support for loading segments from "foreign" object memories.  Alas GSoC did not approve our organization this year.  Maybe soon.  Or maybe some student at one of the teaching institutions using Squeak or Pharo would be interested in having a go.

Can you or Bert give me an outline or a heads up?

An ImageSegment is a WordArray whose contents are a sequence of heap objects in the VM's native object format.  A segment is loaded by invoking a primitive that swizzles the pointers in the word array so that they become valid objects, and then shortening the word array, leaving the objects behind in memory.  Since the contents of the WordArray are specific to a given VM architecture one cannot use the primitive loading mechanism to load segments from VMs with different object representations.  Here different means 
    - 32-bit V3 vs 64-bit V3
    - the contents of the CompactClassesArray (if they're not the same in the saving and loading system, objects could be loaded with the wrong class, which could crash the VM)
    - V3 vs Spur
    - Spur 32-bit vs Spur 64-bit

But there is no reason why Smalltalk code could not parse the contents of the WordArray and hence load objects by creating them based on the data in the WordArray.  This would allow image segments to be loaded in any format the Smalltalk code supported (and it would be interesting to compare performance and see how much slower "synthesized" loading would be).  This would provide legacy support for segments from "old" versions, and maybe even provide a oath to adopting a more neutral format such as Fuel.

To implement such a system one would have to understand the object formats of each system, using the VM source code as a guide, both the object representation and the load and save primitives themselves.

Does this answer your question?

 
(maybe off-list…)

Best
        -Tobias



--
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

Tobias Pape
Hi eliot
On 14.03.2015, at 13:32, Eliot Miranda <[hidden email]> wrote:

> On Thu, Mar 12, 2015 at 3:56 PM, Tobias Pape <[hidden email]> wrote:
> Hey Eliot,
>
> On 12.03.2015, at 17:44, Eliot Miranda <[hidden email]> wrote:
>
> >
> > The problem with image segments is that they are in the format of the VM's heap objects.  However, Bert and I want co co-mentor a project to implement image-level support for loading segments from "foreign" object memories.  Alas GSoC did not approve our organization this year.  Maybe soon.  Or maybe some student at one of the teaching institutions using Squeak or Pharo would be interested in having a go.
>
> Can you or Bert give me an outline or a heads up?



> An ImageSegment is a WordArray whose contents are a sequence of heap objects in the VM's native object format.  A segment is loaded by invoking a primitive that swizzles the pointers in the word array so that they become valid objects, and then shortening the word array, leaving the objects behind in memory.  Since the contents of the WordArray are specific to a given VM architecture one cannot use the primitive loading mechanism to load segments from VMs with different object representations.  Here different means
>     - 32-bit V3 vs 64-bit V3
>     - the contents of the CompactClassesArray (if they're not the same in the saving and loading system, objects could be loaded with the wrong class, which could crash the VM)
>     - V3 vs Spur
>     - Spur 32-bit vs Spur 64-bit
>
> But there is no reason why Smalltalk code could not parse the contents of the WordArray and hence load objects by creating them based on the data in the WordArray.  This would allow image segments to be loaded in any format the Smalltalk code supported (and it would be interesting to compare performance and see how much slower "synthesized" loading would be).  This would provide legacy support for segments from "old" versions, and maybe even provide a oath to adopting a more neutral format such as Fuel.
>
> To implement such a system one would have to understand the object formats of each system, using the VM source code as a guide, both the object representation and the load and save primitives themselves.
>
> Does this answer your question?
>

Yes. Thank you :)
I'll see whether some students could work with that… (no promise tho)

Best
        -Tobias

>  
> (maybe off-list…)
>
> Best
>         -Tobias




Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

Sean P. DeNigris
Administrator
In reply to this post by Chris Muller-3
Chris Muller-3 wrote
Now that we've answered the troll-style question extensively
Coincidentally, we just discussed "trolling" on the Pharo list. The consensus was that the key is whether the intention is disruption rather than productivity (and Wikipedia agrees [1]). I interpreted the post rather as a concern about a core value.

And I hope we can give the inventor of MVC the benefit of the doubt ;) IMHO is is an honor to have him still a part of our community.

[1] "posting… with the deliberate intent of… disrupting normal on-topic discussion" - http://en.wikipedia.org/wiki/Troll_%28Internet%29
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

Chris Muller-3
On Sat, Mar 14, 2015 at 10:40 AM, Sean P. DeNigris
<[hidden email]> wrote:

> Chris Muller-3 wrote
>> Now that we've answered the troll-style question extensively
>
> Coincidentally, we just discussed "trolling" on the Pharo list. The
> consensus was that the key is whether the intention is disruption rather
> than productivity (and Wikipedia agrees [1]). I interpreted the post rather
> as a concern about a core value.
>
> And I hope we can give the inventor of MVC the benefit of the doubt ;) IMHO
> is is an honor to have him still a part of our community.

I have no doubts Trygve was joking, and so was I, while trying to
discuss this issue of having multiple formats..

Reply | Threaded
Open this post in threaded view
|

re: 4.6/5.0 -- how to help users manage different formats?

ccrraaiigg
In reply to this post by David T. Lewis

> At the risk of giving myself an assignment ...

     Yes, I vote for you to create an issue for this on bugs.squeak.org
and assign it to yourself. I'll happily review it. ;)


-C

--
Craig Latta
netjam.org
March 2015:
+ 1 510  480 5860 (SMS ok)
+31  20  893 2796 (no SMS)
afterward:
+31   6 2757 7177 (SMS ok)
+ 1 415  287 3547 (no SMS)


Reply | Threaded
Open this post in threaded view
|

Re: 4.6/5.0 -- how to help users manage different formats?

Chris Muller-3
In reply to this post by Chris Muller-3
A couple of folks have contacted me privately, and suggested that I
may have misinterpreted Trygve.  Trygve, if you were not making a
joke, I apologize.  Spur is remarkably backward compatible code-wise,
I thought you were giving Eliot a hard time (in fun) about the image
format change.  My comment was a dumb attempt to play along in the
"joke" too.  If I've offended you, I apologize, please forgive my
misunderstanding.


On Sun, Mar 15, 2015 at 1:48 PM, Chris Muller <[hidden email]> wrote:

> On Sat, Mar 14, 2015 at 10:40 AM, Sean P. DeNigris
> <[hidden email]> wrote:
>> Chris Muller-3 wrote
>>> Now that we've answered the troll-style question extensively
>>
>> Coincidentally, we just discussed "trolling" on the Pharo list. The
>> consensus was that the key is whether the intention is disruption rather
>> than productivity (and Wikipedia agrees [1]). I interpreted the post rather
>> as a concern about a core value.
>>
>> And I hope we can give the inventor of MVC the benefit of the doubt ;) IMHO
>> is is an honor to have him still a part of our community.
>
> I have no doubts Trygve was joking, and so was I, while trying to
> discuss this issue of having multiple formats..