Fuel transfer from Pharo 1.4 to Pharo 6.1

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Fuel transfer from Pharo 1.4 to Pharo 6.1

Andreas Brodbeck-3
Hi all,

I have a long running Seaside/Pharo application which still runs perfect
on Pharo 1.4. I want to step into the future and migrate to the newest
Pharo 6.1.

The application's persistence is just in the image, so my plan is to use
Fuel to transfer the objects from Pharo 1.4 to Pharo 6.1.

My first attempt:

--- Using Fuel version 1.9.4 on both VMs.
--- Class TimeStamp was missing, so I created it.
--- I use a Fuel-migration for the renaming of "MethodContext" into the
new "Context" class.


But now I am stuck with FLMethodChanged errors. My investigation shows
that there are unequal values for the method Context>>bytecodesHash when
comparing Pharo 1.4 with 6.1. (BTW: Pharo 5 gives same values like 6.1).

So, basically Fuel can not materialize the instances because the
bytecode hash of all methods have changed, comparing 1.4 to 6.1. My
feeling says that this could be a major problem for my plans to migrate
from Pharo 1.4 to 6.1.

Is there a chance to still load those instances with Fuel into the new
6.1 Pharo?

Thanks!
Andreas

--
Andreas Brodbeck
www.mindclue.ch


Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Hannes Hirzel
Hello Andreas

Unfortunately Fuel is not meant for this kind of use case of migrating
an object model.

So other options have to be considered.

Does your object model have cycles?

Regards
Hannes

On 11/28/17, Andreas Brodbeck <[hidden email]> wrote:

> Hi all,
>
> I have a long running Seaside/Pharo application which still runs perfect
> on Pharo 1.4. I want to step into the future and migrate to the newest
> Pharo 6.1.
>
> The application's persistence is just in the image, so my plan is to use
> Fuel to transfer the objects from Pharo 1.4 to Pharo 6.1.
>
> My first attempt:
>
> --- Using Fuel version 1.9.4 on both VMs.
> --- Class TimeStamp was missing, so I created it.
> --- I use a Fuel-migration for the renaming of "MethodContext" into the
> new "Context" class.
>
>
> But now I am stuck with FLMethodChanged errors. My investigation shows
> that there are unequal values for the method Context>>bytecodesHash when
> comparing Pharo 1.4 with 6.1. (BTW: Pharo 5 gives same values like 6.1).
>
> So, basically Fuel can not materialize the instances because the
> bytecode hash of all methods have changed, comparing 1.4 to 6.1. My
> feeling says that this could be a major problem for my plans to migrate
> from Pharo 1.4 to 6.1.
>
> Is there a chance to still load those instances with Fuel into the new
> 6.1 Pharo?
>
> Thanks!
> Andreas
>
> --
> Andreas Brodbeck
> www.mindclue.ch
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Stephan Eggermont-3
In reply to this post by Andreas Brodbeck-3
On 28-11-17 11:01, Andreas Brodbeck wrote:
> So, basically Fuel can not materialize the instances because the
> bytecode hash of all methods have changed, comparing 1.4 to 6.1. My
> feeling says that this could be a major problem for my plans to migrate
> from Pharo 1.4 to 6.1.

If you export the bytecode hash of the 1.4 methods, you might be able to
use those to materialize again in 6.1. Or at least get a step further,
as there will be more things new & renamed & missing.

Stephan


Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Mariano Martinez Peck
In reply to this post by Andreas Brodbeck-3
Hi Andreas,

Do you know why method contexts are trying to be serialized? Of course, probably because of closures.  Do you think / know / are aware of closures as part of your graph? Maybe Sorted Collection? 
I am asking because if the only thing is SortedCollection then we can use some hook... 

Let me know,

On Tue, Nov 28, 2017 at 7:01 AM, Andreas Brodbeck <[hidden email]> wrote:
Hi all,

I have a long running Seaside/Pharo application which still runs perfect
on Pharo 1.4. I want to step into the future and migrate to the newest
Pharo 6.1.

The application's persistence is just in the image, so my plan is to use
Fuel to transfer the objects from Pharo 1.4 to Pharo 6.1.

My first attempt:

--- Using Fuel version 1.9.4 on both VMs.
--- Class TimeStamp was missing, so I created it.
--- I use a Fuel-migration for the renaming of "MethodContext" into the
new "Context" class.


But now I am stuck with FLMethodChanged errors. My investigation shows
that there are unequal values for the method Context>>bytecodesHash when
comparing Pharo 1.4 with 6.1. (BTW: Pharo 5 gives same values like 6.1).

So, basically Fuel can not materialize the instances because the
bytecode hash of all methods have changed, comparing 1.4 to 6.1. My
feeling says that this could be a major problem for my plans to migrate
from Pharo 1.4 to 6.1.

Is there a chance to still load those instances with Fuel into the new
6.1 Pharo?

Thanks!
Andreas

--
Andreas Brodbeck
www.mindclue.ch





--
Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Andreas Brodbeck-3
In reply to this post by Hannes Hirzel
Am 28.11.17 um 11:13 schrieb H. Hirzel:
> Hello Andreas
>
> Unfortunately Fuel is not meant for this kind of use case of migrating
> an object model.

That's sad to hear. From 1.4 to 6.1 is a big step, I agree, but I really
thought that Fuel can be considered as a good "transport vehicle" at
least for small steps like from Pharo 5 to 6.

>
> So other options have to be considered.

That probably would be a custom mechanism storing objects in a database
or fileformat and re-import, I think. Lots of work and very error prone.
But maybe the only way unfortunately ...

SIXX would be an option you think?

>
> Does your object model have cycles?

Yes.


Regards,
Andreas

--
Andreas Brodbeck
www.mindclue.ch


Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Sean P. DeNigris
Administrator
Andreas Brodbeck-3 wrote
>> So other options have to be considered.
>
> That probably would be a custom mechanism storing objects in a database
> or fileformat and re-import,…

My two primary workarounds in this scenario are:
1. Try STON
2. Try Fuel in smaller hops (e.g. 1.4 -> 2 -> 4, etc), although I'm not sure
that would work in your case. This seemed more useful when I didn't seem
able to load the same version of Fuel in the source and target.

Here is a little working document that might help get you started:
https://github.com/seandenigris/pharo/wiki/Data-Migration



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Andreas Brodbeck-3
In reply to this post by Mariano Martinez Peck
Am 28.11.17 um 12:46 schrieb Mariano Martinez Peck:
> Hi Andreas,
>
> Do you know why method contexts are trying to be serialized? Of course,
> probably because of closures.  Do you think / know / are aware of closures
> as part of your graph? Maybe Sorted Collection?
> I am asking because if the only thing is SortedCollection then we can use
> some hook...

I have closures in several places in the object graph. These are objects
with some pluggable functionality, which I rely on.

Just for my better understanding, I like to ask: The FLMethodChanged
errors will show up for *EVERY* method of a materialized object (Since
the bytecodesHash changed for every method) or just those methods which
involve fuel-persisted MethodContexts? (Sorry, if the question is
stupid, but I am rather new to Fuel internals)


Cheers,
Andreas

--
Andreas Brodbeck
www.mindclue.ch


Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Mariano Martinez Peck


On Tue, Nov 28, 2017 at 9:31 AM, Andreas Brodbeck <[hidden email]> wrote:
Am 28.11.17 um 12:46 schrieb Mariano Martinez Peck:
> Hi Andreas,
>
> Do you know why method contexts are trying to be serialized? Of course,
> probably because of closures.  Do you think / know / are aware of closures
> as part of your graph? Maybe Sorted Collection?
> I am asking because if the only thing is SortedCollection then we can use
> some hook...

I have closures in several places in the object graph. These are objects
with some pluggable functionality, which I rely on.


OK. The question is then if those closures are "clean" or not. In other words, what's their scope? Do they refer to variables defined outside  the closure (in which case you would need the methods contexts / stack) or are they clean in the sense that you could be able to replace them via a string and then compile them again ? 
Some time ago we added a #isClean to BlockClosure. You can test a few of your closures to see if this is the case or not. 

Btw, I have an application for a client in which we also have closures to define some pluggable behavior. But what I do is:

1) guarantee they are 100% clean (does not go outside of scope) 
2) aside from the "block" instVar I also add another instVar which is the "string" version of it. Then I always implement #fuelIgnoredInstanceVariableNames to ignore all those "block" instVars, yet DO NOT ignore the "string" like of those closures. Then, of course, the getter of the block instvar does a lazy compilation if the block instVar is nil...

I know you already have your application written, but I am trying to explain how I was able to use Fuel for this case... 
 

Just for my better understanding, I like to ask: The FLMethodChanged
errors will show up for *EVERY* method of a materialized object (Since
the bytecodesHash changed for every method) or just those methods which
involve fuel-persisted MethodContexts? (Sorry, if the question is
stupid, but I am rather new to Fuel internals)


As far as I can remember, the latter. 


 

Cheers,
Andreas

--
Andreas Brodbeck
www.mindclue.ch





--
Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Mariano Martinez Peck
In reply to this post by Andreas Brodbeck-3


On Tue, Nov 28, 2017 at 9:20 AM, Andreas Brodbeck <[hidden email]> wrote:
Am 28.11.17 um 11:13 schrieb H. Hirzel:
> Hello Andreas
>
> Unfortunately Fuel is not meant for this kind of use case of migrating
> an object model.

That's sad to hear. From 1.4 to 6.1 is a big step, I agree, but I really
thought that Fuel can be considered as a good "transport vehicle" at
least for small steps like from Pharo 5 to 6.



Yes, watching it from now, I think this was our biggest drawback of fuel. 


 
>
> So other options have to be considered.

That probably would be a custom mechanism storing objects in a database
or fileformat and re-import, I think. Lots of work and very error prone.
But maybe the only way unfortunately ...

SIXX would be an option you think?



No. SIXX doesn't know how to deal with closures.


 
>
> Does your object model have cycles?

Yes.


Regards,
Andreas

--
Andreas Brodbeck
www.mindclue.ch





--
Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Sven Van Caekenberghe-2
In reply to this post by Mariano Martinez Peck


> On 28 Nov 2017, at 13:52, Mariano Martinez Peck <[hidden email]> wrote:
>
>
>
> On Tue, Nov 28, 2017 at 9:31 AM, Andreas Brodbeck <[hidden email]> wrote:
> Am 28.11.17 um 12:46 schrieb Mariano Martinez Peck:
> > Hi Andreas,
> >
> > Do you know why method contexts are trying to be serialized? Of course,
> > probably because of closures.  Do you think / know / are aware of closures
> > as part of your graph? Maybe Sorted Collection?
> > I am asking because if the only thing is SortedCollection then we can use
> > some hook...
>
> I have closures in several places in the object graph. These are objects
> with some pluggable functionality, which I rely on.
>
>
> OK. The question is then if those closures are "clean" or not. In other words, what's their scope? Do they refer to variables defined outside  the closure (in which case you would need the methods contexts / stack) or are they clean in the sense that you could be able to replace them via a string and then compile them again ?
> Some time ago we added a #isClean to BlockClosure. You can test a few of your closures to see if this is the case or not.
>
> Btw, I have an application for a client in which we also have closures to define some pluggable behavior. But what I do is:
>
> 1) guarantee they are 100% clean (does not go outside of scope)
> 2) aside from the "block" instVar I also add another instVar which is the "string" version of it. Then I always implement #fuelIgnoredInstanceVariableNames to ignore all those "block" instVars, yet DO NOT ignore the "string" like of those closures. Then, of course, the getter of the block instvar does a lazy compilation if the block instVar is nil...

Ah, that's cheating ;-)

But totally acceptable with clean blocks.

So,

[ :x :y | x < y ] isClean.

  => true

[ :x :y | x < y ] sourceNode formattedCode.

  => '[ :x :y | x < y ]'

But

Compiler evaluate: '[ :x :y | x < y ]'.

  => [ :arg1 :arg2 | arg1 < arg2 ]

So the argument names get lost, which seems like a pity. How are you doing it ? Can it be done differently ?

> I know you already have your application written, but I am trying to explain how I was able to use Fuel for this case...
>  
>
> Just for my better understanding, I like to ask: The FLMethodChanged
> errors will show up for *EVERY* method of a materialized object (Since
> the bytecodesHash changed for every method) or just those methods which
> involve fuel-persisted MethodContexts? (Sorry, if the question is
> stupid, but I am rather new to Fuel internals)
>
>
> As far as I can remember, the latter.
>
>
>  
>
> Cheers,
> Andreas
>
> --
> Andreas Brodbeck
> www.mindclue.ch
>
>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com


Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Mariano Martinez Peck


On Tue, Nov 28, 2017 at 10:20 AM, Sven Van Caekenberghe <[hidden email]> wrote:


> On 28 Nov 2017, at 13:52, Mariano Martinez Peck <[hidden email]> wrote:
>
>
>
> On Tue, Nov 28, 2017 at 9:31 AM, Andreas Brodbeck <[hidden email]> wrote:
> Am 28.11.17 um 12:46 schrieb Mariano Martinez Peck:
> > Hi Andreas,
> >
> > Do you know why method contexts are trying to be serialized? Of course,
> > probably because of closures.  Do you think / know / are aware of closures
> > as part of your graph? Maybe Sorted Collection?
> > I am asking because if the only thing is SortedCollection then we can use
> > some hook...
>
> I have closures in several places in the object graph. These are objects
> with some pluggable functionality, which I rely on.
>
>
> OK. The question is then if those closures are "clean" or not. In other words, what's their scope? Do they refer to variables defined outside  the closure (in which case you would need the methods contexts / stack) or are they clean in the sense that you could be able to replace them via a string and then compile them again ?
> Some time ago we added a #isClean to BlockClosure. You can test a few of your closures to see if this is the case or not.
>
> Btw, I have an application for a client in which we also have closures to define some pluggable behavior. But what I do is:
>
> 1) guarantee they are 100% clean (does not go outside of scope)
> 2) aside from the "block" instVar I also add another instVar which is the "string" version of it. Then I always implement #fuelIgnoredInstanceVariableNames to ignore all those "block" instVars, yet DO NOT ignore the "string" like of those closures. Then, of course, the getter of the block instvar does a lazy compilation if the block instVar is nil...

Ah, that's cheating ;-)


I didn't say it wasn't hahahahahaha
 

But totally acceptable with clean blocks.

So,

[ :x :y | x < y ] isClean.

  => true

[ :x :y | x < y ] sourceNode formattedCode.

  => '[ :x :y | x < y ]'

But

Compiler evaluate: '[ :x :y | x < y ]'.

  => [ :arg1 :arg2 | arg1 < arg2 ]

So the argument names get lost, which seems like a pity. How are you doing it ? Can it be done differently ?



I really don't care at all how closures look like. The idea is: always keep the string version instVars and sixx/fuel ignore the block version. block version getter compiles from string if nil. 
I think you might have understood I use the same instVar and that sometimes I set a string and sometimes I set a closure. If that's the case, then yes, I ended up with 2 different instVars...


Example:


FaAction class >> fuelIgnoredInstanceVariableNames
^ #('actionBlock')


FaPersistentObject subclass: #FaAction
instanceVariableNames: 'action actionBlock'
classVariableNames: ''
package: 'FA-Core'


FaAction  >> actionBlock
^ actionBlock
ifNil: [ action isEmptyOrNil
ifTrue: [ nil ]
ifFalse: [actionBlock := self compile: action ] ]

That's is obviously a simplified example and it might have some syntax error but I guess you get the idea, right?


So this has worked very well for me with Fuel on Pharo and Sixx on GemStone. There are obvious drawbacks like: it only works for clean closures, having to have 2 instVar per "concept" , have to compile when I only have the string, etc etc.




 
> I know you already have your application written, but I am trying to explain how I was able to use Fuel for this case...
>
>
> Just for my better understanding, I like to ask: The FLMethodChanged
> errors will show up for *EVERY* method of a materialized object (Since
> the bytecodesHash changed for every method) or just those methods which
> involve fuel-persisted MethodContexts? (Sorry, if the question is
> stupid, but I am rather new to Fuel internals)
>
>
> As far as I can remember, the latter.
>
>
>
>
> Cheers,
> Andreas
>
> --
> Andreas Brodbeck
> www.mindclue.ch
>
>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com





--
Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Andreas Brodbeck-3
In reply to this post by Mariano Martinez Peck
Am 28.11.17 um 13:52 schrieb Mariano Martinez Peck:

> On Tue, Nov 28, 2017 at 9:31 AM, Andreas Brodbeck <[hidden email]> wrote:
>
>> Am 28.11.17 um 12:46 schrieb Mariano Martinez Peck:
>>> Hi Andreas,
>>>
>>> Do you know why method contexts are trying to be serialized? Of course,
>>> probably because of closures.  Do you think / know / are aware of
>> closures
>>> as part of your graph? Maybe Sorted Collection?
>>> I am asking because if the only thing is SortedCollection then we can use
>>> some hook...
>>
>> I have closures in several places in the object graph. These are objects
>> with some pluggable functionality, which I rely on.
>>
>
>
> *OK. The question is then if those closures are "clean" or not.* In other
> words, what's their scope? Do they refer to variables defined outside  the
> closure (in which case you would need the methods contexts / stack) or are
> they clean in the sense that you could be able to replace them via a string
> and then compile them again ?
> Some time ago we added a #isClean to BlockClosure. You can test a few of
> your closures to see if this is the case or not.
>
> Btw, I have an application for a client in which we also have closures to
> define some pluggable behavior. But what I do is:
>
> 1) guarantee they are 100% clean (does not go outside of scope)
> 2) aside from the "block" instVar I also add a>
> On Tue, Nov 28, 2017 at 9:31 AM, Andreas Brodbeck <[hidden email]> wrote:
>
>> Am 28.11.17 um 12:46 schrieb Mariano Martinez Peck:
>>> Hi Andreas,
>>>
>>> Do you know why method contexts are trying to be serialized? Of course,
>>> probably because of closures.  Do you think / know / are aware of
>> closures
>>> as part of your graph? Maybe Sorted Collection?
>>> I am asking because if the only thing is SortedCollection then we can use
>>> some hook...
>>
>> I have closures in several places in the object graph. These are objects
>> with some pluggable functionality, which I rely on.
>>
>
>
> *OK. The question is then if those closures are "clean" or not.* In other
> words, what's their scope? Do they refer to variables defined outside  the
> closure (in which case you would need the methods contexts / stack) or are
> they clean in the sense that you could be able to replace them via a string
> and then compile them again ?
> Some time ago we added a #isClean to BlockClosure. You can test a few of
> your closures to see if this is the case or not.
>
> Btw, I have an application for a client in which we also have closures to
> define some pluggable behavior. But what I do is:
>
> 1) guarantee they are 100% clean (does not go outside of scope)
> 2) aside from the "block" instVar I also add another instVar which is the
> "string" version of it. Then I always implement
> #fuelIgnoredInstanceVariableNames to ignore all those "block" instVars, yet
> DO NOT ignore the "string" like of those closures. Then, of course, the
> getter of the block instvar does a lazy compilation if the block instVar is
> nil...
>
> I know you already have your application written, but I am trying to
> explain how I was able to use Fuel for this case...

Just for the record, here is the solution what I ended up with:

--- Install Fuel version 1.9.4 for debugging:

(ConfigurationOfFuel project version: '1.9.4')
        load: #(default FuelDebug FuelPreview).

--- Analyze the fuel serialization of the root object myDatabaseObject
in the Pharo 1.4 image:

FileStream forceNewFileNamed: 'debug.fuel' do: [:aFile |
        FLSerializer newDefault
                setDebug;
                serialize: myDatabaseObject on: aFile binary].

FLDebugSerialization last log inspect.


--- In the inspected Fuel-Log, look for the FLCompiledMethodCluster,
which told me in which classes I use BlockClosures which fuel wants to
persist.

--- Changed my source code and the live objects where I use persisted
BlockClosures to only use "clean" BlockClosure and only fuel-store them
as Strings (Using fuelIgnoredInstanceVariableNames) an recompile them
when the instance is materialized (Using fuelAfterMaterialization), as
suggested by Mariano.

--- Searched for some few usages of SortedCollection and made sure, that
there is no sortBlock but only the default sort behaviour.


So finally I got rid of persisted BlockClosures in the fuel file. That
1.9.4 Fuel file stored with Pharo 1.4 could then be materialized in
Pharo 6.1. with the same Fuel version 1.9.4 without problems! I'm happy.


Cheers,
Andreas



--
Andreas Brodbeck
www.mindclue.ch


Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Mariano Martinez Peck


On Tue, Dec 5, 2017 at 6:39 AM, Andreas Brodbeck <[hidden email]> wrote:
Am 28.11.17 um 13:52 schrieb Mariano Martinez Peck:
> On Tue, Nov 28, 2017 at 9:31 AM, Andreas Brodbeck <[hidden email]> wrote:
>
>> Am 28.11.17 um 12:46 schrieb Mariano Martinez Peck:
>>> Hi Andreas,
>>>
>>> Do you know why method contexts are trying to be serialized? Of course,
>>> probably because of closures.  Do you think / know / are aware of
>> closures
>>> as part of your graph? Maybe Sorted Collection?
>>> I am asking because if the only thing is SortedCollection then we can use
>>> some hook...
>>
>> I have closures in several places in the object graph. These are objects
>> with some pluggable functionality, which I rely on.
>>
>
>
> *OK. The question is then if those closures are "clean" or not.* In other
> words, what's their scope? Do they refer to variables defined outside  the
> closure (in which case you would need the methods contexts / stack) or are
> they clean in the sense that you could be able to replace them via a string
> and then compile them again ?
> Some time ago we added a #isClean to BlockClosure. You can test a few of
> your closures to see if this is the case or not.
>
> Btw, I have an application for a client in which we also have closures to
> define some pluggable behavior. But what I do is:
>
> 1) guarantee they are 100% clean (does not go outside of scope)
> 2) aside from the "block" instVar I also add another instVar which is the
> "string" version of it. Then I always implement
> #fuelIgnoredInstanceVariableNames to ignore all those "block" instVars, yet
> DO NOT ignore the "string" like of those closures. Then, of course, the
> getter of the block instvar does a lazy compilation if the block instVar is
> nil...
>
> I know you already have your application written, but I am trying to
> explain how I was able to use Fuel for this case...

Just for the record, here is the solution what I ended up with:

--- Install Fuel version 1.9.4 for debugging:

(ConfigurationOfFuel project version: '1.9.4')
        load: #(default FuelDebug FuelPreview).

--- Analyze the fuel serialization of the root object myDatabaseObject
in the Pharo 1.4 image:

FileStream forceNewFileNamed: 'debug.fuel' do: [:aFile |
        FLSerializer newDefault
                setDebug;
                serialize: myDatabaseObject on: aFile binary].

FLDebugSerialization last log inspect.


--- In the inspected Fuel-Log, look for the FLCompiledMethodCluster,
which told me in which classes I use BlockClosures which fuel wants to
persist.

--- Changed my source code and the live objects where I use persisted
BlockClosures to only use "clean" BlockClosure and only fuel-store them
as Strings (Using fuelIgnoredInstanceVariableNames) an recompile them
when the instance is materialized (Using fuelAfterMaterialization), as
suggested by Mariano.

--- Searched for some few usages of SortedCollection and made sure, that
there is no sortBlock but only the default sort behaviour.


So finally I got rid of persisted BlockClosures in the fuel file. That
1.9.4 Fuel file stored with Pharo 1.4 could then be materialized in
Pharo 6.1. with the same Fuel version 1.9.4 without problems! I'm happy.




Awesome! That's exactly what I would have done. 



--
Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Sean P. DeNigris
Administrator
In reply to this post by Andreas Brodbeck-3
Andreas Brodbeck-3 wrote
> --- Searched for some few usages of SortedCollection and made sure, that
> there is no sortBlock but only the default sort behaviour.

Can't we have Fuel handle the sort block if it's clean? I feel like we
discussed this before but can't recall the conclusion…



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Andreas Brodbeck-3
Am 05.12.17 um 17:25 schrieb Sean P. DeNigris:
> Andreas Brodbeck-3 wrote
>> --- Searched for some few usages of SortedCollection and made sure, that
>> there is no sortBlock but only the default sort behaviour.
>
> Can't we have Fuel handle the sort block if it's clean? I feel like we
> discussed this before but can't recall the conclusion…

Fuel does handle blocks perfectly, as long as you don't upgrade the
underlying Pharo from version 1.4 to 6.1 (or similar big step), like I
did... The reason is the change of the bytecodes of a compiled method
somewhere between 1.4 and 6.1, which will then not materialize correctly
and raises a FLMethodChanged error. Therefore I had to deal with it,
like described before.

Cheers, Andreas


--
Andreas Brodbeck
www.mindclue.ch


Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Mariano Martinez Peck
Hi guys,

Let me try to explain the current situation. Fuel does check if a closure is clean or not. If clean, it avoids serializing the whole stack. The way it does this is to simply clean the outerContext of the block to NOT have a sender. See methods CompiledMethod >> fuelAccept: 
So..a clean closure would avoid serializing the sender and the sender of the sender and ... the whole stack. But... at the very least it does need to serialize the outerContext of that closure. That outerContext (even if the block is clean) has a reference to the method in which the closure was compiled. See this example:

TestCase compile: 'deleteMe

^ [ :aNumber | Transcript show: aNumber ]'.

(TestCase new perform: #deleteMe) cleanCopy outerContext method.


Do you  see how even a clean closure will still serialize ONE compiled method? (#deleteMe in this case). That is the method that could have changed bytecodes (remember that by default, Fuel does NOT fully serialize the methods if they are installed in classes...it will simply serialize the necessary information for looking it up at runtime) and that's where Fuel could bark even for clean closures. But of course, far less chances than serializing all the methods of the stack. 


A more robust solution might be that if the block is clean, then do not serialize the closure as a closure with a clean context, but as a string (source). Then at materialization time, compile. The problem is that as far as I remember we never implemented substitution hook at materialization (only for serialization) so I don't have a place to hook. Yeah... you could do a become: with a postMaterialziationAction but that;s too hackish hahaha. 

Hope this helps,



On Wed, Dec 6, 2017 at 9:58 AM, Andreas Brodbeck <[hidden email]> wrote:
Am 05.12.17 um 17:25 schrieb Sean P. DeNigris:
> Andreas Brodbeck-3 wrote
>> --- Searched for some few usages of SortedCollection and made sure, that
>> there is no sortBlock but only the default sort behaviour.
>
> Can't we have Fuel handle the sort block if it's clean? I feel like we
> discussed this before but can't recall the conclusion…

Fuel does handle blocks perfectly, as long as you don't upgrade the
underlying Pharo from version 1.4 to 6.1 (or similar big step), like I
did... The reason is the change of the bytecodes of a compiled method
somewhere between 1.4 and 6.1, which will then not materialize correctly
and raises a FLMethodChanged error. Therefore I had to deal with it,
like described before.

Cheers, Andreas


--
Andreas Brodbeck
www.mindclue.ch





--
Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Andreas Brodbeck-3
In reply to this post by Andreas Brodbeck-3
As this topic has led to a solution for me, here is a more convenient
document:

https://gist.github.com/dassi/9da1aa153635f3df99643149d111021f

Cheers, Andreas

Am 28.11.17 um 11:01 schrieb Andreas Brodbeck:

> Hi all,
>
> I have a long running Seaside/Pharo application which still runs perfect
> on Pharo 1.4. I want to step into the future and migrate to the newest
> Pharo 6.1.
>
> The application's persistence is just in the image, so my plan is to use
> Fuel to transfer the objects from Pharo 1.4 to Pharo 6.1.
>
> [...]


--
Andreas Brodbeck
www.mindclue.ch


Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Sean P. DeNigris
Administrator
In reply to this post by Andreas Brodbeck-3
Andreas Brodbeck-3 wrote
> --- Install Fuel version 1.9.4 for debugging:
> (ConfigurationOfFuel project version: '1.9.4')
> load: #(default FuelDebug FuelPreview).

Hmmm. In Pharo #60510, I got: MetacelloProjectSpecLoadError: No version
found for '1.61' of ConfigurationOfRoassal because: Version '1.61' is not
defined in ConfigurationOfRoassal. Possible versions include:
#(#bleedingEdge #development #stable '0.5-baseline' '0.6-baseline'
'0.7-baseline' '1.6-snapshot' '1.400' '1.401' '1.402' '1.403' '1.404'
'1.405' '1.406' '1.407' '1.408' '1.409' '1.410' '1.411' '1.412' '1.413'
'1.414' '1.415' '1.416' '1.417' '1.418' '1.419' '1.420' '1.421' '1.422'
'1.423' '1.424' '1.425' '1.426' '1.427' '1.428-snapshot' '1.429' '1.430')



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Sean P. DeNigris
Administrator
Sean P. DeNigris wrote
> I got: MetacelloProjectSpecLoadError:…

I then removed the FuelPreview group and got a missing NativeBoost dependecy
(NBExternalHandle)!



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: Fuel transfer from Pharo 1.4 to Pharo 6.1

Stephane Ducasse-3
In reply to this post by Andreas Brodbeck-3
I would use STON because this is a solid solution.

Stef

On Tue, Nov 28, 2017 at 1:20 PM, Andreas Brodbeck <[hidden email]> wrote:

> Am 28.11.17 um 11:13 schrieb H. Hirzel:
>> Hello Andreas
>>
>> Unfortunately Fuel is not meant for this kind of use case of migrating
>> an object model.
>
> That's sad to hear. From 1.4 to 6.1 is a big step, I agree, but I really
> thought that Fuel can be considered as a good "transport vehicle" at
> least for small steps like from Pharo 5 to 6.
>
>>
>> So other options have to be considered.
>
> That probably would be a custom mechanism storing objects in a database
> or fileformat and re-import, I think. Lots of work and very error prone.
> But maybe the only way unfortunately ...
>
> SIXX would be an option you think?
>
>>
>> Does your object model have cycles?
>
> Yes.
>
>
> Regards,
> Andreas
>
> --
> Andreas Brodbeck
> www.mindclue.ch
>
>