Smalltalk › Pharo › Pharo Smalltalk Developers

Fuel materialization on spur

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

30 messages Options

NorbertHartl

Fuel materialization on spur

We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course.

I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?

thanks,

Norbert

primitive #value: in Character class failed

Stacktrace

Character class(Object)>>primitiveFailed:
Character class(Object)>>primitiveFailed
Character class>>value:
Character class>>materializeFrom:
FLHookPrimitiveCluster>>materializeInstanceWith:
FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith:
FLMaterialization>>clusterInstancesStep
[ self clusterInstancesStep ] in FLMaterialization>>instancesStep
SmallInteger(Integer)>>timesRepeat:
FLMaterialization>>instancesStep
FLMaterialization>>run
[ :aDecoder | 
(FLMaterialization with: aDecoder)
	run;
	yourself ] in FLMaterializer>>setDefaultMaterialization
FLMaterializer>>materializeFrom:
FLMaterializer class>>materializeFromByteArray:
MAPExampleModels class>>readModelFromSelector:
MAPExampleModels class>>readModelNamed:
[ self readModelNamed: aString ] in MAPExampleModels class>>named:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
MAPExampleModels class>>named:
MAPTest>>model
MAPTest>>tcapModule
MAPTest>>testInsertSubscriberDataBitStringAccess
[ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
Given>>produceReturnValueAt:
MAPTest(Phexample)>>performTest

Robert Withers

Re: Fuel materialization on spur

On 12/15/2015 07:18 AM, Norbert Hartl wrote:

We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course.
I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that...

or will it just not be possible to exchange objects between spur and non-spur using fuel?

thanks,

Norbert

primitive #value: in Character class failed

Stacktrace

Character class(Object)>>primitiveFailed:
Character class(Object)>>primitiveFailed
Character class>>value:
Character class>>materializeFrom:
FLHookPrimitiveCluster>>materializeInstanceWith:
FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith:
FLMaterialization>>clusterInstancesStep
[ self clusterInstancesStep ] in FLMaterialization>>instancesStep
SmallInteger(Integer)>>timesRepeat:
FLMaterialization>>instancesStep
FLMaterialization>>run
[ :aDecoder | 
(FLMaterialization with: aDecoder)
	run;
	yourself ] in FLMaterializer>>setDefaultMaterialization
FLMaterializer>>materializeFrom:
FLMaterializer class>>materializeFromByteArray:
MAPExampleModels class>>readModelFromSelector:
MAPExampleModels class>>readModelNamed:
[ self readModelNamed: aString ] in MAPExampleModels class>>named:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
MAPExampleModels class>>named:
MAPTest>>model
MAPTest>>tcapModule
MAPTest>>testInsertSubscriberDataBitStringAccess
[ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
Given>>produceReturnValueAt:
MAPTest(Phexample)>>performTest

. .. .. ^,^ best, robert

EstebanLM

Re: Fuel materialization on spur

In reply to this post by NorbertHartl

On 15 Dec 2015, at 13:18, Norbert Hartl <[hidden email]> wrote:

We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course.
I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?

I think is not possible, but I will let fuel maintainers to have the last word :)

Esteban

thanks,

Norbert

primitive #value: in Character class failed

Stacktrace

Character class(Object)>>primitiveFailed:
Character class(Object)>>primitiveFailed
Character class>>value:
Character class>>materializeFrom:
FLHookPrimitiveCluster>>materializeInstanceWith:
FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith:
FLMaterialization>>clusterInstancesStep
[ self clusterInstancesStep ] in FLMaterialization>>instancesStep
SmallInteger(Integer)>>timesRepeat:
FLMaterialization>>instancesStep
FLMaterialization>>run
[ :aDecoder | 
(FLMaterialization with: aDecoder)
	run;
	yourself ] in FLMaterializer>>setDefaultMaterialization
FLMaterializer>>materializeFrom:
FLMaterializer class>>materializeFromByteArray:
MAPExampleModels class>>readModelFromSelector:
MAPExampleModels class>>readModelNamed:
[ self readModelNamed: aString ] in MAPExampleModels class>>named:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
MAPExampleModels class>>named:
MAPTest>>model
MAPTest>>tcapModule
MAPTest>>testInsertSubscriberDataBitStringAccess
[ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
Given>>produceReturnValueAt:
MAPTest(Phexample)>>performTest

Max Leske

Re: Fuel materialization on spur

In reply to this post by NorbertHartl

Characters are now primitive. There shouldn’t be any need to change Character serialization though.

I tried to reproduce your case and I can’t. The only way I can get the primitive to fail is by supplying a negative value. I suspect that the problem happens beforehand, so that the byte being read is now at the wrong offset. Can you give me a runnable example?

Cheers,

Max

On 15 Dec 2015, at 13:18, Norbert Hartl <[hidden email]> wrote:

We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course.

thanks,

Norbert

primitive #value: in Character class failed

Stacktrace

Character class(Object)>>primitiveFailed:
Character class(Object)>>primitiveFailed
Character class>>value:
Character class>>materializeFrom:
FLHookPrimitiveCluster>>materializeInstanceWith:
FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith:
FLMaterialization>>clusterInstancesStep
[ self clusterInstancesStep ] in FLMaterialization>>instancesStep
SmallInteger(Integer)>>timesRepeat:
FLMaterialization>>instancesStep
FLMaterialization>>run
[ :aDecoder | 
(FLMaterialization with: aDecoder)
	run;
	yourself ] in FLMaterializer>>setDefaultMaterialization
FLMaterializer>>materializeFrom:
FLMaterializer class>>materializeFromByteArray:
MAPExampleModels class>>readModelFromSelector:
MAPExampleModels class>>readModelNamed:
[ self readModelNamed: aString ] in MAPExampleModels class>>named:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
MAPExampleModels class>>named:
MAPTest>>model
MAPTest>>tcapModule
MAPTest>>testInsertSubscriberDataBitStringAccess
[ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
Given>>produceReturnValueAt:
MAPTest(Phexample)>>performTest

Max Leske

Re: Fuel materialization on spur

In reply to this post by EstebanLM

On 15 Dec 2015, at 14:06, Esteban Lorenzano <[hidden email]> wrote:

On 15 Dec 2015, at 13:18, Norbert Hartl <[hidden email]> wrote:

We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course.
I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?

I think is not possible, but I will let fuel maintainers to have the last word :)

That’s right. In general it’s not possible (at the moment). If there’s demand, we could try to come up with a conversion package which can be used to load non-spur files.

Esteban

thanks,

Norbert

primitive #value: in Character class failed

Stacktrace

Character class(Object)>>primitiveFailed:
Character class(Object)>>primitiveFailed
Character class>>value:
Character class>>materializeFrom:
FLHookPrimitiveCluster>>materializeInstanceWith:
FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith:
FLMaterialization>>clusterInstancesStep
[ self clusterInstancesStep ] in FLMaterialization>>instancesStep
SmallInteger(Integer)>>timesRepeat:
FLMaterialization>>instancesStep
FLMaterialization>>run
[ :aDecoder | 
(FLMaterialization with: aDecoder)
	run;
	yourself ] in FLMaterializer>>setDefaultMaterialization
FLMaterializer>>materializeFrom:
FLMaterializer class>>materializeFromByteArray:
MAPExampleModels class>>readModelFromSelector:
MAPExampleModels class>>readModelNamed:
[ self readModelNamed: aString ] in MAPExampleModels class>>named:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
MAPExampleModels class>>named:
MAPTest>>model
MAPTest>>tcapModule
MAPTest>>testInsertSubscriberDataBitStringAccess
[ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
Given>>produceReturnValueAt:
MAPTest(Phexample)>>performTest

Eliot Miranda-2

Re: Fuel materialization on spur

In reply to this post by EstebanLM

Hi Esteban,

On Dec 15, 2015, at 5:06 AM, Esteban Lorenzano <[hidden email]> wrote:

On 15 Dec 2015, at 13:18, Norbert Hartl <[hidden email]> wrote:

We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course.
I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?

I think is not possible, but I will let fuel maintainers to have the last word :)

Of course it's possible; in VW we exchanged objects between 32- & 64-but images using a format much like fuel and in 32-bits all floats are arrays of 32-bit bit objects whereas in 64-bits a range of floats are immediate.

In spur characters are immediate (Max, "primitive" is not the right term) whereas in v3 they're objects with a single isn't var. So at least fuel needs to be modified to read characters specially. One way to architect this is to ask the class (SmallInteger, Character, Float etc) to materialize, and then the class can decide how to represent its instances. The question is whether characters in spur should serialize themselves using the same wire format as v3 or whether fuel should modify its wire format to serialize immediate characters specially and have v3 materialize characters as serialized by spur. The issue is whether it's important to preserve object identity of characters with code >= 256 or not.

Esteban

thanks,

Norbert

primitive #value: in Character class failed

Stacktrace

Character class(Object)>>primitiveFailed:
Character class(Object)>>primitiveFailed
Character class>>value:
Character class>>materializeFrom:
FLHookPrimitiveCluster>>materializeInstanceWith:
FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith:
FLMaterialization>>clusterInstancesStep
[ self clusterInstancesStep ] in FLMaterialization>>instancesStep
SmallInteger(Integer)>>timesRepeat:
FLMaterialization>>instancesStep
FLMaterialization>>run
[ :aDecoder | 
(FLMaterialization with: aDecoder)
	run;
	yourself ] in FLMaterializer>>setDefaultMaterialization
FLMaterializer>>materializeFrom:
FLMaterializer class>>materializeFromByteArray:
MAPExampleModels class>>readModelFromSelector:
MAPExampleModels class>>readModelNamed:
[ self readModelNamed: aString ] in MAPExampleModels class>>named:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
MAPExampleModels class>>named:
MAPTest>>model
MAPTest>>tcapModule
MAPTest>>testInsertSubscriberDataBitStringAccess
[ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
Given>>produceReturnValueAt:
MAPTest(Phexample)>>performTest

EstebanLM

Re: Fuel materialization on spur

Hi,

On 16 Dec 2015, at 08:21, Eliot Miranda <[hidden email]> wrote:

Hi Esteban,

On Dec 15, 2015, at 5:06 AM, Esteban Lorenzano <[hidden email]> wrote:

On 15 Dec 2015, at 13:18, Norbert Hartl <[hidden email]> wrote:

We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course.
I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?

I think is not possible, but I will let fuel maintainers to have the last word :)

Of course it's possible; in VW we exchanged objects between 32- & 64-but images using a format much like fuel and in 32-bits all floats are arrays of 32-bit bit objects whereas in 64-bits a range of floats are immediate.

Oh well… of course is possible. What I meant is that I think current implementation of Fuel does not support it, just that :)

but again, I’m not sure: Martin and Mariano can for sure say something more detailed about :)

(problem is Martin is on vacations after getting his PhD and Mariano didn’t followed in close the spur migration)

cheers,

Esteban

In spur characters are immediate (Max, "primitive" is not the right term) whereas in v3 they're objects with a single isn't var. So at least fuel needs to be modified to read characters specially. One way to architect this is to ask the class (SmallInteger, Character, Float etc) to materialize, and then the class can decide how to represent its instances. The question is whether characters in spur should serialize themselves using the same wire format as v3 or whether fuel should modify its wire format to serialize immediate characters specially and have v3 materialize characters as serialized by spur. The issue is whether it's important to preserve object identity of characters with code >= 256 or not.
Esteban
thanks,

Norbert
primitive #value: in Character class failed
Stacktrace
Character class(Object)>>primitiveFailed:
Character class(Object)>>primitiveFailed
Character class>>value:
Character class>>materializeFrom:
FLHookPrimitiveCluster>>materializeInstanceWith:
FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith:
FLMaterialization>>clusterInstancesStep
[ self clusterInstancesStep ] in FLMaterialization>>instancesStep
SmallInteger(Integer)>>timesRepeat:
FLMaterialization>>instancesStep
FLMaterialization>>run
[ :aDecoder | 
(FLMaterialization with: aDecoder)
	run;
	yourself ] in FLMaterializer>>setDefaultMaterialization
FLMaterializer>>materializeFrom:
FLMaterializer class>>materializeFromByteArray:
MAPExampleModels class>>readModelFromSelector:
MAPExampleModels class>>readModelNamed:
[ self readModelNamed: aString ] in MAPExampleModels class>>named:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
MAPExampleModels class>>named:
MAPTest>>model
MAPTest>>tcapModule
MAPTest>>testInsertSubscriberDataBitStringAccess
[ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
Given>>produceReturnValueAt:
MAPTest(Phexample)>>performTest

Clément Béra

Re: Fuel materialization on spur

I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.

2015-12-16 10:34 GMT+01:00 Esteban Lorenzano <[hidden email]>:

Hi,
On 16 Dec 2015, at 08:21, Eliot Miranda <[hidden email]> wrote:

Hi Esteban,

On Dec 15, 2015, at 5:06 AM, Esteban Lorenzano <[hidden email]> wrote:

On 15 Dec 2015, at 13:18, Norbert Hartl <[hidden email]> wrote:

We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course.
I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?

I think is not possible, but I will let fuel maintainers to have the last word :)

Of course it's possible; in VW we exchanged objects between 32- & 64-but images using a format much like fuel and in 32-bits all floats are arrays of 32-bit bit objects whereas in 64-bits a range of floats are immediate.

Oh well… of course is possible. What I meant is that I think current implementation of Fuel does not support it, just that :)
but again, I’m not sure: Martin and Mariano can for sure say something more detailed about :)
(problem is Martin is on vacations after getting his PhD and Mariano didn’t followed in close the spur migration)

cheers,
Esteban
In spur characters are immediate (Max, "primitive" is not the right term) whereas in v3 they're objects with a single isn't var. So at least fuel needs to be modified to read characters specially. One way to architect this is to ask the class (SmallInteger, Character, Float etc) to materialize, and then the class can decide how to represent its instances. The question is whether characters in spur should serialize themselves using the same wire format as v3 or whether fuel should modify its wire format to serialize immediate characters specially and have v3 materialize characters as serialized by spur. The issue is whether it's important to preserve object identity of characters with code >= 256 or not.
Esteban
thanks,

Norbert
primitive #value: in Character class failed
Stacktrace
Character class(Object)>>primitiveFailed:
Character class(Object)>>primitiveFailed
Character class>>value:
Character class>>materializeFrom:
FLHookPrimitiveCluster>>materializeInstanceWith:
FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith:
FLMaterialization>>clusterInstancesStep
[ self clusterInstancesStep ] in FLMaterialization>>instancesStep
SmallInteger(Integer)>>timesRepeat:
FLMaterialization>>instancesStep
FLMaterialization>>run
[ :aDecoder | 
(FLMaterialization with: aDecoder)
	run;
	yourself ] in FLMaterializer>>setDefaultMaterialization
FLMaterializer>>materializeFrom:
FLMaterializer class>>materializeFromByteArray:
MAPExampleModels class>>readModelFromSelector:
MAPExampleModels class>>readModelNamed:
[ self readModelNamed: aString ] in MAPExampleModels class>>named:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
MAPExampleModels class>>named:
MAPTest>>model
MAPTest>>tcapModule
MAPTest>>testInsertSubscriberDataBitStringAccess
[ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt:
[ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
Dictionary>>at:ifAbsent:
Dictionary>>at:ifAbsentPut:
Given>>produceReturnValueAt:
MAPTest(Phexample)>>performTest

Sven Van Caekenberghe-2

Re: Fuel materialization on spur

> On 16 Dec 2015, at 13:24, Clément Bera <[hidden email]> wrote:
>
> I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.

I don't know.

Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?

From my POV FUEL is only compatible with its own version, a sacrifice made to gain performance. IIUC it is not a stable external format for long lived data with backward/forward compatibility.

> 2015-12-16 10:34 GMT+01:00 Esteban Lorenzano <[hidden email]>:
> Hi,
>
>> On 16 Dec 2015, at 08:21, Eliot Miranda <[hidden email]> wrote:
>>
>> Hi Esteban,
>>
>> On Dec 15, 2015, at 5:06 AM, Esteban Lorenzano <[hidden email]> wrote:
>>
>>>
>>>> On 15 Dec 2015, at 13:18, Norbert Hartl <[hidden email]> wrote:
>>>>
>>>> We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course.
>>>> I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?
>>>
>>> I think is not possible, but I will let fuel maintainers to have the last word :)
>>
>>
>> Of course it's possible; in VW we exchanged objects between 32- & 64-but images using a format much like fuel and in 32-bits all floats are arrays of 32-bit bit objects whereas in 64-bits a range of floats are immediate.
>
> Oh well… of course is possible. What I meant is that I think current implementation of Fuel does not support it, just that :)
> but again, I’m not sure: Martin and Mariano can for sure say something more detailed about :)
> (problem is Martin is on vacations after getting his PhD and Mariano didn’t followed in close the spur migration)
>
> cheers,
> Esteban
>
>>
>> In spur characters are immediate (Max, "primitive" is not the right term) whereas in v3 they're objects with a single isn't var. So at least fuel needs to be modified to read characters specially. One way to architect this is to ask the class (SmallInteger, Character, Float etc) to materialize, and then the class can decide how to represent its instances. The question is whether characters in spur should serialize themselves using the same wire format as v3 or whether fuel should modify its wire format to serialize immediate characters specially and have v3 materialize characters as serialized by spur. The issue is whether it's important to preserve object identity of characters with code >= 256 or not.
>>
>>>
>>> Esteban
>>>
>>>>
>>>> thanks,
>>>>
>>>> Norbert
>>>>
>>>>
>>>> primitive #value: in Character class failed
>>>> Stacktrace
>>>>
>>>> Character class(Object)>>primitiveFailed:
>>>> Character class(Object)>>primitiveFailed
>>>> Character class>>value:
>>>> Character class>>materializeFrom:
>>>> FLHookPrimitiveCluster>>materializeInstanceWith:
>>>> FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith:
>>>> FLMaterialization>>clusterInstancesStep
>>>> [ self clusterInstancesStep ] in FLMaterialization>>instancesStep
>>>> SmallInteger(Integer)>>timesRepeat:
>>>> FLMaterialization>>instancesStep
>>>> FLMaterialization>>run
>>>> [ :aDecoder |
>>>> (FLMaterialization with: aDecoder)
>>>> run;
>>>> yourself ] in FLMaterializer>>setDefaultMaterialization
>>>> FLMaterializer>>materializeFrom:
>>>> FLMaterializer class>>materializeFromByteArray:
>>>> MAPExampleModels class>>readModelFromSelector:
>>>> MAPExampleModels class>>readModelNamed:
>>>> [ self readModelNamed: aString ] in MAPExampleModels class>>named:
>>>> [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
>>>> Dictionary>>at:ifAbsent:
>>>> Dictionary>>at:ifAbsentPut:
>>>> MAPExampleModels class>>named:
>>>> MAPTest>>model
>>>> MAPTest>>tcapModule
>>>> MAPTest>>testInsertSubscriberDataBitStringAccess
>>>> [ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt:
>>>> [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut:
>>>> Dictionary>>at:ifAbsent:
>>>> Dictionary>>at:ifAbsentPut:
>>>> Given>>produceReturnValueAt:
>>>> MAPTest(Phexample)>>performTest
>>>>
>>>>
>>>>
>>>
>
>

Denis Kudriashov

Re: Fuel materialization on spur

Hi.

2015-12-16 13:33 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:

> On 16 Dec 2015, at 13:24, Clément Bera <[hidden email]> wrote:
>
> I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.

I don't know.

Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?

It is needed to use fuel as communication format between services which run on different platforms. For example you can exchange data between 64bits server and 32bits android phone.

Sven Van Caekenberghe-2

Re: Fuel materialization on spur

> On 16 Dec 2015, at 13:46, Denis Kudriashov <[hidden email]> wrote:
>
> Hi.
>
> 2015-12-16 13:33 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:
> > On 16 Dec 2015, at 13:24, Clément Bera <[hidden email]> wrote:
> >
> > I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.
>
> I don't know.
>
> Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?
>
> It is needed to use fuel as communication format between services which run on different platforms. For example you can exchange data between 64bits server and 32bits android phone.

What you want might not be the same to what it was designed for.

I might be wrong, but that is how I understood it from discussion in the past when moving data from one fuel version to another.

Let's see what the Fuel guys have to say about this.

Denis Kudriashov

Re: Fuel materialization on spur

2015-12-16 13:49 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:

I might be wrong, but that is how I understood it from discussion in the past when moving data from one fuel version to another.

Will see what they answer. But supporting different versions of Fuel is not same as supporting single format for any platform

Mariano Martinez Peck

Re: Fuel materialization on spur

In reply to this post by Sven Van Caekenberghe-2

Hi guys,

I would like to say a couple of things:

1) Fuel format: Sven is right in the sense that Fuel has problems of migrating to another Fuel version. However.... there is some explanation needed here. Previously (years ago), a given Fuel version was not compatible with previous versions because we were still changing/improving what we call Fuel format, that is, basically, how objects are written and read from the stream. We have stopped with those optimization/changes years ago. So the "Fuel format" has been stable so far for a couple of years.

2) Fuel gets in the internals: In Pharo 4.0 Fuel was not compatible with Fuel from 3.0. I don't remember the exact detail but it was a change in Pharo's Date (or DateAndTime I cannot remember). Now the version that should work in 5.0 changes the Characater serialization. What I want to make clear here is that it's NOT that we are changing the fuel format 1) as years ago when Fuel as still evolving from this point of view. It's Pharo changes that make our incompatibility. But that doesn't mean we cannot fix it.

3) Yes, compromises for speed: I think that both examples that break compatibility in 4.0 (Date / DateAndTime change) and in 5.0 with Spur (Character) is because we target speed. We do have special clusters for both of those cases (Date and Character), and so, we get into it's internals for performance reasons. If the internals change, then we break. This is why a text serializer for example may simply not break, because it continues using text representation that might not have been affected by the underlaying internal representation.

We will try to do a Pharo 40 - Pharo 5.0 (or spur actually) compatibility.

Best,

On Wed, Dec 16, 2015 at 9:49 AM, Sven Van Caekenberghe <[hidden email]> wrote:

> On 16 Dec 2015, at 13:46, Denis Kudriashov <[hidden email]> wrote:
>
> Hi.
>
> 2015-12-16 13:33 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:
> > On 16 Dec 2015, at 13:24, Clément Bera <[hidden email]> wrote:
> >
> > I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.
>
> I don't know.
>
> Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?
>
> It is needed to use fuel as communication format between services which run on different platforms. For example you can exchange data between 64bits server and 32bits android phone.

What you want might not be the same to what it was designed for.

I might be wrong, but that is how I understood it from discussion in the past when moving data from one fuel version to another.

Let's see what the Fuel guys have to say about this.

Mariano
http://marianopeck.wordpress.com

Sven Van Caekenberghe-2

Re: Fuel materialization on spur

OK, thanks for the explanation, good to hear it.

(But, just nitpicking, it still does not say officially 'you can safely save your data in Fuel format and rest assured that you will always be able to read it back').

(Also, if there are compatibility problems between Pharo versions (understandably Pharo's doing), how can there be compatibility with other Smalltalk implementation, not that I expect any, but again, what is the official word ?)

> On 16 Dec 2015, at 14:10, Mariano Martinez Peck <[hidden email]> wrote:
>
> Hi guys,
>
> I would like to say a couple of things:
>
> 1) Fuel format: Sven is right in the sense that Fuel has problems of migrating to another Fuel version. However.... there is some explanation needed here. Previously (years ago), a given Fuel version was not compatible with previous versions because we were still changing/improving what we call Fuel format, that is, basically, how objects are written and read from the stream. We have stopped with those optimization/changes years ago. So the "Fuel format" has been stable so far for a couple of years.
>
> 2) Fuel gets in the internals: In Pharo 4.0 Fuel was not compatible with Fuel from 3.0. I don't remember the exact detail but it was a change in Pharo's Date (or DateAndTime I cannot remember). Now the version that should work in 5.0 changes the Characater serialization. What I want to make clear here is that it's NOT that we are changing the fuel format 1) as years ago when Fuel as still evolving from this point of view. It's Pharo changes that make our incompatibility. But that doesn't mean we cannot fix it.
>
> 3) Yes, compromises for speed: I think that both examples that break compatibility in 4.0 (Date / DateAndTime change) and in 5.0 with Spur (Character) is because we target speed. We do have special clusters for both of those cases (Date and Character), and so, we get into it's internals for performance reasons. If the internals change, then we break. This is why a text serializer for example may simply not break, because it continues using text representation that might not have been affected by the underlaying internal representation.
>
> We will try to do a Pharo 40 - Pharo 5.0 (or spur actually) compatibility.
>
> Best,
>
>
> On Wed, Dec 16, 2015 at 9:49 AM, Sven Van Caekenberghe <[hidden email]> wrote:
>
> > On 16 Dec 2015, at 13:46, Denis Kudriashov <[hidden email]> wrote:
> >
> > Hi.
> >
> > 2015-12-16 13:33 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:
> > > On 16 Dec 2015, at 13:24, Clément Bera <[hidden email]> wrote:
> > >
> > > I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.
> >
> > I don't know.
> >
> > Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?
> >
> > It is needed to use fuel as communication format between services which run on different platforms. For example you can exchange data between 64bits server and 32bits android phone.
>
> What you want might not be the same to what it was designed for.
>
> I might be wrong, but that is how I understood it from discussion in the past when moving data from one fuel version to another.
>
> Let's see what the Fuel guys have to say about this.
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com

Mariano Martinez Peck

Re: Fuel materialization on spur

On Wed, Dec 16, 2015 at 10:17 AM, Sven Van Caekenberghe <[hidden email]> wrote:

OK, thanks for the explanation, good to hear it.

(But, just nitpicking, it still does not say officially 'you can safely save your data in Fuel format and rest assured that you will always be able to read it back').

No, I can't say that (at least myself). It does mean it's not possible, it simply means that it will take some time to make it work and so if nobody does it, it won't work.

However, note that the efforts seem to have been much smaller than years ago. For example, with very little changes, it's likely we could have manage automatically from Pharo 3.0 until now.

(Also, if there are compatibility problems between Pharo versions (understandably Pharo's doing), how can there be compatibility with other Smalltalk implementation, not that I expect any, but again, what is the official word ?)

There is none compatibility with other Smalltalk versions in the sense that a serialized stream in one st could be materialized in another st.

However, note that what we DO support is that Fuel works in every possible version of Pharo and in a wide range of latest squeak images.

> On 16 Dec 2015, at 14:10, Mariano Martinez Peck <[hidden email]> wrote:
>
> Hi guys,
>
> I would like to say a couple of things:
>
> 1) Fuel format: Sven is right in the sense that Fuel has problems of migrating to another Fuel version. However.... there is some explanation needed here. Previously (years ago), a given Fuel version was not compatible with previous versions because we were still changing/improving what we call Fuel format, that is, basically, how objects are written and read from the stream. We have stopped with those optimization/changes years ago. So the "Fuel format" has been stable so far for a couple of years.
>
> 2) Fuel gets in the internals: In Pharo 4.0 Fuel was not compatible with Fuel from 3.0. I don't remember the exact detail but it was a change in Pharo's Date (or DateAndTime I cannot remember). Now the version that should work in 5.0 changes the Characater serialization. What I want to make clear here is that it's NOT that we are changing the fuel format 1) as years ago when Fuel as still evolving from this point of view. It's Pharo changes that make our incompatibility. But that doesn't mean we cannot fix it.
>
> 3) Yes, compromises for speed: I think that both examples that break compatibility in 4.0 (Date / DateAndTime change) and in 5.0 with Spur (Character) is because we target speed. We do have special clusters for both of those cases (Date and Character), and so, we get into it's internals for performance reasons. If the internals change, then we break. This is why a text serializer for example may simply not break, because it continues using text representation that might not have been affected by the underlaying internal representation.
>
> We will try to do a Pharo 40 - Pharo 5.0 (or spur actually) compatibility.
>
> Best,
>
>
> On Wed, Dec 16, 2015 at 9:49 AM, Sven Van Caekenberghe <[hidden email]> wrote:
>
> > On 16 Dec 2015, at 13:46, Denis Kudriashov <[hidden email]> wrote:
> >
> > Hi.
> >
> > 2015-12-16 13:33 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:
> > > On 16 Dec 2015, at 13:24, Clément Bera <[hidden email]> wrote:
> > >
> > > I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.
> >
> > I don't know.
> >
> > Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?
> >
> > It is needed to use fuel as communication format between services which run on different platforms. For example you can exchange data between 64bits server and 32bits android phone.
>
> What you want might not be the same to what it was designed for.
>
> I might be wrong, but that is how I understood it from discussion in the past when moving data from one fuel version to another.
>
> Let's see what the Fuel guys have to say about this.
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com

Mariano
http://marianopeck.wordpress.com

Hannes Hirzel

Re: Fuel materialization on spur

It probably should be noted here as well that

http://pharo.gemtalksystems.com/book/PharoTools/SIXX/
(Smalltalk Instance eXchange in XML)
http://www.mars.dti.ne.jp/~umejava/smalltalk/sixx/index.html

allows to transfer data from one image version to another image
version / Smalltalk dialect.

The format is useful for long-term storage. It is actively maintained.

--Hannes

On 12/16/15, Mariano Martinez Peck <[hidden email]> wrote:

> On Wed, Dec 16, 2015 at 10:17 AM, Sven Van Caekenberghe <[hidden email]>
> wrote:
>
>> OK, thanks for the explanation, good to hear it.
>>
>> (But, just nitpicking, it still does not say officially 'you can safely
>> save your data in Fuel format and rest assured that you will always be
>> able
>> to read it back').
>>
>>
> No, I can't say that (at least myself). It does mean it's not possible, it
> simply means that it will take some time to make it work and so if nobody
> does it, it won't work.
>
> However, note that the efforts seem to have been much smaller than years
> ago. For example, with very little changes, it's likely we could have
> manage automatically from Pharo 3.0 until now.
>
>
>> (Also, if there are compatibility problems between Pharo versions
>> (understandably Pharo's doing), how can there be compatibility with other
>> Smalltalk implementation, not that I expect any, but again, what is the
>> official word ?)
>>
>
> There is none compatibility with other Smalltalk versions in the sense that
> a serialized stream in one st could be materialized in another st.
>
> However, note that what we DO support is that Fuel works in every possible
> version of Pharo and in a wide range of latest squeak images.
>
>
>>
>> > On 16 Dec 2015, at 14:10, Mariano Martinez Peck <[hidden email]>
>> wrote:
>> >
>> > Hi guys,
>> >
>> > I would like to say a couple of things:
>> >
>> > 1) Fuel format: Sven is right in the sense that Fuel has problems of
>> migrating to another Fuel version. However.... there is some explanation
>> needed here. Previously (years ago), a given Fuel version was not
>> compatible with previous versions because we were still
>> changing/improving
>> what we call Fuel format, that is, basically, how objects are written and
>> read from the stream. We have stopped with those optimization/changes
>> years
>> ago. So the "Fuel format" has been stable so far for a couple of years.
>> >
>> > 2) Fuel gets in the internals: In Pharo 4.0 Fuel was not compatible
>> with Fuel from 3.0. I don't remember the exact detail but it was a change
>> in Pharo's Date (or DateAndTime I cannot remember). Now the version that
>> should work in 5.0 changes the Characater serialization. What I want to
>> make clear here is that it's NOT that we are changing the fuel format 1)
>> as
>> years ago when Fuel as still evolving from this point of view. It's Pharo
>> changes that make our incompatibility. But that doesn't mean we cannot
>> fix
>> it.
>> >
>> > 3) Yes, compromises for speed: I think that both examples that break
>> compatibility in 4.0 (Date / DateAndTime change) and in 5.0 with Spur
>> (Character) is because we target speed. We do have special clusters for
>> both of those cases (Date and Character), and so, we get into it's
>> internals for performance reasons. If the internals change, then we
>> break.
>> This is why a text serializer for example may simply not break, because
>> it
>> continues using text representation that might not have been affected by
>> the underlaying internal representation.
>> >
>> > We will try to do a Pharo 40 - Pharo 5.0 (or spur actually)
>> compatibility.
>> >
>> > Best,
>> >
>> >
>> > On Wed, Dec 16, 2015 at 9:49 AM, Sven Van Caekenberghe <[hidden email]>
>> wrote:
>> >
>> > > On 16 Dec 2015, at 13:46, Denis Kudriashov <[hidden email]>
>> wrote:
>> > >
>> > > Hi.
>> > >
>> > > 2015-12-16 13:33 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:
>> > > > On 16 Dec 2015, at 13:24, Clément Bera <[hidden email]>
>> wrote:
>> > > >
>> > > > I don't know if SqueakV3 - Spur32 support is needed, however, I
>> think it is really important to be able to serialize and materialize
>> objects between 32 bits spur images and 64bits spur images.
>> > >
>> > > I don't know.
>> > >
>> > > Yes it would be nice, but since Fuel cannot even exchange data
>> > > between
>> different version itself (If I understand the situation correctly), why
>> should that have to work ?
>> > >
>> > > It is needed to use fuel as communication format between services
>> which run on different platforms. For example you can exchange data
>> between
>> 64bits server and 32bits android phone.
>> >
>> > What you want might not be the same to what it was designed for.
>> >
>> > I might be wrong, but that is how I understood it from discussion in
>> > the
>> past when moving data from one fuel version to another.
>> >
>> > Let's see what the Fuel guys have to say about this.
>> >
>> >
>> >
>> > --
>> > Mariano
>> > http://marianopeck.wordpress.com
>>
>>
>>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>

Denis Kudriashov

Re: Fuel materialization on spur

2015-12-16 14:41 GMT+01:00 H. Hirzel <[hidden email]>:

It probably should be noted here as well that

http://pharo.gemtalksystems.com/book/PharoTools/SIXX/

Just to know does java implementation exists?

Hannes Hirzel

Re: Fuel materialization on spur

No, it is a Smalltalk format.

On 12/16/15, Denis Kudriashov <[hidden email]> wrote:
> 2015-12-16 14:41 GMT+01:00 H. Hirzel <[hidden email]>:
>
>> It probably should be noted here as well that
>>
>> http://pharo.gemtalksystems.com/book/PharoTools/SIXX/
>>
>
> Just to know does java implementation exists?
>

Hannes Hirzel

Re: Fuel materialization on spur

If you want to move data to Java then you probably go for JSON or a
particuar XML format.

On 12/16/15, H. Hirzel <[hidden email]> wrote:

> No, it is a Smalltalk format.
>
> On 12/16/15, Denis Kudriashov <[hidden email]> wrote:
>> 2015-12-16 14:41 GMT+01:00 H. Hirzel <[hidden email]>:
>>
>>> It probably should be noted here as well that
>>>
>>> http://pharo.gemtalksystems.com/book/PharoTools/SIXX/
>>>
>>
>> Just to know does java implementation exists?
>>
>

Denis Kudriashov

Re: Fuel materialization on spur

2015-12-16 14:58 GMT+01:00 H. Hirzel <[hidden email]>:

If you want to move data to Java then you probably go for JSON or a
particuar XML format.

Of course. Or any other format you can find :)

SIXX is XML. And maybe somebody implement it in different languages. I just interesting about it.