Fuel serialization - feature/bug
Locked
 
|
In my test case for serializing a Pier kernel with Fuel, a FSMemoryStore
(from the FileSystem package) is used. The FSMemoryStore holds the "file" bytes in a ByteArray. During serialization, I got a stack trace with KeyNotFound due to FLSerializer>>nextPutReferenceTo: when the object is the "file" ByteArray. Since this ByteArray is where the serialization is being written to, then it's hash value is changing, so Fuel serialization cannot find it in the instancesIndex. I think this is expected behaviour, and the fix is to not serialize certain inst vars. I was using the "stable" 1.4 version. Do I need to change to the 1.5 baseline to get the #fuelIgnoredInstanceVariableNames feature? -- Yanni |
Re: Fuel serialization - feature/bug
|
|
On Mon, Jun 13, 2011 at 11:28 PM, Yanni Chiu <[hidden email]> wrote: In my test case for serializing a Pier kernel with Fuel, a FSMemoryStore (from the FileSystem package) is used. I really didn't understand, sorry :( Why the following is not enought? (FLSerializer on: myStream) serialize: myPRKernel. and (FLMaterializer on: myStream) materialize The FSMemoryStore holds the "file" bytes in a ByteArray. During serialization, I got a stack trace with KeyNotFound due to FLSerializer>>nextPutReferenceTo: when the object is the "file" ByteArray. mmmmmm so you are using a stream to serialize and such stream is also being refenced from the object to serialize? then it's hash value is changing, so Fuel serialization cannot find it in the instancesIndex. -- Mariano http://marianopeck.wordpress.com |
Re: Fuel serialization - feature/bug
|
|
On 13/06/11 5:49 PM, Mariano Martinez Peck wrote:
> > Why the following is not enought? > > (FLSerializer on: myStream) serialize: myPRKernel. > > and > > (FLMaterializer on: myStream) materialize Yes, that's pretty much the code. Except that it's in a test case that uses FSMemoryStore. The tests run a lot faster when writing to memory, instead of to disk. > The FSMemoryStore holds the "file" bytes in a ByteArray. During > serialization, I got a stack trace with KeyNotFound due to > FLSerializer>>nextPutReferenceTo: when the object is the "file" > ByteArray. > > Since this ByteArray is where the serialization is being written to, > > > mmmmmm so you are using a stream to serialize and such stream is also > being refenced from the object to serialize? Yes, that's it - except that the stream is an FSWriteStream on a FSMemoryStore "file", which I think is an issue here. The PRKernel is holding an instance of PRFuelPersistency, which holds an FSReference. That FSReference happens to refer to a memory-based "file". |
Re: Fuel serialization - feature/bug
|
|
On 13/06/11 6:56 PM, Yanni Chiu wrote:
> On 13/06/11 5:49 PM, Mariano Martinez Peck wrote: >> >> mmmmmm so you are using a stream to serialize and such stream is also >> being refenced from the object to serialize? > > Yes, that's it - except that the stream is an FSWriteStream on a > FSMemoryStore "file", which I think is an issue here. The PRKernel is > holding an instance of PRFuelPersistency, which holds an FSReference. > That FSReference happens to refer to a memory-based "file". The code is at http://ss3.gemstone.com/ss/pierfuel.html You'll first need a Pier image, with the following loaded: (ConfigurationOfFilesystem project version: '2.0.1-baseline') load (ConfigurationOfFuel project version: '1.4') load OR (ConfigurationOfFuel project version: '1.5-baseline') load If you run PRFuelPersistencyTest, then you'll see the stack trace. I saw the same problem with 1.4 and with 1.5-baseline. In the 1.5-baseline, I added #fuelIgnoredInstanceVariableNames to PRFuelPersistency and FSMemoryStore (to ignore 'directory' and 'root', respectively) but the same problem remained. So I think it's a bug. Despite the test failing, the PRFuelPersistency works as expected to save/restore a Pier kernel to/from a file. The file is 30 times smaller than the corresponding (uncompressed) SIXX file. |
Re: Fuel serialization - feature/bug
|
|
In reply to this post by Yanni Chiu
On Tue, Jun 14, 2011 at 12:56 AM, Yanni Chiu <[hidden email]> wrote:
Yes, but if you want to serialize in memory you can also use for example: myStream := (MultiByteBinaryOrTextStream on: ''). (FLSerializer on: myStream) serialize: myPRKernel. ... myStream reset. (FLMaterializer on: myStream) materialize That would use a stream in memory. And even more, you can serialize in memory with Fuel this way: testSerializationAndMaterializationInMemory | byteArray materializedObject | byteArray := FLSerializer serializeInMemory: 'mariano'. materializedObject := FLMaterializer materializeFromByteArray: byteArray. self assert: materializedObject equals: 'mariano'.
-- Mariano http://marianopeck.wordpress.com |
Re: Fuel serialization - feature/bug
|
|
In reply to this post by Yanni Chiu
On Tue, Jun 14, 2011 at 8:12 AM, Yanni Chiu <[hidden email]> wrote:
Yes, We think you have spot a bug :) Martin is working on that. We will let you know. Thanks Despite the test failing, the PRFuelPersistency works as expected to save/restore a Pier kernel to/from a file. The file is 30 times smaller than the corresponding (uncompressed) SIXX file. and speed? could you measure serialization / materialization time ? Thanks a lot, -- Mariano http://marianopeck.wordpress.com |
Re: Fuel serialization - feature/bug
|
|
On Tue, Jun 14, 2011 at 9:37 AM, Mariano Martinez Peck <[hidden email]> wrote:
BTW you put them as class side methods, don't you?
-- Mariano http://marianopeck.wordpress.com |
Re: Fuel serialization - feature/bug
|
|
In reply to this post by Yanni Chiu
Hi Yanni
On Tue, Jun 14, 2011 at 3:12 AM, Yanni Chiu <[hidden email]> wrote:
I didn't try in the Pier image, but I hope your problem is fixed with the last version (Fuel-MartinDias.259). serialize the stream where you are serializing is a test case I had never thought! BTW, is it fine to get nil on these variables? I mean, is the FSMemoryStore materialized useful?
good! so it can be useful for scenarios where you don't need to be cross-dialect or to have XML format. |
Re: Fuel serialization - feature/bug
|
|
So..using (Fuel-MartinDias.259) I added
FSMemoryStore class >> fuelIgnoredInstanceVariableNames ^#(root) PRFuelPersistency class >> fuelIgnoredInstanceVariableNames ^#(directory) The tests are all green. Cheers On Tue, Jun 14, 2011 at 10:15 AM, Martin Dias <[hidden email]> wrote: Hi Yanni -- Mariano http://marianopeck.wordpress.com |
Re: Fuel serialization - feature/bug
|
|
In reply to this post by Mariano Martinez Peck
On 14/06/11 3:37 AM, Mariano Martinez Peck wrote:
> > and speed? could you measure serialization / materialization time ? The kernel was just a few pages, and SIXX took about 5 sec to save to disk, and a slight pause to materialize. With Fuel, it was all done almost instantaneously. I don't have any benchmarks ... now added to TODO list. |
Re: Fuel serialization - feature/bug
|
|
In reply to this post by tinchodias
On 14/06/11 4:15 AM, Martin Dias wrote:
> > BTW, is it fine to get nil on these variables? I mean, is the > FSMemoryStore materialized useful? Yes, that's exactly the result expected when defining #fuelIgnoredInstanceVariableNames to return #('root') for FSMemoryStore. |
Re: Fuel serialization - feature/bug
|
|
In reply to this post by Mariano Martinez Peck
On 14/06/11 3:35 AM, Mariano Martinez Peck wrote:
> > Yes, but if you want to serialize in memory you can also use for example: > > myStream := (MultiByteBinaryOrTextStream on: ''). > (FLSerializer on: myStream) serialize: myPRKernel. At the time, I wanted to use the FileSystem protocol. It was a struggle to understand how to use FileSystem. For tests of code that don't involve the file system in the public API, I could avoid the "complexity" of FileSystem, and use the streams directly (as in your example). |
Re: Fuel serialization - feature/bug
|
|
In reply to this post by Mariano Martinez Peck
On 14/06/11 4:41 AM, Mariano Martinez Peck wrote:
> > PRFuelPersistency class >> fuelIgnoredInstanceVariableNames > ^#(directory) Using either one, the test now passes when using Fuel-MartinDias.259. Only one of the #fuelIgnoredInstanceVariableNames is needed. I'll go with the above one, so that FSMemoryStore does not have to be extended. |
Re: Fuel serialization - feature/bug
|
|
On Tue, Jun 14, 2011 at 6:09 PM, Yanni Chiu <[hidden email]> wrote: On 14/06/11 4:41 AM, Mariano Martinez Peck wrote: Excellent. I'll go with the above one, so that FSMemoryStore does not have to be extended. +1 -- Mariano http://marianopeck.wordpress.com |
Re: Fuel serialization - feature/bug
|
|
Yanni. The tests are green, but I have an error while trying to persist. If I go to:
http://localhost:8888/pier/system/management/persistency Choose Fuel, and click on "snapshot" I have a key not found error. Do you have the same or is it working for you? Thanks On Wed, Jun 15, 2011 at 12:01 PM, Mariano Martinez Peck <[hidden email]> wrote:
-- Mariano http://marianopeck.wordpress.com |
Re: Fuel serialization - feature/bug
|
|
Sorry, one more question. One you did seems to be a persistency strategy for Pier, but I thought it was the import/export utility:
http://www.lukas-renggli.ch/blog/export-import But it seems I was wrong. There is no way so that I can use that import/export utility with Fuel also? does that require extra effort or it can use what you did ? Thanks! On Thu, Jun 16, 2011 at 10:24 AM, Mariano Martinez Peck <[hidden email]> wrote: Yanni. The tests are green, but I have an error while trying to persist. If I go to: -- Mariano http://marianopeck.wordpress.com |
Re: Fuel serialization - feature/bug
|
|
In reply to this post by Mariano Martinez Peck
On 16/06/11 4:24 AM, Mariano Martinez Peck wrote:
> Yanni. The tests are green, but I have an error while trying to persist. > If I go to: > > http://localhost:8888/pier/system/management/persistency > > Choose Fuel, and click on "snapshot" I have a key not found error. Do > you have the same or is it working for you? That worked, the one time I tried it. I'm still integrating the code, so I don't have a clear answer at the moment. My system is set up to read in the saved kernel on image startup, so it may not be obvious how to use PRFuelPersistency, in its present form. |
Re: Fuel serialization - feature/bug
|
|
In reply to this post by Mariano Martinez Peck
On 16/06/11 4:44 AM, Mariano Martinez Peck wrote:
> Sorry, one more question. One you did seems to be a persistency strategy > for Pier, but I thought it was the import/export utility: > > http://www.lukas-renggli.ch/blog/export-import > > But it seems I was wrong. There is no way so that I can use that > import/export utility with Fuel also? does that require extra effort or > it can use what you did ? What I did, had more to do with implementing a persistency strategy than with serialization. I took a persistency strategy I'd written, that was using SIXX, and changed it to use Fuel. The changes were very minor - just changed the serialize/deserialize methods. I imagine doing the same for the import/export would be just as straightforward. I'm planning to use Fuel for an import/export facility, but I'm trying to export only a portion of a kernel. Since a Pier kernel's objects are highly interconnected then it's a challenge. On the import side of things, one problem is that the transaction log with be voided (i.e. you would be able to reconstruct the kernel from the log - but maybe that's already a problem when you import a whole kernel). |
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |