Smalltalk › Pharo › Pharo Smalltalk Users

Who is using Fuel?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

5 messages Options

Mariano Martinez Peck

Who is using Fuel?

Of course, not the fuel you use to go to work every day ;) I am talking about Fuel serializer.
In particular, I wonder if our page deserves an update: rmod.lille.inria.fr/web/pier/software/Fuel/Software-using-Fuel

Let me know if there is something I should add.

Thanks,

--
Mariano
http://marianopeck.wordpress.com

NorbertHartl

Re: Who is using Fuel?

Am 28.02.2012 um 16:41 schrieb Mariano Martinez Peck:

Of course, not the fuel you use to go to work every day ;) I am talking about Fuel serializer.
In particular, I wonder if our page deserves an update: rmod.lille.inria.fr/web/pier/software/Fuel/Software-using-Fuel

Let me know if there is something I should add.

We are using it in

http://www.2denker.de/product/apitester

I first used image persistence but the image grow to large. Then I added fuel as journal to be written before image save in order to be able to recover from an emergency.

Now we redo the persistence part. An account object graph is dissected into configuration part and payload part. The payload is written directly to mongo db in a Json format. The configuration part will be written as fuel blob to mongo. Planned is that at startup a configured amount of account fuel blobs are loaded and started. Those configurations have their own processes that write back payload to mongo. Etc.

Just in case you want to know :)

Norbert

Mariano Martinez Peck

Re: [Pharo-project] Who is using Fuel?

On Tue, Feb 28, 2012 at 5:03 PM, Norbert Hartl <[hidden email]> wrote:

Am 28.02.2012 um 16:41 schrieb Mariano Martinez Peck:

Of course, not the fuel you use to go to work every day ;) I am talking about Fuel serializer.
In particular, I wonder if our page deserves an update: rmod.lille.inria.fr/web/pier/software/Fuel/Software-using-Fuel

Let me know if there is something I should add.

We are using it in

http://www.2denker.de/product/apitester

I first used image persistence but the image grow to large. Then I added fuel as journal to be written before image save in order to be able to recover from an emergency.

I didn't undersantd this one. What is what you are serializing that you want to be able to recover in case of an emergency?

Now we redo the persistence part. An account object graph is dissected into configuration part and payload part. The payload is written directly to mongo db in a Json format. The configuration part will be written as fuel blob to mongo. Planned is that at startup a configured amount of account fuel blobs are loaded and started. Those configurations have their own processes that write back payload to mongo. Etc.

Just in case you want to know :)

Interested indeed :) I added it under "Custom applications"

Thanks!

--
Mariano
http://marianopeck.wordpress.com

NorbertHartl

Re: [Pharo-project] Who is using Fuel?

Am 28.02.2012 um 17:23 schrieb Mariano Martinez Peck:

On Tue, Feb 28, 2012 at 5:03 PM, Norbert Hartl <[hidden email]> wrote:

Am 28.02.2012 um 16:41 schrieb Mariano Martinez Peck:

Of course, not the fuel you use to go to work every day ;) I am talking about Fuel serializer.
In particular, I wonder if our page deserves an update: rmod.lille.inria.fr/web/pier/software/Fuel/Software-using-Fuel

Let me know if there is something I should add.

We are using it in

http://www.2denker.de/product/apitester

I first used image persistence but the image grow to large. Then I added fuel as journal to be written before image save in order to be able to recover from an emergency.

I didn't undersantd this one. What is what you are serializing that you want to be able to recover in case of an emergency?

All of my data. An account can log in and configure the service. The service collects data. All of these modifications should be persisted. I experience image corruption with big images. Every week or so the vm just didn't start the images anymore because it stated "premature end of image file". So I write down all my objects via fuel to disk before I save the image. In case of an emergency (image corruption) I take a fresh image and load back the saved fuel graph. And within 2 minutes I'm once read to go.

Now the big parts of the data are in mongo so I don't expect it to happen real soon again.

Now we redo the persistence part. An account object graph is dissected into configuration part and payload part. The payload is written directly to mongo db in a Json format. The configuration part will be written as fuel blob to mongo. Planned is that at startup a configured amount of account fuel blobs are loaded and started. Those configurations have their own processes that write back payload to mongo. Etc.

Just in case you want to know :)

Interested indeed :) I added it under "Custom applications"

thanks,

Norbert

NorbertHartl

Re: [Pharo-project] Who is using Fuel?

In reply to this post by NorbertHartl

Am 28.02.2012 um 19:15 schrieb Yanni Chiu:

> On 28/02/12 11:03 AM, Norbert Hartl wrote:
>>
>> I first used image persistence but the image grow to large. Then I added
>> fuel as journal to be written before image save in order to be able to
>> recover from an emergency.
>> Now we redo the persistence part. An account object graph is dissected
>> into configuration part and payload part. The payload is written
>> directly to mongo db in a Json format. The configuration part will be
>> written as fuel blob to mongo. Planned is that at startup a configured
>> amount of account fuel blobs are loaded and started. Those
>> configurations have their own proces ses that write back payload to
>> mongo. Etc.
>
> That's similar to what I do, except no image save and no mongo db.
>

As soon as the fuel blobs are in mongo (or on disk) no image save is needed then and I will start fresh everytime which is more reliable. Mongo is quite nice because I have a few hundred megabytes of data. And via cron it runs a map/reduce task to create the statistics about the data that has been fetched. The statistical data is displayed in charts on the web site again.

Norbert

> Here's how it goes:
> - on restart the image reconstructs a Pier kernel saved via Fuel to a file
> - included in the Pier kernel is what you're calling the "configuration part" - there are custom (non-std Pier) components here.
> - the components in the reconstructed image can read/write a "payload" part, which is serialized to files using Fuel and SandstoneDb.
> - additionally, a "sub-tree" of the Pier kernel can be exported to a file using Fuel serialization, and later imported using Fuel deserialization.
>
> No URL available at the moment.
>
> Aside - before Fuel/SandstoneDb, I was using Glorp/PostgreSQL. I've gained a lot of simplicity, but lost sequence numbers and object caching (and maybe more things, that have not been a problem so far). I've written a seqno replacement, but the lack of object caching is not a problem at the moment (i.e. small data sets, so far).
>
>