When will Fuel file format stabilize?
Locked
123
123
When will Fuel file format stabilize?
 
|
Hi guys,
We are starting to use Fuel to exchange data daily, but without exactly the same version of Fuel loaded, exported data is just too frequently not importable anymore. And exception report is meaningless, you can only guess that Fuel version is to blame. So, it is more that a time to: - stabilize and freeze the file format - put a version info into a file and check it while importing - meaningful error reporting in such cases This is IMHO more than necessary for Fuel to become a production ready serializer and I'd say Fuel is now "old enough" to become such :) Best regards Janko -- Janko Mivšek Svetovalec za informatiko Eranova d.o.o. Ljubljana, Slovenija www.eranova.si tel: 01 514 22 55 faks: 01 514 22 56 gsm: 031 674 565 |
Re: When will Fuel file format stabilize?
|
|
On Oct 8, 2011, at 5:56 PM, Janko Mivšek wrote: > Hi guys, > > We are starting to use Fuel to exchange data daily, but without exactly > the same version of Fuel loaded, exported data is just too frequently > not importable anymore. And exception report is meaningless, you can > only guess that Fuel version is to blame. > > So, it is more that a time to: > > - stabilize and freeze the file format > - put a version info into a file and check it while importing there is one already. > - meaningful error reporting in such cases > > This is IMHO more than necessary for Fuel to become a production ready > serializer and I'd say Fuel is now "old enough" to become such :) Yes. Now what I would love is that even if fuel changes that the evolution of information is taken into account because like that it will be exercised for real. > > Best regards > Janko > > > > -- > Janko Mivšek > Svetovalec za informatiko > Eranova d.o.o. > Ljubljana, Slovenija > www.eranova.si > tel: 01 514 22 55 > faks: 01 514 22 56 > gsm: 031 674 565 > |
Re: When will Fuel file format stabilize?
|
|
In reply to this post by Janko Mivšek
2011/10/8 Janko Mivšek <[hidden email]> Hi guys, Yes, that's true. We are changing its format all the time. And exception report is meaningless, you can Yes, thanks for reporting this. In fact, we have already fix it yesterday ;) Now, if you try to materialize with a different version, you will have a FLBadVersion exception. Check the tests FLVersionTest and FLSignatureTest for more details. So, it is more that a time to: We cannot. Let me just give you my thoughts: - It is a pity that we cannot make progress just because we break the format. If improving means breaking the format, we will do it. - The main question is WHY you need to update Fuel all the time? which are the reasons behind that? I have only 3 things in mind: a) because Fuel is too slow for you and hence you need a newer/faster version b) because you have changed Pharo version and Fuel doesn't work anymore there with the same version? c) because you found a bug. a) looks unlikely. I think Fuel was fast enough for most scenarios from the beginning even if new versions are faster. b) is unlikely also. For example, the last version (1.6) works out of the box in Pharo 1.2, 1.3 and 1.4. So it means that even between 3 versions of Pharo you don't need to update Fuel. c) we haven't found so far a critical bug which would requires to update Fuel. So...my question is, WHY you need to always use a new version of Fuel and why you cannot stay with a stable/fixed Metacello version? That's why we have metecello configurations. You can stick with version 1.6 (or whatever version) and just migrate when it is really necessary. For example, Francois Stephany is using Fuel for his website, and he is still using Fuel 1.4. He didn't need to update yet, because there aren't bugs and it is fast enough for what he needs. - put a version info into a file and check it while importing Done. Thanks for the suggestion. - meaningful error reporting in such cases Done also :) This is IMHO more than necessary for Fuel to become a production ready I totally agree. serializer and I'd say Fuel is now "old enough" to become such :) heheheh yes. It is already more than year year old hhaa. BTW...yesterday we have finished with Martin the version 1.7 which we will release probably monday or tuesday. Cheers Best regards -- Mariano http://marianopeck.wordpress.com |
Re: When will Fuel file format stabilize?
|
|
In reply to this post by Stéphane Ducasse
s
No, that's impossible, and if posible, it is not worth it. Migrating from an old format to a new one is extremelly complicated and innecessary. The easiest way to solve this is to take the correct version of Fuel, materialize the graph from the stream, load new version of Fuel, and seriaize it again. That the easiest, more secure, and more practical approach I can see. Cheers
-- Mariano http://marianopeck.wordpress.com |
Re: When will Fuel file format stabilize?
|
|
Mariano
> No, that's impossible, and if posible, it is not worth it. Migrating from an old format to a new one is extremelly complicated and innecessary. The easiest way to solve this is to take the correct version of Fuel, materialize the graph from the stream, load new version of Fuel, and seriaize it again. That the easiest, more secure, and more practical approach I can see. It would be interesting to see if you can get two different versions in memory :) (a need for a module system) so that you can load with one and save with the other. Stef |
Re: When will Fuel file format stabilize?
|
|
On Sun, Oct 9, 2011 at 12:03 AM, Stéphane Ducasse <[hidden email]> wrote: Mariano heheh that would be nice :) (a need for a module system) yep, so far it is not possible. so that you can load with one and save with the other. -- Mariano http://marianopeck.wordpress.com |
Re: When will Fuel file format stabilize?
|
|
I wonder if it would be possible for you to come up with a separate
format, whose purpose was only ever for this purpose. It would not need to be efficient in any respect, just something you could write out to file (say) during the process. It would be a shame to rely on the whole graph needing to be in memory. Maybe something very crude and text based? It could even be considered a debugging format, if it was somewhat human readable. If so, you could easily send this over a socket from an old version of a running system to a newer one, without relying on either serialising format. It could be quite handy. cheers, Mike On Sun, Oct 9, 2011 at 10:27 AM, Mariano Martinez Peck <[hidden email]> wrote: > > > On Sun, Oct 9, 2011 at 12:03 AM, Stéphane Ducasse > <[hidden email]> wrote: >> >> Mariano >> >> >> > No, that's impossible, and if posible, it is not worth it. Migrating >> > from an old format to a new one is extremelly complicated and innecessary. >> > The easiest way to solve this is to take the correct version of Fuel, >> > materialize the graph from the stream, load new version of Fuel, and >> > seriaize it again. That the easiest, more secure, and more practical >> > approach I can see. >> >> It would be interesting to see if you can get two different versions in >> memory :) > > heheh that would be nice :) > >> >> (a need for a module system) > > yep, so far it is not possible. > >> >> so that you can load with one and save with the other. >> >> Stef >> >> > > > > -- > Mariano > http://marianopeck.wordpress.com > > |
Re: When will Fuel file format stabilize?
|
|
That's a good idea. Then every version of Fuel would pass an ever growing set of tests designed to ensure that the "backup" format works as intended.
Hopefully, the format will eventually stabilize. I might be alone in this, but I see serializers as a very simple object oriented database, aka a "free" file format. Yes, one could do much the same thing with an OODMS, but there were reasons to avoid that path, especially at the time we got started. One needs to cope with schema changes (instance variable layout) in the application classes, but that can be done (Object Arts strikes again). *IF* Fuel is to serve a market that archives data, the format needs to be readable well into the future. Bill ________________________________________ From: [hidden email] [[hidden email]] On Behalf Of Michael Roberts [[hidden email]] Sent: Sunday, October 09, 2011 10:30 AM To: [hidden email] Subject: Re: [Pharo-project] When will Fuel file format stabilize? I wonder if it would be possible for you to come up with a separate format, whose purpose was only ever for this purpose. It would not need to be efficient in any respect, just something you could write out to file (say) during the process. It would be a shame to rely on the whole graph needing to be in memory. Maybe something very crude and text based? It could even be considered a debugging format, if it was somewhat human readable. If so, you could easily send this over a socket from an old version of a running system to a newer one, without relying on either serialising format. It could be quite handy. cheers, Mike On Sun, Oct 9, 2011 at 10:27 AM, Mariano Martinez Peck <[hidden email]> wrote: > > > On Sun, Oct 9, 2011 at 12:03 AM, Stéphane Ducasse > <[hidden email]> wrote: >> >> Mariano >> >> >> > No, that's impossible, and if posible, it is not worth it. Migrating >> > from an old format to a new one is extremelly complicated and innecessary. >> > The easiest way to solve this is to take the correct version of Fuel, >> > materialize the graph from the stream, load new version of Fuel, and >> > seriaize it again. That the easiest, more secure, and more practical >> > approach I can see. >> >> It would be interesting to see if you can get two different versions in >> memory :) > > heheh that would be nice :) > >> >> (a need for a module system) > > yep, so far it is not possible. > >> >> so that you can load with one and save with the other. >> >> Stef >> >> > > > > -- > Mariano > http://marianopeck.wordpress.com > > |
Re: When will Fuel file format stabilize?
|
|
In reply to this post by Stéphane Ducasse
On 08/10/11 6:03 PM, Stéphane Ducasse wrote:
> Mariano > > >> No, that's impossible, and if posible, it is not worth it. >> Migrating from an old format to a new one is extremelly complicated >> and innecessary. The easiest way to solve this is to take the >> correct version of Fuel, materialize the graph from the stream, >> load new version of Fuel, and seriaize it again. That the easiest, >> more secure, and more practical approach I can see. > > It would be interesting to see if you can get two different versions > in memory :) (a need for a module system) so that you can load with > one and save with the other. For the case where a lot of serialized files need to be converted, having two versions in memory is almost a necessity; otherwise, the Fuel code update would have to be applied for each file being converted. A separate issue is that at some point I wanted to serialize something differently, in two different contexts. In one case, serializing a Pier kernel, I want Fuel to just do its thing. In the other case, when exporting a portion a Pier kernel, I want to fine tune (i.e. map to different objects) the objects being exported/imported. There is a hook to allow the object swap, but the swap would then take place for the other case of serializing to whole kernel (where the desired behaviour was for no such mapping). -- Yanni |
|
|
Maybe the way to go is to provide a metadescription of the changes done to the format since last version. That way, you can use it to convert your serialized objects from one version to another (even back and forward). That's what I think databases do, if you update your version, you have to update the database. It seems like a less effort approach.
Cheers! Pocho.
On Mon, Oct 10, 2011 at 1:50 PM, Yanni Chiu <[hidden email]> wrote:
Lic. Javier Pimás Ciudad de Buenos Aires |
Re: When will Fuel file format stabilize?
|
|
Thanks guys for all the answers and ideas. I really appreaciate all of them. And in fact, this thread would probably be a source of information for Fuel's future work.
But so far, nobody (janko?) yet answer my initial question of this thread. So I will copy paste it here and I will be willing to hear the answers: "We cannot. Let me just give you my thoughts: - It is a pity that we cannot make progress just because we break the format. If improving means breaking the format, we will do it. - The main question is WHY you need to update Fuel all the time? which are the reasons behind that? I have only 3 things in mind: a) because Fuel is too slow for you and hence you need a newer/faster version b) because you have changed Pharo version and Fuel doesn't work anymore there with the same version? c) because you found a bug. a) looks unlikely. I think Fuel was fast enough for most scenarios from the beginning even if new versions are faster. b) is unlikely also. For example, the last version (1.6) works out of the box in Pharo 1.2, 1.3 and 1.4. So it means that even between 3 versions of Pharo you don't need to update Fuel. c) we haven't found so far a critical bug which would requires to update Fuel. So...my question is, WHY you need to always use a new version of Fuel and why you cannot stay with a stable/fixed Metacello version? That's why we have metecello configurations. You can stick with version 1.6 (or whatever version) and just migrate when it is really necessary. For example, Francois Stephany is using Fuel for his website, and he is still using Fuel 1.4. He didn't need to update yet, because there aren't bugs and it is fast enough for what he needs. " On Mon, Oct 10, 2011 at 7:24 PM, Javier Pimás <[hidden email]> wrote: Maybe the way to go is to provide a metadescription of the changes done to the format since last version. That way, you can use it to convert your serialized objects from one version to another (even back and forward). That's what I think databases do, if you update your version, you have to update the database. It seems like a less effort approach. -- Mariano http://marianopeck.wordpress.com |
Re: When will Fuel file format stabilize?
|
|
In reply to this post by Yanni Chiu
On Mon, Oct 10, 2011 at 6:50 PM, Yanni Chiu <[hidden email]> wrote:
heheheheh how people could live with ReferenceStream for 10 years? otherwise, the Fuel code update would have to be applied for each file being converted. but answer my question: why would you need to update Fuel so frequently ? A separate issue is that at some point I wanted to serialize something differently, in two different contexts. In one case, serializing a Pier kernel, I want Fuel to just do its thing. In the other case, when exporting a portion a Pier kernel, I want to fine tune (i.e. map to different objects) the objects being exported/imported. There is a hook to allow the object swap, but the swap would then take place for the other case of serializing to whole kernel (where the desired behaviour was for no such mapping). That's an interesting point....Let me see if I understood. Let's say that you want to exclude certain instance variables of a class. What you mean is that you want to apply this hook (or another one) in an specific serialization and not always? because right now with Fuel (and SIXX, and I guess StOMP also) since you put the hooks in methods of the class, they are always taken into account. How would the API for the user should be? you can you specify for an specific serialization run that certain hook of certain object/class must be applied and some should not ? Apart from that, do you have a real example where you need such behavior ? Beause it looks quite complex/rare scenario ;) Thanks guys for the nice thread.
-- Mariano http://marianopeck.wordpress.com |
Re: When will Fuel file format stabilize?
|
|
I concur with Mariano.
In production images, you want probably stick with concrete verision(s) of frameworks you using and don't follow the bleeding edge. Because if you do, then you spending time on coping up with latest and greatest versions of these frameworks instead of spending time on developing your project :) -- Best regards, Igor Stasenko. |
Re: When will Fuel file format stabilize?
|
|
True, but what about reading data from five years ago? I have ten year old Dolphin STB that I can still load.
________________________________________ From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]] Sent: Monday, October 10, 2011 4:59 PM To: [hidden email] Subject: Re: [Pharo-project] When will Fuel file format stabilize? I concur with Mariano. In production images, you want probably stick with concrete verision(s) of frameworks you using and don't follow the bleeding edge. Because if you do, then you spending time on coping up with latest and greatest versions of these frameworks instead of spending time on developing your project :) -- Best regards, Igor Stasenko. |
Re: When will Fuel file format stabilize?
|
|
On 11 October 2011 00:19, Schwab,Wilhelm K <[hidden email]> wrote:
> True, but what about reading data from five years ago? I have ten year old Dolphin STB that I can still load. > > to my experience a 10 years old data has very small chances to be relevant :) > > ________________________________________ > From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]] > Sent: Monday, October 10, 2011 4:59 PM > To: [hidden email] > Subject: Re: [Pharo-project] When will Fuel file format stabilize? > > I concur with Mariano. > In production images, you want probably stick with concrete > verision(s) of frameworks you using and don't > follow the bleeding edge. Because if you do, then you spending time on > coping up with latest and greatest > versions of these frameworks instead of spending time on developing > your project :) > > -- > Best regards, > Igor Stasenko. > > > -- Best regards, Igor Stasenko. |
Re: When will Fuel file format stabilize?
|
|
In reply to this post by Schwab,Wilhelm K
On Mon, Oct 10, 2011 at 11:19 PM, Schwab,Wilhelm K <[hidden email]> wrote: True, but what about reading data from five years ago? Well, can you load your 5-year-old code out of the box? No. You will probably need to update its code. The same here. You can always take the same Fuel version and updated (without changing the format) to work in a new Pharo image. Just as you do with your own code. As an example, the last stable version 1.6 of Fuel works if Pharo 1.2, 1.3 and 1.4. I think it is good enough. Cheers I have ten year old Dolphin STB that I can still load. -- Mariano http://marianopeck.wordpress.com |
Re: When will Fuel file format stabilize?
|
|
On Mon, Oct 10, 2011 at 11:25 PM, Mariano Martinez Peck <[hidden email]> wrote:
Ahhh and if you are so worried about that, then use a proper database strategy than a simple/plain object serializer.
-- Mariano http://marianopeck.wordpress.com |
Re: When will Fuel file format stabilize?
|
|
In reply to this post by Igor Stasenko
2011/10/10 Igor Stasenko <[hidden email]>:
> On 11 October 2011 00:19, Schwab,Wilhelm K <[hidden email]> wrote: >> True, but what about reading data from five years ago? I have ten year old Dolphin STB that I can still load. >> >> > to my experience a 10 years old data has very small chances to be relevant :) > The problem is that if it ever is relevant, you have very small chance to know because you cannot load ;) Nicolas >> >> ________________________________________ >> From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]] >> Sent: Monday, October 10, 2011 4:59 PM >> To: [hidden email] >> Subject: Re: [Pharo-project] When will Fuel file format stabilize? >> >> I concur with Mariano. >> In production images, you want probably stick with concrete >> verision(s) of frameworks you using and don't >> follow the bleeding edge. Because if you do, then you spending time on >> coping up with latest and greatest >> versions of these frameworks instead of spending time on developing >> your project :) >> >> -- >> Best regards, >> Igor Stasenko. >> >> >> > > > > -- > Best regards, > Igor Stasenko. > > |
Re: When will Fuel file format stabilize?
|
|
In reply to this post by Igor Stasenko
On 10 Oct 2011, at 23:25, Igor Stasenko wrote: > On 11 October 2011 00:19, Schwab,Wilhelm K <[hidden email]> wrote: >> True, but what about reading data from five years ago? I have ten year old Dolphin STB that I can still load. >> >> > to my experience a 10 years old data has very small chances to be relevant :) You might want to reconsider that: Some countries have laws, that regulate how long and what has to be archived: Switzerland: http://www.itandlaw.ch/html/publikationen/pdf/KMUMagazin04.pdf Germany: http://www.recht-im-internet.de/themen/archivierung.htm 10 years is considered the minimum. But that might not be what Pharo or Fuel is supposed to be used for. -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525 |
Re: When will Fuel file format stabilize?
|
|
In reply to this post by Nicolas Cellier
The data in question are medical records - they *really* need to load when demanded.
________________________________________ From: [hidden email] [[hidden email]] On Behalf Of Nicolas Cellier [[hidden email]] Sent: Monday, October 10, 2011 5:30 PM To: [hidden email] Subject: Re: [Pharo-project] When will Fuel file format stabilize? 2011/10/10 Igor Stasenko <[hidden email]>: > On 11 October 2011 00:19, Schwab,Wilhelm K <[hidden email]> wrote: >> True, but what about reading data from five years ago? I have ten year old Dolphin STB that I can still load. >> >> > to my experience a 10 years old data has very small chances to be relevant :) > The problem is that if it ever is relevant, you have very small chance to know because you cannot load ;) Nicolas >> >> ________________________________________ >> From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]] >> Sent: Monday, October 10, 2011 4:59 PM >> To: [hidden email] >> Subject: Re: [Pharo-project] When will Fuel file format stabilize? >> >> I concur with Mariano. >> In production images, you want probably stick with concrete >> verision(s) of frameworks you using and don't >> follow the bleeding edge. Because if you do, then you spending time on >> coping up with latest and greatest >> versions of these frameworks instead of spending time on developing >> your project :) >> >> -- >> Best regards, >> Igor Stasenko. >> >> >> > > > > -- > Best regards, > Igor Stasenko. > > |
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |