DeltaStreams small update
Locked
12
12
 
|
Hi all!
Ok, managed to get some more code written tonight - but I am still on a low level here - matching up with ChangeSet etc. But if anyone wants to join in: See details here: http://wiki.squeak.org/squeak/6001 The first goal is to make DSDelta work as a "true log" capturing all changes in full detail. One question for you gurus out there - do we care about changing the ORDER of say instvars? regards, Göran |
|
|
Hi Göran-- > ...do we care about changing the ORDER of say instvars? I think we should, if we are to support direct installation of compiled methods (without recompiling from source code), since methods use instructions that refer to the receiver's instance variables explicitly by field index. -C -- Craig Latta improvisational musical informaticist www.netjam.org Smalltalkers do: [:it | All with: Class, (And love: it)] |
|
|
In reply to this post by Göran Krampe
Hi,
<[hidden email]> wrote in message news:[hidden email]... > Hi all! > > Ok, managed to get some more code written tonight - but I am still on a > low level here - matching up with ChangeSet etc. But if anyone wants to > join in: > > See details here: http://wiki.squeak.org/squeak/6001 > > The first goal is to make DSDelta work as a "true log" capturing all > changes in full detail. > > One question for you gurus out there - do we care about changing the > ORDER of say instvars? yes,the order can be important. A concrete example... FT2Plugin>>primitiveGetFaceCharMap makes an assumption about the position (and therefore the order) of an FT2Face's instVars... "platformId 23" interpreterProxy storeInteger: 23 ofObject: rcvr withValue: (self cCode: 'charmap->platform_id'). "encodingId 24" interpreterProxy storeInteger: 24 ofObject: rcvr withValue: (self cCode: 'charmap->encoding_id'). If the order of the platformId and encodingId instVars were somehow reversed, then the plugin would write to the wrong slots. Cheers, Andy > > regards, Göran > > |
|
|
Hi!
> Hi, > <[hidden email]> wrote in message > news:[hidden email]... >> Hi all! >> >> Ok, managed to get some more code written tonight - but I am still on a >> low level here - matching up with ChangeSet etc. But if anyone wants to >> join in: >> >> See details here: http://wiki.squeak.org/squeak/6001 >> >> The first goal is to make DSDelta work as a "true log" capturing all >> changes in full detail. >> >> One question for you gurus out there - do we care about changing the >> ORDER of say instvars? > > yes,the order can be important. > A concrete example... Ok, someone said something similar - perhaps it was you (on IRC) :) Sigh, well, then I will just have to add one more Change class then that captures any change in relative order between the ivars that are not added nor removed. So "a b c" -> "a c" would only generate a removeIvar Change and not a orderIvar Change since c still follows after a. And "a b c d" -> "c a e b f" would generate 3 changes: 1 addIvarsChange adding e and f. 1 orderIvarsChange reordering "a b c" to "c a b" (relatively) 1 removeIvarsChange removing d. Sounds fair? regards, Göran regards, Göran |
|
|
In reply to this post by Göran Krampe
[hidden email] wrote:
> Hi all! > > Ok, managed to get some more code written tonight - but I am still on a > low level here - matching up with ChangeSet etc. But if anyone wants to > join in: > > See details here: http://wiki.squeak.org/squeak/6001 > > The first goal is to make DSDelta work as a "true log" capturing all > changes in full detail. > > One question for you gurus out there - do we care about changing the > ORDER of say instvars? > > regards, Göran > uses DataStream's for any form of serialization. best regards Keith |
|
|
In reply to this post by Göran Krampe
On 21/08/07, Göran Krampe <[hidden email]> wrote:
> Hi! > > > Hi, > > <[hidden email]> wrote in message > > news:[hidden email]... > >> Hi all! > >> > >> Ok, managed to get some more code written tonight - but I am still on a > >> low level here - matching up with ChangeSet etc. But if anyone wants to > >> join in: > >> > >> See details here: http://wiki.squeak.org/squeak/6001 > >> > >> The first goal is to make DSDelta work as a "true log" capturing all > >> changes in full detail. > >> > >> One question for you gurus out there - do we care about changing the > >> ORDER of say instvars? > > > > yes,the order can be important. > > A concrete example... > [SNIP] > > Ok, someone said something similar - perhaps it was you (on IRC) :) > Sigh, well, then I will just have to add one more Change class then that > captures any change in relative order between the ivars that are not added > nor removed. > > So "a b c" -> "a c" would only generate a removeIvar Change and not a > orderIvar Change since c still follows after a. > > And "a b c d" -> "c a e b f" would generate 3 changes: > > 1 addIvarsChange adding e and f. > 1 orderIvarsChange reordering "a b c" to "c a b" (relatively) > 1 removeIvarsChange removing d. > IFAIK compiler recompiles class (and all its subclasses) every time you changing ivars. So, there is no difference in undergoing actions between adding/removing/reordering ivars. What the purpose in differentiating kinds of ivar operations? > Sounds fair? > > regards, Göran > > > regards, Göran > > > -- Best regards, Igor Stasenko AKA sig. |
|
|
In reply to this post by ccrraaiigg
Ya, in fact when traits (If I recall correctly) were first introduced
I started getting complains about the JPEG primitive decoder failing. Turns out it uses the "known" order of instance variables to read data into C variables so it can do calculations. Earlier versions of Traits, changed the order, that would break the math and decisions in the decoder primitive, so it would fail. Of course everything in the smalltalk walkback looked fine... On Aug 20, 2007, at 10:29 PM, Craig Latta wrote: > > Hi Göran-- > > > ...do we care about changing the ORDER of say instvars? > > I think we should, if we are to support direct installation of > compiled methods (without recompiling from source code), since > methods use instructions that refer to the receiver's instance > variables explicitly by field index. > > > -C > > -- > Craig Latta > improvisational musical informaticist > www.netjam.org > Smalltalkers do: [:it | All with: Class, (And love: it)] > > -- ======================================================================== === John M. McIntosh <[hidden email]> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== === |
Re: DeltaStreams small update
|
|
In reply to this post by Göran Krampe
Hi Göran,
I refer to your proposed Delta File Format, with your remarks Human readable? encoding , (utf8? BOM or no BOM?) (put encoding in header?) and the Method Source preamble. Can I suggest to consider a small XML-parseable format, which is so flat that it is easy to parse by just the two methods #upTo: and #scanTokens: (perhaps assisted by #replaceAll: $" with: $' and #utf8ToIso). This way Deltas could perfectly be used in other markedup docs, for example with xhtml. Also transport (http+friends) and transfer encoding can be handled by existing (hope so :) protocols. It would look like <?xml version="1.0" encoding="UTF-8"?><... st:xmlns="http://www.squeak.org/2007/deltaFileFormat"> <st:class name="SuperDeltaObject" superclass="Object" dialect="Squeak"> <st:classcomment /> <!--uhm, what class comment ;-) --> <st:method selector="with:do:" category="--all--" timestamp="8/23/2004 14:41" author="st"> with: this do: that ^ this perform: that </st:method> <st:metaclass instvars=""> <st:method selector="initialize" category="class initialization" timestamp="8/23/2004 14:42" author="st"> initialize </st:method> </st:metaclass> </st:class></...> All the not explicitly mentioned attributes can have an intuitive default (like format="fixed" and instvars="", etc) and complex attributes (like oldinstvarorder="c b a") can be ignored by simple readers. It looks a bit wordy but, what do you think? /Klaus On Tue, 21 Aug 2007 03:01:45 +0200, <[hidden email]> wrote: > Hi all! > > Ok, managed to get some more code written tonight - but I am still on a > low level here - matching up with ChangeSet etc. But if anyone wants to > join in: > > See details here: http://wiki.squeak.org/squeak/6001 > > The first goal is to make DSDelta work as a "true log" capturing all > changes in full detail. > > One question for you gurus out there - do we care about changing the > ORDER of say instvars? > > regards, Göran > > |
|
|
Hi!
> Hi Göran, > > I refer to your proposed Delta File Format, with your remarks Human > readable? encoding , (utf8? BOM or no BOM?) (put encoding in header?) and > the Method Source preamble. Note that I didn't write the lower part of the Swiki page. :) Not sure who did. But I kept it anyway. [SNIP] > It looks a bit wordy but, what do you think? Well, honestly I don't like XML much. :) But either we pick ONE single format (for simplicity) and then I am leaning towards a simple chunk format very much like previous formats (.st .cs). You know - KISS. ...or we allow multiple formats from the start and then an XML format is of course a useful one to have for some people (I guess). Definitely for cross-Smalltalk use. Since I can't do all work I will focus on the simple chunk-format but will take into account that the system may have multiple "parsers" to use when reading/writing Deltas. Thus we can add more formats as we go. If you are interested in helping with an XML-format, that would be nice! :) Note though that file formats are still way off - my current goal is to make a DSDelta be able to "record" all kinds of Changes in a correct way using SystemNotifier, I am almost there. Next goal is probably to implement #apply and #revert using SystemEditor. After that we might be looking at external read/write. :) regards, Göran |
Re: DeltaStreams small update
|
|
In reply to this post by Klaus D. Witzel
Klaus D. Witzel wrote:
> Hi Göran, > > I refer to your proposed Delta File Format, with your remarks Human > readable? encoding , (utf8? BOM or no BOM?) (put encoding in header?) > and the Method Source preamble. > > Can I suggest to consider a small XML-parseable format, which is so > flat that it is easy to parse by just the two methods #upTo: and > #scanTokens: (perhaps assisted by #replaceAll: $" with: $' and > #utf8ToIso). > > This way Deltas could perfectly be used in other markedup docs, for > example with xhtml. Also transport (http+friends) and transfer > encoding can be handled by existing (hope so :) protocols. It would > look like > brought the product no discernable advantage except that it is aware of packages, properties etc that were never added to the standard bang fileout format (which was an oversight by someone at some point in the past). At one point there was some fevered excitement over the idea of presenting smalltalk xml fileouts using XSLT to make XHTML.. but at the end of the day, the outcome was hardly spectacular - you could generate something far more interesting directly from the image and make it interactive with seaside, for example. As to human readable - well.. let's just say XML leaves a lot to be desired there :) I'd much rather see pure message sends going on here, eg: self encoding: #utf8. self class: 'SuperDeltaObject' superclass: 'Object' do: [ self instanceVariables: #(a b c). self comment: 'uhm, what class comment ;-)'. self method: 'with:do:' category: '--all--' timestamp: '8/23/2004 14:41' author: 'st' body: [ ^this perform: that ]. self classMethod: 'initialize' category: 'class initialization' timestamp: '8/23/2004 14:42' author: 'st' body: [ ]. ] I'm sure there would be plenty of debate on the actual methods used to build things, mode versus scope and things like that.. all worthwhile. But the beauty of it is that it doesn't introduce a non-smalltalk syntax such as the bangs or xml. Just my 2c. I've got a library in VW that does most of this. I'm sure the technique could be ported fairly readily. Michael > <?xml version="1.0" encoding="UTF-8"?><... > st:xmlns="http://www.squeak.org/2007/deltaFileFormat"> > <st:class name="SuperDeltaObject" superclass="Object" dialect="Squeak"> > <st:classcomment /> <!--uhm, what class comment ;-) --> > <st:method selector="with:do:" category="--all--" > timestamp="8/23/2004 14:41" author="st"> > with: this do: that > ^ this perform: that > </st:method> > <st:metaclass instvars=""> > <st:method selector="initialize" category="class initialization" > timestamp="8/23/2004 14:42" author="st"> > initialize > </st:method> > </st:metaclass> > </st:class></...> > > All the not explicitly mentioned attributes can have an intuitive > default (like format="fixed" and instvars="", etc) and complex > attributes (like oldinstvarorder="c b a") can be ignored by simple > readers. > > It looks a bit wordy but, what do you think? > > /Klaus > > On Tue, 21 Aug 2007 03:01:45 +0200, <[hidden email]> wrote: > >> Hi all! >> >> Ok, managed to get some more code written tonight - but I am still on a >> low level here - matching up with ChangeSet etc. But if anyone wants to >> join in: >> >> See details here: http://wiki.squeak.org/squeak/6001 >> >> The first goal is to make DSDelta work as a "true log" capturing all >> changes in full detail. >> >> One question for you gurus out there - do we care about changing the >> ORDER of say instvars? >> >> regards, Göran >> >> > > > |
|
|
In reply to this post by Göran Krampe
"Göran Krampe" <[hidden email]> wrote in message news:[hidden email]... > Hi! > >> Hi Göran, >> >> I refer to your proposed Delta File Format, with your remarks Human >> readable? encoding , (utf8? BOM or no BOM?) (put encoding in header?) and >> the Method Source preamble. > > Note that I didn't write the lower part of the Swiki page. :) Not sure who > did. But I kept it anyway. ok. It was me that wrote that stuff. (I should have put my name on it somewhere). I started adding some use cases and got carried away :) > > [SNIP] >> It looks a bit wordy but, what do you think? > > Well, honestly I don't like XML much. :) > > But either we pick ONE single format (for simplicity) and then I am > leaning towards a simple chunk format very much like previous formats (.st > .cs). You know - KISS. Chunks will work fine. My doodlings on the wiki were an attempt to find a 'chunk like' format where !s don't have to be escaped/doubled. [snip] Cheers, Andy |
Re: DeltaStreams small update
|
|
In reply to this post by Michael Lucas-Smith-3
Thank you Michael for your 2c's :)
The format you suggested is so easy to translate into the XML-format that I had in mind, just by supplying an appropriate self (calls for a traits implementation ;-). Even the text body with its indentations and whitespace is taken care of. Of course the other direction (XML to your 2c's) is a bit more complicated but doesn't look undoable. Thanks again, will keep in mind you have a library for such things :) /Klaus On Thu, 23 Aug 2007 15:30:06 +0200, Michael wrote: > Klaus D. Witzel wrote: >> Hi Göran, >> >> I refer to your proposed Delta File Format, with your remarks Human >> readable? encoding , (utf8? BOM or no BOM?) (put encoding in header?) >> and the Method Source preamble. >> >> Can I suggest to consider a small XML-parseable format, which is so >> flat that it is easy to parse by just the two methods #upTo: and >> #scanTokens: (perhaps assisted by #replaceAll: $" with: $' and >> #utf8ToIso). >> >> This way Deltas could perfectly be used in other markedup docs, for >> example with xhtml. Also transport (http+friends) and transfer encoding >> can be handled by existing (hope so :) protocols. It would look like >> > VisualWorks has an XML file out format that is the default. It has > brought the product no discernable advantage except that it is aware of > packages, properties etc that were never added to the standard bang > fileout format (which was an oversight by someone at some point in the > past). > > At one point there was some fevered excitement over the idea of > presenting smalltalk xml fileouts using XSLT to make XHTML.. but at the > end of the day, the outcome was hardly spectacular - you could generate > something far more interesting directly from the image and make it > interactive with seaside, for example. > > As to human readable - well.. let's just say XML leaves a lot to be > desired there :) I'd much rather see pure message sends going on here, > eg: > > self encoding: #utf8. > self class: 'SuperDeltaObject' superclass: 'Object' do: [ > self instanceVariables: #(a b c). > self comment: 'uhm, what class comment ;-)'. > self method: 'with:do:' category: '--all--' timestamp: '8/23/2004 > 14:41' author: 'st' body: [ > ^this perform: that > ]. > self classMethod: 'initialize' category: 'class initialization' > timestamp: '8/23/2004 14:42' author: 'st' body: [ > ]. > ] > > I'm sure there would be plenty of debate on the actual methods used to > build things, mode versus scope and things like that.. all worthwhile. > But the beauty of it is that it doesn't introduce a non-smalltalk syntax > such as the bangs or xml. > > Just my 2c. I've got a library in VW that does most of this. I'm sure > the technique could be ported fairly readily. > > Michael >> <?xml version="1.0" encoding="UTF-8"?><... >> st:xmlns="http://www.squeak.org/2007/deltaFileFormat"> >> <st:class name="SuperDeltaObject" superclass="Object" dialect="Squeak"> >> <st:classcomment /> <!--uhm, what class comment ;-) --> >> <st:method selector="with:do:" category="--all--" timestamp="8/23/2004 >> 14:41" author="st"> >> with: this do: that >> ^ this perform: that >> </st:method> >> <st:metaclass instvars=""> >> <st:method selector="initialize" category="class initialization" >> timestamp="8/23/2004 14:42" author="st"> >> initialize >> </st:method> >> </st:metaclass> >> </st:class></...> >> >> All the not explicitly mentioned attributes can have an intuitive >> default (like format="fixed" and instvars="", etc) and complex >> attributes (like oldinstvarorder="c b a") can be ignored by simple >> readers. >> >> It looks a bit wordy but, what do you think? >> >> /Klaus >> >> On Tue, 21 Aug 2007 03:01:45 +0200, <[hidden email]> wrote: >> >>> Hi all! >>> >>> Ok, managed to get some more code written tonight - but I am still on a >>> low level here - matching up with ChangeSet etc. But if anyone wants to >>> join in: >>> >>> See details here: http://wiki.squeak.org/squeak/6001 >>> >>> The first goal is to make DSDelta work as a "true log" capturing all >>> changes in full detail. >>> >>> One question for you gurus out there - do we care about changing the >>> ORDER of say instvars? >>> >>> regards, Göran >>> >>> >> >> >> > > > |
|
|
In reply to this post by Göran Krampe
Göran Krampe wrote:
>> It looks a bit wordy but, what do you think? > > Well, honestly I don't like XML much. :) > > But either we pick ONE single format (for simplicity) and then I am > leaning towards a simple chunk format very much like previous formats (.st > .cs). You know - KISS. XML is not necessarily a bad option for this goal. We have an XML parser that works very well, is relatively small and fast and allows for all but the most esoteric stuff. Chunks are okay but the disadvantage with them is that they are context-sensitive and that a parser can therefore not ignore something it doesn't understand which makes extensibility a brittle thing. A good example is that a 3.8 file contents browser will be unable to correctly parse fileOuts created from 3.9 - that's because it expects to find a class definition being made up of a particular number of tokens and that number of tokens has changed in 3.9 (which is impossible to tell without some sort of meta information). However, if that problem had been addressed by another (context-sensitive) chunk it would probably have exploded the FCB altogether ;-) > Since I can't do all work I will focus on the simple chunk-format but will > take into account that the system may have multiple "parsers" to use when > reading/writing Deltas. Thus we can add more formats as we go. If you are > interested in helping with an XML-format, that would be nice! :) I'll check it out. Unfortunately, I'm completely swamped with work right now. Cheers, - Andreas |
|
|
In reply to this post by Klaus D. Witzel
> Of course the other direction (XML to your 2c's) is a bit more > complicated but doesn't look undoable. That's what I'm afraid of, that we'll introduce XML into the mix and won't be able to un-do it. :) -C -- Craig Latta improvisational musical informaticist www.netjam.org Smalltalkers do: [:it | All with: Class, (And love: it)] |
Re: DeltaStreams small update
|
|
Craig Latta wrote:
> > > Of course the other direction (XML to your 2c's) is a bit more > > complicated but doesn't look undoable. > > That's what I'm afraid of, that we'll introduce XML into the mix > and won't be able to un-do it. :) > Let's put it this way - if we use Smalltalk instead of XML, then we can program stuff and extend it easily. If we use XML, then we end up with something like Ant.. when we could have just used Smalltalk. Using Smalltalk fulfills two requirements - a file out format that describes declarative code structures - and a disk format for writing smalltalk programs that doesn't involve weird !'s or xml tags/namespaces. Just another 2c.. almost up to 5c now. Michael |
|
|
Michael Lucas-Smith wrote:
> Let's put it this way - if we use Smalltalk instead of XML, then we can > program stuff and extend it easily. If we use XML, then we end up with > something like Ant.. Ant is an *excellent* tool. We use it for our product builds and it's a great choice - definitely beats reinventing the whole build infrastructure. But then I don't have prejudices against XML either. Cheers, - Andreas |
|
|
In reply to this post by Michael Lucas-Smith-3
I agree, avoiding XML and sticking with Smalltalk is a big plus.
Travis. Michael Lucas-Smith wrote: > Craig Latta wrote: >> >> > Of course the other direction (XML to your 2c's) is a bit more >> > complicated but doesn't look undoable. >> >> That's what I'm afraid of, that we'll introduce XML into the mix >> and won't be able to un-do it. :) >> > Let's put it this way - if we use Smalltalk instead of XML, then we > can program stuff and extend it easily. If we use XML, then we end up > with something like Ant.. when we could have just used Smalltalk. > Using Smalltalk fulfills two requirements - a file out format that > describes declarative code structures - and a disk format for writing > smalltalk programs that doesn't involve weird !'s or xml tags/namespaces. > > Just another 2c.. almost up to 5c now. > > Michael > |
|
|
In reply to this post by Michael Lucas-Smith-3
Hi,
<snip> > As to human readable - well.. let's just say XML leaves a lot to be > desired there :) I'd much rather see pure message sends going on here, eg: > > self encoding: #utf8. > self class: 'SuperDeltaObject' superclass: 'Object' do: [ > self instanceVariables: #(a b c). > self comment: 'uhm, what class comment ;-)'. > self method: 'with:do:' category: '--all--' timestamp: '8/23/2004 > 14:41' author: 'st' body: [ > ^this perform: that > ]. > self classMethod: 'initialize' category: 'class initialization' > timestamp: '8/23/2004 14:42' author: 'st' body: [ > ]. > ] That looks good. How does the body:[ ... ] part parse? Is there a way of quickly 'skipping' from start '[' to the end ']' without having to parse the source in the midlle with a full Smalltalk parser? It can't just skip matching '[ ]' pairs, in case there are ']' embedded in strings etc. , right? DeltaStreams, on disk, could be large. Especially if used as a kind of finer grained .changes file. Quickly scanning the file to find 'interesting' entries, without deserializing the whole file into memory, would be useful. Cheers, Andy |
|
|
Hi guys!
First - about file format(s) - I really don't think it is a key issue but I personally would prefer something very simple. But not hardwiring a specific format into the code seems a smart move. And also - I am not close to fileout/in yet anyway. ;) [SNIP] > DeltaStreams, on disk, could be large. Especially if used as a kind of > finer > grained .changes file. Quickly scanning the file to find 'interesting' > entries, without deserializing the whole file into memory, would be > useful. Again, I was thinking KISS. A stream is a directory of deltas. A delta is a gzipped file of a readable format with a name that has a number prefix so that the files have an order in the directory. Lots of small files? Sure. A problem? I presume not with modern filesystems. We can also just zip up such a directory if we like. regards, Göran |
Re: DeltaStreams small update
|
|
In reply to this post by ccrraaiigg
Hi Craig,
I see the smiley and the fun of your un-do interpretation :) But I alwo want to see that Smalltalk/Squeak code is found and properly indexed by *all* the crawlers on the net. And for the case that crawlers don't eat St code then give them a honey pot. How goes the saying, if the big battle can't be won, win all the small ones :) /Klaus On Fri, 24 Aug 2007 03:39:35 +0200, you wrote: > > > Of course the other direction (XML to your 2c's) is a bit more > > complicated but doesn't look undoable. > > That's what I'm afraid of, that we'll introduce XML into the mix > and won't be able to un-do it. :) > > > -C > > -- > Craig Latta > improvisational musical informaticist > www.netjam.org > Smalltalkers do: [:it | All with: Class, (And love: it)] > > > |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |