[squeak-dev] DeltaStreams at ESUG in Brest

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[squeak-dev] DeltaStreams at ESUG in Brest

Göran Krampe
Hi folks!

Back from Brest and a GREAT conference. This post is not about the ESUG
conference though (I could write a lot about that) but about
DeltaStreams that I presented and worked hard on while being there.

We had a lot of interest in it, and not only from Squeakers.
Unfortunately my presentation collided with Lukas/Seaside so many missed
it. Slides are here:

http://www.slideshare.net/esug/deltastreams

...and perhaps a movie is available somewhere, dunno.

Status:

- Igor joined me and it was real fun to finally meet in real life and
work together. Igor is now working on a Toolbuilder based UI for Deltas
using TestRunner as inspiration for code etc.

- I hooked in Tirade fully. This means Deltas can now be filedOut and
read back in using Tirade format. This is also covered by some smart
tests written originally by Matthew Fulmer. Simplified Tirade in the
process.

- We fixed lots of Delta tests and the tests are now almost fully in the
green. Hopefully in a few days more I can say 100% green.

- We moved over development from 3.10.2-7179 to trunk! :) Not a big deal
actually but we also turned the base patches into MCs so you can
actually pull DeltaStreams into trunk by loading one MC, and then
running a single liner that uses Installer to pull in the rest.

- We added code for tracking DoIts and I almost burned my braincells
figuring out a *sensible* way to deal with them. This is an interesting
area... I will post separately I think :)

- Lots and lots of other details, don't remember them all.

- We also talked a lot with Pharo people about our dependencies. Deltas
so far depend on four important pieces:
        - SystemChangeNotification and the event classes for it.
        - SystemEditor mainly written by Colin Putney for MC2.
        - Toolbuilder for UIs.
        - Tirade for serialization.

We hope these dependencies will be kept up to date in most Squeak flavors.

The change notification stuff originally written by Roel Wuyts is a bit
messy, but it works. It could be rewritten using Announcements or
whatever (Stephane wants that) but if we do that then we really need to
make sure ALL Squeaks do it together. Otherwise we are going into a
"world of hurt" for building cross fork tools.

SystemEditor works in trunk, although has 11 failures. I think it is
broken in Pharo, haven't looked closer yet.

Toolbuilder is of course kept up to date in trunk by Andreas (I guess),
I hope that Pharo will sync those changes?!

Finally Tirade should work fine in all Squeaks and probably other
Smalltalks too, just vanilla code. And since it is my code I can make
sure it does work wherever we want Deltas to work :)

Now, regarding these dependencies - Toolbuilder is of course just for
the UIs. Deltas can still be used in a Squeak without Toolbuilder.
Tirade is "just" for serialization, but hey, that is kinda useful :)

And if SystemEditor does not work (does it work in Cuis?) then we will
need an extra applier written that uses the "regular style" for messing
with classes. That should not be hard to do, just mimic what a Changeset
does more or less. And I have had that on the todo list for a while.

regards, Göran


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] DeltaStreams at ESUG in Brest

Juan Vuletich-4
Hi Göran,

Göran Krampe wrote:

> Hi folks!
>
> Back from Brest and a GREAT conference. This post is not about the
> ESUG conference though (I could write a lot about that) but about
> DeltaStreams that I presented and worked hard on while being there.
>
> We had a lot of interest in it, and not only from Squeakers.
> Unfortunately my presentation collided with Lukas/Seaside so many
> missed it. Slides are here:
>
> http://www.slideshare.net/esug/deltastreams
>
> ...and perhaps a movie is available somewhere, dunno.
>
> Status:
>
> - Igor joined me and it was real fun to finally meet in real life and
> work together. Igor is now working on a Toolbuilder based UI for
> Deltas using TestRunner as inspiration for code etc.
>
> - I hooked in Tirade fully. This means Deltas can now be filedOut and
> read back in using Tirade format. This is also covered by some smart
> tests written originally by Matthew Fulmer. Simplified Tirade in the
> process.
>
> - We fixed lots of Delta tests and the tests are now almost fully in
> the green. Hopefully in a few days more I can say 100% green.
>
> - We moved over development from 3.10.2-7179 to trunk! :) Not a big
> deal actually but we also turned the base patches into MCs so you can
> actually pull DeltaStreams into trunk by loading one MC, and then
> running a single liner that uses Installer to pull in the rest.
>
> - We added code for tracking DoIts and I almost burned my braincells
> figuring out a *sensible* way to deal with them. This is an
> interesting area... I will post separately I think :)
>
> - Lots and lots of other details, don't remember them all.
>
> - We also talked a lot with Pharo people about our dependencies.
> Deltas so far depend on four important pieces:
>     - SystemChangeNotification and the event classes for it.
>     - SystemEditor mainly written by Colin Putney for MC2.
>     - Toolbuilder for UIs.
>     - Tirade for serialization.
>
> We hope these dependencies will be kept up to date in most Squeak
> flavors.
>
> The change notification stuff originally written by Roel Wuyts is a
> bit messy, but it works. It could be rewritten using Announcements or
> whatever (Stephane wants that) but if we do that then we really need
> to make sure ALL Squeaks do it together. Otherwise we are going into a
> "world of hurt" for building cross fork tools.
>
> SystemEditor works in trunk, although has 11 failures. I think it is
> broken in Pharo, haven't looked closer yet.
>
> Toolbuilder is of course kept up to date in trunk by Andreas (I
> guess), I hope that Pharo will sync those changes?!
>
> Finally Tirade should work fine in all Squeaks and probably other
> Smalltalks too, just vanilla code. And since it is my code I can make
> sure it does work wherever we want Deltas to work :)
>
> Now, regarding these dependencies - Toolbuilder is of course just for
> the UIs. Deltas can still be used in a Squeak without Toolbuilder.
> Tirade is "just" for serialization, but hey, that is kinda useful :)
>
> And if SystemEditor does not work (does it work in Cuis?) then we will
> need an extra applier written that uses the "regular style" for
> messing with classes. That should not be hard to do, just mimic what a
> Changeset does more or less. And I have had that on the todo list for
> a while.
>
> regards, Göran

It is great to know DeltaStreams keeps advancing!

I didn't have much time to understand the dependencies yet. I just want
to say that I want to support DeltaStreams in Cuis.

I guess I'd need to work on ToolBuilder, but I think it is a good
addition to Cuis.

I need to study a bit SystemEditor and the SystemChangeNotification. Not
sure if I'll want to load that code (I really need to check it), but in
any case I'm willing to make easonable alternatives to work in Cuis to
support DeltaStreams.

Cheers,
Juan Vuletich

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] DeltaStreams at ESUG in Brest

Igor Stasenko
2009/9/7 Juan Vuletich <[hidden email]>:

> Hi Göran,
>
> Göran Krampe wrote:
>>
>> Hi folks!
>>
>> Back from Brest and a GREAT conference. This post is not about the ESUG
>> conference though (I could write a lot about that) but about DeltaStreams
>> that I presented and worked hard on while being there.
>>
>> We had a lot of interest in it, and not only from Squeakers. Unfortunately
>> my presentation collided with Lukas/Seaside so many missed it. Slides are
>> here:
>>
>> http://www.slideshare.net/esug/deltastreams
>>
>> ...and perhaps a movie is available somewhere, dunno.
>>
>> Status:
>>
>> - Igor joined me and it was real fun to finally meet in real life and work
>> together. Igor is now working on a Toolbuilder based UI for Deltas using
>> TestRunner as inspiration for code etc.
>>
>> - I hooked in Tirade fully. This means Deltas can now be filedOut and read
>> back in using Tirade format. This is also covered by some smart tests
>> written originally by Matthew Fulmer. Simplified Tirade in the process.
>>
>> - We fixed lots of Delta tests and the tests are now almost fully in the
>> green. Hopefully in a few days more I can say 100% green.
>>
>> - We moved over development from 3.10.2-7179 to trunk! :) Not a big deal
>> actually but we also turned the base patches into MCs so you can actually
>> pull DeltaStreams into trunk by loading one MC, and then running a single
>> liner that uses Installer to pull in the rest.
>>
>> - We added code for tracking DoIts and I almost burned my braincells
>> figuring out a *sensible* way to deal with them. This is an interesting
>> area... I will post separately I think :)
>>
>> - Lots and lots of other details, don't remember them all.
>>
>> - We also talked a lot with Pharo people about our dependencies. Deltas so
>> far depend on four important pieces:
>>    - SystemChangeNotification and the event classes for it.
>>    - SystemEditor mainly written by Colin Putney for MC2.
>>    - Toolbuilder for UIs.
>>    - Tirade for serialization.
>>
>> We hope these dependencies will be kept up to date in most Squeak flavors.
>>
>> The change notification stuff originally written by Roel Wuyts is a bit
>> messy, but it works. It could be rewritten using Announcements or whatever
>> (Stephane wants that) but if we do that then we really need to make sure ALL
>> Squeaks do it together. Otherwise we are going into a "world of hurt" for
>> building cross fork tools.
>>
>> SystemEditor works in trunk, although has 11 failures. I think it is
>> broken in Pharo, haven't looked closer yet.
>>
>> Toolbuilder is of course kept up to date in trunk by Andreas (I guess), I
>> hope that Pharo will sync those changes?!
>>
>> Finally Tirade should work fine in all Squeaks and probably other
>> Smalltalks too, just vanilla code. And since it is my code I can make sure
>> it does work wherever we want Deltas to work :)
>>
>> Now, regarding these dependencies - Toolbuilder is of course just for the
>> UIs. Deltas can still be used in a Squeak without Toolbuilder. Tirade is
>> "just" for serialization, but hey, that is kinda useful :)
>>
>> And if SystemEditor does not work (does it work in Cuis?) then we will
>> need an extra applier written that uses the "regular style" for messing with
>> classes. That should not be hard to do, just mimic what a Changeset does
>> more or less. And I have had that on the todo list for a while.
>>
>> regards, Göran
>
> It is great to know DeltaStreams keeps advancing!
>
> I didn't have much time to understand the dependencies yet. I just want to
> say that I want to support DeltaStreams in Cuis.
>
> I guess I'd need to work on ToolBuilder, but I think it is a good addition
> to Cuis.
>
The ToolBuilder is a bare-bone tool which allow us to build a UI in
declarative style, without
specific knowledge, what exactly framework will be used to preset it to user.
If you having something different in mind (like making own UI-building
tool) , it could be done pretty easily ;)

Our main design point for DeltaStreams that it _should_ be useful even
without any UI, and it should be possible to at least file-in or
file-out DS. So, Deltas is targeted for a very constrained
environments.

But then, of course, if you have a UI, you , as developer, could do a
lot more using deltas..  :)

> I need to study a bit SystemEditor and the SystemChangeNotification. Not
> sure if I'll want to load that code (I really need to check it), but in any
> case I'm willing to make easonable alternatives to work in Cuis to support
> DeltaStreams.
>
To work properly , delta needs to hook in to some places of system ,
in order to be able to record all events related to changes.
It is not really relevant , in what way it hooking up, what is
relevant is to be able to nicely capture all things the developer
doing.
Thus, you can provide own layer for capturing system editing events
and deltas will happily use it.


> Cheers,
> Juan Vuletich
>
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] DeltaStreams at ESUG in Brest

Juan Vuletich-4
Hi Igor,

Igor Stasenko wrote:

>
>> It is great to know DeltaStreams keeps advancing!
>>
>> I didn't have much time to understand the dependencies yet. I just want to
>> say that I want to support DeltaStreams in Cuis.
>>
>> I guess I'd need to work on ToolBuilder, but I think it is a good addition
>> to Cuis.
>>
>>    
> The ToolBuilder is a bare-bone tool which allow us to build a UI in
> declarative style, without
> specific knowledge, what exactly framework will be used to preset it to user.
> If you having something different in mind (like making own UI-building
> tool) , it could be done pretty easily ;)
>
>  

No, I know ToolBuilder since quite long.

> Our main design point for DeltaStreams that it _should_ be useful even
> without any UI, and it should be possible to at least file-in or
> file-out DS. So, Deltas is targeted for a very constrained
> environments.
>
> But then, of course, if you have a UI, you , as developer, could do a
> lot more using deltas..  :)
>  

I want to use DS to replace change sets. So I need the traditional gui:
Change Lists, Change Sorters, etc, or something similar to them.

>> I need to study a bit SystemEditor and the SystemChangeNotification. Not
>> sure if I'll want to load that code (I really need to check it), but in any
>> case I'm willing to make easonable alternatives to work in Cuis to support
>> DeltaStreams.
>>
>>    
> To work properly , delta needs to hook in to some places of system ,
> in order to be able to record all events related to changes.
> It is not really relevant , in what way it hooking up, what is
> relevant is to be able to nicely capture all things the developer
> doing.
> Thus, you can provide own layer for capturing system editing events
> and deltas will happily use it.

I know.

Thanks,
Juan Vuletich

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] DeltaStreams at ESUG in Brest

Göran Krampe
In reply to this post by Juan Vuletich-4

Hi!

Igor answered but I am doing it too:

Juan Vuletich wrote:
> It is great to know DeltaStreams keeps advancing!

Yes, it is great fun right now - especially when Igor joined.

> I didn't have much time to understand the dependencies yet. I just want
> to say that I want to support DeltaStreams in Cuis.

Yes, that is important for us. It also means we will take care trying to
make that possible. :)

> I guess I'd need to work on ToolBuilder, but I think it is a good
> addition to Cuis.

It probably is, and if you don't then you could always write your own
Delta tool - but then we would be doing redundant work. Also, the latest
TestRunner for unit tests is built on top of ToolBuilder...

> I need to study a bit SystemEditor and the SystemChangeNotification. Not
> sure if I'll want to load that code (I really need to check it), but in
> any case I'm willing to make easonable alternatives to work in Cuis to
> support DeltaStreams.

As Igor explained SystemChangeNotification is about capturing events. If
you have alternative ways in Cuis (haven't looked at Cuis yet) it is
easy to do it differently. The domain model does not know anything about
events etc, for trunk/3.10 and Pharo that code is in
DeltaStreams-Logging and I think it works for those flavors. AFAIK it
was originally written by Roel Wuyts.

The SystemEditor is the mechanism we are using today to *apply* a Delta.
  It is written originally by Colin Putney for use in MC2 and it offers
a way to make changes in an atomic fashion. Matthew has fixed several
issues with it along the way. Personally I find the code in SE to be...
confusing. But atomicity is of course nice.

We will most likely also need to implement a "regular" applier using the
same mechanisms that Changeset does today, but that has not yet been
done. So yes, if SystemEditor does not work in Cuis (or you don't want
it) then we will need to first write either a Cuis-specific applier or
write this "regular" applier, and use that one.

On my blog is the latest instruction on how to get started in trunk:

http://goran.krampe.se/blog/Squeak/hacking-on-deltastreams2.rdoc

Feel free to give us more info on the matters above, eventually we will
take a look at Cuis but we are still busy getting "stuff to work right". :)

regards, Göran


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] DeltaStreams at ESUG in Brest

Göran Krampe
In reply to this post by Juan Vuletich-4
Hi!

Juan Vuletich wrote:
> I want to use DS to replace change sets. So I need the traditional gui:
> Change Lists, Change Sorters, etc, or something similar to them.

Yeah, one idea that is a "hack" of course would be to see if we could
make a Delta "mimic" a Changeset or if we could make a wrapper to make
it look like one.

...or just copy those tools and change the parts that needs to be changed.

regards, Göran


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] DeltaStreams at ESUG in Brest

Igor Stasenko
Goran,
at early stages, and for generic backwards compatibility it would be
nice to have a
Delta -> ChangeSet converter
this will mean that we could stay compatible with any historical
version(s) of Squeak at some point.
>From other point , by making this converter available, there is a
potential risk that people will avoid integrating
deltas into own forks, just because they can stick with the old ways :)

2009/9/7 Göran Krampe <[hidden email]>:

> Hi!
>
> Juan Vuletich wrote:
>>
>> I want to use DS to replace change sets. So I need the traditional gui:
>> Change Lists, Change Sorters, etc, or something similar to them.
>
> Yeah, one idea that is a "hack" of course would be to see if we could make a
> Delta "mimic" a Changeset or if we could make a wrapper to make it look like
> one.
>
> ...or just copy those tools and change the parts that needs to be changed.
>
> regards, Göran
>
>
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] DeltaStreams at ESUG in Brest

Juan Vuletich-4
In reply to this post by Göran Krampe
Hi Göran,

Göran Krampe wrote:
>
>> I didn't have much time to understand the dependencies yet. I just
>> want to say that I want to support DeltaStreams in Cuis.
>
> Yes, that is important for us. It also means we will take care trying
> to make that possible. :)

Great!

>
>> I need to study a bit SystemEditor and the SystemChangeNotification.
>> Not sure if I'll want to load that code (I really need to check it),
>> but in any case I'm willing to make easonable alternatives to work in
>> Cuis to support DeltaStreams.
>
> As Igor explained SystemChangeNotification is about capturing events.
> If you have alternative ways in Cuis (haven't looked at Cuis yet) it
> is easy to do it differently. The domain model does not know anything
> about events etc, for trunk/3.10 and Pharo that code is in
> DeltaStreams-Logging and I think it works for those flavors. AFAIK it
> was originally written by Roel Wuyts.

I'd forgotten about it, but System-Change Notification is already in
Cuis, and used for ChangeSets, etc.

> The SystemEditor is the mechanism we are using today to *apply* a
> Delta.  It is written originally by Colin Putney for use in MC2 and it
> offers a way to make changes in an atomic fashion. Matthew has fixed
> several issues with it along the way. Personally I find the code in SE
> to be... confusing. But atomicity is of course nice.

Just took a quick look at it. I understand that if you're using MC you
want such a thing. But it is a bit too much to be preloaded in Cuis. I
guess I prefer to leave it as part of MC2 for those who load it.

> We will most likely also need to implement a "regular" applier using
> the same mechanisms that Changeset does today, but that has not yet
> been done. So yes, if SystemEditor does not work in Cuis (or you don't
> want it) then we will need to first write either a Cuis-specific
> applier or write this "regular" applier, and use that one.

Yes. I guess this will be best for Cuis.

> On my blog is the latest instruction on how to get started in trunk:
>
> http://goran.krampe.se/blog/Squeak/hacking-on-deltastreams2.rdoc
>
> Feel free to give us more info on the matters above, eventually we
> will take a look at Cuis but we are still busy getting "stuff to work
> right". :)
>
> regards, Göran

Keep up the good work!

Cheers,
Juan Vuletich

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] DeltaStreams at ESUG in Brest

Igor Stasenko
2009/9/7 Juan Vuletich <[hidden email]>:

> Hi Göran,
>
> Göran Krampe wrote:
>>
>>> I didn't have much time to understand the dependencies yet. I just want
>>> to say that I want to support DeltaStreams in Cuis.
>>
>> Yes, that is important for us. It also means we will take care trying to
>> make that possible. :)
>
> Great!
>
>>
>>> I need to study a bit SystemEditor and the SystemChangeNotification. Not
>>> sure if I'll want to load that code (I really need to check it), but in any
>>> case I'm willing to make easonable alternatives to work in Cuis to support
>>> DeltaStreams.
>>
>> As Igor explained SystemChangeNotification is about capturing events. If
>> you have alternative ways in Cuis (haven't looked at Cuis yet) it is easy to
>> do it differently. The domain model does not know anything about events etc,
>> for trunk/3.10 and Pharo that code is in DeltaStreams-Logging and I think it
>> works for those flavors. AFAIK it was originally written by Roel Wuyts.
>
> I'd forgotten about it, but System-Change Notification is already in Cuis,
> and used for ChangeSets, etc.
>
>> The SystemEditor is the mechanism we are using today to *apply* a Delta.
>>  It is written originally by Colin Putney for use in MC2 and it offers a way
>> to make changes in an atomic fashion. Matthew has fixed several issues with
>> it along the way. Personally I find the code in SE to be... confusing. But
>> atomicity is of course nice.
>
> Just took a quick look at it. I understand that if you're using MC you want
> such a thing. But it is a bit too much to be preloaded in Cuis. I guess I
> prefer to leave it as part of MC2 for those who load it.
>
>> We will most likely also need to implement a "regular" applier using the
>> same mechanisms that Changeset does today, but that has not yet been done.
>> So yes, if SystemEditor does not work in Cuis (or you don't want it) then we
>> will need to first write either a Cuis-specific applier or write this
>> "regular" applier, and use that one.
>
> Yes. I guess this will be best for Cuis.
>

Okay, then i will avoid using SystemEditor in UI.
The only place where i need it currently is to represent a class
change by printing it as a class definition.

Then , i think, Goran, we should take care implementing
DSClassChange>>asClassDefinition
DSClassChange>>asOldClassDefinition
DSClassChange>>asMetaclassDefinition
DSClassChange>>asOldMetaclassDefinition

to represent the class definition before and after the change.
Other than that, the UI depends only on a toolbuilder (and a bit of morphic ;)

>> On my blog is the latest instruction on how to get started in trunk:
>>
>> http://goran.krampe.se/blog/Squeak/hacking-on-deltastreams2.rdoc
>>
>> Feel free to give us more info on the matters above, eventually we will
>> take a look at Cuis but we are still busy getting "stuff to work right". :)
>>
>> regards, Göran
>
> Keep up the good work!
>
> Cheers,
> Juan Vuletich
>
>



--
Best regards,
Igor Stasenko AKA sig.