CQRS and Event Sourcing in Smalltalk

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

CQRS and Event Sourcing in Smalltalk

Rafael Luque

Hi all,

Please, anyone knows a Pharo framework to implement CQRS and/or Event Sourcing?

Thank you!

Rafael Luque
Reply | Threaded
Open this post in threaded view
|

Re: CQRS and Event Sourcing in Smalltalk

Stéphane Ducasse

>
> Hi all,
>
> Please, anyone knows a Pharo framework to implement CQRS and/or Event Sourcing?

I read in diagonal the paper of martin fowler and I would say just some command patterns?
+ some meta modeling?
no?

>
> Thank you!
>
> Rafael Luque


Reply | Threaded
Open this post in threaded view
|

Re: CQRS and Event Sourcing in Smalltalk

NorbertHartl

Am 13.12.2012 um 17:27 schrieb Stéphane Ducasse <[hidden email]>:

>
>>
>> Hi all,
>>
>> Please, anyone knows a Pharo framework to implement CQRS and/or Event Sourcing?
>
> I read in diagonal the paper of martin fowler and I would say just some command patterns?
> + some meta modeling?
> no?


Well, you can give that particular answer to _a lot_ of problems :) I think the important part is the "event store" and how you snapshot state. So it is more or less a persistence question. Staying in-image it is rather easy. I use an approach where I still have the business objects holding state. But state is altered through commands and every command emits a set of events that get written to a timeline. The objects analyzing the timeline are more CQRS like because the state evolves out of the timeline. So I have an

Event class with instVar time (aDateAndTime)
A timeline is a sorted collection (by time) of events.
An object has a stream on the timeline. Its position is the positional information about the snapshot state.
Snapshot state is updated by following the stream (next: 100, upToDate: … upToEventType:…)

Altogether you have sources that put events on the timeline. Separated from that you have the consumer that reads the snapshot state and updates it.

Norbert
Reply | Threaded
Open this post in threaded view
|

Re: CQRS and Event Sourcing in Smalltalk

Stéphane Ducasse
>>> Please, anyone knows a Pharo framework to implement CQRS and/or Event Sourcing?
>>
>> I read in diagonal the paper of martin fowler and I would say just some command patterns?
>> + some meta modeling?
>> no?
>
> Well, you can give that particular answer to _a lot_ of problems :)

yes the article was not really clear.

> I think the important part is the "event store" and how you snapshot state. So it is more or less a persistence question. Staying in-image it is rather easy. I use an approach where I still have the business objects holding state. But state is altered through commands and every command emits a set of events that get written to a timeline. The objects analyzing the timeline are more CQRS like because the state evolves out of the timeline. So I have an
>
> Event class with instVar time (aDateAndTime)
> A timeline is a sorted collection (by time) of events.
> An object has a stream on the timeline. Its position is the positional information about the snapshot state.
> Snapshot state is updated by following the stream (next: 100, upToDate: … upToEventType:…)
>
> Altogether you have sources that put events on the timeline. Separated from that you have the consumer that reads the snapshot state and updates it.
>
> Norbert


Reply | Threaded
Open this post in threaded view
|

Re: CQRS and Event Sourcing in Smalltalk

Sven Van Caekenberghe-2
In reply to this post by NorbertHartl
On 13 Dec 2012, at 17:56, Norbert Hartl <[hidden email]> wrote:

> Am 13.12.2012 um 17:27 schrieb Stéphane Ducasse <[hidden email]>:
>
>>> Hi all,
>>>
>>> Please, anyone knows a Pharo framework to implement CQRS and/or Event Sourcing?
>>
>> I read in diagonal the paper of martin fowler and I would say just some command patterns?
>> + some meta modeling?
>> no?
>
> Well, you can give that particular answer to _a lot_ of problems :) I think the important part is the "event store" and how you snapshot state. So it is more or less a persistence question. Staying in-image it is rather easy. I use an approach where I still have the business objects holding state. But state is altered through commands and every command emits a set of events that get written to a timeline. The objects analyzing the timeline are more CQRS like because the state evolves out of the timeline. So I have an
>
> Event class with instVar time (aDateAndTime)
> A timeline is a sorted collection (by time) of events.
> An object has a stream on the timeline. Its position is the positional information about the snapshot state.
> Snapshot state is updated by following the stream (next: 100, upToDate: … upToEventType:…)
>
> Altogether you have sources that put events on the timeline. Separated from that you have the consumer that reads the snapshot state and updates it.
>
> Norbert

Sounds really cool and exciting, Norbert.
I think many people including myself are thinking along those lines.
Next time we meet you'll have to show me ;-)

Sven

--
Sven Van Caekenberghe
http://stfx.eu
Smalltalk is the Red Pill


Reply | Threaded
Open this post in threaded view
|

Re: CQRS and Event Sourcing in Smalltalk

NorbertHartl

Am 13.12.2012 um 20:36 schrieb Sven Van Caekenberghe <[hidden email]>:

> On 13 Dec 2012, at 17:56, Norbert Hartl <[hidden email]> wrote:
>
>> Am 13.12.2012 um 17:27 schrieb Stéphane Ducasse <[hidden email]>:
>>
>>>> Hi all,
>>>>
>>>> Please, anyone knows a Pharo framework to implement CQRS and/or Event Sourcing?
>>>
>>> I read in diagonal the paper of martin fowler and I would say just some command patterns?
>>> + some meta modeling?
>>> no?
>>
>> Well, you can give that particular answer to _a lot_ of problems :) I think the important part is the "event store" and how you snapshot state. So it is more or less a persistence question. Staying in-image it is rather easy. I use an approach where I still have the business objects holding state. But state is altered through commands and every command emits a set of events that get written to a timeline. The objects analyzing the timeline are more CQRS like because the state evolves out of the timeline. So I have an
>>
>> Event class with instVar time (aDateAndTime)
>> A timeline is a sorted collection (by time) of events.
>> An object has a stream on the timeline. Its position is the positional information about the snapshot state.
>> Snapshot state is updated by following the stream (next: 100, upToDate: … upToEventType:…)
>>
>> Altogether you have sources that put events on the timeline. Separated from that you have the consumer that reads the snapshot state and updates it.
>>
>> Norbert
>
> Sounds really cool and exciting, Norbert.
> I think many people including myself are thinking along those lines.
> Next time we meet you'll have to show me ;-)
>
Sure :) I think this approach has some benefits.  Normal objects can be seen as the only and most actual snapshot of a state changing history. The history gets discarded immediately. But this history has a great value in itself being able to replay/undo things or develop new values that can be applied at a later moment. We just need to look at the debugger. A stack or context chain is just a collection of moments in time. If you go up the stack we go back in time. But objects stay in the now. It would just be perfect if the object state would go back in time as well.

Let's take a networked service as an example. If we look more closely we can see a snapshot of data is about actuality and consistency. We always like to have consistent data. But actuality is ambiguous in a concurrent media with non-deterministic latencies like the internet. If we separate these we find the need to have data that is consistent at any point in time. But reading the data does not have to return the newest state available.
So we divide the process into three things: adding state change, updating state snapshot, reading state snapshot. The adding is an append-only operation that can be done in a very fast way by the persistence and has less issues when it comes to concurrency. Updating the state snapshot from the state change events can be done according to the needs for actuality or performance. Just to make it clear: In this approach there is also just one entity that is updating the state, so more concurrency problems solved. Finally reading the state can be done in multiple ways. Either you read the provided snapshot or you update the snapshot to the newest value in your private copy of a state holder. This time again you are not changing something concurrent.

A more real life example is one of our products. I changed to model to emit events 6 month ago. Well, using GemStone I didn't have to think about persistence. It is just a persisted collection with a few million events in it. 3 month _after_ I started to collect events I started to think about what data I actually need to show on the web UI. Having all the history in my hand I can derive plenty of different values just afterwards. Every value has a stream on the events collection. So every value has its own cursor in history. New values just start at the beginning of the collection and catch to now. Old ones have just little to update.

I hope this is useful somehow,

Norbert
 


Reply | Threaded
Open this post in threaded view
|

Re: CQRS and Event Sourcing in Smalltalk

Rafael Luque
Thank you for your responses,

Stéphane, I think Fowler is not the best source for this topic, in spite of his fame. Sorry, but I'm a bit iconoclastic and my impression about him changed after the "prevalence case" [1]. I think Greg Young coined the term and is the main evangelist about CQRS and Event Sourcing.

Norbert, I find your descriptions really interesting, but I assume that you have implemented all the required infrastructure: event bus, event store, snapshotting mechanism, etc.

What I'd like to know is if there is an Smalltalk project similar to the Axon framework for Java [2] providing implementations of the most important building blocks in CQRS architecture, such as aggregates, repositories and event buses. For instance, I think it would be specially useful a framework to build webapps after the CQRS architecture.

[1] https://gist.github.com/1186975
[2] http://www.axonframework.org/

Rafa

2012/12/14 Norbert Hartl <[hidden email]>

Am 13.12.2012 um 20:36 schrieb Sven Van Caekenberghe <[hidden email]>:

> On 13 Dec 2012, at 17:56, Norbert Hartl <[hidden email]> wrote:
>
>> Am 13.12.2012 um 17:27 schrieb Stéphane Ducasse <[hidden email]>:
>>
>>>> Hi all,
>>>>
>>>> Please, anyone knows a Pharo framework to implement CQRS and/or Event Sourcing?
>>>
>>> I read in diagonal the paper of martin fowler and I would say just some command patterns?
>>> + some meta modeling?
>>> no?
>>
>> Well, you can give that particular answer to _a lot_ of problems :) I think the important part is the "event store" and how you snapshot state. So it is more or less a persistence question. Staying in-image it is rather easy. I use an approach where I still have the business objects holding state. But state is altered through commands and every command emits a set of events that get written to a timeline. The objects analyzing the timeline are more CQRS like because the state evolves out of the timeline. So I have an
>>
>> Event class with instVar time (aDateAndTime)
>> A timeline is a sorted collection (by time) of events.
>> An object has a stream on the timeline. Its position is the positional information about the snapshot state.
>> Snapshot state is updated by following the stream (next: 100, upToDate: … upToEventType:…)
>>
>> Altogether you have sources that put events on the timeline. Separated from that you have the consumer that reads the snapshot state and updates it.
>>
>> Norbert
>
> Sounds really cool and exciting, Norbert.
> I think many people including myself are thinking along those lines.
> Next time we meet you'll have to show me ;-)
>
Sure :) I think this approach has some benefits.  Normal objects can be seen as the only and most actual snapshot of a state changing history. The history gets discarded immediately. But this history has a great value in itself being able to replay/undo things or develop new values that can be applied at a later moment. We just need to look at the debugger. A stack or context chain is just a collection of moments in time. If you go up the stack we go back in time. But objects stay in the now. It would just be perfect if the object state would go back in time as well.

Let's take a networked service as an example. If we look more closely we can see a snapshot of data is about actuality and consistency. We always like to have consistent data. But actuality is ambiguous in a concurrent media with non-deterministic latencies like the internet. If we separate these we find the need to have data that is consistent at any point in time. But reading the data does not have to return the newest state available.
So we divide the process into three things: adding state change, updating state snapshot, reading state snapshot. The adding is an append-only operation that can be done in a very fast way by the persistence and has less issues when it comes to concurrency. Updating the state snapshot from the state change events can be done according to the needs for actuality or performance. Just to make it clear: In this approach there is also just one entity that is updating the state, so more concurrency problems solved. Finally reading the state can be done in multiple ways. Either you read the provided snapshot or you update the snapshot to the newest value in your private copy of a state holder. This time again you are not changing something concurrent.

A more real life example is one of our products. I changed to model to emit events 6 month ago. Well, using GemStone I didn't have to think about persistence. It is just a persisted collection with a few million events in it. 3 month _after_ I started to collect events I started to think about what data I actually need to show on the web UI. Having all the history in my hand I can derive plenty of different values just afterwards. Every value has a stream on the events collection. So every value has its own cursor in history. New values just start at the beginning of the collection and catch to now. Old ones have just little to update.

I hope this is useful somehow,

Norbert




Reply | Threaded
Open this post in threaded view
|

Re: CQRS and Event Sourcing in Smalltalk

NorbertHartl
Rafael,

Am 17.12.2012 um 20:38 schrieb Rafael Luque <[hidden email]>:

Thank you for your responses,

Stéphane, I think Fowler is not the best source for this topic, in spite of his fame. Sorry, but I'm a bit iconoclastic and my impression about him changed after the "prevalence case" [1]. I think Greg Young coined the term and is the main evangelist about CQRS and Event Sourcing.

Norbert, I find your descriptions really interesting, but I assume that you have implemented all the required infrastructure: event bus, event store, snapshotting mechanism, etc.

I had a look at the axon framework. In order to cope with a lot of stuff the axon thing sounds utterly complex. Unless you have a project that is really complex it is probably not a good idea to go for something like axon. Smaller problems are easier and more suitable to solve.
What you said that I "implemented all the required infrastructure: event bus, event store, snapshotting mechanism, etc." might be true. But

- there is no event bus. Events get emitted at the most important object in the domain model (you'd probably call that aggregate root)
- the event store is a collection. 
- commands also do not need a CommandBus because there is a central registry for handlers and commands are dispatched at the very same place
- the snapshotting mechanism is implemented by me. But I would guess this is quite normal because use cases are hard to generalize. 

So my general advize would be to start small. Think about the central concepts of CQRS and what you need from. If you are looking at the java API and do a solution in smalltalk you just need some confidence that you didn't forget a ton of stuff only because it is much smaller and easier :)

I really like the topic. If you have other inquiries or insights regarding this topic I'm always interested. If you can tell what you need I can think about what of the stuff I did might be useful to others.

Norbert

What I'd like to know is if there is an Smalltalk project similar to the Axon framework for Java [2] providing implementations of the most important building blocks in CQRS architecture, such as aggregates, repositories and event buses. For instance, I think it would be specially useful a framework to build webapps after the CQRS architecture.

[1] https://gist.github.com/1186975
[2] http://www.axonframework.org/

Rafa

2012/12/14 Norbert Hartl <[hidden email]>

Am 13.12.2012 um 20:36 schrieb Sven Van Caekenberghe <[hidden email]>:

> On 13 Dec 2012, at 17:56, Norbert Hartl <[hidden email]> wrote:
>
>> Am 13.12.2012 um 17:27 schrieb Stéphane Ducasse <[hidden email]>:
>>
>>>> Hi all,
>>>>
>>>> Please, anyone knows a Pharo framework to implement CQRS and/or Event Sourcing?
>>>
>>> I read in diagonal the paper of martin fowler and I would say just some command patterns?
>>> + some meta modeling?
>>> no?
>>
>> Well, you can give that particular answer to _a lot_ of problems :) I think the important part is the "event store" and how you snapshot state. So it is more or less a persistence question. Staying in-image it is rather easy. I use an approach where I still have the business objects holding state. But state is altered through commands and every command emits a set of events that get written to a timeline. The objects analyzing the timeline are more CQRS like because the state evolves out of the timeline. So I have an
>>
>> Event class with instVar time (aDateAndTime)
>> A timeline is a sorted collection (by time) of events.
>> An object has a stream on the timeline. Its position is the positional information about the snapshot state.
>> Snapshot state is updated by following the stream (next: 100, upToDate: … upToEventType:…)
>>
>> Altogether you have sources that put events on the timeline. Separated from that you have the consumer that reads the snapshot state and updates it.
>>
>> Norbert
>
> Sounds really cool and exciting, Norbert.
> I think many people including myself are thinking along those lines.
> Next time we meet you'll have to show me ;-)
>
Sure :) I think this approach has some benefits.  Normal objects can be seen as the only and most actual snapshot of a state changing history. The history gets discarded immediately. But this history has a great value in itself being able to replay/undo things or develop new values that can be applied at a later moment. We just need to look at the debugger. A stack or context chain is just a collection of moments in time. If you go up the stack we go back in time. But objects stay in the now. It would just be perfect if the object state would go back in time as well.

Let's take a networked service as an example. If we look more closely we can see a snapshot of data is about actuality and consistency. We always like to have consistent data. But actuality is ambiguous in a concurrent media with non-deterministic latencies like the internet. If we separate these we find the need to have data that is consistent at any point in time. But reading the data does not have to return the newest state available.
So we divide the process into three things: adding state change, updating state snapshot, reading state snapshot. The adding is an append-only operation that can be done in a very fast way by the persistence and has less issues when it comes to concurrency. Updating the state snapshot from the state change events can be done according to the needs for actuality or performance. Just to make it clear: In this approach there is also just one entity that is updating the state, so more concurrency problems solved. Finally reading the state can be done in multiple ways. Either you read the provided snapshot or you update the snapshot to the newest value in your private copy of a state holder. This time again you are not changing something concurrent.

A more real life example is one of our products. I changed to model to emit events 6 month ago. Well, using GemStone I didn't have to think about persistence. It is just a persisted collection with a few million events in it. 3 month _after_ I started to collect events I started to think about what data I actually need to show on the web UI. Having all the history in my hand I can derive plenty of different values just afterwards. Every value has a stream on the events collection. So every value has its own cursor in history. New values just start at the beginning of the collection and catch to now. Old ones have just little to update.

I hope this is useful somehow,

Norbert





Reply | Threaded
Open this post in threaded view
|

Re: CQRS and Event Sourcing in Smalltalk

Rafael Luque

Thank you again Norbert,

So my general advize would be to start small. Think about the central concepts of CQRS and what you need from. If you are looking at the java API and do a solution in smalltalk you just need some confidence that you didn't forget a ton of stuff only because it is much smaller and easier :)

I agree with you and this is the reason why I was looking for an Smalltalk framework to build CQRS webapps.
 
I really like the topic. If you have other inquiries or insights regarding this topic I'm always interested. If you can tell what you need I can think about what of the stuff I did might be useful to others.

Unfortunately, what I really need is to implement CQRS, or some of its concepts as you said, in a Java-based webapp, but not in Smalltalk. We are suffering severe performance issues and I'm evaluating if the "read model" of CQRS could be a good solution for our problem. Then I thought: why not to look up inspiration in an Smalltalk framework, if it exists :-)

Rafael Luque

Reply | Threaded
Open this post in threaded view
|

Re: CQRS and Event Sourcing in Smalltalk

NorbertHartl

Am 19.12.2012 um 13:50 schrieb Rafael Luque <[hidden email]>:

Unfortunately, what I really need is to implement CQRS, or some of its concepts as you said, in a Java-based webapp, but not in Smalltalk. We are suffering severe performance issues and I'm evaluating if the "read model" of CQRS could be a good solution for our problem. Then I thought: why not to look up inspiration in an Smalltalk framework, if it exists :-)

Please keep me updated. I didn't think of using CQRS like stuff for performance reasons. From the gut I would say if you haven't a lot of monitors/locks active or you can't afford to update the read model less often then the odds are not that high in helping with performance  issues. But interesting indeed.

Norbert