Aare questions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Aare questions

hernanmd
Hi,

I decided to give a try to Aare or Workflow in Pharo 6, following
instructions in https://github.com/Netstyle/Workflow . However, the
installation script raised a Warning about missing WADynamicVariable
class. Should I install Seaside?

I've been using a couple of bioinformatics workflows the last years,
specially Taverna (https://en.wikipedia.org/wiki/Apache_Taverna) and
Galaxy (https://www.galaxyproject.org/) both of them huge code bases
maintained since some years now and with very active communities from
biological sciences. I found a cool video showing Galaxy features:
https://www.youtube.com/watch?v=rVYzMWWRMN8

As impressive as you may think, they are not a Panacea for workflow
execution. An installation process could get really complicated, for
example interfacing with nginx and PostgreSQL, and customization of
toolshed could end in endless editing XML specifications.

I just skim through the white paper, but unfortunately didn't
recognized any of the compared features or workflow systems.
Is Aare intended to support only business workflows patterns?
Considering the Galaxy video above, do you think scientific workflows
could be supported in Aare or it is a completely different route?

Cheers,

Hernán

Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

Stephane Ducasse-3
Hi hernan

I started to clean the code in my fork.
If you want to contribute let me know.
I started to clean all misclassified methods to force me to  read the
code. (I used them as a marker to know if I passed on the code).
For the moment I made sure that the code loads.
- I created the variable as subclass of Object
- I removed the dependencies to omnibase.

I added class comments about what I understood.
I'm about to continue writing structural tests
and I will start to try to understand how to run a workflow.

Stef


On Sun, Jan 21, 2018 at 9:02 AM, Hernán Morales Durand
<[hidden email]> wrote:

> Hi,
>
> I decided to give a try to Aare or Workflow in Pharo 6, following
> instructions in https://github.com/Netstyle/Workflow . However, the
> installation script raised a Warning about missing WADynamicVariable
> class. Should I install Seaside?
>
> I've been using a couple of bioinformatics workflows the last years,
> specially Taverna (https://en.wikipedia.org/wiki/Apache_Taverna) and
> Galaxy (https://www.galaxyproject.org/) both of them huge code bases
> maintained since some years now and with very active communities from
> biological sciences. I found a cool video showing Galaxy features:
> https://www.youtube.com/watch?v=rVYzMWWRMN8
>
> As impressive as you may think, they are not a Panacea for workflow
> execution. An installation process could get really complicated, for
> example interfacing with nginx and PostgreSQL, and customization of
> toolshed could end in endless editing XML specifications.
>
> I just skim through the white paper, but unfortunately didn't
> recognized any of the compared features or workflow systems.
> Is Aare intended to support only business workflows patterns?
> Considering the Galaxy video above, do you think scientific workflows
> could be supported in Aare or it is a completely different route?
>
> Cheers,
>
> Hernán
>

Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

hernanmd
Hi Stef,

2018-01-21 5:17 GMT-03:00 Stephane Ducasse <[hidden email]>:
> Hi hernan
>
> I started to clean the code in my fork.
> If you want to contribute let me know.

I don't know enough about workflow implementations yet. However I know
how to run or what to expect from them.
So I could give some feedback from there at first.

> I started to clean all misclassified methods to force me to  read the
> code. (I used them as a marker to know if I passed on the code).

That could result in a nice browser feature: To flag methods from menu
item adding specific icon, maybe possible in Calypso? :)

> For the moment I made sure that the code loads.
> - I created the variable as subclass of Object
> - I removed the dependencies to omnibase.
>
> I added class comments about what I understood.
> I'm about to continue writing structural tests
> and I will start to try to understand how to run a workflow.
>

Maybe someone at Netstyle has some expressions to begin with?

Hernán


> Stef
>
>
> On Sun, Jan 21, 2018 at 9:02 AM, Hernán Morales Durand
> <[hidden email]> wrote:
>> Hi,
>>
>> I decided to give a try to Aare or Workflow in Pharo 6, following
>> instructions in https://github.com/Netstyle/Workflow . However, the
>> installation script raised a Warning about missing WADynamicVariable
>> class. Should I install Seaside?
>>
>> I've been using a couple of bioinformatics workflows the last years,
>> specially Taverna (https://en.wikipedia.org/wiki/Apache_Taverna) and
>> Galaxy (https://www.galaxyproject.org/) both of them huge code bases
>> maintained since some years now and with very active communities from
>> biological sciences. I found a cool video showing Galaxy features:
>> https://www.youtube.com/watch?v=rVYzMWWRMN8
>>
>> As impressive as you may think, they are not a Panacea for workflow
>> execution. An installation process could get really complicated, for
>> example interfacing with nginx and PostgreSQL, and customization of
>> toolshed could end in endless editing XML specifications.
>>
>> I just skim through the white paper, but unfortunately didn't
>> recognized any of the compared features or workflow systems.
>> Is Aare intended to support only business workflows patterns?
>> Considering the Galaxy video above, do you think scientific workflows
>> could be supported in Aare or it is a completely different route?
>>
>> Cheers,
>>
>> Hernán
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

Stephane Ducasse-3
Hi hernan

I will try to allocate more time to understand the execution because I
think that this is important
for our community to get a workflow engine.
So I take it as a reverse engineering exercise. Now I'm super busy so
I will get slow on it.

Stef

On Sun, Jan 21, 2018 at 9:31 AM, Hernán Morales Durand
<[hidden email]> wrote:

> Hi Stef,
>
> 2018-01-21 5:17 GMT-03:00 Stephane Ducasse <[hidden email]>:
>> Hi hernan
>>
>> I started to clean the code in my fork.
>> If you want to contribute let me know.
>
> I don't know enough about workflow implementations yet. However I know
> how to run or what to expect from them.
> So I could give some feedback from there at first.
>
>> I started to clean all misclassified methods to force me to  read the
>> code. (I used them as a marker to know if I passed on the code).
>
> That could result in a nice browser feature: To flag methods from menu
> item adding specific icon, maybe possible in Calypso? :)
>
>> For the moment I made sure that the code loads.
>> - I created the variable as subclass of Object
>> - I removed the dependencies to omnibase.
>>
>> I added class comments about what I understood.
>> I'm about to continue writing structural tests
>> and I will start to try to understand how to run a workflow.
>>
>
> Maybe someone at Netstyle has some expressions to begin with?
>
> Hernán
>
>
>> Stef
>>
>>
>> On Sun, Jan 21, 2018 at 9:02 AM, Hernán Morales Durand
>> <[hidden email]> wrote:
>>> Hi,
>>>
>>> I decided to give a try to Aare or Workflow in Pharo 6, following
>>> instructions in https://github.com/Netstyle/Workflow . However, the
>>> installation script raised a Warning about missing WADynamicVariable
>>> class. Should I install Seaside?
>>>
>>> I've been using a couple of bioinformatics workflows the last years,
>>> specially Taverna (https://en.wikipedia.org/wiki/Apache_Taverna) and
>>> Galaxy (https://www.galaxyproject.org/) both of them huge code bases
>>> maintained since some years now and with very active communities from
>>> biological sciences. I found a cool video showing Galaxy features:
>>> https://www.youtube.com/watch?v=rVYzMWWRMN8
>>>
>>> As impressive as you may think, they are not a Panacea for workflow
>>> execution. An installation process could get really complicated, for
>>> example interfacing with nginx and PostgreSQL, and customization of
>>> toolshed could end in endless editing XML specifications.
>>>
>>> I just skim through the white paper, but unfortunately didn't
>>> recognized any of the compared features or workflow systems.
>>> Is Aare intended to support only business workflows patterns?
>>> Considering the Galaxy video above, do you think scientific workflows
>>> could be supported in Aare or it is a completely different route?
>>>
>>> Cheers,
>>>
>>> Hernán
>>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

NorbertHartl
I‘m in there, too, because I feel the same importance. Need to allocate some time

Norbert

> Am 21.01.2018 um 11:46 schrieb Stephane Ducasse <[hidden email]>:
>
> Hi hernan
>
> I will try to allocate more time to understand the execution because I
> think that this is important
> for our community to get a workflow engine.
> So I take it as a reverse engineering exercise. Now I'm super busy so
> I will get slow on it.
>
> Stef
>
> On Sun, Jan 21, 2018 at 9:31 AM, Hernán Morales Durand
> <[hidden email]> wrote:
>> Hi Stef,
>>
>> 2018-01-21 5:17 GMT-03:00 Stephane Ducasse <[hidden email]>:
>>> Hi hernan
>>>
>>> I started to clean the code in my fork.
>>> If you want to contribute let me know.
>>
>> I don't know enough about workflow implementations yet. However I know
>> how to run or what to expect from them.
>> So I could give some feedback from there at first.
>>
>>> I started to clean all misclassified methods to force me to  read the
>>> code. (I used them as a marker to know if I passed on the code).
>>
>> That could result in a nice browser feature: To flag methods from menu
>> item adding specific icon, maybe possible in Calypso? :)
>>
>>> For the moment I made sure that the code loads.
>>> - I created the variable as subclass of Object
>>> - I removed the dependencies to omnibase.
>>>
>>> I added class comments about what I understood.
>>> I'm about to continue writing structural tests
>>> and I will start to try to understand how to run a workflow.
>>>
>>
>> Maybe someone at Netstyle has some expressions to begin with?
>>
>> Hernán
>>
>>
>>> Stef
>>>
>>>
>>> On Sun, Jan 21, 2018 at 9:02 AM, Hernán Morales Durand
>>> <[hidden email]> wrote:
>>>> Hi,
>>>>
>>>> I decided to give a try to Aare or Workflow in Pharo 6, following
>>>> instructions in https://github.com/Netstyle/Workflow . However, the
>>>> installation script raised a Warning about missing WADynamicVariable
>>>> class. Should I install Seaside?
>>>>
>>>> I've been using a couple of bioinformatics workflows the last years,
>>>> specially Taverna (https://en.wikipedia.org/wiki/Apache_Taverna) and
>>>> Galaxy (https://www.galaxyproject.org/) both of them huge code bases
>>>> maintained since some years now and with very active communities from
>>>> biological sciences. I found a cool video showing Galaxy features:
>>>> https://www.youtube.com/watch?v=rVYzMWWRMN8
>>>>
>>>> As impressive as you may think, they are not a Panacea for workflow
>>>> execution. An installation process could get really complicated, for
>>>> example interfacing with nginx and PostgreSQL, and customization of
>>>> toolshed could end in endless editing XML specifications.
>>>>
>>>> I just skim through the white paper, but unfortunately didn't
>>>> recognized any of the compared features or workflow systems.
>>>> Is Aare intended to support only business workflows patterns?
>>>> Considering the Galaxy video above, do you think scientific workflows
>>>> could be supported in Aare or it is a completely different route?
>>>>
>>>> Cheers,
>>>>
>>>> Hernán
>>>>
>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

cedreek
In reply to this post by hernanmd
Hi Hernan,

I decided to give a try to Aare or Workflow in Pharo 6, following
instructions in https://github.com/Netstyle/Workflow .

Cool :)

I started but had to stop until the semester is finish. Next month should be ok to continue on that.


However, the
installation script raised a Warning about missing WADynamicVariable
class. Should I install Seaside?



Happy to see people having interest in such tools :)

No.

I discussed with Max Leske.

To solve this « pb », just create WADynamicVariable as a subclass of DynamicVariable already in Pharo. There are also missing part for the export/import in XML (should be easy to fix and Max propose to give the code - DOMBuilder and co.)
I tried to start a booklet with Stephane but it is too complicated right now. Stephane started a simpler version. 

I put below a summary of the reflexion/discussion I had with Max.


Cheers,

Cédrick




========= INTRO

What's missing compared to the full (proprietary) solution are user interfaces that were web based (Seaside \+ Magritte). Also, persistence was achived through Omnibase. References were removed but they are stil some disfunctionning. Moreover, the completion of activities were mainly manual and achieved through web interface (web forms). They are, for now, no real way to introduce different kind of activities like those being message-based, event-based, service-based.

To evolve, one has to remove all dependancies to Omnibase and Magritte/Seaside. 

There are also important design decisions to ensure WfWorkflow being a powerful and extensible workflow management system for Pharo. To me there are 2 main tasks:
- rethink persistence (process definition and orchestration, realization)
=> persistance is essential and was central to Aare. Beside removing Omnibase references (and implication of WfManagedObject), some of the application features were very dependant on the persistence (like logging, tracing process evolution). All of these features are to be thing again considering as much as possible a loosely coupled solution. A general purpose storage solution like Voyage could be used. At first, in image storage will be used to focus on the running aspects of the library.   
- rethink interaction 
=> Designing a general interaction sub-system is probably the more challenging tasks. Aare was web based. We want workflow to be UI agnostic. Moreover, we want to associate  ,
remove Magritte (replace UI or build an independant system based on Annoucement for instance)

========= RANDOM QUESTIONS AND ANSWER (in black me - below max)

1)

The static definition of workflow is done through WfWorkflow, WfOutgoingEdge and WfSteps (+ WfConditions). The easy part :)

The runtime aspect are achieved via WfActivation, WfWorkList, WfWorkflowHistory et WfWorkflowManager. More difficult to understand and make evolve.

WfWorkflowHistory represents the different versions of a workflow, so it belongs to the static part. WfWorkflowManager managers workflow definitions (static) and activations (dynamic).
WfFrame is also a part of the dynamic side, connecting activations to a workflow.
Workflows are hierarchical, i.e. a step can have a sub workflow (see WfWorkflowLibrary>>simpleSubflow)



2)

To me activations are really central entities (and I need to be sure of its correct role). They are said to the realization of an activity/step. 

Can we consider the static definition of a workflow as « Plan »  and the runtime as the concrete realization (like a job in a scheduler) ?  

Yes, I would say so.



I see that:


FUTURE: Static BP définitions with steps (plans)

PRESENT: activation (realization)

PAST: History (experience feedback)

Yes, that's certainly one way to look at it. Although I'd say that the static definition is independent from time once it exists and that "future" would be the set of steps of the current definition that have not yet been activated (just a little nit picking :) ).



3)

Going back to activations, here is my current understanding.

Once a step is activated (all condition on Edges leading to are true - to evaluate outgoing conditions of a step, the step execution has to be complete first) , we get an activation instance (I’m not 100% sure how completion of an activity and outgoing conditions are managed).

As far as I understand, a step can always be activated (e.g. manually). However, activating a step propagates the values of conditions on the edges to the next steps and their activations ("dead" or "alive"). Hence, an activation responds to #isAlive with true if at leat one incoming token was true ("alive") (see WfStep>>continueInFrame:fromStep:alive: and WfActivation>>isAlive). The conditions are evaluated once #complete is sent to the activation.
Activation instances are created by two ways (slightly different for the start step), but generally they are created when a step receives tokens from the outgoing edges of an upstream activation (see WfStep>>continueInFrame:fromStep:alive:).


On the exemple of BLActivationHistoryItem that you gave, I’m not sure if it’s an activation of a history item stored in the history (subclasss of WfHistory?).

My idea was that it should represent a record for an activation (there might be multiple records for a single activation). So "HistoryItem" would be the general description of a record, of which "ActivationHistoryItem" is a special case for activations (I hope that makes sense).


4)

Moreover I don’t see clearly the difference with a scheduler and a BP engine (they are at least really complementary). Do you see any ressemblance between Activation and ScheduledJob ?

I actually would like to have a (simple) scheduler with the BP engine :)

Well, it's certainly a matter of point of view. I would argue that an activation is something close to a "job". A "scheduled job" would be a more specific case of a "job", a job that is in a queue. An activation has no notion of scheduling but simply represents the idea of what happens when traversal between two nodes occurs. A scheduler would be one way of managing activations (WfWorklist seems pretty close to a scheduler, holding running and completed activations).


To me the scheduler right now is not really existing besides having a list of simultaneous/parralelized steps to realize (activated or not). Activated means currently being done.

You're right, I think. The application that Workflow was designed for required manual interaction, hence there was no need for other methods of scheduling (e.g. by time or event).



This activation role is about logging and interacting with responsible users. But it is more than that. It’s the actual job being done.

Yes.



5)

Concerning run time modification of a process. Is it true that we can modify a currently executed workflow (steps that are not yet activated ?) ? I think that is possible (an cool). When activated, it shouldn’t. Would you think it would be interesting to allow activations to be suspended/resume (frozen), cancelled ?

The history of workflows consists of copies. The steps of a workflow reference the workflow they belong to. Therefore, while it is possible to modify a workflow, modifications do not impact running workflows, as the steps of that workflow continue to reference the previous version. The model suggests that it is ok to create a new version of a workflow but not to modify existing workflows.
The model does *allow* (in principle) to modify an existing workflow but I doubt that was the intention. However, I don't see why modifying an existing workflow has to be bad thing, as long as you can guarantee that only edges and steps are modified that have not yet been activated. That could open up some interesting possibilities.
Cancellation of an activation could be very interesting. It would basically allow reversal of complex effects. Very nice idea! I'm not so sure though whether suspending makes much sense. Maybe when a step has the potential to take a long time for some operation. But then you may run into transaction problems. The nice thing about an activation at the moment is that it's atomic (in theory). But again, done right that could open up interesting possibilities.



If a running process is modified, do we track its evolutions in WfWorkflowManager ?

The modification of a workflow is indeed announced to WfWorkflowManager, yes. Workflow versions are stored in WfWorkflowHistory. For modifying a *running* workflow you will probably want to create a specific workflow manager for that, that does validation etc.



The goal of history(s) are to log the realization of the process. But is it one of its aim to « replay » the process execution ?

No. WfWorkflowHistory only stores the static definitions. However, the history of activations could be used to replay an execution (nice idea!).

 i.o.w.,  Is it more than log/report ?

I think the activation history was designed as a log, the workflow history as a log and also as a way to undo changes. As you suggest, however, they could certainly be used for more things.


I agree with the separation (WfWFManager for the run time evolution of the process definition => adaptation    and WfActivationHistory for the trace of interactive/automatic step realizations information).


#beginHistory implementation example was very helpful. I knew I missed something  Just, as I see a MailManager (with role is obvious), is it something that has to be generalizable ? Like InteroperableManager ? MessageManager ? To get user interactions

That would be very nice, yes. You're completely free to do what you like. It would be cool though if we could evolve the project as opposed to you creating a fork to your own liking. If you're game we can set up a workflow on github where you open pull requests against our repository.



6) I’d like to remove fully the reference and need of Omnibase and Magritte/seaside forms.

Yes please!


I think Omnibase is ok, even storing in image for instance.

What’s more problematic to me is for user interactions (and even automatic interaction through program for instance). There are achieved in your example with seaside. It is ok but what if I’d like to do it more generally ? Would we use Annoucement for example ? 

I think that is completely open to implementation. In my eyes, Workflow should remain a separate library that does not enforce any kind of specific presentation or persistence. What's critical for that is of course the right kind of interface.
Announcements would certainly be an interesting way to decouple Workflow from other parts of the system.


I’d like to have an incomingMessageBox for each user and eventually transform these message in concrete action (user i/o) ? Or dispatch in JobQueues (each user being a distant worker thread). Again, the distinction between Scheduler and BPEngin is fuzzy to me.


Does it make sense to you ?

Yes, it does :) What I've asked myself (out of curiosity) is whether you want to have 
- a dedicated instance for executing workflows
- a distributed workflow with autonomous synchronisation (parts of a workflow could run on every uses instance, others would need to be executed through agreement)
- a partially distributed workflow with one or more managing nodes that control the core steps (e.g. you could designate parts of a workflow as independant)


PS: some random comments on your proposals

Parts of Workflow were never cleanly separated from the proprietary application, so there are indeed a couple of things that are missing (mostly #subclassResponsibility stuff etc.)



- initializeDefaults is not present for steps but thats ok.


- in #createIncoming: (doesn’t exist), I don’t get ….. => makePersistent asPersistentReference.   You store persistentReference of steps. Is it like the step object itself ?  I would store the step objects (as no persistance right now). 

An activity step is a specific subclass of a step. Such a step includes additional information such as documents, duration and user roles. I'm not sure why "makePersistent asPersistentReference" is sent there, as WfStep already inherits from WfManagedObject and would, therefore, already be managed by OmniBase. But I don't know OmniBase, maybe some things need to be announced specifically.


- I don’t understand the offsetSeconds in your BLActivationHistoryItem

That's really just some business rule to modify the timestamp of the history item.


============= Others responses from Max

WfWorkflow - Load and Run
All instances that inherit from WfManagedObject were managed by OmniBase, yes.
DomBuilder is a small package for wrapping XML-Parser. I'll publish it so you can use it. The XML-Parser package has been significantly modified and is now named XMLParser (you can find it in the catalog). You should make the changes necessary to use the current incarnation of XMLParser (decide for yourself whether you still need DomBuilder or not).


Those IDXXX classes belong to another package a didn't publish (but will). This package also includes an OmniBase specific class.
Magritte has changed significantly over the years but the core should be pretty much the same. Just load the newest Magritte and try working with that.
> ok but avoidable ?
> try loading Magritte ?   Long …

#beginInHistory should be implemented with #subclassResponsibility on WfActivation
Your explicit implementation could then look something like this:
beginInHistory
self history add: (MyActivationHistoryItem new
message: 'started';
user: (WACurrentSession value 
ifNotNilDo: [ :session | session user ]);
type: #started;
yourself).
MyMailManager noteNewActivation: self

Basically, what the history contains is up to the workflow implementation. I think it would makes sense though to have an abstract WfActivationHistoryItem.
WfActivation>>history should be implemented like this:
history
^self propertyAt: #history ifAbsentPut: [ OrderedCollection new ].

Although I think that it would be good to have a dedicated WfActivationHistory class. WfWorkflowHistory is associated with changes to the workflow, not activations. Both history classes should probably inherit from an abstract class WFAbstractHistory.


Seaside interaction

Here are a couple of action implementations in Seaside components that should give you an idea of how to interact with the workflow:
MyActivationComponent>>complete
self activation hasCompleted 
ifTrue: [ ^ self ].
self history add: (BLActivationHistoryItem new
message: 'completed';
user: self session user;
type: #completed;
offsetSeconds: -1;
yourself).
self activation complete.

MyActivationForm>>accept: aString
self activation authorized: true.
self history add: (BLActivationHistoryItem new
comment: aString;
message: 'accepted';
user: self session user;
type: #accepted;
offsetSeconds: -2;
yourself).
self completeIfSatisfied.

MyActivityList>>addActivity
self call: (MyActivityEditor new
workflow: self workflow;
yourself). 
self refresh.

MyActivityEditor>>buildActivity
^(self workflow newActivityStep 
initializeDefaults;
yourself) 
asPersistentReference.

MyTransitionEditor>>createIncoming
| step | 
step := (self workflow newActivityStep 
initializeDefaults;
yourself) makePersistent asPersistentReference. 
self addIncoming: step.
MyCurrentWorkflowComponent value showActivity: step.

MyTransitionEditor>>addIncoming: anActivity
^self add: anActivity to: self activity.

MyTransitionEditor>>add: aSourceActivity to: aDestinationActivity
^aSourceActivity addOutgoingEdgeFor: aDestinationActivity.

MyTransitionEditor>>editCondtions: anEdge
| component |
anEdge condition isTrueCondition
ifTrue: [ anEdge condition: WfAllCondition new ].
component := (MyConditionEditor activity: self activity)
container: anEdge;
addDecoration: (MyToolbarDecoration new
title: 'Edit Transition Conditions to: ' , anEdge to name;
toolbar: [ :html | 
html 
icon: 'icon_close.gif' 
text: 'Close' 
action: [ component answer ].
html helpIconOn: component ];
yourself);
yourself.
self call: component.

MyConditionEditor>>addRule: aDescription
| new |
new := MyCondition newPersistent
field: aDescription;
asPersistentReference.
new := self call: (self 
buildEditorFor: new 
titled: 'Add Rule for ' , aDescription asLabel).
new isNil ifFalse: [ 
self condition add: new; markDirty.
self changedForm ].


> WfWorflowManager class is empty defining empty stubs/protocols. Its aim seems to be about “logging/noting” changes. Is is it’s only role ?
Correct. You would use WfWorkflowManager to update the workflow in OmniBase (e.g. mark dirty, add new activation etc.). #generateID would generate an ID from OmniBase and #activationClass would answer your specific activation class, if you choose to override WfActivation.

Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

cedreek
Besides all suggested ideas on the previous mail, what’s very important to me is being able to run processes between several images considering 1 image = 1 actor.

At first, we can consider both actor being on the same image (what Stephane wants first) but my requirement number 1 is having 1 actor by image and therefore having the possibility to trigger DistantSubWorkflow.

Cheers,

Cédrick
Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

cedreek
To complement, I found this reference useful (Generating Business Process Models from Object Behavior Models - joined).
The model underlying Workflow-aare ressembles a Heuristic Networks that are explained in this paper.

HTH,

Cédrick





I like this approach where there are incoming signals (announcements ?) compared to Aare.




ootopc.pdf (689K) Download Attachment
Pre Post Gateway of a state.png (93K) Download Attachment
Symbios planif bib 2.png (215K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

Stephane Ducasse-3
In reply to this post by cedreek
Cedrick

My goal is one step at a time.
I do not want grand plan. Just something working in one image.
Stef

On Sun, Jan 21, 2018 at 12:19 PM, Cédrick Béler <[hidden email]> wrote:
> Besides all suggested ideas on the previous mail, what’s very important to me is being able to run processes between several images considering 1 image = 1 actor.
>
> At first, we can consider both actor being on the same image (what Stephane wants first) but my requirement number 1 is having 1 actor by image and therefore having the possibility to trigger DistantSubWorkflow.
>
> Cheers,
>
> Cédrick

Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

SergeStinckwich
In reply to this post by hernanmd


On Sun, Jan 21, 2018 at 9:02 AM, Hernán Morales Durand <[hidden email]> wrote:
Hi,

I decided to give a try to Aare or Workflow in Pharo 6, following
instructions in https://github.com/Netstyle/Workflow . However, the
installation script raised a Warning about missing WADynamicVariable
class. Should I install Seaside?

I've been using a couple of bioinformatics workflows the last years,
specially Taverna (https://en.wikipedia.org/wiki/Apache_Taverna) and
Galaxy (https://www.galaxyproject.org/) both of them huge code bases
maintained since some years now and with very active communities from
biological sciences. I found a cool video showing Galaxy features:
https://www.youtube.com/watch?v=rVYzMWWRMN8

As impressive as you may think, they are not a Panacea for workflow
execution. An installation process could get really complicated, for
example interfacing with nginx and PostgreSQL, and customization of
toolshed could end in endless editing XML specifications.

I just skim through the white paper, but unfortunately didn't
recognized any of the compared features or workflow systems.
Is Aare intended to support only business workflows patterns?
Considering the Galaxy video above, do you think scientific workflows
could be supported in Aare or it is a completely different route?


​Yes this is very interesting.
Even for my work on epidemiological modeling, it make sense
to have such worflows, but I think this is possible to compete with tools
like Galaxy.​

​Regards,​

--
Serge Stinckwich
UMI UMMISCO 209 (IRD/UPMC/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/
Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

SergeStinckwich
In reply to this post by Stephane Ducasse-3


On Sun, Jan 21, 2018 at 3:06 PM, Stephane Ducasse <[hidden email]> wrote:
Cedrick

My goal is one step at a time.
I do not want grand plan. Just something working in one image.

​Yes be able to reproduce/run old cold is the first step.

​There was some discussions some months ago about resu​rrecting
​micro-workflow framework. Nobody try to do it ?​

--
Serge Stinckwich
UMI UMMISCO 209 (IRD/UPMC/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/
Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

Stephane Ducasse-3
In reply to this post by cedreek
HI cedrick


> To solve this « pb », just create WADynamicVariable as a subclass of
> DynamicVariable already in Pharo.

Ok I will do it. What is important is to *share* knowledge and code.
So I changed and publish the code in my branch to make
CurrentWorkflowManager an dynamic variable.
Before I just inherited from Object.
Now we get a version that can be loaded.


There are also missing part for the
> export/import in XML (should be easy to fix and Max propose to give the code
> - DOMBuilder and co.)
> I tried to start a booklet with Stephane but it is too complicated right
> now. Stephane started a simpler version.

For now saving and loading is not important. :)


> I put below a summary of the reflexion/discussion I had with Max.

Thanks.
Now you should publish your code.
I would have preferred to work from a working code. Because now it is
like poking in the dark.

Cedrick what did you get running?

Stef
Some comments below


> ========= INTRO
>
> What's missing compared to the full (proprietary) solution are user
> interfaces that were web based (Seaside \+ Magritte). Also, persistence was
> achived through Omnibase. References were removed but they are stil some
> disfunctionning. Moreover, the completion of activities were mainly manual
> and achieved through web interface (web forms). They are, for now, no real
> way to introduce different kind of activities like those being
> message-based, event-based, service-based.
>
> To evolve, one has to remove all dependancies to Omnibase and
> Magritte/Seaside.
>
> There are also important design decisions to ensure WfWorkflow being a
> powerful and extensible workflow management system for Pharo. To me there
> are 2 main tasks:
> - rethink persistence (process definition and orchestration, realization)
> => persistance is essential and was central to Aare. Beside removing
> Omnibase references (and implication of WfManagedObject), some of the
> application features were very dependant on the persistence (like logging,
> tracing process evolution). All of these features are to be thing again
> considering as much as possible a loosely coupled solution. A general
> purpose storage solution like Voyage could be used. At first, in image
> storage will be used to focus on the running aspects of the library.
> - rethink interaction
> => Designing a general interaction sub-system is probably the more
> challenging tasks. Aare was web based. We want workflow to be UI agnostic.
> Moreover, we want to associate  ,
> remove Magritte (replace UI or build an independant system based on
> Annoucement for instance)


Yes I agree. Now I would like to be able to
- define a super simple workflow: I picked sequence: start -> task1 -> task2
- execute it and script it resolution.

I would like to understand the difference between frame and activation.

> ========= RANDOM QUESTIONS AND ANSWER (in black me - below max)
>
> 1)
>
> The static definition of workflow is done through WfWorkflow, WfOutgoingEdge
> and WfSteps (+ WfConditions). The easy part :)

Yes the easy part.
I started to add a test to cover his

>
> The runtime aspect are achieved via WfActivation, WfWorkList,
> WfWorkflowHistory et WfWorkflowManager. More difficult to understand and
> make evolve.
>
> WfWorkflowHistory represents the different versions of a workflow, so it
> belongs to the static part.

I saw and documented the interaction between a workflow history and
its workflows.


WfWorkflowManager managers workflow definitions
> (static) and activations (dynamic).
> WfFrame is also a part of the dynamic side, connecting activations to a
> workflow.
> Workflows are hierarchical, i.e. a step can have a sub workflow (see
> WfWorkflowLibrary>>simpleSubflow)

Yes for now I do not want to understand this part.

>
> 3)
>
> Going back to activations, here is my current understanding.
>
> Once a step is activated (all condition on Edges leading to are true - to
> evaluate outgoing conditions of a step, the step execution has to be
> complete first) , we get an activation instance (I’m not 100% sure how
> completion of an activity and outgoing conditions are managed).
>
> As far as I understand, a step can always be activated (e.g. manually).
> However, activating a step propagates the values of conditions on the edges
> to the next steps and their activations ("dead" or "alive"). Hence, an
> activation responds to #isAlive with true if at leat one incoming token was
> true ("alive") (see WfStep>>continueInFrame:fromStep:alive: and
> WfActivation>>isAlive). The conditions are evaluated once #complete is sent
> to the activation.
> Activation instances are created by two ways (slightly different for the
> start step), but generally they are created when a step receives tokens from
> the outgoing edges of an upstream activation (see
> WfStep>>continueInFrame:fromStep:alive:).
>
>
> On the exemple of BLActivationHistoryItem that you gave, I’m not sure if
> it’s an activation of a history item stored in the history (subclasss of
> WfHistory?).
>
> My idea was that it should represent a record for an activation (there might
> be multiple records for a single activation). So "HistoryItem" would be the
> general description of a record, of which "ActivationHistoryItem" is a
> special case for activations (I hope that makes sense).
>
>
> 4)
> Moreover I don’t see clearly the difference with a scheduler and a BP engine
> (they are at least really complementary). Do you see any ressemblance
> between Activation and ScheduledJob ?
>
> I actually would like to have a (simple) scheduler with the BP engine :)
>
> Well, it's certainly a matter of point of view. I would argue that an
> activation is something close to a "job". A "scheduled job" would be a more
> specific case of a "job", a job that is in a queue. An activation has no
> notion of scheduling but simply represents the idea of what happens when
> traversal between two nodes occurs. A scheduler would be one way of managing
> activations (WfWorklist seems pretty close to a scheduler, holding running
> and completed activations).


Too subtle for me.


> To me the scheduler right now is not really existing besides having a list
> of simultaneous/parralelized steps to realize (activated or not). Activated
> means currently being done.
>
> You're right, I think. The application that Workflow was designed for
> required manual interaction, hence there was no need for other methods of
> scheduling (e.g. by time or event).
>
>
>
> This activation role is about logging and interacting with responsible
> users. But it is more than that. It’s the actual job being done.
>
> Yes.
>
>
>
> 5)
>
> Concerning run time modification of a process. Is it true that we can modify
> a currently executed workflow (steps that are not yet activated ?) ? I think
> that is possible (an cool). When activated, it shouldn’t. Would you think it
> would be interesting to allow activations to be suspended/resume (frozen),
> cancelled ?
>
> The history of workflows consists of copies. The steps of a workflow
> reference the workflow they belong to. Therefore, while it is possible to
> modify a workflow, modifications do not impact running workflows, as the
> steps of that workflow continue to reference the previous version. The model
> suggests that it is ok to create a new version of a workflow but not to
> modify existing workflows.

For now I would like to get a workflow running without modifying it


> The model does *allow* (in principle) to modify an existing workflow but I
> doubt that was the intention.

I think that it was because else why to bother.

However, I don't see why modifying an existing

> workflow has to be bad thing, as long as you can guarantee that only edges
> and steps are modified that have not yet been activated. That could open up
> some interesting possibilities.
> Cancellation of an activation could be very interesting. It would basically
> allow reversal of complex effects. Very nice idea! I'm not so sure though
> whether suspending makes much sense. Maybe when a step has the potential to
> take a long time for some operation. But then you may run into transaction
> problems. The nice thing about an activation at the moment is that it's
> atomic (in theory). But again, done right that could open up interesting
> possibilities.
>
>
>
> If a running process is modified, do we track its evolutions in
> WfWorkflowManager ?
>
> The modification of a workflow is indeed announced to WfWorkflowManager,
> yes. Workflow versions are stored in WfWorkflowHistory. For modifying a
> *running* workflow you will probably want to create a specific workflow
> manager for that, that does validation etc.
>
>
>
> The goal of history(s) are to log the realization of the process. But is it
> one of its aim to « replay » the process execution ?
>
> No. WfWorkflowHistory only stores the static definitions. However, the
> history of activations could be used to replay an execution (nice idea!).
>
>  i.o.w.,  Is it more than log/report ?
>
> I think the activation history was designed as a log, the workflow history
> as a log and also as a way to undo changes. As you suggest, however, they
> could certainly be used for more things.
>
>
> I agree with the separation (WfWFManager for the run time evolution of the
> process definition => adaptation    and WfActivationHistory for the trace of
> interactive/automatic step realizations information).
>
>
> #beginHistory implementation example was very helpful. I knew I missed
> something  Just, as I see a MailManager (with role is obvious), is it
> something that has to be generalizable ? Like InteroperableManager ?
> MessageManager ? To get user interactions
>
> That would be very nice, yes. You're completely free to do what you like. It
> would be cool though if we could evolve the project as opposed to you
> creating a fork to your own liking. If you're game we can set up a workflow
> on github where you open pull requests against our repository.
>
>
>
> 6) I’d like to remove fully the reference and need of Omnibase and
> Magritte/seaside forms.
>
> Yes please!
>
>
> I think Omnibase is ok, even storing in image for instance.
>
> What’s more problematic to me is for user interactions (and even automatic
> interaction through program for instance). There are achieved in your
> example with seaside. It is ok but what if I’d like to do it more generally
> ? Would we use Annoucement for example ?
>
> I think that is completely open to implementation. In my eyes, Workflow
> should remain a separate library that does not enforce any kind of specific
> presentation or persistence. What's critical for that is of course the right
> kind of interface.
> Announcements would certainly be an interesting way to decouple Workflow
> from other parts of the system.
>
>
> I’d like to have an incomingMessageBox for each user and eventually
> transform these message in concrete action (user i/o) ? Or dispatch in
> JobQueues (each user being a distant worker thread). Again, the distinction
> between Scheduler and BPEngin is fuzzy to me.
>
>
> Does it make sense to you ?
>
> Yes, it does :) What I've asked myself (out of curiosity) is whether you
> want to have
> - a dedicated instance for executing workflows
> - a distributed workflow with autonomous synchronisation (parts of a
> workflow could run on every uses instance, others would need to be executed
> through agreement)
> - a partially distributed workflow with one or more managing nodes that
> control the core steps (e.g. you could designate parts of a workflow as
> independant)
>
>
> PS: some random comments on your proposals
>
> Parts of Workflow were never cleanly separated from the proprietary
> application, so there are indeed a couple of things that are missing (mostly
> #subclassResponsibility stuff etc.)
>
>
>
> - initializeDefaults is not present for steps but thats ok.
>
>
> - in #createIncoming: (doesn’t exist), I don’t get ….. => makePersistent
> asPersistentReference.   You store persistentReference of steps. Is it like
> the step object itself ?  I would store the step objects (as no persistance
> right now).
>
> An activity step is a specific subclass of a step. Such a step includes
> additional information such as documents, duration and user roles. I'm not
> sure why "makePersistent asPersistentReference" is sent there, as WfStep
> already inherits from WfManagedObject and would, therefore, already be
> managed by OmniBase. But I don't know OmniBase, maybe some things need to be
> announced specifically.
>
>
> - I don’t understand the offsetSeconds in your BLActivationHistoryItem
>
> That's really just some business rule to modify the timestamp of the history
> item.
>
>
> ============= Others responses from Max
>
> WfWorkflow - Load and Run
> All instances that inherit from WfManagedObject were managed by OmniBase,
> yes.
> DomBuilder is a small package for wrapping XML-Parser. I'll publish it so
> you can use it.

Cedric did you got this package?


The XML-Parser package has been significantly modified and
> is now named XMLParser (you can find it in the catalog). You should make the
> changes necessary to use the current incarnation of XMLParser (decide for
> yourself whether you still need DomBuilder or not).

But this is not really important.

>
>
> Those IDXXX classes belong to another package a didn't publish (but will).
> This package also includes an OmniBase specific class.
> Magritte has changed significantly over the years but the core should be
> pretty much the same. Just load the newest Magritte and try working with
> that.
>> ok but avoidable ?
>> try loading Magritte ?   Long …
>
> #beginInHistory should be implemented with #subclassResponsibility on
> WfActivation.
> Your explicit implementation could then look something like this:
> beginInHistory
> self history add: (MyActivationHistoryItem new
> message: 'started';
> user: (WACurrentSession value
> ifNotNilDo: [ :session | session user ]);
> type: #started;
> yourself).
> MyMailManager noteNewActivation: self
>
> Basically, what the history contains is up to the workflow implementation. I
> think it would makes sense though to have an abstract
> WfActivationHistoryItem.
> WfActivation>>history should be implemented like this:
> history
> ^self propertyAt: #history ifAbsentPut: [ OrderedCollection new ].
>
> Although I think that it would be good to have a dedicated
> WfActivationHistory class. WfWorkflowHistory is associated with changes to
> the workflow, not activations. Both history classes should probably inherit
> from an abstract class WFAbstractHistory.
>
>
> Seaside interaction
>
> Here are a couple of action implementations in Seaside components that
> should give you an idea of how to interact with the workflow:
> MyActivationComponent>>complete
> self activation hasCompleted
> ifTrue: [ ^ self ].
> self history add: (BLActivationHistoryItem new
> message: 'completed';
> user: self session user;
> type: #completed;
> offsetSeconds: -1;
> yourself).
> self activation complete.
>
> MyActivationForm>>accept: aString
> self activation authorized: true.
> self history add: (BLActivationHistoryItem new
> comment: aString;
> message: 'accepted';
> user: self session user;
> type: #accepted;
> offsetSeconds: -2;
> yourself).
> self completeIfSatisfied.
>
> MyActivityList>>addActivity
> self call: (MyActivityEditor new
> workflow: self workflow;
> yourself).
> self refresh.
>
> MyActivityEditor>>buildActivity
> ^(self workflow newActivityStep
> initializeDefaults;
> yourself)
> asPersistentReference.
>
> MyTransitionEditor>>createIncoming
> | step |
> step := (self workflow newActivityStep
> initializeDefaults;
> yourself) makePersistent asPersistentReference.
> self addIncoming: step.
> MyCurrentWorkflowComponent value showActivity: step.
>
> MyTransitionEditor>>addIncoming: anActivity
> ^self add: anActivity to: self activity.
>
> MyTransitionEditor>>add: aSourceActivity to: aDestinationActivity
> ^aSourceActivity addOutgoingEdgeFor: aDestinationActivity.
>
> MyTransitionEditor>>editCondtions: anEdge
> | component |
> anEdge condition isTrueCondition
> ifTrue: [ anEdge condition: WfAllCondition new ].
> component := (MyConditionEditor activity: self activity)
> container: anEdge;
> addDecoration: (MyToolbarDecoration new
> title: 'Edit Transition Conditions to: ' , anEdge to name;
> toolbar: [ :html |
> html
> icon: 'icon_close.gif'
> text: 'Close'
> action: [ component answer ].
> html helpIconOn: component ];
> yourself);
> yourself.
> self call: component.
>
> MyConditionEditor>>addRule: aDescription
> | new |
> new := MyCondition newPersistent
> field: aDescription;
> asPersistentReference.
> new := self call: (self
> buildEditorFor: new
> titled: 'Add Rule for ' , aDescription asLabel).
> new isNil ifFalse: [
> self condition add: new; markDirty.
> self changedForm ].
>
>
>> WfWorflowManager class is empty defining empty stubs/protocols. Its aim
>> seems to be about “logging/noting” changes. Is is it’s only role ?
> Correct. You would use WfWorkflowManager to update the workflow in OmniBase
> (e.g. mark dirty, add new activation etc.). #generateID would generate an ID
> from OmniBase and #activationClass would answer your specific activation
> class, if you choose to override WfActivation.
>

Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

Stephane Ducasse-3
I got all the WfTestFlow green.
So now we can study the execution a bit better.

On Sun, Jan 21, 2018 at 3:28 PM, Stephane Ducasse
<[hidden email]> wrote:

> HI cedrick
>
>
>> To solve this « pb », just create WADynamicVariable as a subclass of
>> DynamicVariable already in Pharo.
>
> Ok I will do it. What is important is to *share* knowledge and code.
> So I changed and publish the code in my branch to make
> CurrentWorkflowManager an dynamic variable.
> Before I just inherited from Object.
> Now we get a version that can be loaded.
>
>
> There are also missing part for the
>> export/import in XML (should be easy to fix and Max propose to give the code
>> - DOMBuilder and co.)
>> I tried to start a booklet with Stephane but it is too complicated right
>> now. Stephane started a simpler version.
>
> For now saving and loading is not important. :)
>
>
>> I put below a summary of the reflexion/discussion I had with Max.
>
> Thanks.
> Now you should publish your code.
> I would have preferred to work from a working code. Because now it is
> like poking in the dark.
>
> Cedrick what did you get running?
>
> Stef
> Some comments below
>
>
>> ========= INTRO
>>
>> What's missing compared to the full (proprietary) solution are user
>> interfaces that were web based (Seaside \+ Magritte). Also, persistence was
>> achived through Omnibase. References were removed but they are stil some
>> disfunctionning. Moreover, the completion of activities were mainly manual
>> and achieved through web interface (web forms). They are, for now, no real
>> way to introduce different kind of activities like those being
>> message-based, event-based, service-based.
>>
>> To evolve, one has to remove all dependancies to Omnibase and
>> Magritte/Seaside.
>>
>> There are also important design decisions to ensure WfWorkflow being a
>> powerful and extensible workflow management system for Pharo. To me there
>> are 2 main tasks:
>> - rethink persistence (process definition and orchestration, realization)
>> => persistance is essential and was central to Aare. Beside removing
>> Omnibase references (and implication of WfManagedObject), some of the
>> application features were very dependant on the persistence (like logging,
>> tracing process evolution). All of these features are to be thing again
>> considering as much as possible a loosely coupled solution. A general
>> purpose storage solution like Voyage could be used. At first, in image
>> storage will be used to focus on the running aspects of the library.
>> - rethink interaction
>> => Designing a general interaction sub-system is probably the more
>> challenging tasks. Aare was web based. We want workflow to be UI agnostic.
>> Moreover, we want to associate  ,
>> remove Magritte (replace UI or build an independant system based on
>> Annoucement for instance)
>
>
> Yes I agree. Now I would like to be able to
> - define a super simple workflow: I picked sequence: start -> task1 -> task2
> - execute it and script it resolution.
>
> I would like to understand the difference between frame and activation.
>
>> ========= RANDOM QUESTIONS AND ANSWER (in black me - below max)
>>
>> 1)
>>
>> The static definition of workflow is done through WfWorkflow, WfOutgoingEdge
>> and WfSteps (+ WfConditions). The easy part :)
>
> Yes the easy part.
> I started to add a test to cover his
>
>>
>> The runtime aspect are achieved via WfActivation, WfWorkList,
>> WfWorkflowHistory et WfWorkflowManager. More difficult to understand and
>> make evolve.
>>
>> WfWorkflowHistory represents the different versions of a workflow, so it
>> belongs to the static part.
>
> I saw and documented the interaction between a workflow history and
> its workflows.
>
>
> WfWorkflowManager managers workflow definitions
>> (static) and activations (dynamic).
>> WfFrame is also a part of the dynamic side, connecting activations to a
>> workflow.
>> Workflows are hierarchical, i.e. a step can have a sub workflow (see
>> WfWorkflowLibrary>>simpleSubflow)
>
> Yes for now I do not want to understand this part.
>
>>
>> 3)
>>
>> Going back to activations, here is my current understanding.
>>
>> Once a step is activated (all condition on Edges leading to are true - to
>> evaluate outgoing conditions of a step, the step execution has to be
>> complete first) , we get an activation instance (I’m not 100% sure how
>> completion of an activity and outgoing conditions are managed).
>>
>> As far as I understand, a step can always be activated (e.g. manually).
>> However, activating a step propagates the values of conditions on the edges
>> to the next steps and their activations ("dead" or "alive"). Hence, an
>> activation responds to #isAlive with true if at leat one incoming token was
>> true ("alive") (see WfStep>>continueInFrame:fromStep:alive: and
>> WfActivation>>isAlive). The conditions are evaluated once #complete is sent
>> to the activation.
>> Activation instances are created by two ways (slightly different for the
>> start step), but generally they are created when a step receives tokens from
>> the outgoing edges of an upstream activation (see
>> WfStep>>continueInFrame:fromStep:alive:).
>>
>>
>> On the exemple of BLActivationHistoryItem that you gave, I’m not sure if
>> it’s an activation of a history item stored in the history (subclasss of
>> WfHistory?).
>>
>> My idea was that it should represent a record for an activation (there might
>> be multiple records for a single activation). So "HistoryItem" would be the
>> general description of a record, of which "ActivationHistoryItem" is a
>> special case for activations (I hope that makes sense).
>>
>>
>> 4)
>> Moreover I don’t see clearly the difference with a scheduler and a BP engine
>> (they are at least really complementary). Do you see any ressemblance
>> between Activation and ScheduledJob ?
>>
>> I actually would like to have a (simple) scheduler with the BP engine :)
>>
>> Well, it's certainly a matter of point of view. I would argue that an
>> activation is something close to a "job". A "scheduled job" would be a more
>> specific case of a "job", a job that is in a queue. An activation has no
>> notion of scheduling but simply represents the idea of what happens when
>> traversal between two nodes occurs. A scheduler would be one way of managing
>> activations (WfWorklist seems pretty close to a scheduler, holding running
>> and completed activations).
>
>
> Too subtle for me.
>
>
>> To me the scheduler right now is not really existing besides having a list
>> of simultaneous/parralelized steps to realize (activated or not). Activated
>> means currently being done.
>>
>> You're right, I think. The application that Workflow was designed for
>> required manual interaction, hence there was no need for other methods of
>> scheduling (e.g. by time or event).
>>
>>
>>
>> This activation role is about logging and interacting with responsible
>> users. But it is more than that. It’s the actual job being done.
>>
>> Yes.
>>
>>
>>
>> 5)
>>
>> Concerning run time modification of a process. Is it true that we can modify
>> a currently executed workflow (steps that are not yet activated ?) ? I think
>> that is possible (an cool). When activated, it shouldn’t. Would you think it
>> would be interesting to allow activations to be suspended/resume (frozen),
>> cancelled ?
>>
>> The history of workflows consists of copies. The steps of a workflow
>> reference the workflow they belong to. Therefore, while it is possible to
>> modify a workflow, modifications do not impact running workflows, as the
>> steps of that workflow continue to reference the previous version. The model
>> suggests that it is ok to create a new version of a workflow but not to
>> modify existing workflows.
>
> For now I would like to get a workflow running without modifying it
>
>
>> The model does *allow* (in principle) to modify an existing workflow but I
>> doubt that was the intention.
>
> I think that it was because else why to bother.
>
> However, I don't see why modifying an existing
>> workflow has to be bad thing, as long as you can guarantee that only edges
>> and steps are modified that have not yet been activated. That could open up
>> some interesting possibilities.
>> Cancellation of an activation could be very interesting. It would basically
>> allow reversal of complex effects. Very nice idea! I'm not so sure though
>> whether suspending makes much sense. Maybe when a step has the potential to
>> take a long time for some operation. But then you may run into transaction
>> problems. The nice thing about an activation at the moment is that it's
>> atomic (in theory). But again, done right that could open up interesting
>> possibilities.
>>
>>
>>
>> If a running process is modified, do we track its evolutions in
>> WfWorkflowManager ?
>>
>> The modification of a workflow is indeed announced to WfWorkflowManager,
>> yes. Workflow versions are stored in WfWorkflowHistory. For modifying a
>> *running* workflow you will probably want to create a specific workflow
>> manager for that, that does validation etc.
>>
>>
>>
>> The goal of history(s) are to log the realization of the process. But is it
>> one of its aim to « replay » the process execution ?
>>
>> No. WfWorkflowHistory only stores the static definitions. However, the
>> history of activations could be used to replay an execution (nice idea!).
>>
>>  i.o.w.,  Is it more than log/report ?
>>
>> I think the activation history was designed as a log, the workflow history
>> as a log and also as a way to undo changes. As you suggest, however, they
>> could certainly be used for more things.
>>
>>
>> I agree with the separation (WfWFManager for the run time evolution of the
>> process definition => adaptation    and WfActivationHistory for the trace of
>> interactive/automatic step realizations information).
>>
>>
>> #beginHistory implementation example was very helpful. I knew I missed
>> something  Just, as I see a MailManager (with role is obvious), is it
>> something that has to be generalizable ? Like InteroperableManager ?
>> MessageManager ? To get user interactions
>>
>> That would be very nice, yes. You're completely free to do what you like. It
>> would be cool though if we could evolve the project as opposed to you
>> creating a fork to your own liking. If you're game we can set up a workflow
>> on github where you open pull requests against our repository.
>>
>>
>>
>> 6) I’d like to remove fully the reference and need of Omnibase and
>> Magritte/seaside forms.
>>
>> Yes please!
>>
>>
>> I think Omnibase is ok, even storing in image for instance.
>>
>> What’s more problematic to me is for user interactions (and even automatic
>> interaction through program for instance). There are achieved in your
>> example with seaside. It is ok but what if I’d like to do it more generally
>> ? Would we use Annoucement for example ?
>>
>> I think that is completely open to implementation. In my eyes, Workflow
>> should remain a separate library that does not enforce any kind of specific
>> presentation or persistence. What's critical for that is of course the right
>> kind of interface.
>> Announcements would certainly be an interesting way to decouple Workflow
>> from other parts of the system.
>>
>>
>> I’d like to have an incomingMessageBox for each user and eventually
>> transform these message in concrete action (user i/o) ? Or dispatch in
>> JobQueues (each user being a distant worker thread). Again, the distinction
>> between Scheduler and BPEngin is fuzzy to me.
>>
>>
>> Does it make sense to you ?
>>
>> Yes, it does :) What I've asked myself (out of curiosity) is whether you
>> want to have
>> - a dedicated instance for executing workflows
>> - a distributed workflow with autonomous synchronisation (parts of a
>> workflow could run on every uses instance, others would need to be executed
>> through agreement)
>> - a partially distributed workflow with one or more managing nodes that
>> control the core steps (e.g. you could designate parts of a workflow as
>> independant)
>>
>>
>> PS: some random comments on your proposals
>>
>> Parts of Workflow were never cleanly separated from the proprietary
>> application, so there are indeed a couple of things that are missing (mostly
>> #subclassResponsibility stuff etc.)
>>
>>
>>
>> - initializeDefaults is not present for steps but thats ok.
>>
>>
>> - in #createIncoming: (doesn’t exist), I don’t get ….. => makePersistent
>> asPersistentReference.   You store persistentReference of steps. Is it like
>> the step object itself ?  I would store the step objects (as no persistance
>> right now).
>>
>> An activity step is a specific subclass of a step. Such a step includes
>> additional information such as documents, duration and user roles. I'm not
>> sure why "makePersistent asPersistentReference" is sent there, as WfStep
>> already inherits from WfManagedObject and would, therefore, already be
>> managed by OmniBase. But I don't know OmniBase, maybe some things need to be
>> announced specifically.
>>
>>
>> - I don’t understand the offsetSeconds in your BLActivationHistoryItem
>>
>> That's really just some business rule to modify the timestamp of the history
>> item.
>>
>>
>> ============= Others responses from Max
>>
>> WfWorkflow - Load and Run
>> All instances that inherit from WfManagedObject were managed by OmniBase,
>> yes.
>> DomBuilder is a small package for wrapping XML-Parser. I'll publish it so
>> you can use it.
>
> Cedric did you got this package?
>
>
> The XML-Parser package has been significantly modified and
>> is now named XMLParser (you can find it in the catalog). You should make the
>> changes necessary to use the current incarnation of XMLParser (decide for
>> yourself whether you still need DomBuilder or not).
>
> But this is not really important.
>
>>
>>
>> Those IDXXX classes belong to another package a didn't publish (but will).
>> This package also includes an OmniBase specific class.
>> Magritte has changed significantly over the years but the core should be
>> pretty much the same. Just load the newest Magritte and try working with
>> that.
>>> ok but avoidable ?
>>> try loading Magritte ?   Long …
>>
>> #beginInHistory should be implemented with #subclassResponsibility on
>> WfActivation.
>> Your explicit implementation could then look something like this:
>> beginInHistory
>> self history add: (MyActivationHistoryItem new
>> message: 'started';
>> user: (WACurrentSession value
>> ifNotNilDo: [ :session | session user ]);
>> type: #started;
>> yourself).
>> MyMailManager noteNewActivation: self
>>
>> Basically, what the history contains is up to the workflow implementation. I
>> think it would makes sense though to have an abstract
>> WfActivationHistoryItem.
>> WfActivation>>history should be implemented like this:
>> history
>> ^self propertyAt: #history ifAbsentPut: [ OrderedCollection new ].
>>
>> Although I think that it would be good to have a dedicated
>> WfActivationHistory class. WfWorkflowHistory is associated with changes to
>> the workflow, not activations. Both history classes should probably inherit
>> from an abstract class WFAbstractHistory.
>>
>>
>> Seaside interaction
>>
>> Here are a couple of action implementations in Seaside components that
>> should give you an idea of how to interact with the workflow:
>> MyActivationComponent>>complete
>> self activation hasCompleted
>> ifTrue: [ ^ self ].
>> self history add: (BLActivationHistoryItem new
>> message: 'completed';
>> user: self session user;
>> type: #completed;
>> offsetSeconds: -1;
>> yourself).
>> self activation complete.
>>
>> MyActivationForm>>accept: aString
>> self activation authorized: true.
>> self history add: (BLActivationHistoryItem new
>> comment: aString;
>> message: 'accepted';
>> user: self session user;
>> type: #accepted;
>> offsetSeconds: -2;
>> yourself).
>> self completeIfSatisfied.
>>
>> MyActivityList>>addActivity
>> self call: (MyActivityEditor new
>> workflow: self workflow;
>> yourself).
>> self refresh.
>>
>> MyActivityEditor>>buildActivity
>> ^(self workflow newActivityStep
>> initializeDefaults;
>> yourself)
>> asPersistentReference.
>>
>> MyTransitionEditor>>createIncoming
>> | step |
>> step := (self workflow newActivityStep
>> initializeDefaults;
>> yourself) makePersistent asPersistentReference.
>> self addIncoming: step.
>> MyCurrentWorkflowComponent value showActivity: step.
>>
>> MyTransitionEditor>>addIncoming: anActivity
>> ^self add: anActivity to: self activity.
>>
>> MyTransitionEditor>>add: aSourceActivity to: aDestinationActivity
>> ^aSourceActivity addOutgoingEdgeFor: aDestinationActivity.
>>
>> MyTransitionEditor>>editCondtions: anEdge
>> | component |
>> anEdge condition isTrueCondition
>> ifTrue: [ anEdge condition: WfAllCondition new ].
>> component := (MyConditionEditor activity: self activity)
>> container: anEdge;
>> addDecoration: (MyToolbarDecoration new
>> title: 'Edit Transition Conditions to: ' , anEdge to name;
>> toolbar: [ :html |
>> html
>> icon: 'icon_close.gif'
>> text: 'Close'
>> action: [ component answer ].
>> html helpIconOn: component ];
>> yourself);
>> yourself.
>> self call: component.
>>
>> MyConditionEditor>>addRule: aDescription
>> | new |
>> new := MyCondition newPersistent
>> field: aDescription;
>> asPersistentReference.
>> new := self call: (self
>> buildEditorFor: new
>> titled: 'Add Rule for ' , aDescription asLabel).
>> new isNil ifFalse: [
>> self condition add: new; markDirty.
>> self changedForm ].
>>
>>
>>> WfWorflowManager class is empty defining empty stubs/protocols. Its aim
>>> seems to be about “logging/noting” changes. Is is it’s only role ?
>> Correct. You would use WfWorkflowManager to update the workflow in OmniBase
>> (e.g. mark dirty, add new activation etc.). #generateID would generate an ID
>> from OmniBase and #activationClass would answer your specific activation
>> class, if you choose to override WfActivation.
>>

Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

Stephane Ducasse-3
Now I got all the TestGraphs green by commenting the graphiz dependencies.

On Sun, Jan 21, 2018 at 3:33 PM, Stephane Ducasse
<[hidden email]> wrote:

> I got all the WfTestFlow green.
> So now we can study the execution a bit better.
>
> On Sun, Jan 21, 2018 at 3:28 PM, Stephane Ducasse
> <[hidden email]> wrote:
>> HI cedrick
>>
>>
>>> To solve this « pb », just create WADynamicVariable as a subclass of
>>> DynamicVariable already in Pharo.
>>
>> Ok I will do it. What is important is to *share* knowledge and code.
>> So I changed and publish the code in my branch to make
>> CurrentWorkflowManager an dynamic variable.
>> Before I just inherited from Object.
>> Now we get a version that can be loaded.
>>
>>
>> There are also missing part for the
>>> export/import in XML (should be easy to fix and Max propose to give the code
>>> - DOMBuilder and co.)
>>> I tried to start a booklet with Stephane but it is too complicated right
>>> now. Stephane started a simpler version.
>>
>> For now saving and loading is not important. :)
>>
>>
>>> I put below a summary of the reflexion/discussion I had with Max.
>>
>> Thanks.
>> Now you should publish your code.
>> I would have preferred to work from a working code. Because now it is
>> like poking in the dark.
>>
>> Cedrick what did you get running?
>>
>> Stef
>> Some comments below
>>
>>
>>> ========= INTRO
>>>
>>> What's missing compared to the full (proprietary) solution are user
>>> interfaces that were web based (Seaside \+ Magritte). Also, persistence was
>>> achived through Omnibase. References were removed but they are stil some
>>> disfunctionning. Moreover, the completion of activities were mainly manual
>>> and achieved through web interface (web forms). They are, for now, no real
>>> way to introduce different kind of activities like those being
>>> message-based, event-based, service-based.
>>>
>>> To evolve, one has to remove all dependancies to Omnibase and
>>> Magritte/Seaside.
>>>
>>> There are also important design decisions to ensure WfWorkflow being a
>>> powerful and extensible workflow management system for Pharo. To me there
>>> are 2 main tasks:
>>> - rethink persistence (process definition and orchestration, realization)
>>> => persistance is essential and was central to Aare. Beside removing
>>> Omnibase references (and implication of WfManagedObject), some of the
>>> application features were very dependant on the persistence (like logging,
>>> tracing process evolution). All of these features are to be thing again
>>> considering as much as possible a loosely coupled solution. A general
>>> purpose storage solution like Voyage could be used. At first, in image
>>> storage will be used to focus on the running aspects of the library.
>>> - rethink interaction
>>> => Designing a general interaction sub-system is probably the more
>>> challenging tasks. Aare was web based. We want workflow to be UI agnostic.
>>> Moreover, we want to associate  ,
>>> remove Magritte (replace UI or build an independant system based on
>>> Annoucement for instance)
>>
>>
>> Yes I agree. Now I would like to be able to
>> - define a super simple workflow: I picked sequence: start -> task1 -> task2
>> - execute it and script it resolution.
>>
>> I would like to understand the difference between frame and activation.
>>
>>> ========= RANDOM QUESTIONS AND ANSWER (in black me - below max)
>>>
>>> 1)
>>>
>>> The static definition of workflow is done through WfWorkflow, WfOutgoingEdge
>>> and WfSteps (+ WfConditions). The easy part :)
>>
>> Yes the easy part.
>> I started to add a test to cover his
>>
>>>
>>> The runtime aspect are achieved via WfActivation, WfWorkList,
>>> WfWorkflowHistory et WfWorkflowManager. More difficult to understand and
>>> make evolve.
>>>
>>> WfWorkflowHistory represents the different versions of a workflow, so it
>>> belongs to the static part.
>>
>> I saw and documented the interaction between a workflow history and
>> its workflows.
>>
>>
>> WfWorkflowManager managers workflow definitions
>>> (static) and activations (dynamic).
>>> WfFrame is also a part of the dynamic side, connecting activations to a
>>> workflow.
>>> Workflows are hierarchical, i.e. a step can have a sub workflow (see
>>> WfWorkflowLibrary>>simpleSubflow)
>>
>> Yes for now I do not want to understand this part.
>>
>>>
>>> 3)
>>>
>>> Going back to activations, here is my current understanding.
>>>
>>> Once a step is activated (all condition on Edges leading to are true - to
>>> evaluate outgoing conditions of a step, the step execution has to be
>>> complete first) , we get an activation instance (I’m not 100% sure how
>>> completion of an activity and outgoing conditions are managed).
>>>
>>> As far as I understand, a step can always be activated (e.g. manually).
>>> However, activating a step propagates the values of conditions on the edges
>>> to the next steps and their activations ("dead" or "alive"). Hence, an
>>> activation responds to #isAlive with true if at leat one incoming token was
>>> true ("alive") (see WfStep>>continueInFrame:fromStep:alive: and
>>> WfActivation>>isAlive). The conditions are evaluated once #complete is sent
>>> to the activation.
>>> Activation instances are created by two ways (slightly different for the
>>> start step), but generally they are created when a step receives tokens from
>>> the outgoing edges of an upstream activation (see
>>> WfStep>>continueInFrame:fromStep:alive:).
>>>
>>>
>>> On the exemple of BLActivationHistoryItem that you gave, I’m not sure if
>>> it’s an activation of a history item stored in the history (subclasss of
>>> WfHistory?).
>>>
>>> My idea was that it should represent a record for an activation (there might
>>> be multiple records for a single activation). So "HistoryItem" would be the
>>> general description of a record, of which "ActivationHistoryItem" is a
>>> special case for activations (I hope that makes sense).
>>>
>>>
>>> 4)
>>> Moreover I don’t see clearly the difference with a scheduler and a BP engine
>>> (they are at least really complementary). Do you see any ressemblance
>>> between Activation and ScheduledJob ?
>>>
>>> I actually would like to have a (simple) scheduler with the BP engine :)
>>>
>>> Well, it's certainly a matter of point of view. I would argue that an
>>> activation is something close to a "job". A "scheduled job" would be a more
>>> specific case of a "job", a job that is in a queue. An activation has no
>>> notion of scheduling but simply represents the idea of what happens when
>>> traversal between two nodes occurs. A scheduler would be one way of managing
>>> activations (WfWorklist seems pretty close to a scheduler, holding running
>>> and completed activations).
>>
>>
>> Too subtle for me.
>>
>>
>>> To me the scheduler right now is not really existing besides having a list
>>> of simultaneous/parralelized steps to realize (activated or not). Activated
>>> means currently being done.
>>>
>>> You're right, I think. The application that Workflow was designed for
>>> required manual interaction, hence there was no need for other methods of
>>> scheduling (e.g. by time or event).
>>>
>>>
>>>
>>> This activation role is about logging and interacting with responsible
>>> users. But it is more than that. It’s the actual job being done.
>>>
>>> Yes.
>>>
>>>
>>>
>>> 5)
>>>
>>> Concerning run time modification of a process. Is it true that we can modify
>>> a currently executed workflow (steps that are not yet activated ?) ? I think
>>> that is possible (an cool). When activated, it shouldn’t. Would you think it
>>> would be interesting to allow activations to be suspended/resume (frozen),
>>> cancelled ?
>>>
>>> The history of workflows consists of copies. The steps of a workflow
>>> reference the workflow they belong to. Therefore, while it is possible to
>>> modify a workflow, modifications do not impact running workflows, as the
>>> steps of that workflow continue to reference the previous version. The model
>>> suggests that it is ok to create a new version of a workflow but not to
>>> modify existing workflows.
>>
>> For now I would like to get a workflow running without modifying it
>>
>>
>>> The model does *allow* (in principle) to modify an existing workflow but I
>>> doubt that was the intention.
>>
>> I think that it was because else why to bother.
>>
>> However, I don't see why modifying an existing
>>> workflow has to be bad thing, as long as you can guarantee that only edges
>>> and steps are modified that have not yet been activated. That could open up
>>> some interesting possibilities.
>>> Cancellation of an activation could be very interesting. It would basically
>>> allow reversal of complex effects. Very nice idea! I'm not so sure though
>>> whether suspending makes much sense. Maybe when a step has the potential to
>>> take a long time for some operation. But then you may run into transaction
>>> problems. The nice thing about an activation at the moment is that it's
>>> atomic (in theory). But again, done right that could open up interesting
>>> possibilities.
>>>
>>>
>>>
>>> If a running process is modified, do we track its evolutions in
>>> WfWorkflowManager ?
>>>
>>> The modification of a workflow is indeed announced to WfWorkflowManager,
>>> yes. Workflow versions are stored in WfWorkflowHistory. For modifying a
>>> *running* workflow you will probably want to create a specific workflow
>>> manager for that, that does validation etc.
>>>
>>>
>>>
>>> The goal of history(s) are to log the realization of the process. But is it
>>> one of its aim to « replay » the process execution ?
>>>
>>> No. WfWorkflowHistory only stores the static definitions. However, the
>>> history of activations could be used to replay an execution (nice idea!).
>>>
>>>  i.o.w.,  Is it more than log/report ?
>>>
>>> I think the activation history was designed as a log, the workflow history
>>> as a log and also as a way to undo changes. As you suggest, however, they
>>> could certainly be used for more things.
>>>
>>>
>>> I agree with the separation (WfWFManager for the run time evolution of the
>>> process definition => adaptation    and WfActivationHistory for the trace of
>>> interactive/automatic step realizations information).
>>>
>>>
>>> #beginHistory implementation example was very helpful. I knew I missed
>>> something  Just, as I see a MailManager (with role is obvious), is it
>>> something that has to be generalizable ? Like InteroperableManager ?
>>> MessageManager ? To get user interactions
>>>
>>> That would be very nice, yes. You're completely free to do what you like. It
>>> would be cool though if we could evolve the project as opposed to you
>>> creating a fork to your own liking. If you're game we can set up a workflow
>>> on github where you open pull requests against our repository.
>>>
>>>
>>>
>>> 6) I’d like to remove fully the reference and need of Omnibase and
>>> Magritte/seaside forms.
>>>
>>> Yes please!
>>>
>>>
>>> I think Omnibase is ok, even storing in image for instance.
>>>
>>> What’s more problematic to me is for user interactions (and even automatic
>>> interaction through program for instance). There are achieved in your
>>> example with seaside. It is ok but what if I’d like to do it more generally
>>> ? Would we use Annoucement for example ?
>>>
>>> I think that is completely open to implementation. In my eyes, Workflow
>>> should remain a separate library that does not enforce any kind of specific
>>> presentation or persistence. What's critical for that is of course the right
>>> kind of interface.
>>> Announcements would certainly be an interesting way to decouple Workflow
>>> from other parts of the system.
>>>
>>>
>>> I’d like to have an incomingMessageBox for each user and eventually
>>> transform these message in concrete action (user i/o) ? Or dispatch in
>>> JobQueues (each user being a distant worker thread). Again, the distinction
>>> between Scheduler and BPEngin is fuzzy to me.
>>>
>>>
>>> Does it make sense to you ?
>>>
>>> Yes, it does :) What I've asked myself (out of curiosity) is whether you
>>> want to have
>>> - a dedicated instance for executing workflows
>>> - a distributed workflow with autonomous synchronisation (parts of a
>>> workflow could run on every uses instance, others would need to be executed
>>> through agreement)
>>> - a partially distributed workflow with one or more managing nodes that
>>> control the core steps (e.g. you could designate parts of a workflow as
>>> independant)
>>>
>>>
>>> PS: some random comments on your proposals
>>>
>>> Parts of Workflow were never cleanly separated from the proprietary
>>> application, so there are indeed a couple of things that are missing (mostly
>>> #subclassResponsibility stuff etc.)
>>>
>>>
>>>
>>> - initializeDefaults is not present for steps but thats ok.
>>>
>>>
>>> - in #createIncoming: (doesn’t exist), I don’t get ….. => makePersistent
>>> asPersistentReference.   You store persistentReference of steps. Is it like
>>> the step object itself ?  I would store the step objects (as no persistance
>>> right now).
>>>
>>> An activity step is a specific subclass of a step. Such a step includes
>>> additional information such as documents, duration and user roles. I'm not
>>> sure why "makePersistent asPersistentReference" is sent there, as WfStep
>>> already inherits from WfManagedObject and would, therefore, already be
>>> managed by OmniBase. But I don't know OmniBase, maybe some things need to be
>>> announced specifically.
>>>
>>>
>>> - I don’t understand the offsetSeconds in your BLActivationHistoryItem
>>>
>>> That's really just some business rule to modify the timestamp of the history
>>> item.
>>>
>>>
>>> ============= Others responses from Max
>>>
>>> WfWorkflow - Load and Run
>>> All instances that inherit from WfManagedObject were managed by OmniBase,
>>> yes.
>>> DomBuilder is a small package for wrapping XML-Parser. I'll publish it so
>>> you can use it.
>>
>> Cedric did you got this package?
>>
>>
>> The XML-Parser package has been significantly modified and
>>> is now named XMLParser (you can find it in the catalog). You should make the
>>> changes necessary to use the current incarnation of XMLParser (decide for
>>> yourself whether you still need DomBuilder or not).
>>
>> But this is not really important.
>>
>>>
>>>
>>> Those IDXXX classes belong to another package a didn't publish (but will).
>>> This package also includes an OmniBase specific class.
>>> Magritte has changed significantly over the years but the core should be
>>> pretty much the same. Just load the newest Magritte and try working with
>>> that.
>>>> ok but avoidable ?
>>>> try loading Magritte ?   Long …
>>>
>>> #beginInHistory should be implemented with #subclassResponsibility on
>>> WfActivation.
>>> Your explicit implementation could then look something like this:
>>> beginInHistory
>>> self history add: (MyActivationHistoryItem new
>>> message: 'started';
>>> user: (WACurrentSession value
>>> ifNotNilDo: [ :session | session user ]);
>>> type: #started;
>>> yourself).
>>> MyMailManager noteNewActivation: self
>>>
>>> Basically, what the history contains is up to the workflow implementation. I
>>> think it would makes sense though to have an abstract
>>> WfActivationHistoryItem.
>>> WfActivation>>history should be implemented like this:
>>> history
>>> ^self propertyAt: #history ifAbsentPut: [ OrderedCollection new ].
>>>
>>> Although I think that it would be good to have a dedicated
>>> WfActivationHistory class. WfWorkflowHistory is associated with changes to
>>> the workflow, not activations. Both history classes should probably inherit
>>> from an abstract class WFAbstractHistory.
>>>
>>>
>>> Seaside interaction
>>>
>>> Here are a couple of action implementations in Seaside components that
>>> should give you an idea of how to interact with the workflow:
>>> MyActivationComponent>>complete
>>> self activation hasCompleted
>>> ifTrue: [ ^ self ].
>>> self history add: (BLActivationHistoryItem new
>>> message: 'completed';
>>> user: self session user;
>>> type: #completed;
>>> offsetSeconds: -1;
>>> yourself).
>>> self activation complete.
>>>
>>> MyActivationForm>>accept: aString
>>> self activation authorized: true.
>>> self history add: (BLActivationHistoryItem new
>>> comment: aString;
>>> message: 'accepted';
>>> user: self session user;
>>> type: #accepted;
>>> offsetSeconds: -2;
>>> yourself).
>>> self completeIfSatisfied.
>>>
>>> MyActivityList>>addActivity
>>> self call: (MyActivityEditor new
>>> workflow: self workflow;
>>> yourself).
>>> self refresh.
>>>
>>> MyActivityEditor>>buildActivity
>>> ^(self workflow newActivityStep
>>> initializeDefaults;
>>> yourself)
>>> asPersistentReference.
>>>
>>> MyTransitionEditor>>createIncoming
>>> | step |
>>> step := (self workflow newActivityStep
>>> initializeDefaults;
>>> yourself) makePersistent asPersistentReference.
>>> self addIncoming: step.
>>> MyCurrentWorkflowComponent value showActivity: step.
>>>
>>> MyTransitionEditor>>addIncoming: anActivity
>>> ^self add: anActivity to: self activity.
>>>
>>> MyTransitionEditor>>add: aSourceActivity to: aDestinationActivity
>>> ^aSourceActivity addOutgoingEdgeFor: aDestinationActivity.
>>>
>>> MyTransitionEditor>>editCondtions: anEdge
>>> | component |
>>> anEdge condition isTrueCondition
>>> ifTrue: [ anEdge condition: WfAllCondition new ].
>>> component := (MyConditionEditor activity: self activity)
>>> container: anEdge;
>>> addDecoration: (MyToolbarDecoration new
>>> title: 'Edit Transition Conditions to: ' , anEdge to name;
>>> toolbar: [ :html |
>>> html
>>> icon: 'icon_close.gif'
>>> text: 'Close'
>>> action: [ component answer ].
>>> html helpIconOn: component ];
>>> yourself);
>>> yourself.
>>> self call: component.
>>>
>>> MyConditionEditor>>addRule: aDescription
>>> | new |
>>> new := MyCondition newPersistent
>>> field: aDescription;
>>> asPersistentReference.
>>> new := self call: (self
>>> buildEditorFor: new
>>> titled: 'Add Rule for ' , aDescription asLabel).
>>> new isNil ifFalse: [
>>> self condition add: new; markDirty.
>>> self changedForm ].
>>>
>>>
>>>> WfWorflowManager class is empty defining empty stubs/protocols. Its aim
>>>> seems to be about “logging/noting” changes. Is is it’s only role ?
>>> Correct. You would use WfWorkflowManager to update the workflow in OmniBase
>>> (e.g. mark dirty, add new activation etc.). #generateID would generate an ID
>>> from OmniBase and #activationClass would answer your specific activation
>>> class, if you choose to override WfActivation.
>>>

Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

cedreek
In reply to this post by Stephane Ducasse-3
>
>
> Cedrick
>
> My goal is one step at a time.
> I do not want grand plan. Just something working in one image.

Yes sure. But at this is an important requirement for me, I prefer to talk about it first :)


> Stef
>
> On Sun, Jan 21, 2018 at 12:19 PM, Cédrick Béler <[hidden email]> wrote:
>> Besides all suggested ideas on the previous mail, what’s very important to me is being able to run processes between several images considering 1 image = 1 actor.
>>
>> At first, we can consider both actor being on the same image (what Stephane wants first) but my requirement number 1 is having 1 actor by image and therefore having the possibility to trigger DistantSubWorkflow.
>>
>> Cheers,
>>
>> Cédrick
>


Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

hernanmd
In reply to this post by cedreek
Hi Cédrick,

2018-01-21 8:04 GMT-03:00 Cédrick Béler <[hidden email]>:

> Hi Hernan,
>
> I decided to give a try to Aare or Workflow in Pharo 6, following
> instructions in https://github.com/Netstyle/Workflow .
>
>
> Cool :)
>
> I started but had to stop until the semester is finish. Next month should be
> ok to continue on that.
>
>
> However, the
> installation script raised a Warning about missing WADynamicVariable
> class. Should I install Seaside?
>
>
>
> Happy to see people having interest in such tools :)
>
> No.
>
> I discussed with Max Leske.
>
> To solve this « pb », just create WADynamicVariable as a subclass of
> DynamicVariable already in Pharo. There are also missing part for the
> export/import in XML (should be easy to fix and Max propose to give the code
> - DOMBuilder and co.)

Thanks. I cloned the Ducasse fork which already solved the issue.

> I tried to start a booklet with Stephane but it is too complicated right
> now. Stephane started a simpler version.
>
> I put below a summary of the reflexion/discussion I had with Max.
>
>
> Cheers,
>
> Cédrick
>
>
>
>
> ========= INTRO
>
> What's missing compared to the full (proprietary) solution are user
> interfaces that were web based (Seaside \+ Magritte). Also, persistence was
> achived through Omnibase. References were removed but they are stil some
> disfunctionning. Moreover, the completion of activities were mainly manual
> and achieved through web interface (web forms). They are, for now, no real
> way to introduce different kind of activities like those being
> message-based, event-based, service-based.
>
> To evolve, one has to remove all dependancies to Omnibase and
> Magritte/Seaside.
>
> There are also important design decisions to ensure WfWorkflow being a
> powerful and extensible workflow management system for Pharo. To me there
> are 2 main tasks:
> - rethink persistence (process definition and orchestration, realization)
> => persistance is essential and was central to Aare. Beside removing
> Omnibase references (and implication of WfManagedObject), some of the
> application features were very dependant on the persistence (like logging,
> tracing process evolution). All of these features are to be thing again
> considering as much as possible a loosely coupled solution. A general
> purpose storage solution like Voyage could be used. At first, in image
> storage will be used to focus on the running aspects of the library.
> - rethink interaction
> => Designing a general interaction sub-system is probably the more
> challenging tasks. Aare was web based. We want workflow to be UI agnostic.
> Moreover, we want to associate  ,
> remove Magritte (replace UI or build an independant system based on
> Annoucement for instance)
>

It could be me that I am used to execute and wait processes jobs for
hours or days in HPC: typical workflows of NGS trimming, QC, genome
indexing, sequence alignment, etc. But integration of a monitoring
system (like Ganglia) and interaction with a work load manager (like
PBS, Condor, Slurm, etc.) would be also giant tasks.

> ========= RANDOM QUESTIONS AND ANSWER (in black me - below max)
>
> 1)
>
> The static definition of workflow is done through WfWorkflow, WfOutgoingEdge
> and WfSteps (+ WfConditions). The easy part :)
>
> The runtime aspect are achieved via WfActivation, WfWorkList,
> WfWorkflowHistory et WfWorkflowManager. More difficult to understand and
> make evolve.
>
> WfWorkflowHistory represents the different versions of a workflow, so it
> belongs to the static part. WfWorkflowManager managers workflow definitions
> (static) and activations (dynamic).
> WfFrame is also a part of the dynamic side, connecting activations to a
> workflow.
> Workflows are hierarchical, i.e. a step can have a sub workflow (see
> WfWorkflowLibrary>>simpleSubflow)
>
>
>
> 2)
>
> To me activations are really central entities (and I need to be sure of its
> correct role). They are said to the realization of an activity/step.
>
> Can we consider the static definition of a workflow as « Plan »  and the
> runtime as the concrete realization (like a job in a scheduler) ?
>
> Yes, I would say so.
>
>
>
> I see that:
>
>
> FUTURE: Static BP définitions with steps (plans)
>
> PRESENT: activation (realization)
>
> PAST: History (experience feedback)
>
> Yes, that's certainly one way to look at it. Although I'd say that the
> static definition is independent from time once it exists and that "future"
> would be the set of steps of the current definition that have not yet been
> activated (just a little nit picking :) ).
>
>
>
> 3)
>
> Going back to activations, here is my current understanding.
>
> Once a step is activated (all condition on Edges leading to are true - to
> evaluate outgoing conditions of a step, the step execution has to be
> complete first) , we get an activation instance (I’m not 100% sure how
> completion of an activity and outgoing conditions are managed).
>
> As far as I understand, a step can always be activated (e.g. manually).
> However, activating a step propagates the values of conditions on the edges
> to the next steps and their activations ("dead" or "alive"). Hence, an
> activation responds to #isAlive with true if at leat one incoming token was
> true ("alive") (see WfStep>>continueInFrame:fromStep:alive: and
> WfActivation>>isAlive). The conditions are evaluated once #complete is sent
> to the activation.
> Activation instances are created by two ways (slightly different for the
> start step), but generally they are created when a step receives tokens from
> the outgoing edges of an upstream activation (see
> WfStep>>continueInFrame:fromStep:alive:).
>
>
> On the exemple of BLActivationHistoryItem that you gave, I’m not sure if
> it’s an activation of a history item stored in the history (subclasss of
> WfHistory?).
>
> My idea was that it should represent a record for an activation (there might
> be multiple records for a single activation). So "HistoryItem" would be the
> general description of a record, of which "ActivationHistoryItem" is a
> special case for activations (I hope that makes sense).
>
>
> 4)
>
> Moreover I don’t see clearly the difference with a scheduler and a BP engine
> (they are at least really complementary). Do you see any ressemblance
> between Activation and ScheduledJob ?
>
> I actually would like to have a (simple) scheduler with the BP engine :)
>
> Well, it's certainly a matter of point of view. I would argue that an
> activation is something close to a "job". A "scheduled job" would be a more
> specific case of a "job", a job that is in a queue. An activation has no
> notion of scheduling but simply represents the idea of what happens when
> traversal between two nodes occurs. A scheduler would be one way of managing
> activations (WfWorklist seems pretty close to a scheduler, holding running
> and completed activations).
>
>
> To me the scheduler right now is not really existing besides having a list
> of simultaneous/parralelized steps to realize (activated or not). Activated
> means currently being done.
>
> You're right, I think. The application that Workflow was designed for
> required manual interaction, hence there was no need for other methods of
> scheduling (e.g. by time or event).
>
>
>
> This activation role is about logging and interacting with responsible
> users. But it is more than that. It’s the actual job being done.
>
> Yes.
>
>
>
> 5)
>
> Concerning run time modification of a process. Is it true that we can modify
> a currently executed workflow (steps that are not yet activated ?) ? I think
> that is possible (an cool). When activated, it shouldn’t. Would you think it
> would be interesting to allow activations to be suspended/resume (frozen),
> cancelled ?
>
> The history of workflows consists of copies. The steps of a workflow
> reference the workflow they belong to. Therefore, while it is possible to
> modify a workflow, modifications do not impact running workflows, as the
> steps of that workflow continue to reference the previous version. The model
> suggests that it is ok to create a new version of a workflow but not to
> modify existing workflows.
> The model does *allow* (in principle) to modify an existing workflow but I
> doubt that was the intention. However, I don't see why modifying an existing
> workflow has to be bad thing, as long as you can guarantee that only edges
> and steps are modified that have not yet been activated. That could open up
> some interesting possibilities.
> Cancellation of an activation could be very interesting. It would basically
> allow reversal of complex effects. Very nice idea! I'm not so sure though
> whether suspending makes much sense. Maybe when a step has the potential to
> take a long time for some operation. But then you may run into transaction
> problems. The nice thing about an activation at the moment is that it's
> atomic (in theory). But again, done right that could open up interesting
> possibilities.
>
>
>
> If a running process is modified, do we track its evolutions in
> WfWorkflowManager ?
>
> The modification of a workflow is indeed announced to WfWorkflowManager,
> yes. Workflow versions are stored in WfWorkflowHistory. For modifying a
> *running* workflow you will probably want to create a specific workflow
> manager for that, that does validation etc.
>
>
>
> The goal of history(s) are to log the realization of the process. But is it
> one of its aim to « replay » the process execution ?
>
> No. WfWorkflowHistory only stores the static definitions. However, the
> history of activations could be used to replay an execution (nice idea!).
>
>  i.o.w.,  Is it more than log/report ?
>
> I think the activation history was designed as a log, the workflow history
> as a log and also as a way to undo changes. As you suggest, however, they
> could certainly be used for more things.
>
>
> I agree with the separation (WfWFManager for the run time evolution of the
> process definition => adaptation    and WfActivationHistory for the trace of
> interactive/automatic step realizations information).
>
>
> #beginHistory implementation example was very helpful. I knew I missed
> something  Just, as I see a MailManager (with role is obvious), is it
> something that has to be generalizable ? Like InteroperableManager ?
> MessageManager ? To get user interactions
>
> That would be very nice, yes. You're completely free to do what you like. It
> would be cool though if we could evolve the project as opposed to you
> creating a fork to your own liking. If you're game we can set up a workflow
> on github where you open pull requests against our repository.
>
>
>
> 6) I’d like to remove fully the reference and need of Omnibase and
> Magritte/seaside forms.
>
> Yes please!
>
>
> I think Omnibase is ok, even storing in image for instance.
>
> What’s more problematic to me is for user interactions (and even automatic
> interaction through program for instance). There are achieved in your
> example with seaside. It is ok but what if I’d like to do it more generally
> ? Would we use Annoucement for example ?
>
> I think that is completely open to implementation. In my eyes, Workflow
> should remain a separate library that does not enforce any kind of specific
> presentation or persistence. What's critical for that is of course the right
> kind of interface.
> Announcements would certainly be an interesting way to decouple Workflow
> from other parts of the system.
>
>
> I’d like to have an incomingMessageBox for each user and eventually
> transform these message in concrete action (user i/o) ? Or dispatch in
> JobQueues (each user being a distant worker thread). Again, the distinction
> between Scheduler and BPEngin is fuzzy to me.
>
>
> Does it make sense to you ?
>
> Yes, it does :) What I've asked myself (out of curiosity) is whether you
> want to have
> - a dedicated instance for executing workflows
> - a distributed workflow with autonomous synchronisation (parts of a
> workflow could run on every uses instance, others would need to be executed
> through agreement)
> - a partially distributed workflow with one or more managing nodes that
> control the core steps (e.g. you could designate parts of a workflow as
> independant)
>
>
> PS: some random comments on your proposals
>
> Parts of Workflow were never cleanly separated from the proprietary
> application, so there are indeed a couple of things that are missing (mostly
> #subclassResponsibility stuff etc.)
>
>
>
> - initializeDefaults is not present for steps but thats ok.
>
>
> - in #createIncoming: (doesn’t exist), I don’t get ….. => makePersistent
> asPersistentReference.   You store persistentReference of steps. Is it like
> the step object itself ?  I would store the step objects (as no persistance
> right now).
>
> An activity step is a specific subclass of a step. Such a step includes
> additional information such as documents, duration and user roles. I'm not
> sure why "makePersistent asPersistentReference" is sent there, as WfStep
> already inherits from WfManagedObject and would, therefore, already be
> managed by OmniBase. But I don't know OmniBase, maybe some things need to be
> announced specifically.
>
>
> - I don’t understand the offsetSeconds in your BLActivationHistoryItem
>
> That's really just some business rule to modify the timestamp of the history
> item.
>
>
> ============= Others responses from Max
>
> WfWorkflow - Load and Run
> All instances that inherit from WfManagedObject were managed by OmniBase,
> yes.
> DomBuilder is a small package for wrapping XML-Parser. I'll publish it so
> you can use it. The XML-Parser package has been significantly modified and
> is now named XMLParser (you can find it in the catalog). You should make the
> changes necessary to use the current incarnation of XMLParser (decide for
> yourself whether you still need DomBuilder or not).
>
>
> Those IDXXX classes belong to another package a didn't publish (but will).
> This package also includes an OmniBase specific class.
> Magritte has changed significantly over the years but the core should be
> pretty much the same. Just load the newest Magritte and try working with
> that.
>> ok but avoidable ?
>> try loading Magritte ?   Long …
>
> #beginInHistory should be implemented with #subclassResponsibility on
> WfActivation.
> Your explicit implementation could then look something like this:
> beginInHistory
> self history add: (MyActivationHistoryItem new
> message: 'started';
> user: (WACurrentSession value
> ifNotNilDo: [ :session | session user ]);
> type: #started;
> yourself).
> MyMailManager noteNewActivation: self
>
> Basically, what the history contains is up to the workflow implementation. I
> think it would makes sense though to have an abstract
> WfActivationHistoryItem.
> WfActivation>>history should be implemented like this:
> history
> ^self propertyAt: #history ifAbsentPut: [ OrderedCollection new ].
>
> Although I think that it would be good to have a dedicated
> WfActivationHistory class. WfWorkflowHistory is associated with changes to
> the workflow, not activations. Both history classes should probably inherit
> from an abstract class WFAbstractHistory.
>
>
> Seaside interaction
>
> Here are a couple of action implementations in Seaside components that
> should give you an idea of how to interact with the workflow:
> MyActivationComponent>>complete
> self activation hasCompleted
> ifTrue: [ ^ self ].
> self history add: (BLActivationHistoryItem new
> message: 'completed';
> user: self session user;
> type: #completed;
> offsetSeconds: -1;
> yourself).
> self activation complete.
>
> MyActivationForm>>accept: aString
> self activation authorized: true.
> self history add: (BLActivationHistoryItem new
> comment: aString;
> message: 'accepted';
> user: self session user;
> type: #accepted;
> offsetSeconds: -2;
> yourself).
> self completeIfSatisfied.
>
> MyActivityList>>addActivity
> self call: (MyActivityEditor new
> workflow: self workflow;
> yourself).
> self refresh.
>
> MyActivityEditor>>buildActivity
> ^(self workflow newActivityStep
> initializeDefaults;
> yourself)
> asPersistentReference.
>
> MyTransitionEditor>>createIncoming
> | step |
> step := (self workflow newActivityStep
> initializeDefaults;
> yourself) makePersistent asPersistentReference.
> self addIncoming: step.
> MyCurrentWorkflowComponent value showActivity: step.
>
> MyTransitionEditor>>addIncoming: anActivity
> ^self add: anActivity to: self activity.
>
> MyTransitionEditor>>add: aSourceActivity to: aDestinationActivity
> ^aSourceActivity addOutgoingEdgeFor: aDestinationActivity.
>
> MyTransitionEditor>>editCondtions: anEdge
> | component |
> anEdge condition isTrueCondition
> ifTrue: [ anEdge condition: WfAllCondition new ].
> component := (MyConditionEditor activity: self activity)
> container: anEdge;
> addDecoration: (MyToolbarDecoration new
> title: 'Edit Transition Conditions to: ' , anEdge to name;
> toolbar: [ :html |
> html
> icon: 'icon_close.gif'
> text: 'Close'
> action: [ component answer ].
> html helpIconOn: component ];
> yourself);
> yourself.
> self call: component.
>
> MyConditionEditor>>addRule: aDescription
> | new |
> new := MyCondition newPersistent
> field: aDescription;
> asPersistentReference.
> new := self call: (self
> buildEditorFor: new
> titled: 'Add Rule for ' , aDescription asLabel).
> new isNil ifFalse: [
> self condition add: new; markDirty.
> self changedForm ].
>
>
>> WfWorflowManager class is empty defining empty stubs/protocols. Its aim
>> seems to be about “logging/noting” changes. Is is it’s only role ?
> Correct. You would use WfWorkflowManager to update the workflow in OmniBase
> (e.g. mark dirty, add new activation etc.). #generateID would generate an ID
> from OmniBase and #activationClass would answer your specific activation
> class, if you choose to override WfActivation.
>

Cool, thank you both for including your very valuable comments!

I did my first experiment and have a script to start:

| wf task1 task2 startStep frame |

wf := WfWorkflowLibrary default workflowNamed: 'Test1'.
task1 := wf newStepNamed: 'Step 1'.
task2 := wf newStepNamed: 'Step 2'.
startStep := wf start.
startStep addOutgoingEdgeFor: task1.
task1 addOutgoingEdgeFor: task2.

frame := wf executeInNewFrame.
frame workList running first complete.
frame workList runningSteps.
frame workList running first complete.
frame workList runningSteps.
wf

To see how it matches with scientific workflows I will need more experiments.

Cheers,

Hernán

Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

cedreek
In reply to this post by Stephane Ducasse-3

HI cedrick


I tried to start a booklet with Stephane but it is too complicated right
now. Stephane started a simpler version.

For now saving and loading is not important. :)

Yes



I put below a summary of the reflexion/discussion I had with Max.

Thanks.
Now you should publish your code.
I would have preferred to work from a working code. Because now it is
like poking in the dark.

Cedrick what did you get running?

Actually, I got the workflow working and the tests as you. And then I was able to run some examples (as I tried to wrote in the booklet).

I just din’t know yet how to publish on GitHub and friends… I discussed with Max about then… I’m learned a bit after how to use these tools especially for the booklet as we chatted….
Then end of the semester….  No time…. Should be better soon.

So sorry for that :)

I join the last pdf I had…. Fuzzy (especially since I tried to include automatically classes comments)  but there are details of installation and what I could make work:


- rethink persistence (process definition and orchestration, realization)
=> persistance is essential and was central to Aare. Beside removing
Omnibase references (and implication of WfManagedObject), some of the
application features were very dependant on the persistence (like logging,
tracing process evolution). All of these features are to be thing again
considering as much as possible a loosely coupled solution. A general
purpose storage solution like Voyage could be used. At first, in image
storage will be used to focus on the running aspects of the library.
- rethink interaction
=> Designing a general interaction sub-system is probably the more
challenging tasks. Aare was web based. We want workflow to be UI agnostic.
Moreover, we want to associate  ,
remove Magritte (replace UI or build an independant system based on
Annoucement for instance)


Yes I agree. Now I would like to be able to
- define a super simple workflow: I picked sequence: start -> task1 -> task2

Done in WfWorkflowLibrary (with convenience method). An example in 2.4




- execute it and script it resolution.

See 2.5.

My way of running them and where I stopped and is a bit stuck.

To me we need now to design an interaction model (based on Annoucement ?)



Running a workflow. 

Once a workflow is defined, to be executed, he calls the method #executeIn- newFrame. This method instanciate and populate the worklist (WfWorklist) that keep track of the state of the running workflow.

      workflow := WfWorkflowLibrary simpleBranch.
      frame := workflow executeInNewFrame.

To see running activitions (activities that are started but not completed yet):

      frame worklist running.

Once a activation is over, it has to be completed by sending the message #completed. For now, there are only manual completions but we could imag- ine in the future automatic or semi-atomatic completion based on extrenal interaction or if delegated to a program for instance.

      anActivation := frame worklist any.
      anActivation complete. "complete mannully one activation"
      "or"
      frame complete: anActivation.

The workflow is running until there are no possible (accessible) activation left. This state is called implicit termination and can be checked with:

      frame isCompleted.

Here is the code of the WfFrame»>completeAll

      WfFrame>>>completeAll
        [self workList allRunning isEmpty] whileFalse:
          [self completeActivation: self workList allRunning first].




I would like to understand the difference between frame and activation.

The frame is the container of the workflow resolution, the « manager » … A workflow runs in a frame (subworkflows in subframes)…

Activations are when a step (one activity defined in a workflow) is activated (we are starting for real to do the activity).



WfWorkflowManager managers workflow definitions
(static) and activations (dynamic).
WfFrame is also a part of the dynamic side, connecting activations to a
workflow.
Workflows are hierarchical, i.e. a step can have a sub workflow (see
WfWorkflowLibrary>>simpleSubflow)

Yes for now I do not want to understand this part.

You have too to get the workflow running I think.




3)

Going back to activations, here is my current understanding.

Once a step is activated (all condition on Edges leading to are true - to
evaluate outgoing conditions of a step, the step execution has to be
complete first) , we get an activation instance (I’m not 100% sure how
completion of an activity and outgoing conditions are managed).

As far as I understand, a step can always be activated (e.g. manually).
However, activating a step propagates the values of conditions on the edges
to the next steps and their activations ("dead" or "alive"). Hence, an
activation responds to #isAlive with true if at leat one incoming token was
true ("alive") (see WfStep>>continueInFrame:fromStep:alive: and
WfActivation>>isAlive). The conditions are evaluated once #complete is sent
to the activation.
Activation instances are created by two ways (slightly different for the
start step), but generally they are created when a step receives tokens from
the outgoing edges of an upstream activation (see
WfStep>>continueInFrame:fromStep:alive:).

Activations are central.



On the exemple of BLActivationHistoryItem that you gave, I’m not sure if
it’s an activation of a history item stored in the history (subclasss of
WfHistory?).

My idea was that it should represent a record for an activation (there might
be multiple records for a single activation). So "HistoryItem" would be the
general description of a record, of which "ActivationHistoryItem" is a
special case for activations (I hope that makes sense).


4)
Moreover I don’t see clearly the difference with a scheduler and a BP engine
(they are at least really complementary). Do you see any ressemblance
between Activation and ScheduledJob ?

I actually would like to have a (simple) scheduler with the BP engine :)

Well, it's certainly a matter of point of view. I would argue that an
activation is something close to a "job". A "scheduled job" would be a more
specific case of a "job", a job that is in a queue. An activation has no
notion of scheduling but simply represents the idea of what happens when
traversal between two nodes occurs. A scheduler would be one way of managing
activations (WfWorklist seems pretty close to a scheduler, holding running
and completed activations).


Too subtle for me.

For now, the are no real workflow engine. More a state machine providing access to activable steps (and running one… hence remaining ones).

I/We will need a scheduler (even simple ones). 

Max explains below: 



To me the scheduler right now is not really existing besides having a list
of simultaneous/parralelized steps to realize (activated or not). Activated
means currently being done.

You're right, I think. The application that Workflow was designed for
required manual interaction, hence there was no need for other methods of
scheduling (e.g. by time or event).



This activation role is about logging and interacting with responsible
users. But it is more than that. It’s the actual job being done.

Yes.



5)

Concerning run time modification of a process. Is it true that we can modify
a currently executed workflow (steps that are not yet activated ?) ? I think
that is possible (an cool). When activated, it shouldn’t. Would you think it
would be interesting to allow activations to be suspended/resume (frozen),
cancelled ?

The history of workflows consists of copies. The steps of a workflow
reference the workflow they belong to. Therefore, while it is possible to
modify a workflow, modifications do not impact running workflows, as the
steps of that workflow continue to reference the previous version. The model
suggests that it is ok to create a new version of a workflow but not to
modify existing workflows.

For now I would like to get a workflow running without modifying it

Sure. But again an interesting decision to have first.

To me, what more interesting is to be able :
- to cancel a task (an activation actually),
- change dynamically the remaining workflow (to adapt)… 
- eventually suspend/resume (nice to time track but I also thin it’s not hard to have)

I know the second point seems too early but again this is important to consider now.

Designs decisions of Aare leaves some room to build more (and constrain its use).


[…]

DomBuilder is a small package for wrapping XML-Parser. I'll publish it so
you can use it.

Cedric did you got this package?

No but I didn’t ask either. It’s possible to ask Max. But I think first, as you, we need to be able to run a process.



The XML-Parser package has been significantly modified and
is now named XMLParser (you can find it in the catalog). You should make the
changes necessary to use the current incarnation of XMLParser (decide for
yourself whether you still need DomBuilder or not).

But this is not really important.



Yep.

Cheers,

Cédrick


book.pdf (216K) Download Attachment
PastedGraphic-4.png (180K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

Stephane Ducasse-3
Ok so we redid everything in parallel. Too bad anyway I will continue
and my changes are in my clone.
I will update the booklet once I get it.

Stef

On Sun, Jan 21, 2018 at 6:05 PM, Cédrick Béler <[hidden email]> wrote:

>
> HI cedrick
>
>
> I tried to start a booklet with Stephane but it is too complicated right
> now. Stephane started a simpler version.
>
>
> For now saving and loading is not important. :)
>
>
> Yes
>
>
>
> I put below a summary of the reflexion/discussion I had with Max.
>
>
> Thanks.
> Now you should publish your code.
> I would have preferred to work from a working code. Because now it is
> like poking in the dark.
>
> Cedrick what did you get running?
>
>
> Actually, I got the workflow working and the tests as you. And then I was
> able to run some examples (as I tried to wrote in the booklet).
>
> I just din’t know yet how to publish on GitHub and friends… I discussed with
> Max about then… I’m learned a bit after how to use these tools especially
> for the booklet as we chatted….
> Then end of the semester….  No time…. Should be better soon.
>
> So sorry for that :)
>
> I join the last pdf I had…. Fuzzy (especially since I tried to include
> automatically classes comments)  but there are details of installation and
> what I could make work:
>
>
> - rethink persistence (process definition and orchestration, realization)
>
> => persistance is essential and was central to Aare. Beside removing
> Omnibase references (and implication of WfManagedObject), some of the
> application features were very dependant on the persistence (like logging,
> tracing process evolution). All of these features are to be thing again
> considering as much as possible a loosely coupled solution. A general
> purpose storage solution like Voyage could be used. At first, in image
> storage will be used to focus on the running aspects of the library.
> - rethink interaction
> => Designing a general interaction sub-system is probably the more
> challenging tasks. Aare was web based. We want workflow to be UI agnostic.
> Moreover, we want to associate  ,
> remove Magritte (replace UI or build an independant system based on
> Annoucement for instance)
>
>
>
> Yes I agree. Now I would like to be able to
> - define a super simple workflow: I picked sequence: start -> task1 -> task2
>
>
> Done in WfWorkflowLibrary (with convenience method). An example in 2.4
>
>
>
>
> - execute it and script it resolution.
>
>
> See 2.5.
>
> My way of running them and where I stopped and is a bit stuck.
>
> To me we need now to design an interaction model (based on Annoucement ?)
>
>
>
> Running a workflow.
>
> Once a workflow is defined, to be executed, he calls the method #executeIn-
> newFrame. This method instanciate and populate the worklist (WfWorklist)
> that keep track of the state of the running workflow.
>
>       workflow := WfWorkflowLibrary simpleBranch.
>       frame := workflow executeInNewFrame.
>
> To see running activitions (activities that are started but not completed
> yet):
>
>       frame worklist running.
>
> Once a activation is over, it has to be completed by sending the message
> #completed. For now, there are only manual completions but we could imag-
> ine in the future automatic or semi-atomatic completion based on extrenal
> interaction or if delegated to a program for instance.
>
>       anActivation := frame worklist any.
>       anActivation complete. "complete mannully one activation"
>       "or"
>       frame complete: anActivation.
>
> The workflow is running until there are no possible (accessible) activation
> left. This state is called implicit termination and can be checked with:
>
>       frame isCompleted.
>
> Here is the code of the WfFrame»>completeAll
>
>       WfFrame>>>completeAll
>         [self workList allRunning isEmpty] whileFalse:
>
>           [self completeActivation: self workList allRunning first].
>
>
>
>
>
> I would like to understand the difference between frame and activation.
>
>
> The frame is the container of the workflow resolution, the « manager » … A
> workflow runs in a frame (subworkflows in subframes)…
>
> Activations are when a step (one activity defined in a workflow) is
> activated (we are starting for real to do the activity).
>
>
>
> WfWorkflowManager managers workflow definitions
>
> (static) and activations (dynamic).
> WfFrame is also a part of the dynamic side, connecting activations to a
> workflow.
> Workflows are hierarchical, i.e. a step can have a sub workflow (see
> WfWorkflowLibrary>>simpleSubflow)
>
>
> Yes for now I do not want to understand this part.
>
>
> You have too to get the workflow running I think.
>
>
>
>
> 3)
>
> Going back to activations, here is my current understanding.
>
> Once a step is activated (all condition on Edges leading to are true - to
> evaluate outgoing conditions of a step, the step execution has to be
> complete first) , we get an activation instance (I’m not 100% sure how
> completion of an activity and outgoing conditions are managed).
>
> As far as I understand, a step can always be activated (e.g. manually).
> However, activating a step propagates the values of conditions on the edges
> to the next steps and their activations ("dead" or "alive"). Hence, an
> activation responds to #isAlive with true if at leat one incoming token was
> true ("alive") (see WfStep>>continueInFrame:fromStep:alive: and
> WfActivation>>isAlive). The conditions are evaluated once #complete is sent
> to the activation.
> Activation instances are created by two ways (slightly different for the
> start step), but generally they are created when a step receives tokens from
> the outgoing edges of an upstream activation (see
> WfStep>>continueInFrame:fromStep:alive:).
>
>
> Activations are central.
>
>
>
> On the exemple of BLActivationHistoryItem that you gave, I’m not sure if
> it’s an activation of a history item stored in the history (subclasss of
> WfHistory?).
>
> My idea was that it should represent a record for an activation (there might
> be multiple records for a single activation). So "HistoryItem" would be the
> general description of a record, of which "ActivationHistoryItem" is a
> special case for activations (I hope that makes sense).
>
>
> 4)
> Moreover I don’t see clearly the difference with a scheduler and a BP engine
> (they are at least really complementary). Do you see any ressemblance
> between Activation and ScheduledJob ?
>
> I actually would like to have a (simple) scheduler with the BP engine :)
>
> Well, it's certainly a matter of point of view. I would argue that an
> activation is something close to a "job". A "scheduled job" would be a more
> specific case of a "job", a job that is in a queue. An activation has no
> notion of scheduling but simply represents the idea of what happens when
> traversal between two nodes occurs. A scheduler would be one way of managing
> activations (WfWorklist seems pretty close to a scheduler, holding running
> and completed activations).
>
>
>
> Too subtle for me.
>
>
> For now, the are no real workflow engine. More a state machine providing
> access to activable steps (and running one… hence remaining ones).
>
> I/We will need a scheduler (even simple ones).
>
> Max explains below:
>
>
>
> To me the scheduler right now is not really existing besides having a list
> of simultaneous/parralelized steps to realize (activated or not). Activated
> means currently being done.
>
> You're right, I think. The application that Workflow was designed for
> required manual interaction, hence there was no need for other methods of
> scheduling (e.g. by time or event).
>
>
>
> This activation role is about logging and interacting with responsible
> users. But it is more than that. It’s the actual job being done.
>
> Yes.
>
>
>
> 5)
>
> Concerning run time modification of a process. Is it true that we can modify
> a currently executed workflow (steps that are not yet activated ?) ? I think
> that is possible (an cool). When activated, it shouldn’t. Would you think it
> would be interesting to allow activations to be suspended/resume (frozen),
> cancelled ?
>
> The history of workflows consists of copies. The steps of a workflow
> reference the workflow they belong to. Therefore, while it is possible to
> modify a workflow, modifications do not impact running workflows, as the
> steps of that workflow continue to reference the previous version. The model
> suggests that it is ok to create a new version of a workflow but not to
> modify existing workflows.
>
>
> For now I would like to get a workflow running without modifying it
>
>
> Sure. But again an interesting decision to have first.
>
> To me, what more interesting is to be able :
> - to cancel a task (an activation actually),
> - change dynamically the remaining workflow (to adapt)…
> - eventually suspend/resume (nice to time track but I also thin it’s not
> hard to have)
>
> I know the second point seems too early but again this is important to
> consider now.
>
> Designs decisions of Aare leaves some room to build more (and constrain its
> use).
>
>
> […]
>
> DomBuilder is a small package for wrapping XML-Parser. I'll publish it so
> you can use it.
>
>
> Cedric did you got this package?
>
>
> No but I didn’t ask either. It’s possible to ask Max. But I think first, as
> you, we need to be able to run a process.
>
>
>
> The XML-Parser package has been significantly modified and
>
> is now named XMLParser (you can find it in the catalog). You should make the
> changes necessary to use the current incarnation of XMLParser (decide for
> yourself whether you still need DomBuilder or not).
>
>
> But this is not really important.
>
>
>
> Yep.
>
> Cheers,
>
> Cédrick
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Aare questions

Stephan Eggermont-3
In reply to this post by cedreek
Cédrick Béler <[hidden email]> wrote:
> Yes sure. But at this is an important requirement for me, I prefer to
> talk about it first :)

Distributed systems design is a different subject. Different constraints,
different solutions.

Stephan




Reply | Threaded
Open this post in threaded view
|

Workflow/Aare design - distribution or not ? And how ? (Was Re: (Aare questions))

cedreek
Just change the subject to discuss on one point. 

=> How make workflow application (in one image) delegate activation (and/or subworkflow) to another image.


Yes sure. But at this is an important requirement for me, I prefer to
talk about it first :)

Distributed systems design is a different subject. Different constraints,
different solutions.

I don’t really agree. Of course you’re right but I see little interest in having a process engine in a centralized manner (today is more about decentralized solution than ever and we have smalltalk heritage about message passing). 

Because, in centralized workflow, complexity is the proper use of the tool. You work in a decentralized world with a centralized tool. You have to plug to existing IS through mails, sms, REST API or whatever… All workflow systems I’ve seen (I’m not expert though) are that way and quite boring to use by the way… Then, we enter the real of model transformation so as to be interoperable. Whereas I kind of like the concepts, I find it’s like programming in C vs. in Smalltalk fo information exchange… Slow, couter intuitive, web based most of the time… 

So to come back to the subject, to me, to have a powerful workflow engine, one has first to consider actors on several applications hence images (at starts). Of course several actors on the same image can be interesting, so doing it first has sense.

To me, but I know I see things superficially (I’m not a programmer), we could have something quite simple that does the job. 

Of course it raises questions like we had with Max. I copy below as my response (see PS). 
But to sumup, the simple version I need will be one person runs a workflow in a RootFrame. Then he has activities he is responsible but I can delegate some other. It’s possible to have subworkflows (but again associated to him or an actor in the image). We could « just add » the possibility to activate a distant step (or a distant subworkflow). Me (Alice) ask someone else  (Bob) to do an activity (a step). 

Here is how I see it:

-My process activate this task (image Alice). I send a signal, a message to the appropriate actor (image of Bob). I just wait until I receive other signals (progression metadata, resources, deliverables…). Of course cancelling an activation is useful there as Bob can never answer.
- I (image Alice) have not a lot of responsibility except waiting. 
- An interesting option would be to be able to ping for completion. 

-The distant image manager (and scheduler) receive a signal, a message so that a step has to be done. 
-Bob does the job on its own. Send progressively elements (eventually). He has to send at least when the job is over (eventually devliverables as documents, informations, …).

The key to me is to consider systems that are distributed (decentralized actually) not in computer terms but in human terms… It changes everything as less complex. Look at workflow right now, it only allows manual completion of activations (step being realized)…. So it’s basically a human clicking on a button to tell the system the activity is over ! I would prefer having this functionality in a distributed/decentralized manner than having workflow more powerful in a centralized manner.

So to me we need to extend and reify better the possibility to delegate and clicks on buttons (regarding questions) and of course in a decentralized manner.

I hope you understand what I mean. I hope too not being to naive about such a design. But to me, we need two basic building blocks to provide what I (we?) need:
- the possibility to inform to someone or to a group (wait no response - but eventually information on status like reception, first reading, complete reading)
- the possibility to request someone or to a group (I wait for a response)

Don’t hesitate to criticize, discuss, I’m all ears ;-)

Cheers,

Cédrick


PS: The discussion about this point with Max

Announcements would certainly be an interesting way to decouple Workflow from other parts of the system.



Yes an important design decision.



I’d like to have an incomingMessageBox for each user and eventually transform these message in concrete action (user i/o) ? Or dispatch in JobQueues (each user being a distant worker thread). Again, the distinction between Scheduler and BPEngin is fuzzy to me.

Does it make sense to you ?

Yes, it does :) What I've asked myself (out of curiosity) is whether you want to have 

- a dedicated instance for executing workflows

- a distributed workflow with autonomous synchronisation (parts of a workflow could run on every uses instance, others would need to be executed through agreement)

- a partially distributed workflow with one or more managing nodes that control the core steps (e.g. you could designate parts of a workflow as independant)



I’m not really sure what I want :) . 
I think the 3 are possible/interesting. What I don’t want is a centralized approach (if it exists, it has to be a very special case).

My project is about having autonomous personnel systems. My system have a priori several nodes (one application per device). It’s a decentralized (federated/distributed) network of nodes.
My system interact (exchange information basically through messages) with other species having an equivalent node-based system.
All is about keeping version history of the information (yes it looks like GIT that I try not to use ^^ - GITHUB could probably be a special kind of node BTW). 

So I was thinking that one of the exchanged « messages » type could be « activations » sent by someone responsible of the workflow. So :

At first, I want each personal system to have the BP Engine and be able to execute workflow alone (personal routine, planification, suggestions, …) => normal usage
Soon, it becomes distributed, for instance if the person have several devices that share the system and the activations.

And if we consider interactions with other persons (and organisations which are group of persons), then we quickly have distributed (or partially distributed) workflows. 
Not crystal clear I guess :-) 
This is a subject I’d like to propose a PhD on. I try to convince a web dev company to develop such a distributed organizational tool but… no positive answer for now). I need to push more !
So I give it a try by mysel and here we are.

To better respond to you question, here is a simple use case I imagine:
- I’m the responsible of the workflow execution.
- When I assign a task A to someone (or an organization, or something like a smart object), I delegate the realization and then it would be a special case of SubWorkflow that is distant.
1) once activated, I send a special message to the other personal system. 
+ deadline/metadata + interaction contracts (do I only wait or can I see the actual progression ? To which level?)
2) I wait. 
— I'm able to ask for the realization status, a degree of completion,  
—— Depending on the contract (level of agreement), I can have more or less right on seing the realization but that's not the point here. First, I want to delegate and have information about the realization degree (+eventual bad event that could have arose)
3) I received the completion message + eventual document, artefacts, summarized information (note that the system respect privacy by defaults - all exchange is under control).
4) my workflow continues

So it’s a bit of a distributed workflow under a central control node but with distributed sub-workflows. It seems doable without too much effort. Maybe declaring DistantSubFlow and having proper history (using messaging functionalities to get status update)
But again, quickly, that would be very interesting to consider several managing nodes (I guess another PHD on itself for this one). 



12