Smalltalk › Pharo › Pharo Smalltalk Users

Workflows and possible usages

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

10 messages Options

Tim Mackinnon

Workflows and possible usages

Hi everyone, in my day job I encountered an interesting domain problem that a previous java team has made a bit of a hash of ...

I thought it might be interesting to model it in Pharo to help explain it better to a new team and I was interested in community thoughts as we have some of the best thinkers on tap.

The scenario is that a user is presented with several input fields to apply for a loan , one of them is Amount and another the type of loan (cash, invoice, machinery etc).

Depending on these answers, they are next presented with different questions - if the amount > 100k there are some high value questions, if the loan type is machinery there are some more additional questions on the type of machine etc. This progresses to a third form and potentially a 4th and 5th.

So it’s effectively a decision tree, that rules out different loan types.

The UI is a React web form, and I’ve suggested to the team to do some experiments with a dynamic UI that interprets a Json payload that describes the questions, so it’s really about how to model the questions and then render them out in this Json format.

I was wondering if this was a typical workflow problem (and whether the netstyle engine was appropriate or wasn’t there some other ones announced recently too?)

I’ve never used a workflow engine - But it sounds a bit like the problem above? Or not?

Alternatively, I was thinking I could model groups of questions into a Decision, and then have a DecisionJourney of Decisions. Each decision could have some match and dependency criteria - e.g. some sort of matcher block eg [:d | d amount > 100 & d type = #machinery] and then some method to add questions to a DecisionForm on a match which would then render the Json payload.

But I wondered if workflow engines just do this, and they might handle extra stuff for free.

Anyone have any thoughts or directions to explore?

Tim

Sent from my iPhone

Tim Mackinnon

Re: Workflows and possible usages

If this does become of interest to anyone, I had a quick look at the workflow landscape in Pharo and found it rather unfathomable - Netsyle seems very comprehensive but complicated and lacking syntactic sugar to make it easy to just pick up and run with. It’s reified up the hilt, so simple stuff seems to need lots of code, which the examples emphasise.

NewWave has a great readme.md, so quickly got some initial concepts but it’s very work in progress and key base classes have no documentation so it’s not easy to understand what does what for more detailed things (Sebastijan did respond quickly to some questions, so hope to see it continue to improve). It looked promising but it seems to want to “own the flow” and run everything, so my Teapot endpoints couldn’t easily get the next workflow from the engine - the engine wanted to push things to me (I messed with a semaphore to try and sync things, but decided it’s ethos was just different - I think the netstyle engine would let me pull, but didn’t get that far with it).

There was a gemstone workflow engine - it’s not ported, it looked interesting but used a Windows bpml tool and again wasn’t sure if it wanted to own the world, and porting it would be more than I wanted to do.

After an hour, I went with my own domain model and some condition blocks to just get something going. Teapot and NeoJsonWriter really let you prototype quickly! Had something in 15 mins to let me mull over the problem.

Tim

Sent from my iPhone

> On 26 Sep 2019, at 09:12, Tim Mackinnon <[hidden email]> wrote:
>
> Hi everyone, in my day job I encountered an interesting domain problem that a previous java team has made a bit of a hash of ...
>
> I thought it might be interesting to model it in Pharo to help explain it better to a new team and I was interested in community thoughts as we have some of the best thinkers on tap.
>
> The scenario is that a user is presented with several input fields to apply for a loan , one of them is Amount and another the type of loan (cash, invoice, machinery etc).
>
> Depending on these answers, they are next presented with different questions - if the amount > 100k there are some high value questions, if the loan type is machinery there are some more additional questions on the type of machine etc. This progresses to a third form and potentially a 4th and 5th.
>
> So it’s effectively a decision tree, that rules out different loan types.
>
> The UI is a React web form, and I’ve suggested to the team to do some experiments with a dynamic UI that interprets a Json payload that describes the questions, so it’s really about how to model the questions and then render them out in this Json format.
>
> I was wondering if this was a typical workflow problem (and whether the netstyle engine was appropriate or wasn’t there some other ones announced recently too?)
>
> I’ve never used a workflow engine - But it sounds a bit like the problem above? Or not?
>
> Alternatively, I was thinking I could model groups of questions into a Decision, and then have a DecisionJourney of Decisions. Each decision could have some match and dependency criteria - e.g. some sort of matcher block eg [:d | d amount > 100 & d type = #machinery] and then some method to add questions to a DecisionForm on a match which would then render the Json payload.
>
> But I wondered if workflow engines just do this, and they might handle extra stuff for free.
>
> Anyone have any thoughts or directions to explore?
>
> Tim
>
> Sent from my iPhone
>

NorbertHartl

Re: Workflows and possible usages

> Am 27.09.2019 um 10:10 schrieb Tim Mackinnon <[hidden email]>:
>
> If this does become of interest to anyone, I had a quick look at the workflow landscape in Pharo and found it rather unfathomable - Netsyle seems very comprehensive but complicated and lacking syntactic sugar to make it easy to just pick up and run with. It’s reified up the hilt, so simple stuff seems to need lots of code, which the examples emphasise.
>
> NewWave has a great readme.md, so quickly got some initial concepts but it’s very work in progress and key base classes have no documentation so it’s not easy to understand what does what for more detailed things (Sebastijan did respond quickly to some questions, so hope to see it continue to improve). It looked promising but it seems to want to “own the flow” and run everything, so my Teapot endpoints couldn’t easily get the next workflow from the engine - the engine wanted to push things to me (I messed with a semaphore to try and sync things, but decided it’s ethos was just different - I think the netstyle engine would let me pull, but didn’t get that far with it).
>
> There was a gemstone workflow engine - it’s not ported, it looked interesting but used a Windows bpml tool and again wasn’t sure if it wanted to own the world, and porting it would be more than I wanted to do.
>
> After an hour, I went with my own domain model and some condition blocks to just get something going. Teapot and NeoJsonWriter really let you prototype quickly! Had something in 15 mins to let me mull over the problem.
>

That‘s where most „workflow engines“ start _and_ end ;)
Workflow engines are simple if the flow is simple. Most people come up with a simple sequence of tasks. To do it properly with all the fork and join possibilities is not that easy.

I think the Netstyle engine is kind of ok but never got past the point where seaside dependencies and Omnibase dependencies were properly refactored to be independent of its roots.

NewWave is the new trial. We are talking with the guys to help build a decent machine. At ESUG we talked more than an hour about approaches. The last thing we acknowledged was to create a mailing list for it to make the discussion more public. Stef wanted it to happen on this list but I think a dedicated one would be better. So we are trying to find a place for the list.

Marcus, I think we got stuck in the process finding a proper mailing list provider, right?

Norbert

> Tim
>
> Sent from my iPhone
>
>> On 26 Sep 2019, at 09:12, Tim Mackinnon <[hidden email]> wrote:
>>
>> Hi everyone, in my day job I encountered an interesting domain problem that a previous java team has made a bit of a hash of ...
>>
>> I thought it might be interesting to model it in Pharo to help explain it better to a new team and I was interested in community thoughts as we have some of the best thinkers on tap.
>>
>> The scenario is that a user is presented with several input fields to apply for a loan , one of them is Amount and another the type of loan (cash, invoice, machinery etc).
>>
>> Depending on these answers, they are next presented with different questions - if the amount > 100k there are some high value questions, if the loan type is machinery there are some more additional questions on the type of machine etc. This progresses to a third form and potentially a 4th and 5th.
>>
>> So it’s effectively a decision tree, that rules out different loan types.
>>
>> The UI is a React web form, and I’ve suggested to the team to do some experiments with a dynamic UI that interprets a Json payload that describes the questions, so it’s really about how to model the questions and then render them out in this Json format.
>>
>> I was wondering if this was a typical workflow problem (and whether the netstyle engine was appropriate or wasn’t there some other ones announced recently too?)
>>
>> I’ve never used a workflow engine - But it sounds a bit like the problem above? Or not?
>>
>> Alternatively, I was thinking I could model groups of questions into a Decision, and then have a DecisionJourney of Decisions. Each decision could have some match and dependency criteria - e.g. some sort of matcher block eg [:d | d amount > 100 & d type = #machinery] and then some method to add questions to a DecisionForm on a match which would then render the Json payload.
>>
>> But I wondered if workflow engines just do this, and they might handle extra stuff for free.
>>
>> Anyone have any thoughts or directions to explore?
>>
>> Tim
>>
>> Sent from my iPhone
>>
>
>

Christopher Fuhrman-4

Re: Workflows and possible usages

In reply to this post by Tim Mackinnon

On Thu, 26 Sep 2019 at 04:27, Tim Mackinnon <[hidden email]> wrote:

Anyone have any thoughts or directions to explore?

My contribution is very academic, and not really my field of expertise (we evaluated a BPML/BPMN tool about 10 years ago which I think is dead now). The conference paper from 2016 at https://link.springer.com/chapter/10.1007/978-3-319-42887-1_2 (which is behind a paywall) mentions lots of acronyms for the notations and languages. I put the date of the cited reference for each:

BPMN Business Process Modelling Notation (2013)
CMMN Case Management Modeling and Notation (2014, OMG)
Declare (framework, 2009)
DPIL Declarative Process Intermediate Language (2014)
WRP Workflow Resource Patterns (2005)
Dynamic Condition Response-Graphs (DCR-Graphs, 2013)
EM-BrA2CE (framework, 2008)

My intuition says that most software projects don't last long enough to invest in a general solution to supporting workflows.

Stephan Eggermont-3

Re: Workflows and possible usages

Christopher Fuhrman
<[hidden email]> wrote:
> On Thu, 26 Sep 2019 at 04:27, Tim Mackinnon
> <[hidden email]> wrote:
>> Anyone have any thoughts or directions to explore?

> My intuition says that most software projects don't last long enough to
> invest in a general solution to supporting workflows.

So, is there a causal relationship? And in which direction?

Stephan

eftomi

Re: Workflows and possible usages

> Anyone have any thoughts or directions to explore?

The main assumption about workflows is that they are easily understood by
non-developers, and that non-developers can be involved in the development,
within business environments. However, the workflows per-se were not
accepted well. For instance, even Microsoft's Windows Workflow Foundation in
.NET will not have any future in .NET Core, as it seems.

The idea went on, but on a totally different "track" and by different
people: today BPMN notation attracts most of the attention. It is being
used in business process management, processes design and reingeneering, and
it can be directly used in software development, together with use cases. It
has it's own role in information architecture frameworks. There are many
systems that support a direct SW automation in process-driven development
(e. g. Bonita https://www.bonitasoft.com/, ProcessMaker
https://www.processmaker.com/, Bizagi https://www.bizagi.com/, jBPM, Joget
... to list just a few). Most influential big players on this were Oracle
with SOA & BPM Suite and IBM WebSphere Process Server.

Most of these platforms are web based, they have various designers (like
process, data, forms designer), and extensive authentication and
authorization facilities. They use reliable technologies for database
access.

So, as for the workflow core engine, I would bet onto BPMN and a direct
support for BPMN 2.0 XML standard
(https://www.omg.org/oceb-2/documents/BPMN_Interchange.pdf). This would be
the core.

Just my two cents ...

Best wishes,
Tomaz

--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Christopher Fuhrman-4

Re: Workflows and possible usages

In reply to this post by Stephan Eggermont-3

On Sun, 29 Sep 2019 at 11:01, Stephan Eggermont <[hidden email]> wrote:

Christopher Fuhrman
<[hidden email]> wrote:
> On Thu, 26 Sep 2019 at 04:27, Tim Mackinnon
> <[hidden email]> wrote:
>> Anyone have any thoughts or directions to explore?

> My intuition says that most software projects don't last long enough to
> invest in a general solution to supporting workflows.

So, is there a causal relationship? And in which direction?

Interesting question, for which I have no answer.

At the risk of going off topic, here's more context for my "intuition": I'm talking about startups (or even companies such as Google who have many "beta" projects that will likely die because they're experiments). The goal is a Minimal Viable Product (MVP) first, because many projects (or features?) don't continue for reasons that are often not technical.

So spending resources (especially time) on workflow frameworks to support change is possibly a waste until some other (bigger) risks are mitigated. It's a form of YAGNI. The MVP is usually a form of cowboy hacking (hard-coded workflow), and if the project makes it to the next round, the company will manage the technical debt if changes are needed. We as software architects/designers are put off by these kinds of things. Maybe this is the kind of scenario Tim describes in the first post?

Technical debt due to brittle design could be a reason a project dies, but I think it's not really the common reason. I believe we as software engineers are biased, because software was less prevalent when the discipline of SE was created. It's still true today that it costs too much to maintain software that lasts a long time. But how many projects last long enough for it to be a problem?

A related topic about traditional CS/SE curriculum not being adequate for the context of startups was discussed recently [1].

Cris

[1] https://spectrum.ieee.org/the-institute/ieee-member-news/software-engineering-grads-lack-the-skills-startups-need

Stephan

Christopher Fuhrman, P.Eng., PhD

Professeur au Département de génie logiciel et des technologies de l'information
ÉTS (École de technologie supérieure)

http://profs.etsmtl.ca/cfuhrman

+1 514 396 8638

L'ÉTS est une constituante de l'Université du Québec

eftomi

Re: Workflows and possible usages

Please see also http://www.workflowpatterns.com/.

Historically, BPMN by itself was regarded difficult to automate. BPEL as a
workflow language (Oracle SOA, Microsoft BizTalk server) can be interpreted
easily by machine, but it is not so easy to learn by the people who have to
model the process. There were some effors to make an automatic translation
from BPMN to BPEL but it didn't succeed - it was a true challenge, and the
platforms that I mentioned in previous email were already coming to the
stage, so this idea went downhill. YAWL is another workflow language
attempt, however it is not widely accepted, in my opinion from similar
reasons as for BPEL.

As I said, a possible direction is to go for BPMN. It can be based on
Workflow Patterns "behind the scenes". Here's an example of this approach in
Python https://github.com/knipknap/SpiffWorkflow. However, the platforms
that I mentioned have "... an embeddable, lightweight process engine in
Java, supporting native BPMN 2.0 execution ..."
(https://en.wikipedia.org/wiki/JBPM) and similar, so a direct support for
BPMN 2.0 shouldn't be too difficult after all.

One of the advantages to build an engine in Pharo around BPMN is that it
would be directly usable by a large community that works on processes
digitalization. They understand the processes and the notation.

Best wishes,
Tomaz

--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

BrunoBB

Re: Workflows and possible usages

In reply to this post by Tim Mackinnon

Hi,

"There was a gemstone workflow engine - it’s not ported, it looked interesting but used a Windows bpml tool and again wasn’t sure if it wanted to own the world, and porting it would be more than I wanted to do."

I think you are talking about BpmFlow (https://github.com/brunobuzzi/BpmFlow) if not excuse me :)

A couple of month ago a created an issue for the port (https://github.com/brunobuzzi/BpmFlow/issues/843) to annotate all information related to a possible port.

The most challenging task is how to persist BpmProcess data within Pharo, use an ORM (GLORP) or use GemStone ?
This task is the most complex.

But first there are some minor issue to tackle:
* Replace GS ReduceConflict classes with normal Pharo collections.
* GsQuery
   - Replace GsQuery with normal blocks 
   - (or) Find out if GsQuery was ported (or it could be ported) to Pharo
* Replace GS Selection Block with normal Pharo blocks. {:each | each.address.state = 'OR'}

Develop in Pharo and Deploy in GemStone strategy can be a very good source of information to solve the above issues.

I use Jade client to directly develop on GemStone/S so a research is needed on "Develop in Pharo and Deploy in GemStone".

Modeling tools:
The goal is to be able to import any XPDL file exported by any BPMN tool. 
For now we use Bizagi but others tools that export to XPDL standard can be added.
This will require some effort but is possible (we do an extensive use of Bizagi extended attributes but if the BPMN tool to be added support some kind of mechanism to add custom attributes to tasks there will be no problem).

regards,
bruno

Esteban A. Maringolo

Re: Workflows and possible usages

Bruno,

There was the MockGemstone package that created "compatible" reduced conflict classes (that provide name compatibility but no RC behavior) and other Gemstone globals (such as System) that you can use.

The develop in Pharo deploy in Gemstone is used by a few projects that I'm aware of.

As for storing the rules, I rather go with Fuel or GLORP+SQlite. I don't know how complex the object graph can get. But these two options are the most easy to setup, since they have no dependencies and are self contained.

Regards,

Esteban A. Maringolo

On Mon, Sep 30, 2019 at 11:56 AM Smalltalk <[hidden email]> wrote:

Hi,

"There was a gemstone workflow engine - it’s not ported, it looked interesting but used a Windows bpml tool and again wasn’t sure if it wanted to own the world, and porting it would be more than I wanted to do."

I think you are talking about BpmFlow (https://github.com/brunobuzzi/BpmFlow) if not excuse me :)

A couple of month ago a created an issue for the port (https://github.com/brunobuzzi/BpmFlow/issues/843) to annotate all information related to a possible port.

The most challenging task is how to persist BpmProcess data within Pharo, use an ORM (GLORP) or use GemStone ?
This task is the most complex.

But first there are some minor issue to tackle:
* Replace GS ReduceConflict classes with normal Pharo collections.
* GsQuery
   - Replace GsQuery with normal blocks 
   - (or) Find out if GsQuery was ported (or it could be ported) to Pharo
* Replace GS Selection Block with normal Pharo blocks. {:each | each.address.state = 'OR'}

Develop in Pharo and Deploy in GemStone strategy can be a very good source of information to solve the above issues.

I use Jade client to directly develop on GemStone/S so a research is needed on "Develop in Pharo and Deploy in GemStone".

Modeling tools:
The goal is to be able to import any XPDL file exported by any BPMN tool. 
For now we use Bizagi but others tools that export to XPDL standard can be added.
This will require some effort but is possible (we do an extensive use of Bizagi extended attributes but if the BPMN tool to be added support some kind of mechanism to add custom attributes to tasks there will be no problem).

regards,
bruno