[Glass] GsDevKit Server Blocks for Thin Client appications ... pre-announcement

A GsDevKit Server Block[1] is a block that is written in-line in client Smalltalk, but is executed in GemStone. For example the following expression is executed in a standard Pharo workspace:

  | shell x y |
  shell := TDShell forSessionNamed: 'devKit'.
  x := 3.
  y := 4.
  shell onServerDo: [ x + y ].

and the `[3 + 4 ]` block is executed in GemStone using the `devKit` session description to log into the stone. The temp vars x and y referenced in the server block and defined in Pharo are shipped across the wire to GemStone along with block source where the block source is compiled and executed. The result is then shipped back across the wire and returned as the result of #onServerDo: message in Pharo. Pharo execution can continue on using the result. STON[2] is used to serialize the objects that are shipped across the wire.

For any of you familiar with underpinnings of GemTools, Jade or tODE, this is not necessarily ground-breaking technology, however, exposing this capability to developers just may be.

It has been a long standing crime that developers in the Pharo community choose to use  MongoDB and MySQL over GemStone, but frankly the problem is that (until now) we have not had a simple client-based solution for adding GemStone-based persistence for native Pharo applications - the pharo developers have not really had a choice.

Being able to embed server blocks in client code certainly qualifies as simple. Solution(?), well that remains to be seen, but I am optimistic.

As a more concrete example, here's Pharo workspace code that uses NeoCSV running in Pharo to load stone objects in a Dictionary in GemStone:

  'NeoCSVBenchmark.csv' asFileReference
  readStreamDo: [ :stream |
  | reader converter buffer bufCnt numRecords records |
  converter := [ :string | NeoNumberParser parse: string ].
  reader := NeoCSVReader on: (ZnBufferedReadStream on: stream).
  reader
    recordClass: NeoCSVTestObject;
    addIntegerField: #'x:';
    addIntegerField: #'y:';
    addIntegerField: #'z:'.
  buffer := Array new: 1000.
  bufCnt := 0.
  [ reader atEnd ]
    whileFalse: [
      bufCnt := bufCnt + 1.
      buffer at: bufCnt put: reader next.
      bufCnt = buffer size
        ifTrue: [
          numRecords := bufCnt.
          records := buffer.
          DevKitShell
            onServerDo: [
              1 to: numRecords do: [ :index |
                | record |
                record := records at: index.
                NeoCSVDictionary at: record x put: record ].
              System commitTransaction.
              nil ].
          bufCnt := 0 ] ] ].
   DevKitShell onServerDo: [ System commitTransaction ]

The code ships 1000 instances of NeoCSVTestObject at a pop to GemStone. Using the above technique, one can easily arrange to store some pretty large object graphs in GemStone ... Efficient queries based on standard Smalltalk can be written on the client and transparently performed in GemStone (see the GsDevKitServerBlocks doc[1] for the complete example).

Server blocks do not duplicate the functionality GemBuilder for Smalltalk[6][7] which provides transparent replication and maintenance of objects between the client and server. With server blocks you end up with disconnected copies of server objects.

Because of this disconnect, I think the best way to architect an application using server blocks, is to plan on "executing all business logic" on the server --- If you are using an MVC pattern, the M would primarily be managed on the server while the VC would be managed on the client.

As an application evolves, the code can migrate back and forth between client and server as needed.

Most of the server blocks code leverages tODE and has been in use for several years. The code that spelunks in the block structure and extracts the _value_ of temp variables is only a couple of days old and has some pretty rough edges (notice the odd placement of temp variables and declarations in the above example).

The server-side debugger and inspectors, etc. will use tODE (at least for now) ...  in the server blocks doc[1] I demonstrate an #exportClassToServer: to illustrate the potential to share code in weird and wonderful ways between the client and server.

If you have the interest/opportunity to take this code for a spin, let me know. I have written instructions[5] for installing the experimental Roassal Visualization code[4] (GemStone and Pharo3.0 or Pharo4.0) for Pierre Chanson and those instructions can be used for doing work with GsDevKit
Server Blocks. There are a handful of obvious things that need to be done:
  - connection pools
  - coordinated client/server debuggers
  - client-side exception handlers for server errors
  - more???
and if folks express interest in start to do exploratory work with server blocks, then I will make time to provide support.

I am hoping to have something to announce by Smalltalks in November,so it would be useful if some experienced GemStoners tried things out before then...

I do have to finish up the documentation for GsDevKitHome 1.0.0 and tODE 0.1.0 and I'm also committed to doing some work on the 3.3 GemStone release, so we'll see how that goes:)

I also think that server blocks can be very useful for the "develop in Pharo, deploy in GemSstone" crowd, since it will be possible to write "pharo-based scripts" to perform server-side tasks ...

Questions or comments?

Dale

[1] https://github.com/GsDevKit/gsDevKitHome/blob/dev/docs/articles/gsDevKitServerBlocks.md#gsdevkit-server-blocks
[2] https://github.com/GsDevKit/ston#ston---smalltalk-object-notation
[3] https://vimeo.com/123261640
[4] https://github.com/GsDevKit/gsDevKitHome/tree/dev/projects/roassal#roassal-visualization
[5] https://github.com/GsDevKit/gsDevKitHome/blob/dev/projects/roassal/devBootstrap.md
[6] http://gemtalksystems.com/products/gbs-vw/
[7] http://gemtalksystems.com/products/gbs-va/

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Hi Dale,

that is amazing news! Thank you!

There is one thing that I would love to try out!
Could you please compile the needed communication dlls for RaspbianOS on ARM architecture.

I would love to try out to connect ScratchOnPi with the needed tODE layer and serverBlocks during the Spring CampSmalltalk in Nanaimo,BC Canada end of April.

The only thing I am not really aware of is the question if it is possible at all to load the tODE communication layers into a Squeak image? Would it?

It is really too unfortunate that Pharo's VirtualGPU needs NativeBoost which is not available for the ARM VMs.... The other thing I would love to try out is having a Gemstone Pharo VirtualGPU tODE-ServerBlock Parallella board setup.

A GsDevKit Server Block[1] is a block that is written in-line in client Smalltalk, but is executed in GemStone. For example the following expression is executed in a standard Pharo workspace:

  | shell x y |
  shell := TDShell forSessionNamed: 'devKit'.
  x := 3.
  y := 4.
  shell onServerDo: [ x + y ].

and the `[3 + 4 ]` block is executed in GemStone using the `devKit` session description to log into the stone. The temp vars x and y referenced in the server block and defined in Pharo are shipped across the wire to GemStone along with block source where the block source is compiled and executed. The result is then shipped back across the wire and returned as the result of #onServerDo: message in Pharo. Pharo execution can continue on using the result. STON[2] is used to serialize the objects that are shipped across the wire.

For any of you familiar with underpinnings of GemTools, Jade or tODE, this is not necessarily ground-breaking technology, however, exposing this capability to developers just may be.

It has been a long standing crime that developers in the Pharo community choose to use  MongoDB and MySQL over GemStone, but frankly the problem is that (until now) we have not had a simple client-based solution for adding GemStone-based persistence for native Pharo applications - the pharo developers have not really had a choice.

Being able to embed server blocks in client code certainly qualifies as simple. Solution(?), well that remains to be seen, but I am optimistic.

As a more concrete example, here's Pharo workspace code that uses NeoCSV running in Pharo to load stone objects in a Dictionary in GemStone:

  'NeoCSVBenchmark.csv' asFileReference
  readStreamDo: [ :stream |
  | reader converter buffer bufCnt numRecords records |
  converter := [ :string | NeoNumberParser parse: string ].
  reader := NeoCSVReader on: (ZnBufferedReadStream on: stream).
  reader
    recordClass: NeoCSVTestObject;
    addIntegerField: #'x:';
    addIntegerField: #'y:';
    addIntegerField: #'z:'.
  buffer := Array new: 1000.
  bufCnt := 0.
  [ reader atEnd ]
    whileFalse: [
      bufCnt := bufCnt + 1.
      buffer at: bufCnt put: reader next.
      bufCnt = buffer size
        ifTrue: [
          numRecords := bufCnt.
          records := buffer.
          DevKitShell
            onServerDo: [
              1 to: numRecords do: [ :index |
                | record |
                record := records at: index.
                NeoCSVDictionary at: record x put: record ].
              System commitTransaction.
              nil ].
          bufCnt := 0 ] ] ].
   DevKitShell onServerDo: [ System commitTransaction ]

The code ships 1000 instances of NeoCSVTestObject at a pop to GemStone. Using the above technique, one can easily arrange to store some pretty large object graphs in GemStone ... Efficient queries based on standard Smalltalk can be written on the client and transparently performed in GemStone (see the GsDevKitServerBlocks doc[1] for the complete example).

Server blocks do not duplicate the functionality GemBuilder for Smalltalk[6][7] which provides transparent replication and maintenance of objects between the client and server. With server blocks you end up with disconnected copies of server objects.

Because of this disconnect, I think the best way to architect an application using server blocks, is to plan on "executing all business logic" on the server --- If you are using an MVC pattern, the M would primarily be managed on the server while the VC would be managed on the client.

As an application evolves, the code can migrate back and forth between client and server as needed.

Most of the server blocks code leverages tODE and has been in use for several years. The code that spelunks in the block structure and extracts the _value_ of temp variables is only a couple of days old and has some pretty rough edges (notice the odd placement of temp variables and declarations in the above example).

The server-side debugger and inspectors, etc. will use tODE (at least for now) ...  in the server blocks doc[1] I demonstrate an #exportClassToServer: to illustrate the potential to share code in weird and wonderful ways between the client and server.

If you have the interest/opportunity to take this code for a spin, let me know. I have written instructions[5] for installing the experimental Roassal Visualization code[4] (GemStone and Pharo3.0 or Pharo4.0) for Pierre Chanson and those instructions can be used for doing work with GsDevKit
Server Blocks. There are a handful of obvious things that need to be done:
  - connection pools
  - coordinated client/server debuggers
  - client-side exception handlers for server errors
  - more???
and if folks express interest in start to do exploratory work with server blocks, then I will make time to provide support.

I am hoping to have something to announce by Smalltalks in November,so it would be useful if some experienced GemStoners tried things out before then...

I do have to finish up the documentation for GsDevKitHome 1.0.0 and tODE 0.1.0 and I'm also committed to doing some work on the 3.3 GemStone release, so we'll see how that goes:)

I also think that server blocks can be very useful for the "develop in Pharo, deploy in GemSstone" crowd, since it will be possible to write "pharo-based scripts" to perform server-side tasks ...

Questions or comments?

Dale

[1] https://github.com/GsDevKit/gsDevKitHome/blob/dev/docs/articles/gsDevKitServerBlocks.md#gsdevkit-server-blocks
[2] https://github.com/GsDevKit/ston#ston---smalltalk-object-notation
[3] https://vimeo.com/123261640
[4] https://github.com/GsDevKit/gsDevKitHome/tree/dev/projects/roassal#roassal-visualization
[5] https://github.com/GsDevKit/gsDevKitHome/blob/dev/projects/roassal/devBootstrap.md
[6] http://gemtalksystems.com/products/gbs-vw/
[7] http://gemtalksystems.com/products/gbs-va/

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Hi Dale,

which are the branches on has to load in order to get this package in a tODE image?

I am missing he TDShell forSessionNamed: method.

Am I right that this would be the implementation?
TDShell class >>
forSessionNamed: aSessionName
  ^ self sessionDescription: (TDSessionDescription importFrom: self sessionDescriptionHome, aSessionName)

Cheers
Sebastian

On 2015-04-15 10:23 PM, Dale Henrichs via Glass wrote:

A GsDevKit Server Block[1] is a block that is written in-line in client Smalltalk, but is executed in GemStone. For example the following expression is executed in a standard Pharo workspace:

  | shell x y |
  shell := TDShell forSessionNamed: 'devKit'.
  x := 3.
  y := 4.
  shell onServerDo: [ x + y ].

and the `[3 + 4 ]` block is executed in GemStone using the `devKit` session description to log into the stone. The temp vars x and y referenced in the server block and defined in Pharo are shipped across the wire to GemStone along with block source where the block source is compiled and executed. The result is then shipped back across the wire and returned as the result of #onServerDo: message in Pharo. Pharo execution can continue on using the result. STON[2] is used to serialize the objects that are shipped across the wire.

For any of you familiar with underpinnings of GemTools, Jade or tODE, this is not necessarily ground-breaking technology, however, exposing this capability to developers just may be.

It has been a long standing crime that developers in the Pharo community choose to use  MongoDB and MySQL over GemStone, but frankly the problem is that (until now) we have not had a simple client-based solution for adding GemStone-based persistence for native Pharo applications - the pharo developers have not really had a choice.

Being able to embed server blocks in client code certainly qualifies as simple. Solution(?), well that remains to be seen, but I am optimistic.

As a more concrete example, here's Pharo workspace code that uses NeoCSV running in Pharo to load stone objects in a Dictionary in GemStone:

  'NeoCSVBenchmark.csv' asFileReference
  readStreamDo: [ :stream |
  | reader converter buffer bufCnt numRecords records |
  converter := [ :string | NeoNumberParser parse: string ].
  reader := NeoCSVReader on: (ZnBufferedReadStream on: stream).
  reader
    recordClass: NeoCSVTestObject;
    addIntegerField: #'x:';
    addIntegerField: #'y:';
    addIntegerField: #'z:'.
  buffer := Array new: 1000.
  bufCnt := 0.
  [ reader atEnd ]
    whileFalse: [
      bufCnt := bufCnt + 1.
      buffer at: bufCnt put: reader next.
      bufCnt = buffer size
        ifTrue: [
          numRecords := bufCnt.
          records := buffer.
          DevKitShell
            onServerDo: [
              1 to: numRecords do: [ :index |
                | record |
                record := records at: index.
                NeoCSVDictionary at: record x put: record ].
              System commitTransaction.
              nil ].
          bufCnt := 0 ] ] ].
   DevKitShell onServerDo: [ System commitTransaction ]

The code ships 1000 instances of NeoCSVTestObject at a pop to GemStone. Using the above technique, one can easily arrange to store some pretty large object graphs in GemStone ... Efficient queries based on standard Smalltalk can be written on the client and transparently performed in GemStone (see the GsDevKitServerBlocks doc[1] for the complete example).

Server blocks do not duplicate the functionality GemBuilder for Smalltalk[6][7] which provides transparent replication and maintenance of objects between the client and server. With server blocks you end up with disconnected copies of server objects.

Because of this disconnect, I think the best way to architect an application using server blocks, is to plan on "executing all business logic" on the server --- If you are using an MVC pattern, the M would primarily be managed on the server while the VC would be managed on the client.

As an application evolves, the code can migrate back and forth between client and server as needed.

Most of the server blocks code leverages tODE and has been in use for several years. The code that spelunks in the block structure and extracts the _value_ of temp variables is only a couple of days old and has some pretty rough edges (notice the odd placement of temp variables and declarations in the above example).

The server-side debugger and inspectors, etc. will use tODE (at least for now) ...  in the server blocks doc[1] I demonstrate an #exportClassToServer: to illustrate the potential to share code in weird and wonderful ways between the client and server.

If you have the interest/opportunity to take this code for a spin, let me know. I have written instructions[5] for installing the experimental Roassal Visualization code[4] (GemStone and Pharo3.0 or Pharo4.0) for Pierre Chanson and those instructions can be used for doing work with GsDevKit
Server Blocks. There are a handful of obvious things that need to be done:
  - connection pools
  - coordinated client/server debuggers
  - client-side exception handlers for server errors
  - more???
and if folks express interest in start to do exploratory work with server blocks, then I will make time to provide support.

I am hoping to have something to announce by Smalltalks in November,so it would be useful if some experienced GemStoners tried things out before then...

I do have to finish up the documentation for GsDevKitHome 1.0.0 and tODE 0.1.0 and I'm also committed to doing some work on the 3.3 GemStone release, so we'll see how that goes:)

I also think that server blocks can be very useful for the "develop in Pharo, deploy in GemSstone" crowd, since it will be possible to write "pharo-based scripts" to perform server-side tasks ...

Questions or comments?

Dale

[1] https://github.com/GsDevKit/gsDevKitHome/blob/dev/docs/articles/gsDevKitServerBlocks.md#gsdevkit-server-blocks
[2] https://github.com/GsDevKit/ston#ston---smalltalk-object-notation
[3] https://vimeo.com/123261640
[4] https://github.com/GsDevKit/gsDevKitHome/tree/dev/projects/roassal#roassal-visualization
[5] https://github.com/GsDevKit/gsDevKitHome/blob/dev/projects/roassal/devBootstrap.md
[6] http://gemtalksystems.com/products/gbs-vw/
[7] http://gemtalksystems.com/products/gbs-va/

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

On Tue, Apr 21, 2015 at 1:24 PM, Dale Henrichs <[hidden email]> wrote:

On 04/21/2015 08:57 AM, Mariano Martinez Peck wrote:

Hi Dale,

This is very interesting!!! Not exactly right now, BUT at some point in the future, i was thinking an scenario where this could really be of help. Basically, I have some jobs that would take hours/days to run (for example neural networks implementations for backtesting and automated trading). And these jobs may not read/write much of the data in GemStone. So what I thought is to fire up several Pharo images to split the work and to avoid collapsing the stone that will be used for other things as well.

Yes this type of "light weight" stone access is one of the things that server blocks are good for ... you can login and logout on demand so there is no need to keep sessions alive for long periods of time ...

With that strategy in mind, I needed to somehow resolve the read/write of the little data I need (either REST + json or whatever). But this server blocks idea may be exactly what I need.

Yep ... I think the best model is to keep things simple (thus the emphasis on thin client) ... but the built-in STON access and the ability to execute arbitrary Smalltalk on the server makes server blocks superior to a restful interface ... Obviously for external use REST is the right thing but for trusted clients it's really a good way to go ...

Dale,

I fully agree with all you said. There is something I would really like to know and it's about the exact type of primitive objects that can successfully be serialized from Pharo to GemStone (are these the tests of STONWriterTests?) . That is... the tempvars 'x' and 'y' of your example. I think I remember you saying there were some problems with certain type of objects.... 

 

I hope I have the chance in the future to try this out.

Drop me a line before trying in case the api or dev instructions change[1].

[1] https://github.com/GsDevKit/gsDevKitHome/blob/dev/projects/roassal/devBootstrap.md#roassal-visualization-setup-and-install

Thanks a lot, very interesting.  I have some other ideas in mind...like consuming a no-sql or sql db from pharo and push data into gemstone (otherwise you would need to find another way). Anyway...lots of ideas come to my mind where this could be useful.

That's what is nice about server Blocks ... they provide a low-level and simple interface to GemStone upon which interesting capabilities can be built ...

Dale

--

Mariano
http://marianopeck.wordpress.com

On Thu, Apr 16, 2015 at 2:23 AM, Dale Henrichs via Glass <[hidden email]> wrote:

A GsDevKit Server Block[1] is a block that is written in-line in client Smalltalk, but is executed in GemStone. For example the following expression is executed in a standard Pharo workspace:

  | shell x y |
  shell := TDShell forSessionNamed: 'devKit'.
  x := 3.
  y := 4.
  shell onServerDo: [ x + y ].

Dale, 

I know (because I already asked a few months/years ago) that from a stone X you can do a remote login on stone Y and execute stuff in Y. But now I wonder....could server blocks also work for gemstone-gemstone? (my gut feelings tell me that yes) I mean, could I run the above code from GemStone itself?   That would automatically resolve all the remote login stuff and the ston serialization.

Mariano,

Depending on what you mean by "execute stuff on server Y", 3.2 includes something called GsExternalSession. It it capable of executing Smalltalk specified in  Block on a separate session against the same stone or a different one.

It doesn't support copying object graphs, but if your definition of execute stuff has limited requirements for exchanging data, it could be what you are looking for.
See the 3.2 documentation for details.

Thanks again Dale !

here are three small examples using these blocks in the roassal workspace of tODE:  https://vimeo.com/126435220

I highlight here the server blocks in red.

The first one is a Mondrian view of the class Collection (server side) with all it's subclasses.

The classes are processed in the block and stocked statically on the variable "classes".

--------------------------------------------------------------------------------------------------------------------------------------
| b classes |
classes := shell onServerDo:[Collection withAllSubclasses collect: [ :c | c name -> {(c superClass name). ((c instVarNamed: #methDicts asString) numElements)}]].

b := RTMondrian new.
b shape circle.
b nodes: classes.
b edges connectFrom: [:e | classes detect: [ :c | c name = e value first ] ifNone:[ nil ]] .

b shape bezierLineFollowing: [:e | classes detect: [ :c | c name = e value first ] ifNone:[ nil ]];
     color: (Color blue alpha: 0.2).
   

b normalizer
    normalizeSize: [:e | e value second ] using: #sqrt;
    normalizeColor: [:e | e value second ] using: (Array with: Color green with: Color red) using: #sqrt.
b layout cluster.
b build.
b view open
--------------------------------------------------------------------------------------------------------------------------------------

The second script use the RTExploraBuilder to do the same process, but this time the builder call dynamically the server on clics (we are using the oop of objects).

--------------------------------------------------------------------------------------------------------------------------------------
| builder |
    builder := RTExploraBuilder new.
    builder shape circle
        size: 30;
        color: (Color blue alpha: 0.5);
        if: [ :cls |
            (shell onServerDo: [ (Object _objectForOop: (cls value)) subclasses size]) = 0 ] fillColor: (Color red alpha: 0.5).
    builder
        layout: [RTClusterLayout new horizontalGap: 80];
        onClickExplore: [ :cls |
            shell onServerDo: [ ((Object _objectForOop: (cls value)) subclasses collect: [:c | c asString -> c asOop]) asArray ]
            ];
        withPopup: [:cls | cls key];
        dragChildren;
        node: (shell onServerDo: [Collection asString -> Collection asOop]);
        open.
--------------------------------------------------------------------------------------------------------------------------------------

The last script is about users. This need the last version of Roassal. We present some selection menus, a button and a radar Chart. Created with the RTMenuBuilder the button add a new user on the server with the characteristics selected in the other menus and present the characteristics of the added users on a Kiviat chart on the view.

--------------------------------------------------------------------------------------------------------------------------------------
|nameMenus privilegeMenus groupMenus|
    v := RTView new.
   
    kiv := RTKiviatBuilder new view: v.
    n := RTMultiLinearColorForIdentity new objects: (kiv objects).
    kiv shape circle color: [ :value | n rtValue: value kiviatNode named]; size: 10.


    kiv addMetric: [ :v | v second size] max: 5 title: 'groups'.
    kiv addMetric: [ :v | v third size] max: 5 title: 'privileges'.
    kiv addMetric: [ :v | v first size] max: 10 title: 'name'.


    kiv activatePolygons.
    kiv build.
   
    b := RTMenuBuilder new view: v.
   
    nameMenus := Array with: ('Henri'->[:m |]) with: ('Bob'->[:m |]) with: ('Robert'->[:m |]).
    groupMenus := Array with: ('Subscribers'->[:m |]) with:('Publishers'->[:m |]).
    privilegeMenus := Array with: ('ObsoleteStatistics'->[:m |]) with:('UserPassword'->[:m |]) with:('SessionAccess'->[:m |]).

   
    b menu: 'add User' callback: [
        |name groups privileges |
       
        name := (nameMenus detect: [ :m | m selected ]) name.
        groups := (groupMenus select: [:m | m selected]) collect: [:g | g name].
        privileges := (privilegeMenus select: [:m | m selected])  collect: [:p | p name].
        shell onServerDo: [
            AllUsers addNewUserWithId: name password: '' defaultObjectSecurityPolicy: nil privileges: privileges inGroups: groups .
            System commitTransaction.].
        kiv addDynamicObject: (Array with: (name) with: (groups) with: (privileges))
].

    nameMenus := b menu: 'user' subcheckmenus: nameMenus background: (Color red alpha: 0.3).
    RTMenuGroup on: nameMenus.
    nameMenus first selected: true.


    groupMenus := b menu: 'groups' subcheckmenus: groupMenus background: (Color blue alpha: 0.3).
    groupMenus first selected: true.
   

    privilegeMenus := b menu: 'privileges' subcheckmenus: privilegeMenus background: (Color green alpha: 0.3).
   
    privilegeMenus first selected: true.
   
    b view open.
--------------------------------------------------------------------------------------------------------------------------------------

 Pierre

 

2015-04-23 19:58 GMT+02:00 Mariano Martinez Peck via Glass <[hidden email]>:

On Wed, Apr 22, 2015 at 1:53 AM, Richard Sargent <[hidden email]> wrote:

Mariano,

Depending on what you mean by "execute stuff on server Y", 3.2 includes something called GsExternalSession. It it capable of executing Smalltalk specified in  Block on a separate session against the same stone or a different one.

It doesn't support copying object graphs, but if your definition of execute stuff has limited requirements for exchanging data, it could be what you are looking for.
See the 3.2 documentation for details.

Good to know too. Thanks Richard. 

 

On Apr 21, 2015 7:01 PM, "Mariano Martinez Peck via Glass" <[hidden email]> wrote:

On Thu, Apr 16, 2015 at 2:23 AM, Dale Henrichs via Glass <[hidden email]> wrote:

A GsDevKit Server Block[1] is a block that is written in-line in client Smalltalk, but is executed in GemStone. For example the following expression is executed in a standard Pharo workspace:

  | shell x y |
  shell := TDShell forSessionNamed: 'devKit'.
  x := 3.
  y := 4.
  shell onServerDo: [ x + y ].

Dale, 

I know (because I already asked a few months/years ago) that from a stone X you can do a remote login on stone Y and execute stuff in Y. But now I wonder....could server blocks also work for gemstone-gemstone? (my gut feelings tell me that yes) I mean, could I run the above code from GemStone itself?   That would automatically resolve all the remote login stuff and the ston serialization. 

 

Thanks in advance, 

and the `[3 + 4 ]` block is executed in GemStone using the `devKit` session description to log into the stone. The temp vars x and y referenced in the server block and defined in Pharo are shipped across the wire to GemStone along with block source where the block source is compiled and executed. The result is then shipped back across the wire and returned as the result of #onServerDo: message in Pharo. Pharo execution can continue on using the result. STON[2] is used to serialize the objects that are shipped across the wire.

For any of you familiar with underpinnings of GemTools, Jade or tODE, this is not necessarily ground-breaking technology, however, exposing this capability to developers just may be.

It has been a long standing crime that developers in the Pharo community choose to use  MongoDB and MySQL over GemStone, but frankly the problem is that (until now) we have not had a simple client-based solution for adding GemStone-based persistence for native Pharo applications - the pharo developers have not really had a choice.

Being able to embed server blocks in client code certainly qualifies as simple. Solution(?), well that remains to be seen, but I am optimistic.

As a more concrete example, here's Pharo workspace code that uses NeoCSV running in Pharo to load stone objects in a Dictionary in GemStone:

  'NeoCSVBenchmark.csv' asFileReference
  readStreamDo: [ :stream |
  | reader converter buffer bufCnt numRecords records |
  converter := [ :string | NeoNumberParser parse: string ].
  reader := NeoCSVReader on: (ZnBufferedReadStream on: stream).
  reader
    recordClass: NeoCSVTestObject;
    addIntegerField: #'x:';
    addIntegerField: #'y:';
    addIntegerField: #'z:'.
  buffer := Array new: 1000.
  bufCnt := 0.
  [ reader atEnd ]
    whileFalse: [
      bufCnt := bufCnt + 1.
      buffer at: bufCnt put: reader next.
      bufCnt = buffer size
        ifTrue: [
          numRecords := bufCnt.
          records := buffer.
          DevKitShell
            onServerDo: [
              1 to: numRecords do: [ :index |
                | record |
                record := records at: index.
                NeoCSVDictionary at: record x put: record ].
              System commitTransaction.
              nil ].
          bufCnt := 0 ] ] ].
   DevKitShell onServerDo: [ System commitTransaction ]

The code ships 1000 instances of NeoCSVTestObject at a pop to GemStone. Using the above technique, one can easily arrange to store some pretty large object graphs in GemStone ... Efficient queries based on standard Smalltalk can be written on the client and transparently performed in GemStone (see the GsDevKitServerBlocks doc[1] for the complete example).

Server blocks do not duplicate the functionality GemBuilder for Smalltalk[6][7] which provides transparent replication and maintenance of objects between the client and server. With server blocks you end up with disconnected copies of server objects.

Because of this disconnect, I think the best way to architect an application using server blocks, is to plan on "executing all business logic" on the server --- If you are using an MVC pattern, the M would primarily be managed on the server while the VC would be managed on the client.

As an application evolves, the code can migrate back and forth between client and server as needed.

Most of the server blocks code leverages tODE and has been in use for several years. The code that spelunks in the block structure and extracts the _value_ of temp variables is only a couple of days old and has some pretty rough edges (notice the odd placement of temp variables and declarations in the above example).

The server-side debugger and inspectors, etc. will use tODE (at least for now) ...  in the server blocks doc[1] I demonstrate an #exportClassToServer: to illustrate the potential to share code in weird and wonderful ways between the client and server.

If you have the interest/opportunity to take this code for a spin, let me know. I have written instructions[5] for installing the experimental Roassal Visualization code[4] (GemStone and Pharo3.0 or Pharo4.0) for Pierre Chanson and those instructions can be used for doing work with GsDevKit
Server Blocks. There are a handful of obvious things that need to be done:
  - connection pools
  - coordinated client/server debuggers
  - client-side exception handlers for server errors
  - more???
and if folks express interest in start to do exploratory work with server blocks, then I will make time to provide support.

I am hoping to have something to announce by Smalltalks in November,so it would be useful if some experienced GemStoners tried things out before then...

I do have to finish up the documentation for GsDevKitHome 1.0.0 and tODE 0.1.0 and I'm also committed to doing some work on the 3.3 GemStone release, so we'll see how that goes:)

I also think that server blocks can be very useful for the "develop in Pharo, deploy in GemSstone" crowd, since it will be possible to write "pharo-based scripts" to perform server-side tasks ...

Questions or comments?

Dale

[1] https://github.com/GsDevKit/gsDevKitHome/blob/dev/docs/articles/gsDevKitServerBlocks.md#gsdevkit-server-blocks
[2] https://github.com/GsDevKit/ston#ston---smalltalk-object-notation
[3] https://vimeo.com/123261640
[4] https://github.com/GsDevKit/gsDevKitHome/tree/dev/projects/roassal#roassal-visualization
[5] https://github.com/GsDevKit/gsDevKitHome/blob/dev/projects/roassal/devBootstrap.md
[6] http://gemtalksystems.com/products/gbs-vw/
[7] http://gemtalksystems.com/products/gbs-va/

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Hi Pierre,

I dabbled around with the serverBlocks during CampSmalltalk, too and I like your ObjectPointer solution.

I just wanted to ask whethere there is some kind of lazy initialiyation in Roassal? I encountered some issues.

I also want to report a bug in the implementation and will file it as soo nas I am working on this again.
For the executeOnServer code string creation there is printString used,.... this results in issues once you would like to upload longer csv strings because they are trimmed and end with "...".

I also have trouble using this functionality from outside the workspace.
E.g. this is more or less the same implementatin as the one in the example and still the onserverDo: Block return an exception complaining about a Parameter 1.


'mycsvFile.csv' asFileReference readStreamDo: [ :stream |  | records numRecords line buffer bufCnt|
        stream nextLine.
         buffer := Array new: 10000.
  bufCnt := 0.
    line:= stream nextLine.
  [ line notNil ]
    whileFalse: [
      bufCnt := bufCnt + 1.
      buffer at: bufCnt put: line.
      bufCnt = buffer size
        ifTrue: [
          numRecords := bufCnt.
          records := buffer.
          DevKitShell
            onServerDo: [
              1 to: numRecords do: [ :index |
                | record |
                record := records at: index.
                MyLineprocessor processLine: record.].
              MySystemExtensionClass markForCollectionAndCommit .
              nil ].
          bufCnt := 0 ].
    line:= stream nextLine].

 ].

Did you encounter these problems during your tests, too?

I also want to add that there seems to be a limitation on connections this interface can handle. During debugging I accidently didn't qiot the shell properly and it doesn'T take long until the server had to be restarted due to too many open connections errors.