[Glass] GsDevKit Server Blocks for Thin Client appications ... pre-announcement

+1 :)

On 04/30/2015 10:26 AM, Mariano Martinez Peck via Glass wrote:

Hi Pierre,

Do you want to do a super killer example of using Roassal and ServerBlocks?

As you may known, since in Smalltalk you cannot delete an object (compared to relational DBs where you do delete rows), removing all references to an object so that is GCed,  sometimes is not a walk in park. Ben was tired of using the Pharo Explorer for that, and he did this: http://smalltalkhub.com/#!/~BenComan/PointerDetective

If that is valuable in Pharo, imagine in GemStone. And now, with the serverBlocks, that would be very easy. 

Just see today's email of Otto saying he couldn't GC an object.

To find pointers to an object in GemStone:

SystemRepository findReferencePathToObject: 123456789 asObject

yeah...maybe I should try it myself :)

On Wed, Apr 29, 2015 at 7:42 PM, Pierre CHANSON <[hidden email]> wrote:

Thanks again Dale !

here are three small examples using these blocks in the roassal workspace of tODE:  https://vimeo.com/126435220

I highlight here the server blocks in red.

The first one is a Mondrian view of the class Collection (server side) with all it's subclasses.

The classes are processed in the block and stocked statically on the variable "classes".

--------------------------------------------------------------------------------------------------------------------------------------
| b classes |
classes := shell onServerDo:[Collection withAllSubclasses collect: [ :c | c name -> {(c superClass name). ((c instVarNamed: #methDicts asString) numElements)}]].

b := RTMondrian new.
b shape circle.
b nodes: classes.
b edges connectFrom: [:e | classes detect: [ :c | c name = e value first ] ifNone:[ nil ]] .

b shape bezierLineFollowing: [:e | classes detect: [ :c | c name = e value first ] ifNone:[ nil ]];
     color: (Color blue alpha: 0.2).
   

b normalizer
    normalizeSize: [:e | e value second ] using: #sqrt;
    normalizeColor: [:e | e value second ] using: (Array with: Color green with: Color red) using: #sqrt.
b layout cluster.
b build.
b view open
--------------------------------------------------------------------------------------------------------------------------------------

The second script use the RTExploraBuilder to do the same process, but this time the builder call dynamically the server on clics (we are using the oop of objects).

--------------------------------------------------------------------------------------------------------------------------------------
| builder |
    builder := RTExploraBuilder new.
    builder shape circle
        size: 30;
        color: (Color blue alpha: 0.5);
        if: [ :cls |
            (shell onServerDo: [ (Object _objectForOop: (cls value)) subclasses size]) = 0 ] fillColor: (Color red alpha: 0.5).
    builder
        layout: [RTClusterLayout new horizontalGap: 80];
        onClickExplore: [ :cls |
            shell onServerDo: [ ((Object _objectForOop: (cls value)) subclasses collect: [:c | c asString -> c asOop]) asArray ]
            ];
        withPopup: [:cls | cls key];
        dragChildren;
        node: (shell onServerDo: [Collection asString -> Collection asOop]);
        open.
--------------------------------------------------------------------------------------------------------------------------------------

The last script is about users. This need the last version of Roassal. We present some selection menus, a button and a radar Chart. Created with the RTMenuBuilder the button add a new user on the server with the characteristics selected in the other menus and present the characteristics of the added users on a Kiviat chart on the view.

--------------------------------------------------------------------------------------------------------------------------------------
|nameMenus privilegeMenus groupMenus|
    v := RTView new.
   
    kiv := RTKiviatBuilder new view: v.
    n := RTMultiLinearColorForIdentity new objects: (kiv objects).
    kiv shape circle color: [ :value | n rtValue: value kiviatNode named]; size: 10.


    kiv addMetric: [ :v | v second size] max: 5 title: 'groups'.
    kiv addMetric: [ :v | v third size] max: 5 title: 'privileges'.
    kiv addMetric: [ :v | v first size] max: 10 title: 'name'.


    kiv activatePolygons.
    kiv build.
   
    b := RTMenuBuilder new view: v.
   
    nameMenus := Array with: ('Henri'->[:m |]) with: ('Bob'->[:m |]) with: ('Robert'->[:m |]).
    groupMenus := Array with: ('Subscribers'->[:m |]) with:('Publishers'->[:m |]).
    privilegeMenus := Array with: ('ObsoleteStatistics'->[:m |]) with:('UserPassword'->[:m |]) with:('SessionAccess'->[:m |]).

   
    b menu: 'add User' callback: [
        |name groups privileges |
       
        name := (nameMenus detect: [ :m | m selected ]) name.
        groups := (groupMenus select: [:m | m selected]) collect: [:g | g name].
        privileges := (privilegeMenus select: [:m | m selected])  collect: [:p | p name].
        shell onServerDo: [
            AllUsers addNewUserWithId: name password: '' defaultObjectSecurityPolicy: nil privileges: privileges inGroups: groups .
            System commitTransaction.].
        kiv addDynamicObject: (Array with: (name) with: (groups) with: (privileges))
].

    nameMenus := b menu: 'user' subcheckmenus: nameMenus background: (Color red alpha: 0.3).
    RTMenuGroup on: nameMenus.
    nameMenus first selected: true.


    groupMenus := b menu: 'groups' subcheckmenus: groupMenus background: (Color blue alpha: 0.3).
    groupMenus first selected: true.
   

    privilegeMenus := b menu: 'privileges' subcheckmenus: privilegeMenus background: (Color green alpha: 0.3).
   
    privilegeMenus first selected: true.
   
    b view open.
--------------------------------------------------------------------------------------------------------------------------------------

 Pierre

 

2015-04-23 19:58 GMT+02:00 Mariano Martinez Peck via Glass <[hidden email]>:

On Wed, Apr 22, 2015 at 1:53 AM, Richard Sargent <[hidden email]> wrote:

Mariano,

Depending on what you mean by "execute stuff on server Y", 3.2 includes something called GsExternalSession. It it capable of executing Smalltalk specified in  Block on a separate session against the same stone or a different one.

It doesn't support copying object graphs, but if your definition of execute stuff has limited requirements for exchanging data, it could be what you are looking for.
See the 3.2 documentation for details.

Good to know too. Thanks Richard. 

 

On Apr 21, 2015 7:01 PM, "Mariano Martinez Peck via Glass" <[hidden email]> wrote:

On Thu, Apr 16, 2015 at 2:23 AM, Dale Henrichs via Glass <[hidden email]> wrote:

A GsDevKit Server Block[1] is a block that is written in-line in client Smalltalk, but is executed in GemStone. For example the following expression is executed in a standard Pharo workspace:

  | shell x y |
  shell := TDShell forSessionNamed: 'devKit'.
  x := 3.
  y := 4.
  shell onServerDo: [ x + y ].

Dale, 

I know (because I already asked a few months/years ago) that from a stone X you can do a remote login on stone Y and execute stuff in Y. But now I wonder....could server blocks also work for gemstone-gemstone? (my gut feelings tell me that yes) I mean, could I run the above code from GemStone itself?   That would automatically resolve all the remote login stuff and the ston serialization. 

 

Thanks in advance, 

and the `[3 + 4 ]` block is executed in GemStone using the `devKit` session description to log into the stone. The temp vars x and y referenced in the server block and defined in Pharo are shipped across the wire to GemStone along with block source where the block source is compiled and executed. The result is then shipped back across the wire and returned as the result of #onServerDo: message in Pharo. Pharo execution can continue on using the result. STON[2] is used to serialize the objects that are shipped across the wire.

For any of you familiar with underpinnings of GemTools, Jade or tODE, this is not necessarily ground-breaking technology, however, exposing this capability to developers just may be.

It has been a long standing crime that developers in the Pharo community choose to use  MongoDB and MySQL over GemStone, but frankly the problem is that (until now) we have not had a simple client-based solution for adding GemStone-based persistence for native Pharo applications - the pharo developers have not really had a choice.

Being able to embed server blocks in client code certainly qualifies as simple. Solution(?), well that remains to be seen, but I am optimistic.

As a more concrete example, here's Pharo workspace code that uses NeoCSV running in Pharo to load stone objects in a Dictionary in GemStone:

  'NeoCSVBenchmark.csv' asFileReference
  readStreamDo: [ :stream |
  | reader converter buffer bufCnt numRecords records |
  converter := [ :string | NeoNumberParser parse: string ].
  reader := NeoCSVReader on: (ZnBufferedReadStream on: stream).
  reader
    recordClass: NeoCSVTestObject;
    addIntegerField: #'x:';
    addIntegerField: #'y:';
    addIntegerField: #'z:'.
  buffer := Array new: 1000.
  bufCnt := 0.
  [ reader atEnd ]
    whileFalse: [
      bufCnt := bufCnt + 1.
      buffer at: bufCnt put: reader next.
      bufCnt = buffer size
        ifTrue: [
          numRecords := bufCnt.
          records := buffer.
          DevKitShell
            onServerDo: [
              1 to: numRecords do: [ :index |
                | record |
                record := records at: index.
                NeoCSVDictionary at: record x put: record ].
              System commitTransaction.
              nil ].
          bufCnt := 0 ] ] ].
   DevKitShell onServerDo: [ System commitTransaction ]

The code ships 1000 instances of NeoCSVTestObject at a pop to GemStone. Using the above technique, one can easily arrange to store some pretty large object graphs in GemStone ... Efficient queries based on standard Smalltalk can be written on the client and transparently performed in GemStone (see the GsDevKitServerBlocks doc[1] for the complete example).

Server blocks do not duplicate the functionality GemBuilder for Smalltalk[6][7] which provides transparent replication and maintenance of objects between the client and server. With server blocks you end up with disconnected copies of server objects.

Because of this disconnect, I think the best way to architect an application using server blocks, is to plan on "executing all business logic" on the server --- If you are using an MVC pattern, the M would primarily be managed on the server while the VC would be managed on the client.

As an application evolves, the code can migrate back and forth between client and server as needed.

Most of the server blocks code leverages tODE and has been in use for several years. The code that spelunks in the block structure and extracts the _value_ of temp variables is only a couple of days old and has some pretty rough edges (notice the odd placement of temp variables and declarations in the above example).

The server-side debugger and inspectors, etc. will use tODE (at least for now) ...  in the server blocks doc[1] I demonstrate an #exportClassToServer: to illustrate the potential to share code in weird and wonderful ways between the client and server.

If you have the interest/opportunity to take this code for a spin, let me know. I have written instructions[5] for installing the experimental Roassal Visualization code[4] (GemStone and Pharo3.0 or Pharo4.0) for Pierre Chanson and those instructions can be used for doing work with GsDevKit
Server Blocks. There are a handful of obvious things that need to be done:
  - connection pools
  - coordinated client/server debuggers
  - client-side exception handlers for server errors
  - more???
and if folks express interest in start to do exploratory work with server blocks, then I will make time to provide support.

I am hoping to have something to announce by Smalltalks in November,so it would be useful if some experienced GemStoners tried things out before then...

I do have to finish up the documentation for GsDevKitHome 1.0.0 and tODE 0.1.0 and I'm also committed to doing some work on the 3.3 GemStone release, so we'll see how that goes:)

I also think that server blocks can be very useful for the "develop in Pharo, deploy in GemSstone" crowd, since it will be possible to write "pharo-based scripts" to perform server-side tasks ...

Questions or comments?

Dale

[1] https://github.com/GsDevKit/gsDevKitHome/blob/dev/docs/articles/gsDevKitServerBlocks.md#gsdevkit-server-blocks
[2] https://github.com/GsDevKit/ston#ston---smalltalk-object-notation
[3] https://vimeo.com/123261640
[4] https://github.com/GsDevKit/gsDevKitHome/tree/dev/projects/roassal#roassal-visualization
[5] https://github.com/GsDevKit/gsDevKitHome/blob/dev/projects/roassal/devBootstrap.md
[6] http://gemtalksystems.com/products/gbs-vw/
[7] http://gemtalksystems.com/products/gbs-va/

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Oh thanks ! I like this method now,  I am going to give a try :)

2015-04-30 14:59 GMT-03:00 Dale Henrichs via Glass <[hidden email]>:

+1 :)

On 04/30/2015 10:26 AM, Mariano Martinez Peck via Glass wrote:

Hi Pierre,

Do you want to do a super killer example of using Roassal and ServerBlocks?

As you may known, since in Smalltalk you cannot delete an object (compared to relational DBs where you do delete rows), removing all references to an object so that is GCed,  sometimes is not a walk in park. Ben was tired of using the Pharo Explorer for that, and he did this: http://smalltalkhub.com/#!/~BenComan/PointerDetective

If that is valuable in Pharo, imagine in GemStone. And now, with the serverBlocks, that would be very easy. 

Just see today's email of Otto saying he couldn't GC an object.

To find pointers to an object in GemStone:

SystemRepository findReferencePathToObject: 123456789 asObject

yeah...maybe I should try it myself :)

On Wed, Apr 29, 2015 at 7:42 PM, Pierre CHANSON <[hidden email]> wrote:

Thanks again Dale !

here are three small examples using these blocks in the roassal workspace of tODE:  https://vimeo.com/126435220

I highlight here the server blocks in red.

The first one is a Mondrian view of the class Collection (server side) with all it's subclasses.

The classes are processed in the block and stocked statically on the variable "classes".

--------------------------------------------------------------------------------------------------------------------------------------
| b classes |
classes := shell onServerDo:[Collection withAllSubclasses collect: [ :c | c name -> {(c superClass name). ((c instVarNamed: #methDicts asString) numElements)}]].

b := RTMondrian new.
b shape circle.
b nodes: classes.
b edges connectFrom: [:e | classes detect: [ :c | c name = e value first ] ifNone:[ nil ]] .

b shape bezierLineFollowing: [:e | classes detect: [ :c | c name = e value first ] ifNone:[ nil ]];
     color: (Color blue alpha: 0.2).
   

b normalizer
    normalizeSize: [:e | e value second ] using: #sqrt;
    normalizeColor: [:e | e value second ] using: (Array with: Color green with: Color red) using: #sqrt.
b layout cluster.
b build.
b view open
--------------------------------------------------------------------------------------------------------------------------------------

The second script use the RTExploraBuilder to do the same process, but this time the builder call dynamically the server on clics (we are using the oop of objects).

--------------------------------------------------------------------------------------------------------------------------------------
| builder |
    builder := RTExploraBuilder new.
    builder shape circle
        size: 30;
        color: (Color blue alpha: 0.5);
        if: [ :cls |
            (shell onServerDo: [ (Object _objectForOop: (cls value)) subclasses size]) = 0 ] fillColor: (Color red alpha: 0.5).
    builder
        layout: [RTClusterLayout new horizontalGap: 80];
        onClickExplore: [ :cls |
            shell onServerDo: [ ((Object _objectForOop: (cls value)) subclasses collect: [:c | c asString -> c asOop]) asArray ]
            ];
        withPopup: [:cls | cls key];
        dragChildren;
        node: (shell onServerDo: [Collection asString -> Collection asOop]);
        open.
--------------------------------------------------------------------------------------------------------------------------------------

The last script is about users. This need the last version of Roassal. We present some selection menus, a button and a radar Chart. Created with the RTMenuBuilder the button add a new user on the server with the characteristics selected in the other menus and present the characteristics of the added users on a Kiviat chart on the view.

--------------------------------------------------------------------------------------------------------------------------------------
|nameMenus privilegeMenus groupMenus|
    v := RTView new.
   
    kiv := RTKiviatBuilder new view: v.
    n := RTMultiLinearColorForIdentity new objects: (kiv objects).
    kiv shape circle color: [ :value | n rtValue: value kiviatNode named]; size: 10.


    kiv addMetric: [ :v | v second size] max: 5 title: 'groups'.
    kiv addMetric: [ :v | v third size] max: 5 title: 'privileges'.
    kiv addMetric: [ :v | v first size] max: 10 title: 'name'.


    kiv activatePolygons.
    kiv build.
   
    b := RTMenuBuilder new view: v.
   
    nameMenus := Array with: ('Henri'->[:m |]) with: ('Bob'->[:m |]) with: ('Robert'->[:m |]).
    groupMenus := Array with: ('Subscribers'->[:m |]) with:('Publishers'->[:m |]).
    privilegeMenus := Array with: ('ObsoleteStatistics'->[:m |]) with:('UserPassword'->[:m |]) with:('SessionAccess'->[:m |]).

   
    b menu: 'add User' callback: [
        |name groups privileges |
       
        name := (nameMenus detect: [ :m | m selected ]) name.
        groups := (groupMenus select: [:m | m selected]) collect: [:g | g name].
        privileges := (privilegeMenus select: [:m | m selected])  collect: [:p | p name].
        shell onServerDo: [
            AllUsers addNewUserWithId: name password: '' defaultObjectSecurityPolicy: nil privileges: privileges inGroups: groups .
            System commitTransaction.].
        kiv addDynamicObject: (Array with: (name) with: (groups) with: (privileges))
].

    nameMenus := b menu: 'user' subcheckmenus: nameMenus background: (Color red alpha: 0.3).
    RTMenuGroup on: nameMenus.
    nameMenus first selected: true.


    groupMenus := b menu: 'groups' subcheckmenus: groupMenus background: (Color blue alpha: 0.3).
    groupMenus first selected: true.
   

    privilegeMenus := b menu: 'privileges' subcheckmenus: privilegeMenus background: (Color green alpha: 0.3).
   
    privilegeMenus first selected: true.
   
    b view open.
--------------------------------------------------------------------------------------------------------------------------------------

 Pierre

 

2015-04-23 19:58 GMT+02:00 Mariano Martinez Peck via Glass <[hidden email]>:

On Wed, Apr 22, 2015 at 1:53 AM, Richard Sargent <[hidden email]> wrote:

Mariano,

Depending on what you mean by "execute stuff on server Y", 3.2 includes something called GsExternalSession. It it capable of executing Smalltalk specified in  Block on a separate session against the same stone or a different one.

It doesn't support copying object graphs, but if your definition of execute stuff has limited requirements for exchanging data, it could be what you are looking for.
See the 3.2 documentation for details.

Good to know too. Thanks Richard. 

 

On Apr 21, 2015 7:01 PM, "Mariano Martinez Peck via Glass" <[hidden email]> wrote:

On Thu, Apr 16, 2015 at 2:23 AM, Dale Henrichs via Glass <[hidden email]> wrote:

A GsDevKit Server Block[1] is a block that is written in-line in client Smalltalk, but is executed in GemStone. For example the following expression is executed in a standard Pharo workspace:

  | shell x y |
  shell := TDShell forSessionNamed: 'devKit'.
  x := 3.
  y := 4.
  shell onServerDo: [ x + y ].

Dale, 

I know (because I already asked a few months/years ago) that from a stone X you can do a remote login on stone Y and execute stuff in Y. But now I wonder....could server blocks also work for gemstone-gemstone? (my gut feelings tell me that yes) I mean, could I run the above code from GemStone itself?   That would automatically resolve all the remote login stuff and the ston serialization. 

 

Thanks in advance, 

and the `[3 + 4 ]` block is executed in GemStone using the `devKit` session description to log into the stone. The temp vars x and y referenced in the server block and defined in Pharo are shipped across the wire to GemStone along with block source where the block source is compiled and executed. The result is then shipped back across the wire and returned as the result of #onServerDo: message in Pharo. Pharo execution can continue on using the result. STON[2] is used to serialize the objects that are shipped across the wire.

For any of you familiar with underpinnings of GemTools, Jade or tODE, this is not necessarily ground-breaking technology, however, exposing this capability to developers just may be.

It has been a long standing crime that developers in the Pharo community choose to use  MongoDB and MySQL over GemStone, but frankly the problem is that (until now) we have not had a simple client-based solution for adding GemStone-based persistence for native Pharo applications - the pharo developers have not really had a choice.

Being able to embed server blocks in client code certainly qualifies as simple. Solution(?), well that remains to be seen, but I am optimistic.

As a more concrete example, here's Pharo workspace code that uses NeoCSV running in Pharo to load stone objects in a Dictionary in GemStone:

  'NeoCSVBenchmark.csv' asFileReference
  readStreamDo: [ :stream |
  | reader converter buffer bufCnt numRecords records |
  converter := [ :string | NeoNumberParser parse: string ].
  reader := NeoCSVReader on: (ZnBufferedReadStream on: stream).
  reader
    recordClass: NeoCSVTestObject;
    addIntegerField: #'x:';
    addIntegerField: #'y:';
    addIntegerField: #'z:'.
  buffer := Array new: 1000.
  bufCnt := 0.
  [ reader atEnd ]
    whileFalse: [
      bufCnt := bufCnt + 1.
      buffer at: bufCnt put: reader next.
      bufCnt = buffer size
        ifTrue: [
          numRecords := bufCnt.
          records := buffer.
          DevKitShell
            onServerDo: [
              1 to: numRecords do: [ :index |
                | record |
                record := records at: index.
                NeoCSVDictionary at: record x put: record ].
              System commitTransaction.
              nil ].
          bufCnt := 0 ] ] ].
   DevKitShell onServerDo: [ System commitTransaction ]

The code ships 1000 instances of NeoCSVTestObject at a pop to GemStone. Using the above technique, one can easily arrange to store some pretty large object graphs in GemStone ... Efficient queries based on standard Smalltalk can be written on the client and transparently performed in GemStone (see the GsDevKitServerBlocks doc[1] for the complete example).

Server blocks do not duplicate the functionality GemBuilder for Smalltalk[6][7] which provides transparent replication and maintenance of objects between the client and server. With server blocks you end up with disconnected copies of server objects.

Because of this disconnect, I think the best way to architect an application using server blocks, is to plan on "executing all business logic" on the server --- If you are using an MVC pattern, the M would primarily be managed on the server while the VC would be managed on the client.

As an application evolves, the code can migrate back and forth between client and server as needed.

Most of the server blocks code leverages tODE and has been in use for several years. The code that spelunks in the block structure and extracts the _value_ of temp variables is only a couple of days old and has some pretty rough edges (notice the odd placement of temp variables and declarations in the above example).

The server-side debugger and inspectors, etc. will use tODE (at least for now) ...  in the server blocks doc[1] I demonstrate an #exportClassToServer: to illustrate the potential to share code in weird and wonderful ways between the client and server.

If you have the interest/opportunity to take this code for a spin, let me know. I have written instructions[5] for installing the experimental Roassal Visualization code[4] (GemStone and Pharo3.0 or Pharo4.0) for Pierre Chanson and those instructions can be used for doing work with GsDevKit
Server Blocks. There are a handful of obvious things that need to be done:
  - connection pools
  - coordinated client/server debuggers
  - client-side exception handlers for server errors
  - more???
and if folks express interest in start to do exploratory work with server blocks, then I will make time to provide support.

I am hoping to have something to announce by Smalltalks in November,so it would be useful if some experienced GemStoners tried things out before then...

I do have to finish up the documentation for GsDevKitHome 1.0.0 and tODE 0.1.0 and I'm also committed to doing some work on the 3.3 GemStone release, so we'll see how that goes:)

I also think that server blocks can be very useful for the "develop in Pharo, deploy in GemSstone" crowd, since it will be possible to write "pharo-based scripts" to perform server-side tasks ...

Questions or comments?

Dale

[1] https://github.com/GsDevKit/gsDevKitHome/blob/dev/docs/articles/gsDevKitServerBlocks.md#gsdevkit-server-blocks
[2] https://github.com/GsDevKit/ston#ston---smalltalk-object-notation
[3] https://vimeo.com/123261640
[4] https://github.com/GsDevKit/gsDevKitHome/tree/dev/projects/roassal#roassal-visualization
[5] https://github.com/GsDevKit/gsDevKitHome/blob/dev/projects/roassal/devBootstrap.md
[6] http://gemtalksystems.com/products/gbs-vw/
[7] http://gemtalksystems.com/products/gbs-va/

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Thanks a lot, great ! I am still learning a lot for now about Gemstone and it's applications (and also Pharo).

On 28 mai 2015, at 19:38, Dale Henrichs <[hidden email]> wrote:

Pierre,

Again, very cool ... I am nearing the end of my (intense) work on GemStone 3.3 so we should touch bases when I am done.

Dale

On 5/25/15 1:47 PM, Pierre CHANSON wrote:

Hi all, finally I found some time to work on this.

first, based on the PointerDetective of Ben (http://smalltalkhub.com/#!/~BenComan/PointerDetective) I did a RPointerDetective using Roassal and usable on a Pharo image.

Then I made a second class adapted to work on tODE, given a TDShell and using server blocks.

Here is a little video of the result: https://vimeo.com/128790347

The prerequisite are the last version of Roassal and the package: http://smalltalkhub.com/#!/~PierreChanson/RPointerDetective

The first part is on a Pharo image (could be a tODE image too) with the script:

--------------------------------------------
ob := 'OBJ'.
ar := { ob. 12 }.
e := (RTBox new) elementOn: ar.
v := RTView new.
v add: e.

RPointerDetective on: ob
-------------------------------------------------

Second part is on tODE, given an oop of server object.

The first script :

RPointerDetective onServerFindAllReferencePathsToObject: 154679809 shell: shell

And the second:

RPointerDetective onServer: 154679809 shell: shell

The first script is using the findAllReferencePathsToObjects:limitObjArray:printToLog:. In that case the graph is directly given by the gemstone method and displayed on the view on clic.

The second script is using the gemstone method findAllReferences. here tODE is calling the method when the object is clicked on the view. This one can be dangerous because we do not take care of the number of object that are going to be displayed...

There is still a lot of interesting things to do, with interactions on the view for example with the possibility to dinamically change the colors given a block or display the pointersFrom of a node or the possibility to open a tODE inspector etc.

cheers,

Pierre

2015-04-30 20:14 GMT-03:00 Pierre CHANSON <[hidden email]>:

Oh thanks ! I like this method now,  I am going to give a try :)

2015-04-30 14:59 GMT-03:00 Dale Henrichs via Glass <[hidden email]>:

+1 :)

On 04/30/2015 10:26 AM, Mariano Martinez Peck via Glass wrote:

Hi Pierre,

Do you want to do a super killer example of using Roassal and ServerBlocks?

As you may known, since in Smalltalk you cannot delete an object (compared to relational DBs where you do delete rows), removing all references to an object so that is GCed,  sometimes is not a walk in park. Ben was tired of using the Pharo Explorer for that, and he did this: http://smalltalkhub.com/#!/~BenComan/PointerDetective

If that is valuable in Pharo, imagine in GemStone. And now, with the serverBlocks, that would be very easy. 

Just see today's email of Otto saying he couldn't GC an object.

To find pointers to an object in GemStone:

SystemRepository findReferencePathToObject: 123456789 asObject

yeah...maybe I should try it myself :)

On Wed, Apr 29, 2015 at 7:42 PM, Pierre CHANSON <[hidden email]> wrote:

Thanks again Dale !

here are three small examples using these blocks in the roassal workspace of tODE:  https://vimeo.com/126435220

I highlight here the server blocks in red.

The first one is a Mondrian view of the class Collection (server side) with all it's subclasses.

The classes are processed in the block and stocked statically on the variable "classes".

--------------------------------------------------------------------------------------------------------------------------------------
| b classes |
classes := shell onServerDo:[Collection withAllSubclasses collect: [ :c | c name -> {(c superClass name). ((c instVarNamed: #methDicts asString) numElements)}]].

b := RTMondrian new.
b shape circle.
b nodes: classes.
b edges connectFrom: [:e | classes detect: [ :c | c name = e value first ] ifNone:[ nil ]] .

b shape bezierLineFollowing: [:e | classes detect: [ :c | c name = e value first ] ifNone:[ nil ]];
     color: (Color blue alpha: 0.2).
   

b normalizer
    normalizeSize: [:e | e value second ] using: #sqrt;
    normalizeColor: [:e | e value second ] using: (Array with: Color green with: Color red) using: #sqrt.
b layout cluster.
b build.
b view open
--------------------------------------------------------------------------------------------------------------------------------------

The second script use the RTExploraBuilder to do the same process, but this time the builder call dynamically the server on clics (we are using the oop of objects).

--------------------------------------------------------------------------------------------------------------------------------------
| builder |
    builder := RTExploraBuilder new.
    builder shape circle
        size: 30;
        color: (Color blue alpha: 0.5);
        if: [ :cls |
            (shell onServerDo: [ (Object _objectForOop: (cls value)) subclasses size]) = 0 ] fillColor: (Color red alpha: 0.5).
    builder
        layout: [RTClusterLayout new horizontalGap: 80];
        onClickExplore: [ :cls |
            shell onServerDo: [ ((Object _objectForOop: (cls value)) subclasses collect: [:c | c asString -> c asOop]) asArray ]
            ];
        withPopup: [:cls | cls key];
        dragChildren;
        node: (shell onServerDo: [Collection asString -> Collection asOop]);
        open.
--------------------------------------------------------------------------------------------------------------------------------------

The last script is about users. This need the last version of Roassal. We present some selection menus, a button and a radar Chart. Created with the RTMenuBuilder the button add a new user on the server with the characteristics selected in the other menus and present the characteristics of the added users on a Kiviat chart on the view.

--------------------------------------------------------------------------------------------------------------------------------------
|nameMenus privilegeMenus groupMenus|
    v := RTView new.
   
    kiv := RTKiviatBuilder new view: v.
    n := RTMultiLinearColorForIdentity new objects: (kiv objects).
    kiv shape circle color: [ :value | n rtValue: value kiviatNode named]; size: 10.


    kiv addMetric: [ :v | v second size] max: 5 title: 'groups'.
    kiv addMetric: [ :v | v third size] max: 5 title: 'privileges'.
    kiv addMetric: [ :v | v first size] max: 10 title: 'name'.


    kiv activatePolygons.
    kiv build.
   
    b := RTMenuBuilder new view: v.
   
    nameMenus := Array with: ('Henri'->[:m |]) with: ('Bob'->[:m |]) with: ('Robert'->[:m |]).
    groupMenus := Array with: ('Subscribers'->[:m |]) with:('Publishers'->[:m |]).
    privilegeMenus := Array with: ('ObsoleteStatistics'->[:m |]) with:('UserPassword'->[:m |]) with:('SessionAccess'->[:m |]).

   
    b menu: 'add User' callback: [
        |name groups privileges |
       
        name := (nameMenus detect: [ :m | m selected ]) name.
        groups := (groupMenus select: [:m | m selected]) collect: [:g | g name].
        privileges := (privilegeMenus select: [:m | m selected])  collect: [:p | p name].
        shell onServerDo: [
            AllUsers addNewUserWithId: name password: '' defaultObjectSecurityPolicy: nil privileges: privileges inGroups: groups .
            System commitTransaction.].
        kiv addDynamicObject: (Array with: (name) with: (groups) with: (privileges))
].

    nameMenus := b menu: 'user' subcheckmenus: nameMenus background: (Color red alpha: 0.3).
    RTMenuGroup on: nameMenus.
    nameMenus first selected: true.


    groupMenus := b menu: 'groups' subcheckmenus: groupMenus background: (Color blue alpha: 0.3).
    groupMenus first selected: true.
   

    privilegeMenus := b menu: 'privileges' subcheckmenus: privilegeMenus background: (Color green alpha: 0.3).
   
    privilegeMenus first selected: true.
   
    b view open.
--------------------------------------------------------------------------------------------------------------------------------------

 Pierre

 

2015-04-23 19:58 GMT+02:00 Mariano Martinez Peck via Glass <[hidden email]>:

On Wed, Apr 22, 2015 at 1:53 AM, Richard Sargent <[hidden email]> wrote:

Mariano,

Depending on what you mean by "execute stuff on server Y", 3.2 includes something called GsExternalSession. It it capable of executing Smalltalk specified in  Block on a separate session against the same stone or a different one.

It doesn't support copying object graphs, but if your definition of execute stuff has limited requirements for exchanging data, it could be what you are looking for.
See the 3.2 documentation for details.

Good to know too. Thanks Richard. 

 

On Apr 21, 2015 7:01 PM, "Mariano Martinez Peck via Glass" <[hidden email]> wrote:

On Thu, Apr 16, 2015 at 2:23 AM, Dale Henrichs via Glass <[hidden email]> wrote:

A GsDevKit Server Block[1] is a block that is written in-line in client Smalltalk, but is executed in GemStone. For example the following expression is executed in a standard Pharo workspace:

  | shell x y |
  shell := TDShell forSessionNamed: 'devKit'.
  x := 3.
  y := 4.
  shell onServerDo: [ x + y ].

Dale, 

I know (because I already asked a few months/years ago) that from a stone X you can do a remote login on stone Y and execute stuff in Y. But now I wonder....could server blocks also work for gemstone-gemstone? (my gut feelings tell me that yes) I mean, could I run the above code from GemStone itself?   That would automatically resolve all the remote login stuff and the ston serialization. 

 

Thanks in advance, 

and the `[3 + 4 ]` block is executed in GemStone using the `devKit` session description to log into the stone. The temp vars x and y referenced in the server block and defined in Pharo are shipped across the wire to GemStone along with block source where the block source is compiled and executed. The result is then shipped back across the wire and returned as the result of #onServerDo: message in Pharo. Pharo execution can continue on using the result. STON[2] is used to serialize the objects that are shipped across the wire.

For any of you familiar with underpinnings of GemTools, Jade or tODE, this is not necessarily ground-breaking technology, however, exposing this capability to developers just may be.

It has been a long standing crime that developers in the Pharo community choose to use  MongoDB and MySQL over GemStone, but frankly the problem is that (until now) we have not had a simple client-based solution for adding GemStone-based persistence for native Pharo applications - the pharo developers have not really had a choice.

Being able to embed server blocks in client code certainly qualifies as simple. Solution(?), well that remains to be seen, but I am optimistic.

As a more concrete example, here's Pharo workspace code that uses NeoCSV running in Pharo to load stone objects in a Dictionary in GemStone:

  'NeoCSVBenchmark.csv' asFileReference
  readStreamDo: [ :stream |
  | reader converter buffer bufCnt numRecords records |
  converter := [ :string | NeoNumberParser parse: string ].
  reader := NeoCSVReader on: (ZnBufferedReadStream on: stream).
  reader
    recordClass: NeoCSVTestObject;
    addIntegerField: #'x:';
    addIntegerField: #'y:';
    addIntegerField: #'z:'.
  buffer := Array new: 1000.
  bufCnt := 0.
  [ reader atEnd ]
    whileFalse: [
      bufCnt := bufCnt + 1.
      buffer at: bufCnt put: reader next.
      bufCnt = buffer size
        ifTrue: [
          numRecords := bufCnt.
          records := buffer.
          DevKitShell
            onServerDo: [
              1 to: numRecords do: [ :index |
                | record |
                record := records at: index.
                NeoCSVDictionary at: record x put: record ].
              System commitTransaction.
              nil ].
          bufCnt := 0 ] ] ].
   DevKitShell onServerDo: [ System commitTransaction ]

The code ships 1000 instances of NeoCSVTestObject at a pop to GemStone. Using the above technique, one can easily arrange to store some pretty large object graphs in GemStone ... Efficient queries based on standard Smalltalk can be written on the client and transparently performed in GemStone (see the GsDevKitServerBlocks doc[1] for the complete example).

Server blocks do not duplicate the functionality GemBuilder for Smalltalk[6][7] which provides transparent replication and maintenance of objects between the client and server. With server blocks you end up with disconnected copies of server objects.

Because of this disconnect, I think the best way to architect an application using server blocks, is to plan on "executing all business logic" on the server --- If you are using an MVC pattern, the M would primarily be managed on the server while the VC would be managed on the client.

As an application evolves, the code can migrate back and forth between client and server as needed.

Most of the server blocks code leverages tODE and has been in use for several years. The code that spelunks in the block structure and extracts the _value_ of temp variables is only a couple of days old and has some pretty rough edges (notice the odd placement of temp variables and declarations in the above example).

The server-side debugger and inspectors, etc. will use tODE (at least for now) ...  in the server blocks doc[1] I demonstrate an #exportClassToServer: to illustrate the potential to share code in weird and wonderful ways between the client and server.

If you have the interest/opportunity to take this code for a spin, let me know. I have written instructions[5] for installing the experimental Roassal Visualization code[4] (GemStone and Pharo3.0 or Pharo4.0) for Pierre Chanson and those instructions can be used for doing work with GsDevKit
Server Blocks. There are a handful of obvious things that need to be done:
  - connection pools
  - coordinated client/server debuggers
  - client-side exception handlers for server errors
  - more???
and if folks express interest in start to do exploratory work with server blocks, then I will make time to provide support.

I am hoping to have something to announce by Smalltalks in November,so it would be useful if some experienced GemStoners tried things out before then...

I do have to finish up the documentation for GsDevKitHome 1.0.0 and tODE 0.1.0 and I'm also committed to doing some work on the 3.3 GemStone release, so we'll see how that goes:)

I also think that server blocks can be very useful for the "develop in Pharo, deploy in GemSstone" crowd, since it will be possible to write "pharo-based scripts" to perform server-side tasks ...

Questions or comments?

Dale

[1] https://github.com/GsDevKit/gsDevKitHome/blob/dev/docs/articles/gsDevKitServerBlocks.md#gsdevkit-server-blocks
[2] https://github.com/GsDevKit/ston#ston---smalltalk-object-notation
[3] https://vimeo.com/123261640
[4] https://github.com/GsDevKit/gsDevKitHome/tree/dev/projects/roassal#roassal-visualization
[5] https://github.com/GsDevKit/gsDevKitHome/blob/dev/projects/roassal/devBootstrap.md
[6] http://gemtalksystems.com/products/gbs-vw/
[7] http://gemtalksystems.com/products/gbs-va/

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Wow...this is so fuc... cool!!!!! AWESOME. Really... as soon as I need to trace pointers again in GemStone I DO KNOW how to do it :)

Yes... you should be really really careful about the number if items the #findAllReferences can answer. Remember GemStone is also a database...so the "image" may have billions of objects. So if the tool hangs hangs quite frequently, then i won't work. I think you should define a treshold and if more than that, then somehow present to the user the number of items and allow HIM (the user) to decide wether to display such items or not....

Thanks!

On Thu, May 28, 2015 at 7:52 PM, Pierre Chanson via Glass <[hidden email]> wrote:

Thanks a lot, great ! I am still learning a lot for now about Gemstone and it's applications (and also Pharo).

On 28 mai 2015, at 19:38, Dale Henrichs <[hidden email]> wrote:

Pierre,

Again, very cool ... I am nearing the end of my (intense) work on GemStone 3.3 so we should touch bases when I am done.

Dale

On 5/25/15 1:47 PM, Pierre CHANSON wrote:

Hi all, finally I found some time to work on this.

first, based on the PointerDetective of Ben (http://smalltalkhub.com/#!/~BenComan/PointerDetective) I did a RPointerDetective using Roassal and usable on a Pharo image.

Then I made a second class adapted to work on tODE, given a TDShell and using server blocks.

Here is a little video of the result: https://vimeo.com/128790347

The prerequisite are the last version of Roassal and the package: http://smalltalkhub.com/#!/~PierreChanson/RPointerDetective

The first part is on a Pharo image (could be a tODE image too) with the script:

--------------------------------------------
ob := 'OBJ'.
ar := { ob. 12 }.
e := (RTBox new) elementOn: ar.
v := RTView new.
v add: e.

RPointerDetective on: ob
-------------------------------------------------

Second part is on tODE, given an oop of server object.

The first script :

RPointerDetective onServerFindAllReferencePathsToObject: 154679809 shell: shell

And the second:

RPointerDetective onServer: 154679809 shell: shell

The first script is using the findAllReferencePathsToObjects:limitObjArray:printToLog:. In that case the graph is directly given by the gemstone method and displayed on the view on clic.

The second script is using the gemstone method findAllReferences. here tODE is calling the method when the object is clicked on the view. This one can be dangerous because we do not take care of the number of object that are going to be displayed...

There is still a lot of interesting things to do, with interactions on the view for example with the possibility to dinamically change the colors given a block or display the pointersFrom of a node or the possibility to open a tODE inspector etc.

cheers,

Pierre

2015-04-30 20:14 GMT-03:00 Pierre CHANSON <[hidden email]>:

Oh thanks ! I like this method now,  I am going to give a try :)

2015-04-30 14:59 GMT-03:00 Dale Henrichs via Glass <[hidden email]>:

+1 :)

On 04/30/2015 10:26 AM, Mariano Martinez Peck via Glass wrote:

Hi Pierre,

Do you want to do a super killer example of using Roassal and ServerBlocks?

As you may known, since in Smalltalk you cannot delete an object (compared to relational DBs where you do delete rows), removing all references to an object so that is GCed,  sometimes is not a walk in park. Ben was tired of using the Pharo Explorer for that, and he did this: http://smalltalkhub.com/#!/~BenComan/PointerDetective

If that is valuable in Pharo, imagine in GemStone. And now, with the serverBlocks, that would be very easy. 

Just see today's email of Otto saying he couldn't GC an object.

To find pointers to an object in GemStone:

SystemRepository findReferencePathToObject: 123456789 asObject

yeah...maybe I should try it myself :)

On Wed, Apr 29, 2015 at 7:42 PM, Pierre CHANSON <[hidden email]> wrote:

Thanks again Dale !

here are three small examples using these blocks in the roassal workspace of tODE:  https://vimeo.com/126435220

I highlight here the server blocks in red.

The first one is a Mondrian view of the class Collection (server side) with all it's subclasses.

The classes are processed in the block and stocked statically on the variable "classes".

--------------------------------------------------------------------------------------------------------------------------------------
| b classes |
classes := shell onServerDo:[Collection withAllSubclasses collect: [ :c | c name -> {(c superClass name). ((c instVarNamed: #methDicts asString) numElements)}]].

b := RTMondrian new.
b shape circle.
b nodes: classes.
b edges connectFrom: [:e | classes detect: [ :c | c name = e value first ] ifNone:[ nil ]] .

b shape bezierLineFollowing: [:e | classes detect: [ :c | c name = e value first ] ifNone:[ nil ]];
     color: (Color blue alpha: 0.2).
   

b normalizer
    normalizeSize: [:e | e value second ] using: #sqrt;
    normalizeColor: [:e | e value second ] using: (Array with: Color green with: Color red) using: #sqrt.
b layout cluster.
b build.
b view open
--------------------------------------------------------------------------------------------------------------------------------------

The second script use the RTExploraBuilder to do the same process, but this time the builder call dynamically the server on clics (we are using the oop of objects).

--------------------------------------------------------------------------------------------------------------------------------------
| builder |
    builder := RTExploraBuilder new.
    builder shape circle
        size: 30;
        color: (Color blue alpha: 0.5);
        if: [ :cls |
            (shell onServerDo: [ (Object _objectForOop: (cls value)) subclasses size]) = 0 ] fillColor: (Color red alpha: 0.5).
    builder
        layout: [RTClusterLayout new horizontalGap: 80];
        onClickExplore: [ :cls |
            shell onServerDo: [ ((Object _objectForOop: (cls value)) subclasses collect: [:c | c asString -> c asOop]) asArray ]
            ];
        withPopup: [:cls | cls key];
        dragChildren;
        node: (shell onServerDo: [Collection asString -> Collection asOop]);
        open.
--------------------------------------------------------------------------------------------------------------------------------------

The last script is about users. This need the last version of Roassal. We present some selection menus, a button and a radar Chart. Created with the RTMenuBuilder the button add a new user on the server with the characteristics selected in the other menus and present the characteristics of the added users on a Kiviat chart on the view.

--------------------------------------------------------------------------------------------------------------------------------------
|nameMenus privilegeMenus groupMenus|
    v := RTView new.
   
    kiv := RTKiviatBuilder new view: v.
    n := RTMultiLinearColorForIdentity new objects: (kiv objects).
    kiv shape circle color: [ :value | n rtValue: value kiviatNode named]; size: 10.


    kiv addMetric: [ :v | v second size] max: 5 title: 'groups'.
    kiv addMetric: [ :v | v third size] max: 5 title: 'privileges'.
    kiv addMetric: [ :v | v first size] max: 10 title: 'name'.


    kiv activatePolygons.
    kiv build.
   
    b := RTMenuBuilder new view: v.
   
    nameMenus := Array with: ('Henri'->[:m |]) with: ('Bob'->[:m |]) with: ('Robert'->[:m |]).
    groupMenus := Array with: ('Subscribers'->[:m |]) with:('Publishers'->[:m |]).
    privilegeMenus := Array with: ('ObsoleteStatistics'->[:m |]) with:('UserPassword'->[:m |]) with:('SessionAccess'->[:m |]).

   
    b menu: 'add User' callback: [
        |name groups privileges |
       
        name := (nameMenus detect: [ :m | m selected ]) name.
        groups := (groupMenus select: [:m | m selected]) collect: [:g | g name].
        privileges := (privilegeMenus select: [:m | m selected])  collect: [:p | p name].
        shell onServerDo: [
            AllUsers addNewUserWithId: name password: '' defaultObjectSecurityPolicy: nil privileges: privileges inGroups: groups .
            System commitTransaction.].
        kiv addDynamicObject: (Array with: (name) with: (groups) with: (privileges))
].

    nameMenus := b menu: 'user' subcheckmenus: nameMenus background: (Color red alpha: 0.3).
    RTMenuGroup on: nameMenus.
    nameMenus first selected: true.


    groupMenus := b menu: 'groups' subcheckmenus: groupMenus background: (Color blue alpha: 0.3).
    groupMenus first selected: true.
   

    privilegeMenus := b menu: 'privileges' subcheckmenus: privilegeMenus background: (Color green alpha: 0.3).
   
    privilegeMenus first selected: true.
   
    b view open.
--------------------------------------------------------------------------------------------------------------------------------------

 Pierre

 

2015-04-23 19:58 GMT+02:00 Mariano Martinez Peck via Glass <[hidden email]>:

On Wed, Apr 22, 2015 at 1:53 AM, Richard Sargent <[hidden email]> wrote:

Mariano,

Depending on what you mean by "execute stuff on server Y", 3.2 includes something called GsExternalSession. It it capable of executing Smalltalk specified in  Block on a separate session against the same stone or a different one.

It doesn't support copying object graphs, but if your definition of execute stuff has limited requirements for exchanging data, it could be what you are looking for.
See the 3.2 documentation for details.

Good to know too. Thanks Richard. 

 

On Apr 21, 2015 7:01 PM, "Mariano Martinez Peck via Glass" <[hidden email]> wrote:

On Thu, Apr 16, 2015 at 2:23 AM, Dale Henrichs via Glass <[hidden email]> wrote:

A GsDevKit Server Block[1] is a block that is written in-line in client Smalltalk, but is executed in GemStone. For example the following expression is executed in a standard Pharo workspace:

  | shell x y |
  shell := TDShell forSessionNamed: 'devKit'.
  x := 3.
  y := 4.
  shell onServerDo: [ x + y ].

Dale, 

I know (because I already asked a few months/years ago) that from a stone X you can do a remote login on stone Y and execute stuff in Y. But now I wonder....could server blocks also work for gemstone-gemstone? (my gut feelings tell me that yes) I mean, could I run the above code from GemStone itself?   That would automatically resolve all the remote login stuff and the ston serialization. 

 

Thanks in advance, 

and the `[3 + 4 ]` block is executed in GemStone using the `devKit` session description to log into the stone. The temp vars x and y referenced in the server block and defined in Pharo are shipped across the wire to GemStone along with block source where the block source is compiled and executed. The result is then shipped back across the wire and returned as the result of #onServerDo: message in Pharo. Pharo execution can continue on using the result. STON[2] is used to serialize the objects that are shipped across the wire.

For any of you familiar with underpinnings of GemTools, Jade or tODE, this is not necessarily ground-breaking technology, however, exposing this capability to developers just may be.

It has been a long standing crime that developers in the Pharo community choose to use  MongoDB and MySQL over GemStone, but frankly the problem is that (until now) we have not had a simple client-based solution for adding GemStone-based persistence for native Pharo applications - the pharo developers have not really had a choice.

Being able to embed server blocks in client code certainly qualifies as simple. Solution(?), well that remains to be seen, but I am optimistic.

As a more concrete example, here's Pharo workspace code that uses NeoCSV running in Pharo to load stone objects in a Dictionary in GemStone:

  'NeoCSVBenchmark.csv' asFileReference
  readStreamDo: [ :stream |
  | reader converter buffer bufCnt numRecords records |
  converter := [ :string | NeoNumberParser parse: string ].
  reader := NeoCSVReader on: (ZnBufferedReadStream on: stream).
  reader
    recordClass: NeoCSVTestObject;
    addIntegerField: #'x:';
    addIntegerField: #'y:';
    addIntegerField: #'z:'.
  buffer := Array new: 1000.
  bufCnt := 0.
  [ reader atEnd ]
    whileFalse: [
      bufCnt := bufCnt + 1.
      buffer at: bufCnt put: reader next.
      bufCnt = buffer size
        ifTrue: [
          numRecords := bufCnt.
          records := buffer.
          DevKitShell
            onServerDo: [
              1 to: numRecords do: [ :index |
                | record |
                record := records at: index.
                NeoCSVDictionary at: record x put: record ].
              System commitTransaction.
              nil ].
          bufCnt := 0 ] ] ].
   DevKitShell onServerDo: [ System commitTransaction ]

The code ships 1000 instances of NeoCSVTestObject at a pop to GemStone. Using the above technique, one can easily arrange to store some pretty large object graphs in GemStone ... Efficient queries based on standard Smalltalk can be written on the client and transparently performed in GemStone (see the GsDevKitServerBlocks doc[1] for the complete example).

Server blocks do not duplicate the functionality GemBuilder for Smalltalk[6][7] which provides transparent replication and maintenance of objects between the client and server. With server blocks you end up with disconnected copies of server objects.

Because of this disconnect, I think the best way to architect an application using server blocks, is to plan on "executing all business logic" on the server --- If you are using an MVC pattern, the M would primarily be managed on the server while the VC would be managed on the client.

As an application evolves, the code can migrate back and forth between client and server as needed.

Most of the server blocks code leverages tODE and has been in use for several years. The code that spelunks in the block structure and extracts the _value_ of temp variables is only a couple of days old and has some pretty rough edges (notice the odd placement of temp variables and declarations in the above example).

The server-side debugger and inspectors, etc. will use tODE (at least for now) ...  in the server blocks doc[1] I demonstrate an #exportClassToServer: to illustrate the potential to share code in weird and wonderful ways between the client and server.

If you have the interest/opportunity to take this code for a spin, let me know. I have written instructions[5] for installing the experimental Roassal Visualization code[4] (GemStone and Pharo3.0 or Pharo4.0) for Pierre Chanson and those instructions can be used for doing work with GsDevKit
Server Blocks. There are a handful of obvious things that need to be done:
  - connection pools
  - coordinated client/server debuggers
  - client-side exception handlers for server errors
  - more???
and if folks express interest in start to do exploratory work with server blocks, then I will make time to provide support.

I am hoping to have something to announce by Smalltalks in November,so it would be useful if some experienced GemStoners tried things out before then...

I do have to finish up the documentation for GsDevKitHome 1.0.0 and tODE 0.1.0 and I'm also committed to doing some work on the 3.3 GemStone release, so we'll see how that goes:)

I also think that server blocks can be very useful for the "develop in Pharo, deploy in GemSstone" crowd, since it will be possible to write "pharo-based scripts" to perform server-side tasks ...

Questions or comments?

Dale

[1] https://github.com/GsDevKit/gsDevKitHome/blob/dev/docs/articles/gsDevKitServerBlocks.md#gsdevkit-server-blocks
[2] https://github.com/GsDevKit/ston#ston---smalltalk-object-notation
[3] https://vimeo.com/123261640
[4] https://github.com/GsDevKit/gsDevKitHome/tree/dev/projects/roassal#roassal-visualization
[5] https://github.com/GsDevKit/gsDevKitHome/blob/dev/projects/roassal/devBootstrap.md
[6] http://gemtalksystems.com/products/gbs-vw/
[7] http://gemtalksystems.com/products/gbs-va/

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

You are right, there are several ways to solve this, you'll hear from me soon

cheers,

Pierre

2015-05-28 22:24 GMT-03:00 Mariano Martinez Peck <[hidden email]>:

Wow...this is so fuc... cool!!!!! AWESOME. Really... as soon as I need to trace pointers again in GemStone I DO KNOW how to do it :)

Yes... you should be really really careful about the number if items the #findAllReferences can answer. Remember GemStone is also a database...so the "image" may have billions of objects. So if the tool hangs hangs quite frequently, then i won't work. I think you should define a treshold and if more than that, then somehow present to the user the number of items and allow HIM (the user) to decide wether to display such items or not....

Thanks!

On Thu, May 28, 2015 at 7:52 PM, Pierre Chanson via Glass <[hidden email]> wrote:

Thanks a lot, great ! I am still learning a lot for now about Gemstone and it's applications (and also Pharo).

On 28 mai 2015, at 19:38, Dale Henrichs <[hidden email]> wrote:

Pierre,

Again, very cool ... I am nearing the end of my (intense) work on GemStone 3.3 so we should touch bases when I am done.

Dale

On 5/25/15 1:47 PM, Pierre CHANSON wrote:

Hi all, finally I found some time to work on this.

first, based on the PointerDetective of Ben (http://smalltalkhub.com/#!/~BenComan/PointerDetective) I did a RPointerDetective using Roassal and usable on a Pharo image.

Then I made a second class adapted to work on tODE, given a TDShell and using server blocks.

Here is a little video of the result: https://vimeo.com/128790347

The prerequisite are the last version of Roassal and the package: http://smalltalkhub.com/#!/~PierreChanson/RPointerDetective

The first part is on a Pharo image (could be a tODE image too) with the script:

--------------------------------------------
ob := 'OBJ'.
ar := { ob. 12 }.
e := (RTBox new) elementOn: ar.
v := RTView new.
v add: e.

RPointerDetective on: ob
-------------------------------------------------

Second part is on tODE, given an oop of server object.

The first script :

RPointerDetective onServerFindAllReferencePathsToObject: 154679809 shell: shell

And the second:

RPointerDetective onServer: 154679809 shell: shell

The first script is using the findAllReferencePathsToObjects:limitObjArray:printToLog:. In that case the graph is directly given by the gemstone method and displayed on the view on clic.

The second script is using the gemstone method findAllReferences. here tODE is calling the method when the object is clicked on the view. This one can be dangerous because we do not take care of the number of object that are going to be displayed...

There is still a lot of interesting things to do, with interactions on the view for example with the possibility to dinamically change the colors given a block or display the pointersFrom of a node or the possibility to open a tODE inspector etc.

cheers,

Pierre

2015-04-30 20:14 GMT-03:00 Pierre CHANSON <[hidden email]>:

Oh thanks ! I like this method now,  I am going to give a try :)

2015-04-30 14:59 GMT-03:00 Dale Henrichs via Glass <[hidden email]>:

+1 :)

On 04/30/2015 10:26 AM, Mariano Martinez Peck via Glass wrote:

Hi Pierre,

Do you want to do a super killer example of using Roassal and ServerBlocks?

As you may known, since in Smalltalk you cannot delete an object (compared to relational DBs where you do delete rows), removing all references to an object so that is GCed,  sometimes is not a walk in park. Ben was tired of using the Pharo Explorer for that, and he did this: http://smalltalkhub.com/#!/~BenComan/PointerDetective

If that is valuable in Pharo, imagine in GemStone. And now, with the serverBlocks, that would be very easy. 

Just see today's email of Otto saying he couldn't GC an object.

To find pointers to an object in GemStone:

SystemRepository findReferencePathToObject: 123456789 asObject

yeah...maybe I should try it myself :)

On Wed, Apr 29, 2015 at 7:42 PM, Pierre CHANSON <[hidden email]> wrote:

Thanks again Dale !

here are three small examples using these blocks in the roassal workspace of tODE:  https://vimeo.com/126435220

I highlight here the server blocks in red.

The first one is a Mondrian view of the class Collection (server side) with all it's subclasses.

The classes are processed in the block and stocked statically on the variable "classes".

--------------------------------------------------------------------------------------------------------------------------------------
| b classes |
classes := shell onServerDo:[Collection withAllSubclasses collect: [ :c | c name -> {(c superClass name). ((c instVarNamed: #methDicts asString) numElements)}]].

b := RTMondrian new.
b shape circle.
b nodes: classes.
b edges connectFrom: [:e | classes detect: [ :c | c name = e value first ] ifNone:[ nil ]] .

b shape bezierLineFollowing: [:e | classes detect: [ :c | c name = e value first ] ifNone:[ nil ]];
     color: (Color blue alpha: 0.2).
   

b normalizer
    normalizeSize: [:e | e value second ] using: #sqrt;
    normalizeColor: [:e | e value second ] using: (Array with: Color green with: Color red) using: #sqrt.
b layout cluster.
b build.
b view open
--------------------------------------------------------------------------------------------------------------------------------------

The second script use the RTExploraBuilder to do the same process, but this time the builder call dynamically the server on clics (we are using the oop of objects).

--------------------------------------------------------------------------------------------------------------------------------------
| builder |
    builder := RTExploraBuilder new.
    builder shape circle
        size: 30;
        color: (Color blue alpha: 0.5);
        if: [ :cls |
            (shell onServerDo: [ (Object _objectForOop: (cls value)) subclasses size]) = 0 ] fillColor: (Color red alpha: 0.5).
    builder
        layout: [RTClusterLayout new horizontalGap: 80];
        onClickExplore: [ :cls |
            shell onServerDo: [ ((Object _objectForOop: (cls value)) subclasses collect: [:c | c asString -> c asOop]) asArray ]
            ];
        withPopup: [:cls | cls key];
        dragChildren;
        node: (shell onServerDo: [Collection asString -> Collection asOop]);
        open.
--------------------------------------------------------------------------------------------------------------------------------------

The last script is about users. This need the last version of Roassal. We present some selection menus, a button and a radar Chart. Created with the RTMenuBuilder the button add a new user on the server with the characteristics selected in the other menus and present the characteristics of the added users on a Kiviat chart on the view.

--------------------------------------------------------------------------------------------------------------------------------------
|nameMenus privilegeMenus groupMenus|
    v := RTView new.
   
    kiv := RTKiviatBuilder new view: v.
    n := RTMultiLinearColorForIdentity new objects: (kiv objects).
    kiv shape circle color: [ :value | n rtValue: value kiviatNode named]; size: 10.


    kiv addMetric: [ :v | v second size] max: 5 title: 'groups'.
    kiv addMetric: [ :v | v third size] max: 5 title: 'privileges'.
    kiv addMetric: [ :v | v first size] max: 10 title: 'name'.


    kiv activatePolygons.
    kiv build.
   
    b := RTMenuBuilder new view: v.
   
    nameMenus := Array with: ('Henri'->[:m |]) with: ('Bob'->[:m |]) with: ('Robert'->[:m |]).
    groupMenus := Array with: ('Subscribers'->[:m |]) with:('Publishers'->[:m |]).
    privilegeMenus := Array with: ('ObsoleteStatistics'->[:m |]) with:('UserPassword'->[:m |]) with:('SessionAccess'->[:m |]).

   
    b menu: 'add User' callback: [
        |name groups privileges |
       
        name := (nameMenus detect: [ :m | m selected ]) name.
        groups := (groupMenus select: [:m | m selected]) collect: [:g | g name].
        privileges := (privilegeMenus select: [:m | m selected])  collect: [:p | p name].
        shell onServerDo: [
            AllUsers addNewUserWithId: name password: '' defaultObjectSecurityPolicy: nil privileges: privileges inGroups: groups .
            System commitTransaction.].
        kiv addDynamicObject: (Array with: (name) with: (groups) with: (privileges))
].

    nameMenus := b menu: 'user' subcheckmenus: nameMenus background: (Color red alpha: 0.3).
    RTMenuGroup on: nameMenus.
    nameMenus first selected: true.


    groupMenus := b menu: 'groups' subcheckmenus: groupMenus background: (Color blue alpha: 0.3).
    groupMenus first selected: true.
   

    privilegeMenus := b menu: 'privileges' subcheckmenus: privilegeMenus background: (Color green alpha: 0.3).
   
    privilegeMenus first selected: true.
   
    b view open.
--------------------------------------------------------------------------------------------------------------------------------------

 Pierre

 

2015-04-23 19:58 GMT+02:00 Mariano Martinez Peck via Glass <[hidden email]>:

On Wed, Apr 22, 2015 at 1:53 AM, Richard Sargent <[hidden email]> wrote:

Mariano,

Depending on what you mean by "execute stuff on server Y", 3.2 includes something called GsExternalSession. It it capable of executing Smalltalk specified in  Block on a separate session against the same stone or a different one.

It doesn't support copying object graphs, but if your definition of execute stuff has limited requirements for exchanging data, it could be what you are looking for.
See the 3.2 documentation for details.

Good to know too. Thanks Richard. 

 

On Apr 21, 2015 7:01 PM, "Mariano Martinez Peck via Glass" <[hidden email]> wrote:

On Thu, Apr 16, 2015 at 2:23 AM, Dale Henrichs via Glass <[hidden email]> wrote:

A GsDevKit Server Block[1] is a block that is written in-line in client Smalltalk, but is executed in GemStone. For example the following expression is executed in a standard Pharo workspace:

  | shell x y |
  shell := TDShell forSessionNamed: 'devKit'.
  x := 3.
  y := 4.
  shell onServerDo: [ x + y ].

Dale, 

I know (because I already asked a few months/years ago) that from a stone X you can do a remote login on stone Y and execute stuff in Y. But now I wonder....could server blocks also work for gemstone-gemstone? (my gut feelings tell me that yes) I mean, could I run the above code from GemStone itself?   That would automatically resolve all the remote login stuff and the ston serialization. 

 

Thanks in advance, 

and the `[3 + 4 ]` block is executed in GemStone using the `devKit` session description to log into the stone. The temp vars x and y referenced in the server block and defined in Pharo are shipped across the wire to GemStone along with block source where the block source is compiled and executed. The result is then shipped back across the wire and returned as the result of #onServerDo: message in Pharo. Pharo execution can continue on using the result. STON[2] is used to serialize the objects that are shipped across the wire.

For any of you familiar with underpinnings of GemTools, Jade or tODE, this is not necessarily ground-breaking technology, however, exposing this capability to developers just may be.

It has been a long standing crime that developers in the Pharo community choose to use  MongoDB and MySQL over GemStone, but frankly the problem is that (until now) we have not had a simple client-based solution for adding GemStone-based persistence for native Pharo applications - the pharo developers have not really had a choice.

Being able to embed server blocks in client code certainly qualifies as simple. Solution(?), well that remains to be seen, but I am optimistic.

As a more concrete example, here's Pharo workspace code that uses NeoCSV running in Pharo to load stone objects in a Dictionary in GemStone:

  'NeoCSVBenchmark.csv' asFileReference
  readStreamDo: [ :stream |
  | reader converter buffer bufCnt numRecords records |
  converter := [ :string | NeoNumberParser parse: string ].
  reader := NeoCSVReader on: (ZnBufferedReadStream on: stream).
  reader
    recordClass: NeoCSVTestObject;
    addIntegerField: #'x:';
    addIntegerField: #'y:';
    addIntegerField: #'z:'.
  buffer := Array new: 1000.
  bufCnt := 0.
  [ reader atEnd ]
    whileFalse: [
      bufCnt := bufCnt + 1.
      buffer at: bufCnt put: reader next.
      bufCnt = buffer size
        ifTrue: [
          numRecords := bufCnt.
          records := buffer.
          DevKitShell
            onServerDo: [
              1 to: numRecords do: [ :index |
                | record |
                record := records at: index.
                NeoCSVDictionary at: record x put: record ].
              System commitTransaction.
              nil ].
          bufCnt := 0 ] ] ].
   DevKitShell onServerDo: [ System commitTransaction ]

The code ships 1000 instances of NeoCSVTestObject at a pop to GemStone. Using the above technique, one can easily arrange to store some pretty large object graphs in GemStone ... Efficient queries based on standard Smalltalk can be written on the client and transparently performed in GemStone (see the GsDevKitServerBlocks doc[1] for the complete example).

Server blocks do not duplicate the functionality GemBuilder for Smalltalk[6][7] which provides transparent replication and maintenance of objects between the client and server. With server blocks you end up with disconnected copies of server objects.

Because of this disconnect, I think the best way to architect an application using server blocks, is to plan on "executing all business logic" on the server --- If you are using an MVC pattern, the M would primarily be managed on the server while the VC would be managed on the client.

As an application evolves, the code can migrate back and forth between client and server as needed.

Most of the server blocks code leverages tODE and has been in use for several years. The code that spelunks in the block structure and extracts the _value_ of temp variables is only a couple of days old and has some pretty rough edges (notice the odd placement of temp variables and declarations in the above example).

The server-side debugger and inspectors, etc. will use tODE (at least for now) ...  in the server blocks doc[1] I demonstrate an #exportClassToServer: to illustrate the potential to share code in weird and wonderful ways between the client and server.

If you have the interest/opportunity to take this code for a spin, let me know. I have written instructions[5] for installing the experimental Roassal Visualization code[4] (GemStone and Pharo3.0 or Pharo4.0) for Pierre Chanson and those instructions can be used for doing work with GsDevKit
Server Blocks. There are a handful of obvious things that need to be done:
  - connection pools
  - coordinated client/server debuggers
  - client-side exception handlers for server errors
  - more???
and if folks express interest in start to do exploratory work with server blocks, then I will make time to provide support.

I am hoping to have something to announce by Smalltalks in November,so it would be useful if some experienced GemStoners tried things out before then...

I do have to finish up the documentation for GsDevKitHome 1.0.0 and tODE 0.1.0 and I'm also committed to doing some work on the 3.3 GemStone release, so we'll see how that goes:)

I also think that server blocks can be very useful for the "develop in Pharo, deploy in GemSstone" crowd, since it will be possible to write "pharo-based scripts" to perform server-side tasks ...

Questions or comments?

Dale

[1] https://github.com/GsDevKit/gsDevKitHome/blob/dev/docs/articles/gsDevKitServerBlocks.md#gsdevkit-server-blocks
[2] https://github.com/GsDevKit/ston#ston---smalltalk-object-notation
[3] https://vimeo.com/123261640
[4] https://github.com/GsDevKit/gsDevKitHome/tree/dev/projects/roassal#roassal-visualization
[5] https://github.com/GsDevKit/gsDevKitHome/blob/dev/projects/roassal/devBootstrap.md
[6] http://gemtalksystems.com/products/gbs-vw/
[7] http://gemtalksystems.com/products/gbs-va/

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com