[tode_st] Gemstone & Seaside

> Hello All,
>
> Please pardon the newbie question. Trying to get my head around the
> Gemstone64/S Seaside work flow.
>
> I am wanting to use Gemstone as a persistent object store.
> I do not want it to run any client facing applications.
> I want to run all client facing applications in pharo images of there own
> connected to a central backing stone. Is this the wrong way to think about
> this?

I'm just curious - How do you solve communication between the server (Gemstone/backing store) and the client (Pharo images with Seaside)? JSON REST API? Or some transparent object layer?

If you run Seaside apps inside the Gemstone VM, you can access all Gemstone objects directly, which is luxury (and fast).

pf

>
> All the information (at least that I can find) forces the installation of
> seaside into the Gemstone server.  From there worked on in the client
> (tODE).  Having played around with the single installation version where
> all reside on a local machine I notice that I can created multiple stones
> and multiple clients which is great.  But doesn't this take away from
> separating application space from backing store?  Or is the Idea that one
> application gets one stone?
>
> Sorry for the confusing question but I can't find any literature on how
> this is all supposed to fit together in the real world.
>
> Kindly
>
> --
> You received this message because you are subscribed to the Google Groups "tODE" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

Ian,

Most of the focus for GLASS/GsDevKit over the last several years has been on porting headless applications to GemStone from Pharo so there has not been a lot of work done on enabling the kind of applications you are talking about ...

As Petr points out your best bet is to write your application as a RESTful GemStone server and use JSON or STON to pass objects back and forth between Pharo and GemStone.

tODE is not really a classic "client application using GemStone as a backing store". tODE is what I call a "very thin client" where the Pharo client is only used for presentation (much like a web browser) and all of the real work is done on the server.

At ESUG in 2016, I presented a rough framework called Tugrik[1],[2],[3] that was aimed at supporting tODE-like applications based on Voyage, but I think that there is a lot of work to be done to move Tugrik into reality ... I would be willing to work with you if you wanted to push on Tugrik.

But I really think that your best bet at the moment would be to go with a RESTful architecture as all of the technology is well developed.

Let me know if you have additional questions,

Dale

[1] https://github.com/dalehenrich/Tugrik
[2] http://esug.org/data/ESUG2016/03-Wednesday/1400-1445%20Tugrik/2016-08-24-Voyage-Tugrik.key.pdf
[3] https://www.youtube.com/watch?v=YwlUdRaqTwE

On 1/4/18 12:47 PM, Ian Ian wrote:

Hello All,

Please pardon the newbie question. Trying to get my head around the Gemstone64/S Seaside work flow.

I am wanting to use Gemstone as a persistent object store.
I do not want it to run any client facing applications.
I want to run all client facing applications in pharo images of there own connected to a central backing stone. Is this the wrong way to think about this?

All the information (at least that I can find) forces the installation of seaside into the Gemstone server.  From there worked on in the client (tODE).  Having played around with the single installation version where all reside on a local machine I notice that I can created multiple stones and multiple clients which is great.  But doesn't this take away from separating application space from backing store?  Or is the Idea that one application gets one stone?

Sorry for the confusing question but I can't find any literature on how this is all supposed to fit together in the real world.

Kindly

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "tODE" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tode_st/DhOb1p0H5LE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

Ian Ian wrote
> I am used to thinking in a relational sense and was looking at GemStone as
> a 'database' which, it seems, in the traditional sense, it is not.

In some respects, it is better to think of the GemStone repository as a
Smalltalk /image/. Then you think of the Gems as concurrently executing VMs
running the same image, but with transaction management to make it safe.


An SQL database has stored procedures; GemStone has "stored procedures on
steroids". With complete access to all the code in a Smalltalk image, you
can literally do any computable thing you can imagine.

It is a vastly different paradigm from plucking bits of data from a data
store and playing with them in a tier that is essentially completely
isolated from the data. Object Oriented is code + data while SQL is code
versus data.


It's a lot of fun, so jump in, the water's fine!




--
Sent from: http://forum.world.st/tODE-f3744225.html

--
You received this message because you are subscribed to a topic in the Google Groups "tODE" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tode_st/DhOb1p0H5LE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

On 1/11/18 2:24 PM, Ian Ian wrote:

Hi Dale,

I am evaluating Gemstone as an application server/big data repository.  Hence my confusion up till now.

I work for an organization that has rather large data storage requirements (understatement:  into the 100's of petabyte range).

Currently, several independent Apache Hadoop stacks are used to manage this data.  There are several serious issues that are evident with this setup, prime of which is object to relational mapping errors (as datasets in a given space change over time - data is mandated to be kept into perpetuity) regardless of whether the database is NoSQL, SQL of any other schema.  The issue bubbles up into the organization application layers of many diverse and distributed verticals.  Maintenance of the system is not trivial and is very expensive and time consuming.

We use virtualization (in the VMWare sense) to create views into the data.  Generally, access is a split between virtualized kiosk systems, another level of indirection, where by mandate, we must now archive the state of the vm's as well as business data, and direct workstation access, Not fun, error prone and hardware intensive (from a data storage perspective).

As I have been reading about object databases and what they are said to buy in terms of efficiency (objects being stored natively and directly) coupled with the fact the we are virtualizing everything we can, I, along with a couple of colleagues, thought that we should take a look at what new (laugh) technologies were available.

After a little time I came across Gemstone.  Did my reading and research and after a period of semantic confusion (still going) became enamored with the possibilities.

Interesting ... I guess I have sorta been waiting for someone to discover us as I have thought for a while now that data modeling and exploration based on extractions from  big data streams would be a niche where GemStone could provide some real value...

All this said, from a network, sys admin point of view, an image that contains the application and it's objects is a godsend if object persistence was guaranteed.  It would allow me to archive a self contained living blob that manages its own living state (in the VMWare snapshot sense) and at the same time the data is available to other blobs/views (seaside).

On views in to the data store:

Currently data mining occurs on the application layer (rather than database level views).  Various development groups create views (applications) to access (possibly update, if they are the owning group) the data.  As mandate trickle (waterfall) from above new data points can be added or removed (some of the mandate for this is governed by changing regulations - in other words it must be done regardless of the technical difficulties involved).  You can image the difficulties and manpower involved.

This brings me back to the application server. Currently, whatever tool a group uses to access the data, be it browser, remote client application, whatever have you (generally based on the skill sets in the group), requires a set of policies and procedures wrapped around the effort (mandated disaster recovery and etc...).  Tones of work that can never be standardized in any way but nevertheless must be documented and managed.

Okay (to long winded self):

Having spent some time looking at this problem from a smalltalk, image virtualization, integrated application management perspective, all my issues could be alleviated within a single technology.  Yippee!

Haha ...

The spawning of an idea:

Amber -> Seaside -> Gemstone.  One language (integrated properly -> one tool).

A full-stack Smalltalk solution makes a ton of sense.

Amber:

*  Each group, with a small learning curve could create and manage their own views into the data.
*  The serialized javascript, once the client was developed, could be streamed from the application server (Seaside/Gemstone) on demand.  It just becomes another object.

Seaside:

* Client interface manager transport system (HTTP) for any and all views.

Gemstone:

* Object level persistence with no forced mappings of any kind.
* Extensibility. (can gemstone handle data levels in the petadata range?)

The current release has a design limit of about 7 petabytes and an object count of 128 billion (there is database metadata that needs to be accounted for as well:)... depending upon the nature of the data being stored there would be some simple changes we could make that could extend the limit into the 100's of petabytes, of course there are additional changes that we could make to increase the bytecount limit.

Cool!:

Now of course the devil is in the details, but this makes a lot of sense ... as an alternate architecture to study, I would point to the work of Martin Feldtmann[1]. Marten uses Swagger to generate multi-language mappings for the RESTful API that he is using where GemStone/S is the database of record ... I'm sure that Marten would be willing to provide additional details if asked.

A full-stack Smalltalk application of course has tons of appeal, but the realist in me knows that the entire world does not use Smalltalk, so the Swagger approach simplifies the process of maintaining a mulit-language API against a GemStone dB.

[1] https://schrievkrom.wordpress.com/2016/03/15/gemstones-application-example/
[2] https://swagger.io/

* Users/Developers of the system have a completely controlled view into the image/system (perhaps a clustered NginX front end?).
* Image access at the machine level is completely controlled/secure.
* Everything in one place with system knowledge transfer and expertise leveraged vertically and horizontally.
* The images can easily be archived in whatever manner (at the bit level using any version control system if desired) at whatever schedule.
* Another level of 'complete system' security achieved. (Way cool!!)
* Whole systems is a single entity!  Fantastical!
* A significantly smaller foot print.

* Cost reductions, increased efficiency all the way through...

Pipe dream?  Maybe but I think it is doable over time with smalltalk.  The same system is not doable in any other way with any other technology without extensive dollars and effort and once built making dynamic unplanned for changes virtually impossible.  You are locked in (Silly but happens all the time).  With the system I envision, changes, even schema changes, are relatively easy.

Agreed.

For my money:

* Gemtalk should integrate development tools directly into Gemstone (Gemstone with seaside is an order of magnitude more useful than without it)

This is now on the roadmap and things should start showing up in our next release.

* Gemtools should be in the reference client - Pharo - permanently (CGI/RPC, everything).

We are definitely headed in this direction.

  What does the tODE buy me, other than access, but a shell?

tODE is actually involves a couple of different distinct technologies beyond the shell: an ultra-thin client, a multi-client tools architecture, and a prototype for git-based development.

tODE uses an ultra-thin Pharo client[6]. There is no business logic on the Pharo client, it is simply used for it's display capabilities --- think of html in browsers where you have a variety of widgets, but the contents of the widgets are described by objects passed back and forth between the server and client, rather than html. The result is that object replication is restricted to only display artifacts ... the business objects remain on the server at all times and all code is actually executed in server...

The ultra-thin client has the advantage of eliminating the need to implement any kind of replication scheme between the client and server, except that needed for the display which are only strings and numbers and objects of limited depth. Replicating arbitrary and/or deeply nested objects is where a lot of complication comes into play.

The complication for the ultra-thin client comes in providing a set of rich and responsive widgets for implementing the GUI -- presumably a much simpler.

Speaking of alternate approaches, a year ago I did a simple proof of concept[7] aimed at allowing the Pharo class library to be run natively in a GemStone repository using some of the base technology that we developed for Maglev[8]. In theory, Pharo programs (and even the Pharo GUI) would run "natively" in GemStone.

There is a group of bachelor students at HPI[9] currently working on running Squeak programs (and the Squeak GUI) in GemStone.

[6] https://www.slideshare.net/esug/tode-and-now-for-something-completely-different
[7] https://github.com/dalehenrich/PharoGs
[8] https://maglev.github.io/
[9] https://hpi.de/en/

  Give Pharo a permanent seamless object store - make it easy for me/others. The user base will grow.

Giving "Pharo a permanent seamless object store" is not explicitly on the roadmap as we are still in the early stages of exploration. We will need some sort of object replication scheme to implement the development environment. GemBuilder for Smalltalk (GBS)[3] is pretty much a "a permanent seamless object store"  that exists for VW[4] and VA[5], but it is also incredibly complex and we will be exploring simpler/cheaper to implement alternatives before investing in a full-blown GBS for Pharo.

Martin McClure (responsible for GBS) has a new replication architecture in mind for the upcoming Pharo client that eliminates much of the complication entailed in GBS, however, it _is_ still in development, so we won't know for a while how far it goes in providing a "Pharo a permanent seamless object store."

[3] https://downloads.gemtalksystems.com/docs/GBS/7.6.x/GBS-VW-UsersGuide-7.6/GBS-VW-UsersGuide-7.6.htm
[4] https://gemtalksystems.com/products/gbs-vw/
[5] https://gemtalksystems.com/products/gbs-va/

  Gemtalk will sell more licenses in the long run.

Ultimately where we head and invest resources will be driven by customer demand ... and at the moment we are moving towards providing support for git-based development in the kernel with a Pharo client for development.

* Gemstone should be re-branded as a fully functional pure OO application server with extensible secure persistence built in. (slight difference but HUGE!!! - MS, in truth, is a marketing company)
  Let's see if java can compete with that! (Not - they would be forced to make changes just to try.)

I think we would need some pioneering work by folks with domain knowledge so that we can provide concrete examples and (more importantly) make sure that the path is a smooth one ... and I see that you are proposing a proof of concept below ..

* There are more but I am saying too much as it is....

Lastly,

My proof of concept:

1.  Create a seaside app
2.  Create its client in amber.
3.  Upload the client js to the stone.
4.  Stream the client(s) to the browser based on some form of uri
5.  Manipulated the data store (gemstone) with the client.
6.  Cluster the system.
7.  Develop a DR process.
8.  Sell it

Should on the face of it be simple?

It should be relatively straightforward.

Let me know when you are planning on starting your proof of concept project as I would like to update the GsDevit_home Seaside installation instructions[10] to use Seaside 3.2 instead of Seaside 3.1 (you should be using the latest version of Seaside).

It is worth getting on the GLASS[11], Seaside[12], and Amber[13] mailing lists so that you can get quick answers to your questions.

For GsDevKit_home and GemStone specific advice I am definitely interested in helping.

Dale

[10] https://github.com/GsDevKit/GsDevKit_home/blob/master/docs/gettingStartedWithSeaside.md
[11] http://forum.world.st/mailing_list/MailingListOptions.jtp?forum=1460844
[12] http://forum.world.st/mailing_list/MailingListOptions.jtp?forum=86180
[13] http://forum.world.st/mailing_list/MailingListOptions.jtp?forum=3824463

--
You received this message because you are subscribed to a topic in the Google Groups "tODE" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tode_st/DhOb1p0H5LE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.