[tode_st] Gemstone & Seaside

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

[tode_st] Gemstone & Seaside

Ian Ian
Hello All,

Please pardon the newbie question. Trying to get my head around the Gemstone64/S Seaside work flow.

I am wanting to use Gemstone as a persistent object store.
I do not want it to run any client facing applications.
I want to run all client facing applications in pharo images of there own connected to a central backing stone. Is this the wrong way to think about this?

All the information (at least that I can find) forces the installation of seaside into the Gemstone server.  From there worked on in the client (tODE).  Having played around with the single installation version where all reside on a local machine I notice that I can created multiple stones and multiple clients which is great.  But doesn't this take away from separating application space from backing store?  Or is the Idea that one application gets one stone?

Sorry for the confusing question but I can't find any literature on how this is all supposed to fit together in the real world.

Kindly

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Petr Fischer
> Hello All,
>
> Please pardon the newbie question. Trying to get my head around the
> Gemstone64/S Seaside work flow.
>
> I am wanting to use Gemstone as a persistent object store.
> I do not want it to run any client facing applications.
> I want to run all client facing applications in pharo images of there own
> connected to a central backing stone. Is this the wrong way to think about
> this?

I'm just curious - How do you solve communication between the server (Gemstone/backing store) and the client (Pharo images with Seaside)? JSON REST API? Or some transparent object layer?

If you run Seaside apps inside the Gemstone VM, you can access all Gemstone objects directly, which is luxury (and fast).

pf

>
> All the information (at least that I can find) forces the installation of
> seaside into the Gemstone server.  From there worked on in the client
> (tODE).  Having played around with the single installation version where
> all reside on a local machine I notice that I can created multiple stones
> and multiple clients which is great.  But doesn't this take away from
> separating application space from backing store?  Or is the Idea that one
> application gets one stone?
>
> Sorry for the confusing question but I can't find any literature on how
> this is all supposed to fit together in the real world.
>
> Kindly
>
> --
> You received this message because you are subscribed to the Google Groups "tODE" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Mariano Martinez Peck
Hi,

Before  a more detailed answer, I guess I need more details on what is "client facing applications" ? Is that somehow something web? or is that a desktop kind of application?
Why this is important? Because it will impact the possible solutions. For example, if it is a desktop kind of app, then yes, you would need Pharo or something doing the UI and connecting to GemStone. If the UI is web then you may be able to run the server side also in GemStone (as most Seaside applications on gemstone) and no need from Pharo. You do the famous "develop in pharo and deploy in gemstone".

Cheers,



On Thu, Jan 4, 2018 at 7:21 PM, Petr Fischer <[hidden email]> wrote:
> Hello All,
>
> Please pardon the newbie question. Trying to get my head around the
> Gemstone64/S Seaside work flow.
>
> I am wanting to use Gemstone as a persistent object store.
> I do not want it to run any client facing applications.
> I want to run all client facing applications in pharo images of there own
> connected to a central backing stone. Is this the wrong way to think about
> this?

I'm just curious - How do you solve communication between the server (Gemstone/backing store) and the client (Pharo images with Seaside)? JSON REST API? Or some transparent object layer?

If you run Seaside apps inside the Gemstone VM, you can access all Gemstone objects directly, which is luxury (and fast).

pf

>
> All the information (at least that I can find) forces the installation of
> seaside into the Gemstone server.  From there worked on in the client
> (tODE).  Having played around with the single installation version where
> all reside on a local machine I notice that I can created multiple stones
> and multiple clients which is great.  But doesn't this take away from
> separating application space from backing store?  Or is the Idea that one
> application gets one stone?
>
> Sorry for the confusing question but I can't find any literature on how
> this is all supposed to fit together in the real world.
>
> Kindly
>
> --
> You received this message because you are subscribed to the Google Groups "tODE" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.



--

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Dale Henrichs-3
In reply to this post by Ian Ian

Ian,

Most of the focus for GLASS/GsDevKit over the last several years has been on porting headless applications to GemStone from Pharo so there has not been a lot of work done on enabling the kind of applications you are talking about ...

As Petr points out your best bet is to write your application as a RESTful GemStone server and use JSON or STON to pass objects back and forth between Pharo and GemStone.

tODE is not really a classic "client application using GemStone as a backing store". tODE is what I call a "very thin client" where the Pharo client is only used for presentation (much like a web browser) and all of the real work is done on the server.

At ESUG in 2016, I presented a rough framework called Tugrik[1],[2],[3] that was aimed at supporting tODE-like applications based on Voyage, but I think that there is a lot of work to be done to move Tugrik into reality ... I would be willing to work with you if you wanted to push on Tugrik.

But I really think that your best bet at the moment would be to go with a RESTful architecture as all of the technology is well developed.

Let me know if you have additional questions,

Dale

[1] https://github.com/dalehenrich/Tugrik
[2] http://esug.org/data/ESUG2016/03-Wednesday/1400-1445%20Tugrik/2016-08-24-Voyage-Tugrik.key.pdf
[3] https://www.youtube.com/watch?v=YwlUdRaqTwE

On 1/4/18 12:47 PM, Ian Ian wrote:
Hello All,

Please pardon the newbie question. Trying to get my head around the Gemstone64/S Seaside work flow.

I am wanting to use Gemstone as a persistent object store.
I do not want it to run any client facing applications.
I want to run all client facing applications in pharo images of there own connected to a central backing stone. Is this the wrong way to think about this?

All the information (at least that I can find) forces the installation of seaside into the Gemstone server.  From there worked on in the client (tODE).  Having played around with the single installation version where all reside on a local machine I notice that I can created multiple stones and multiple clients which is great.  But doesn't this take away from separating application space from backing store?  Or is the Idea that one application gets one stone?

Sorry for the confusing question but I can't find any literature on how this is all supposed to fit together in the real world.

Kindly

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Ian Ian
Thanks for your response Dale,

After posting this question and given the responses I have received to date I realized that I needed to back away and re-think.

It seems that I had it all wrong much as you suggested.  I was looking for an object store able to persist object data as is.  It seems the only way to achieve that is to have whichever type of application you are working on run directly in a stone, further that, given the way the DevKit is setup, create a stone for each and every (non-related, in terms of data) application.

This is something I did not 'get' as, for me, I am used to thinking in a relational sense and was looking at GemStone as a 'database' which, it seems, in the traditional sense, it is not.

If I now have the sense of it understood correctly, If I want to separate the data from the application I must use an object store much like, as you suggest, a RESTful, or some other mechanism is the SaaS world and transfer JSON of STON strings around.  It was this level of indirection that I thought GemStone would remove.  And...,  it seems it does if you use the stone as an application layer and a data store rolled into one.

The part that is still fuzzy for me is the packaging of the application/database for production use.  With only a cursory level of understanding it seems that I would need to take the Gemstone install base and all image modifications along with the extents files for the app and move all to a production box and configure the supporting OS level requirements in the new environment.  Do I have the right sense of this.

Thank you for taking the time to set me straight.

On 8 January 2018 at 09:43, Dale Henrichs <[hidden email]> wrote:

Ian,

Most of the focus for GLASS/GsDevKit over the last several years has been on porting headless applications to GemStone from Pharo so there has not been a lot of work done on enabling the kind of applications you are talking about ...

As Petr points out your best bet is to write your application as a RESTful GemStone server and use JSON or STON to pass objects back and forth between Pharo and GemStone.

tODE is not really a classic "client application using GemStone as a backing store". tODE is what I call a "very thin client" where the Pharo client is only used for presentation (much like a web browser) and all of the real work is done on the server.

At ESUG in 2016, I presented a rough framework called Tugrik[1],[2],[3] that was aimed at supporting tODE-like applications based on Voyage, but I think that there is a lot of work to be done to move Tugrik into reality ... I would be willing to work with you if you wanted to push on Tugrik.

But I really think that your best bet at the moment would be to go with a RESTful architecture as all of the technology is well developed.

Let me know if you have additional questions,

Dale

[1] https://github.com/dalehenrich/Tugrik
[2] http://esug.org/data/ESUG2016/03-Wednesday/1400-1445%20Tugrik/2016-08-24-Voyage-Tugrik.key.pdf
[3] https://www.youtube.com/watch?v=YwlUdRaqTwE

On 1/4/18 12:47 PM, Ian Ian wrote:
Hello All,

Please pardon the newbie question. Trying to get my head around the Gemstone64/S Seaside work flow.

I am wanting to use Gemstone as a persistent object store.
I do not want it to run any client facing applications.
I want to run all client facing applications in pharo images of there own connected to a central backing stone. Is this the wrong way to think about this?

All the information (at least that I can find) forces the installation of seaside into the Gemstone server.  From there worked on in the client (tODE).  Having played around with the single installation version where all reside on a local machine I notice that I can created multiple stones and multiple clients which is great.  But doesn't this take away from separating application space from backing store?  Or is the Idea that one application gets one stone?

Sorry for the confusing question but I can't find any literature on how this is all supposed to fit together in the real world.

Kindly

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "tODE" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tode_st/DhOb1p0H5LE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Dale Henrichs-3


On 01/08/2018 07:06 AM, Ian Ian wrote:
> Thanks for your response Dale,
>
> After posting this question and given the responses I have received to
> date I realized that I needed to back away and re-think.
>
> It seems that I had it all wrong much as you suggested.  I was looking
> for an object store able to persist object data as is.
Well Tugrik[1] actually started as a means for storing arbitrary objects
in GemStone via the MongoTalk/Voyage API, so that GemStone _could_ be
used as a pure object store, but doing a better job at being mongo than
mongo for Smalltalk clients is not necessarily something that is worth
investing a lot of time into ... so actually having your schema on a
server can be advantageous, since you can run Smalltalk on the server,
you can run complex queries over large data sets without having to copy
all of your data to the client (or write your query in JSON for Mongo:).

If you see advantages to to storing remote Smalltalk objects to leverage
server-side functionality, then GemStone is still worth looking at

[1] https://github.com/dalehenrich/Tugrik
> It seems the only way to achieve that is to have whichever type of
> application you are working on run directly in a stone, further that,
> given the way the DevKit is setup, create a stone for each and every
> (non-related, in terms of data) application.
Well, this is not necessarily true. GemStone has a pretty good namespace
implementation that would make it practical to use a single  stone as a
Smalltalk object store for a number of totally unrelated shema ... a
Seaside REST interface could be implemented that extracted data or ran
server-side queries/operations within a particular namespace ... some
GemStone specific work would have to be done and I would be willing to
provide help doing this, but for the most part it would be relatively
straightforward to put into a Seaside application.
>
> This is something I did not 'get' as, for me, I am used to thinking in
> a relational sense and was looking at GemStone as a 'database' which,
> it seems, in the traditional sense, it is not.
Yes there are a couple of things that set it apart from the traditional
view of a database. The fact that GemStone stores data and code is the
real differentiator and can be quite advantageous. However, because
GemStone does store data and code, some of the things easily done with
traditional databases are not quite as straightforward (or even
possible) in GemStone.
>
> If I now have the sense of it understood correctly, If I want to
> separate the data from the application I must use an object store much
> like, as you suggest, a RESTful, or some other mechanism is the SaaS
> world and transfer JSON of STON strings around.  It was this level of
> indirection that I thought GemStone would remove.  And...,  it seems
> it does if you use the stone as an application layer and a data store
> rolled into one.
Yes ... if you can conceive of your RESTful api as allowing for
execution of business logic on the server and/or returning the results
of "arbitraty code" executed on the server (like queries) then the
overhead of passing the result sets back to the client using STON/JSON
may not be as bad as it sounds at first blush ..
>
> The part that is still fuzzy for me is the packaging of the
> application/database for production use.  With only a cursory level of
> understanding it seems that I would need to take the Gemstone install
> base and all image modifications along with the extents files for the
> app and move all to a production box and configure the supporting OS
> level requirements in the new environment.  Do I have the right sense
> of this.
>
Paul DeBruicker has shared his ansible-based deployment scripts in the
GsDevKit_ansible[2] project and GsDevKit_home has formulas for
installing the required OS level dependencies.

There are also some students at HPI that are working on setting up a
Docker container for GemStone, but AFAIK that work has not been made
public yet.

Personally I think the pendulum is swinging away from fat clients
(again) and with code and state stored in a server, I think that
GemStone provides a lot of value in that universe:)

Dale

[2] https://github.com/GsDevKit/GsDevKit_ansible
[3] https://github.com/GsDevKit/GsDevKit_home


--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Richard Sargent
Administrator
In reply to this post by Ian Ian
Ian Ian wrote
> I am used to thinking in a relational sense and was looking at GemStone as
> a 'database' which, it seems, in the traditional sense, it is not.

In some respects, it is better to think of the GemStone repository as a
Smalltalk /image/. Then you think of the Gems as concurrently executing VMs
running the same image, but with transaction management to make it safe.


An SQL database has stored procedures; GemStone has "stored procedures on
steroids". With complete access to all the code in a Smalltalk image, you
can literally do any computable thing you can imagine.

It is a vastly different paradigm from plucking bits of data from a data
store and playing with them in a tier that is essentially completely
isolated from the data. Object Oriented is code + data while SQL is code
versus data.


It's a lot of fun, so jump in, the water's fine!




--
Sent from: http://forum.world.st/tODE-f3744225.html

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Ian Ian
Thanks Richard,

I am 'beginning' to get it.

One thing:  does gemstone have its own package management interface such that from Topaz I can install 'some package'?

Thanks!

On 11 January 2018 at 13:20, Richard Sargent <[hidden email]> wrote:
Ian Ian wrote
> I am used to thinking in a relational sense and was looking at GemStone as
> a 'database' which, it seems, in the traditional sense, it is not.

In some respects, it is better to think of the GemStone repository as a
Smalltalk /image/. Then you think of the Gems as concurrently executing VMs
running the same image, but with transaction management to make it safe.


An SQL database has stored procedures; GemStone has "stored procedures on
steroids". With complete access to all the code in a Smalltalk image, you
can literally do any computable thing you can imagine.

It is a vastly different paradigm from plucking bits of data from a data
store and playing with them in a tier that is essentially completely
isolated from the data. Object Oriented is code + data while SQL is code
versus data.


It's a lot of fun, so jump in, the water's fine!




--
Sent from: http://forum.world.st/tODE-f3744225.html

--
You received this message because you are subscribed to a topic in the Google Groups "tODE" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tode_st/DhOb1p0H5LE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Dale Henrichs-3

Ian,

The GemStone base system does not really support loading packages ...

In 3.4 there is a version of CypressReferenceImplementation[1] and Metacello[2] that have been ported to the base, but those packages are not officially supported for 3.4 (you would have to install using $GEMSTONE/upgrade/bootstrapCypressSupport.topaz and $GEMSTONE/upgrade/bootstrapMetacelloSupport.topaz) ...

If you really wanted to use "packages" and especially if you plan on using Seaside (Seaside requires a GLASS/GsDevKit and GsDevKit_home builds stones with the latest versions of GLASS/GsDevKit installed), then I really recommend that you use GsDevKit_home[3]. You haven't mentioned (I don't think) how you are running and installing GemStone. tODE isn't required, but it provides some very useful features ...

topaz does not directly support loading packages, unless you count running Smalltalk expressions (i.e., standard Metacello load expressions) as supporting package loading, but with GLASS/GsDevKit installed, you can use Monticello and Filetree repositories. In future releases we are planning to support "loading packages from topaz" since we will be officially supporting tonel, filetree and Metacello in the base image along with a full array of package manage tools (in the past we talked about Cypress2 and now we are calling the project Rowan).

With tODE you can run GsDevKit_home scripts like the following in bash to install project XXX:

  "$GS_HOME/bin/todeIt" $stoneName project load XXX

A little more information about how your using/installing/running GemStone right now would help me answer your questions  and posting questions on the GLASS list[1] will reach a broader audience of GemStone users into the mix as well:)

Dale

[1] https://github.com/dalehenrich/CypressReferenceImplementation

[2] https://github.com/Metacello/metacello

[3] https://github.com/GsDevKit/GsDevKit_home

[4] http://forum.world.st/GLASS-f1460844.html


On 1/11/18 11:21 AM, Ian Ian wrote:
Thanks Richard,

I am 'beginning' to get it.

One thing:  does gemstone have its own package management interface such that from Topaz I can install 'some package'?

Thanks!

On 11 January 2018 at 13:20, Richard Sargent <[hidden email]> wrote:
Ian Ian wrote
> I am used to thinking in a relational sense and was looking at GemStone as
> a 'database' which, it seems, in the traditional sense, it is not.

In some respects, it is better to think of the GemStone repository as a
Smalltalk /image/. Then you think of the Gems as concurrently executing VMs
running the same image, but with transaction management to make it safe.


An SQL database has stored procedures; GemStone has "stored procedures on
steroids". With complete access to all the code in a Smalltalk image, you
can literally do any computable thing you can imagine.

It is a vastly different paradigm from plucking bits of data from a data
store and playing with them in a tier that is essentially completely
isolated from the data. Object Oriented is code + data while SQL is code
versus data.


It's a lot of fun, so jump in, the water's fine!




--
Sent from: http://forum.world.st/tODE-f3744225.html

--
You received this message because you are subscribed to a topic in the Google Groups "tODE" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tode_st/DhOb1p0H5LE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Ian Ian
Hi Dale,

I am evaluating Gemstone as an application server/big data repository.  Hence my confusion up till now.

I work for an organization that has rather large data storage requirements (understatement:  into the 100's of petabyte range).

Currently, several independent Apache Hadoop stacks are used to manage this data.  There are several serious issues that are evident with this setup, prime of which is object to relational mapping errors (as datasets in a given space change over time - data is mandated to be kept into perpetuity) regardless of whether the database is NoSQL, SQL of any other schema.  The issue bubbles up into the organization application layers of many diverse and distributed verticals.  Maintenance of the system is not trivial and is very expensive and time consuming.

We use virtualization (in the VMWare sense) to create views into the data.  Generally, access is a split between virtualized kiosk systems, another level of indirection, where by mandate, we must now archive the state of the vm's as well as business data, and direct workstation access,   Not fun, error prone and hardware intensive (from a data storage perspective). 

As I have been reading about object databases and what they are said to buy in terms of efficiency (objects being stored natively and directly) coupled with the fact the we are virtualizing everything we can, I, along with a couple of colleagues, thought that we should take a look at what new (laugh) technologies were available.

After a little time I came across Gemstone.  Did my reading and research and after a period of semantic confusion (still going) became enamored with the possibilities.

All this said, from a network, sys admin point of view, an image that contains the application and it's objects is a godsend if object persistence was guaranteed.  It would allow me to archive a self contained living blob that manages its own living state (in the VMWare snapshot sense) and at the same time the data is available to other blobs/views (seaside).

On views in to the data store:

Currently data mining occurs on the application layer (rather than database level views).  Various development groups create views (applications) to access (possibly update, if they are the owning group) the data.  As mandate trickle (waterfall) from above new data points can be added or removed (some of the mandate for this is governed by changing regulations - in other words it must be done regardless of the technical difficulties involved).  You can image the difficulties and manpower involved.

This brings me back to the application server.  Currently, whatever tool a group uses to access the data, be it browser, remote client application, whatever have you (generally based on the skill sets in the group), requires a set of policies and procedures wrapped around the effort (mandated disaster recovery and etc...).  Tones of work that can never be standardized in any way but nevertheless must be documented and managed.

Okay (to long winded self):

Having spent some time looking at this problem from a smalltalk, image virtualization, integrated application management perspective, all my issues could be alleviated within a single technology.  Yippee!

The spawning of an idea:

Amber -> Seaside -> Gemstone.  One language (integrated properly -> one tool). 

Amber: 

*  Each group, with a small learning curve could create and manage their own views into the data.
*  The serialized javascript, once the client was developed, could be streamed from the application server (Seaside/Gemstone) on demand.  It just becomes another object.

Seaside:

* Client interface manager transport system (HTTP) for any and all views.

Gemstone:

* Object level persistence with no forced mappings of any kind.
* Extensibility. (can gemstone handle data levels in the petadata range?)

Cool!:

* Users/Developers of the system have a completely controlled view into the image/system (perhaps a clustered NginX front end?).
* Image access at the machine level is completely controlled/secure.
* Everything in one place with system knowledge transfer and expertise leveraged vertically and horizontally.
* The images can easily be archived in whatever manner (at the bit level using any version control system if desired) at whatever schedule.
* Another level of 'complete system' security achieved. (Way cool!!)
* Whole systems is a single entity!  Fantastical! 
* A significantly smaller foot print.

* Cost reductions, increased efficiency all the way through...

Pipe dream?  Maybe but I think it is doable over time with smalltalk.  The same system is not doable in any other way with any other technology without extensive dollars and effort and once built making dynamic unplanned for changes virtually impossible.  You are locked in (Silly but happens all the time).  With the system I envision, changes, even schema changes, are relatively easy.

For my money:

* Gemtalk should integrate development tools directly into Gemstone (Gemstone with seaside is an order of magnitude more useful than without it)
* Gemtools should be in the reference client - Pharo - permanently (CGI/RPC, everything).
  What does the tODE buy me, other than access, but a shell?
  Give Pharo a permanent seamless object store - make it easy for me/others. The user base will grow. 
  Gemtalk will sell more licenses in the long run.
* Gemstone should be re-branded as a fully functional pure OO application server with extensible secure persistence built in. (slight difference but HUGE!!! - MS, in truth, is a marketing company)
  Let's see if java can compete with that! (Not - they would be forced to make changes just to try.)
* There are more but I am saying too much as it is....

Lastly,

My proof of concept:

1.  Create a seaside app
2.  Create its client in amber.
3.  Upload the client js to the stone.
4.  Stream the client(s) to the browser based on some form of uri
5.  Manipulated the data store (gemstone) with the client.
6.  Cluster the system.
7.  Develop a DR process.
8.  Sell it

Should on the face of it be simple?




 
 



On 11 January 2018 at 14:53, Dale Henrichs <[hidden email]> wrote:

Ian,

The GemStone base system does not really support loading packages ...

In 3.4 there is a version of CypressReferenceImplementation[1] and Metacello[2] that have been ported to the base, but those packages are not officially supported for 3.4 (you would have to install using $GEMSTONE/upgrade/bootstrapCypressSupport.topaz and $GEMSTONE/upgrade/bootstrapMetacelloSupport.topaz) ...

If you really wanted to use "packages" and especially if you plan on using Seaside (Seaside requires a GLASS/GsDevKit and GsDevKit_home builds stones with the latest versions of GLASS/GsDevKit installed), then I really recommend that you use GsDevKit_home[3]. You haven't mentioned (I don't think) how you are running and installing GemStone. tODE isn't required, but it provides some very useful features ...

topaz does not directly support loading packages, unless you count running Smalltalk expressions (i.e., standard Metacello load expressions) as supporting package loading, but with GLASS/GsDevKit installed, you can use Monticello and Filetree repositories. In future releases we are planning to support "loading packages from topaz" since we will be officially supporting tonel, filetree and Metacello in the base image along with a full array of package manage tools (in the past we talked about Cypress2 and now we are calling the project Rowan).

With tODE you can run GsDevKit_home scripts like the following in bash to install project XXX:

  "$GS_HOME/bin/todeIt" $stoneName project load XXX

A little more information about how your using/installing/running GemStone right now would help me answer your questions  and posting questions on the GLASS list[1] will reach a broader audience of GemStone users into the mix as well:)

Dale

[1] https://github.com/dalehenrich/CypressReferenceImplementation

[2] https://github.com/Metacello/metacello

[3] https://github.com/GsDevKit/GsDevKit_home

[4] http://forum.world.st/GLASS-f1460844.html


On 1/11/18 11:21 AM, Ian Ian wrote:
Thanks Richard,

I am 'beginning' to get it.

One thing:  does gemstone have its own package management interface such that from Topaz I can install 'some package'?

Thanks!

On 11 January 2018 at 13:20, Richard Sargent <[hidden email]> wrote:
Ian Ian wrote
> I am used to thinking in a relational sense and was looking at GemStone as
> a 'database' which, it seems, in the traditional sense, it is not.

In some respects, it is better to think of the GemStone repository as a
Smalltalk /image/. Then you think of the Gems as concurrently executing VMs
running the same image, but with transaction management to make it safe.


An SQL database has stored procedures; GemStone has "stored procedures on
steroids". With complete access to all the code in a Smalltalk image, you
can literally do any computable thing you can imagine.

It is a vastly different paradigm from plucking bits of data from a data
store and playing with them in a tier that is essentially completely
isolated from the data. Object Oriented is code + data while SQL is code
versus data.


It's a lot of fun, so jump in, the water's fine!




--
Sent from: http://forum.world.st/tODE-f3744225.html

--
You received this message because you are subscribed to a topic in the Google Groups "tODE" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tode_st/DhOb1p0H5LE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "tODE" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tode_st/DhOb1p0H5LE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Petr Fischer
I like your "For my money" section.
+1

pf


> Hi Dale,
>
> I am evaluating Gemstone as an application server/big data repository.
> Hence my confusion up till now.
>
> I work for an organization that has rather large data storage requirements
> (understatement:  into the 100's of petabyte range).
>
> Currently, several independent Apache Hadoop stacks are used to manage this
> data.  There are several serious issues that are evident with this setup,
> prime of which is object to relational mapping errors (as datasets in a
> given space change over time - data is mandated to be kept into perpetuity)
> regardless of whether the database is NoSQL, SQL of any other schema.  The
> issue bubbles up into the organization application layers of many diverse
> and distributed verticals.  Maintenance of the system is not trivial and is
> very expensive and time consuming.
>
> We use virtualization (in the VMWare sense) to create views into the data.
> Generally, access is a split between virtualized kiosk systems, another
> level of indirection, where by mandate, we must now archive the state of
> the vm's as well as business data, and direct workstation access,   Not
> fun, error prone and hardware intensive (from a data storage perspective).
>
> As I have been reading about object databases and what they are said to buy
> in terms of efficiency (objects being stored natively and directly) coupled
> with the fact the we are virtualizing everything we can, I, along with a
> couple of colleagues, thought that we should take a look at what new
> (laugh) technologies were available.
>
> After a little time I came across Gemstone.  Did my reading and research
> and after a period of semantic confusion (still going) became enamored with
> the possibilities.
>
> All this said, from a network, sys admin point of view, an image that
> contains the application and it's objects is a godsend if object
> persistence was guaranteed.  It would allow me to archive a self contained
> living blob that manages its own living state (in the VMWare snapshot
> sense) and at the same time the data is available to other blobs/views
> (seaside).
>
> On views in to the data store:
>
> Currently data mining occurs on the application layer (rather than database
> level views).  Various development groups create views (applications) to
> access (possibly update, if they are the owning group) the data.  As
> mandate trickle (waterfall) from above new data points can be added or
> removed (some of the mandate for this is governed by changing regulations -
> in other words it must be done regardless of the technical difficulties
> involved).  You can image the difficulties and manpower involved.
>
> This brings me back to the application server.  Currently, whatever tool a
> group uses to access the data, be it browser, remote client application,
> whatever have you (generally based on the skill sets in the group),
> requires a set of policies and procedures wrapped around the effort
> (mandated disaster recovery and etc...).  Tones of work that can never be
> standardized in any way but nevertheless must be documented and managed.
>
> Okay (to long winded self):
>
> Having spent some time looking at this problem from a smalltalk, image
> virtualization, integrated application management perspective, all my
> issues could be alleviated within a single technology.  Yippee!
>
> The spawning of an idea:
>
> Amber -> Seaside -> Gemstone.  One language (integrated properly -> one
> tool).
>
> Amber:
>
> *  Each group, with a small learning curve could create and manage their
> own views into the data.
> *  The serialized javascript, once the client was developed, could be
> streamed from the application server (Seaside/Gemstone) on demand.  It just
> becomes another object.
>
> Seaside:
>
> * Client interface manager transport system (HTTP) for any and all views.
>
> Gemstone:
>
> * Object level persistence with no forced mappings of any kind.
> * Extensibility. (can gemstone handle data levels in the petadata range?)
>
> Cool!:
>
> * Users/Developers of the system have a completely controlled view into the
> image/system (perhaps a clustered NginX front end?).
> * Image access at the machine level is completely controlled/secure.
> * Everything in one place with system knowledge transfer and expertise
> leveraged vertically and horizontally.
> * The images can easily be archived in whatever manner (at the bit level
> using any version control system if desired) at whatever schedule.
> * Another level of 'complete system' security achieved. (Way cool!!)
> * Whole systems is a single entity!  Fantastical!
> * A significantly smaller foot print.
>
> * Cost reductions, increased efficiency all the way through...
>
> Pipe dream?  Maybe but I think it is doable over time with smalltalk.  The
> same system is not doable in any other way with any other technology
> without extensive dollars and effort and once built making dynamic
> unplanned for changes virtually impossible.  You are locked in (Silly but
> happens all the time).  With the system I envision, changes, even schema
> changes, are relatively easy.
>
> For my money:
>
> * Gemtalk should integrate development tools directly into Gemstone
> (Gemstone with seaside is an order of magnitude more useful than without it)
> * Gemtools should be in the reference client - Pharo - permanently
> (CGI/RPC, everything).
>   What does the tODE buy me, other than access, but a shell?
>   Give Pharo a permanent seamless object store - make it easy for
> me/others. The user base will grow.
>   Gemtalk will sell more licenses in the long run.
> * Gemstone should be re-branded as a fully functional pure OO application
> server with extensible secure persistence built in. (slight difference but
> HUGE!!! - MS, in truth, is a marketing company)
>   Let's see if java can compete with that! (Not - they would be forced to
> make changes just to try.)
> * There are more but I am saying too much as it is....
>
> Lastly,
>
> My proof of concept:
>
> 1.  Create a seaside app
> 2.  Create its client in amber.
> 3.  Upload the client js to the stone.
> 4.  Stream the client(s) to the browser based on some form of uri
> 5.  Manipulated the data store (gemstone) with the client.
> 6.  Cluster the system.
> 7.  Develop a DR process.
> 8.  Sell it
>
> Should on the face of it be simple?
>
>
>
>
>
>
>
>
>
> On 11 January 2018 at 14:53, Dale Henrichs <[hidden email]
> > wrote:
>
> > Ian,
> >
> > The GemStone base system does not really support loading packages ...
> >
> > In 3.4 there is a version of CypressReferenceImplementation[1] and
> > Metacello[2] that have been ported to the base, but those packages are not
> > officially supported for 3.4 (you would have to install using
> > $GEMSTONE/upgrade/bootstrapCypressSupport.topaz and $GEMSTONE/upgrade/
> > bootstrapMetacelloSupport.topaz) ...
> >
> > If you really wanted to use "packages" and especially if you plan on using
> > Seaside (Seaside requires a GLASS/GsDevKit and GsDevKit_home builds stones
> > with the latest versions of GLASS/GsDevKit installed), then I really
> > recommend that you use GsDevKit_home[3]. You haven't mentioned (I don't
> > think) how you are running and installing GemStone. tODE isn't required,
> > but it provides some very useful features ...
> >
> > topaz does not directly support loading packages, unless you count running
> > Smalltalk expressions (i.e., standard Metacello load expressions) as
> > supporting package loading, but with GLASS/GsDevKit installed, you can use
> > Monticello and Filetree repositories. In future releases we are planning to
> > support "loading packages from topaz" since we will be officially
> > supporting tonel, filetree and Metacello in the base image along with a
> > full array of package manage tools (in the past we talked about Cypress2
> > and now we are calling the project Rowan).
> >
> > With tODE you can run GsDevKit_home scripts like the following in bash to
> > install project XXX:
> >
> >   "$GS_HOME/bin/todeIt" $stoneName project load XXX
> >
> > A little more information about how your using/installing/running GemStone
> > right now would help me answer your questions  and posting questions on the
> > GLASS list[1] will reach a broader audience of GemStone users into the mix
> > as well:)
> >
> > Dale
> >
> > [1] https://github.com/dalehenrich/CypressReferenceImplementation
> >
> > [2] https://github.com/Metacello/metacello
> >
> > [3] https://github.com/GsDevKit/GsDevKit_home
> >
> > [4] http://forum.world.st/GLASS-f1460844.html
> >
> > On 1/11/18 11:21 AM, Ian Ian wrote:
> >
> > Thanks Richard,
> >
> > I am 'beginning' to get it.
> >
> > One thing:  does gemstone have its own package management interface such
> > that from Topaz I can install 'some package'?
> >
> > Thanks!
> >
> > On 11 January 2018 at 13:20, Richard Sargent <richard.sargent@
> > gemtalksystems.com> wrote:
> >
> >> Ian Ian wrote
> >> > I am used to thinking in a relational sense and was looking at GemStone
> >> as
> >> > a 'database' which, it seems, in the traditional sense, it is not.
> >>
> >> In some respects, it is better to think of the GemStone repository as a
> >> Smalltalk /image/. Then you think of the Gems as concurrently executing
> >> VMs
> >> running the same image, but with transaction management to make it safe.
> >>
> >>
> >> An SQL database has stored procedures; GemStone has "stored procedures on
> >> steroids". With complete access to all the code in a Smalltalk image, you
> >> can literally do any computable thing you can imagine.
> >>
> >> It is a vastly different paradigm from plucking bits of data from a data
> >> store and playing with them in a tier that is essentially completely
> >> isolated from the data. Object Oriented is code + data while SQL is code
> >> versus data.
> >>
> >>
> >> It's a lot of fun, so jump in, the water's fine!
> >>
> >>
> >>
> >>
> >> --
> >> Sent from: http://forum.world.st/tODE-f3744225.html
> >>
> >> --
> >> You received this message because you are subscribed to a topic in the
> >> Google Groups "tODE" group.
> >> To unsubscribe from this topic, visit https://groups.google.com/d/to
> >> pic/tode_st/DhOb1p0H5LE/unsubscribe.
> >> To unsubscribe from this group and all its topics, send an email to
> >> [hidden email].
> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "tODE" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [hidden email].
> > For more options, visit https://groups.google.com/d/optout.
> >
> >
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "tODE" group.
> > To unsubscribe from this topic, visit https://groups.google.com/d/
> > topic/tode_st/DhOb1p0H5LE/unsubscribe.
> > To unsubscribe from this group and all its topics, send an email to
> > [hidden email].
> > For more options, visit https://groups.google.com/d/optout.
> >
>
> --
> You received this message because you are subscribed to the Google Groups "tODE" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Richard Sargent
Administrator
In reply to this post by Ian Ian
Ian Ian wrote
> Hi Dale,
>
> I am evaluating Gemstone as an application server/big data repository.
> Hence my confusion up till now.
>
> I work for an organization that has rather large data storage requirements
> (understatement:  into the 100's of petabyte range).

How much repetition is present in the data? One advantage of an object
database is that instances with the same representation can be represented
by a single object. So, if the "columns" of your current data have often
recurring strings, dates, etc., you can greatly reduce the space
requirements for them.

Please feel free to elaborate on your application and its data, if you can.



> Currently, several independent Apache Hadoop stacks are used to manage
> this
> data.  There are several serious issues that are evident with this setup,
> prime of which is object to relational mapping errors (as datasets in a
> given space change over time - data is mandated to be kept into
> perpetuity)
> regardless of whether the database is NoSQL, SQL of any other schema.  The
> issue bubbles up into the organization application layers of many diverse
> and distributed verticals.  Maintenance of the system is not trivial and
> is
> very expensive and time consuming.
>
> We use virtualization (in the VMWare sense) to create views into the data.
> Generally, access is a split between virtualized kiosk systems, another
> level of indirection, where by mandate, we must now archive the state of
> the vm's as well as business data, and direct workstation access,   Not
> fun, error prone and hardware intensive (from a data storage perspective).
>
> As I have been reading about object databases and what they are said to
> buy
> in terms of efficiency (objects being stored natively and directly)
> coupled
> with the fact the we are virtualizing everything we can, I, along with a
> couple of colleagues, thought that we should take a look at what new
> (laugh) technologies were available.
>
> After a little time I came across Gemstone.  Did my reading and research
> and after a period of semantic confusion (still going) became enamored
> with
> the possibilities.
>
> All this said, from a network, sys admin point of view, an image that
> contains the application and it's objects is a godsend if object
> persistence was guaranteed.  It would allow me to archive a self contained
> living blob that manages its own living state (in the VMWare snapshot
> sense) and at the same time the data is available to other blobs/views
> (seaside).
>
> On views in to the data store:
>
> Currently data mining occurs on the application layer (rather than
> database
> level views).  Various development groups create views (applications) to
> access (possibly update, if they are the owning group) the data.  As
> mandate trickle (waterfall) from above new data points can be added or
> removed (some of the mandate for this is governed by changing regulations
> -
> in other words it must be done regardless of the technical difficulties
> involved).  You can image the difficulties and manpower involved.
>
> This brings me back to the application server.  Currently, whatever tool a
> group uses to access the data, be it browser, remote client application,
> whatever have you (generally based on the skill sets in the group),
> requires a set of policies and procedures wrapped around the effort
> (mandated disaster recovery and etc...).  Tones of work that can never be
> standardized in any way but nevertheless must be documented and managed.
>
> Okay (to long winded self):
>
> Having spent some time looking at this problem from a smalltalk, image
> virtualization, integrated application management perspective, all my
> issues could be alleviated within a single technology.  Yippee!
>
> The spawning of an idea:
>
> Amber -> Seaside -> Gemstone.  One language (integrated properly -> one
> tool).
>
> Amber:
>
> *  Each group, with a small learning curve could create and manage their
> own views into the data.
> *  The serialized javascript, once the client was developed, could be
> streamed from the application server (Seaside/Gemstone) on demand.  It
> just
> becomes another object.
>
> Seaside:
>
> * Client interface manager transport system (HTTP) for any and all views.
>
> Gemstone:
>
> * Object level persistence with no forced mappings of any kind.
> * Extensibility. (can gemstone handle data levels in the petadata range?)
>
> Cool!:
>
> * Users/Developers of the system have a completely controlled view into
> the
> image/system (perhaps a clustered NginX front end?).
> * Image access at the machine level is completely controlled/secure.
> * Everything in one place with system knowledge transfer and expertise
> leveraged vertically and horizontally.
> * The images can easily be archived in whatever manner (at the bit level
> using any version control system if desired) at whatever schedule.
> * Another level of 'complete system' security achieved. (Way cool!!)
> * Whole systems is a single entity!  Fantastical!
> * A significantly smaller foot print.
>
> * Cost reductions, increased efficiency all the way through...
>
> Pipe dream?  Maybe but I think it is doable over time with smalltalk.  The
> same system is not doable in any other way with any other technology
> without extensive dollars and effort and once built making dynamic
> unplanned for changes virtually impossible.  You are locked in (Silly but
> happens all the time).  With the system I envision, changes, even schema
> changes, are relatively easy.
>
> For my money:
>
> * Gemtalk should integrate development tools directly into Gemstone
> (Gemstone with seaside is an order of magnitude more useful than without
> it)
> * Gemtools should be in the reference client - Pharo - permanently
> (CGI/RPC, everything).
>   What does the tODE buy me, other than access, but a shell?
>   Give Pharo a permanent seamless object store - make it easy for
> me/others. The user base will grow.
>   Gemtalk will sell more licenses in the long run.
> * Gemstone should be re-branded as a fully functional pure OO application
> server with extensible secure persistence built in. (slight difference but
> HUGE!!! - MS, in truth, is a marketing company)
>   Let's see if java can compete with that! (Not - they would be forced to
> make changes just to try.)
> * There are more but I am saying too much as it is....
>
> Lastly,
>
> My proof of concept:
>
> 1.  Create a seaside app
> 2.  Create its client in amber.
> 3.  Upload the client js to the stone.
> 4.  Stream the client(s) to the browser based on some form of uri
> 5.  Manipulated the data store (gemstone) with the client.
> 6.  Cluster the system.
> 7.  Develop a DR process.
> 8.  Sell it
>
> Should on the face of it be simple?
>
>
>
>
>
>
>
>
>
> On 11 January 2018 at 14:53, Dale Henrichs &lt;

> dale.henrichs@

> &gt; wrote:
>
>> Ian,
>>
>> The GemStone base system does not really support loading packages ...
>>
>> In 3.4 there is a version of CypressReferenceImplementation[1] and
>> Metacello[2] that have been ported to the base, but those packages are
>> not
>> officially supported for 3.4 (you would have to install using
>> $GEMSTONE/upgrade/bootstrapCypressSupport.topaz and $GEMSTONE/upgrade/
>> bootstrapMetacelloSupport.topaz) ...
>>
>> If you really wanted to use "packages" and especially if you plan on
>> using
>> Seaside (Seaside requires a GLASS/GsDevKit and GsDevKit_home builds
>> stones
>> with the latest versions of GLASS/GsDevKit installed), then I really
>> recommend that you use GsDevKit_home[3]. You haven't mentioned (I don't
>> think) how you are running and installing GemStone. tODE isn't required,
>> but it provides some very useful features ...
>>
>> topaz does not directly support loading packages, unless you count
>> running
>> Smalltalk expressions (i.e., standard Metacello load expressions) as
>> supporting package loading, but with GLASS/GsDevKit installed, you can
>> use
>> Monticello and Filetree repositories. In future releases we are planning
>> to
>> support "loading packages from topaz" since we will be officially
>> supporting tonel, filetree and Metacello in the base image along with a
>> full array of package manage tools (in the past we talked about Cypress2
>> and now we are calling the project Rowan).
>>
>> With tODE you can run GsDevKit_home scripts like the following in bash to
>> install project XXX:
>>
>>   "$GS_HOME/bin/todeIt" $stoneName project load XXX
>>
>> A little more information about how your using/installing/running
>> GemStone
>> right now would help me answer your questions  and posting questions on
>> the
>> GLASS list[1] will reach a broader audience of GemStone users into the
>> mix
>> as well:)
>>
>> Dale
>>
>> [1] https://github.com/dalehenrich/CypressReferenceImplementation
>>
>> [2] https://github.com/Metacello/metacello
>>
>> [3] https://github.com/GsDevKit/GsDevKit_home
>>
>> [4] http://forum.world.st/GLASS-f1460844.html
>>
>> On 1/11/18 11:21 AM, Ian Ian wrote:
>>
>> Thanks Richard,
>>
>> I am 'beginning' to get it.
>>
>> One thing:  does gemstone have its own package management interface such
>> that from Topaz I can install 'some package'?
>>
>> Thanks!
>>
>> On 11 January 2018 at 13:20, Richard Sargent &lt;richard.sargent@
> &gt; gemtalksystems.com> wrote:
>>
>>> Ian Ian wrote
>>> > I am used to thinking in a relational sense and was looking at
>>> GemStone
>>> as
>>> > a 'database' which, it seems, in the traditional sense, it is not.
>>>
>>> In some respects, it is better to think of the GemStone repository as a
>>> Smalltalk /image/. Then you think of the Gems as concurrently executing
>>> VMs
>>> running the same image, but with transaction management to make it safe.
>>>
>>>
>>> An SQL database has stored procedures; GemStone has "stored procedures
>>> on
>>> steroids". With complete access to all the code in a Smalltalk image,
>>> you
>>> can literally do any computable thing you can imagine.
>>>
>>> It is a vastly different paradigm from plucking bits of data from a data
>>> store and playing with them in a tier that is essentially completely
>>> isolated from the data. Object Oriented is code + data while SQL is code
>>> versus data.
>>>
>>>
>>> It's a lot of fun, so jump in, the water's fine!
>>>
>>>
>>>
>>>
>>> --
>>> Sent from: http://forum.world.st/tODE-f3744225.html
>>>
>>> --
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "tODE" group.
>>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>>> pic/tode_st/DhOb1p0H5LE/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>>

> tode_st+unsubscribe@

> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "tODE" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to

> tode_st+unsubscribe@

> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "tODE" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/
>> topic/tode_st/DhOb1p0H5LE/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>>

> tode_st+unsubscribe@

> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "tODE" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to

> tode_st+unsubscribe@

> .
> For more options, visit https://groups.google.com/d/optout.





--
Sent from: http://forum.world.st/tODE-f3744225.html

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Dale Henrichs-3
In reply to this post by Ian Ian


On 1/11/18 2:24 PM, Ian Ian wrote:

> Hi Dale,
>
> I am evaluating Gemstone as an application server/big data
> repository.  Hence my confusion up till now.
>
> I work for an organization that has rather large data storage
> requirements (understatement:  into the 100's of petabyte range).
>
> Currently, several independent Apache Hadoop stacks are used to manage
> this data.  There are several serious issues that are evident with
> this setup, prime of which is object to relational mapping errors (as
> datasets in a given space change over time - data is mandated to be
> kept into perpetuity) regardless of whether the database is NoSQL, SQL
> of any other schema.  The issue bubbles up into the organization
> application layers of many diverse and distributed verticals. 
> Maintenance of the system is not trivial and is very expensive and
> time consuming.
>
> We use virtualization (in the VMWare sense) to create views into the
> data.  Generally, access is a split between virtualized kiosk systems,
> another level of indirection, where by mandate, we must now archive
> the state of the vm's as well as business data, and direct workstation
> access, Not fun, error prone and hardware intensive (from a data
> storage perspective).
>
> As I have been reading about object databases and what they are said
> to buy in terms of efficiency (objects being stored natively and
> directly) coupled with the fact the we are virtualizing everything we
> can, I, along with a couple of colleagues, thought that we should take
> a look at what new (laugh) technologies were available.
>
> After a little time I came across Gemstone.  Did my reading and
> research and after a period of semantic confusion (still going) became
> enamored with the possibilities.
Interesting ... I guess I have sorta been waiting for someone to
discover us as I have thought for a while now that data modeling and
exploration based on extractions from  big data streams would be a niche
where GemStone could provide some real value...

>
> All this said, from a network, sys admin point of view, an image that
> contains the application and it's objects is a godsend if object
> persistence was guaranteed.  It would allow me to archive a self
> contained living blob that manages its own living state (in the VMWare
> snapshot sense) and at the same time the data is available to other
> blobs/views (seaside).
>
> On views in to the data store:
>
> Currently data mining occurs on the application layer (rather than
> database level views).  Various development groups create views
> (applications) to access (possibly update, if they are the owning
> group) the data.  As mandate trickle (waterfall) from above new data
> points can be added or removed (some of the mandate for this is
> governed by changing regulations - in other words it must be done
> regardless of the technical difficulties involved).  You can image the
> difficulties and manpower involved.
>
> This brings me back to the application server. Currently, whatever
> tool a group uses to access the data, be it browser, remote client
> application, whatever have you (generally based on the skill sets in
> the group), requires a set of policies and procedures wrapped around
> the effort (mandated disaster recovery and etc...).  Tones of work
> that can never be standardized in any way but nevertheless must be
> documented and managed.
>
> Okay (to long winded self):
>
> Having spent some time looking at this problem from a smalltalk, image
> virtualization, integrated application management perspective, all my
> issues could be alleviated within a single technology.  Yippee!
Haha ...
>
> The spawning of an idea:
>
> Amber -> Seaside -> Gemstone.  One language (integrated properly ->
> one tool).
A full-stack Smalltalk solution makes a ton of sense.

>
> Amber:
>
> *  Each group, with a small learning curve could create and manage
> their own views into the data.
> *  The serialized javascript, once the client was developed, could be
> streamed from the application server (Seaside/Gemstone) on demand.  It
> just becomes another object.
>
> Seaside:
>
> * Client interface manager transport system (HTTP) for any and all views.
>
> Gemstone:
>
> * Object level persistence with no forced mappings of any kind.
> * Extensibility. (can gemstone handle data levels in the petadata range?)
The current release has a design limit of about 7 petabytes and an
object count of 128 billion (there is database metadata that needs to be
accounted for as well:)... depending upon the nature of the data being
stored there would be some simple changes we could make that could
extend the limit into the 100's of petabytes, of course there are
additional changes that we could make to increase the bytecount limit.
>
> Cool!:
Now of course the devil is in the details, but this makes a lot of sense
... as an alternate architecture to study, I would point to the work of
Martin Feldtmann[1]. Marten uses Swagger to generate multi-language
mappings for the RESTful API that he is using where GemStone/S is the
database of record ... I'm sure that Marten would be willing to provide
additional details if asked.

A full-stack Smalltalk application of course has tons of appeal, but the
realist in me knows that the entire world does not use Smalltalk, so the
Swagger approach simplifies the process of maintaining a mulit-language
API against a GemStone dB.

[1]
https://schrievkrom.wordpress.com/2016/03/15/gemstones-application-example/
[2] https://swagger.io/

>
> * Users/Developers of the system have a completely controlled view
> into the image/system (perhaps a clustered NginX front end?).
> * Image access at the machine level is completely controlled/secure.
> * Everything in one place with system knowledge transfer and expertise
> leveraged vertically and horizontally.
> * The images can easily be archived in whatever manner (at the bit
> level using any version control system if desired) at whatever schedule.
> * Another level of 'complete system' security achieved. (Way cool!!)
> * Whole systems is a single entity!  Fantastical!
> * A significantly smaller foot print.
>
> * Cost reductions, increased efficiency all the way through...
>
> Pipe dream?  Maybe but I think it is doable over time with smalltalk. 
> The same system is not doable in any other way with any other
> technology without extensive dollars and effort and once built making
> dynamic unplanned for changes virtually impossible.  You are locked in
> (Silly but happens all the time).  With the system I envision,
> changes, even schema changes, are relatively easy.
Agreed.
>
> For my money:
>
> * Gemtalk should integrate development tools directly into Gemstone
> (Gemstone with seaside is an order of magnitude more useful than
> without it)
This is now on the roadmap and things should start showing up in our
next release.
> * Gemtools should be in the reference client - Pharo - permanently
> (CGI/RPC, everything).
We are definitely headed in this direction.
>   What does the tODE buy me, other than access, but a shell?
tODE is actually involves a couple of different distinct technologies
beyond the shell: an ultra-thin client, a multi-client tools
architecture, and a prototype for git-based development.

tODE uses an ultra-thin Pharo client[6]. There is no business logic on
the Pharo client, it is simply used for it's display capabilities ---
think of html in browsers where you have a variety of widgets, but the
contents of the widgets are described by objects passed back and forth
between the server and client, rather than html. The result is that
object replication is restricted to only display artifacts ... the
business objects remain on the server at all times and all code is
actually executed in server...

The ultra-thin client has the advantage of eliminating the need to
implement any kind of replication scheme between the client and server,
except that needed for the display which are only strings and numbers
and objects of limited depth. Replicating arbitrary and/or deeply nested
objects is where a lot of complication comes into play.

The complication for the ultra-thin client comes in providing a set of
rich and responsive widgets for implementing the GUI -- presumably a
much simpler.

Speaking of alternate approaches, a year ago I did a simple proof of
concept[7] aimed at allowing the Pharo class library to be run natively
in a GemStone repository using some of the base technology that we
developed for Maglev[8]. In theory, Pharo programs (and even the Pharo
GUI) would run "natively" in GemStone.

There is a group of bachelor students at HPI[9] currently working on
running Squeak programs (and the Squeak GUI) in GemStone.

[6]
https://www.slideshare.net/esug/tode-and-now-for-something-completely-different
[7] https://github.com/dalehenrich/PharoGs
[8] https://maglev.github.io/
[9] https://hpi.de/en/
>   Give Pharo a permanent seamless object store - make it easy for
> me/others. The user base will grow.
Giving "Pharo a permanent seamless object store" is not explicitly on
the roadmap as we are still in the early stages of exploration. We will
need some sort of object replication scheme to implement the development
environment. GemBuilder for Smalltalk (GBS)[3] is pretty much a "a
permanent seamless object store"  that exists for VW[4] and VA[5], but
it is also incredibly complex and we will be exploring simpler/cheaper
to implement alternatives before investing in a full-blown GBS for Pharo.

Martin McClure (responsible for GBS) has a new replication architecture
in mind for the upcoming Pharo client that eliminates much of the
complication entailed in GBS, however, it _is_ still in development, so
we won't know for a while how far it goes in providing a "Pharo a
permanent seamless object store."

[3]
https://downloads.gemtalksystems.com/docs/GBS/7.6.x/GBS-VW-UsersGuide-7.6/GBS-VW-UsersGuide-7.6.htm
[4] https://gemtalksystems.com/products/gbs-vw/
[5] https://gemtalksystems.com/products/gbs-va/

>   Gemtalk will sell more licenses in the long run.

Ultimately where we head and invest resources will be driven by customer
demand ... and at the moment we are moving towards providing support for
git-based development in the kernel with a Pharo client for development.

> * Gemstone should be re-branded as a fully functional pure OO
> application server with extensible secure persistence built in.
> (slight difference but HUGE!!! - MS, in truth, is a marketing company)
>   Let's see if java can compete with that! (Not - they would be forced
> to make changes just to try.)
I think we would need some pioneering work by folks with domain
knowledge so that we can provide concrete examples and (more
importantly) make sure that the path is a smooth one ... and I see that
you are proposing a proof of concept below ..

> * There are more but I am saying too much as it is....
>
> Lastly,
>
> My proof of concept:
>
> 1.  Create a seaside app
> 2.  Create its client in amber.
> 3.  Upload the client js to the stone.
> 4.  Stream the client(s) to the browser based on some form of uri
> 5.  Manipulated the data store (gemstone) with the client.
> 6.  Cluster the system.
> 7.  Develop a DR process.
> 8.  Sell it
>
> Should on the face of it be simple?
>
It should be relatively straightforward.

Let me know when you are planning on starting your proof of concept
project as I would like to update the GsDevit_home Seaside installation
instructions[10] to use Seaside 3.2 instead of Seaside 3.1 (you should
be using the latest version of Seaside).

It is worth getting on the GLASS[11], Seaside[12], and Amber[13] mailing
lists so that you can get quick answers to your questions.

For GsDevKit_home and GemStone specific advice I am definitely
interested in helping.

Dale

[10]
https://github.com/GsDevKit/GsDevKit_home/blob/master/docs/gettingStartedWithSeaside.md
[11] http://forum.world.st/mailing_list/MailingListOptions.jtp?forum=1460844
[12] http://forum.world.st/mailing_list/MailingListOptions.jtp?forum=86180
[13] http://forum.world.st/mailing_list/MailingListOptions.jtp?forum=3824463

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [tode_st] Gemstone & Seaside

Ian Ian
Thanks Dale. 

On Jan 15, 2018 2:28 PM, "Dale Henrichs" <[hidden email]> wrote:


On 1/11/18 2:24 PM, Ian Ian wrote:
Hi Dale,

I am evaluating Gemstone as an application server/big data repository.  Hence my confusion up till now.

I work for an organization that has rather large data storage requirements (understatement:  into the 100's of petabyte range).

Currently, several independent Apache Hadoop stacks are used to manage this data.  There are several serious issues that are evident with this setup, prime of which is object to relational mapping errors (as datasets in a given space change over time - data is mandated to be kept into perpetuity) regardless of whether the database is NoSQL, SQL of any other schema.  The issue bubbles up into the organization application layers of many diverse and distributed verticals.  Maintenance of the system is not trivial and is very expensive and time consuming.

We use virtualization (in the VMWare sense) to create views into the data.  Generally, access is a split between virtualized kiosk systems, another level of indirection, where by mandate, we must now archive the state of the vm's as well as business data, and direct workstation access, Not fun, error prone and hardware intensive (from a data storage perspective).

As I have been reading about object databases and what they are said to buy in terms of efficiency (objects being stored natively and directly) coupled with the fact the we are virtualizing everything we can, I, along with a couple of colleagues, thought that we should take a look at what new (laugh) technologies were available.

After a little time I came across Gemstone.  Did my reading and research and after a period of semantic confusion (still going) became enamored with the possibilities.
Interesting ... I guess I have sorta been waiting for someone to discover us as I have thought for a while now that data modeling and exploration based on extractions from  big data streams would be a niche where GemStone could provide some real value...

All this said, from a network, sys admin point of view, an image that contains the application and it's objects is a godsend if object persistence was guaranteed.  It would allow me to archive a self contained living blob that manages its own living state (in the VMWare snapshot sense) and at the same time the data is available to other blobs/views (seaside).

On views in to the data store:

Currently data mining occurs on the application layer (rather than database level views).  Various development groups create views (applications) to access (possibly update, if they are the owning group) the data.  As mandate trickle (waterfall) from above new data points can be added or removed (some of the mandate for this is governed by changing regulations - in other words it must be done regardless of the technical difficulties involved).  You can image the difficulties and manpower involved.

This brings me back to the application server. Currently, whatever tool a group uses to access the data, be it browser, remote client application, whatever have you (generally based on the skill sets in the group), requires a set of policies and procedures wrapped around the effort (mandated disaster recovery and etc...).  Tones of work that can never be standardized in any way but nevertheless must be documented and managed.

Okay (to long winded self):

Having spent some time looking at this problem from a smalltalk, image virtualization, integrated application management perspective, all my issues could be alleviated within a single technology.  Yippee!
Haha ...

The spawning of an idea:

Amber -> Seaside -> Gemstone.  One language (integrated properly -> one tool).
A full-stack Smalltalk solution makes a ton of sense.

Amber:

*  Each group, with a small learning curve could create and manage their own views into the data.
*  The serialized javascript, once the client was developed, could be streamed from the application server (Seaside/Gemstone) on demand.  It just becomes another object.

Seaside:

* Client interface manager transport system (HTTP) for any and all views.

Gemstone:

* Object level persistence with no forced mappings of any kind.
* Extensibility. (can gemstone handle data levels in the petadata range?)
The current release has a design limit of about 7 petabytes and an object count of 128 billion (there is database metadata that needs to be accounted for as well:)... depending upon the nature of the data being stored there would be some simple changes we could make that could extend the limit into the 100's of petabytes, of course there are additional changes that we could make to increase the bytecount limit.

Cool!:
Now of course the devil is in the details, but this makes a lot of sense ... as an alternate architecture to study, I would point to the work of Martin Feldtmann[1]. Marten uses Swagger to generate multi-language mappings for the RESTful API that he is using where GemStone/S is the database of record ... I'm sure that Marten would be willing to provide additional details if asked.

A full-stack Smalltalk application of course has tons of appeal, but the realist in me knows that the entire world does not use Smalltalk, so the Swagger approach simplifies the process of maintaining a mulit-language API against a GemStone dB.

[1] https://schrievkrom.wordpress.com/2016/03/15/gemstones-application-example/
[2] https://swagger.io/

* Users/Developers of the system have a completely controlled view into the image/system (perhaps a clustered NginX front end?).
* Image access at the machine level is completely controlled/secure.
* Everything in one place with system knowledge transfer and expertise leveraged vertically and horizontally.
* The images can easily be archived in whatever manner (at the bit level using any version control system if desired) at whatever schedule.
* Another level of 'complete system' security achieved. (Way cool!!)
* Whole systems is a single entity!  Fantastical!
* A significantly smaller foot print.

* Cost reductions, increased efficiency all the way through...

Pipe dream?  Maybe but I think it is doable over time with smalltalk.  The same system is not doable in any other way with any other technology without extensive dollars and effort and once built making dynamic unplanned for changes virtually impossible.  You are locked in (Silly but happens all the time).  With the system I envision, changes, even schema changes, are relatively easy.
Agreed.

For my money:

* Gemtalk should integrate development tools directly into Gemstone (Gemstone with seaside is an order of magnitude more useful than without it)
This is now on the roadmap and things should start showing up in our next release.
* Gemtools should be in the reference client - Pharo - permanently (CGI/RPC, everything).
We are definitely headed in this direction.
  What does the tODE buy me, other than access, but a shell?
tODE is actually involves a couple of different distinct technologies beyond the shell: an ultra-thin client, a multi-client tools architecture, and a prototype for git-based development.

tODE uses an ultra-thin Pharo client[6]. There is no business logic on the Pharo client, it is simply used for it's display capabilities --- think of html in browsers where you have a variety of widgets, but the contents of the widgets are described by objects passed back and forth between the server and client, rather than html. The result is that object replication is restricted to only display artifacts ... the business objects remain on the server at all times and all code is actually executed in server...

The ultra-thin client has the advantage of eliminating the need to implement any kind of replication scheme between the client and server, except that needed for the display which are only strings and numbers and objects of limited depth. Replicating arbitrary and/or deeply nested objects is where a lot of complication comes into play.

The complication for the ultra-thin client comes in providing a set of rich and responsive widgets for implementing the GUI -- presumably a much simpler.

Speaking of alternate approaches, a year ago I did a simple proof of concept[7] aimed at allowing the Pharo class library to be run natively in a GemStone repository using some of the base technology that we developed for Maglev[8]. In theory, Pharo programs (and even the Pharo GUI) would run "natively" in GemStone.

There is a group of bachelor students at HPI[9] currently working on running Squeak programs (and the Squeak GUI) in GemStone.

[6] https://www.slideshare.net/esug/tode-and-now-for-something-completely-different
[7] https://github.com/dalehenrich/PharoGs
[8] https://maglev.github.io/
[9] https://hpi.de/en/
  Give Pharo a permanent seamless object store - make it easy for me/others. The user base will grow.
Giving "Pharo a permanent seamless object store" is not explicitly on the roadmap as we are still in the early stages of exploration. We will need some sort of object replication scheme to implement the development environment. GemBuilder for Smalltalk (GBS)[3] is pretty much a "a permanent seamless object store"  that exists for VW[4] and VA[5], but it is also incredibly complex and we will be exploring simpler/cheaper to implement alternatives before investing in a full-blown GBS for Pharo.

Martin McClure (responsible for GBS) has a new replication architecture in mind for the upcoming Pharo client that eliminates much of the complication entailed in GBS, however, it _is_ still in development, so we won't know for a while how far it goes in providing a "Pharo a permanent seamless object store."

[3] https://downloads.gemtalksystems.com/docs/GBS/7.6.x/GBS-VW-UsersGuide-7.6/GBS-VW-UsersGuide-7.6.htm
[4] https://gemtalksystems.com/products/gbs-vw/
[5] https://gemtalksystems.com/products/gbs-va/

  Gemtalk will sell more licenses in the long run.

Ultimately where we head and invest resources will be driven by customer demand ... and at the moment we are moving towards providing support for git-based development in the kernel with a Pharo client for development.

* Gemstone should be re-branded as a fully functional pure OO application server with extensible secure persistence built in. (slight difference but HUGE!!! - MS, in truth, is a marketing company)
  Let's see if java can compete with that! (Not - they would be forced to make changes just to try.)
I think we would need some pioneering work by folks with domain knowledge so that we can provide concrete examples and (more importantly) make sure that the path is a smooth one ... and I see that you are proposing a proof of concept below ..
* There are more but I am saying too much as it is....

Lastly,

My proof of concept:

1.  Create a seaside app
2.  Create its client in amber.
3.  Upload the client js to the stone.
4.  Stream the client(s) to the browser based on some form of uri
5.  Manipulated the data store (gemstone) with the client.
6.  Cluster the system.
7.  Develop a DR process.
8.  Sell it

Should on the face of it be simple?

It should be relatively straightforward.

Let me know when you are planning on starting your proof of concept project as I would like to update the GsDevit_home Seaside installation instructions[10] to use Seaside 3.2 instead of Seaside 3.1 (you should be using the latest version of Seaside).

It is worth getting on the GLASS[11], Seaside[12], and Amber[13] mailing lists so that you can get quick answers to your questions.

For GsDevKit_home and GemStone specific advice I am definitely interested in helping.

Dale

[10] https://github.com/GsDevKit/GsDevKit_home/blob/master/docs/gettingStartedWithSeaside.md
[11] http://forum.world.st/mailing_list/MailingListOptions.jtp?forum=1460844
[12] http://forum.world.st/mailing_list/MailingListOptions.jtp?forum=86180
[13] http://forum.world.st/mailing_list/MailingListOptions.jtp?forum=3824463

--
You received this message because you are subscribed to a topic in the Google Groups "tODE" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tode_st/DhOb1p0H5LE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "tODE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.