Re: Pharo-users Digest, Vol 57, Issue 39

Ian,

You are not being completely transparent ....

My initial response[2] (on the8th of January) to your initial post on the tode mailing list[1] (from the 4th of January):

I am wanting to use Gemstone as a persistent object store.
I do not want it to run any client facing applications.
I want to run all client facing applications in pharo images of there own connected to a central backing stone. Is this the wrong way to think about this?

"Your best bet is to use REST"

You continued to ask several questions about how GemStone actually works (calling it a backing store is not a fair characterization of GemStone) and I tried to fill in various additional details about other possible approaches since you didn't seem to be very clear about what you were interested in.

It wasn't until the 11th that you wrote the nicely detailed post[4] about intentions to use Amber and GemStone and a RESTful API for your application that involves petabytes of data.

And then this morning you decided to post some non-flattering comments on the Pharo mailing list of all places quoting the post from the 11th before I even responded.

I did not have a chance to respond to your nicely detailed post until late this morning[5] and never did see your post on this list until this afternoon, because it was titled "Pharo-users Digest, Vol 57, Issue 39".

At the end of the day you seem to be upset that an out-of-the-box client/server solution does not exist for GemStone and my answer to that is that we have had out-of-the-box client/server solutions for over 2 decades; for VW and VA  and we have active customers who have been running applications using this technology for just as long ...

We just do not have the same kind of solution for Pharo, but as I mentioned in my email from this morning[5] we are headed in that direction. However, with 2 decades of experience under our belts we will take a slightly different approach to the object replication architecture this time around - and the reality is that there are several possible directions to take..

It is also worth mentioning that we have Seaside customers using a develop in Pharo deploy in GemStone methodology who have been running production applications continuously for over a decade.

I mentioned in my mail from this morning that I would be willing to work with on your proof of concept and that statement still stands.

Using GemStone as an object store for visualizing and inspecting objects derived from big data streams does seem to be an application that GemStone is well suited for but there is work that needs to be done to make it a reality as there is no out-of-box solution for this problem at the moment.

Dale

W.R.T Database Connectivity:

Hi All, 

The database connectivity issue as discussed is near and dear to my evaluation of Smalltalk(s).  It is a huge problem/stumbling block as I see it.

Here Is a cut and paste of my replay to the Gemstone users group:

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Hi Dale,

I am evaluating Gemstone as an application server/big data repository.  Hence my confusion up till now.

I work for an organization that has rather large data storage requirements (understatement:  into the 100's of petabyte range).

Currently, several independent Apache Hadoop stacks are used to manage this data.  There are several serious issues that are evident with this setup, prime of which is object to relational mapping errors (as datasets in a given space change over time - data is mandated to be kept into perpetuity) regardless of whether the database is NoSQL, SQL of any other schema.  The issue bubbles up into the organization application layers of many diverse and distributed verticals.  Maintenance of the system is not trivial and is very expensive and time consuming.

We use virtualization (in the VMWare sense) to create views into the data.  Generally, access is a split between virtualized kiosk systems, another level of indirection, where by mandate, we must now archive the state of the vm's as well as business data, and direct workstation access,   Not fun, error prone and hardware intensive (from a data storage perspective). 

As I have been reading about object databases and what they are said to buy in terms of efficiency (objects being stored natively and directly) coupled with the fact the we are virtualizing everything we can, I, along with a couple of colleagues, thought that we should take a look at what new (laugh) technologies were available.

After a little time I came across Gemstone.  Did my reading and research and after a period of semantic confusion (still going) became enamored with the possibilities.

All this said, from a network, sys admin point of view, an image that contains the application and it's objects is a godsend if object persistence was guaranteed.  It would allow me to archive a self contained living blob that manages its own living state (in the VMWare snapshot sense) and at the same time the data is available to other blobs/views (seaside).

On views in to the data store:

Currently data mining occurs on the application layer (rather than database level views).  Various development groups create views (applications) to access (possibly update, if they are the owning group) the data.  As mandate trickle (waterfall) from above new data points can be added or removed (some of the mandate for this is governed by changing regulations - in other words it must be done regardless of the technical difficulties involved).  You can image the difficulties and manpower involved.

This brings me back to the application server.  Currently, whatever tool a group uses to access the data, be it browser, remote client application, whatever have you (generally based on the skill sets in the group), requires a set of policies and procedures wrapped around the effort (mandated disaster recovery and etc...).  Tones of work that can never be standardized in any way but nevertheless must be documented and managed.

Okay (to long winded self):

Having spent some time looking at this problem from a smalltalk, image virtualization, integrated application management perspective, all my issues could be alleviated within a single technology.  Yippee!

The spawning of an idea:

Amber -> Seaside -> Gemstone.  One language (integrated properly -> one tool). 

Amber: 

*  Each group, with a small learning curve could create and manage their own views into the data.

*  The serialized javascript, once the client was developed, could be streamed from the application server (Seaside/Gemstone) on demand.  It just becomes another object.

Seaside:

* Client interface manager transport system (HTTP) for any and all views.

Gemstone:

* Object level persistence with no forced mappings of any kind.

* Extensibility. (can gemstone handle data levels in the petadata range?)

Cool!:

* Users/Developers of the system have a completely controlled view into the image/system (perhaps a clustered NginX front end?).
* Image access at the machine level is completely controlled/secure.

* Everything in one place with system knowledge transfer and expertise leveraged vertically and horizontally.

* The images can easily be archived in whatever manner (at the bit level using any version control system if desired) at whatever schedule.
* Another level of 'complete system' security achieved. (Way cool!!)
* Whole systems is a single entity!  Fantastical! 

* A significantly smaller foot print.

* Cost reductions, increased efficiency all the way through...

Pipe dream?  Maybe but I think it is doable over time with smalltalk.  The same system is not doable in any other way with any other technology without extensive dollars and effort and once built making dynamic unplanned for changes virtually impossible.  You are locked in (Silly but happens all the time).  With the system I envision, changes, even schema changes, are relatively easy.

For my money:

* Gemtalk should integrate development tools directly into Gemstone (Gemstone with seaside is an order of magnitude more useful than without it)

* Gemtools should be in the reference client - Pharo - permanently (CGI/RPC, everything).

  What does the tODE buy me, other than access, but a shell?

  Give Pharo a permanent seamless object store - make it easy for me/others. The user base will grow. 

  Gemtalk will sell more licenses in the long run.

* Gemstone should be re-branded as a fully functional pure OO application server with extensible secure persistence built in. (slight difference but HUGE!!! - MS, in truth, is a marketing company)

  Let's see if java can compete with that! (Not - they would be forced to make changes just to try.)

* There are more but I am saying too much as it is....

Lastly,

My proof of concept:

1.  Create a seaside app

2.  Create its client in amber.

3.  Upload the client js to the stone.

4.  Stream the client(s) to the browser based on some form of uri

5.  Manipulated the data store (gemstone) with the client.

6.  Cluster the system.

7.  Develop a DR process.

8.  Sell it

Should on the face of it be simple?

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

I know what this means for me as an evaluator of Smalltalk as a viable system.  I only include my response to that group to simply to show my pain.

The responses I got were on the order of 'you could do this,' or write that...  Neither answer, from my perspective is valid.

I reiterate that Pharo with built in persistence as in a fully integrated Gemstone is, IMHO, the best approach to take.  The fact that gemstone as the only non-flattened oo extensible database is sad.   Worse is that Gemtalk sees Gemstone as a separate entity.  Object persistence requires application level integration ie. gemstone requires you to develop your app and have it run in the stone in order to persist.  The Gemtalk philosophy is thus flawed on the face of it. 

I don't means to disparage anyone just stating facts as I see them.

On 15 January 2018 at 08:22, <[hidden email]> wrote:

Send Pharo-users mailing list submissions to
        [hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
        http://lists.pharo.org/mailman/listinfo/pharo-users_lists.pharo.org
or, via email, send a message with subject or body 'help' to
        [hidden email]

You can reach the person managing the list at
        [hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Pharo-users digest..."

Today's Topics:

   1. Re: Databases (Sven Van Caekenberghe)
   2. Re: [Moose-dev] [ann] gt connector (Sven Van Caekenberghe)
   3. Re: [Moose-dev] [ann] gt connector (Kjell Godo)
   4. Re: [ANN] OpenSSL-Pharo works on Windows (Tudor Girba)
   5. Set Rounding mode for IEEE floating point operations
      (Steffen M?rcker)
   6. [Seaside] WAComponent instances tree + proper page refreshing
      for user (back button) - confusion (Petr Fischer)


---------- Forwarded message ----------
From: Sven Van Caekenberghe <[hidden email]>
To: Benoit St-Jean <[hidden email]>, Any question about pharo is welcome <[hidden email]>
Cc: 
Bcc: 
Date: Mon, 15 Jan 2018 10:55:04 +0100
Subject: Re: [Pharo-users] Databases
Benoît,

Yes, of course databases are important and I know many users ask for them, but on the other hand, the low development activity can only mean that there is not enough interest (or some existing solutions work well enough).

There are many SQL and non-SQL databases out there, as well as many, incompatible Smalltalk implementations. I do not believe that there can be one magic solution to fix all problems everywhere.

Remember that setting up a database is often a challenge by itself.

I do not think the current situation is bad, it is just confusing. Making a good meta overview (i.e. documentation writing) and testing solutions would already help many people.

To add to your list, I recently wrote a new PostgreSQL client, P3, that is using only TCP networking (no native library mess). It supports most recent Pharo distributions.

  https://github.com/svenvc/P3

P3 can be used under Glorp. Glorp in still being maintained and in active use, but could use some love in the promotion department.

Here is some documentation

 http://files.pharo.org/books-pdfs/booklet-Glorp/2017-05-02-Glorp-Spiral.pdf
 https://medium.com/concerning-pharo/reddit-st-in-10-cool-pharo-classes-1b5327ca0740
 https://github.com/SquareBracketAssociates/Booklet-Reddit (updated version of the former)
 https://bintray.com/squarebracketassociates/wip/download_file?file_path=reddit-wip.pdf

Actually, few languages have as nice an Object Relational Mapper as Glorp.

There is also Voyage that seems quite popular.

Sven

> On 15 Jan 2018, at 10:13, Benoit St-Jean via Pharo-users <[hidden email]> wrote:
>
>
> From: Benoit St-Jean <[hidden email]>
> Subject: Databases
> Date: 15 January 2018 at 10:13:10 GMT+1
> To: "[hidden email]" <[hidden email]>, "[hidden email]" <[hidden email]>
> Reply-To: Benoit St-Jean <[hidden email]>
>
>
> Bear with me for a moment...
>
> First and foremost, this post is *not intended in any way* to criticize the work of anyone who could have worked on those packages/projects/frameworks.
>
> Now, I'll tell you a story to make my point.
>
> Often times, in public or in private, I help newcomers on Quora, IRC, the mailing list(s) or elsewhere with various Smalltalk environments.  Often times, I try to be helpful getting them started on Squeak or Pharo.  Often times, they are just amazed by how powerful Smalltalk is...
>
> Now, the sad part.  Everything goes well until they just want to "code something" to experiment and they ask me about databases...
>
> -How can I connect to a MySQL (or insert your favorite database here) with Squeak/Pharo ?
>
>
> As far as I can remember, the database connectivity with Squeak/Pharo has always been an area where, as a Squeak/Pharo community, we suck big time.  I don't recall the last time I loaded a package, ran the tests and everything was fine.  There is almost ALWAYS something broken with the databases packages.  Understandibly, Squeak and Pharo were for a long time moving targets.  Whether it was VM changes (FFI, NativeBoost, uFFI, 32 vs 64 bit, OS differences/problems, differences between Squeak and Pharo, etc), code refactoring (cryptography class name changes), the DateAndTime/Timestamp/TimeStamp/Date(Magnitude)/Date(Timespan) were used from one version to the other in the driver, code that was removed by some cleanup, compiler changes/problems/incompatibilities, etc.  In other words, database connectivity needs some love.
>
> Unfortunately, it looks like there doesn't seem to be a consensus or a shared desire to fix things *together*.  There is duplicate effort all over the place.  Just as an example, I listed the various implementations of MySQL packages that exists (I did not list the 8-9 of them that had no code but a project listed on one of those sites).  So, let's say you are a newcomer and you want to connect to a MySQL database.  Here are the choices you are offered (and remember, as a newcomer, you have NO clue which one to pick) :
>
>
> Databases supported : Project name
>
> 1) SqueakSource
>
> MySQL : Automatic Object Storage To MySQL
> OpenDBX
> SqueakDBX
>
>
> 2) SmalltalkHub
>
> DBX/Talk / DBXDatabaseModel
> GlorpDriverMySQL
> PharoExtras / ODBC
> DBXTalk / Garage
> UDBC
> UDBC2
>
> 3) GitHub.io
>
> dbxtalk
>
> 4) GitHub
>
> pharo-rdbms/garage
>
> Now, let's say you picked one package.  In some cases, if you are lucky, the code will load without any problem or warning.  And then, you run the tests.  That is where it almost *always* breaks!
>
> Today, I tried to load *each and everyone of them* in a Pharo 6.1 (Windows) image : none of them worked right out of the box!
>
> In other words, that newcomer I told you about just goes from "WOW!!!" to "WHAT?!?!" in 10 seconds.  People are usually not so impressed with a TestRunner that fails a gazillion unit tests...
>
> So, now the nice part...
>
> Database connectivity is, in my humble opinion, a MUST for Pharo and Squeak. I think it's about time we do *something* about the current state of this mess.  There are enough brilliant and talented people in the Squeak and Pharo communities to fix this situation.  At least, we could start *talking* about it if we want to address the problems.  Ideally, we would come up with a will to change things and establish a plan.
>
> In a perfect world, we'd have something like JDBC (or some kind of Smalltalk universal driver) in Smalltalk for native drivers.  Plus ODBC connectivity. Plus it would have to work right-out-of-the-box.  Plus documentation that has at least an example with Connect/SELECT/UPDATE/DELETE/CREATE/DROP/disconnect simple examples to get newbies started.  Plus a solid suite of tests to beat the sh*t out of that code. Plus a compatibility layer for Squeak/Pharo differences.  To that regard, Glorp is an example to follow!  IMHO, it's about time we share stuff to maximize our efforts!
>
> As I told someone recently, it's about time we do something about databases (RDBMS).  And as I told that person, I don't mind writing Smalltalk code and SQL scripts and running tests.  Even do performance testing.  My MySQL Server currently has close to 1 billion records to torture the MySQL driver if needed.  And I can also work on SQLite.  And Oracle.  And DB/2.  And MySQL Server.  And I don't mind installing other RDBMS on my machine and test other driver ports.  Now, it's just a matter of knowing if we all want to work together.
>
> As for OpenDBX/DBX, I'm not sure.  Having another dependency (and C code) that we more or less control doesn't inspire me.  I have yet to see how DBX/OpenDBX gives me any advantage over, say, ODBC...
>
> As I said, it's time to discuss!
>
> Anyone interested?
>
> -----------------
> Benoît St-Jean
> Yahoo! Messenger: bstjean
> Twitter: @BenLeChialeux
> Pinterest: benoitstjean
> Instagram: Chef_Benito
> IRC: lamneth
> Blogue: endormitoire.wordpress.com
> "A standpoint is an intellectual horizon of radius zero".  (A. Einstein)
>
>





---------- Forwarded message ----------
From: Sven Van Caekenberghe <[hidden email]>
To: Moose-related development <[hidden email]>
Cc: Pharo Development List <[hidden email]>, Any question about pharo is welcome <[hidden email]>
Bcc: 
Date: Mon, 15 Jan 2018 11:19:59 +0100
Subject: Re: [Pharo-users] [Moose-dev] [ann] gt connector


> On 14 Jan 2018, at 22:54, Tudor Girba <[hidden email]> wrote:
>
> Hi,
>
> Towards the end of last year we worked on GT Connector, a new kind of interface that allows us to exercise and test the limits (or the lack thereof) of Bloc.
>
> It looks like this:
> <connector2.jpeg>
>
> You can see it in action here:
> https://twitter.com/feenkcom/status/936109463462965248

Incredible stuff !

Keep on pushing the boundaries.

> In the current implementation, the Connector allows us to navigate and connect example methods. The focus is not on examples, but on the connections. We used examples because the engine was already around and offered us a nice use case. We want to extend it in the near future to other kinds of objects.
>
> There are a couple of things that are worth noting:
> • The editor works live, and the connection points appear and disappear as you type.
> • The layout of the editor elements is based on a tree-based graph layout that only works with constrains (no actual visible edges between the editor elements).
> • The editor works live, so adding new elements to the scene properly rearranges the scene.
> • But, perhaps, the most exciting part is the fact that the lines connect an element from inside the text editor element with another that lives outside of the editor element.
>
> All these validate the architecture of Bloc of having exactly one rendering tree. It was not an obvious goal a couple of years ago, but we are really happy that it works.
>
> To put it in perspective, let's compare this with the html world. Text is text is rendered through the DOM tree. If you want graphics you might use something like SVG which comes with its own tree. However, these are two distinct worlds, and you cannot go from one to another, or at least not easily. This is the case in most engines we looked at.
>
> Why is this important? One thing we learn in the Smalltalk world is that covering the same space with less concepts opens up a whole dimension of creativity that is simply not possible outside of it.
>
> The goal with Bloc is to enable new kinds of user interfaces. As we are late to the game of modern interfaces, even though the field was invented in Smalltalk, our only chance to take the lead again is to rethink the model.
>
> Let's look at the Connector again. In most user interfaces we have panes on the outside, and visuals confined within the boundaries of those panes. Interestingly, we can trace this pattern to the very first Smalltalk interfaces. In the Connector interface we have no boundaries with text and visualization being intertwined to form a new kind of workflow.
>
> Talking about workflows, we now have two distinct and novel ways to explore examples: one is Connector, and the other one is the expandable code editor. For example, the scene from above looks like this in the example expanding editor:
>
>  <editor.jpg>
>
>
> Both of these interfaces are not found in other infrastructures, and yet they were both inexpensive to implement in Bloc.
>
> We believe this will have a deep impact for all sorts of interfaces, and especially for the IDE. If you are interested in more details related to the IDE, take a look at the following paper from 2015:
> http://scg.unibe.ch/archive/papers/Girb15b-PervasiveSoftwareVisualizations.pdf
>
> Please let us know what you think.
>
> Cheers,
> The feenk team
>
>
> --
> www.tudorgirba.com
> www.feenk.com
>
> "What is more important: To be happy, or to make happy?"
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.list.inf.unibe.ch/listinfo/moose-dev





---------- Forwarded message ----------
From: Kjell Godo <[hidden email]>
To: Moose-related development <[hidden email]>
Cc: Any question about pharo is welcome <[hidden email]>, Pharo Development List <[hidden email]>
Bcc: 
Date: Mon, 15 Jan 2018 10:44:26 +0000
Subject: Re: [Pharo-users] [Moose-dev] [ann] gt connector

i want to use it

---------- Forwarded message ----------
From: Tudor Girba <[hidden email]>
To: Any question about pharo is welcome <[hidden email]>
Cc: 
Bcc: 
Date: Mon, 15 Jan 2018 12:20:00 +0100
Subject: Re: [Pharo-users] [ANN] OpenSSL-Pharo works on Windows
Thanks a lot!

Doru


> On Dec 20, 2017, at 8:55 AM, Pierce Ng <[hidden email]> wrote:
>
> Hi all,
>
> OpenSSL-Pharo now works on Windows. Tested on Windows 10 with a fresh 32-bit
> Pharo 6.1 zip package downloaded from pharo.org. On Windows this library uses
> libeay.dll which is bundled with the Pharo VM.
>
>  Metacello new
>    baseline: 'OpenSSL';
>    smalltalkhubUser: 'PierceNg' project: 'OpenSSL-Pharo';
>    load.
>
> Pierce
>
>

--
www.tudorgirba.com
www.feenk.com

"No matter how many recipes we know, we still value a chef."











---------- Forwarded message ----------
From: "Steffen Märcker" <[hidden email]>
To: Any question about pharo is welcome <[hidden email]>
Cc: 
Bcc: 
Date: Mon, 15 Jan 2018 12:27:06 +0100
Subject: [Pharo-users] Set Rounding mode for IEEE floating point operations
Hi,

is there any way to set the rounding mode for IEEE floating point
operations? Maybe something like

Double roundToMinusInfWhile: [... code goes here ...]Double roundToZeroWhile: [... more code here ...]

If not, is it possible to add this behavior, e.g., via a custom primitive?

Best, Steffen




---------- Forwarded message ----------
From: Petr Fischer <[hidden email]>
To: [hidden email]
Cc: 
Bcc: 
Date: Mon, 15 Jan 2018 14:21:28 +0100
Subject: [Pharo-users] [Seaside] WAComponent instances tree + proper page refreshing for user (back button) - confusion
Hello, sorry for crossposting (from Seaside list), but I probably do not understand something very basic in Seaside - how to properly refresh instantiated WAComponent tree (data views) and when (+ back button problems).

Seaside manual says ("About Callbacks" section):

"Do not change state while rendering... Just produce output and define callbacks that do the fancy stuff you can’t do while rendering."

OK, then my small example:

1) Suppose I have a page - WAMyListPage (WAComponent sub class), which presents a list of items from DB table, easy.

2) When WAMyListPage is instantiated at the beginnig, it loads 10 rows from the DB, creates WAMyItem (WAComponent subclass) for every DB row and puts these WAMyItem instances to the WAMyListPage instance variable "items" (OrderedCollection)

3) WAMyListPage is rendered to the browser (simple render call on all WAMyItem "items" components in a loop) - 10 items (db rows) are displayed - OK

4) User 1 clicks on the WAMyItem component from the rendered list (on the WAMyListPage) and goes into item detail page (detail page is not interesting here)

5) User 2 deletes 5 items/rows from the database (some way)

6) User 1 clicks browser back button and returns to the WAMyListPage, two cases can occur:

                A) typically, the "list page" is cached in the browser and browser immediatelly renders the list page from the cache (without request) - with all 10 old items (no refresh, broken old view with some non existing items)

                B) request from the browser to the Seaside app is made again (with old Seaside URL from browser history), then Seaside app just renders old state of WAMyListPage instance from the history and again - there is all 10 old WAMyItem items instantiated, so, broken again, old items in the view

My question - where can I do a proper refresh of WAMyListPage list items? (manual refresh button in the Seaside app is unacceptable)

Note: I can solve this by reloading WAMyListPage "items" instvar collection in every renderContentOn: (reload from DB, reload all WAMyItem instances) + instruct the browser to not cache any web page (in HTTP headers), so in point (6A) above, browser will ignore caching and make request to Seaside app again.
In this way, everything works, WAMyListPage is actual every time, but main Seaside rule "Do not change state while rendering" is violated (WAMyItems are reinstantiated in every renderContentOn:).

So, how do I face this? Thanks very much! pf



_______________________________________________
Pharo-users mailing list
[hidden email]
http://lists.pharo.org/mailman/listinfo/pharo-users_lists.pharo.org