Easiest light weight cloud/web persistence for Pharo?

I have been using STON for a while now.  It does a really nice job of keeping everything just the way I need it - and the read/write - even for larger files process very quickly.  Limitations are full writes, no updates, but hey, it is incredibly easy to use and works in both v8 and v9.  I built a flashcard application in Pharo as well, and the STON files load in a fraction of a second.

Cheers!

Russ

On Wed, Oct 7, 2020 at 12:04 AM [hidden email] <[hidden email]> wrote:

Am 06.10.20 um 22:41 schrieb Tim Mackinnon:

> Gosh - this is proving much more interesting than I had imagined, and I’m getting lots of useful input, so I double appreciate the time and thoughts from everyone.

>

> I probably should have said that my "super awesome idea" is just a little flashcard spelling app for my daughter (and possibly a few friends in her class) - so probably SimplePersistance will do in this case,

yes, sounds like it. You mentioned it is not a super-complex 30-year

project you are planning. It sounds like you can load all the data into

memory in a few msec and just go form there, Saving the whole model as a

file also sounds like perfectly doable in such a project. So, heck, you

could probably simply just save the image and be good. A chunk of JSON,

Fuel, whatever object serialization is probably second best.

The great thing about this: nothing external (to the image) to install

or maintain. It is just you and the file system.

> however the comments about making a persistence decision at the right moment are super interesting - and as always, its about spotting the right moment to do that…

Oh no, don't fear this too much. That's not what I wanted to point out.

Most projects will do well on either NoSQL or using an ORM or an OODB.

Almost anything is possible on all of them. We do boring business stuff

(accounting) using Glorp as an ORM and DB2. Another well-known project

in the Pharo world does something extremely similar (travel cost) with

Voyage/Mongo. It is not so much a question of lost opportunities. The

problem here is that you have to deal with the very same problems

(concurrent access and isolation, fast lookups, caching, ghost objects)

on each of these technologies, but differently. It is the different

approaches to the same problems that make switching from one to the

other that makes the decision so hard.

Example: we had a hard time finding out why some objects never went away

although we had deleted them from the database. It took a while of

logging all SQL statements until we found out first that they were

really deleted in the Transaction when the user clicked OK. But they

were inserted back in the next, possibly completely unrelated,

transaction, because we had some dangling backpointers that tricked

glorp into thinking "oops, there is this bunch of new objects I have to

insert now". This is why I used object deletion as an example. Smalltalk

has no concept of deleting an object. Databases do. So whatever approach

you chose, deleting an object looks different. You either make the

object completely (!) unreachable from the rest of the object model and

thus just make it irrelevant, or you tell the databse to just throw it

away. Sounds like no big deal. But it can be.

> and ideally you have well factored code with ample tests that help you easily move from the idea, to something more scalable or robust…

The hard part here: the example with the reinserted objects is what is

called an unknown unknown in the boringtech site ;-) You would need a

test to check whether an object you deleted really isn't reoccuring

after the next Transaction. You'd have to imagine that there might be a

small chance that you delete an object successfully, but for some reason

you distorted the ORMs internal bookkeeping and trocked it into

inserting teh object back in some consecutive Transaction. A simple fact

once you understood why it happened, but I am 99% sure almost nobody

would come around the corner and say: well, we'll have to write a test

for this scenario where an object still lingers somewhere and gets

reinserted. The whole team would by this guy a beer and move to another

table.

> however, move too soon and you get bogged down with the details and you lose site of an MVP.

move and live with your decision. Or be prepared for a much more

complicated transition than you thought.

The problem is that nobody (at least that I am aware of) has come up

with an abstraction good enough to make the persistence implications

irrelevant enough and provide good performance and feature richness at

the same time. I know it's been tried.

>

> It has been interesting hearing peoples thoughts on all of this… the turning tide on ORM’s, the potential sweet allure of a NoSql (but can you query it easily) - and then the overarching element of just setting this shit up (where hopefully Docker steps in to make that bit at least easy).

Not sure about the need for Docker. You just throw more tech at the

problem. I mean, installing and setting up PostgreSQL or MySQL on a

Linux distro these days is matter of a few commands. apt install, enter

a db administrator password, answer a few questions and go. Same with

Mongo or CouchDB.

> I suppose this is where the Rails scaffolding was/is? such a jumpstart,

Don't get me started ;-)

Scaffolding is great if you need to sell a technology to management on a

few slides. Slip in some comment like "works on existing database

schemas too, at no extra cost" and these guys are ready to write

whatever it takes onto a cheque. I mean, come on, that guy showed us how

to make the whole mapping and transaction management for a flight

booking system in 15 minutes, how much harder can our project be? That's

how most multi-million desasters start.

> you can get a full thing going quite easily, and deploying seems relatively easy too… for us in Smalltalk land, its still a bit too much work for my liking, compared to the ease of getting an image and coding, and seeing it all work.

I totally agree. These days, everything is easy. Just install NodeJS and

some super sophisticated package manager and let it install everything

for you in just 5 minutes and you're ready to go. Just make sure you

have at least 12 GB of free memory and a good machine with fast

internet. Your data will be stored in the *bling, stars and glitter *

Cloud and you just forget about this detail. You want a login screen,

sure, just type 'init -n -tfgr "login"' and provide your github

credentials. We'll create a directory structure for you with a src, an

html, a helpers and a .gitignore file. Don't worry, some of that you'll

never touch. Oh, sure just make sure you have a Facebook account and

consent with the cookie policy of that super-georgous technology company

that provides this piece of code.

Not sure if that is what is really needed, but it's where we're heading

at the moment. Open any JS related book these days and read the first

chapter. I bet it's going to step you through the installation process

of at least three major super-cool open source, reliable and tested

frameworks or infrastructure monsters before you can start. From there

on, everything is a breeze. It's like a free lunch.

But boy, somebody will have to run this stuff for a while.

I think the Smalltalk vendors, both commercial and open source are

making great progress here, and I like the fact that we are not followng

the bloating trend. Maybe the "not invented here" meme makes sense to

some degree. What we as developers need is some understanding of what

we're doing and not so much the latest bells and whistles. Storing data

in a file may be unsophisticated, but it does the job even under hard

conditions.

..but I am getting slightly off-topic ;-)

Joachim

> It does seem to be getting marginally better at least, but I do wish there was super easy setup with all the pieces nicely in place so it was just your idea that you could focus on…

>

> Anyway, that login screen… oh crap I have to write one of those…

>

> Tim

>

>> On 6 Oct 2020, at 20:56, [hidden email] wrote:

>>

>> Sean,

>>

>> thanks for your short overview of what SimplePersistence does. Sounds useful for quite a few scenarios and might even carry you well through production stages for some projects.

>> What I was talking about is also not meant to frustrate people. I've only played with Mongo/Voyage for a few hours and I must say I was blown away by the speed and ease of that stack. We got something running in a few hours and it was impressive. So Mongo/Voyage is a cool thing to use.

>>

>> I always wanted to use Magna on Pharo. I even started to implement my own little clone of Magna in VA Smalltalk. I got into troubles when I tried to nest transactions and get this concurrency stuff streamlined somehow. We all know that besides naming and one-off problems, caching and concurreny are the hardest problems in computing. I think there is something about this. And so I gave up on that project... ;-)

>>

>> In the end I went with Glorp and DB2 (soon PostgreSQL). So far I am in a very solid state somewhere between complete despair and freaking out about how cool things are. I love and hate that stack from the bottom of my heart.

>>

>> The cool thing about an RDB is (and will sure be for quite a while) three letters. S, Q and L. There are lots of highly sophisticated GUI tools to query, manage, correct your data. And you can simply do everything form a command prompt, in an ssh session from your smartphone in a hotel toilet on the other side of the planet.

>>

>> Sure, using pure Smalltalk objects and not worry about n:m relationships, not having to write mappings and not having to end up with an object model that is driven mostly by what your O/R mapper can handle, sound great. And it is. Until you realize you also need to think about query optimizations, reorganizations, indexes and whatnot in an object database. There are also compromises to make.

>>

>> But, hey, I said all of that before.

>>

>> So maybe approaches like fuel, SimplePersistence (or BOSS or Object Swapper) are the best thing to start with when you need to find out about your architectural and business ideas first (am I building the right thing, will this feel good to a user, etc.), but once you are beyond that state, you better dive into your options and decide soon. Maybe using image saving or SimplePersistence is even good for production in your case. It was good enough for dabbleDB for quite a while, iirc, so why shouldn't it work for others? And maybe that is even the best you can do to postpone the decision for as long as possible at minimum opportunity cost.

>>

>> I didn't dig deep enough into Voyage/Mongo to judge how expensive or risky the changes to the design are. How hard is it to restructure the root trees - say you need something that is now beneath some root to be a root of its own? How would you do such changes?

>>

>> I know I can do a lot of things of that kind with SQL. It is a second looking glass and set of tools to view and manipulate the data. Sometimes things are easier to do in Smalltalk, sometimes it is way too slow on top of an ORM and a SQL query can do the same thing in a few milliseconds.

>>

>> But maybe I am asking the wrong questions fo Tim's purposes. I think I understand what you (Tim) are looking for is not a big, complex project but more like an experiment? I don't want to invalidate any of the given suggestions, I know or at least believe that they each do a good job. All I really wanted to warn you is that you will not easily be able to go from one option to another, because each will have a deep impact on your object model and application architecture.

>>

>> Joachim

>>

>>

>>

>>

>> Am 06.10.20 um 16:34 schrieb Sean P. DeNigris via Pharo-users:

>>> jtuchel wrote

>>>> Sigh. Forget about the idea that it will be easy to switch your

>>>> persistence later....I am not commenting on SimplePersistence here, I

>>>> don't even know what it

>>>> does or doesn't.

>>> Joachim,

>>> Thanks for this interesting perspective. I've never had the (mis?!)fortune

>>> of a project growing enough to force me to make those tough choices! For

>>> SimplePersistence I will say that I view it as a way to *delay* making *any*

>>> choices until you are forced to. It's really just a layer of sugar on top of

>>> Fuel (it used to use the old school Squeak equivalent serialization

>>> mechanism - I forget the name and that might still work). You tell it what

>>> classes to serialize. Implement two methods for each class that get and set

>>> the data, and then it saves the whole thing as one object graph.

>>>

>>> Tim,

>>> If you use SimplePersistence, please keep me posted about your experience.

>>> I'm happy to help.

>>>

>>> NB I have maintained and extended the library, but it is the work of Ramon

>>> Leon

>>>

>>>

>>>

>>> -----

>>> Cheers,

>>> Sean

>>> --

>>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

>>>

>> --

>> -----------------------------------------------------------------------

>> Objektfabrik Joachim Tuchel          mailto:[hidden email]

>> Fliederweg 1                         http://www.objektfabrik.de

>> D-71640 Ludwigsburg                  http://joachimtuchel.wordpress.com

>> Telefon: +49 7141 56 10 86 0         Fax: +49 7141 56 10 86 1

>>

--

-----------------------------------------------------------------------

Objektfabrik Joachim Tuchel          mailto:[hidden email]

Fliederweg 1                         http://www.objektfabrik.de

D-71640 Ludwigsburg                  http://joachimtuchel.wordpress.com

Telefon: +49 7141 56 10 86 0         Fax: +49 7141 56 10 86 1

--

Russ Whaley

[hidden email]