Pharo - database without database
Locked
 
|
Hello,
I was wondering if it is possible for Pharo to load lots of data and query them without using external DB, just using object memory. Well, someting like GemStone/S, but not so sophisticated. I did some research so here is details what I found. I would appreciate if someone can tell me if any points of this agenda can be allready done or need to be programmed, or if this whole idea is waste of time and it is better to use external DB. 1) Object memory limit: Current object memory of Pharo is limited to 4GB. Free version of GemStone/S is also limited to 4GB, so I would rather stick with Pharo until I reach this level. And in the future this limit grow due to 64bit version of VM. 2) Saving to disk: Is it possible to save object or group of objects to disk and ensure consistency? With some transaction mechanism and transaction log? 3) RAM limit: Is it possible to delete objects from memory, move them to the disk and not break its connections to other objects? For example by redirecting object internal ID to some kind of hollow object, that will renew receiver of message from disk if necessary... This approach can also possibly split object memory into more "remote" sources. 4) Indexing: What optimalization for searching current collections in Pharo do? For example: how to store instances of City if I would like to search through it by name and also by its postal code? Is there some library for creating fulltext search (maybe Spotter uses that - I did not study it yet). Have a nice day, and thanks for any answer. Adam |
Re: Pharo - database without database
|
|
Adam, SandstoneDb may be of interest to you: "SandstoneDb is a lightweight Prevayler style embedded object database with an ActiveRecord API that doesn't require a command pattern and works for small apps that a single Squeak image can handle. The idea is to make a Squeak image durable and crash proof and suitable for use in small office applications. Data is kept in ram for speed and on disk for safety. All data is reloaded from disk on image startup. Since we're dealing with live objects in memory, concurrency is handled via optional record level critical sections rather than optimistic locking and commit failures. It's up to the developer to use critical sections at the appropriate points by using the critical method on the record. Saves are atomic for an ActiveRecord and all its non ActiveRecord children, for example, an order and its items. There is no atomic save across multiple ActiveRecords. A record is a cluster of objects that are stored in a single file together. After installing SandstoneDb make sure to restart your image before attempting to use it or running the tests." Fuel may also have something in stock: I also use Mapless http://sebastianconcept.github.io/Mapless/ HTH Phil On Sun, Sep 27, 2015 at 6:47 PM, Adam <[hidden email]> wrote: Hello, |
|
|
Hello,
thank You for great hints. I will look at this. Have a nice day, Adam. Dne Ne 27. září 2015 19:16:10, [hidden email] napsal(a): > Adam, > > SandstoneDb may be of interest to you: > > http://www.smalltalkhub.com/#!/~gnaritas/SandstoneDb > "SandstoneDb is a lightweight Prevayler style embedded object database with > an ActiveRecord API that doesn't require a command pattern and works for > small apps that a single Squeak image can handle. The idea is to make a > Squeak image durable and crash proof and suitable for use in small office > applications. > > Data is kept in ram for speed and on disk for safety. All data is reloaded > from disk on image startup. > > Since we're dealing with live objects in memory, concurrency is handled via > optional record level critical sections rather than optimistic locking and > commit failures. It's up to the developer to use critical sections at the > appropriate points by using the critical method on the record. > > Saves are atomic for an ActiveRecord and all its non ActiveRecord children, > for example, an order and its items. There is no atomic save across > multiple ActiveRecords. A record is a cluster of objects that are stored in > a single file together. > > After installing SandstoneDb make sure to restart your image before > attempting to use it or running the tests." > > > Fuel may also have something in stock: > > https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la > stSuccessfulBuild/artifact/book-result/Fuel/Fuel.pdf > > > I also use Mapless > > http://sebastianconcept.github.io/Mapless/ > > > HTH > > Phil > > On Sun, Sep 27, 2015 at 6:47 PM, Adam <[hidden email]> wrote: > > Hello, > > > > I was wondering if it is possible for Pharo to load lots of data and query > > them without using external DB, just using object memory. Well, someting > > like > > GemStone/S, but not so sophisticated. > > > > I did some research so here is details what I found. I would appreciate if > > someone can tell me if any points of this agenda can be allready done or > > need > > to be programmed, or if this whole idea is waste of time and it is better > > to > > use external DB. > > > > 1) Object memory limit: Current object memory of Pharo is limited to 4GB. > > Free > > version of GemStone/S is also limited to 4GB, so I would rather stick with > > Pharo until I reach this level. And in the future this limit grow due to > > 64bit > > version of VM. > > 2) Saving to disk: Is it possible to save object or group of objects to > > disk > > and ensure consistency? With some transaction mechanism and transaction > > log? > > 3) RAM limit: Is it possible to delete objects from memory, move them to > > the > > disk and not break its connections to other objects? For example by > > redirecting object internal ID to some kind of hollow object, that will > > renew > > receiver of message from disk if necessary... This approach can also > > possibly > > split object memory into more "remote" sources. > > 4) Indexing: What optimalization for searching current collections in > > Pharo > > do? For example: how to store instances of City if I would like to search > > through it by name and also by its postal code? Is there some library for > > creating fulltext search (maybe Spotter uses that - I did not study it > > yet). > > > > Have a nice day, and thanks for any answer. > > > > Adam |
|
|
If I'm correct we wrote about it on the seaside book
http://book.seaside.st Le 27/9/15 20:23, Adam a écrit : > Hello, > > thank You for great hints. I will look at this. > > Have a nice day, > > Adam. > > > Dne Ne 27. září 2015 19:16:10, [hidden email] napsal(a): >> Adam, >> >> SandstoneDb may be of interest to you: >> >> http://www.smalltalkhub.com/#!/~gnaritas/SandstoneDb >> "SandstoneDb is a lightweight Prevayler style embedded object database with >> an ActiveRecord API that doesn't require a command pattern and works for >> small apps that a single Squeak image can handle. The idea is to make a >> Squeak image durable and crash proof and suitable for use in small office >> applications. >> >> Data is kept in ram for speed and on disk for safety. All data is reloaded >> from disk on image startup. >> >> Since we're dealing with live objects in memory, concurrency is handled via >> optional record level critical sections rather than optimistic locking and >> commit failures. It's up to the developer to use critical sections at the >> appropriate points by using the critical method on the record. >> >> Saves are atomic for an ActiveRecord and all its non ActiveRecord children, >> for example, an order and its items. There is no atomic save across >> multiple ActiveRecords. A record is a cluster of objects that are stored in >> a single file together. >> >> After installing SandstoneDb make sure to restart your image before >> attempting to use it or running the tests." >> >> >> Fuel may also have something in stock: >> >> https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/la >> stSuccessfulBuild/artifact/book-result/Fuel/Fuel.pdf >> >> >> I also use Mapless >> >> http://sebastianconcept.github.io/Mapless/ >> >> >> HTH >> >> Phil >> >> On Sun, Sep 27, 2015 at 6:47 PM, Adam <[hidden email]> wrote: >>> Hello, >>> >>> I was wondering if it is possible for Pharo to load lots of data and query >>> them without using external DB, just using object memory. Well, someting >>> like >>> GemStone/S, but not so sophisticated. >>> >>> I did some research so here is details what I found. I would appreciate if >>> someone can tell me if any points of this agenda can be allready done or >>> need >>> to be programmed, or if this whole idea is waste of time and it is better >>> to >>> use external DB. >>> >>> 1) Object memory limit: Current object memory of Pharo is limited to 4GB. >>> Free >>> version of GemStone/S is also limited to 4GB, so I would rather stick with >>> Pharo until I reach this level. And in the future this limit grow due to >>> 64bit >>> version of VM. >>> 2) Saving to disk: Is it possible to save object or group of objects to >>> disk >>> and ensure consistency? With some transaction mechanism and transaction >>> log? >>> 3) RAM limit: Is it possible to delete objects from memory, move them to >>> the >>> disk and not break its connections to other objects? For example by >>> redirecting object internal ID to some kind of hollow object, that will >>> renew >>> receiver of message from disk if necessary... This approach can also >>> possibly >>> split object memory into more "remote" sources. >>> 4) Indexing: What optimalization for searching current collections in >>> Pharo >>> do? For example: how to store instances of City if I would like to search >>> through it by name and also by its postal code? Is there some library for >>> creating fulltext search (maybe Spotter uses that - I did not study it >>> yet). >>> >>> Have a nice day, and thanks for any answer. >>> >>> Adam > |
|
|
Hello,
it is true: http://book.seaside.st/book/advanced/persistency Thank You very much. Adam Dne Po 28. září 2015 08:50:28, stepharo napsal(a): > If I'm correct we wrote about it on the seaside book > http://book.seaside.st > > Le 27/9/15 20:23, Adam a écrit : > > Hello, > > > > thank You for great hints. I will look at this. > > > > Have a nice day, > > > > Adam. > > > > Dne Ne 27. září 2015 19:16:10, [hidden email] napsal(a): > >> Adam, > >> > >> SandstoneDb may be of interest to you: > >> > >> http://www.smalltalkhub.com/#!/~gnaritas/SandstoneDb > >> "SandstoneDb is a lightweight Prevayler style embedded object database > >> with > >> an ActiveRecord API that doesn't require a command pattern and works for > >> small apps that a single Squeak image can handle. The idea is to make a > >> Squeak image durable and crash proof and suitable for use in small office > >> applications. > >> > >> Data is kept in ram for speed and on disk for safety. All data is > >> reloaded > >> from disk on image startup. > >> > >> Since we're dealing with live objects in memory, concurrency is handled > >> via > >> optional record level critical sections rather than optimistic locking > >> and > >> commit failures. It's up to the developer to use critical sections at the > >> appropriate points by using the critical method on the record. > >> > >> Saves are atomic for an ActiveRecord and all its non ActiveRecord > >> children, > >> for example, an order and its items. There is no atomic save across > >> multiple ActiveRecords. A record is a cluster of objects that are stored > >> in > >> a single file together. > >> > >> After installing SandstoneDb make sure to restart your image before > >> attempting to use it or running the tests." > >> > >> > >> Fuel may also have something in stock: > >> > >> https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook > >> /la stSuccessfulBuild/artifact/book-result/Fuel/Fuel.pdf > >> > >> > >> I also use Mapless > >> > >> http://sebastianconcept.github.io/Mapless/ > >> > >> > >> HTH > >> > >> Phil > >> > >> On Sun, Sep 27, 2015 at 6:47 PM, Adam <[hidden email]> wrote: > >>> Hello, > >>> > >>> I was wondering if it is possible for Pharo to load lots of data and > >>> query > >>> them without using external DB, just using object memory. Well, someting > >>> like > >>> GemStone/S, but not so sophisticated. > >>> > >>> I did some research so here is details what I found. I would appreciate > >>> if > >>> someone can tell me if any points of this agenda can be allready done or > >>> need > >>> to be programmed, or if this whole idea is waste of time and it is > >>> better > >>> to > >>> use external DB. > >>> > >>> 1) Object memory limit: Current object memory of Pharo is limited to > >>> 4GB. > >>> Free > >>> version of GemStone/S is also limited to 4GB, so I would rather stick > >>> with > >>> Pharo until I reach this level. And in the future this limit grow due to > >>> 64bit > >>> version of VM. > >>> 2) Saving to disk: Is it possible to save object or group of objects to > >>> disk > >>> and ensure consistency? With some transaction mechanism and transaction > >>> log? > >>> 3) RAM limit: Is it possible to delete objects from memory, move them to > >>> the > >>> disk and not break its connections to other objects? For example by > >>> redirecting object internal ID to some kind of hollow object, that will > >>> renew > >>> receiver of message from disk if necessary... This approach can also > >>> possibly > >>> split object memory into more "remote" sources. > >>> 4) Indexing: What optimalization for searching current collections in > >>> Pharo > >>> do? For example: how to store instances of City if I would like to > >>> search > >>> through it by name and also by its postal code? Is there some library > >>> for > >>> creating fulltext search (maybe Spotter uses that - I did not study it > >>> yet). > >>> > >>> Have a nice day, and thanks for any answer. > >>> > >>> Adam |
Re: Pharo - database without database
|
|
In reply to this post by Adam-2
On 27-09-15 18:47, Adam wrote:
> 1) Object memory limit: Current object memory of Pharo is limited to 4GB. Free > version of GemStone/S is also limited to 4GB, so I would rather stick with > Pharo until I reach this level. And in the future this limit grow due to 64bit > version of VM. The information on GemStone seems incorrect. https://gemtalksystems.com/licensing/ What I read there is that the Limited (email address required) license supports 50 GB of (on-disk) objects, with a 2GB shared page cache. Stephan |
Re: Pharo - database without database
|
|
On Mon, Sep 28, 2015 at 7:17 AM, Stephan Eggermont <[hidden email]> wrote: On 27-09-15 18:47, Adam wrote: On the one hand, the license has changed starting in GemStone version 3.2. On the other hand, the "memory" limit you are talking about is called Share Page Cache and it's more a "soft" limit in the sense that is simply a cache of the persistent objects brought from disk. If you have a small "cache" the performance will be bad because of trashing (objects going and coming back from disk) but it doesn't mean the program won't be able to run because it doesn't fit. Free GemStone version involves a 2GB SPC. That is normally enough for small/medium apps. That is completely different as the "memory" limit for Pharo. |
|
|
In reply to this post by Stephan Eggermont-3
Hello,
Yes, I was confused by some old discussion. New licence is different. Thanks for correction. Adam. Dne Po 28. září 2015 12:17:19, Stephan Eggermont napsal(a): > On 27-09-15 18:47, Adam wrote: > > 1) Object memory limit: Current object memory of Pharo is limited to 4GB. > > Free version of GemStone/S is also limited to 4GB, so I would rather > > stick with Pharo until I reach this level. And in the future this limit > > grow due to 64bit version of VM. > > The information on GemStone seems incorrect. > https://gemtalksystems.com/licensing/ > What I read there is that the Limited (email address required) license > supports 50 GB of (on-disk) objects, with a 2GB shared page cache. > > Stephan |
Re: Pharo - database without database
|
|
Adam,
Keep an eye on the GLASS list[1] for an upcoming (within 2 months - I promise:) announcement about GsDevKit_home (not to be confused with gsDevKitHome:)... as it will provide much needed support for creating/starting/managing multiple GemStone stones.... If you are interested in working with GemStone:) Dale [1] http://forum.world.st/GLASS-f1460844.html On 09/28/2015 12:48 PM, Adam wrote: > Hello, > > Yes, I was confused by some old discussion. New licence is different. Thanks > for correction. > > Adam. > > Dne Po 28. září 2015 12:17:19, Stephan Eggermont napsal(a): >> On 27-09-15 18:47, Adam wrote: >>> 1) Object memory limit: Current object memory of Pharo is limited to 4GB. >>> Free version of GemStone/S is also limited to 4GB, so I would rather >>> stick with Pharo until I reach this level. And in the future this limit >>> grow due to 64bit version of VM. >> The information on GemStone seems incorrect. >> https://gemtalksystems.com/licensing/ >> What I read there is that the Limited (email address required) license >> supports 50 GB of (on-disk) objects, with a 2GB shared page cache. >> >> Stephan |
|
|
In reply to this post by Mariano Martinez Peck
Mariano soon with spur 64bits we will see :)
Le 28/9/15 14:57, Mariano Martinez Peck
a écrit :
|
|
|
In reply to this post by Dale Henrichs-3
Hello,
looking forward to use it :) Adam. Dne Po 28. září 2015 16:38:58, Dale Henrichs napsal(a): > Adam, > > Keep an eye on the GLASS list[1] for an upcoming (within 2 months - I > promise:) announcement about GsDevKit_home (not to be confused with > gsDevKitHome:)... as it will provide much needed support for > creating/starting/managing multiple GemStone stones.... If you are > interested in working with GemStone:) > > Dale > > [1] http://forum.world.st/GLASS-f1460844.html > > On 09/28/2015 12:48 PM, Adam wrote: > > Hello, > > > > Yes, I was confused by some old discussion. New licence is different. > > Thanks for correction. > > > > Adam. > > > > Dne Po 28. září 2015 12:17:19, Stephan Eggermont napsal(a): > >> On 27-09-15 18:47, Adam wrote: > >>> 1) Object memory limit: Current object memory of Pharo is limited to > >>> 4GB. > >>> Free version of GemStone/S is also limited to 4GB, so I would rather > >>> stick with Pharo until I reach this level. And in the future this limit > >>> grow due to 64bit version of VM. > >> > >> The information on GemStone seems incorrect. > >> https://gemtalksystems.com/licensing/ > >> What I read there is that the Limited (email address required) license > >> supports 50 GB of (on-disk) objects, with a 2GB shared page cache. > >> > >> Stephan |
«
Return to Pharo Smalltalk Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |