[squeak-dev] Smalltalk and flash memory

Hi All,

I am sorry for the length of this post and its slow start.  Please bear with me
as I think you will fine it interesting.

I have been thinking about how the recent and continuing advancements in
persistent memory (flash memory) in terms of speed, size and cost may impact
Smalltalk programs.  The available speed and sizes are increasing and the prices
are dropping.  There are already products that use flash memory to replace disk
drives.

Many programs maintain data in a database.  Often this is an SQL database,
sometimes it is an object database.  In either case it is not as easy to
maintain the data in these databases as it is to maintain data, as in memory
objects.

A Smalltalk program running on a server with a large amount of flash memory,
could run without the need for another form of database.  The flash memory would
be in addition to the normal memory, with both available as RAM.  The flash
memory would not look like a hard disk replacement.

To do this the Smalltalk VM would need a means of accessing the flash memory
differently than normal memory and would need a way of "reconnecting" to the
flash memory upon a system restart.  If the OS knows about the flash memory
there would need to be a means of connecting the flash memory to one program or
another.  The OS/program interaction can be addressed in many ways, I won't go
into them here.  Lets just say there is a need for an OS/program interaction,
and that need can reasonably be met.

I would like to talk about the Smalltalk VM and its role in this new
environment.  I don't know a lot about the workings of Smalltalk VMs, so I will
only talk about it at a very high level of functionality.  If a program could
tell the VM to instantiate an object in flash memory, based upon an indicator as
to whether an instance is to be persistent or not, the need for database
specific code almost disappears.  The indicator to be persistent could propagate
down to any instance pointed to by a persistent (in flash memory) object.  The
VM would need to be smart enough to move instances from regular memory to flash
memory when needed.

A program would only need to identify a few instances of collections
(LookupTable, Dictionary, SortedCollection etc.) as persistent object and it has
a database.  The code would change (or add to) these objects as it normally
would, without care for saving them in any other way.  There would be no need to
serialize an object to save them in any kind of an object database.  The
database is just the objects in the flash memory.

The flash memory would be RAM and NOT a disk replacement.  I don't know this for
sure but I don't think there are currently any motherboards that can have direct
CPU instruction access to large amounts of flash memory as if it is regular RAM
but I don't think it would be hard to do.  I would expect a 64 bit CPU is
needed, so we could have direct access to many gigabytes of flash memory beyond
the 4 gigabytes available with a 32 bit CPU.  

Different Smalltalks like VA Smalltalk and Squeak might deal with the flash
memory differently.  VA Smalltalk programs (not the development environment but
packaged images) don't normally save the image between runs.  Squeak is a more
traditional Smalltalk where programs aren't packaged the way VA Smalltalk
programs and the development environment and run time image are often the same
and the image is saved between runs.  In a Squeak program the objects in the
flash memory would probably not need to be save to a hard drive with the rest of
the image.

I think that what I am proposing could also work with any language that uses a
VM to support objects.  I would prefer Smalltalk to do it first.

Lou
-----------------------------------------------------------
Louis LaBrunda
Keystone Software Corp.
SkypeMe callto://PhotonDemon
mailto:[hidden email] http://www.Keystone-Software.com

On 6-Jun-08, at 3:24 PM, Louis LaBrunda wrote:

[snip]

What you're describing is very close to the way VisualWorks  uses  
PermSpace (which is sharable on system that have suitable memory  
sharing arrangements) and also very similar to the ROMSpace I  
implemented for the ActiveBook system almost 20 years ago. So, yes, it  
can be done, and no, I don't think it is currently available for Squeak

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Useful random insult:- Proof that evolution CAN go in reverse.

On Fri, Jun 6, 2008 at 3:24 PM, Louis LaBrunda
<[hidden email]> wrote:
> A Smalltalk program running on a server with a large amount of flash memory,
> could run without the need for another form of database.  The flash memory would
> be in addition to the normal memory, with both available as RAM.  The flash
> memory would not look like a hard disk replacement.

You might want to do some reading about GemStone
http://www.gemstone.com/products/smalltalk/

It does much of what you discuss. They have a free version so you can
explore more how it behaves.

Your comments about not having to intentionally manage writes of objects indicates that you believe that the read/write/erase characteristics of flash memory are similar to  conventional memory.  Cells in flash memory can't be re-written without first erasing.  Erasure occurs on block boundaries.  You can only erase a block a certain number of times (typically in excess of 100,000).  I find it convenient to think of flash as a very fast replacement for disk drives.  Compared to disks, it has very fast read access, especially for random reads, but the block structured erase/write protocol look more like a disk drive.  See http://en.wikipedia.org/wiki/Flash_memory for more.

Object persistence managers like GemStone, Glorp, etc, are still needed, but flash memory will definitely change the way we think about persistent storage.

-david

PS - I did a quick search to see if there have been any breakthroughs in flash other than price/density and didn't find anything.  If anybody has more info on this, please pass it along.

Hi David,

I am being a little sloppy calling the persistent memory I'm talking about
"flash memory".  I of course don't mean the exact kind of flash memory used for
memory sticks or disk drive replacements.  You are correct about their block
nature and that is clearly not what I want.  NOR flash memory is what I am
talking about.

Maybe I'm looking too far into the future but NOR flash memory is here today and
I think the prices are coming down.  I would like to generate a conversation
that might show a need for this memory and motherboards that support it, sooner
rather than later.

>Object persistence managers like GemStone, Glorp, etc, are still needed, but
>flash memory will definitely change the way we think about persistent
>storage.

They are still and will be needed for quite some time.  Again I'm trying to push
things a little here.  NOR flash memory is used in PDAs.  I think there is a use
for it in laptops and the use I described for servers.

>-david

>PS - I did a quick search to see if there have been any breakthroughs in
>flash other than price/density and didn't find anything.  If anybody has
>more info on this, please pass it along.

Try looking for NOR flash memory.  The price and sizes aren't where I want them
but we can still look toward the day that they will be and think about the VM
changes needed to use NOR flash memory.

Thanks, Lou
-----------------------------------------------------------
Louis LaBrunda
Keystone Software Corp.
SkypeMe callto://PhotonDemon
mailto:[hidden email] http://www.Keystone-Software.com

>PS - I did a quick search to see if there have been any breakthroughs in flash other
>than price/density and didn't find anything.  If anybody has more info on this, please pass it along.

Samsung also has a new persistent memory called PRAM.  It is too new to know how
well it will do but another player is interesting.
-----------------------------------------------------------
Louis LaBrunda
Keystone Software Corp.
SkypeMe callto://PhotonDemon
mailto:[hidden email] http://www.Keystone-Software.com