Smalltalk › Squeak › Squeak VM

Where is the heap?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

7 messages Options

melkyades

Where is the heap?

Hi,
I'm working in memory management of SqueakNOS platform. Currently we use a static size heap, and we'll probably have to change that. I read in the blue book that the heap is implemented as a contiguous chunk of memory, is the heap done that way in squeak?
        I looked how it's implemented in unix and saw that sqAllocateMemory uses mmap. I don't have experience with this function, but I wonder if it means that the heap is allocated in a virtual contiguous chunk that gets mapped to some place in disk. Or maybe it just reserves contiguous virtual addresses that are in different blocks of real memory?

        The alloc line is this, (sqUnixMemory.c)

if (MAP_FAILED == (heap= mmap(0, heapLimit, MAP_PROT, MAP_FLAGS, devZero, 0)))

        Also, other question is what is this mapping to /dev/zero? Then reading from the heap wouldn't return allways zero? Is it because this mmap isn't actually backed by any file? Does mmap reserve heapLimit bytes in real memory? I mean, if you write to the mapped address, you write to real memory, device memory, or both?

        Lastly, growing the heap gets to this:

if (MAP_FAILED == mmap(base, newDelta, MAP_PROT, MAP_FLAGS | MAP_FIXED, devZero, heapSize))

which doesn't alter heap start, so it has to be that when resizing the heap the only thing that happens is that some memory is reserved and mapped to the ending address of it.

Well as you can see I'm rather confused so any help is wellcome, thanks for reading.

Javier.

--
Javier Pimás
Ciudad de Buenos Aires

Bert Freudenberg

Re: Where is the heap?

Am 11.05.2010 um 15:15 schrieb Javier Pimás:

Hi,
I'm working in memory management of SqueakNOS platform. Currently we use a static size heap, and we'll probably have to change that. I read in the blue book that the heap is implemented as a contiguous chunk of memory, is the heap done that way in squeak?

Yes.

For SqueakNOS I'd think you should simply use all the available memory.

- Bert -

johnmci

Re: Where is the heap?

In reply to this post by melkyades

How mmap works is somewhat dependent on the operating system.

see

http://isqueak.org/sqAllocateMemory

The iphone and os-x version use for example:

fstat(fileno((FILE *)f), &sb);

fileSize = sb.st_size;

fileRoundedUpToPageSize = valign(fileSize+pageSize-1);

startOfmmapForImageFile = mmap(possibleLocation, fileRoundedUpToPageSize, PROT_READ|PROT_WRITE, MAP_FILE|MAP_PRIVATE,fileno((FILE *)f), (off_t)0);

So let's talk intent.

Yes the oops space currently is a continuous chuck of memory

It has the basic form of:

start < - > oldspace <-> youngStart <-> freeblock <-> ??? <-> GC remap pointers <->end of memory

The most simple case is to use malloc and allocate N bytes of memory.

However in general most platform VM's use much more complex solutions, where they ask the operating system virtual memory for N pages of memory, at even a particular start location to avoid re-calculating the oops offsets at startup tie. There are two targets here, one is the total amount of memory that the Image *could* use, and then controlling the actual usage.

The sqGrowMemoryBy & sqShrinkMemoryBy are then used by the VM to indicate how much memory Squeak is using so that perhaps the platform code will give memory back to the operating system to reduce page counts for the app. So although you might indicate a VM *can* get at 500 MB of memory you'll find maybe only 20MB is actually used, so things get adjusted to tell the operating system you only want 20+ MB of ram.

I note on os-x and the iPhone I just go ahead and suggest I'm going to use the 500MB because BSD is optimistic about page usage and will overallocate, so asking for 500MB doesn't mean I might get 500MB when I need it, but this does result in simpler mmap logic. Note that mmap is fraught with bugs over the years and using munmap to readjust your original allocation *might* result in surprises depending on your choice of operating system (and version of operating system).

So as Bert said malloc() is the simplest solution...

On 2010-05-11, at 3:15 PM, Javier Pimás wrote:

Hi,
I'm working in memory management of SqueakNOS platform. Currently we use a static size heap, and we'll probably have to change that. I read in the blue book that the heap is implemented as a contiguous chunk of memory, is the heap done that way in squeak?
        I looked how it's implemented in unix and saw that sqAllocateMemory uses mmap. I don't have experience with this function, but I wonder if it means that the heap is allocated in a virtual contiguous chunk that gets mapped to some place in disk. Or maybe it just reserves contiguous virtual addresses that are in different blocks of real memory?

        The alloc line is this, (sqUnixMemory.c)

if (MAP_FAILED == (heap= mmap(0, heapLimit, MAP_PROT, MAP_FLAGS, devZero, 0)))

        Also, other question is what is this mapping to /dev/zero? Then reading from the heap wouldn't return allways zero? Is it because this mmap isn't actually backed by any file? Does mmap reserve heapLimit bytes in real memory? I mean, if you write to the mapped address, you write to real memory, device memory, or both?

        Lastly, growing the heap gets to this:

if (MAP_FAILED == mmap(base, newDelta, MAP_PROT, MAP_FLAGS | MAP_FIXED, devZero, heapSize))

which doesn't alter heap start, so it has to be that when resizing the heap the only thing that happens is that some memory is reserved and mapped to the ending address of it.

Well as you can see I'm rather confused so any help is wellcome, thanks for reading.

Javier.

--
Javier Pimás
Ciudad de Buenos Aires

===========================================================================

John M. McIntosh <[hidden email]> Twitter: squeaker68882

Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com

===========================================================================

smime.p7s (3K) Download Attachment

Bert Freudenberg

Re: Where is the heap?

Am 11.05.2010 um 15:50 schrieb John M McIntosh:
> So as Bert said malloc() is the simplest solution...

I actually didn't mention malloc(). Since in SqueakNOS there *is* no operating system, I'd have thought you don't even need to malloc().

- Bert -

Igor Stasenko

Re: Where is the heap?

On 12 May 2010 01:58, Bert Freudenberg <[hidden email]> wrote:
>
> Am 11.05.2010 um 15:50 schrieb John M McIntosh:
>> So as Bert said malloc() is the simplest solution...
>
> I actually didn't mention malloc(). Since in SqueakNOS there *is* no operating system, I'd have thought you don't even need to malloc().
>

Yeah, i think SqueakNOS could simply declare an appropriate address
space range as its own, and just use it :)

> - Bert -
>
>
>

--
Best regards,
Igor Stasenko AKA sig.

melkyades

Re: Where is the heap?

Well well well, this is nice, now I'm getting more into this. Thanks all for the quick answers! The iSqueak wiki (which I had seen before but thought it only had info about iPhone) helped a lot.

Here are some concepts I got (hope they are right)

The heap is at a virtual address that depends on the platform. Many platforms try to put it starting at a fixed virtual address (like 500M or 512M). Unix doesn't bother.

The use of /dev/null mapping in unix is not that important because of mmap's MAP_PRIVATE flag, which makes a copy of the mapped object after first write. In mac it uses the image file instead and MAP_SHARED, which may actually change the image file on memory writes. So in that case I wonder why it doesn't happen (or it does?), maybe because the file is closed after initialization?

After loading, as oops are virtual memory addressed, they are iterated (in case the heap isn't loaded in a fixed position, ie. unix) and adjusted to the new heap start.

Also -this confused me a bit-, in some platforms you mmap a heap of size much bigger than image size, like 512MB (this was what puzzled me, I confused it with memory address offset). In the end you get 512MB mapped at offset 500M, of which the OS only reserves the used part. In the space between 0 and 500M lies the VM, stack globals and other stuff.

Excelent, now for SqueakNOS we may use the total amount of available memory, as Bert said, but we have to be carefull: Saving the image requires disk access, which is now done by a beautiful disk driver written in smalltalk (no file-write primitives). Our idea then is that inside primitiveSnapshot we make the snapshot go temporarily to a byte array instead of disk, and after resuming the interpreter we send the order to write to disk. As you can imagine, this has the drawback of requiring double of image size be reserved in memory among other things, but should work. We are open to more elegant ideas, of course!

As for heap mem allocating have a look at what we have now in squeaknos...

sqInt sqGrowMemoryBy(sqInt oldLimit, sqInt delta)    {
       // XXX: We should check available memory here.
    return oldLimit+delta;
}

Thanks!
           Javier.

On Tue, May 11, 2010 at 9:10 PM, Igor Stasenko <[hidden email]> wrote:

On 12 May 2010 01:58, Bert Freudenberg <[hidden email]> wrote:
>
> Am 11.05.2010 um 15:50 schrieb John M McIntosh:
>> So as Bert said malloc() is the simplest solution...
>
> I actually didn't mention malloc(). Since in SqueakNOS there *is* no operating system, I'd have thought you don't even need to malloc().
>

Yeah, i think SqueakNOS could simply declare an appropriate address
space range as its own, and just use it :)

> - Bert -

>
>
>

--
Best regards,
Igor Stasenko AKA sig.

--
Javier Pimás
Ciudad de Buenos Aires

johnmci

Re: Where is the heap?

Ok, well there is a number of optimizations on the os-x and iphone VM code that I should explain. I choose 500M as a start value because then I can save an image with an os-x based VM and move it to the iPhone without having the oops pointer logic wanting to reset all the oops offset values, ie if the saved start of memory is the same as the new start of memory then we skip the oops reset.

On the iPhone the mmap logic maps ram memory to the flash memory pages of the image, so if you don't touch the ram then it doesn't swap the page from flash into ram. So if you avoid doing the oops pointer reset, and the dozen or so allInstances that occur that at image startup time, then you avoid 2 or 3 seconds of clock time that was taken to swap in oops pages from flash to ram (for no purpose). This could be fixed if you could create images that only contain objects that are actually used, but that problem is hard to solve.

For the iPhone I could make the image file mmap writable, but by design image files are read-only, although I could make a writable copy, but was unsure of issues of crashes and what state the image file would be in. Well I could set a flag, but this issue is less important now from the days when an iphone had only about 64MB of ram to work with, versus the 150MB in an iphone 3GS. Still I might revisit this since it technically provides virtual memory backing to storage (flash) if ram becomes constraint.

For os-x and the iphone if you look closer I did mmap the image file bytes into space starting at 500MB, but then I mmaped anonymous memory after the end of the image file to the upper limit. That works quite well on the iphone and on testing on an older powerbook 3400 showed an interesting time/work curves as it pages in the oops pages, but we discovered that this particular setup does not properly work on NFS mounted volumes so the logic is turned off on os-x. Although there is a flag to enable it if you wish to shave a few 100ms off the os-x vm startup time. Otherwise on os-x we just dump the image file data into mmap memory via an fread() Yes mmap was faster (but faulty), and on os-x it would (unlike iphone) swap in all the image file pages at once.

On Tue, May 11, 2010 at 10:54 PM, Javier Pimás <[hidden email]> wrote:

The heap is at a virtual address that depends on the platform. Many platforms try to put it starting at a fixed virtual address (like 500M or 512M). Unix doesn't bother.