mmap, changes to co-ordinate mapping to particular address to avoid oops address remapping at startup time.

 
In the past year or so there has been discussion of using mmap to set  
a start address for the offset of object memory so that memory address  
swizzling does not occur.
Also various operating systems vendors say using mmap to page in large  
files is faster than using fread, this in fact is true and noticeable  
on the iPhone.

This requires a change to VMMaker so that sqAllocateMemory can pass  
the file pointer and the header size, since in the
sqAllocateMemory we need to figure out the file (image size) and also  
understand the headersize of the image file to return a proper address  
to work with.

#define sqAllocateMemory(x,y,f,headersize)

(a) map in the image file as MAP_PRIVATE (copy on write) at a set  
location,
         noting that on os-x mmap (file offset) does not allow for non-
multiples of page size, so we return the offset into the file as the  
address.
(b) map in the free space at the page boundary rounded up based on the  
file size, plus the set location. This should give two mmap memory  
regions side by side.

Obvious issues, if any mmap fails we exit. We cannot be sure what will  
happen for memory mapping in the future, should we attempt recover at  
some other start location?

Source code below for comment.

?bug memory refers to (startOfAnonymousMemory+headersize) yet  
calculation in sqGrowMemoryBy uses memory and gMaxHeapSize.
lazy adding of extra page size to freeSpaceRoundedUpToPageSize is  
preventing disaster, proper fix is exercise for the reader...


  extern usqInt  gMaxHeapSize;
  static usqInt gHeapSize;
  static  void *startOfmmapForANONMemory,*startOfmmapForImageFile;
  static size_t fileRoundedUpToPageSize,freeSpaceRoundedUpToPageSize;

  /* compute the desired memory allocation */

  extern unsigned char *memory;

  usqInt sqGetAvailableMemory() {
         return gMaxHeapSize;
  }

  usqInt sqAllocateMemoryMac(sqInt minHeapSize, sqInt  
*desiredHeapSize, FILE * f,usqInt headersize) {
         void  *possibleLocation,*startOfAnonymousMemory;
         off_t fileSize;
         struct stat sb;
         size_t pageSize= getpagesize();
         size_t pageMask= ~(pageSize - 1);
       
        #define valign(x) ((x) & pageMask)
        #pragma unused(minHeapSize,desiredHeapSize)
       
         possibleLocation = 500*1024*1024;
         gHeapSize = gMaxHeapSize;

         /* Lets see about mmap the image file into a chunk of memory at the  
500MB boundary rounding up to the page size
          Then we on the next page anonymously allocate the required free  
space for young space*/
       
         /* Thanks to David Pennell for suggesting this */
       
         fstat(fileno((FILE *)f), &sb);
         fileSize = sb.st_size;
         fileRoundedUpToPageSize = valign(fileSize+pageSize-1);
         startOfAnonymousMemory = (void *) ((size_t) fileRoundedUpToPageSize  
+ (size_t) possibleLocation);
       
         startOfmmapForImageFile = mmap(possibleLocation,  
fileRoundedUpToPageSize, PROT_READ|PROT_WRITE, MAP_FILE|
MAP_PRIVATE,fileno((FILE *)f), 0);
       
         if (startOfmmapForImageFile != possibleLocation) {
                 fprintf(stderr, "errno %d\n", errno);
                 perror("startOfmmapForImageFile failed");
                 exit(42);
         }
       
         freeSpaceRoundedUpToPageSize = valign(gMaxHeapSize)-
fileRoundedUpToPageSize+pageSize;
         startOfmmapForANONMemory = mmap(startOfAnonymousMemory,  
freeSpaceRoundedUpToPageSize, PROT_READ|PROT_WRITE, MAP_ANON|
MAP_SHARED,0,0);
       
         if (startOfmmapForANONMemory != startOfAnonymousMemory) {
                 fprintf(stderr, "errno %d\n", errno);
                 perror("startOfmmapForANONMemory failed");
                 exit(42);
         }
       
         return (usqInt) startOfmmapForImageFile+headersize;
  }

sqInt sqGrowMemoryBy(sqInt memoryLimit, sqInt delta) {
        if ((usqInt) memoryLimit + (usqInt) delta - (usqInt) memory >  
gMaxHeapSize)
                        return memoryLimit;

        gHeapSize += delta;
        return memoryLimit + delta;
  }

sqInt sqShrinkMemoryBy(sqInt memoryLimit, sqInt delta) {
         returnsqGrowMemoryBy(memoryLimit,0-delta);
}

sqInt sqMemoryExtraBytesLeft(sqInt flag) {
        return gMaxHeapSize - gHeapSize;
}

void sqMacMemoryFree() {
        if (memory == NULL)
                return;
        munmap(startOfmmapForImageFile,fileRoundedUpToPageSize);
        munmap(startOfmmapForANONMemory,freeSpaceRoundedUpToPageSize);
        memory = NULL;
}

 
On Fri, Oct 10, 2008 at 09:36:51PM -0700, John M McIntosh wrote: John,

If I understand correctly, the change in #readImageFromFile:HeapSize:StartingAt:
might look like this:

  "allocate a contiguous block of memory for the Squeak heap"
  memory := self
      allocateMemory: heapSize
      minimum: minimumMemory
      imageFile: f
      headerSize: headerSize.

For backward compatibility in the support code (e.g. for the existing unix
platform code), a define could be provided like this:

  #define allocateMemoryminimumimageFileheaderSize(heapSize, minimumMemory, f, headerSize) \
      sqAllocateMemory(minimumMemory, heapSize)

And for the iPhone VM, you could have a macro like this:

  #define allocateMemoryminimumimageFileheaderSize(heapSize, minimumMemory, f, headerSize) \
      sqAllocateMemoryMac(minimumMemory, heapSize, f, headerSize) {

Does that sound right? Change set attached for the VMMaker part.

- Dave

 
Well it's not quite a straight forward

(a) you need to disable the image read logic I guess this could be in  
the #define

So in  readImageFromFile: f HeapSize: desiredHeapSize StartingAt:  
imageOffset

we have
        "read in the image in bulk, then swap the bytes if necessary"
        bytesRead := self cCode: 'sqImageFileRead(pointerForOop(memory),  
sizeof(unsigned char), dataSize, f)'.
        bytesRead ~= dataSize ifTrue: [self unableToReadImageError].

That I think needs to become
        bytesRead := self cCode:  
'sqImageFileReadEntireImage(pointerForOop(memory), sizeof(unsigned  
char), dataSize, f)'.
        bytesRead ~= dataSize ifTrue: [self unableToReadImageError].

so that you can say
        #define s#define sqImageFileRead(ptr, sz, count, f)   count
which then nulls out what sqImageFileReadEntireImage does since the  
image reading is really done via the mmap which was done in the  
allocateMemoryminimumimageFileheaderSize
otherwise by default it would be
        #define sqImageFileReadEntireImage(ptr, sz, count, f)  
sqImageFileRead(ptr, sz, count, f)

(b) free?
Actually I'm not sure who if anyone ummaps memory on exit, however in  
this case one does need to remember that memory points to the start of  
file, not 0+headerSize
For testing I've just remember the mmap for the file, and mmap for the  
anonymous section.  However I'd guess a Hydra VM would have to do a  
bit more to remember those
two memory locations somewhere since I suspect the ability to startup  
and shutdown an image is more likely without terminating the process  
space.


On Oct 23, 2008, at 6:16 PM, David T. Lewis wrote:

--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
=
=
=
========================================================================

 
On Thu, Oct 23, 2008 at 06:52:25PM -0700, John M McIntosh wrote:
> Well it's not quite a straight forward
>
> (a) you need to disable the image read logic I guess this could be in  
> the #define
>
> So in  readImageFromFile: f HeapSize: desiredHeapSize StartingAt:  
> imageOffset

John,

Here is another try at it. The idea is to maintain backward compatibility
with the existing support code, so the Windows, Unix, and RiskOS support
code require no changes, but you can define your new mmap() allocators
in sqConfig.h:

#define allocateMemoryMinimumImageFileHeaderSize(heapSize, minimumMemory, fileStream, headerSize) \
    myMemoryAllocator(heapSize, minimumMemory, fileStream, headerSize)
#define sqImageFileReadEntireImage(memoryAddress, fileStream, elementSize,  length) \
    myImageFileReader(memoryAddress, fileStream, elementSize,  length)
It would be trivial to add this later on, so I think it would be best
to wait and see if Hydra needs it.

Dave

 
Ok, I check this out, seems ok. I check in the macintosh changes.  I  
assume here if you stick this in VMMaker the
windows and unix version will compile without changes.  If so can you  
update VMMaker then?


On 30-Oct-08, at 5:00 AM, David T. Lewis wrote:

--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
=
=
=
========================================================================

 
On Tue, Nov 25, 2008 at 01:39:30PM -0800, John M McIntosh wrote:
> Ok, I check this out, seems ok. I check in the macintosh changes.  I  
> assume here if you stick this in VMMaker the
> windows and unix version will compile without changes.  If so can you  
> update VMMaker then?
>

The changes are now in VMMaker-dtl.109 on SqueakSource.

The change set is also on Mantis 7233: mmap mapping to particular address
to avoid oops address remapping at startup time.

Andreas, Ian: No changes are required for Windows or Unix platform sources.

- Dave

>From the preamble:

Change Set: Interpreter-readImageFromFile-jmm-dtl
Date: 23 October 2008
Author: David T. Lewis

Pass image file and image header length to object memory allocation
function in order to enable mmap loading without address swizzling,
per jmm proposal.

CCodeGenerator will provide default implementations in interp.h that are
backward compatible with the existing platform support code. These defaults
may be overridden by adding definitions such as the following to sqConfig.h
(or config.h via configure for the unix VM).

#define allocateMemoryMinimumImageFileHeaderSize(heapSize, minimumMemory, fileStream, headerSize) \
    myMemoryAllocator(heapSize, minimumMemory, fileStream, headerSize)


#define sqImageFileReadEntireImage(memoryAddress, fileStream, elementSize,  length) \
    myImageFileReader(memoryAddress, fileStream, elementSize,  length)