Low Space Warning

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Low Space Warning

Jon Hylands

Hi guys,

I'm getting a weird problem, and its been so many years since I saw
this, I'm at a loss as to what to do...

On my robot Brainbot, I have a 2.83 GHz Core 2 Quad with 2 GB RAM and
a 32 GB solid state HD. It is running Windows XP Pro.

I'm getting a Low Space Notifier whenever I try and copy text to the
clipboard, or when I try and paste. I can do other development things,
writing a saving code, and opening browsers and windows with no
problems.

Task Manager says Squeak is taking up 37,980 K, and the machine has
1.2 GB of free physical memory.

I'm running the following VM:

Squeak 3.10.6 (release) from Aug 30 2007

And the following image:

Squeak 3.10.2
latest update: #7179

When I run Utilities garbageCollectAndReport, I get this:

6,038,864 bytes (internal)
506,832,208 bytes (physical)
373,974,352 bytes (total)

I am running the following GC setup code in that image:

        Smalltalk setGCBiasToGrow: 1.

        "do an  incremental GC after  this many allocations"
        SmalltalkImage current vmParameterAt: 5 put: 8000.  
        "tenure when more  than this  many objects survive the GC"
        SmalltalkImage current vmParameterAt: 6 put: 4000.  

        "grow headroom"
        SmalltalkImage current vmParameterAt: 25 put: 24*1024*1024.
        "shrink threshold"
        SmalltalkImage current vmParameterAt: 24 put: 48*1024*1024.

        Smalltalk setGCBiasToGrowGCLimit: 16*1024*1024.
        Smalltalk setGCBiasToGrow: 1.

Those are basically values I copied from something that appeared on
this list a few years ago.

Any ideas?

Thanks,
Jon


Reply | Threaded
Open this post in threaded view
|

Re: Low Space Warning

Jon Hylands
On Fri, 19 Feb 2010 16:25:20 -0500, Jon Hylands <[hidden email]> wrote:

>I'm getting a weird problem, and its been so many years since I saw
>this, I'm at a loss as to what to do...
>
>On my robot Brainbot, I have a 2.83 GHz Core 2 Quad with 2 GB RAM and
>a 32 GB solid state HD. It is running Windows XP Pro.
>
>I'm getting a Low Space Notifier whenever I try and copy text to the
>clipboard, or when I try and paste. I can do other development things,
>writing a saving code, and opening browsers and windows with no
>problems.
>
>Task Manager says Squeak is taking up 37,980 K, and the machine has
>1.2 GB of free physical memory.
>
>I'm running the following VM:
>
>Squeak 3.10.6 (release) from Aug 30 2007
>
>And the following image:
>
>Squeak 3.10.2
>latest update: #7179

Okay, so this is still bothering me, and I dug a little deeper, and
came across something that blew me away...

http://www.huv.com/Clipboard-OutOfMemory.png

Basically, when I do 'copy', the primitive returns a ByteString that
is 4 billion bytes long -- 2^32 - 1 to be exact. Clearly that is not
really allocated, because even when I have this inspector open, Squeak
is still only taking about 37 MB of memory.

However, the method String >> #withSqueakLineEndings is trying to
create a new string that is 4 billion bytes long, and that is where I
run into the low space warning.

Primitive 141 appears to be causing the issue.

I'm not much of a VM person, so hopefully someone who is can look at
that primitive and figure out what case is causing it.

One more thing to note - I'm running Windows in a Remote Desktop, from
a Linux machine. To clarify, my development machine is a desktop,
running Ubuntu linux. The robot is a mini-itx running Windows XP Pro.
I use remote desktop from linux to access the Windows GUI on the
robot.

Later,
Jon


Reply | Threaded
Open this post in threaded view
|

Re: Low Space Warning

Eliot Miranda-2


On Wed, Mar 10, 2010 at 6:39 AM, Jon Hylands <[hidden email]> wrote:
On Fri, 19 Feb 2010 16:25:20 -0500, Jon Hylands <[hidden email]> wrote:

>I'm getting a weird problem, and its been so many years since I saw
>this, I'm at a loss as to what to do...
>
>On my robot Brainbot, I have a 2.83 GHz Core 2 Quad with 2 GB RAM and
>a 32 GB solid state HD. It is running Windows XP Pro.
>
>I'm getting a Low Space Notifier whenever I try and copy text to the
>clipboard, or when I try and paste. I can do other development things,
>writing a saving code, and opening browsers and windows with no
>problems.
>
>Task Manager says Squeak is taking up 37,980 K, and the machine has
>1.2 GB of free physical memory.
>
>I'm running the following VM:
>
>Squeak 3.10.6 (release) from Aug 30 2007
>
>And the following image:
>
>Squeak 3.10.2
>latest update: #7179

Okay, so this is still bothering me, and I dug a little deeper, and
came across something that blew me away...

http://www.huv.com/Clipboard-OutOfMemory.png

Basically, when I do 'copy', the primitive returns a ByteString that
is 4 billion bytes long -- 2^32 - 1 to be exact. Clearly that is not
really allocated, because even when I have this inspector open, Squeak
is still only taking about 37 MB of memory.

However, the method String >> #withSqueakLineEndings is trying to
create a new string that is 4 billion bytes long, and that is where I
run into the low space warning.

Primitive 141 appears to be causing the issue.

The primitive looks like this in my quite possibly obsolete VM sources:

primitiveClipboardText
"When called with a single string argument, post the string to 
the clipboard. When called with zero arguments, return a 
string containing the current clipboard contents."
| s sz |
argumentCount = 1
ifTrue: [s := self stackTop.
(self isBytes: s) ifFalse: [^ self primitiveFail].
self successful
ifTrue: [sz := self stSizeOf: s.
self clipboardWrite: sz From: s + BaseHeaderSize At: 0.
self pop: 1]]
ifFalse: [sz := self clipboardSize.
(self sufficientSpaceToAllocate: sz) ifFalse:[^self primitiveFail].
s := self instantiateClass: (self splObj: ClassString) indexableSize: sz.
self clipboardRead: sz Into: s + BaseHeaderSize At: 0.
self pop: 1 thenPush: s]

I'm willing to bet that clipboardSize is returning -1.

It probably needs to read something like

primitiveClipboardText
"When called with a single string argument, post the string to 
the clipboard. When called with zero arguments, return a 
string containing the current clipboard contents."
| s sz |
argumentCount = 1
ifTrue: [s := self stackTop.
(self isBytes: s) ifFalse: [^ self primitiveFail].
self successful
ifTrue: [sz := self stSizeOf: s.
self clipboardWrite: sz From: s + BaseHeaderSize At: 0.
self pop: 1]]
ifFalse: [sz := self clipboardSize.
>> (sz >= 0 and: [self sufficientSpaceToAllocate: sz]) ifFalse:[^self primitiveFail].
s := self instantiateClass: (self splObj: ClassString) indexableSize: sz.
self clipboardRead: sz Into: s + BaseHeaderSize At: 0.
self pop: 1 thenPush: s] 

I'm not much of a VM person, so hopefully someone who is can look at
that primitive and figure out what case is causing it.

One more thing to note - I'm running Windows in a Remote Desktop, from
a Linux machine. To clarify, my development machine is a desktop,
running Ubuntu linux. The robot is a mini-itx running Windows XP Pro.
I use remote desktop from linux to access the Windows GUI on the
robot.

The Windows VM's clipboardText function is a tad unsafe:

int clipboardSize(void) {
  HANDLE h;
  WCHAR *src;
  unsigned char *tmp;
  int i, count, bytesNeeded;

  /* Do we have text in the clipboard? */
  if(!IsClipboardFormatAvailable(CF_UNICODETEXT))
    return 0;

  if(!OpenClipboard(stWindow))
    return 0;

  /* Get it in unicode format. */
  h = GetClipboardData(CF_UNICODETEXT);
  src = GlobalLock(h);

  /* How many bytes do we need to store those unicode chars in UTF8 format? */
  bytesNeeded = WideCharToMultiByte(CP_UTF8, 0, src, -1,
                    NULL, 0, NULL, NULL );
  tmp = malloc(bytesNeeded+1);

  /* Convert Unicode text to UTF8. */
  WideCharToMultiByte(CP_UTF8, 0, src, -1, tmp, bytesNeeded , NULL, NULL);

  /* Count CrLfs for which we remove the extra Lf */
  count = bytesNeeded; /* ex. terminating zero */
  for(i=0; i<count; i++) {
    if((tmp[i] == 13) && (tmp[i+1] == 10)) bytesNeeded--;
  }
  bytesNeeded--; /* discount terminating zero */
  free(tmp); /* no longer needed */

  GlobalUnlock(h);
  CloseClipboard();

  return bytesNeeded;
}

If bytesNeeded is zero then the function will return (int)-1. 

From the MS manual page:

Return Value

Returns the number of bytes written to the buffer pointed to by lpMultiByteStr if successful. If the function succeeds and cbMultiByte is 0, the return value is the required size, in bytes, for the buffer indicated by lpMultiByteStr.

The function returns 0 if it does not succeed. To get extended error information, the application can call GetLastError, which can return one of the following error codes:

  • ERROR_INSUFFICIENT_BUFFER. A supplied buffer size was not large enough, or it was incorrectly set to NULL.
  • ERROR_INVALID_FLAGS. The values supplied for flags were not valid.
  • ERROR_INVALID_PARAMETER. Any of the parameter values was invalid.
  • ERROR_NO_UNICODE_TRANSLATION. Invalid Unicode was found in a string.

So something like

  /* How many bytes do we need to store those unicode chars in UTF8 format? */
     bytesNeeded = WideCharToMultiByte(CP_UTF8, 0, src, -1,
                    NULL, 0, NULL, NULL );
     if (bytesNeeded <= 0)
          return 0;

would be wise.
 

Later,
Jon