Dynamic setting of memory upper bound?

How would I go about maximizing #memoryUpperBound depending on the
amount of memory available to the OS? Current memory policy settings
require a hard number AFAICT, whereas I'm looking for something along
the lines of,

| totalOsMem |
totalOsMem := ???.
MemoryPolicy.DefaultMemoryUpperBound := (totalOsMem * 0.70) min:
256000000.
(ObjectMemory currentMemoryPolicy)
  setDefaults;
  updateAfterGC.

Any idea how to get a hold of totalOsMem? Is there a better way
altogether?

Cheers!

-Boris

--
+1.604.689.0322
DeepCove Labs Ltd.
4th floor 595 Howe Street
Vancouver, Canada V6C 2T5
http://tinyurl.com/r7uw4

[hidden email]

CONFIDENTIALITY NOTICE

This email is intended only for the persons named in the message
header. Unless otherwise indicated, it contains information that is
private and confidential. If you have received it in error, please
notify the sender and delete the entire message including any
attachments.

Thank you.

So far I'm pondering something along the lines of attached,

Win32SystemSupport new getGlobalMemoryStatus

-self:  a Win32MemoryStatus
-hash:  10070
-identity:  10070
load:  75
totalPhysical:  2144903168
availablePhysical:  532156416
totalPageFile:  4129312768
availablePageFile:  2418712576
totalVirtual:  2147352576
availableVirtual:  1831092224

and then using totalPhysical as a baseline to set the memory policy hard
bound, is this a reasonable approach at all?

Cheers!

-Boris

--
+1.604.689.0322
DeepCove Labs Ltd.
4th floor 595 Howe Street
Vancouver, Canada V6C 2T5
http://tinyurl.com/r7uw4

[hidden email]

CONFIDENTIALITY NOTICE

This email is intended only for the persons named in the message
header. Unless otherwise indicated, it contains information that is
private and confidential. If you have received it in error, please
notify the sender and delete the entire message including any
attachments.

Thank you.

total := Win32SystemSupport new getGlobalMemoryStatus totalPhysical.
mine := (total * 0.70s) truncated max: 256000000.
MemoryPolicy.DefaultMemoryUpperBound := mine.
(ObjectMemory currentMemoryPolicy)
        setDefaults;
        updateAfterGC

Thoughts?

-Boris

--
+1.604.689.0322
DeepCove Labs Ltd.
4th floor 595 Howe Street
Vancouver, Canada V6C 2T5
http://tinyurl.com/r7uw4

[hidden email]

CONFIDENTIALITY NOTICE

This email is intended only for the persons named in the message
header. Unless otherwise indicated, it contains information that is
private and confidential. If you have received it in error, please
notify the sender and delete the entire message including any
attachments.

Thank you.
hard along

Offhand, this looks like a nice idea. But I'm wondering why the limit
should correspond to your physical memory, since everything goes virtual
anyway. Maybe I'm missing something here.

- Dave

-----Original Message-----
From: Boris Popov [mailto:[hidden email]]
Sent: Thursday, December 06, 2007 5:03 PM
To: [hidden email]
Subject: RE: Dynamic setting of memory upper bound?

total := Win32SystemSupport new getGlobalMemoryStatus totalPhysical.
mine := (total * 0.70s) truncated max: 256000000.
MemoryPolicy.DefaultMemoryUpperBound := mine.
(ObjectMemory currentMemoryPolicy)
        setDefaults;
        updateAfterGC

Thoughts?

-Boris

--
+1.604.689.0322
DeepCove Labs Ltd.
4th floor 595 Howe Street
Vancouver, Canada V6C 2T5
http://tinyurl.com/r7uw4

[hidden email]

CONFIDENTIALITY NOTICE

This email is intended only for the persons named in the message
header. Unless otherwise indicated, it contains information that is
private and confidential. If you have received it in error, please
notify the sender and delete the entire message including any
attachments.

Thank you.
hard along

To clarify a bit, I guess you're just trying to peg the upper bound to a
more polite value. Maybe the machine won't be running any other big
applications, so there's a good chance that the entire image will fit in
memory much of the time. Sounds like a good experiment for better
performance.

-----Original Message-----
From: Wallen, David [mailto:[hidden email]]
Sent: Thursday, December 06, 2007 8:36 PM
To: Boris Popov; [hidden email]
Subject: RE: Dynamic setting of memory upper bound?

Offhand, this looks like a nice idea. But I'm wondering why the limit
should correspond to your physical memory, since everything goes virtual
anyway. Maybe I'm missing something here.

- Dave

-----Original Message-----
From: Boris Popov [mailto:[hidden email]]
Sent: Thursday, December 06, 2007 5:03 PM
To: [hidden email]
Subject: RE: Dynamic setting of memory upper bound?

total := Win32SystemSupport new getGlobalMemoryStatus totalPhysical.
mine := (total * 0.70s) truncated max: 256000000.
MemoryPolicy.DefaultMemoryUpperBound := mine.
(ObjectMemory currentMemoryPolicy)
        setDefaults;
        updateAfterGC

Thoughts?

-Boris

--
+1.604.689.0322
DeepCove Labs Ltd.
4th floor 595 Howe Street
Vancouver, Canada V6C 2T5
http://tinyurl.com/r7uw4

[hidden email]

CONFIDENTIALITY NOTICE

This email is intended only for the persons named in the message
header. Unless otherwise indicated, it contains information that is
private and confidential. If you have received it in error, please
notify the sender and delete the entire message including any
attachments.

Thank you.
hard along

David, indeed. Really the only reason I need to go in this direction is to allow users with more ram to run larger reports that bring back a bunch of data from a db, so some sort of dependent sizing would fit nicely. As far as virtual, I guess that would be a more correct way of doing it, yes.

Cheers!

-Boris
(Sent from a BlackBerry)

----- Original Message -----
From: Wallen, David <[hidden email]>
To: Wallen, David <[hidden email]>; Boris Popov; [hidden email] <[hidden email]>
Sent: Thu Dec 06 20:51:24 2007
Subject: RE: Dynamic setting of memory upper bound?

To clarify a bit, I guess you're just trying to peg the upper bound to a
more polite value. Maybe the machine won't be running any other big
applications, so there's a good chance that the entire image will fit in
memory much of the time. Sounds like a good experiment for better
performance.

-----Original Message-----
From: Wallen, David [[hidden email]]
Sent: Thursday, December 06, 2007 8:36 PM
To: Boris Popov; [hidden email]
Subject: RE: Dynamic setting of memory upper bound?

Offhand, this looks like a nice idea. But I'm wondering why the limit
should correspond to your physical memory, since everything goes virtual
anyway. Maybe I'm missing something here.

- Dave

-----Original Message-----
From: Boris Popov [[hidden email]]
Sent: Thursday, December 06, 2007 5:03 PM
To: [hidden email]
Subject: RE: Dynamic setting of memory upper bound?

total := Win32SystemSupport new getGlobalMemoryStatus totalPhysical.
mine := (total * 0.70s) truncated max: 256000000.
MemoryPolicy.DefaultMemoryUpperBound := mine.
(ObjectMemory currentMemoryPolicy)
        setDefaults;
        updateAfterGC

Thoughts?

-Boris

--
+1.604.689.0322
DeepCove Labs Ltd.
4th floor 595 Howe Street
Vancouver, Canada V6C 2T5
http://tinyurl.com/r7uw4

[hidden email]

CONFIDENTIALITY NOTICE

This email is intended only for the persons named in the message
header. Unless otherwise indicated, it contains information that is
private and confidential. If you have received it in error, please
notify the sender and delete the entire message including any
attachments.

Thank you.

> -----Original Message-----
> From: Boris Popov [[hidden email]]
> Sent: Thursday, December 06, 2007 4:48 PM
> To: [hidden email]
> Subject: RE: Dynamic setting of memory upper bound?
>
> So far I'm pondering something along the lines of attached,
>
> Win32SystemSupport new getGlobalMemoryStatus
>
> -self:  a Win32MemoryStatus
> -hash:  10070
> -identity:  10070
> load:  75
> totalPhysical:  2144903168
> availablePhysical:  532156416
> totalPageFile:  4129312768
> availablePageFile:  2418712576
> totalVirtual:  2147352576
> availableVirtual:  1831092224
>
> and then using totalPhysical as a baseline to set the memory policy
hard
> bound, is this a reasonable approach at all?
>
> Cheers!
>
> -Boris
>
> --
> +1.604.689.0322
> DeepCove Labs Ltd.
> 4th floor 595 Howe Street
> Vancouver, Canada V6C 2T5
> http://tinyurl.com/r7uw4
>
> [hidden email]
>
> CONFIDENTIALITY NOTICE
>
> This email is intended only for the persons named in the message
> header. Unless otherwise indicated, it contains information that is
> private and confidential. If you have received it in error, please
> notify the sender and delete the entire message including any
> attachments.
>
> Thank you.
>
> > -----Original Message-----
> > From: Boris Popov [[hidden email]]
> > Sent: Thursday, December 06, 2007 3:55 PM
> > To: [hidden email]
> > Subject: Dynamic setting of memory upper bound?
> >
> > How would I go about maximizing #memoryUpperBound depending on the
> > amount of memory available to the OS? Current memory policy settings
> > require a hard number AFAICT, whereas I'm looking for something
along
> > the lines of,
> >
> > | totalOsMem |
> > totalOsMem := ???.
> > MemoryPolicy.DefaultMemoryUpperBound := (totalOsMem * 0.70) min:
> > 256000000.
> > (ObjectMemory currentMemoryPolicy)
> >   setDefaults;
> >   updateAfterGC.
> >
> > Any idea how to get a hold of totalOsMem? Is there a better way
> > altogether?
> >
> > Cheers!
> >
> > -Boris
> >
> > --
> > +1.604.689.0322
> > DeepCove Labs Ltd.
> > 4th floor 595 Howe Street
> > Vancouver, Canada V6C 2T5
> > http://tinyurl.com/r7uw4
> >
> > [hidden email]
> >
> > CONFIDENTIALITY NOTICE
> >
> > This email is intended only for the persons named in the message
> > header. Unless otherwise indicated, it contains information that is
> > private and confidential. If you have received it in error, please
> > notify the sender and delete the entire message including any
> > attachments.
> >
> > Thank you.

__________________________________________________________
Karl Breith
AREVA NP GmbH
Methods & Codes (FDEEC-G)

Postfach 1109
91001 Erlangen
Phone:   +49 (0) 9131 900 5544 
Fax:     +49 (0) 9131 900 4081         
mail to:  [hidden email]
An AREVA and Siemens company
 
Vorsitzender des Aufsichtsrats: Olivier Wantz - Geschäftsführer: Dr. Ralf Güldner, Christian Hillrichs, Rüdiger Steuerlein
Sitz der Gesellschaft: Erlangen - Registergericht: Fürth, HRB 7817 - Umsatzsteuer-ID: DE 206407096 - www.areva-np.com
Wichtiger Hinweis: Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse bzw. sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank.

Important Note: This e-mail may contain trade secrets or privileged, undisclosed or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.

__________________________________________________________
Karl Breith
AREVA NP GmbH
Methods & Codes (FDEEC-G)

Postfach 1109
91001 Erlangen
Phone:   +49 (0) 9131 900 5544 
Fax:     +49 (0) 9131 900 4081         
mail to:  [hidden email]
An AREVA and Siemens company
 
Vorsitzender des Aufsichtsrats: Olivier Wantz - Geschäftsführer: Dr. Ralf Güldner, Christian Hillrichs, Rüdiger Steuerlein
Sitz der Gesellschaft: Erlangen - Registergericht: Fürth, HRB 7817 - Umsatzsteuer-ID: DE 206407096 - www.areva-np.com
Wichtiger Hinweis: Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse bzw. sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank.

Important Note: This e-mail may contain trade secrets or privileged, undisclosed or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.

__________________________________________________________
Karl Breith
AREVA NP GmbH
Methods & Codes (FDEEC-G)

Postfach 1109
91001 Erlangen
Phone:   +49 (0) 9131 900 5544 
Fax:     +49 (0) 9131 900 4081         
mail to:  [hidden email]
An AREVA and Siemens company
 
Vorsitzender des Aufsichtsrats: Olivier Wantz - Geschäftsführer: Dr. Ralf Güldner, Christian Hillrichs, Rüdiger Steuerlein
Sitz der Gesellschaft: Erlangen - Registergericht: Fürth, HRB 7817 - Umsatzsteuer-ID: DE 206407096 - www.areva-np.com
Wichtiger Hinweis: Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse bzw. sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank.

Important Note: This e-mail may contain trade secrets or privileged, undisclosed or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.

Boris Popov schrieb:
for a 32 Bit VM, max. 2GB are available per process. I doesn't look that
the windows VM is compiled with /LARGEADDRESSAWARE, and I've seen
rather nasty behavior on Windows Server 2003 with 4 GB when VW tried to
get more than 2 GB RAM.

so

"mine := (total * 0.70s) truncated max: 256000000) min: 2147483648"


But I wouldn't use this as a generic approach. Usually there are a lot
of other applications running on a PC, and allocating 70% of the
physical memory seems to be rude.

Boris

You may want to create a different MemoryPolicy that
has two upper memory bounds. The difference would be how
aggressive the gc is. When the first bound is reached
the gc would become more aggressive so as to insure that the
additional memory is really required.

Terry
 
===========================================================
Terry Raymond
Crafted Smalltalk
80 Lazywood Ln.
Tiverton, RI  02878
(401) 624-4517      [hidden email]
<http://www.craftedsmalltalk.com>
===========================================================

That's a good idea, thanks.

-Boris

--
+1.604.689.0322
DeepCove Labs Ltd.
4th floor 595 Howe Street
Vancouver, Canada V6C 2T5
http://tinyurl.com/r7uw4

[hidden email]

CONFIDENTIALITY NOTICE

This email is intended only for the persons named in the message
header. Unless otherwise indicated, it contains information that is
private and confidential. If you have received it in error, please
notify the sender and delete the entire message including any
attachments.

Thank you.
totalPhysical. that
> > the windows VM is compiled with /LARGEADDRESSAWARE, and I've seen
> > rather nasty behavior on Windows Server 2003 with 4 GB when VW tried
to
> > get more than 2 GB RAM.
> >
> > so
> >
> > "mine := (total * 0.70s) truncated max: 256000000) min: 2147483648"
> >
> >
> > But I wouldn't use this as a generic approach. Usually there are a
lot
> > of other applications running on a PC, and allocating 70% of the
> > physical memory seems to be rude.
>

Boris,

for the RUT-K application of German Railways, we are using a customized
MemoryPolicy. RUT-K is used to construct detailed train paths, which
needs detailed information about railway infrastructure and the paths of
hundreds to thousands of other trains for conflict detection and for
detailed graphical views. This is memory-intensive, especially when
users open a dozen graphical views simultaneously.

Our MemoryPolicy gets the GlobalMemoryStatus from Windows, and computes
memoryUpperBound as 90% of physical RAM.  Going beyond that was a recipe
for desaster - once the image grew larger, swapping would cause
disruptions in the order of many seconds to many minutes. A global
garbage collection is the worst thing one can do in this situation... ;-)

The default MemoryPolicy answers constant values for memoryUpperBound
and growthRegimeUpperBound. We have overridden the methods to query the
OS and answer a computed value. Of course, the amount of physical RAM
doesn't change very often. ;-) The dynamic computation avoids problems
when the image is restarted on a different machine.

We compute growthRegimeUpperBound depending on the amount of free
(virtual) memory. When other applications consume more memory, our
application will thus try to avoid the allocation of old space segments
earlier. When enough memory is free, it will prefer memory allocation to
garbage collection.

Since we installed this regime, the application behaves much better when
it is started more than once, and in the presence of other applications.
When run on a computer with more RAM, it takes advantage of it instead
of being limited to a constant memoryUpperBound.

To sum it up: IMO, your idea is the right thing to do.

Best regards,
Joachim Geidel

Boris Popov schrieb am 07.12.2007 05:56: