Smalltalk › Pharo › Pharo Smalltalk Users

PUngLite can't query large database

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

3 messages Options

riverdusty

PUngLite can't query large database

Hi

I've managed to create a large database, with PUnqLite, that is 562.1MB.
I can't seem to query it though.
I've set fetchBufferSize: (1024*1024*1023).
I've set SmalltalkImage current vm extraVMMemory: (1024*1024*1023).
I've played around in the info.plist setting the SqueakMaxHeapSize to ridiculous things.
But I keep getting either out of memory or PqFetchError (probably because it can't fit the whole stream into the image?).
This is why I was asking about the JX9 query and running a query like that to only return to the image with results I want.
Any suggestions on how I can handle this?

Info.plist:

<key>SqueakMaxHeapSize</key>
<integer>1073741824</integer>

Pharo:

Image
-----
*****/Pharo4.0.image
Pharo4.0
Latest update: #40611
Unnamed

Operating System/Hardware
-------------------------
Mac OS 1093 intel

Kind Regards

Gareth Cox
IT Manager/Developer
Inspired Org (PTY) Ltd

Paul DeBruicker

Re: PUngLite can't query large database

Hi Gareth,

Sorry for the stupid question but does your query work on a smaller database, one a tenth the size or so?

Maybe by using a #haltIf: that checks for memory usage growth will show something interesting.

e.g.

self haltIf:[Smalltalk vm memoryEnd > 100000000]. "100MB"

And then running

Smalltalk reportCPUandRAM

to get a few text files of what's using the RAM & CPU.

Hope this helps

Paul

riverdusty wrote

Hi

I've managed to create a large database, with PUnqLite, that is 562.1MB.
I can't seem to query it though.
I've set fetchBufferSize: (1024*1024*1023).
I've set SmalltalkImage current vm extraVMMemory: (1024*1024*1023).
I've played around in the info.plist setting the SqueakMaxHeapSize to
ridiculous things.
But I keep getting either out of memory or PqFetchError (probably
because it can't fit the whole stream into the image?).
This is why I was asking about the JX9 query and running a query like
that to only return to the image with results I want.
Any suggestions on how I can handle this?

Info.plist:
> <key>SqueakMaxHeapSize</key>
> <integer>1073741824</integer>
Pharo:
> Image
> -----
> *****/Pharo4.0.image
> Pharo4.0
> Latest update: #40611
> Unnamed
>
> Operating System/Hardware
> -------------------------
> Mac OS 1093 intel

Kind Regards
--
Gareth Cox
IT Manager/Developer
Inspired Org (PTY) Ltd

riverdusty

Re: PUngLite can't query large database

Hi Paul

Yes, the query works on smaller databases.
I've created quite a few, one of them was 200MB and that worked!
I believe this is some memory/size limit.
I'm curious if there is a suitable way to overcome/override this limit.
I'm working on another project today but I'll try those suggestions perhaps later or tomorrow and get back with the results.

Thanks for the reply and suggestions.

KR