Smalltalk › Gemtalk › GLASS

[Glass] GsSocket can "only" read 10.000.000 Bytes ?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

7 messages Options

marten

[Glass] GsSocket can "only" read 10.000.000 Bytes ?

I've found that GsSocket can only receive maximum of 10.000.000 byte in
ONE call of:

read: maxBytes into: byteObj startingAt: index

I tried to upload 47 MByte of a file via Zinc and then this error is thrown.

Can this be configured ??? Otherwise the logic within Zinc must be
adapted to handle this limitation.

Marten
--
Marten Feldtmann
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

SebastianHC

[Glass] sorting and merging

Hi!

I need to offer access to CSV formated data via a REST interface.

Is there any best practice and or anti pattern to sort / merge / group the data which is held in different keyed collections and/or collections before I write the data onto a stream?
Right now I use a modified NeoCSV to read and write data. I added a CanonStringDict instance variable to the Reader to prevent unneccessary String instance duplication.

I am thinking of memory or processing time issues.

Thanks
Sebastian

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Dale Henrichs-3

Re: [Glass] sorting and merging

Sebastian,

I'd caution against premature optimization ... if you can identify/characterize a particular performance issue then it's worth going after it ...

one of the things with GemStone is that the gc for temporary objects can be VERY efficient as long as you can keep the number of temp objects low ... so a cache of temporary objects could end up causing things to run slower due to increased temp obj pressure... so it's not a given that creating a lot of temps will kill your performance because it might be the case where gc sweeps them up before they create a problem ...

When your entire image is scanned for gc, then it might be more of an issue, but the vm gc tradeoffs are slightly different because large chunks of your object graph are likely to be on disk ...

Dale

From: "Sebastian Heidbrink" <[hidden email]>
To: [hidden email]
Sent: Tuesday, December 10, 2013 12:31:43 PM
Subject: [Glass] sorting and merging

Hi!

I need to offer access to CSV formated data via a REST interface.

Is there any best practice and or anti pattern to sort / merge / group the data which is held in different keyed collections and/or collections before I write the data onto a stream?
Right now I use a modified NeoCSV to read and write data. I added a CanonStringDict instance variable to the Reader to prevent unneccessary String instance duplication.

I am thinking of memory or processing time issues.

Thanks
Sebastian

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Stephan Eggermont-3

Re: [Glass] GsSocket can "only" read 10.000.000 Bytes ?

In reply to this post by marten

Aren't you going to let the front webserver handle file uploads? AFAIK, they are by default limited to much smaller sizes (2 MByte comes to mind).

Stephan

Sent from my iPad

On 10 dec. 2013, at 21:17, "[hidden email]" <[hidden email]> wrote:

> I've found that GsSocket can only receive maximum of 10.000.000 byte in
> ONE call of:
>
> read: maxBytes into: byteObj startingAt: index
>
> I tried to upload 47 MByte of a file via Zinc and then this error is thrown.
>
> Can this be configured ??? Otherwise the logic within Zinc must be
> adapted to handle this limitation.
>
>
> Marten
> --
> Marten Feldtmann
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

marten

Re: [Glass] GsSocket can "only" read 10.000.000 Bytes ?

Yes, that is the idea at the end - but actually in a demo phase it
seemed easier to have one tool.

Marten

Am 15.12.2013 09:23, schrieb Stephan Eggermont:
> Aren't you going to let the front webserver handle file uploads? AFAIK, they are by default limited to much smaller sizes (2 MByte comes to mind).
>

--
Marten Feldtmann
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Stephan Eggermont-3

Re: [Glass] GsSocket can "only" read 10.000.000 Bytes ?

In Pharo the Zinc default maximum entity size is 16 MB. See ZnConstants.
Did you check the streaming support? I don't know how much of that is
in the Gemstone version of Zinc.

Stephan

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

marten

Re: [Glass] GsSocket can "only" read 10.000.000 Bytes ?

Its a limitation of the GsSocket primitive call - not related with Zinc
own limitations.

Marten

Am 15.12.2013 12:26, schrieb Stephan Eggermont:
> In Pharo the Zinc default maximum entity size is 16 MB. See ZnConstants.
> Did you check the streaming support? I don't know how much of that is
> in the Gemstone version of Zinc.
>
> Stephan
>
>

--
Marten Feldtmann
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass