[Glass] Saving "large" documents in GemStone
Locked
[Glass] Saving "large" documents in GemStone
 
|
Hi there,
we're wondering if GLASS users store files like PDF, MP3 or the like in GemStone. Or would you suggest putting such Blobs into external storage. For a possible project, we are thinking about whether it would be better to use a NoSQL DB for pdf files and such and only store "real" Smalltalk objects in GemStone.
The option of having all in one storage, only needing one backup/restore mechanism and such sounds interesting, but we are interested to hear opinions on whether this would work well. We are talking about many thousands of documents, sound snippets and such over time.
Thanks
Joachim
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] Saving "large" documents in GemStone
|
|
Joachim,
I'd like to hear from other users as well ... I think that historically, GemStone file i/o wasn't the fastest kid on the block especially if you needed to chunk up files to get them into memory. We have plans to make improvement in that particular area in 3.3. For the types of files you are talking about, there isn't much value to having them in memory and if you are serving them up through GemStone, then they will be consuming potentially valuable SPC (shared page cache) space. So if you can arrange to server up these files independently of GemStone then that makes a lot of sense. You do have a bit of a "2 phase commit" problem when initially storing the files to disk, because you do have to ensure that the file makes it to disk and the commit with path succeeds, but this is a manageable problem. Dale On 02/10/2015 08:22 AM, Joachim Tuchel
via Glass wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] Saving "large" documents in GemStone
|
|
Same ideas here,
we decided to put the images into the filesystem and only the file path into the database due to several reasons: -> Apache might directly access these pictures and delivers them without interaction with Gemstone (less CPU time needed on Gemstone) -> Database file might increase significantly and this could be a problem with the "free" introduction license -> SPC problem already mentioned -> We even upload the images NOT via Gemstone/HTTP to reduce CPU time for Gemstone and stay away from the 10MB limitation within Zinc/gemstone. Marten Am 10.02.2015 um 18:54 schrieb Dale Henrichs via Glass: > Joachim, > > I'd like to hear from other users as well ... > > I think that historically, GemStone file i/o wasn't the fastest kid on > the block especially if you needed to chunk up files to get them into > memory. We have plans to make improvement in that particular area in 3.3. > > For the types of files you are talking about, there isn't much value to > having them in memory and if you are serving them up through GemStone, > then they will be consuming potentially valuable SPC (shared page cache) > space. So if you can arrange to server up these files independently of > GemStone then that makes a lot of sense. > > You do have a bit of a "2 phase commit" problem when initially storing > the files to disk, because you do have to ensure that the file makes it > to disk and the commit with path succeeds, but this is a manageable problem. > > Dale > > On 02/10/2015 08:22 AM, Joachim Tuchel via Glass wrote: >> Hi there, >> >> we're wondering if GLASS users store files like PDF, MP3 or the like >> in GemStone. Or would you suggest putting such Blobs into external >> storage. For a possible project, we are thinking about whether it >> would be better to use a NoSQL DB for pdf files and such and only >> store "real" Smalltalk objects in GemStone. >> >> The option of having all in one storage, only needing one >> backup/restore mechanism and such sounds interesting, but we are >> interested to hear opinions on whether this would work well. We are >> talking about many thousands of documents, sound snippets and such >> over time. >> >> Thanks >> >> Joachim >> >> >> _______________________________________________ >> Glass mailing list >> [hidden email] >> http://lists.gemtalksystems.com/mailman/listinfo/glass > > > > _______________________________________________ > Glass mailing list > [hidden email] > http://lists.gemtalksystems.com/mailman/listinfo/glass > -- Marten Feldtmann _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] Saving "large" documents in GemStone
|
|
In reply to this post by GLASS mailing list
Hi, We store all such files (attachments) on the filesystem. After the upload (using Dropzone) we create for every fiile a file object in GemStone which holds some informations like: file name, file size, file type, relative url ... Gerhard On Tue, Feb 10, 2015 at 5:22 PM, Joachim Tuchel via Glass <[hidden email]> wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] Saving "large" documents in GemStone
|
|
Hi Dale, Marten, James and Gerhard,
thanks a lot for your comments. I was somewhat expecting something like this. I guess if we go with GemStone, we'll do as you all agree is best: store external files externally and keep the spc for more important things. Joachim Am 10.02.15 um 19:10 schrieb Gerhard Obermann via Glass:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
«
Return to GLASS
|
1 view|%1 views
| Free forum by Nabble | Edit this page |