large blobs

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

large blobs

johnmci
Originally posted to the GLASS list, but suggested I repost to the gemstone list.

I'm collecting thoughts on using gemstone to store large chunks of data (say JPGS).  Say 8 to 16MB in size. I have another case of blob data maybe upwards of 200MB but I'd store that in chunks just to enable better restart capabilities when pushing/pulling to a mobile device as that would need to restart a previously aborted transmission. 

I'm wondering if anyone has had any experience with this? 

--
===========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd. Twitter: squeaker68882
===========================================================================


_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
Reply | Threaded
Open this post in threaded view
|

Re: large blobs

James Foster-9
Hi John,

While GemStone/S is a database, there is some overhead in storing data in it as compared to the file system. In general, if you are dealing with large, static blobs, then I’d suggest that they be kept outside the database and referenced from some internal object. The internal representation can have a path and if you want to represent chunks of the file, you can have an object that has not just a path but a begin and end offset. Of course, if you want, you can just store a ByteArray in GemStone/S and everything would work fine.

James

On Jun 16, 2014, at 4:19 PM, John McIntosh <[hidden email]> wrote:

Originally posted to the GLASS list, but suggested I repost to the gemstone list.

I'm collecting thoughts on using gemstone to store large chunks of data (say JPGS).  Say 8 to 16MB in size. I have another case of blob data maybe upwards of 200MB but I'd store that in chunks just to enable better restart capabilities when pushing/pulling to a mobile device as that would need to restart a previously aborted transmission. 

I'm wondering if anyone has had any experience with this? 

--
===========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd. Twitter: squeaker68882
===========================================================================

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk


_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
Reply | Threaded
Open this post in threaded view
|

Re: large blobs

Dennis smith-4
We had some similar questions a few years ago.

We ended up with two types of things
    - stored images (we used GIF's, small ones), for example where
        an "Employee" had a photo
    - other "documents"
        - might require editing as well as viewing
        - might be .xls, .jpg, .doc
        - might be larger, but that was not a specific criteria

We did have some issues dealing with commit/abort with an external document
        - delete it
        - keep it
        - ...

I would NOT have kept a .doc or .xls inside Gemstone.  A jpeg?? maybe, but not likely -- the only
thing we kept was something the user might want to look at as part of some other object
    (Employee, Customer, Contact, ...)

We did support things such that external vs stored (in GS) could be treated the same
(same behaviour).



On 2014-06-17 5:28 PM, James Foster wrote:
Hi John,

While GemStone/S is a database, there is some overhead in storing data in it as compared to the file system. In general, if you are dealing with large, static blobs, then I’d suggest that they be kept outside the database and referenced from some internal object. The internal representation can have a path and if you want to represent chunks of the file, you can have an object that has not just a path but a begin and end offset. Of course, if you want, you can just store a ByteArray in GemStone/S and everything would work fine.

James

On Jun 16, 2014, at 4:19 PM, John McIntosh <[hidden email]> wrote:

Originally posted to the GLASS list, but suggested I repost to the gemstone list.

I'm collecting thoughts on using gemstone to store large chunks of data (say JPGS).  Say 8 to 16MB in size. I have another case of blob data maybe upwards of 200MB but I'd store that in chunks just to enable better restart capabilities when pushing/pulling to a mobile device as that would need to restart a previously aborted transmission. 

I'm wondering if anyone has had any experience with this? 

--
===========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd. Twitter: squeaker68882
===========================================================================

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk



_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

-- 
Dennis Smith
Cherniak Software Development Corporation
416.798.7948  x208

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk