LZ4 for backups in the near future?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

LZ4 for backups in the near future?

GLASS mailing list
Hi guys,

I was reading the release notes of 3.3 and I was very glad to see you started using LZ4. I made a Pharo wrapper for LZ4 [1] long ago when I was testing Fuel serializer with compressed streams.

Anyway, I was very happy with LZ4 performance. My backups of the extent takes quite some time in the compressing part (much more than in the smalltalk side!). So I was wondering if using LZ4 is in the plans or not (I read in the release notes that this is NOT used currently for backups...but I wonder if there is a technical reason or only a matter of time).

Thanks in advance, 



--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: LZ4 for backups in the near future?

GLASS mailing list
We plan to support LZ4 backups in the next major release (3.4).

On 3/11/2016 4:54 AM, Mariano Martinez Peck via Glass wrote:
Hi guys,

I was reading the release notes of 3.3 and I was very glad to see you started using LZ4. I made a Pharo wrapper for LZ4 [1] long ago when I was testing Fuel serializer with compressed streams.

Anyway, I was very happy with LZ4 performance. My backups of the extent takes quite some time in the compressing part (much more than in the smalltalk side!). So I was wondering if using LZ4 is in the plans or not (I read in the release notes that this is NOT used currently for backups...but I wonder if there is a technical reason or only a matter of time).

Thanks in advance, 



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: LZ4 for backups in the near future?

GLASS mailing list
Thanks Norm. That's good to hear!!


On Fri, Mar 11, 2016 at 10:37 AM, Norm Green via Glass <[hidden email]> wrote:
We plan to support LZ4 backups in the next major release (3.4).


On 3/11/2016 4:54 AM, Mariano Martinez Peck via Glass wrote:
Hi guys,

I was reading the release notes of 3.3 and I was very glad to see you started using LZ4. I made a Pharo wrapper for LZ4 [1] long ago when I was testing Fuel serializer with compressed streams.

Anyway, I was very happy with LZ4 performance. My backups of the extent takes quite some time in the compressing part (much more than in the smalltalk side!). So I was wondering if using LZ4 is in the plans or not (I read in the release notes that this is NOT used currently for backups...but I wonder if there is a technical reason or only a matter of time).

Thanks in advance, 



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: LZ4 for backups in the near future?

GLASS mailing list
Hi Norm,

Do you know if LZ4 backups are finally gonna make it for 3.4?


On Mon, Mar 14, 2016 at 11:28 PM, Mariano Martinez Peck <[hidden email]> wrote:
Thanks Norm. That's good to hear!!


On Fri, Mar 11, 2016 at 10:37 AM, Norm Green via Glass <[hidden email]> wrote:
We plan to support LZ4 backups in the next major release (3.4).


On 3/11/2016 4:54 AM, Mariano Martinez Peck via Glass wrote:
Hi guys,

I was reading the release notes of 3.3 and I was very glad to see you started using LZ4. I made a Pharo wrapper for LZ4 [1] long ago when I was testing Fuel serializer with compressed streams.

Anyway, I was very happy with LZ4 performance. My backups of the extent takes quite some time in the compressing part (much more than in the smalltalk side!). So I was wondering if using LZ4 is in the plans or not (I read in the release notes that this is NOT used currently for backups...but I wonder if there is a technical reason or only a matter of time).

Thanks in advance, 



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--



--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: LZ4 for backups in the near future?

GLASS mailing list
Hi Mariano,

LZ4 backups are in 3.4 for both regular (insecure) backups and the new secure backups.  secure backups are always digitally signed and may be compressed and/or encrypted as well.

Norm


On 6/26/17 12:12, Mariano Martinez Peck wrote:
Hi Norm,

Do you know if LZ4 backups are finally gonna make it for 3.4?


On Mon, Mar 14, 2016 at 11:28 PM, Mariano Martinez Peck <[hidden email]> wrote:
Thanks Norm. That's good to hear!!


On Fri, Mar 11, 2016 at 10:37 AM, Norm Green via Glass <[hidden email]> wrote:
We plan to support LZ4 backups in the next major release (3.4).


On 3/11/2016 4:54 AM, Mariano Martinez Peck via Glass wrote:
Hi guys,

I was reading the release notes of 3.3 and I was very glad to see you started using LZ4. I made a Pharo wrapper for LZ4 [1] long ago when I was testing Fuel serializer with compressed streams.

Anyway, I was very happy with LZ4 performance. My backups of the extent takes quite some time in the compressing part (much more than in the smalltalk side!). So I was wondering if using LZ4 is in the plans or not (I read in the release notes that this is NOT used currently for backups...but I wonder if there is a technical reason or only a matter of time).

Thanks in advance, 



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: LZ4 for backups in the near future?

GLASS mailing list


On Mon, Jun 26, 2017 at 4:22 PM, Norm Green <[hidden email]> wrote:
Hi Mariano,

LZ4 backups are in 3.4 for both regular (insecure) backups and the new secure backups.  secure backups are always digitally signed and may be compressed and/or encrypted as well.


Wow, that's very good news!!!!   
Let me ask... those features will be only for backups or for extent / tranlogs too? 

Cheers,

 
Norm



On 6/26/17 12:12, Mariano Martinez Peck wrote:
Hi Norm,

Do you know if LZ4 backups are finally gonna make it for 3.4?


On Mon, Mar 14, 2016 at 11:28 PM, Mariano Martinez Peck <[hidden email]> wrote:
Thanks Norm. That's good to hear!!


On Fri, Mar 11, 2016 at 10:37 AM, Norm Green via Glass <[hidden email]> wrote:
We plan to support LZ4 backups in the next major release (3.4).


On 3/11/2016 4:54 AM, Mariano Martinez Peck via Glass wrote:
Hi guys,

I was reading the release notes of 3.3 and I was very glad to see you started using LZ4. I made a Pharo wrapper for LZ4 [1] long ago when I was testing Fuel serializer with compressed streams.

Anyway, I was very happy with LZ4 performance. My backups of the extent takes quite some time in the compressing part (much more than in the smalltalk side!). So I was wondering if using LZ4 is in the plans or not (I read in the release notes that this is NOT used currently for backups...but I wonder if there is a technical reason or only a matter of time).

Thanks in advance, 



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--



--




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: LZ4 for backups in the near future?

GLASS mailing list
So far the new security features are just for backups.  We can consider adding it to extents and tranlogs, but we think performance might be better if compression/encryption is handled at the file system level using LUKS/dm-crypt rather than at the GemStone level.

Norm

On 6/26/17 12:28, Mariano Martinez Peck wrote:


On Mon, Jun 26, 2017 at 4:22 PM, Norm Green <[hidden email]> wrote:
Hi Mariano,

LZ4 backups are in 3.4 for both regular (insecure) backups and the new secure backups.  secure backups are always digitally signed and may be compressed and/or encrypted as well.


Wow, that's very good news!!!!   
Let me ask... those features will be only for backups or for extent / tranlogs too? 

Cheers,

 
Norm



On 6/26/17 12:12, Mariano Martinez Peck wrote:
Hi Norm,

Do you know if LZ4 backups are finally gonna make it for 3.4?


On Mon, Mar 14, 2016 at 11:28 PM, Mariano Martinez Peck <[hidden email]> wrote:
Thanks Norm. That's good to hear!!


On Fri, Mar 11, 2016 at 10:37 AM, Norm Green via Glass <[hidden email]> wrote:
We plan to support LZ4 backups in the next major release (3.4).


On 3/11/2016 4:54 AM, Mariano Martinez Peck via Glass wrote:
Hi guys,

I was reading the release notes of 3.3 and I was very glad to see you started using LZ4. I made a Pharo wrapper for LZ4 [1] long ago when I was testing Fuel serializer with compressed streams.

Anyway, I was very happy with LZ4 performance. My backups of the extent takes quite some time in the compressing part (much more than in the smalltalk side!). So I was wondering if using LZ4 is in the plans or not (I read in the release notes that this is NOT used currently for backups...but I wonder if there is a technical reason or only a matter of time).

Thanks in advance, 



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--



--




--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: LZ4 for backups in the near future?

GLASS mailing list


On Mon, Jun 26, 2017 at 4:32 PM, Norm Green <[hidden email]> wrote:
So far the new security features are just for backups.  We can consider adding it to extents and tranlogs, but we think performance might be better if compression/encryption is handled at the file system level using LUKS/dm-crypt rather than at the GemStone level.


Well, I guess your developed scenario is good and it covers serious / large apps. I was just thinking on "smaller" or simpler kind of apps, where the backups may reside in same root directory than the extent. For example, by default (it can be changed), GsDevKit_home  would have both backups and extents  under $GS_HOME/server/stones/stoneXXX. So... if for some reason my server is compromised who ever got access to the backups would also have access to the extent. So... encrypting the backups does not make sense in this case unless I also encrypt the extent. 

But.... as said, I do see the advantages of having encrypted backups. And it's good to know that I can move / share/ copy backups to another place (outside where the extent is) and they are encrypted. That is good!
 
Norm


On 6/26/17 12:28, Mariano Martinez Peck wrote:


On Mon, Jun 26, 2017 at 4:22 PM, Norm Green <[hidden email]> wrote:
Hi Mariano,

LZ4 backups are in 3.4 for both regular (insecure) backups and the new secure backups.  secure backups are always digitally signed and may be compressed and/or encrypted as well.


Wow, that's very good news!!!!   
Let me ask... those features will be only for backups or for extent / tranlogs too? 

Cheers,

 
Norm



On 6/26/17 12:12, Mariano Martinez Peck wrote:
Hi Norm,

Do you know if LZ4 backups are finally gonna make it for 3.4?


On Mon, Mar 14, 2016 at 11:28 PM, Mariano Martinez Peck <[hidden email]> wrote:
Thanks Norm. That's good to hear!!


On Fri, Mar 11, 2016 at 10:37 AM, Norm Green via Glass <[hidden email]> wrote:
We plan to support LZ4 backups in the next major release (3.4).


On 3/11/2016 4:54 AM, Mariano Martinez Peck via Glass wrote:
Hi guys,

I was reading the release notes of 3.3 and I was very glad to see you started using LZ4. I made a Pharo wrapper for LZ4 [1] long ago when I was testing Fuel serializer with compressed streams.

Anyway, I was very happy with LZ4 performance. My backups of the extent takes quite some time in the compressing part (much more than in the smalltalk side!). So I was wondering if using LZ4 is in the plans or not (I read in the release notes that this is NOT used currently for backups...but I wonder if there is a technical reason or only a matter of time).

Thanks in advance, 



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--



--




--




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass