Smalltalk › Gemtalk › Gemstone/S

[GBS] How often does this error affect you?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

13 messages Options

Richard Sargent

[GBS] How often does this error affect you?

Administrator

If you have a class in your client image mapped to the latest (or only) version of a server class, creating a new version of the server class will result in replication errors until you reconnect the client class to the new server class version.

GBS is designed to automatically migrate instances of older versions of a server class to the version that the client is connected to when you replicate such an instance. This is the "skew" case.

The problem case is the anti-skew, in which the automatic migration - if it were to happen - would have migrated instances of newer versions of the class to older versions. GBS throws an error. This error can arise as soon as you change the class definition and agree to "commit and migrate", if you already have an instance of the class replicated into the client image. It can also occur at any point following this event, if any subsequent replication tries to bring back an object graph referencing an instances of the new class version.

The error is a GbsClassGenerationError. It will occur more frequently starting with server version 3.2, since we corrected the problem of #become: not notifying the GCI client (GBS in this case) of the change. Migration of instances to the new class version uses #become:.

There are two workarounds to the error. One is to log out and back in again. The second is to use the Connector Browser and reconnect the class to the server. (The browser would show the class as disconnected, since the class is mapped to an older version, rather than the latest version of the named class.)

We would appreciate your feedback on how much of a problem this has been for you in the past, and if you have started to upgrade to 3.2, how much it impacts you now. Additionally, feedback on how you would like GBS to handle this case (other than "don't throw an error") would help us determine what we need to do.

Thank you,
Richard Sargent

Dennis smith-4

Re: [GBS] How often does this error affect you?

If I understand correctly, the problem occurs if the old class is
connected to the class on the client side?
Then migrate generates an error?

We have a tool UI with a migrate button which logs out and in THEN does
the migrate.

On 2014-05-14 11:44 AM, Richard Sargent wrote:

> If you have a class in your client image mapped to the latest (or only)
> version of a server class, creating a new version of the server class will
> result in replication errors until you reconnect the client class to the new
> server class version.
>
> GBS is designed to automatically migrate instances of older versions of a
> server class to the version that the client is connected to when you
> replicate such an instance. This is the "skew" case.
>
> The problem case is the anti-skew, in which the automatic migration - if it
> were to happen - would have migrated instances of newer versions of the
> class to older versions. GBS throws an error. This error can arise as soon
> as you change the class definition and agree to "commit and migrate", if you
> already have an instance of the class replicated into the client image. It
> can also occur at any point following this event, if any subsequent
> replication tries to bring back an object graph referencing an instances of
> the new class version.
>
> The error is a GbsClassGenerationError. It will occur more frequently
> starting with server version 3.2, since we corrected the problem of #become:
> not notifying the GCI client (GBS in this case) of the change. Migration of
> instances to the new class version uses #become:.
>
> There are two workarounds to the error. One is to log out and back in again.
> The second is to use the Connector Browser and reconnect the class to the
> server. (The browser would show the class as disconnected, since the class
> is mapped to an older version, rather than the latest version of the named
> class.)
>
>
> We would appreciate your feedback on how much of a problem this has been for
> you in the past, and if you have started to upgrade to 3.2, how much it
> impacts you now. Additionally, feedback on how you would like GBS to handle
> this case (other than "don't throw an error") would help us determine what
> we need to do.
>
>
> Thank you,
> Richard Sargent
>
>
>
>
> --
> View this message in context: http://forum.world.st/GBS-How-often-does-this-error-affect-you-tp4759041.html
> Sent from the Gemstone/S mailing list archive at Nabble.com.
> _______________________________________________
> GemStone-Smalltalk mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

Paul Baumann

Re: [GBS] How often does this error affect you?

In reply to this post by Richard Sargent

Hi Richard,

Problems like that are familiar, and are reduced with procedure. A related GBS issue is that replication can't behave according to spec as classes are connected on demand. Primarily from replication side effects, we turn off all on-demand generation of classes and connectors. An error is the proper way to show something as serious as a missing class or connector; arguably, an error is also a proper way to show class skew was discovered since connection. Client and server builds are deployed together. Gem Kit-produced GS patches with class changes or migrations are a special "cold" patch that is deployed with no active clients sessions. If someone changes a GS class that development clients are connected to then that session gets an error and the session works fine after reconnection. It is not a problem for us because the tools and processes we use are oriented toward working around it. With suitable procedure, an error like that becomes an environment management issue more than a GB
S bug.

It is an old problem. GBS attempts to do things that it can't do well (like using traversal content to discover in-traversal what the traversal content should have been). I remember looking at how to avoid it many years ago. The modifications required were in sensitive areas of code with far more risk than potential reward, so the problem endured. It sounds like 3.2 dives deep into those sensitive areas, so good luck. Replication policy is still incorrect even if you avoid the error, so I'd still configure GBS to raise errors instead. The approach that I'd explored was to discard the report and replay after clamps have been rebuilt, but that is imperfect because some of the cached objects have already been updated by the time the problem is discovered.

Paul Baumann

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Richard Sargent
Sent: Wednesday, May 14, 2014 11:44
To: [hidden email]
Subject: [GemStone-Smalltalk] [GBS] How often does this error affect you?

If you have a class in your client image mapped to the latest (or only) version of a server class, creating a new version of the server class will result in replication errors until you reconnect the client class to the new server class version.

GBS is designed to automatically migrate instances of older versions of a server class to the version that the client is connected to when you replicate such an instance. This is the "skew" case.

The problem case is the anti-skew, in which the automatic migration - if it were to happen - would have migrated instances of newer versions of the class to older versions. GBS throws an error. This error can arise as soon as you change the class definition and agree to "commit and migrate", if you already have an instance of the class replicated into the client image. It can also occur at any point following this event, if any subsequent replication tries to bring back an object graph referencing an instances of the new class version.

The error is a GbsClassGenerationError. It will occur more frequently starting with server version 3.2, since we corrected the problem of #become:
not notifying the GCI client (GBS in this case) of the change. Migration of instances to the new class version uses #become:.

There are two workarounds to the error. One is to log out and back in again.
The second is to use the Connector Browser and reconnect the class to the server. (The browser would show the class as disconnected, since the class is mapped to an older version, rather than the latest version of the named
class.)

We would appreciate your feedback on how much of a problem this has been for you in the past, and if you have started to upgrade to 3.2, how much it impacts you now. Additionally, feedback on how you would like GBS to handle this case (other than "don't throw an error") would help us determine what we need to do.

Thank you,
Richard Sargent

--
View this message in context: http://forum.world.st/GBS-How-often-does-this-error-affect-you-tp4759041.html
Sent from the Gemstone/S mailing list archive at Nabble.com.
_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.
_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

Dennis smith-4

Re: [GBS] How often does this error affect you?

I agree with all of this, except the connections -- we cannot connect
all our classes. We have, in the largest client, about 6,000 classes
that can be connected.
Not only would that take a very long time to do, but it tends to slow
things down operationally.

In any given session, the number is more likely on the order 500 or 600,
but we cannot predict -- so connects are done as needed (except for
about 50 or so
that everyone would use).

I agree with errors for non-migrated (either direction) -- they are
likely a serious problem.

On 2014-05-14 12:44 PM, Paul Baumann wrote:

> Hi Richard,
>
> Problems like that are familiar, and are reduced with procedure. A related GBS issue is that replication can't behave according to spec as classes are connected on demand. Primarily from replication side effects, we turn off all on-demand generation of classes and connectors. An error is the proper way to show something as serious as a missing class or connector; arguably, an error is also a proper way to show class skew was discovered since connection. Client and server builds are deployed together. Gem Kit-produced GS patches with class changes or migrations are a special "cold" patch that is deployed with no active clients sessions. If someone changes a GS class that development clients are connected to then that session gets an error and the session works fine after reconnection. It is not a problem for us because the tools and processes we use are oriented toward working around it. With suitable procedure, an error like that becomes an environment management issue more t
> han a GB
> S bug.
>
> It is an old problem. GBS attempts to do things that it can't do well (like using traversal content to discover in-traversal what the traversal content should have been). I remember looking at how to avoid it many years ago. The modifications required were in sensitive areas of code with far more risk than potential reward, so the problem endured. It sounds like 3.2 dives deep into those sensitive areas, so good luck. Replication policy is still incorrect even if you avoid the error, so I'd still configure GBS to raise errors instead. The approach that I'd explored was to discard the report and replay after clamps have been rebuilt, but that is imperfect because some of the cached objects have already been updated by the time the problem is discovered.
>
> Paul Baumann
>
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of Richard Sargent
> Sent: Wednesday, May 14, 2014 11:44
> To: [hidden email]
> Subject: [GemStone-Smalltalk] [GBS] How often does this error affect you?
>
> If you have a class in your client image mapped to the latest (or only) version of a server class, creating a new version of the server class will result in replication errors until you reconnect the client class to the new server class version.
>
> GBS is designed to automatically migrate instances of older versions of a server class to the version that the client is connected to when you replicate such an instance. This is the "skew" case.
>
> The problem case is the anti-skew, in which the automatic migration - if it were to happen - would have migrated instances of newer versions of the class to older versions. GBS throws an error. This error can arise as soon as you change the class definition and agree to "commit and migrate", if you already have an instance of the class replicated into the client image. It can also occur at any point following this event, if any subsequent replication tries to bring back an object graph referencing an instances of the new class version.
>
> The error is a GbsClassGenerationError. It will occur more frequently starting with server version 3.2, since we corrected the problem of #become:
> not notifying the GCI client (GBS in this case) of the change. Migration of instances to the new class version uses #become:.
>
> There are two workarounds to the error. One is to log out and back in again.
> The second is to use the Connector Browser and reconnect the class to the server. (The browser would show the class as disconnected, since the class is mapped to an older version, rather than the latest version of the named
> class.)
>
>
> We would appreciate your feedback on how much of a problem this has been for you in the past, and if you have started to upgrade to 3.2, how much it impacts you now. Additionally, feedback on how you would like GBS to handle this case (other than "don't throw an error") would help us determine what we need to do.
>
>
> Thank you,
> Richard Sargent
>
>
>
>
> --
> View this message in context: http://forum.world.st/GBS-How-often-does-this-error-affect-you-tp4759041.html
> Sent from the Gemstone/S mailing list archive at Nabble.com.
> _______________________________________________
> GemStone-Smalltalk mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
>
> This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.
> _______________________________________________
> GemStone-Smalltalk mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

Richard Sargent

Re: [GBS] How often does this error affect you?

Administrator

In reply to this post by Dennis smith-4

Dennis smith-4 wrote

If I understand correctly, the problem occurs if the old class is
connected to the class on the client side?
Then migrate generates an error?

We have a tool UI with a migrate button which logs out and in THEN does
the migrate.

Yes. However, it isn't migration, per se, that generates the error. It is replication afterward.

As for using the tool and technique you describe, Dennis, it seems that the scenario still can arise. If anyone else is using the database and has a class connected to the 'original' server class version, he will get a replication error any time after he refreshes his view of the database and replicates an instance of the class.

I realize that this problem really is a development issue, not a production issue. (At least, in general. In my philosophy, modifying a server class in a live production environment is something to be avoided.) As it only affects development, I can understand just living with it as an error which is fairly easily worked around.

Dennis smith-4

Re: [GBS] How often does this error affect you?

Yes -- it also insists on no other users logged in -- restricts us for
migrates a bit, but it works.

On 2014-05-14 1:10 PM, Richard Sargent wrote:

> Dennis smith-4 wrote
>> If I understand correctly, the problem occurs if the old class is
>> connected to the class on the client side?
>> Then migrate generates an error?
>>
>> We have a tool UI with a migrate button which logs out and in THEN does
>> the migrate.
> Yes. However, it isn't migration, per se, that generates the error. It is
> replication afterward.
>
> As for using the tool and technique you describe, Dennis, it seems that the
> scenario still can arise. If anyone else is using the database and has a
> class connected to the 'original' server class version, he will get a
> replication error any time after he refreshes his view of the database and
> replicates an instance of the class.
>
> I realize that this problem really is a development issue, not a production
> issue. (At least, in general. In my philosophy, modifying a server class in
> a live production environment is something to be avoided.) As it only
> affects development, I can understand just living with it as an error which
> is fairly easily worked around.
>
>
>
> --
> View this message in context: http://forum.world.st/GBS-How-often-does-this-error-affect-you-tp4759041p4759052.html
> Sent from the Gemstone/S mailing list archive at Nabble.com.
> _______________________________________________
> GemStone-Smalltalk mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

Paul Baumann

Re: [GBS] How often does this error affect you?

In reply to this post by Dennis smith-4

Hi Dennis,

We use bulk class connection (perhaps tuned here) of about 1,000 classes in sessions that are long-lived. Connection performance isn't a concern here anymore. Proper replication-spec behavior is important for us, so on-demand connection is not an option. The specs were required to avoid performance issues that are hopefully fixed since the GBS release we use. If the problems still exist then you may be missing out on huge performance gains by not carefully controlling replication policy. This is a short description of our replication configuration:

1) 2 is default fault level (1 is preferred, but some collections need 2 for specs to behave properly through them)
2) Simple objects like strings and dates are always replicated because a proxy takes as much time to create.
3) A home-grown spec framework generates a type-specific default replication spec that leaves the non-simple objects in GS unless requested.
4) The active window determines what spec is used, and specs can be customized for every window.
5) GBS has a few modifications to cause it to always use the application-specific spec for every replication (including replication by proxy).
6) Deliberate replication is through our own framework that specifies depth.
7) We use the Collection Views framework that I'd demonstrated at a Smalltalk conference some years ago.
8) Code is designed for one-trip-per-click to reduce the number of trips to GS while returning a minimum number of non-simple objects.
9) #instVarMap declarations are used avoid transporting data for unpaired attributes.

The great attention to replication was required to regain adequate performance. Any replication problems get noticed by users.

Paul Baumann

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Dennis Smith
Sent: Wednesday, May 14, 2014 12:49
To: [hidden email]
Subject: Re: [GemStone-Smalltalk] [GBS] How often does this error affect you?

I agree with all of this, except the connections -- we cannot connect all our classes. We have, in the largest client, about 6,000 classes that can be connected.
Not only would that take a very long time to do, but it tends to slow things down operationally.

In any given session, the number is more likely on the order 500 or 600, but we cannot predict -- so connects are done as needed (except for about 50 or so that everyone would use).

I agree with errors for non-migrated (either direction) -- they are likely a serious problem.

On 2014-05-14 12:44 PM, Paul Baumann wrote:

--
Dennis Smith
Cherniak Software Development Corporation
416.798.7948 x208

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.
_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

Dennis smith-4

Re: [GBS] How often does this error affect you?

Sounds like we both had similar issues -- our solutions were a bit
different ...
1) fault level 1 -- two special Array subclasses fault 2 and 3, specific
ivars can get 2 or 3, usually auto-specified
2) individual connects were not a critical issue, but a large number at
login got complaines
- connect about 50 at login
- open UI it figures out what it needs and connects them in a group
3) we deal a lot with Dates/Times/Fixed-Point -- they mostly get stored
as SmallInteger, expanded via read-accessor
4) store lots of nil's and use lazy initialization (actually no, just
returns the initial value, never stored)
5) our major Collections (Customers, Orders, Sales-Reps, .... -- 5000
things) live on Gemstone, objects accessed and cached
to VW

Took some time, but performance is pretty darn good now.

On 2014-05-14 1:32 PM, Paul Baumann wrote:

> Hi Dennis,
>
> We use bulk class connection (perhaps tuned here) of about 1,000 classes in sessions that are long-lived. Connection performance isn't a concern here anymore. Proper replication-spec behavior is important for us, so on-demand connection is not an option. The specs were required to avoid performance issues that are hopefully fixed since the GBS release we use. If the problems still exist then you may be missing out on huge performance gains by not carefully controlling replication policy. This is a short description of our replication configuration:
>
> 1) 2 is default fault level (1 is preferred, but some collections need 2 for specs to behave properly through them)
> 2) Simple objects like strings and dates are always replicated because a proxy takes as much time to create.
> 3) A home-grown spec framework generates a type-specific default replication spec that leaves the non-simple objects in GS unless requested.
> 4) The active window determines what spec is used, and specs can be customized for every window.
> 5) GBS has a few modifications to cause it to always use the application-specific spec for every replication (including replication by proxy).
> 6) Deliberate replication is through our own framework that specifies depth.
> 7) We use the Collection Views framework that I'd demonstrated at a Smalltalk conference some years ago.
> 8) Code is designed for one-trip-per-click to reduce the number of trips to GS while returning a minimum number of non-simple objects.
> 9) #instVarMap declarations are used avoid transporting data for unpaired attributes.
>
> The great attention to replication was required to regain adequate performance. Any replication problems get noticed by users.
>
> Paul Baumann
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of Dennis Smith
> Sent: Wednesday, May 14, 2014 12:49
> To: [hidden email]
> Subject: Re: [GemStone-Smalltalk] [GBS] How often does this error affect you?
>
> I agree with all of this, except the connections -- we cannot connect all our classes. We have, in the largest client, about 6,000 classes that can be connected.
> Not only would that take a very long time to do, but it tends to slow things down operationally.
>
> In any given session, the number is more likely on the order 500 or 600, but we cannot predict -- so connects are done as needed (except for about 50 or so that everyone would use).
>
> I agree with errors for non-migrated (either direction) -- they are likely a serious problem.
>
>
> On 2014-05-14 12:44 PM, Paul Baumann wrote:
>> Hi Richard,
>>
>> Problems like that are familiar, and are reduced with procedure. A related GBS issue is that replication can't behave according to spec as classes are connected on demand. Primarily from replication side effects, we turn off all on-demand generation of classes and connectors. An error is the proper way to show something as serious as a missing class or connector; arguably, an error is also a proper way to show class skew was discovered since connection. Client and server builds are deployed together. Gem Kit-produced GS patches with class changes or migrations are a special "cold" patch that is deployed with no active clients sessions. If someone changes a GS class that development clients are connected to then that session gets an error and the session works fine after reconnection. It is not a problem for us because the tools and processes we use are oriented toward working around it. With suitable procedure, an error like that becomes an environment management issue more t
>> han a GB
>> S bug.
>>
>> It is an old problem. GBS attempts to do things that it can't do well (like using traversal content to discover in-traversal what the traversal content should have been). I remember looking at how to avoid it many years ago. The modifications required were in sensitive areas of code with far more risk than potential reward, so the problem endured. It sounds like 3.2 dives deep into those sensitive areas, so good luck. Replication policy is still incorrect even if you avoid the error, so I'd still configure GBS to raise errors instead. The approach that I'd explored was to discard the report and replay after clamps have been rebuilt, but that is imperfect because some of the cached objects have already been updated by the time the problem is discovered.
>>
>> Paul Baumann
>>
>>
>> -----Original Message-----
>> From: [hidden email] [mailto:[hidden email]] On Behalf Of Richard Sargent
>> Sent: Wednesday, May 14, 2014 11:44
>> To: [hidden email]
>> Subject: [GemStone-Smalltalk] [GBS] How often does this error affect you?
>>
>> If you have a class in your client image mapped to the latest (or only) version of a server class, creating a new version of the server class will result in replication errors until you reconnect the client class to the new server class version.
>>
>> GBS is designed to automatically migrate instances of older versions of a server class to the version that the client is connected to when you replicate such an instance. This is the "skew" case.
>>
>> The problem case is the anti-skew, in which the automatic migration - if it were to happen - would have migrated instances of newer versions of the class to older versions. GBS throws an error. This error can arise as soon as you change the class definition and agree to "commit and migrate", if you already have an instance of the class replicated into the client image. It can also occur at any point following this event, if any subsequent replication tries to bring back an object graph referencing an instances of the new class version.
>>
>> The error is a GbsClassGenerationError. It will occur more frequently starting with server version 3.2, since we corrected the problem of #become:
>> not notifying the GCI client (GBS in this case) of the change. Migration of instances to the new class version uses #become:.
>>
>> There are two workarounds to the error. One is to log out and back in again.
>> The second is to use the Connector Browser and reconnect the class to the server. (The browser would show the class as disconnected, since the class is mapped to an older version, rather than the latest version of the named
>> class.)
>>
>>
>> We would appreciate your feedback on how much of a problem this has been for you in the past, and if you have started to upgrade to 3.2, how much it impacts you now. Additionally, feedback on how you would like GBS to handle this case (other than "don't throw an error") would help us determine what we need to do.
>>
>>
>> Thank you,
>> Richard Sargent
>>
>>
>>
>>
>> --
>> View this message in context: http://forum.world.st/GBS-How-often-does-this-error-affect-you-tp4759041.html
>> Sent from the Gemstone/S mailing list archive at Nabble.com.
>> _______________________________________________
>> GemStone-Smalltalk mailing list
>> [hidden email]
>> http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
>>
>> This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.
>> _______________________________________________
>> GemStone-Smalltalk mailing list
>> [hidden email]
>> http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
> --
> Dennis Smith
> Cherniak Software Development Corporation
> 416.798.7948 x208
>
>
> _______________________________________________
> GemStone-Smalltalk mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
>
> This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.

Richard Sargent

Re: [GBS] How often does this error affect you?

Administrator

In reply to this post by Paul Baumann

This is a great discussion, guys. Thanks!

Paul Baumann wrote

5) GBS has a few modifications to cause it to always use the application-specific spec for every replication (including replication by proxy).

Paul, is this something that should be incorporated into the GBS product? i.e., is it generally useful for any GBS customer?

Paul Baumann

Re: [GBS] How often does this error affect you?

>> Paul, is this something that should be incorporated into the GBS product?
>> i.e., is it generally useful for any GBS customer?

I think that would be useful, but I'm not sure how practical that will be. Martin had already moved in a useful direction by doing queries through #newMessageSend or #newServerTraversal; however, those were not application-configurable and hook points needed a little more control and monitoring than that alone. Our form of the command pattern allows simpler control of what is executed and how it is data is returned; the protocol is different. How it interacts with GBS was able to change as GBS had evolved. When a rare bug in single-trip GBS code appeared we were able to avoid it with one method change because we use our own command pattern. Here are our hooks in case your are interested:

GbsForwarder>>gbxPerform: sel withArguments: args
...
delegate validateSessionForProxy: self.
gsResult := (IcxGsQuery newForActiveView session: delegate session)
spec: (isForwarder ifTrue: [#delegate] ifFalse: [#current])
object: self
execute: selector
withArgs: args.
isForwarder ifFalse: [^gsResult].
gsResult == delegate ifTrue: [^self].
isUncachedForwarder ifTrue: [^gsResult asUncachedForwarder].
^gsResult asForwarder

GbxObjectStub>>faultToLevel: levelOrNil
...
"self faultToLevel: levelOrNil session: session delegate: delegate."
IcxGsQuery stub: self faultToLevel: levelOrNil session: session delegate: delegate.
self zzzisStub
...

IcxGsQuery class>>stub: gbsStub faultToLevel: levelOrNil session: gbsSession delegate: gbsDelegate
"This is our hook into GBS stub faults. We do this so that we can tune replication."

"gbsStub faultToLevel: levelOrNil session: gbsSession delegate: gbsDelegate."
| serverTraversal level result |
level := levelOrNil == nil
ifTrue: [gbsSession defaultGStoSTLevel]
ifFalse: [levelOrNil].
serverTraversal := gbsSession newServerTraversal.
result := serverTraversal
rootDelegate: gbsDelegate;
replicationSpecSelector: #icxRepSpec;
replicationLevel: level;
execute.
result ~~ gbsStub ifTrue: [[result == self] gbxAssert].
#faultToLevel:session:delegate: yourself.

(gbsStub zzzisStub not and: [GbsConfiguration current stubDebugging and: [gbsStub respondsTo: #icxObserveAttributeTypes]])
ifTrue: [gbsStub icxObserveAttributeTypes].

GbxObjectStub>>gbxPerform: selector withArguments: anArray
"This does a double-dispatch so that we can monitor or tune replication."

^IcxGsQuery stub: self gbxPerform: selector withArguments: anArray

IcxGsProxy class>>stub: gbsProxy gbxPerform: sel withArguments: args
"This is our hook into GBS proxy interactions so that we can monitor and tune replication."

| answer |
answer := gbsProxy overridden_gbxPerform: sel withArguments: args.
"The next line was just to aid debugging."
"(#(asText isEmpty) includes: sel) not ifTrue: [gbsProxy yourself]."
^answer

The #icxObserveAttributeTypes automatically updates code for the default type-specific minimal replication (#icxRepSpec). I generally consider stub faulting to be a replication spec bug, and I abhore replication as cached forwarders due to their multi-session limitations. If a fault happens then the specs can often self-tune the problem away for the next session login. Developers see the method changes and publish them with their other changes. It is a form of incremental optimization through automatic discovery of the object model in response to an inefficiency. Stub faulting is configured to use #icxRepSpec instead of the our default #icxRepSpec_minimal. Replication through a forwarder is what gets customized for the active window. It is rare to interact with GS by accident of a proxy; most interactions are through our own command pattern with automatic view-specific replication configuration.

A recent performance issue was the discovery of #remotePerform: type code in the application model that became replication costs that were not being monitored like a fault or forward. It is useful to for application code to be able to monitor the costs of every kind of interaction with GS. It is also useful to add artificial development-only delays to interactions to simulate network latency that can happen outside of development. It is a testing and tuning feature.

Paul Baumann

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Richard Sargent
Sent: Wednesday, May 14, 2014 15:00
To: [hidden email]
Subject: Re: [GemStone-Smalltalk] [GBS] How often does this error affect you?

This is a great discussion, guys. Thanks!

Paul Baumann wrote
> 5) GBS has a few modifications to cause it to always use the
> application-specific spec for every replication (including replication
> by proxy).

Paul, is this something that should be incorporated into the GBS product?
i.e., is it generally useful for any GBS customer?

--
View this message in context: http://forum.world.st/GBS-How-often-does-this-error-affect-you-tp4759041p4759065.html
Sent from the Gemstone/S mailing list archive at Nabble.com.
_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.
_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

Richard Sargent

Re: [GBS] How often does this error affect you?

Administrator

In reply to this post by Dennis smith-4

Dennis smith-4 wrote

I agree with all of this, except the connections -- we cannot connect
all our classes. We have, in the largest client, about 6,000 classes
that can be connected.
Not only would that take a very long time to do, but it tends to slow
things down operationally.

I have looked into this, and it appears to be a failing of how the connectors are meant to be hooked up. There is a "bulk resolution" mechanism, but it doesn't seem to be used for application classes. Consequently, each class connector requires (on my equipment) about 2ms for a round-trip to the server. So, 2 seconds per 1,000 classes to connect. Do these numbers seem to match your experience?

Dennis, is this an issue that you have previously reported via an HR?

Richard Sargent

Re: [GBS] How often does this error affect you?

Administrator

Richard Sargent wrote

I have looked into this, and it appears to be a failing of how the connectors are meant to be hooked up. There is a "bulk resolution" mechanism, but it doesn't seem to be used for application classes. Consequently, each class connector requires (on my equipment) about 2ms for a round-trip to the server. So, 2 seconds per 1,000 classes to connect. Do these numbers seem to match your experience?

I ran a test with a development branch of GBS against a 3.2 server. I like these results much better.

Logged in with 9 extra class connectors in 0.115372 seconds
Logged in with 19 extra class connectors in 0.097572 seconds
Logged in with 29 extra class connectors in 0.082155 seconds
Logged in with 99 extra class connectors in 0.093118 seconds
Logged in with 199 extra class connectors in 0.107003 seconds
Logged in with 299 extra class connectors in 0.143217 seconds
Logged in with 999 extra class connectors in 0.201468 seconds
Logged in with 1999 extra class connectors in 0.393284 seconds
Logged in with 2999 extra class connectors in 0.60691 seconds
Logged in with 9999 extra class connectors in 1.43287 seconds

If my math is right, that works out to about 0.14ms per class connector for the last example, more like 0.2 in the smaller examples.

Martin McClure-5

Re: [GBS] How often does this error affect you?

On 05/21/2014 10:15 AM, Richard Sargent wrote:

> Richard Sargent wrote
>> I have looked into this, and it appears to be a failing of how the
>> connectors are meant to be hooked up. There is a "bulk resolution"
>> mechanism, but it doesn't seem to be used for application classes.
>> Consequently, each class connector requires (on my equipment) about 2ms
>> for a round-trip to the server. So, 2 seconds per 1,000 classes to
>> connect. Do these numbers seem to match your experience?
>
> I ran a test with a development branch of GBS against a 3.2 server. I like
> these results much better.
>
> Logged in with 9 extra class connectors in 0.115372 seconds
> Logged in with 19 extra class connectors in 0.097572 seconds
> Logged in with 29 extra class connectors in 0.082155 seconds
> Logged in with 99 extra class connectors in 0.093118 seconds
> Logged in with 199 extra class connectors in 0.107003 seconds
> Logged in with 299 extra class connectors in 0.143217 seconds
> Logged in with 999 extra class connectors in 0.201468 seconds
> Logged in with 1999 extra class connectors in 0.393284 seconds
> Logged in with 2999 extra class connectors in 0.60691 seconds
> Logged in with 9999 extra class connectors in 1.43287 seconds
>
> If my math is right, that works out to about 0.14ms per class connector for
> the last example, more like 0.2 in the smaller examples.

And note that while there's some fixed overhead, the *incremental* cost
of each new class connector is only about 118 microseconds.

Regards,

-Martin
_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk