Smalltalk › Frameworks & Tools › GLORP

Suggestion For DuplicatePrimaryKey Exception (and a few questions)

‹ Previous Topic Next Topic ›
 
classic Classic list List threaded Threaded
♦
♦
 4 messages Options
jtuchel
Reply | Threaded
Open this post in threaded view
♦
♦
|

Suggestion For DuplicatePrimaryKey Exception (and a few questions)

jtuchel
Hi,


I'm still hunting for "randomly" occurring DuplicatePrimaryKeyExceptions when I insert objects. This may or may not be related to caching erros, like described by Esteban in an answer to this thread: https://groups.google.com/d/msg/glorp-group/eccmfOergeY/zC-JyLK3AQAJ.
So far, I have not found a way to reproduce the problem in a development image, but it happens quite often in production images. The question #isNew: leads to a DuplicateKeyException in Cache>>#includesKey:as: (note: not as a a result of an SQL INSERT, but long before that).

So Esteban has found problems in caching of Objects that are mapped using a FilteredTypeResolver, and I'd like to find out if that is the same problem as mine. But since I cannot reproduce the problem (yet?), all I can do is add some logging output or additional info in Walkbacks. For this I suggest the followong changes / extensions to Glorp:

  1. Add instance variables @cache and @key and accessors to DuplicatePrimaryKeyException
  2. Add a new instance creation method to DuplicatePrimaryKeyException class side:
    forKey: key inCache: aCache new: newObject existing: existingObject

        ^self new
            cache: aCache;
            key: key;
            newObject: newObject;
            existingObject: existingObject;
            yourself
  3. Change DuplicatePrimaryKeyException>>#messageText
    messageText

        ^'Cache lookup for  %1  in Cache %2 found existing  %3 for key: %4'
            bindWith: self newObject class printString
            with:
                self cache
                    printString "The Cache does not know wich Class(es) it caches, so this output is a bit useless, some keyAtValue: magic in the cache's mainCache might help ???"
            with: self existingObject class printString
            with: self key printString
  4. Change Cache>>#includesKey:as:

    includesKey: key as: anObject
        "Return true if we include the object, and it matches the given object. If we include a different object with the same key, raise an exception. Don't listen to any expiry policy"

        | item value |

        item := self basicAt: key ifAbsent: [^false].
        value := policy contentsOf: item.
        value == anObject ifFalse: [(DuplicatePrimaryKeyException forKey: key inCache: self new: anObject existing: value) signal].
        ^true
The idea here is that there will be at least the key and new and old object in our stack traces. The output of the Cache instance is not very helpful, however, because the Cache doesn't know which Class(es) it  caches. This info is only known to the CacheManager, which unfortunately is not accessible in Cache>>#includesKey:as:

I am thinking about catching the DuplicatePrimaryKeyException in CacheManager>>#containsObject:key: and doing some special research when it gets signalled by the Cache, but that will be quite specific to my use case and will most likely be quite "dirty code"...

So what do people think about this? Is this worth pulling upstream into the Glorp base code?
Are there any ideas how to augment the exception with info on which Cache (caching whcih Class, that is) includes that exiting object? ---> I need to find out if the lookup happens in the right Cache, to understand whether this is the same problem that Esteban found. Or if maybe I have a timing problem in my application...

Any comments?


Joachim




--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/glorp-group/ea32a2aa-ea56-41d9-84d8-492dc7f210c4%40googlegroups.com.
Madhu
Reply | Threaded
Open this post in threaded view
♦
♦
|

RE: Suggestion For DuplicatePrimaryKey Exception (and a few questions)

Madhu

Hi Joachim,

 

We use Glorp in our application and many years ago, we used to get DuplicatePrimaryKey exception in some cases. Some cases were reproducible in development image, some not. We did not create new objects in the session, so the case was not regarding sql INSERT nor did we work with two sessions, which might also trigger this exception. After loading some objects from database, and calling beginUnitOfWork caused the exception.

 

In our case (ObjectStudio/VisualWorks), the exception occurred because we use WeakVWCachePolicy. I find WeakVWCachePolicy very nice as objects are thrown out of the cache if not referenced anywhere. Actually the problem was more with Ephemeron than Glorp because Glorp uses EphemeralValueDictionary which is a subclass of EphemeronDictionary. We had the problem with the order in which the objects were finalized by GC. Sometimes, direct references in ephemerons were finalized after indirect references. This triggered the exception.

 

To solve the problem, we increased the number of strong references to keep (WeakVWCachePolicy>>numberOfElements:). By doing so we didn't get the exception anymore.

 

I don't know if increasing the number of elements in your cache would help. But give it a try if you use same cache policy as we do.

 

Regards,

Madhu.

 

From: [hidden email] <[hidden email]> On Behalf Of jtuchel
Sent: 07 September 2019 11:53
To: glorp-group <[hidden email]>
Subject: Suggestion For DuplicatePrimaryKey Exception (and a few questions)

 

Hi,

 

 

I'm still hunting for "randomly" occurring DuplicatePrimaryKeyExceptions when I insert objects. This may or may not be related to caching erros, like described by Esteban in an answer to this thread: https://groups.google.com/d/msg/glorp-group/eccmfOergeY/zC-JyLK3AQAJ.

So far, I have not found a way to reproduce the problem in a development image, but it happens quite often in production images. The question #isNew: leads to a DuplicateKeyException in Cache>>#includesKey:as: (note: not as a a result of an SQL INSERT, but long before that).

 

So Esteban has found problems in caching of Objects that are mapped using a FilteredTypeResolver, and I'd like to find out if that is the same problem as mine. But since I cannot reproduce the problem (yet?), all I can do is add some logging output or additional info in Walkbacks. For this I suggest the followong changes / extensions to Glorp:

 

  1. Add instance variables @cache and @key and accessors to DuplicatePrimaryKeyException
  2. Add a new instance creation method to DuplicatePrimaryKeyException class side:
    forKey: key inCache: aCache new: newObject existing: existingObject

        ^self new
            cache: aCache;
            key: key;
            newObject: newObject;
            existingObject: existingObject;
            yourself
  3. Change DuplicatePrimaryKeyException>>#messageText
    messageText

        ^'Cache lookup for  %1  in Cache %2 found existing  %3 for key: %4'
            bindWith: self newObject class printString
            with:
                self cache
                    printString "The Cache does not know wich Class(es) it caches, so this output is a bit useless, some keyAtValue: magic in the cache's mainCache might help ???"
            with: self existingObject class printString
            with: self key printString
  4. Change Cache>>#includesKey:as:

    includesKey: key as: anObject
        "Return true if we include the object, and it matches the given object. If we include a different object with the same key, raise an exception. Don't listen to any expiry policy"

        | item value |

        item := self basicAt: key ifAbsent: [^false].
        value := policy contentsOf: item.
        value == anObject ifFalse: [(DuplicatePrimaryKeyException forKey: key inCache: self new: anObject existing: value) signal].
        ^true

The idea here is that there will be at least the key and new and old object in our stack traces. The output of the Cache instance is not very helpful, however, because the Cache doesn't know which Class(es) it  caches. This info is only known to the CacheManager, which unfortunately is not accessible in Cache>>#includesKey:as:

 

I am thinking about catching the DuplicatePrimaryKeyException in CacheManager>>#containsObject:key: and doing some special research when it gets signalled by the Cache, but that will be quite specific to my use case and will most likely be quite "dirty code"...

 

So what do people think about this? Is this worth pulling upstream into the Glorp base code?

Are there any ideas how to augment the exception with info on which Cache (caching whcih Class, that is) includes that exiting object? ---> I need to find out if the lookup happens in the right Cache, to understand whether this is the same problem that Esteban found. Or if maybe I have a timing problem in my application...

 

Any comments?

 

 

Joachim

 

 

 

 

--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/glorp-group/ea32a2aa-ea56-41d9-84d8-492dc7f210c4%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/glorp-group/00c401d565a2%24b22145d0%241663d170%24%40gmail.com.
jtuchel
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Suggestion For DuplicatePrimaryKey Exception (and a few questions)

jtuchel
Hi Madhu,

thanks very much for taking your time and answering in such detail. I really appreciate that. We seem to be very few people who both use Glorp and also visit and answer on this group. There is so much to learn and exchange knoweldge and experiences about.

It is hard to believe that the caching policy would yield a cache lookp problem for a duplicate key. I'd expect it to probably not find a previously cached object any more, but not an object with the same key previously registered in the very same session, but not being identical (#==), which this error actually means....

We are using the default CachePolicy default, and I guess we'll have to investigate this a little further...

Thank you very much for sharing and thus giving us a pointer what (else) to look at.


Joachim



Am Dienstag, 10. September 2019 16:54:26 UTC+2 schrieb Madhu:

Hi Joachim,

 

We use Glorp in our application and many years ago, we used to get DuplicatePrimaryKey exception in some cases. Some cases were reproducible in development image, some not. We did not create new objects in the session, so the case was not regarding sql INSERT nor did we work with two sessions, which might also trigger this exception. After loading some objects from database, and calling beginUnitOfWork caused the exception.

 

In our case (ObjectStudio/VisualWorks), the exception occurred because we use WeakVWCachePolicy. I find WeakVWCachePolicy very nice as objects are thrown out of the cache if not referenced anywhere. Actually the problem was more with Ephemeron than Glorp because Glorp uses EphemeralValueDictionary which is a subclass of EphemeronDictionary. We had the problem with the order in which the objects were finalized by GC. Sometimes, direct references in ephemerons were finalized after indirect references. This triggered the exception.

 

To solve the problem, we increased the number of strong references to keep (WeakVWCachePolicy>>numberOfElements:). By doing so we didn't get the exception anymore.

 

I don't know if increasing the number of elements in your cache would help. But give it a try if you use same cache policy as we do.

 

Regards,

Madhu.

 

From: <a href="javascript:" target="_blank" gdf-obfuscated-mailto="aE33kzBWAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">glorp...@... <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="aE33kzBWAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">glorp...@...> On Behalf Of jtuchel
Sent: 07 September 2019 11:53
To: glorp-group <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="aE33kzBWAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">glorp...@...>
Subject: Suggestion For DuplicatePrimaryKey Exception (and a few questions)

 

Hi,

 

 

I'm still hunting for "randomly" occurring DuplicatePrimaryKeyExceptions when I insert objects. This may or may not be related to caching erros, like described by Esteban in an answer to this thread: <a href="https://groups.google.com/d/msg/glorp-group/eccmfOergeY/zC-JyLK3AQAJ" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msg/glorp-group/eccmfOergeY/zC-JyLK3AQAJ&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msg/glorp-group/eccmfOergeY/zC-JyLK3AQAJ&#39;;return true;">https://groups.google.com/d/msg/glorp-group/eccmfOergeY/zC-JyLK3AQAJ.

So far, I have not found a way to reproduce the problem in a development image, but it happens quite often in production images. The question #isNew: leads to a DuplicateKeyException in Cache>>#includesKey:as: (note: not as a a result of an SQL INSERT, but long before that).

 

So Esteban has found problems in caching of Objects that are mapped using a FilteredTypeResolver, and I'd like to find out if that is the same problem as mine. But since I cannot reproduce the problem (yet?), all I can do is add some logging output or additional info in Walkbacks. For this I suggest the followong changes / extensions to Glorp:

 

  1. Add instance variables @cache and @key and accessors to DuplicatePrimaryKeyException
  2. Add a new instance creation method to DuplicatePrimaryKeyException class side:
    forKey: key inCache: aCache new: newObject existing: existingObject

        ^self new
            cache: aCache;
            key: key;
            newObject: newObject;
            existingObject: existingObject;
            yourself
  3. Change DuplicatePrimaryKeyException>>#messageText
    messageText

        ^'Cache lookup for  %1  in Cache %2 found existing  %3 for key: %4'
            bindWith: self newObject class printString
            with:
                self cache
                    printString "The Cache does not know wich Class(es) it caches, so this output is a bit useless, some keyAtValue: magic in the cache's mainCache might help ???"
            with: self existingObject class printString
            with: self key printString
  4. Change Cache>>#includesKey:as:

    includesKey: key as: anObject
        "Return true if we include the object, and it matches the given object. If we include a different object with the same key, raise an exception. Don't listen to any expiry policy"

        | item value |

        item := self basicAt: key ifAbsent: [^false].
        value := policy contentsOf: item.
        value == anObject ifFalse: [(DuplicatePrimaryKeyException forKey: key inCache: self new: anObject existing: value) signal].
        ^true

The idea here is that there will be at least the key and new and old object in our stack traces. The output of the Cache instance is not very helpful, however, because the Cache doesn't know which Class(es) it  caches. This info is only known to the CacheManager, which unfortunately is not accessible in Cache>>#includesKey:as:

 

I am thinking about catching the DuplicatePrimaryKeyException in CacheManager>>#containsObject:key: and doing some special research when it gets signalled by the Cache, but that will be quite specific to my use case and will most likely be quite "dirty code"...

 

So what do people think about this? Is this worth pulling upstream into the Glorp base code?

Are there any ideas how to augment the exception with info on which Cache (caching whcih Class, that is) includes that exiting object? ---> I need to find out if the lookup happens in the right Cache, to understand whether this is the same problem that Esteban found. Or if maybe I have a timing problem in my application...

 

Any comments?

 

 

Joachim

 

 

 

 

--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="aE33kzBWAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">glorp...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/glorp-group/ea32a2aa-ea56-41d9-84d8-492dc7f210c4%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/glorp-group/ea32a2aa-ea56-41d9-84d8-492dc7f210c4%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/glorp-group/ea32a2aa-ea56-41d9-84d8-492dc7f210c4%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/glorp-group/ea32a2aa-ea56-41d9-84d8-492dc7f210c4%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/glorp-group/3665c7f7-699f-4ae5-848c-1d07a2435eb7%40googlegroups.com.
jtuchel
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Suggestion For DuplicatePrimaryKey Exception (and a few questions)

jtuchel
Madhu,

it seems like changing the numberOfElements inst var in CachePolicy won't change much for us, because it is only used in one subclass: WeakVWCachePolicy

We are on VAST, so we cannot even try that CachePolicy.

This is not to say your analysis and suggested fix are wrong. It just isn't applicable to *our* problem...  It seems, however, that there are several things that can go wrong with Caching in Glorp, be it a bug in Glorp or in our use of Glorp.

We currently have a new hypothesis about our DuplicatePrimaryKey exception: since we use FilteredTypeResolvers in many places of our model, but only this specific class shows that effect, we were looking around for things that are different.

And there is a difference: the piece of code that creates the new object runs in a forked background process - including the #commitUnitOfWork, while that is not the case in other places of our app. The background process is needed in order to allow Ajax Callbacks to poll for the progress of this operation that creates the object.

Since we never could recreate the problem in our dev images, we are now "testing" the combination of a Semaphore and teh commitUnitOfWork outside of the forked Block, back in the main thread. (this brings up new problems, because this way Ajax Requests are not handled before the background process finished, but that's a different story). I say "testing", because we just introdced this change in production, the only place we've ever seen this happening. The change has been in production now for a few days and we haven't gotten a DuplicatePrimaryKey any more since then. Not sure this proves anything, wa hadn't had that Error daily, so we can just wait and see. We'll only find out it doesn't solve the problem, when the problem occurs, but we'll never find out if it provably does work...

The sad thing here is: I can not find a plausible explanation for why committing a UOW in a background process that is only forked once would ever try to register (or test for #isNew:) more than once... I'll be happy to hear theories on how this could happen (and we've added some logging just to make sure the backgorund process is really just forked once).

I'll report here or in the other thread on this group in a few days when we have more to say. For now, the "fix" is experimental.


Joachim







Am 11.09.19 um 08:25 schrieb jtuchel:
Hi Madhu,

thanks very much for taking your time and answering in such detail. I really appreciate that. We seem to be very few people who both use Glorp and also visit and answer on this group. There is so much to learn and exchange knoweldge and experiences about.

It is hard to believe that the caching policy would yield a cache lookp problem for a duplicate key. I'd expect it to probably not find a previously cached object any more, but not an object with the same key previously registered in the very same session, but not being identical (#==), which this error actually means....

We are using the default CachePolicy default, and I guess we'll have to investigate this a little further...

Thank you very much for sharing and thus giving us a pointer what (else) to look at.


Joachim



Am Dienstag, 10. September 2019 16:54:26 UTC+2 schrieb Madhu:

Hi Joachim,

 

We use Glorp in our application and many years ago, we used to get DuplicatePrimaryKey exception in some cases. Some cases were reproducible in development image, some not. We did not create new objects in the session, so the case was not regarding sql INSERT nor did we work with two sessions, which might also trigger this exception. After loading some objects from database, and calling beginUnitOfWork caused the exception.

 

In our case (ObjectStudio/VisualWorks), the exception occurred because we use WeakVWCachePolicy. I find WeakVWCachePolicy very nice as objects are thrown out of the cache if not referenced anywhere. Actually the problem was more with Ephemeron than Glorp because Glorp uses EphemeralValueDictionary which is a subclass of EphemeronDictionary. We had the problem with the order in which the objects were finalized by GC. Sometimes, direct references in ephemerons were finalized after indirect references. This triggered the exception.

 

To solve the problem, we increased the number of strong references to keep (WeakVWCachePolicy>>numberOfElements:). By doing so we didn't get the exception anymore.

 

I don't know if increasing the number of elements in your cache would help. But give it a try if you use same cache policy as we do.

 

Regards,

Madhu.

 

From: <a href="javascript:" target="_blank" gdf-obfuscated-mailto="aE33kzBWAwAJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;" moz-do-not-send="true">glorp...@... <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="aE33kzBWAwAJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;" moz-do-not-send="true">glorp...@...> On Behalf Of jtuchel
Sent: 07 September 2019 11:53
To: glorp-group <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="aE33kzBWAwAJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;" moz-do-not-send="true">glorp...@...>
Subject: Suggestion For DuplicatePrimaryKey Exception (and a few questions)

 

Hi,

 

 

I'm still hunting for "randomly" occurring DuplicatePrimaryKeyExceptions when I insert objects. This may or may not be related to caching erros, like described by Esteban in an answer to this thread: <a href="https://groups.google.com/d/msg/glorp-group/eccmfOergeY/zC-JyLK3AQAJ" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/msg/glorp-group/eccmfOergeY/zC-JyLK3AQAJ';return true;" onclick="this.href='https://groups.google.com/d/msg/glorp-group/eccmfOergeY/zC-JyLK3AQAJ';return true;" moz-do-not-send="true">https://groups.google.com/d/msg/glorp-group/eccmfOergeY/zC-JyLK3AQAJ.

So far, I have not found a way to reproduce the problem in a development image, but it happens quite often in production images. The question #isNew: leads to a DuplicateKeyException in Cache>>#includesKey:as: (note: not as a a result of an SQL INSERT, but long before that).

 

So Esteban has found problems in caching of Objects that are mapped using a FilteredTypeResolver, and I'd like to find out if that is the same problem as mine. But since I cannot reproduce the problem (yet?), all I can do is add some logging output or additional info in Walkbacks. For this I suggest the followong changes / extensions to Glorp:

 

  1. Add instance variables @cache and @key and accessors to DuplicatePrimaryKeyException
  2. Add a new instance creation method to DuplicatePrimaryKeyException class side:
    forKey: key inCache: aCache new: newObject existing: existingObject

        ^self new
            cache: aCache;
            key: key;
            newObject: newObject;
            existingObject: existingObject;
            yourself
  3. Change DuplicatePrimaryKeyException>>#messageText
    messageText

        ^'Cache lookup for  %1  in Cache %2 found existing  %3 for key: %4'
            bindWith: self newObject class printString
            with:
                self cache
                    printString "The Cache does not know wich Class(es) it caches, so this output is a bit useless, some keyAtValue: magic in the cache's mainCache might help ???"
            with: self existingObject class printString
            with: self key printString
  4. Change Cache>>#includesKey:as:

    includesKey: key as: anObject
        "Return true if we include the object, and it matches the given object. If we include a different object with the same key, raise an exception. Don't listen to any expiry policy"

        | item value |

        item := self basicAt: key ifAbsent: [^false].
        value := policy contentsOf: item.
        value == anObject ifFalse: [(DuplicatePrimaryKeyException forKey: key inCache: self new: anObject existing: value) signal].
        ^true

The idea here is that there will be at least the key and new and old object in our stack traces. The output of the Cache instance is not very helpful, however, because the Cache doesn't know which Class(es) it  caches. This info is only known to the CacheManager, which unfortunately is not accessible in Cache>>#includesKey:as:

 

I am thinking about catching the DuplicatePrimaryKeyException in CacheManager>>#containsObject:key: and doing some special research when it gets signalled by the Cache, but that will be quite specific to my use case and will most likely be quite "dirty code"...

 

So what do people think about this? Is this worth pulling upstream into the Glorp base code?

Are there any ideas how to augment the exception with info on which Cache (caching whcih Class, that is) includes that exiting object? ---> I need to find out if the lookup happens in the right Cache, to understand whether this is the same problem that Esteban found. Or if maybe I have a timing problem in my application...

 

Any comments?

 

 

Joachim

 

 

 

 

--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="aE33kzBWAwAJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;" moz-do-not-send="true">glorp...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/glorp-group/ea32a2aa-ea56-41d9-84d8-492dc7f210c4%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/msgid/glorp-group/ea32a2aa-ea56-41d9-84d8-492dc7f210c4%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter';return true;" onclick="this.href='https://groups.google.com/d/msgid/glorp-group/ea32a2aa-ea56-41d9-84d8-492dc7f210c4%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter';return true;" moz-do-not-send="true">https://groups.google.com/d/msgid/glorp-group/ea32a2aa-ea56-41d9-84d8-492dc7f210c4%40googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "glorp-group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/glorp-group/m3cGZquuGjE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/glorp-group/3665c7f7-699f-4ae5-848c-1d07a2435eb7%40googlegroups.com.


-- 
-----------------------------------------------------------------------
Objektfabrik Joachim Tuchel          [hidden email]
Fliederweg 1                         http://www.objektfabrik.de
D-71640 Ludwigsburg                  http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0         Fax: +49 7141 56 10 86 1

--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/glorp-group/921ec798-7bb4-f980-3ece-9dad47745aeb%40objektfabrik.de.
Free forum by Nabble Edit this page