Problems bossing out objects with GUI dependents

Dear All,

        I have a problem with BOSS that, I hope, the list may be able to help
me out with.

        Much to my surprise, my well-tested methods for bossing out complex
data objects fail miserably when used from within the GUI. That
happens, I think, because some GUI elements become dependents of the
data objects (as they should be), and BOSSWriter finds itself to deal
with graphic elements. The message error I get, in fact, is
"GraphicsHandles cannot be stored by BOSS."
        The application developer's guide deals with this issue very briefly
in a single paragraph at the end of the discussion of BOSS (p.287 of
7.4.1 edition). However, if I understand the point correctly, the basic
suggestion is to provide a method that will enable the reconstruction
of the object from scratch. This approach may work fine with very
simple objects like the three-dimensional point used in the example,
but does not seem very feasible with the kind of rather complex objects
I am dealing with. In fact, it seems to me the suggested approach goes
directly against the very point of using BOSS, because it essentially
recommends  writing one's own serialization method (or methods, in my
case, given my nested objects). It almost (almost!) seems better to
avoid BOSS altogether and take care of the problem from scratch by
saving out text-based representations. In short, it seems that whenever
a complex objet is used as a direct model for a GUI, BOSS cannot be
used.

Well, this is how I understand the problem, but I have the nagging
feeling I am misunderstanding the issues surrounding BOSS use and
perhaps getting the wrong angle. Unfortunately,  neither VisualWorks
documentation nor books on Smalltalk provide much detail on the
subject. Or at least I could not find it.

Any help is appreciated.


Cheers,

Stefano






__________________________________________________
Stefano Franchi
Department of Philosophy                  Ph:   (64)  9 373-7599 x83940
University of Auckland Fax: (64) 9 373-8768
Private Bag 92019 [hidden email]
Auckland
New Zealand

I had the same issue in BottomFeeder.  What I do is strip dependencies off
the objects before they are Bossed out, and then reattach them
afterwards.  When I save the subscription info, it looks like this:

self
         doFeeds: feeds
         withoutDependents: [self writeBOSS: feeds to: filename].

The #doFeeds:withoutDependents: method looks like this:

doFeeds: feeds withoutDependents: aBlock
         "before a save, dump dependents so as not to fill the save file
with them.  also, simply can't
         save them in a BOSS save"

         | subscribedDeps  depsToRestore cachesToRestore |
         subscribedDeps := feeds myDependents.
         feeds breakDependents.
         depsToRestore := self removeDependenciesOnItems: feeds.
         cachesToRestore := self removeCachesFrom: feeds.
         aBlock
                 valueNowOrOnUnwindDo: [SystemStartup current isShuttingDown
                                                                         ifFalse:
[feeds myDependents: subscribedDeps.
                                                                                         self
restoreNotifiersAfterSave: depsToRestore.
                                                                                         self
restoreCachesAfterSave: cachesToRestore.
                                                                                         SystemStartup
current feedsNeedSave: false]]


Now, there's a bunch of code that is specific to my app there, but you see
the outline:

1) cache the dependents
2) remove them
3) save
4) restore them


At 06:35 PM 6/9/2006, you wrote:
<Talk Small and Carry a Big Class Library>
James Robertson, Product Manager, Cincom Smalltalk
http://www.cincomsmalltalk.com/blog/blogView

Hi James,

        thanks for the very prompt reply. I had a feeling I had to go the
"stripping dependents" way, but very little clue on how to do it. Your
sample code is precious.

Cheers,

Stefano

On Jun 9, 2006, at 3:51 PM, James Robertson wrote:
__________________________________________________
Stefano Franchi
Department of Philosophy                  Ph:  (64)  9 373-7599 x83940
University Of Auckland Fax: (64) 9 373-8768
Private Bag 92019 [hidden email]
Auckland
New Zealand

James Robertson wrote:
> Now, there's a bunch of code that is specific to my app there, but you
> see the outline:
>
> 1) cache the dependents
> 2) remove them
> 3) save
> 4) restore them

You will also have to apply this scheme to objects that contain
SortedCollections of children elements that link back to their parents.
I remove the parent links from each child before saving and retsore them
when done.

This is a bug in BOSS. Probably because the sortBlock causes trouble in
conjunction with the temporary object map during reconstruction of the
cyclic structure. I submitted this bug a while ago.

Andre

Hi Andre,

I use the cyclic structure you describe extensively, but haven't noticed
any problems when bossing out/in.

Can you please elaborate as to what you are encountering?

Cheers,

Stewart
you them
>when done.
>
>This is a bug in BOSS. Probably because the sortBlock causes trouble in
>conjunction with the temporary object map during reconstruction of the
>cyclic structure. I submitted this bug a while ago.
>
>Andre

Stefano Franchi wrote:
 > [...]
> Well, this is how I understand the problem, but I have the nagging
> feeling I am misunderstanding the issues surrounding BOSS use and
> perhaps getting the wrong angle. Unfortunately,  neither VisualWorks
> documentation nor books on Smalltalk provide much detail on the subject.
> Or at least I could not find it.

Hi,

another possible approach (that we used) is to BOSS out your objects as message
sends. To do that, you have to override #representBinaryOn: method to something
like:

        YourObject>>representBinaryOn: aBOSSWriter

                ^MessageSend
                        receiver: self class
                        selector: #iv1:iv2:iv3:
                        arguments:
                                (OrderedCollection new
                                        add: iv1;
                                        add: iv2;
                                        add: iv3;
                                        yourself).

and define BOSS instance creation method(s):

        YourObject class>>iv1: arg1 iv2: arg2: iv2 arg3: iv3
                "These should be all instance variables of YourObject."

                ^self new
                        setIv1: arg1
                        iv2: arg2
                        iv3.

        YourObject>>setIv1: arg1 iv2: arg2 iv3: arg3

                iv1 := arg1.
                iv2 := arg2.
                iv3 := arg3.

Note also, that you can not evaluate code other than assignments here,
because when you use cyclic structures, one or more args can still be
an Array and not your 'full featured object'.

Do the above for all objects that can have dependents or that directly refers
to something that should not be BOSSed at all.

When loading, you can also implement something like #recoverAfterLoad, which
restores all relationships that were not BOSSed out.


[Ignore the following if you don't need backward compatibility]

And one last advice, if you are going to extend your objects and want to be
backward compatible with your older BOSS formats, consider not using BOSS at
all, because I can tell you from my experience, it's hell... BOSS is simply
too level for this kind of job.

In that case consider creating some convert structure on your own, which
deals just with serialization / deserialization. You could then save this
new simplified structure (you can throw away everything that can be
reconstructed when loading) as XML and later when your objects change,
write XSLT transformation from previous XML version to the new one.

Given three versions of your format, the loading scheme for the first
version would be:

        XMLv1 ----> XMLv2 -----> XMLv3 -----> storage structure -----> full structure
               XSLT         XSLT

(Maybe you could omit the storage structure and implement the behavior
directly to your objects, but it could be messy.)


Hope this helps,

Ladislav Lenart

Stewart MacLean wrote:
> I use the cyclic structure you describe extensively, but haven't noticed
> any problems when bossing out/in.
>
> Can you please elaborate as to what you are encountering?
>
>  
Hi Stewart,

see the attached parcel. It contains a simple class that demonstrates
the BOSS bug. Method comments explain what's happening.

Greetings,
Andre


<?xml version="1.0"?>

<st-source>
<!--
Name: BOSS Bug Demonstration
Comment: This parcel demonstrates a bug in BOSS 7.1, which occurs when
filing in objects structured in a certain way. Hopefully it may help
fix the problem.

Problem description: Nested objects with parent references
(cyclic) being members of a SortedCollection are not
converted correctly. They remain Arrays of nils, which
causes the subsequent reSort to fail.

See methods and comments in the class BossBugTest
for further details.

Provided by Andre Schnoor.
[hidden email]

HideSource: false
Parcel: #('BOSS Bug Demonstration')
SaveSource: true
Date: 6:02:19 pm January 14, 2004
 -->
<time-stamp>From VisualWorks® NonCommercial, Release 7 of 21. März 2003 on 14. Januar 2004 at 18:02:19</time-stamp>


<do-it>(Dialog confirm: 'You are filing-in a Parcel source file!\\While this is possible it will not have\the same effect as loading the parcel.\None of the Parcel''s prerequisites will\be loaded and none of its load actions\will be performed.\\Are you sure you want to file-in?' withCRs) ifFalse: [self error: 'Parcel file-in abandoned.  Choose terminate or close.']</do-it>

<name-space>
<name>BossBug</name>
<environment>Core</environment>
<private>false</private>
<imports>
                        private Smalltalk.*
                        </imports>
<category>SortedCollection BOSS Bug</category>
</name-space>

<class>
<name>BossBugTest</name>
<environment>Core.BossBug</environment>
<super>Core.Object</super>
<private>false</private>
<indexed-type>none</indexed-type>
<inst-vars>object </inst-vars>
<class-inst-vars></class-inst-vars>
<imports></imports>
<category>SortedCollection BOSS Bug</category>
</class>

<class>
<name>BossBugComponent</name>
<environment>Core.BossBug</environment>
<super>Core.Object</super>
<private>false</private>
<indexed-type>none</indexed-type>
<inst-vars>owner sort components </inst-vars>
<class-inst-vars></class-inst-vars>
<imports></imports>
<category>SortedCollection BOSS Bug</category>
</class>

<comment>
<class-id>Core.BossBug.BossBugComponent</class-id>
<body>SortedCollectionBugComponent is a recursive object with parent references.

Instance Variables:
        components collection of embedded objects
        owner parent object
        sort random sorting value

</body>
</comment>

<methods>
<class-id>Core.BossBug.BossBugTest</class-id> <category>testing</category>

<body>cleanup
        "Use this message to release all cyclic objects that have been created:"

        "BossBug.BossBugTest new cleanup."

        BossBugComponent allInstances do:[ :c | c release ].</body>

<body>go
        "Run the test, which raises the error. Please use #cleanup message manually to release
         cyclic objects after testing is finished. Evaluate this:"

         "BossBug.BossBugTest new go."
        | bos loaded |
        self makeObject.
        "save it:"
        [ bos := BinaryObjectStorage onNew: self filename writeStream.
          bos nextPut: object ] ensure:[ bos close ].
        "load it:"
        [ bos := BinaryObjectStorage onOld: self filename readStream.
                loaded := bos next ]  ensure:[ bos close ].

        "if we reached here, everything worked fine"
        Transcript cr; show: 'Test passed'.</body>

<body>makeObject
        "Build the object that causes BOSS (7.1) problems: A Dictionary with
         SortedCollections containing objects with (cyclic) parent references."

        object := Dictionary new.
        1 to: self numberOfEntriesInDict do:[:i |
                object at: i printString put: self makeObjectCollection ].</body>

<body>makeObjectCollection
        "Build a SortedCollection with those special objects..."

        | answer |
        answer := self collectionClass new. "may be changed for testing"
        1 to: self numberOfElementsInCollection do:[:i |
                answer add: self makeObjectComponent ].
        ^answer</body>

<body>makeObjectComponent
        "Build the object with cyclic references that causes the BOSS problem"

        | answer |
        answer := BossBugComponent new.
        1 to: self numberOfCyclicComponents do:[:i |
                answer add: BossBugComponent new ].
        ^answer</body>
</methods>

<methods>
<class-id>Core.BossBug.BossBugTest</class-id> <category>constants</category>

<body>collectionClass
        "Change this to OrderedCollection and no error will occur and
         all objects are filed in correctly."

        ^SortedCollection</body>

<body>filename
        ^'test.bos' asFilename.
</body>

<body>numberOfCyclicComponents
        "Only larger values will raise the error ??? Seems to be a problem connected to the
         total number of objects being filed in, not each individial parameter of this
         test."

         "BossBug.BossBugTest new go."
        ^23</body>

<body>numberOfElementsInCollection
        "Values &gt; 23 cause the error ??? Seems to be a problem connected to the
         total number of objects being filed in, not each individial parameter of this
         test."

         "BossBug.BossBugTest new go."
        ^24</body>

<body>numberOfEntriesInDict
        "This also influences the error occurence: Needs to be &gt; 16 ????"

         "BossBug.BossBugTest new go."
        ^17</body>
</methods>

<methods>
<class-id>Core.BossBug.BossBugComponent</class-id> <category>comparing</category>

<body>&lt;= otherComponent
        ^sort &lt;= otherComponent sort</body>
</methods>

<methods>
<class-id>Core.BossBug.BossBugComponent</class-id> <category>accessing</category>

<body>owner
        ^owner</body>

<body>owner: anObject
        owner := anObject</body>

<body>sort
        ^sort</body>

<body>sort: anObject
        sort := anObject</body>
</methods>

<methods>
<class-id>Core.BossBug.BossBugComponent</class-id> <category>adding</category>

<body>add: aComponent

        components add: aComponent.
        aComponent owner: self.</body>
</methods>

<methods>
<class-id>Core.BossBug.BossBugComponent</class-id> <category>initialize-release</category>

<body>initialize

        components := OrderedCollection new.
        sort := Random new next.</body>

<body>release
        "drop cyclic references"
        owner := nil.
        components do:[:c | c release ].</body>
</methods>

<methods>
<class-id>Core.BossBug.BossBugComponent class</class-id> <category>instance creation</category>

<body>new
        ^super new initialize</body>
</methods>

<do-it>"Imported Classes:"</do-it>

<do-it>self error: 'Attempting to file-in parcel imports.  Choose terminate or close'</do-it>

<class>
<name>Object</name>
<environment>Core</environment>
<super></super>
<private>false</private>
<indexed-type>none</indexed-type>
<inst-vars></inst-vars>
<class-inst-vars></class-inst-vars>
<imports></imports>
<category>Kernel-Objects</category>
</class>

</st-source>

Andre Schnoor wrote:
Hi,

well the bug is not in the fact that SortedCollection is trying to sort
Arrays full of nils, but in the fact that it is trying to sort them at all.
To fix this, you have to modify #representBinaryOn: of SortedCollection to:

        SortedCollection>>representBinaryOn: aBOSSWriter

                ^MessageSend
                        receiver: self class
                        selector: #fromSortedArray:sortBlock:
                        arguments: (Array with: self asArray with: sortBlock).

or by other words, comment out the branch for case that actual sortBlock
is DefaultSortBlock.

[Note]
Arrays full of nils is proper BOSS behavior when dealing with cyclic
structures...

Hope this helps,

Ladislav Lenart

Andre Schnoor wrote:
Yes, BOSS preserves identity, so the sortBlock will be identical
*within* the BOSSed structure.

But let's take a look at the following scenario:
        1) Create an empty SortedCollection with DefaultSortBlock.
        2) BOSS the collection to internal stream.
        3) Read it back.
The result without 'my fix' is new empty SortedCollection which
references DefaultSortBlock. But the result after applying the
fix is empty SortedCollection with sortBlock equivalent (but not
identical!) to DefaultSortBlock. So as long as this is not a problem,
my fix will work.

Speaking of this, you may encounter problems when bossing cyclic
structure containing other collections - Sets and Dictionaries.
But in this case, the following might help:

        | boss |
        boss := BinaryObjectStorage onNew: aWriteStream
        boss expectCycles: true.
        boss nextPut: aRootOfCyclicStructure.
        [...]

BOSS differs when #expectCycles is true, see Set>>representBinaryOn:
and BOSSSpecialObjectLoader comment especially.

Hope this helps,

Ladislav Lenart

On 11 Jun, 2006, at 1:39 AM, Ladislav Lenart wrote:


        Well, after considering James's and Ladislav's proposed solutions to
the problem of Bossing out objects with GUI dependents I decided to go
with the latter and wrote out methods to override #representBinary:on:
for all the objects that have dependents. Plus the auxiliary methods
Ladislav suggested. Unfortunately, it does not seem to work.

        I have been digging through the (rather obscure, to me) BOSSWriter
code and what happens is that my top object is correctly assigned a
MessageSend by BOSSWriter>>trace: After the trace: method, though,
BOSSWriter goes into #trace:body:class: which, in its first lines,
retrieves the object's dependents and tries to find a representation
for them. Hence, the problem again, as the recursive descent eventually
ends up on a GraphicsHandle.

Can anyone enlighten me on why this happens? I thought overriding
#representBinary:on: with a MessageSend would avoid the recursive
tracing of the dependents of an object, but apparently this is not what
happens. Is there something I am missing here?


Best,

Stefano




__________________________________________________
Stefano Franchi
Department of Philosophy                  Ph:  (64)  9 373-7599 x83940
University Of Auckland Fax: (64) 9 373-8768
Private Bag 92019 [hidden email]
Auckland
New Zealand

Hi Stefano,

To avoid all these problems I do a very deep copy of the object
structure I wish to persist. During the copy I remove dependents and
break the connections to other parts of the model. When bossing in I
then rebind the structure.

Another point to note is to pass copies of collections to the GUI lists,
as it sometimes holds onto dependents of these, for change notification.

Cheers,

Stewart

PS If you're ever down in Wellington we could have a Smalltalk party
(the other ones have been a bit far away:))


solutions to
>the problem of Bossing out objects with GUI dependents I decided to go
>with the latter and wrote out methods to override #representBinary:on:
>for all the objects that have dependents. Plus the auxiliary methods
>Ladislav suggested. Unfortunately, it does not seem to work.
>
> I have been digging through the (rather obscure, to me)
BOSSWriter

Another simple solution is to not directly attach the GUI dependents to
the object being edited.

When displaying a List (whose contents are some aspect of the object)
for example, you could change the GUI code such that it refers to a copy
of the original collection. When the list is being edited (add/remove)
you need to delegate this behavior to manipulate the object, of course.
But that's what one is doing in most cases anyway.

I solved the dependency problem successfully this way.

HTH,
Andre

Stefano Franchi wrote: Well,

I try another more concrete example so you can verify it...

Let's suppose I have a class Person which has these instance variables:

        name (aString),
        address (anAddress),
        connection (aConnection).

Now I don't want to BOSS out connection instance variable of Person.
So I override #representBinaryOn: with:

        Person>>representBinaryOn: aBOSSWriter
                ^MessageSend
                        receiver: self class
                        selector: #bossName:address:
                        arguments: (Array with: name with: address).

        Person class>>bossName: aString address: anAddress
                ^self new
                        bossSetName: aString
                        address: anAddress.

        Person>>bossSetName: aString address: anAddress
                name := aString.
                address := anAddress.

As you can see, the connection will simply not be BOSSed with the Person.
If you reference the connection from more places, you have to eliminate all
these references in the same manner.

But after loading a Person from BOSS, I have to sent the whole structrure
something like #recoverAfterLoad method, which recreates all such
inpersistent connections. In case of Person, it will look like:
        Person>>recoverAfterLoad
                connection := self createConnection.

Again, you have to implement #recoverAfterLoad everywhere you need to
reconstruct some inpersistent relationship.

Someone mentioned that sometimes a collection in the model can have dependents.
If this is your case, you can override #representBinaryOn: of the collection
holder object to BOSS out copy of your collection. The copy will be definitely
dependents free.

BTW if your only problem are UI dependents, maybe all you have to do to eliminate
it is to inherit your model objects from Object instead of ValueModel...

If your problem remains and this will not help you, could you please write an
example that would clearly demonstrate your problem?

Ladislav Lenart

Ladislav Lenart wrote:
I am sorry - I didn't realize we override BOSSWriter>>trace:body:class: method...

I looked at trace:body:class: method and it turned out that we override
this method to suit our need, which in this case means not to store
dependents from DependentsFields (optionally). In BOSSWriter>>trace:body:class:
method, we replaced

        [...]
        deps == nil ifFalse:
                [self
                        trace: (anObject -> deps)
                        body: (Array with: anObject with: #myDependents: with: deps)
                        class: BinaryObjectStorage indexImportSend].
        ].
        [...]

with

        [...]
        storage storeDependents fTrue: [
                deps == nil ifFalse:
                        [self
                                trace: (anObject -> deps)
                                body: (Array with: anObject with: #myDependents: with: deps)
                                class: BinaryObjectStorage indexImportSend].
                ].
        ].
        [...]

And storeDependents is essentially an instance variable of BinaryObjectStorage,
though it is implemented externally via singleton (like all such overrides in
our image):

        BinaryObjectStorage>>storeDependents

                AnnotationHolder default
                        at: self
                        at: #storeDepenndents
                        ifAbsent: [true]

And setter method:

        BinaryObjectStorage>>storeDependents: aBoolean

                AnnotationHolder default
                        at: self
                        at: #storeDepenndents
                        put: aBoolean

This should finally solve your dependents problem once and for all (hopefully).

Ladislav Lenart

Ladislav

With respect to your comment about preventing bossing out
of a connection, I think there may be an easier way to
prevent an instance of a class from being bossed out.

From my reading of the code in BOSSWriter it appears that if
if #representBinaryOn: were to return 0 then instances of
that class would be dealt with as if they were nil, 0 is
the index for nil. So,
Connection>>representBinaryOn: aBossWriter
        ^0

Would prevent all instances of Connection from being bossed
out and upon reconstruction would set the connection ivar
to nil.

Alternatively, instead of returning 0 one could return a
message send that returns nil.

Please note that I have not tried this.

Terry
 
===========================================================
Terry Raymond       Smalltalk Professional Debug Package
Crafted Smalltalk
80 Lazywood Ln.
Tiverton, RI  02878
(401) 624-4517      [hidden email]
<http://www.craftedsmalltalk.com>
===========================================================

Terry Raymond wrote:
I never tried the 'return zero' variant, but MessageSend returning
nil works for sure - I used it several times myself. I don't know
why I forgot to mention it... It's a pitty it is not a Monday today,
because all I can say now is that I am sorry (again)... :-)

Ladislav Lenart