Persisting 'applications'

*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.

> 4. Do I have to use something like Magma to store
> the entire object tree.

Hi Bob, I've stored basic morphs connected with
Connectors in a Magma database before and it works
fine.

Projects seem to work too although I'd be a little
more cautious with them.  The problem is not in the
serializatino or materialization, its the
*initialization* that must occur.

Several of the projects in 3.7Full image all work and
part of the Magma test cases:

  - client2 expunges several projects ("Fun with
Music", "Smalltalk introduction", "Fun with Morphic",
etc.)
  - client1 persists these projects in the database
  - I am then able to retrieve and access the projects
in client2.

Let me know if you have any questions.

Good luck,
  Chris

> 4. Do I have to use something like Magma to store
> the entire object tree.

Chris Muller wrote:
>> Hi Bob, I've stored basic morphs connected with
>> Connectors in a Magma database before and it works
>> fine.

>> Projects seem to work too although I'd be a little
>> more cautious with them.  The problem is not in the
>> serializatino or materialization, its the
>> *initialization* that must occur.

>> Several of the projects in 3.7Full image all work and
>> part of the Magma test cases:

>>   - client2 expunges several projects ("Fun with
>> Music", "Smalltalk introduction", "Fun with Morphic",
>> etc.)
>>   - client1 persists these projects in the database
>>   - I am then able to retrieve and access the projects
>> in client2.

>> Let me know if you have any questions.

Thanks Chris:
I am completely blown away with the power of Magma. I've used object databases before but not one as transparent as this. It swallowed my whole radio and spat it out again intact. Really there was only one line of active code that saved the PasteUpMorph on which I built the radio. There is only one tiny thing I can't figure. My connectors come back connected unless I close the morph. Then the next load from the database they are disconnected. Still in place but the ends floating. Do you know of any way to close the morph without this happening? Also if I put in a window it restores without the window. I think I understand this because the window is not the root object. If I could get a reference to the window somehow and commit that I'm sure it would work.

Bob

*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.

Chris Muller a écrit :

I have not followed this discussion since the begining but you
*initialization* recall me that Projects seems to loose dependency
hooked with the when:sent:to: message. Indeed this message seems to hook
the dependency with an EventManager, which is a global objet.
I don't know though how Magma will perform there, but I will give a look
 later for sure.

Hilaire Fernandes

Sorry for the delayed response Bob.

> My connectors come
> back connected unless I
> close the morph. Then the next load from the
> database they are disconnected.
> Still in place but the ends floating. Do you know of
> any way to close the
> morph without this happening?

Its possible that the Connectors may need some
initialization on rematerialization.  Magma itself
does not require connectors so I have not extended any
of its classes with any pre or post serialization
stuff.

Did you store the Morphs while they were opened and
running in the world?  I try to support both, but you
might have better luck one way or the other.

Sorry I don't have a more definitive answer.  If you
want to get it working here is a suggested approach
(and I can guide you if you wish).

Explore your Radio in the committing image just before
you commit, getting ready to explore its full tree of
state.

In the retrieving image, after opening up the Radio in
the world, explore it.

Now place the two Squeak images side-by-side and work
down through each tree to find what objects in the
receiving image do not match the ones in the
committing image.

These objects will probably need a pre or post
serialization operation.  You can see some examples of
how I set up various pre/post-ops on some of Squeaks
built-in classes (like Color, PasteUpMorph and
Project) by looking at the method
MaObjectSerializer>>#setUpPreAndPostProcessing.

If you get this far let me know, I can probably help
you implement those methods.

> Also if I put in a
> window it restores without
> the window. I think I understand this because the
> window is not the root
> object. If I could get a reference to the window
> somehow and commit that I'm
> sure it would work.

That's a good guess but I'm not sure..  For Morphs
directly in the World, Magma does not serialize their
'owner' (the World).  However, I would think since you
put the RadioMorph into a window (a SystemWindow?)
that its 'owner' would be changed the window, not the
World, so it should have serialized its owner.

Sigh.  I'm sure I could debug this and figure it out
in person; do you have any money?  :)

Otherwise, just email me private if you wish and I'll
do my best to help you from afar.

Regards,
  Chris

Chris

It didn't click I was talking to the author. Thanks for taking the time, much appreciated. I have managed to get myself a little confused. I'm not long with Squeak so I am still in discovery mode. I found I had hundreds of live objects in my image so thought it best to find out how to manage lifetimes in the image before tackling Magma. I have a new image and everything loaded back in. I think I understand how to delete morphs now, but it does take some effort to make them 'really' go.

However, when I make them persistent I get into some trouble. I just can't seem to delete them from the database I've tried:

sdrBackplane := myMagmaSession root at: 'SDRBackplane'

to get a reference and then

myMagmaSession root: sdrBackplane
MagmaSession cleanup
myMagmaSession root removeKey: 'SDRBackplane'

to remove it (and variations thereof)

If I do:

SDRBackplane allInstances

I can get rid of any instances showing by:

sdrBackplane := nil
CommandHistory resetAllHistory
Smalltalk garbageCollect

Here's the interesting part. Once there are no instances in the image I can get one from the db by:

sdrBackplane := myMagmaSession root at: 'SDRBackplane'
sdrBackplane openInWorld

and lo and behold the connectors are still connected. It seems that if I get an instance or even close one (via the halo) and openInWorld again the connections are broken. It is the act of closing it and opening again that breaks the connections - NOT MAGMA. This happens even when it's not persistent.

So if you tell me how to delete objects reliably from Magma I think I might be there.

I tried this with the SystemWindow. Making sure it was gone from the image. What I get back is the SDRBackplane (which is just a PasteUpMorph) but it has the white background and border of the window rather than its own.

Sorry it's a bit rambling. I haven't won the lottery yet but if I do you will be the first to benefit !

Regards
Bob
--------------------------

Sorry for the delayed response Bob.

> My connectors come
> back connected unless I
> close the morph. Then the next load from the
> database they are disconnected.
> Still in place but the ends floating. Do you know of
> any way to close the
> morph without this happening?

Its possible that the Connectors may need some
initialization on rematerialization.  Magma itself
does not require connectors so I have not extended any
of its classes with any pre or post serialization
stuff.

Did you store the Morphs while they were opened and
running in the world?  I try to support both, but you
might have better luck one way or the other.

Sorry I don't have a more definitive answer.  If you
want to get it working here is a suggested approach
(and I can guide you if you wish).

Explore your Radio in the committing image just before
you commit, getting ready to explore its full tree of
state.

In the retrieving image, after opening up the Radio in
the world, explore it.

Now place the two Squeak images side-by-side and work
down through each tree to find what objects in the
receiving image do not match the ones in the
committing image.

These objects will probably need a pre or post
serialization operation.  You can see some examples of
how I set up various pre/post-ops on some of Squeaks
built-in classes (like Color, PasteUpMorph and
Project) by looking at the method
MaObjectSerializer>>#setUpPreAndPostProcessing.

If you get this far let me know, I can probably help
you implement those methods.

> Also if I put in a
> window it restores without
> the window. I think I understand this because the
> window is not the root
> object. If I could get a reference to the window
> somehow and commit that I'm
> sure it would work.

That's a good guess but I'm not sure..  For Morphs
directly in the World, Magma does not serialize their
'owner' (the World).  However, I would think since you
put the RadioMorph into a window (a SystemWindow?)
that its 'owner' would be changed the window, not the
World, so it should have serialized its owner.

Sigh.  I'm sure I could debug this and figure it out
in person; do you have any money?  :)

Otherwise, just email me private if you wish and I'll
do my best to help you from afar.

Regards,
  Chris

*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.

> sdrBackplane := myMagmaSession root at:
> 'SDRBackplane'
>
> to get a reference and then
>
> myMagmaSession root: sdrBackplane
> MagmaSession cleanup
> myMagmaSession root removeKey: 'SDRBackplane'

There's no need to reset the root again, it was
already there since you just got it in the first line.

The cleanUp is just an old sort of 'advanced utility'
method for dereferencing objects from temps in
MethodContexts..  It has nothing to do with removing
anything from a database, just from image memory; I'm
actually not sure whether its needed anymore since I
discovered #fixTemps..

I line 3 is wrapped in a commit, as in:

  myMagmaSession commit: [ myMagmaSession root
removeKey: 'SDRBackplane' ]

that will ensure the "delete" from the database is
saved to disk.  (I quote delete because I prefer the
term "dereference"; there is no delete, per se).

Ah, ok.  So maybe the connectors should maintain their
state in the World at runtime..  Hard to know.

> So if you tell me how to delete objects reliably
> from Magma I think I might
> be there.

Dereferencing any sub-part of your object-model is a
matter of changing the reference to point to the nil
object (wrapped in commit, of course).

 - Chris

Chris

Thank you very much, I'm on the straight and narrow now. The ability to
build multiple configurations and just persist them is magic.


> sdrBackplane := myMagmaSession root at:
> 'SDRBackplane'
>
> to get a reference and then
>
> myMagmaSession root: sdrBackplane
> MagmaSession cleanup
> myMagmaSession root removeKey: 'SDRBackplane'

>> There's no need to reset the root again, it was
>> already there since you just got it in the first line.

Sorry, bad copy and paste.

>> The cleanUp is just an old sort of 'advanced utility'
>> method for dereferencing objects from temps in
>> MethodContexts..  It has nothing to do with removing
>> anything from a database, just from image memory; I'm
>> actually not sure whether its needed anymore since I
>>discovered #fixTemps..

That explains why it cleaned out objects in my image that I couldn't
seem to get rid of.

>> I line 3 is wrapped in a commit, as in:

>>  myMagmaSession commit: [ myMagmaSession root
>> removeKey: 'SDRBackplane' ]

Of course, having a bit of a blond day.


>> Ah, ok.  So maybe the connectors should maintain their
>> state in the World at runtime..  Hard to know.

For the moment I'm happy I can work round that.

> So if you tell me how to delete objects reliably
> from Magma I think I might
> be there.

>> Dereferencing any sub-part of your object-model is a
>> matter of changing the reference to point to the nil
>> object (wrapped in commit, of course).

Everything cleaned so now I can start building.

 - Bob
*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.

Chris

Sorry, but I am going to have to ask for help again. I can now manage
the lifetimes in the database and image. Although it took me a while to
get the process repeatable and figure out where all the references were.
However, it now works and connectors stay connected. Unfortunately now
that I am really retrieving from the database a retrieved app does not
work. At least none of the UI events work, but then I am using dynamic
techniques to hook everything together. I have a taken a simple case
which is one composite component with one button on it. The button fires
a block which references self in this case. However, self appears to be
nil. This is where it fails.

DoIt
        | t1 t2 t3 t4 t5 |
        super initialize.
        t2 := ImageMorph new.
        t3 := GraphicPath , 'sdr1000.gif'.
        t2
                setNewImageFrom: (Form fromFileNamed: t3).
        t1 := SDRButton
                                newWithParams: 'Reset'
                                actionSelector: #(#RESETHW )
                                actionTarget: [:t6 | self buttonAction:
t6].
        self borderWidth: 2.
        self borderColor: #complexFramed.
        self color: Color gray.
        self addMorph: t2.
        self addMorph: t1.
        t4 := t2 extent.
        t5 := t1 extent.
        self extent: t4 x + t5 x + 8 @ (t4 y + 8)

MessageNotUnderstood Undefined object buttonAction. This method called
DoIt seems to be my Initialize method with the variables replaced with
generated names. GraphicPath refers to a pool dictionary which it also
can't resolve, don't know if that's relevant.

I have attached the file. The button is set up in the SDR1000Morph
Initialize. It is called through the block in the dictionary set up by
SDR1000Controller createServerInterfaceMap at #RESETHW.

I know this is a lot to ask. If it's not obvious what I need to do
please don't labour over it.

Thanks
Bob

*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.

Magma does support blocks and I just verified the
#testBlocks tests three different BlockContexts with
various characteristics, one includes a 'self'
reference.  So on the surface, it seems like it should
work.

Having said that, did you happen to ever save a change
to that #initialize method in the debugger at some
point?  MethodContexts on the stack referring to the
CompiledMethod being changed may undergo some
transformation.  An existing instance of that Morph
was persisted in the db; for one its MethodContext on
the stack at the time is suddenly not pointing to the
CompiledMethod installed in the class.

So you could try to resave the #initialize method and
re-commit all your SDR1000Morphs to the db.  That
might get it working, but there is still the main
caveat with storing BlockContext and MethodContexts in
the database for "long-term behavior" such as this;
they become stale as soon as the code changes because
they refer to the old CompiledMethod object, not the
current one.  This could cause some real confusion
much later if you have lots of stored instances not
behaving as the current method source indicates; they
are running the old CompiledMethod which now executes
in a DoIt Context I think.

This is such as simple block anyway that you may want
to consider using a MessageSend instead.  This is a
standard object that can be used instead of the Block.
 Try changing your assignment to resetButton to
something like this:

        resetButton :=
                (SDRButton
                        newWithParams: 'Reset'
                        actionSelector: #(#RESETHW)
                        actionTarget:
                                (MessageSend
                                        receiver: self
                                        selector: #buttonAction:).

Now, I see MessageSend answers to #value and
#valueWithArguments: but I don't see #value: which may
be required.  So you may need to add
MessageSend>>#value:.

value: anObject
  ^ self valueWithArguments: { anObject }

This might help solve your problem and you might be
happier with a MessageSend in the long run anyway
because it will stay "current" with the code.

Good luck,
  Chris


--- "Cowdery, Bob [UK]" <[hidden email]>
wrote:

Chris

Thanks very much for the response.

> Magma does support blocks and I just verified the
> #testBlocks tests three different BlockContexts with
> various characteristics, one includes a 'self'
> reference.  So on the surface, it seems like it should
> work.

I must be doing something a little off-track somewhere.

> Having said that, did you happen to ever save a change
> to that #initialize method in the debugger at some
> point?  MethodContexts on the stack referring to the
> CompiledMethod being changed may undergo some
> transformation.  An existing instance of that Morph
> was persisted in the db; for one its MethodContext on
> the stack at the time is suddenly not pointing to the
> CompiledMethod installed in the class.

That didn't make any difference. I thought it wouldn't because it's a fresh image and it happens on all the interactors on all components.

> So you could try to resave the #initialize method and
> re-commit all your SDR1000Morphs to the db.  That
> might get it working, but there is still the main
> caveat with storing BlockContext and MethodContexts in
> the database for "long-term behavior" such as this;
> they become stale as soon as the code changes because
> they refer to the old CompiledMethod object, not the
> current one.  This could cause some real confusion
> much later if you have lots of stored instances not
> behaving as the current method source indicates; they
> are running the old CompiledMethod which now executes
> in a DoIt Context I think.

Thanks, yes I understand the problem. 

> This is such as simple block anyway that you may want
> to consider using a MessageSend instead.  This is a
> standard object that can be used instead of the Block.
>  Try changing your assignment to resetButton to
> something like this:
>
>       resetButton :=
>               (SDRButton
>                       newWithParams: 'Reset'
>                       actionSelector: #(#RESETHW)
>                       actionTarget:
>                               (MessageSend
>                                       receiver: self
>                                       selector: #buttonAction:).
>
> Now, I see MessageSend answers to #value and
> #valueWithArguments: but I don't see #value: which may
> be required.  So you may need to add
> MessageSend>>#value:.

> value: anObject
>   ^ self valueWithArguments: { anObject }

> This might help solve your problem and you might be
> happier with a MessageSend in the long run anyway
> because it will stay "current" with the code.

Ok, I will think about doing that. There are two blocks involved here though as there is a level of indirection. The first one above calls a method which looks up #RESETHW in the server dictionary to get another block which calls the target method. In this simple case the target method is also in this component but in most cases it is on one or more of the connected components, the other end of the connector link (i.e. when the link is connected the maps get exchanged and in this case of course self is the target). Do you think I'm flogging a dead horse here? I will fabricate a simple case with message send and see if that works.

Thanks
Bob

--- "Cowdery, Bob [UK]" <[hidden email]>
wrote:

>
> Chris
>
> Sorry, but I am going to have to ask for help again.
> I can now manage
> the lifetimes in the database and image. Although it
> took me a while to
> get the process repeatable and figure out where all
> the references were.
> However, it now works and connectors stay connected.
> Unfortunately now
> that I am really retrieving from the database a
> retrieved app does not
> work. At least none of the UI events work, but then
> I am using dynamic
> techniques to hook everything together. I have a
> taken a simple case
> which is one composite component with one button on
> it. The button fires
> a block which references self in this case. However,
> self appears to be
> nil. This is where it fails.
>
> DoIt
>       | t1 t2 t3 t4 t5 |
>       super initialize.
>       t2 := ImageMorph new.
>       t3 := GraphicPath , 'sdr1000.gif'.
>       t2
>               setNewImageFrom: (Form fromFileNamed: t3).
>       t1 := SDRButton
>                               newWithParams: 'Reset'
>                               actionSelector: #(#RESETHW )
>                               actionTarget: [:t6 | self buttonAction:
> t6].
>       self borderWidth: 2.
>       self borderColor: #complexFramed.
>       self color: Color gray.
>       self addMorph: t2.
>       self addMorph: t1.
>       t4 := t2 extent.
>       t5 := t1 extent.
>       self extent: t4 x + t5 x + 8 @ (t4 y + 8)
>
> MessageNotUnderstood Undefined object buttonAction.
> This method called
> DoIt seems to be my Initialize method with the
> variables replaced with
> generated names. GraphicPath refers to a pool
> dictionary which it also
> can't resolve, don't know if that's relevant.
>
> I have attached the file. The button is set up in
> the SDR1000Morph
> Initialize. It is called through the block in the
> dictionary set up by
> SDR1000Controller createServerInterfaceMap at
> #RESETHW.
>
> I know this is a lot to ask. If it's not obvious
> what I need to do
> please don't labour over it.
>
> Thanks
> Bob
>
> *** Confidentiality Notice ***
> Proprietary/Confidential
> Information belonging to CGI Group Inc. and its
> affiliates
> may be contained in this message. If you are not a
> recipient
> indicated or intended in this message (or
> responsible for
> delivery of this message to such person), or you
> think for
> any reason that this message may have been addressed
> to you
> in error, you may not use or copy or deliver this
> message
> to anyone else.  In such case, you should destroy
> this
> message and are asked to notify the sender by reply
> email.
>

*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else.  In such case, you should destroy this
message and are asked to notify the sender by reply email.