materializing a MaCommitLogRecord

I'm wondering if it is possible to directly materialize the objects in a
MaCommitLogRecord instance.

It seems that the commit log records are used exclusively for
"advancing" the repository.  However I'm wondering if I can use the
commit log as a sort of poor-man's audit log and if I'd be able to
manually extract objects from specific commit log records.  Right now
when I try to materialize a record in Pharo 1.0, I get an error because
the serializer tries to call basicNew: on the MagmaLocalLocation class.

I'm not sure if this is even a good path to take, so fell free to tell
me I'm crazy or that I shouldn't go there :)

I'm also wondering if the commits could be "rolled back" so as to be
able to rewind the database back to a specific commit?  I see how one
can restore up to a certain point in time, but I'm just wondering if
there is any more specific way to "un-apply" the commit records.

--
Dave Woodward

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Hi Dave, thanks for the great questions.

On Fri, Jun 11, 2010 at 1:12 PM, Dave Woodward
<[hidden email]> wrote:
> I'm wondering if it is possible to directly materialize the objects in a
> MaCommitLogRecord instance.

You can materialize the CommitPackage which was applied for that
commit.  The CommitPackage contains the serialized first-class Buffers
which could be used to materialize the domain objects.

So, the domain objects are three-materializations removed from
existence.  These levels of abstraction are necessary for Magma to
handle HA functionality generically; i.e., it never has to deal with
specific classes of domain objects.

> It seems that the commit log records are used exclusively for "advancing"
> the repository.

That is their primary purpose, yes.

>  However I'm wondering if I can use the commit log as a sort
> of poor-man's audit log and if I'd be able to manually extract objects from
> specific commit log records.

Interesting.  Now, you probably noticed already that you have
easy-access to a Magma-provided timestamp and commitNumber for each
record.  Those would seem to be useful for an "audit log".

As far as getting at the objects, it is not difficult.  You would just
need to open the repository to establish the proper class-map, and
then use that serializer to materialize the "objects" instVar of the
MaCommitPackage contained within the MaCommitLogRecord.

>  Right now when I try to materialize a record
> in Pharo 1.0, I get an error because the serializer tries to call basicNew:
> on the MagmaLocalLocation class.
> I'm not sure if this is even a good path to take, so fell free to tell me
> I'm crazy or that I shouldn't go there :)

Not sure..  :)  Why are you inclined to do this?

> I'm also wondering if the commits could be "rolled back" so as to be able to
> rewind the database back to a specific commit?  I see how one can restore up
> to a certain point in time, but I'm just wondering if there is any more
> specific way to "un-apply" the commit records.

Only a partial commit can be rolled back, the last one, and Magma does
it automatically only if necessary.

Beyond that, Magma only supports rolling forward to a point in time.
Domain objects that are updated in the client are replaced "in-place"
on the hard-disk by the server.

Even if you never made a fullBackup, but you have all of the commit
log files from the beginning of time, you can apply them to the "seed"
repository that was automatically created and resides inside the magma
directory.  It is just one of several things Magma does to go out of
its way to protect the users persistent model (another is the SHA
verification of every single commit-record ever applied to any db).

Regards,
  Chris


>
> --
> Dave Woodward
>
> _______________________________________________
> Magma mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

On 6/12/10 11:39 PM, Chris Muller wrote:
> Interesting.  Now, you probably noticed already that you have
> easy-access to a Magma-provided timestamp and commitNumber for each
> record.  Those would seem to be useful for an "audit log".
>
> As far as getting at the objects, it is not difficult.  You would just
> need to open the repository to establish the proper class-map, and
> then use that serializer to materialize the "objects" instVar of the
> MaCommitPackage contained within the MaCommitLogRecord.
>    
Its not clear to me from the code how I access the commit package.  I
see that I can get the byte array from the record method of a
MaCommitRecord, but I can't seem to materialize the contents of that
array.  Is the commit package serialized inside that byte array or is
there another way to rebuild it?

I've tried using the same method calls that are used elsewhere in Magma
to materialize the contents of the commit record as below:

aMaObjectSerializer resetOidManager materializeGraph: aMaCommitRecord
record.

But that doesn't work (see my first e-mail for the error).
>>   Right now when I try to materialize a record
>> in Pharo 1.0, I get an error because the serializer tries to call basicNew:
>> on the MagmaLocalLocation class.
>> I'm not sure if this is even a good path to take, so fell free to tell me
>> I'm crazy or that I shouldn't go there :)
>>      
> Not sure..  :)  Why are you inclined to do this?
>    
Only because a feature requested for my app is "the ability to audit who
changed what... just in case."  It doesn't need to be an end-user
accessible feature, and it won't be used often.  Instead of me building
something new I realized that Magma has these commit records in the db
already... so I'm thinking I can just do something simple like record
which commit numbers a user is responsible for and then if I need to I
can go back later and extract the needed information from that commit.
> Regards,
>    Chris
>    
Thanks again for your help! And for Magma, it is great having an open
source database of this quality!

--
Dave Woodward
...................................................
www.promedmedical.net
317.332.6438
[hidden email]

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Nevermind, I figured it out.  I need to use MagmaSession
newCommitLogSerializer to materialize the graph in a commit record, not
the serializer from the current session.

However... now I'm not really sure what to make of the contents of the
buffers inside the serialized buffer of the commit package.  I don't see
any domain objects in there, just an array of oids, the timestamp and
something else that I can't seem to materialize.

--
Dave Woodward
...................................................
www.promedmedical.net
317.332.6438
[hidden email]

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

> Only because a feature requested for my app is "the ability to audit who
> changed what... just in case."  It doesn't need to be an end-user accessible
> feature, and it won't be used often.  Instead of me building something new I
> realized that Magma has these commit records in the db already... so I'm
> thinking I can just do something simple like record which commit numbers a
> user is responsible for and then if I need to I can go back later and
> extract the needed information from that commit.

If this is for Sarb'Ox then I think you made the right choice;
employing the database-level auditing information, rather putting it
in the app.  Plus, your approach not only captures the auditry about
the current state of all objects, but their entire historical updates
as well.  Excellent.

> Nevermind, I figured it out.  I need to use MagmaSession
> newCommitLogSerializer to materialize the graph in a commit record, not the
> serializer from the current session.

Yep, you found it, good hunting.  I don't know whether you'd found the
"Magma Tools" package but, in there, you'll find a tool called
"MagmaBufferController" which materializes CommitPackages in this
manner as well.  With the tool you can essentially browse the
persistent model without materializing any domain instances..

> However... now I'm not really sure what to make of the contents of the
> buffers inside the serialized buffer of the commit package.  I don't see any
> domain objects in there, just an array of oids, the timestamp and something
> else that I can't seem to materialize.

You could:

  mySession serializer independentCopy materializeObject:
theCommitPackage objects.

#materializeObject: is a private method but given your new
requirements and ideas, you would want a nice, small API for accessing
the objects out of a CommitLogRecord in this manner...

But, allow me to suggest that this may not be necessary.  You just
want to see the "contents" of the objects, you aren't as interested in
executing their behaviors, right?

If so, rather than trying to materialize an old portion of the model,
I would try that MagmaBufferController, et al, to do this job.  With
that tool, you get not only the *shape* of the object model, but also
easy-access to type, numeric, and string contents of objects, not to
mention the audit information as well, all from one single object
(MagmaBufferFacade).

Attempting to materialize a single CommitPackage gives you a "portion"
of an old-version of the model.  It can be done, but it's a bit more
work..

 - Chris
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

On 6/14/10 3:01 PM, Chris Muller wrote: We're a private company... but yeah, its basically a CYA feature. Yep.
> If so, rather than trying to materialize an old portion of the model,
> I would try that MagmaBufferController, et al, to do this job.  With
> that tool, you get not only the *shape* of the object model, but also
> easy-access to type, numeric, and string contents of objects, not to
> mention the audit information as well, all from one single object
> (MagmaBufferFacade).
>    
Ok, I loaded up the Magma Tools package and played with the
MagmaBufferController.  This is pretty cool.  I figured out how to
browse the contents of the buffers for a commit using the
"referencedBuffers" method and inspectors.  This is good enough for now.

I realize that its designed for use with Maui... and Maui doesn't load
in Pharo (I know its an old project).  Maybe I'll see if there is a way
to subclass Pharo's Inspector to make a primitive UI for the Buffer
Controller or something.

Or maybe I'll just load up the DB with my domain in a Squeak image where
all of these tools work :).  I've watched a video of Maui in action and
its pretty cool!
> Attempting to materialize a single CommitPackage gives you a "portion"
> of an old-version of the model.  It can be done, but it's a bit more
> work..
>
>   - Chris
>    
Thanks again for your help!

--
Dave Woodward

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma