[Glass] sixx + XmlPullParser in 3.1?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[Glass] sixx + XmlPullParser in 3.1?

Paul DeBruicker
Does anyone have a working copy of SIXX using the XMLPullParser in 3.1?
 If not I'll work on that soonish
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] sixx + XmlPullParser in 3.1?

Mariano Martinez Peck
Uhhh I will be very interested in this as well. Not only in GemStone but also in Pharo ;)
I found that Johan started to work in a port of the XMLPullParser for GemStone: https://github.com/glassdb/XMLPullParser
But we still miss the SIXX integration. I was reading http://kentreis.wordpress.com/2009/06/01/deep-sixx-with-xml-pull-parser/
And..from the squeak repo from there: http://www.squeaksource.com/XMLPullParser.html  
It seems the adapted/modified SIXX is in that repo. See SIXX-kdt.143. I did a diff against SIXX-mu.141. There are some changes....
I was browsing BaselineOfSixx, and I found SIXX.310-PaulDeBruicker.182. So seems like we are 40 versions ahead? 

 best, 


On Sat, Sep 21, 2013 at 12:51 PM, Paul DeBruicker <[hidden email]> wrote:
Does anyone have a working copy of SIXX using the XMLPullParser in 3.1?
 If not I'll work on that soonish
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass



--
Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] sixx + XmlPullParser in 3.1?

Johan Brichau-3

On 28 Jul 2014, at 15:03, Mariano Martinez Peck <[hidden email]> wrote:

> I found that Johan started to work in a port of the XMLPullParser for GemStone: https://github.com/glassdb/XMLPullParser

The XMLPullParser port is fully working (actually using it in production for a couple of months already), but I think I need to sync with the latest changes of smalltalkhub...

Johan
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] sixx + XmlPullParser in 3.1?

Paul DeBruicker
In reply to this post by Mariano Martinez Peck
I tried to sort the XMLPullParser out for 3.1 but didn't have the right time/brain horsepower combination to figure it out but was able to use STON to transfer the objects from Pharo to GemStone.




Mariano Martinez Peck wrote
Uhhh I will be very interested in this as well. Not only in GemStone but
also in Pharo ;)
I found that Johan started to work in a port of the XMLPullParser for
GemStone: https://github.com/glassdb/XMLPullParser
But we still miss the SIXX integration. I was reading
http://kentreis.wordpress.com/2009/06/01/deep-sixx-with-xml-pull-parser/
And..from the squeak repo from there:
http://www.squeaksource.com/XMLPullParser.html
It seems the adapted/modified SIXX is in that repo. See SIXX-kdt.143. I did
a diff against SIXX-mu.141. There are some changes....
I was browsing BaselineOfSixx, and I found SIXX.310-PaulDeBruicker.182. So
seems like we are 40 versions ahead?

 best,


On Sat, Sep 21, 2013 at 12:51 PM, Paul DeBruicker <[hidden email]>
wrote:

> Does anyone have a working copy of SIXX using the XMLPullParser in 3.1?
>  If not I'll work on that soonish
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>



--
Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] sixx + XmlPullParser in 3.1?

GLASS mailing list


On Tue, Jul 29, 2014 at 3:05 AM, Paul DeBruicker <[hidden email]> wrote:
I tried to sort the XMLPullParser out for 3.1 but didn't have the right
time/brain horsepower combination to figure it out but was able to use STON
to transfer the objects from Pharo to GemStone.


Hi Paul, is that code available somewhere?  (I know where it is STON) but I wonder about all the intercommuication between pharo and gemstone.
Does ston load cleanly in Gemstone?  Did you need to modify code in either side?
 
Did you see that the SIXX serializer has a large problem with memory consumption and the stack limit size in Gemstone? this is mainly because of the type of XML parser used.. But anyway...do you know if the STON parser would have this kind of problem too? In other words....if I am already facing memory/stack limits of SIXX, do you think STON would at least give me some air? Note that I don't have much of a problem that it takes time. 

To be honest, I would prefer ston rather than XML either way.. but sixx seem to have been used quite a lot in Gemstone-Pharo. I am not sure in Gemstone how it would work. 

Anyway....I have large graphs to check. So if you give me some hints I can experiment as well and share my results. 

Thanks, 
 




Mariano Martinez Peck wrote
> Uhhh I will be very interested in this as well. Not only in GemStone but
> also in Pharo ;)
> I found that Johan started to work in a port of the XMLPullParser for
> GemStone: https://github.com/glassdb/XMLPullParser
> But we still miss the SIXX integration. I was reading
> http://kentreis.wordpress.com/2009/06/01/deep-sixx-with-xml-pull-parser/
> And..from the squeak repo from there:
> http://www.squeaksource.com/XMLPullParser.html
> It seems the adapted/modified SIXX is in that repo. See SIXX-kdt.143. I
> did
> a diff against SIXX-mu.141. There are some changes....
> I was browsing BaselineOfSixx, and I found SIXX.310-PaulDeBruicker.182. So
> seems like we are 40 versions ahead?
>
>  best,
>
>
> On Sat, Sep 21, 2013 at 12:51 PM, Paul DeBruicker &lt;

> pdebruic@

> &gt;
> wrote:
>
>> Does anyone have a working copy of SIXX using the XMLPullParser in 3.1?
>>  If not I'll work on that soonish
>> _______________________________________________
>> Glass mailing list
>>

> Glass@.gemtalksystems

>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
> _______________________________________________
> Glass mailing list

> Glass@.gemtalksystems

> http://lists.gemtalksystems.com/mailman/listinfo/glass





--
View this message in context: http://forum.world.st/Glass-sixx-XmlPullParser-in-3-1-tp4709520p4770789.html
Sent from the GLASS mailing list archive at Nabble.com.
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass



--
Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] sixx + XmlPullParser in 3.1?

Richard Sargent
Administrator
GLASS mailing list wrote
Hi Paul, is that code available somewhere?  (I know where it is STON) but I
wonder about all the intercommuication between pharo and gemstone.
Does ston load cleanly in Gemstone?  Did you need to modify code in either
side?

...

To be honest, I would prefer ston rather than XML either way.. but sixx
seem to have been used quite a lot in Gemstone-Pharo. I am not sure in
Gemstone how it would work.
Mariano,

Dale uses STON for all data exchange between GemStone and Pharo in tODE, so I think it is safe to say that there is a viable solution for you.


Richard
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] sixx + XmlPullParser in 3.1?

Paul DeBruicker
In reply to this post by GLASS mailing list
Hi Mariano,

I just exported the graph to a STON file and then read it in from GemStone.  It was a 3.5MB ston file once I finished getting the serialization/materialization right.  

My intuition is that STON is a more compact format than SIXX, but I have no proof.  Maybe serialize a portion of your graph and compare file sizes.  

I wish I had a better answer or better tooling for you.


Paul




GLASS mailing list wrote
On Tue, Jul 29, 2014 at 3:05 AM, Paul DeBruicker <[hidden email]> wrote:

> I tried to sort the XMLPullParser out for 3.1 but didn't have the right
> time/brain horsepower combination to figure it out but was able to use STON
> to transfer the objects from Pharo to GemStone.
>
>
Hi Paul, is that code available somewhere?  (I know where it is STON) but I
wonder about all the intercommuication between pharo and gemstone.
Does ston load cleanly in Gemstone?  Did you need to modify code in either
side?

Did you see that the SIXX serializer has a large problem with memory
consumption and the stack limit size in Gemstone? this is mainly because of
the type of XML parser used.. But anyway...do you know if the STON parser
would have this kind of problem too? In other words....if I am already
facing memory/stack limits of SIXX, do you think STON would at least give
me some air? Note that I don't have much of a problem that it takes time.

To be honest, I would prefer ston rather than XML either way.. but sixx
seem to have been used quite a lot in Gemstone-Pharo. I am not sure in
Gemstone how it would work.

Anyway....I have large graphs to check. So if you give me some hints I can
experiment as well and share my results.

Thanks,


>
>
>
>
> Mariano Martinez Peck wrote
> > Uhhh I will be very interested in this as well. Not only in GemStone but
> > also in Pharo ;)
> > I found that Johan started to work in a port of the XMLPullParser for
> > GemStone: https://github.com/glassdb/XMLPullParser
> > But we still miss the SIXX integration. I was reading
> > http://kentreis.wordpress.com/2009/06/01/deep-sixx-with-xml-pull-parser/
> > And..from the squeak repo from there:
> > http://www.squeaksource.com/XMLPullParser.html
> > It seems the adapted/modified SIXX is in that repo. See SIXX-kdt.143. I
> > did
> > a diff against SIXX-mu.141. There are some changes....
> > I was browsing BaselineOfSixx, and I found SIXX.310-PaulDeBruicker.182.
> So
> > seems like we are 40 versions ahead?
> >
> >  best,
> >
> >
> > On Sat, Sep 21, 2013 at 12:51 PM, Paul DeBruicker <
>
> > pdebruic@
>
> > >
> > wrote:
> >
> >> Does anyone have a working copy of SIXX using the XMLPullParser in 3.1?
> >>  If not I'll work on that soonish
> >> _______________________________________________
> >> Glass mailing list
> >>
>
> > Glass@.gemtalksystems
>
> >> http://lists.gemtalksystems.com/mailman/listinfo/glass
> >>
> >
> >
> >
> > --
> > Mariano
> > http://marianopeck.wordpress.com
> >
> > _______________________________________________
> > Glass mailing list
>
> > Glass@.gemtalksystems
>
> > http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
>
>
>
> --
> View this message in context:
> http://forum.world.st/Glass-sixx-XmlPullParser-in-3-1-tp4709520p4770789.html
> Sent from the GLASS mailing list archive at Nabble.com.
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>



--
Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] sixx + XmlPullParser in 3.1?

GLASS mailing list
In reply to this post by Richard Sargent
Depending upon the structure that you are exporting STON may need work ... variable classes with instance variables are not handled cleanly by STON (see Text) ...

OTOH STON does handle intra-object references ... haven't paid attention to whether STON recurses through data structures or not which is the thing that gets you in trouble with the stack size ...

My plate is pretty full, but if I had some pointers as to the problems that you hit with the XML PULL parser I might be able to pitch in and get over the hump...


Dale

On Fri, Sep 26, 2014 at 1:58 PM, Richard Sargent via Glass <[hidden email]> wrote:
GLASS mailing list wrote
> Hi Paul, is that code available somewhere?  (I know where it is STON) but
> I
> wonder about all the intercommuication between pharo and gemstone.
> Does ston load cleanly in Gemstone?  Did you need to modify code in either
> side?
>
> ...
>
> To be honest, I would prefer ston rather than XML either way.. but sixx
> seem to have been used quite a lot in Gemstone-Pharo. I am not sure in
> Gemstone how it would work.

Mariano,

Dale uses STON for all data exchange between GemStone and Pharo in tODE, so
I think it is safe to say that there is a viable solution for you.


Richard



--
View this message in context: http://forum.world.st/Glass-sixx-XmlPullParser-in-3-1-tp4709520p4780794.html
Sent from the GLASS mailing list archive at Nabble.com.
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] sixx + XmlPullParser in 3.1?

GLASS mailing list
Johan's latest work on the XML-PULL parser is here[1] and all the tests are green[2] as of last week ...

Dale

[1] https://github.com/GsDevKit/XMLParser
[2] https://travis-ci.org/GsDevKit/XMLPullParser

On Fri, Sep 26, 2014 at 4:29 PM, Dale Henrichs <[hidden email]> wrote:
Depending upon the structure that you are exporting STON may need work ... variable classes with instance variables are not handled cleanly by STON (see Text) ...

OTOH STON does handle intra-object references ... haven't paid attention to whether STON recurses through data structures or not which is the thing that gets you in trouble with the stack size ...

My plate is pretty full, but if I had some pointers as to the problems that you hit with the XML PULL parser I might be able to pitch in and get over the hump...


Dale

On Fri, Sep 26, 2014 at 1:58 PM, Richard Sargent via Glass <[hidden email]> wrote:
GLASS mailing list wrote
> Hi Paul, is that code available somewhere?  (I know where it is STON) but
> I
> wonder about all the intercommuication between pharo and gemstone.
> Does ston load cleanly in Gemstone?  Did you need to modify code in either
> side?
>
> ...
>
> To be honest, I would prefer ston rather than XML either way.. but sixx
> seem to have been used quite a lot in Gemstone-Pharo. I am not sure in
> Gemstone how it would work.

Mariano,

Dale uses STON for all data exchange between GemStone and Pharo in tODE, so
I think it is safe to say that there is a viable solution for you.


Richard



--
View this message in context: http://forum.world.st/Glass-sixx-XmlPullParser-in-3-1-tp4709520p4780794.html
Sent from the GLASS mailing list archive at Nabble.com.
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass



_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] sixx + XmlPullParser in 3.1?

GLASS mailing list
oops the project for the XML-PULL parser is here[1]

[1] https://github.com/GsDevKit/XMLPullParser

On Fri, Sep 26, 2014 at 5:03 PM, Dale Henrichs <[hidden email]> wrote:
Johan's latest work on the XML-PULL parser is here[1] and all the tests are green[2] as of last week ...

Dale

[1] https://github.com/GsDevKit/XMLParser
[2] https://travis-ci.org/GsDevKit/XMLPullParser

On Fri, Sep 26, 2014 at 4:29 PM, Dale Henrichs <[hidden email]> wrote:
Depending upon the structure that you are exporting STON may need work ... variable classes with instance variables are not handled cleanly by STON (see Text) ...

OTOH STON does handle intra-object references ... haven't paid attention to whether STON recurses through data structures or not which is the thing that gets you in trouble with the stack size ...

My plate is pretty full, but if I had some pointers as to the problems that you hit with the XML PULL parser I might be able to pitch in and get over the hump...


Dale

On Fri, Sep 26, 2014 at 1:58 PM, Richard Sargent via Glass <[hidden email]> wrote:
GLASS mailing list wrote
> Hi Paul, is that code available somewhere?  (I know where it is STON) but
> I
> wonder about all the intercommuication between pharo and gemstone.
> Does ston load cleanly in Gemstone?  Did you need to modify code in either
> side?
>
> ...
>
> To be honest, I would prefer ston rather than XML either way.. but sixx
> seem to have been used quite a lot in Gemstone-Pharo. I am not sure in
> Gemstone how it would work.

Mariano,

Dale uses STON for all data exchange between GemStone and Pharo in tODE, so
I think it is safe to say that there is a viable solution for you.


Richard



--
View this message in context: http://forum.world.st/Glass-sixx-XmlPullParser-in-3-1-tp4709520p4780794.html
Sent from the GLASS mailing list archive at Nabble.com.
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] sixx + XmlPullParser in 3.1?

GLASS mailing list
Hey guys,

The XMLPullParser works fine but the test coverage is perhaps a bit small.
It’s also almost an exact copy of the smalltalkhub version for Pharo.
I’m using the one right before the latest in production on GS 2.4 without any issues.

My experiments with SIXX date back almost 4 years, and I used the ‘commit on almost out of memory’ trick at that time.
So I’m afraid I don’t know how to set up the pull parser with SIXX.

Johan

On 27 Sep 2014, at 02:08, Dale Henrichs via Glass <[hidden email]> wrote:

oops the project for the XML-PULL parser is here[1]

[1] https://github.com/GsDevKit/XMLPullParser

On Fri, Sep 26, 2014 at 5:03 PM, Dale Henrichs <[hidden email]> wrote:
Johan's latest work on the XML-PULL parser is here[1] and all the tests are green[2] as of last week ...

Dale

[1] https://github.com/GsDevKit/XMLParser
[2] https://travis-ci.org/GsDevKit/XMLPullParser

On Fri, Sep 26, 2014 at 4:29 PM, Dale Henrichs <[hidden email]> wrote:
Depending upon the structure that you are exporting STON may need work ... variable classes with instance variables are not handled cleanly by STON (see Text) ...

OTOH STON does handle intra-object references ... haven't paid attention to whether STON recurses through data structures or not which is the thing that gets you in trouble with the stack size ...

My plate is pretty full, but if I had some pointers as to the problems that you hit with the XML PULL parser I might be able to pitch in and get over the hump...


Dale

On Fri, Sep 26, 2014 at 1:58 PM, Richard Sargent via Glass <[hidden email]> wrote:
GLASS mailing list wrote
> Hi Paul, is that code available somewhere?  (I know where it is STON) but
> I
> wonder about all the intercommuication between pharo and gemstone.
> Does ston load cleanly in Gemstone?  Did you need to modify code in either
> side?
>
> ...
>
> To be honest, I would prefer ston rather than XML either way.. but sixx
> seem to have been used quite a lot in Gemstone-Pharo. I am not sure in
> Gemstone how it would work.

Mariano,

Dale uses STON for all data exchange between GemStone and Pharo in tODE, so
I think it is safe to say that there is a viable solution for you.


Richard



--
View this message in context: http://forum.world.st/Glass-sixx-XmlPullParser-in-3-1-tp4709520p4780794.html
Sent from the GLASS mailing list archive at Nabble.com.
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass



_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass