Tracing a Spur Image from Smalltalk
Locked
Tracing a Spur Image from Smalltalk
 
|
I've updated the SystemTracer package to trace a Spur-format image.
This works as far as I can tell, producing a header that looks just like the header produced by the VM and the objects written also look good. I can also open the image in RSqueak, but not on Spur. It crashes immediately and produces a Smalltalk backtrace that looks like it never manages to return from the image-saving method (but the names in the Stack-trace all look ok, leading me to believe that enough objects got written ok to figure out classes, method names, find the right Process to run and so forth). Curiously, the old SystemTracer2 class that writes Cog-format images also produces images that cannot be opened by Cog VMs, only Interpreter VMs. So I assume there are other assumptions that maybe the JIT makes when starting up. So my question is, what assumptions could there be, and where could I start looking. Cheers, Tim (If you're wondering why I'm using the Smalltalk-level tracing at all - in our quest to create (with RSqueakVM) a Squeak VM that runs Squeak code fast enough that we no longer have to rely on C code from plugins or optional primitives, we're trying to see how far we can push this, and e.g. write the image from within itself, too.) |
Re: Tracing a Spur Image from Smalltalk
|
|
> On 26-02-2016, at 7:59 AM, timfelgentreff <[hidden email]> wrote: > > > I've updated the SystemTracer package to trace a Spur-format image. I’m impressed that you were able to revive the tracer enough to do anything with it; it’s at least 15 years since I last whacked at it and I hadn’t noticed anything much changing since then. Maybe someday I’ll get around to returning to using it to trace out specialised images with only wanted parts… probably shortly after I invent FTL travel. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Useful random insult:- Has a pulse, but that's about all. |
Re: Tracing a Spur Image from Smalltalk
|
|
In reply to this post by timfelgentreff
Hi Tim, > On Feb 26, 2016, at 7:59 AM, timfelgentreff <[hidden email]> wrote: > > > I've updated the SystemTracer package to trace a Spur-format image. Woo hoo! > This works as far as I can tell, producing a header that looks just like the > header produced by the VM and the objects written also look good. I can also > open the image in RSqueak, but not on Spur. It crashes immediately and > produces a Smalltalk backtrace that looks like it never manages to return > from the image-saving method (but the names in the Stack-trace all look ok, > leading me to believe that enough objects got written ok to figure out > classes, method names, find the right Process to run and so forth). Have you tried opening it in the simulator? Also you can use the leak checker in both the simulator and in an assert VM. But that you can run the traced image in RSqueak argues against it being a mist raced heap. > > Curiously, the old SystemTracer2 class that writes Cog-format images also > produces images that cannot be opened by Cog VMs, only Interpreter VMs. So I > assume there are other assumptions that maybe the JIT makes when starting > up. > > So my question is, what assumptions could there be, and where could I start > looking. The things to do are to use the simulator and an assert VM because many of the invariants upon which the VMs depend are checked by asserts. Likely you'll find an assert failure that will point to any issues immediately. Tim, if your image is available I'd love to download it. > > Cheers, > Tim > > (If you're wondering why I'm using the Smalltalk-level tracing at all - in > our quest to create (with RSqueakVM) a Squeak VM that runs Squeak code fast > enough that we no longer have to rely on C code from plugins or optional > primitives, we're trying to see how far we can push this, and e.g. write the > image from within itself, too.) And I hope that Sista will help you get there. BTW, if you have spare muscle in your team Sour could really do with a new compactor. Clément has a design in his head but, like me, no time. I guess you're in the same state but I thought I'd ask just in case. > > > > -- > View this message in context: http://forum.world.st/Tracing-a-Spur-Image-from-Smalltalk-tp4881224.html > Sent from the Squeak VM mailing list archive at Nabble.com. |
Re: Tracing a Spur Image from Smalltalk
|
|
In reply to this post by timfelgentreff
That is very good news! The Cuis folks will be happy to hear about this also, as there has been recent discussion on the Cuis list about how to do a transition to Spur format. If I recall right, the issue with tracing and resuming on Cog had to do with resuming in the middle of a method of a jitted method. I don't think that I ever positively confirmed that theory, but things worked fine under an interpreter and it was a trivial step to re-save from Cog. Dave > > I've updated the SystemTracer package to trace a Spur-format image. > > This works as far as I can tell, producing a header that looks just like > the > header produced by the VM and the objects written also look good. I can > also > open the image in RSqueak, but not on Spur. It crashes immediately and > produces a Smalltalk backtrace that looks like it never manages to return > from the image-saving method (but the names in the Stack-trace all look > ok, > leading me to believe that enough objects got written ok to figure out > classes, method names, find the right Process to run and so forth). > > Curiously, the old SystemTracer2 class that writes Cog-format images also > produces images that cannot be opened by Cog VMs, only Interpreter VMs. So > I > assume there are other assumptions that maybe the JIT makes when starting > up. > > So my question is, what assumptions could there be, and where could I > start > looking. > > Cheers, > Tim > > (If you're wondering why I'm using the Smalltalk-level tracing at all - in > our quest to create (with RSqueakVM) a Squeak VM that runs Squeak code > fast > enough that we no longer have to rely on C code from plugins or optional > primitives, we're trying to see how far we can push this, and e.g. write > the > image from within itself, too.) > > > > -- > View this message in context: > http://forum.world.st/Tracing-a-Spur-Image-from-Smalltalk-tp4881224.html > Sent from the Squeak VM mailing list archive at Nabble.com. > |
Re: Tracing a Spur Image from Smalltalk
|
|
In reply to this post by timrowledge
Hi Tim (Rowledge), The system tracer has been working for quite a few years (http://www.squeaksource.com/SystemTracing). That is how the 64-bit format 68002 images are done: http://build.squeak.org/job/Squeak%2064-bit%20image/ I turned that job off because the upstream job was no longer maintained, but it still works fine. Dave > > >> On 26-02-2016, at 7:59 AM, timfelgentreff <[hidden email]> >> wrote: >> >> >> I've updated the SystemTracer package to trace a Spur-format image. > > Iâm impressed that you were able to revive the tracer enough to do > anything with it; itâs at least 15 years since I last whacked at it and > I hadnât noticed anything much changing since then. Maybe someday > Iâll get around to returning to using it to trace out specialised images > with only wanted parts⦠probably shortly after I invent FTL travel. > > tim > -- > tim Rowledge; [hidden email]; http://www.rowledge.org/tim > Useful random insult:- Has a pulse, but that's about all. > > |
Re: Tracing a Spur Image from Smalltalk
|
|
In reply to this post by David T. Lewis
On Fri, Feb 26, 2016 at 11:27 AM, David T. Lewis <[hidden email]> wrote:
Alas the tracer is unlikely to work as one needs to add extra methods. I have been remiss in helping the Cuis folks transition. I have been distracted with 64-bit work. The situation right now is that a script that installs the revised Float (SmallFloat64 & BoxedFloat64) hierarchy needs rewriting for Cuis and I haven't got round to it. Forgive me.
At least from the image level the difference in e.g. pc in the active context shouldn't be visible. If it is then there's a bug in Cog.
_,,,^..^,,,_ best, Eliot |
Re: Tracing a Spur Image from Smalltalk
|
|
In reply to this post by Eliot Miranda-2
Hi,
by parallel-stepping through the simulator on the traced image and a normal one I found two issues, the first relating to the size of the hidden roots object (was 4096 in my code, apparently needs a few extra slots) and the size of the free lists object (which I left empty, because we're ignoring it in RSqueakVM). After fixing those things and adding an ugly workaround for a file-prim failure post startup, I now get working Spur images out of the tracing process :) Would it make sense to move the SystemTracing package into the VMMaker repo? If not, David, can you give me access to the SystemTracing repo so I can push my version there? I did everything in a new subclass of SystemTracer2 (minus some method extraction refactorings), so the old code for tracing pre-Spur images and the even older version without BlockContexts also works. cheers, Tim
|
Re: Tracing a Spur Image from Smalltalk
|
|
> On Feb 26, 2016, at 1:25 PM, timfelgentreff <[hidden email]> wrote: > > > Hi, > > by parallel-stepping through the simulator on the traced image and a normal > one I found two issues, the first relating to the size of the hidden roots > object (was 4096 in my code, apparently needs a few extra slots) and the > size of the free lists object (which I left empty, because we're ignoring it > in RSqueakVM). After fixing those things and adding an ugly workaround for a > file-prim failure post startup, I now get working Spur images out of the > tracing process :) Congratulations Tim, and thanks! > Would it make sense to move the SystemTracing package into the VMMaker repo? > If not, David, can you give me access to the SystemTracing repo so I can > push my version there? I did everything in a new subclass of SystemTracer2 > (minus some method extraction refactorings), so the old code for tracing > pre-Spur images and the even older version without BlockContexts also works. Nice! Do you select the right tracer based on a vmParameterAt: invocation, or...? > cheers, > Tim > > > Eliot Miranda-2 wrote >> Hi Tim, >> >>> On Feb 26, 2016, at 7:59 AM, timfelgentreff < > >> timfelgentreff@ > >> > wrote: >>> >>> >>> I've updated the SystemTracer package to trace a Spur-format image. >> >> Woo hoo! >> >>> This works as far as I can tell, producing a header that looks just like >>> the >>> header produced by the VM and the objects written also look good. I can >>> also >>> open the image in RSqueak, but not on Spur. It crashes immediately and >>> produces a Smalltalk backtrace that looks like it never manages to return >>> from the image-saving method (but the names in the Stack-trace all look >>> ok, >>> leading me to believe that enough objects got written ok to figure out >>> classes, method names, find the right Process to run and so forth). >> >> Have you tried opening it in the simulator? Also you can use the leak >> checker in both the simulator and in an assert VM. But that you can run >> the traced image in RSqueak argues against it being a mist raced heap. >> >>> >>> Curiously, the old SystemTracer2 class that writes Cog-format images also >>> produces images that cannot be opened by Cog VMs, only Interpreter VMs. >>> So I >>> assume there are other assumptions that maybe the JIT makes when starting >>> up. >>> >>> So my question is, what assumptions could there be, and where could I >>> start >>> looking. >> >> The things to do are to use the simulator and an assert VM because many of >> the invariants upon which the VMs depend are checked by asserts. Likely >> you'll find an assert failure that will point to any issues immediately. >> >> Tim, if your image is available I'd love to download it. >> >> >>> >>> Cheers, >>> Tim >>> >>> (If you're wondering why I'm using the Smalltalk-level tracing at all - >>> in >>> our quest to create (with RSqueakVM) a Squeak VM that runs Squeak code >>> fast >>> enough that we no longer have to rely on C code from plugins or optional >>> primitives, we're trying to see how far we can push this, and e.g. write >>> the >>> image from within itself, too.) >> >> And I hope that Sista will help you get there. >> >> BTW, if you have spare muscle in your team Sour could really do with a new >> compactor. Clément has a design in his head but, like me, no time. I >> guess you're in the same state but I thought I'd ask just in case. >> >>> >>> >>> >>> -- >>> View this message in context: >>> http://forum.world.st/Tracing-a-Spur-Image-from-Smalltalk-tp4881224.html >>> Sent from the Squeak VM mailing list archive at Nabble.com. > > > > > > -- > View this message in context: http://forum.world.st/Tracing-a-Spur-Image-from-Smalltalk-tp4881224p4881300.html > Sent from the Squeak VM mailing list archive at Nabble.com. |
Re: Tracing a Spur Image from Smalltalk
|
|
In reply to this post by timfelgentreff
Hi Tim, I just added you to the repository at http://www.squeaksource.com/SystemTracing. Please commit your updates there. Thanks a lot for doing this! Dave > > Hi, > > by parallel-stepping through the simulator on the traced image and a > normal > one I found two issues, the first relating to the size of the hidden roots > object (was 4096 in my code, apparently needs a few extra slots) and the > size of the free lists object (which I left empty, because we're ignoring > it > in RSqueakVM). After fixing those things and adding an ugly workaround for > a > file-prim failure post startup, I now get working Spur images out of the > tracing process :) > > Would it make sense to move the SystemTracing package into the VMMaker > repo? > If not, David, can you give me access to the SystemTracing repo so I can > push my version there? I did everything in a new subclass of SystemTracer2 > (minus some method extraction refactorings), so the old code for tracing > pre-Spur images and the even older version without BlockContexts also > works. > > cheers, > Tim > > > Eliot Miranda-2 wrote >> Hi Tim, >> >>> On Feb 26, 2016, at 7:59 AM, timfelgentreff < > >> timfelgentreff@ > >> > wrote: >>> >>> >>> I've updated the SystemTracer package to trace a Spur-format image. >> >> Woo hoo! >> >>> This works as far as I can tell, producing a header that looks just >>> like >>> the >>> header produced by the VM and the objects written also look good. I can >>> also >>> open the image in RSqueak, but not on Spur. It crashes immediately and >>> produces a Smalltalk backtrace that looks like it never manages to >>> return >>> from the image-saving method (but the names in the Stack-trace all look >>> ok, >>> leading me to believe that enough objects got written ok to figure out >>> classes, method names, find the right Process to run and so forth). >> >> Have you tried opening it in the simulator? Also you can use the leak >> checker in both the simulator and in an assert VM. But that you can run >> the traced image in RSqueak argues against it being a mist raced heap. >> >>> >>> Curiously, the old SystemTracer2 class that writes Cog-format images >>> also >>> produces images that cannot be opened by Cog VMs, only Interpreter VMs. >>> So I >>> assume there are other assumptions that maybe the JIT makes when >>> starting >>> up. >>> >>> So my question is, what assumptions could there be, and where could I >>> start >>> looking. >> >> The things to do are to use the simulator and an assert VM because many >> of >> the invariants upon which the VMs depend are checked by asserts. Likely >> you'll find an assert failure that will point to any issues immediately. >> >> Tim, if your image is available I'd love to download it. >> >> >>> >>> Cheers, >>> Tim >>> >>> (If you're wondering why I'm using the Smalltalk-level tracing at all - >>> in >>> our quest to create (with RSqueakVM) a Squeak VM that runs Squeak code >>> fast >>> enough that we no longer have to rely on C code from plugins or >>> optional >>> primitives, we're trying to see how far we can push this, and e.g. >>> write >>> the >>> image from within itself, too.) >> >> And I hope that Sista will help you get there. >> >> BTW, if you have spare muscle in your team Sour could really do with a >> new >> compactor. Clément has a design in his head but, like me, no time. I >> guess you're in the same state but I thought I'd ask just in case. >> >>> >>> >>> >>> -- >>> View this message in context: >>> http://forum.world.st/Tracing-a-Spur-Image-from-Smalltalk-tp4881224.html >>> Sent from the Squeak VM mailing list archive at Nabble.com. > > > > > > -- > View this message in context: > http://forum.world.st/Tracing-a-Spur-Image-from-Smalltalk-tp4881224p4881300.html > Sent from the Squeak VM mailing list archive at Nabble.com. > |
Need a good comment for SystemTracerSpur (was: Tracing a Spur Image from Smalltalk)
|
|
In reply to this post by timfelgentreff
Hi Tim, The latest version of your system tracer for Spur (on squeaksource) is in SystemTracing-tfel.41. The SystemTracerSpur class would benefit from a class comment to explain with it can (and cannot) do. Would you mind adding a comment to clarify? My understanding is that SystemTracerSpur traces a 32-bit V3 image to 32-bit Spur, although it is not clear if the traced image can then be run under a Spur VM (I suspect not, but I am not sure). We also have 64-bit Spur images available, and these will require tracing support either with SystemTracer, or using the VMMaker tracing that Eliot has already provided. So I think that this might get confusing soon, and a class comment will help to explain it. Thanks! Dave On Fri, Feb 26, 2016 at 07:59:04AM -0800, timfelgentreff wrote: > > I've updated the SystemTracer package to trace a Spur-format image. > > This works as far as I can tell, producing a header that looks just like the > header produced by the VM and the objects written also look good. I can also > open the image in RSqueak, but not on Spur. It crashes immediately and > produces a Smalltalk backtrace that looks like it never manages to return > from the image-saving method (but the names in the Stack-trace all look ok, > leading me to believe that enough objects got written ok to figure out > classes, method names, find the right Process to run and so forth). > > Curiously, the old SystemTracer2 class that writes Cog-format images also > produces images that cannot be opened by Cog VMs, only Interpreter VMs. So I > assume there are other assumptions that maybe the JIT makes when starting > up. > > So my question is, what assumptions could there be, and where could I start > looking. > > Cheers, > Tim > > (If you're wondering why I'm using the Smalltalk-level tracing at all - in > our quest to create (with RSqueakVM) a Squeak VM that runs Squeak code fast > enough that we no longer have to rely on C code from plugins or optional > primitives, we're trying to see how far we can push this, and e.g. write the > image from within itself, too.) > > > > -- > View this message in context: http://forum.world.st/Tracing-a-Spur-Image-from-Smalltalk-tp4881224.html > Sent from the Squeak VM mailing list archive at Nabble.com. |
Re: Need a good comment for SystemTracerSpur (was: Tracing a Spur Image from Smalltalk)
|
|
Hi David,
sorry for dropping this and not having time to put more work into it. You're right, this will get confusing. Right now the Spur format tracer can trace Spur format images to Spur. Those will load on a Spur VM. I have (locally, or maybe I also pushed it) code to also trace V3 images to Spur, and those won't load on Spur VMs, but they do load on RSqueak, which was one of my use cases. Marcel Taeumel and I were discussing where to go with this and that it might be useful to have support through the SystemTracing package for cross-tracing images between different formats. Imagine that you are working on a 64-bit Sista-enabled VM on your x86 machine and you want to move to (maybe) and older 32-bit ARM system without a Sista VM or to SqueakJS which doesn't support Spur (yet): We'd like the "Save As.." dialog to offer the choice of word size and format as well, and use the SystemTracing package to convert between the formats. This might also be useful to convert the Etoys images into the newer format without losing the in-image content and then using RSqueak or something to do the necessary adjustments to get it to load on the Spur VMs. I will get back to this soon, I hope, but it will be a couple more weeks before I have time again. Is that alright to leave it be until then? Best, Tim
|
Re: Need a good comment for SystemTracerSpur (was: Tracing a Spur Image from Smalltalk)
|
|
Tim, Sure, of course it is ok to update it a few weeks from now. Thanks for the follow up! Dave > > Hi David, > > sorry for dropping this and not having time to put more work into it. > You're > right, this will get confusing. > > Right now the Spur format tracer can trace Spur format images to Spur. > Those > will load on a Spur VM. I have (locally, or maybe I also pushed it) code > to > also trace V3 images to Spur, and those won't load on Spur VMs, but they > do > load on RSqueak, which was one of my use cases. > > Marcel Taeumel and I were discussing where to go with this and that it > might > be useful to have support through the SystemTracing package for > cross-tracing images between different formats. Imagine that you are > working > on a 64-bit Sista-enabled VM on your x86 machine and you want to move to > (maybe) and older 32-bit ARM system without a Sista VM or to SqueakJS > which > doesn't support Spur (yet): We'd like the "Save As.." dialog to offer the > choice of word size and format as well, and use the SystemTracing package > to > convert between the formats. This might also be useful to convert the > Etoys > images into the newer format without losing the in-image content and then > using RSqueak or something to do the necessary adjustments to get it to > load > on the Spur VMs. > > I will get back to this soon, I hope, but it will be a couple more weeks > before I have time again. Is that alright to leave it be until then? > > Best, > Tim > > > David T. Lewis wrote >> Hi Tim, >> >> The latest version of your system tracer for Spur (on squeaksource) is >> in SystemTracing-tfel.41. The SystemTracerSpur class would benefit from >> a class comment to explain with it can (and cannot) do. Would you mind >> adding a comment to clarify? >> >> My understanding is that SystemTracerSpur traces a 32-bit V3 image to >> 32-bit Spur, although it is not clear if the traced image can then be >> run under a Spur VM (I suspect not, but I am not sure). We also have >> 64-bit Spur images available, and these will require tracing support >> either with SystemTracer, or using the VMMaker tracing that Eliot has >> already provided. So I think that this might get confusing soon, and >> a class comment will help to explain it. >> >> Thanks! >> >> Dave >> >> On Fri, Feb 26, 2016 at 07:59:04AM -0800, timfelgentreff wrote: >>> >>> I've updated the SystemTracer package to trace a Spur-format image. >>> >>> This works as far as I can tell, producing a header that looks just >>> like >>> the >>> header produced by the VM and the objects written also look good. I can >>> also >>> open the image in RSqueak, but not on Spur. It crashes immediately and >>> produces a Smalltalk backtrace that looks like it never manages to >>> return >>> from the image-saving method (but the names in the Stack-trace all look >>> ok, >>> leading me to believe that enough objects got written ok to figure out >>> classes, method names, find the right Process to run and so forth). >>> >>> Curiously, the old SystemTracer2 class that writes Cog-format images >>> also >>> produces images that cannot be opened by Cog VMs, only Interpreter VMs. >>> So I >>> assume there are other assumptions that maybe the JIT makes when >>> starting >>> up. >>> >>> So my question is, what assumptions could there be, and where could I >>> start >>> looking. >>> >>> Cheers, >>> Tim >>> >>> (If you're wondering why I'm using the Smalltalk-level tracing at all - >>> in >>> our quest to create (with RSqueakVM) a Squeak VM that runs Squeak code >>> fast >>> enough that we no longer have to rely on C code from plugins or >>> optional >>> primitives, we're trying to see how far we can push this, and e.g. >>> write >>> the >>> image from within itself, too.) >>> >>> >>> >>> -- >>> View this message in context: >>> http://forum.world.st/Tracing-a-Spur-Image-from-Smalltalk-tp4881224.html >>> Sent from the Squeak VM mailing list archive at Nabble.com. > > > > > > -- > View this message in context: > http://forum.world.st/Tracing-a-Spur-Image-from-Smalltalk-tp4881224p4899020.html > Sent from the Squeak VM mailing list archive at Nabble.com. > |
«
Return to Squeak VM
|
1 view|%1 views
| Free forum by Nabble | Edit this page |