Exploring the simulator (was Re: REPL image for simulation)
Locked
 
|
On Mon, May 23, 2016 at 3:14 AM, Eliot Miranda <[hidden email]> wrote: > >> On May 22, 2016, at 9:43 AM, Ben Coman <[hidden email]> wrote: >> >>> On Sat, May 21, 2016 at 4:44 AM, Clément Bera <[hidden email]> wrote: >>> >>>> On Fri, May 20, 2016 at 7:29 PM, Ben Coman <[hidden email]> wrote: >>>> >>>> On Fri, May 20, 2016 at 8:44 PM, Clément Bera <[hidden email]> wrote: >>> >>> Normally if you build a cog development image: >>> $ svn co http://www.squeakvm.org/svn/squeak/branches/Cog/image >>> $ cd ./image >>> $ ./buildsqueaktrunkvmmakerimage.sh >>> >>> You have multiple scripts available with comments to run the simulator that work out of the box. It should take a couple minutes to get it working. Then the easiest is to simulate a REPL image to easily debug what you want. >> >> Hi Clement, >> >> Could you point me to such a REPL image? >> >> cheers -ben > > Hi Ben, > > see buildspurreaderimage.sh, or it may be buildspurtrunkreaderimage.sh, in the image directory. Thanks Eliot. I now have an "Input please" window. I presume I should type Smalltalk expressions in there. For a while it was confusing that I could see no effect entering... 3+4 or... Transcript show:'hello' Nothing moving in the Simulator's trace pane, nor the Transcript within the simulation, nor the host's Transcript, but after selecting menu item "print instructions each instruction" I see 3+4 produce a stream of instructions in the host transcript. With such a long list of menu items, I expect now I'll bug everyone to discover what they all are. I'll use this thread to log random things I discover for posterity for myself and other VM newcomers. There doesn't seem a lot of info out there on the simulator and maybe some will spark some short comments, but I don't expect feedback on everything. But first, what are a few important menu or workflow items a VM newcomer working with the REPL image would find useful? cheers -ben P.S. After running ./buildspurtrunkreaderimage.sh this is the code I ran in the host image... | cos | cos := CogVMSimulator newWithOptions: #(Cogit StackToRegisterMappingCogit "SimpleStackBasedCogit" ObjectMemory Spur32BitCoMemoryManager "ISA ARMv5" "ISA IA32"). "cos initializeThreadSupport." cos desiredNumStackPages: 8. cos openOn: 'spurreader.image'. cos openAsMorph; run |
Re: Exploring the simulator (was Re: REPL image for simulation)
|
|
Hi Ben, Hopefully Chris Cunnigton was doing a tutorial video on the simulator to help. There is one blog post from Eliot on machine code simulation for x86 in his blog (Simulating out of the Bochs or something like that). The REPL image expects chunk format. Hence you need to write "3 + 4 !" Your script looks nice. I have 3 main workflows with the simulator: 1) in-image compilation: I change the way the JIT generate machine code, use in-image compilation to check the machine code generated is as I wanted, put halt in the JIT code and debug it as Smalltalk code while JIT compilation. For that I use the scripts in "In-image compilation" workspace. 2) Ensuring my VM changes are correct: I start the simulator and check there are no assertion failure / error before the display kicks in, sometimes I try to run some code too using my VM changes by writing it in the REPL . 3) Debugging a VM error: I start the simulator with a REPL image, then enter the code triggering the VM crash, and try to figure out what's going on. In the menu, there are in order: 1) toggle Transcript (toggle between simulator and external Transcript the output stream) clone VM (clone the simulator, to have guinea pigs to reproduce bugs, typically bugs hard to reproduce once you've reproduced them once or GC bugs) 2) things related to the stacks. 'printcallStack' is the main one which prints the current stack. 'print ext head frame' prints the current stack frame, very useful too. These 2 are the most useful. Other entries are situational. 3) 'printOop:' expects parameter in hex, printing an oop, if non immediate the header and all the fields. disassemble entries are very useful to disassemble where the VM has crashed or disassemble a method that looks suspicious based on its address. 4) inspect objectMemory or interpreter to look into the variables value, if crash in GC or interpreter Or run the leak checked to look for memory leaks. Or inspect cogit if a bug happened during JIT compilation Or inspect the method zone, typically useful to analyze it 5) print cog methods and trampoline (similar to disassemble, used for debugging the machine code) All the *break* things stop execution on a specific selector / pc / block / .. If single stepping is enabled (you need to do that only on small portion of code, the machine code gets dog slow), then you can report recent instructions to see the register state at each machine instructions, etc). To get warmed-up: 1) Inspect the object memory, then look for the first class table page instance variable. It's an oop referencing an array, try in the simulator to "printOop:" the address of the first class table page that you found. It should print it in the Transcript, the first entries are immediate, in Spur32 SmallInteger/Character/SmallInteger. 2) print the active stack, look for the method's address. Try to print it as an oop, and if it tells you "address in the machine code zone", print the cog method and its machine code instead. Have fun. I'll try to answer further questions as I like to see people hacking the VM, but I am quite busy :-) On Mon, May 23, 2016 at 6:06 PM, Ben Coman <[hidden email]> wrote:
|
|
|
Hi Clement, Thanks for your detailed reply. I particularly liked your warm up exercises. Goal directed learning is better than general browsing. On Tue, May 24, 2016 at 1:29 AM, Clément Bera <[hidden email]> wrote: > > Hi Ben, > > The REPL image expects chunk format. Hence you need to write "3 + 4 !" > > To get warmed-up: > 1) Inspect the object memory, then look for the first class table page instance variable. It's an oop referencing an array, try in the simulator to "printOop:" the address of the first class table page that you found. It should print it in the Transcript, the first entries are immediate, in Spur32 SmallInteger/Character/SmallInteger. The inspector showed a Spur32MMLECoSimulator and classTableFirstPage held 16r5311F8. Plugging that into [print oop...] showed > 2) print the active stack, look for the method's address. Try to print it as an oop, and if it tells you "address in the machine code zone", print the cog method and its machine code instead. > > Have fun. > > I'll try to answer further questions as I like to see people hacking the VM, but I am quite busy :-) > > > > On Mon, May 23, 2016 at 6:06 PM, Ben Coman <[hidden email]> wrote: >> >> >> On Mon, May 23, 2016 at 3:14 AM, Eliot Miranda <[hidden email]> wrote: >> > >> >> On May 22, 2016, at 9:43 AM, Ben Coman <[hidden email]> wrote: >> >> >> >>> On Sat, May 21, 2016 at 4:44 AM, Clément Bera <[hidden email]> wrote: >> >>> >> >>>> On Fri, May 20, 2016 at 7:29 PM, Ben Coman <[hidden email]> wrote: >> >>>> >> >>>> On Fri, May 20, 2016 at 8:44 PM, Clément Bera <[hidden email]> wrote: >> >>> >> >>> Normally if you build a cog development image: >> >>> $ svn co http://www.squeakvm.org/svn/squeak/branches/Cog/image >> >>> $ cd ./image >> >>> $ ./buildsqueaktrunkvmmakerimage.sh >> >>> >> >>> You have multiple scripts available with comments to run the simulator that work out of the box. It should take a couple minutes to get it working. Then the easiest is to simulate a REPL image to easily debug what you want. >> >> >> >> Hi Clement, >> >> >> >> Could you point me to such a REPL image? >> >> >> >> cheers -ben >> >> > >> > Hi Ben, >> > >> > see buildspurreaderimage.sh, or it may be buildspurtrunkreaderimage.sh, in the image directory. >> >> Thanks Eliot. I now have an "Input please" window. I presume I >> should type Smalltalk expressions in there. For a while it was >> confusing that I could see no effect entering... >> 3+4 >> or... >> Transcript show:'hello' >> >> Nothing moving in the Simulator's trace pane, nor the Transcript >> within the simulation, nor the host's Transcript, but after selecting >> menu item "print instructions each instruction" I see 3+4 produce a >> stream of instructions in the host transcript. >> >> With such a long list of menu items, I expect now I'll bug everyone to >> discover what they all are. I'll use this thread to log random things >> I discover for posterity for myself and other VM newcomers. There >> doesn't seem a lot of info out there on the simulator and maybe some >> will spark some short comments, but I don't expect feedback on >> everything. >> >> But first, what are a few important menu or workflow items a VM >> newcomer working with the REPL image would find useful? >> >> cheers -ben >> >> P.S. After running ./buildspurtrunkreaderimage.sh >> this is the code I ran in the host image... >> | cos | >> cos := CogVMSimulator newWithOptions: #(Cogit >> StackToRegisterMappingCogit "SimpleStackBasedCogit" >> ObjectMemory Spur32BitCoMemoryManager >> "ISA ARMv5" "ISA IA32"). >> "cos initializeThreadSupport." >> cos desiredNumStackPages: 8. >> cos openOn: 'spurreader.image'. >> cos openAsMorph; run > > > |
|
|
On Sun, May 29, 2016 at 10:14 AM, Ben Coman <[hidden email]> wrote: > Hi Clement, Thanks for your detailed reply. I particularly liked your > warm up exercises. Goal directed learning is better than general > browsing. > > On Tue, May 24, 2016 at 1:29 AM, Clément Bera <[hidden email]> wrote: >> >> Hi Ben, >> >> The REPL image expects chunk format. Hence you need to write "3 + 4 !" >> >> To get warmed-up: >> 1) Inspect the object memory, then look for the first class table page instance variable. It's an oop referencing an array, try in the simulator to "printOop:" the address of the first class table page that you found. It should print it in the Transcript, the first entries are immediate, in Spur32 SmallInteger/Character/SmallInteger. > > The inspector showed a Spur32MMLECoSimulator and classTableFirstPage > held 16r5311F8. Plugging that into [print oop...] showed... 16r5311F8: a(n) Array 16r52D108 nil 16r15C3A50 class SmallInteger 16r878D70 class Character 16r15C3A50 class SmallInteger 16r1111DC0 class SmallFloat64 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r87AE60 class Array 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r878C58 class LargeNegativeInteger 16r878C90 class LargePositiveInteger 16r10AEAE8 class BoxedFloat64 16r879438 class Message All the nils I guess are due to the class table being a hash map? Is there some way from within the simulation to reference an object by its hex number. For example, to use the size of that array from within the simulation, something like... classTableSize := 16r5311F8 objectFromHex size > > >> 2) print the active stack, look for the method's address. Try to print it as an oop, and if it tells you "address in the machine code zone", print the cog method and its machine code instead. I presume is the active stack is [print call stack] which produces... 16r1012F8 M MultiByteFileStream(StandardFileStream)>basicNext 16r1E7408: a(n) MultiByteFileStream 16r101334 M UTF8TextConverter>nextFromStream: 16r1EA418: a(n) UTF8TextConverter 16r10135C M MultiByteFileStream>next 16r1E7408: a(n) MultiByteFileStream 16r10138C I MultiByteFileStream(PositionableStream)>nextChunkNoTag 16r1E7408: a(n) MultiByteFileStream 16r1013B0 I StdioListener>run 16r1E7C98: a(n) StdioListener 16r1013D0 I [] in UndefinedObject>(nil) 16r52D108: a(n) UndefinedObject 16r1013F0 I [] in BlockClosure>newProcess 16r1E7E00: a(n) BlockClosure --------- [print oop...] 16r1012F8 tells me... 16r1012F8 is in the stack zone [print cog method for...] 16r1012F8 tells me... not a method [print mc/cog frame] says... Assertion failed with debugger at CogVMSimulatorLSD(CoInterpreter)>>isMachineCodeFrame: So I seem to be missing something. I restarted the simulator and this time... [print call stack...] 16r1012F8 M MultiByteFileStream(StandardFileStream)>basicNext 16r2A1BA8: a(n) MultiByteFileStream 16r101334 M UTF8TextConverter>nextFromStream: 16r2A2188: a(n) UTF8TextConverter 16r10135C M MultiByteFileStream>next 16r2A1BA8: a(n) MultiByteFileStream 16r10138C I MultiByteFileStream(PositionableStream)>nextChunkNoTag 16r2A1BA8: a(n) MultiByteFileStream 16r1013B0 I StdioListener>run 16r2A1B00: a(n) StdioListener 16r1013D0 I [] in UndefinedObject>(nil) 16r52D108: a(n) UndefinedObject 16r1013F0 I [] in BlockClosure>newProcess 16r2A1690: a(n) BlockClosure ---------- [print oop...] 16r1012F8 16r1012F8 is in the stack zone [print oop...] 16r2A1BA8 16r2A1BA8: a(n) MultiByteFileStream 16r2A2740 '????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????' 16r1 =0 (16r0) 16r1 =0 (16r0) 16r52D108 nil 16r52D108 nil 16r52D118 false 16r9AA1E8 #stdin 16r24CC18 a ByteArray 16r2A2728 '?' 16r52D108 nil 16r2A2188 an UTF8TextConverter 16r6DF1D8 #lf 16r52D128 true ----------- Now after doing 3+4! several times, [print call stack...] produces... 16r101300 M MultiByteFileStream(StandardFileStream)>basicNext 16r2A1BA8: a(n) MultiByteFileStream 16r10133C M UTF8TextConverter>nextFromStream: 16r2A2188: a(n) UTF8TextConverter 16r101364 M MultiByteFileStream>next 16r2A1BA8: a(n) MultiByteFileStream 16r10138C M MultiByteFileStream(PositionableStream)>nextChunkNoTag 16r2A1BA8: a(n) MultiByteFileStream 16r1013B0 I StdioListener>run 16r2A1B00: a(n) StdioListener 16r1013D0 I [] in UndefinedObject>(nil) 16r52D108: a(n) UndefinedObject 16r1013F0 I [] in BlockClosure>newProcess 16r2A1690: a(n) BlockClosure btw, What is the meaning of the M and I in the second column? I notice that 16r10138C has changed from an I to a M. Also the address associated with basicNext changed from 16r1012F8 to 16r101300. Can some meaning be inferred from that? cheers -ben |
Re: Exploring the simulator (was Re: REPL image for simulation)
|
|
Hi !
On Mon, May 30, 2016 at 2:39 PM, Ben Coman <[hidden email]> wrote:
You got the right result. The the class table is a linked list of pages, each page being an array. The first page, shown here, is reserved for frequently used classes. Indexes 0-15 are reserved for tagged object. Indexes 16-32 are reserved for hidden classes. Typically the class table pages are instances of Array, but the use index 16 so the VM know they are hidden. The rest is for real classes that are frequently used. There are many nils so we have free space for new features. It's not a hash map. I don't think things like that exists: classTableSize := 16r5311F8 objectFromHex size. For oops debugging features are tied to printing through the simulator instance right now. However there is something like that in the JIT. In the machine code zone we can access part of the bytes as CogMethodSurrogate and its subclasses and in the stack we can do the same for stack pages with the corresponding surrogate. In this case one can do something like: CogMethodSurrogate at: 16r51578 objectMemory: objectMemory cogit: cogit And then one can ask the surrogate things like: surrogate cmRefersToYoung And it reads the correct bytes for you, in this case answering if the cog method has a reference to a young object.
Some explanations are needed here :-) The M or I at the beginning of the printing are for 'Interpreted frame' or 'Machine code frame'. When you do [print call stack], you print the list of stack frame in the current stack. For example, 16r101300 M MultiByteFileStream(StandardFileStream)>basicNext means that: - the stack frame address in the stack zone is 16r101300 - the machine code version of the method is executed in this frame (M and not I).- the receiver has the type MultiByteFileStream - the stack frame on top of the stack is the activation for the method StandardFileStream>>basicNext Now what you tried to do is to print the frame as a method, and that won't work (It's not obvious and my exercise was not very precised, sorry). You can use [print frame ...] and put the frame's hex to print it. Alternatively, asyou usually want the top (a.k.a. head) frame, you can directly use [print ext head frame] if it's a machine code frame. That should print something like that (I print a random frame here): 16r103160: arg1: 16r8239 =16668(16r411C) 16r10315C: arg2: 16r825D =16686(16r412E) 16r103158: arg3: 16r2E21C0 =a(n) Point 16r103154: arg4: 16r9BC480 =a(n) StrikeFont 16r103150: arg5: 16r1 =0(16r0) 16r10314C: caller ip: 16r564E0=353504 16r103148: saved fp: 16r103184=1061252 16r103144: method: 16r51578 16r102BDD0 16r102BDD0: a(n) CompiledMethod 16r103144: mcfrm flags: 16r0 numArgs: 6 noContext notBlock 16r103140: context: 16r52D108 nil 16r10313C: receiver: 16r2E0078 a GrafPort 16r103138: stck: 16r2E0078 a GrafPort 16r103134: stck: 16r1A115D0 Now that you've print the frame, you can see the method addresses in this line: 16r103144: method: 16r51578 16r102BDD0 16r102BDD0: a(n) CompiledMethod. This is a machine code frame, so the method has two addresses: 16r51578 => in generated method, so you need to use [disassembleMethod/trampoline...] and write down the hex to see the disassembly. (Toggle Transcript first and open a large Transcript if you do that). 16r102BDD0 => in the heap. This is the bytecode version of the method. You can print it using [print oop...] Ok ! One last warm-up exercise: 3) When the simulator has started and the REPL window has popped up, select [single step]. Then enter something in the REPL window and execute it. Once done, do [report recent instructions]. You should be able to see in the Transcript the last 100 machine instruction with the register state in-between each instruction. I think I should write down those exercise as a blog post...
|
Re: Exploring the simulator (was Re: REPL image for simulation)
|
|
On Mon, May 30, 2016 at 4:12 PM, Clément Bera <[hidden email]> wrote:
|
|
|
On Mon, May 30, 2016 at 11:35 PM, Clément Bera <[hidden email]> wrote: > > I did a post out of this thread: > > https://clementbera.wordpress.com/2016/05/30/simulating-the-cog-vm/ Nice article Clement, thanks. One thing though, I can't think what the "dis" means in genAndDis: ? > On Mon, May 30, 2016 at 4:12 PM, Clément Bera <[hidden email]> wrote: >> >> Hi ! >> >> On Mon, May 30, 2016 at 2:39 PM, Ben Coman <[hidden email]> wrote: >>> >>> >>> On Sun, May 29, 2016 at 10:14 AM, Ben Coman <[hidden email]> wrote: >>> > Hi Clement, Thanks for your detailed reply. I particularly liked your >>> > warm up exercises. Goal directed learning is better than general >>> > browsing. >>> > >>> > On Tue, May 24, 2016 at 1:29 AM, Clément Bera <[hidden email]> wrote: >>> >> >>> >> Hi Ben, >>> >> >>> >> The REPL image expects chunk format. Hence you need to write "3 + 4 !" >>> >> >>> >> To get warmed-up: >>> >> 1) Inspect the object memory, then look for the first class table page instance variable. It's an oop referencing an array, try in the simulator to "printOop:" the address of the first class table page that you found. It should print it in the Transcript, the first entries are immediate, in Spur32 SmallInteger/Character/SmallInteger. >>> > >>> > The inspector showed a Spur32MMLECoSimulator and classTableFirstPage >>> > held 16r5311F8. Plugging that into [print oop...] showed... >>> >>> 16r5311F8: a(n) Array >>> 16r52D108 nil 16r15C3A50 class SmallInteger 16r878D70 class >>> Character 16r15C3A50 class SmallInteger >>> 16r1111DC0 class SmallFloat64 16r52D108 nil 16r52D108 nil 16r52D108 nil >>> 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil >>> 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil >>> 16r87AE60 class Array 16r52D108 nil 16r52D108 nil 16r52D108 nil >>> 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil >>> 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil >>> 16r52D108 nil 16r52D108 nil 16r52D108 nil 16r52D108 nil >>> 16r878C58 class LargeNegativeInteger 16r878C90 class >>> LargePositiveInteger 16r10AEAE8 class BoxedFloat64 16r879438 class >>> Message >>> >>> >>> All the nils I guess are due to the class table being a hash map? >>> >>> Is there some way from within the simulation to reference an object by >>> its hex number. For example, to use the size of that array from >>> within the simulation, something like... >>> >>> classTableSize := 16r5311F8 objectFromHex size >> >> >> You got the right result. The the class table is a linked list of pages, each page being an array. The first page, shown here, is reserved for frequently used classes. >> >> Indexes 0-15 are reserved for tagged object. >> Indexes 16-32 are reserved for hidden classes. Typically the class table pages are instances of Array, but the use index 16 so the VM know they are hidden. >> The rest is for real classes that are frequently used. There are many nils so we have free space for new features. It's not a hash map. >> >> I don't think things like that exists: classTableSize := 16r5311F8 objectFromHex size. For oops debugging features are tied to printing through the simulator instance right now. However there is something like that in the JIT. In the machine code zone we can access part of the bytes as CogMethodSurrogate and its subclasses and in the stack we can do the same for stack pages with the corresponding surrogate. In this case one can do something like: >> CogMethodSurrogate at: 16r51578 objectMemory: objectMemory cogit: cogit >> And then one can ask the surrogate things like: >> surrogate cmRefersToYoung >> And it reads the correct bytes for you, in this case answering if the cog method has a reference to a young object. >> >>> >>> > >>> > >>> >> 2) print the active stack, look for the method's address. Try to print it as an oop, and if it tells you "address in the machine code zone", print the cog method and its machine code instead. >>> >>> I presume is the active stack is >>> [print call stack] which produces... >>> >>> 16r1012F8 M MultiByteFileStream(StandardFileStream)>basicNext >>> 16r1E7408: a(n) MultiByteFileStream >>> 16r101334 M UTF8TextConverter>nextFromStream: 16r1EA418: a(n) >>> UTF8TextConverter >>> 16r10135C M MultiByteFileStream>next 16r1E7408: a(n) MultiByteFileStream >>> 16r10138C I MultiByteFileStream(PositionableStream)>nextChunkNoTag >>> 16r1E7408: a(n) MultiByteFileStream >>> 16r1013B0 I StdioListener>run 16r1E7C98: a(n) StdioListener >>> 16r1013D0 I [] in UndefinedObject>(nil) 16r52D108: a(n) UndefinedObject >>> 16r1013F0 I [] in BlockClosure>newProcess 16r1E7E00: a(n) BlockClosure >>> --------- >>> >>> [print oop...] 16r1012F8 tells me... >>> 16r1012F8 is in the stack zone >>> >>> [print cog method for...] 16r1012F8 tells me... >>> not a method >>> >>> [print mc/cog frame] says... >>> Assertion failed >>> with debugger at CogVMSimulatorLSD(CoInterpreter)>>isMachineCodeFrame: >>> >>> So I seem to be missing something. >>> >>> >>> I restarted the simulator and this time... >>> [print call stack...] >>> 16r1012F8 M MultiByteFileStream(StandardFileStream)>basicNext >>> 16r2A1BA8: a(n) MultiByteFileStream >>> 16r101334 M UTF8TextConverter>nextFromStream: 16r2A2188: a(n) >>> UTF8TextConverter >>> 16r10135C M MultiByteFileStream>next 16r2A1BA8: a(n) MultiByteFileStream >>> 16r10138C I MultiByteFileStream(PositionableStream)>nextChunkNoTag >>> 16r2A1BA8: a(n) MultiByteFileStream >>> 16r1013B0 I StdioListener>run 16r2A1B00: a(n) StdioListener >>> 16r1013D0 I [] in UndefinedObject>(nil) 16r52D108: a(n) UndefinedObject >>> 16r1013F0 I [] in BlockClosure>newProcess 16r2A1690: a(n) BlockClosure >>> ---------- >>> >>> [print oop...] 16r1012F8 >>> 16r1012F8 is in the stack zone >>> >>> [print oop...] 16r2A1BA8 >>> 16r2A1BA8: a(n) MultiByteFileStream >>> 16r2A2740 '????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????' >>> 16r1 =0 (16r0) 16r1 =0 (16r0) 16r52D108 nil >>> 16r52D108 nil 16r52D118 false 16r9AA1E8 #stdin 16r24CC18 a ByteArray >>> 16r2A2728 '?' 16r52D108 nil 16r2A2188 an UTF8TextConverter >>> 16r6DF1D8 #lf >>> 16r52D128 true >>> ----------- >>> >>> Now after doing 3+4! several times, >>> [print call stack...] produces... >>> 16r101300 M MultiByteFileStream(StandardFileStream)>basicNext >>> 16r2A1BA8: a(n) MultiByteFileStream >>> 16r10133C M UTF8TextConverter>nextFromStream: 16r2A2188: a(n) >>> UTF8TextConverter >>> 16r101364 M MultiByteFileStream>next 16r2A1BA8: a(n) MultiByteFileStream >>> 16r10138C M MultiByteFileStream(PositionableStream)>nextChunkNoTag >>> 16r2A1BA8: a(n) MultiByteFileStream >>> 16r1013B0 I StdioListener>run 16r2A1B00: a(n) StdioListener >>> 16r1013D0 I [] in UndefinedObject>(nil) 16r52D108: a(n) UndefinedObject >>> 16r1013F0 I [] in BlockClosure>newProcess 16r2A1690: a(n) BlockClosure >>> >>> btw, What is the meaning of the M and I in the second column? I >>> notice that 16r10138C has changed from an I to a M. >>> >>> Also the address associated with basicNext changed from 16r1012F8 to >>> 16r101300. Can some meaning be inferred from that? >> >> >> Some explanations are needed here :-) >> >> The M or I at the beginning of the printing are for 'Interpreted frame' or 'Machine code frame'. Ahhh. Now obvious of course. >> >> When you do [print call stack], you print the list of stack frame in the current stack. For example, >> 16r101300 M MultiByteFileStream(StandardFileStream)>basicNext >> means that: >> - the stack frame address in the stack zone is 16r101300 >> - the machine code version of the method is executed in this frame (M and not I). >> - the receiver has the type MultiByteFileStream So just to confirm, 16r2A1BA8 is MultiByteFileStream object ? >> - the stack frame on top of the stack is the activation for the method StandardFileStream>>basicNext >> >> Now what you tried to do is to print the frame as a method, and that won't work (It's not obvious and my exercise was not very precise, sorry). No problem at all. I appreciate your attention. Anyway the implicit knowledge of something you work with every day can be hard to identify - until newbie questions shine a light on it. I post my results not just for answers, but for others to reference also. In your blog you only generally describe "If the method is jitted, two address are available", but the concrete hex numbers in the example below were quite useful to my understanding. The following would be good to add to your blog post... >> You can use [print frame ...] and put the frame's hex to print it. Alternatively, asyou usually want the top (a.k.a. head) frame, you can directly use [print ext head frame] if it's a machine code frame. >> Now that you've print the frame, you can see the method addresses in this line: >> 16r103144: method: 16r51578 16r102BDD0 16r102BDD0: a(n) CompiledMethod. >> This is a machine code frame, so the method has two addresses: >> 16r51578 => in generated method, so you need to use [disassembleMethod/trampoline...] and write down the hex to see the disassembly. (Toggle Transcript first and open a large Transcript if you do that). >> 16r102BDD0 => in the heap. This is the bytecode version of the method. You can print it using [print oop...] cheers -ben |
Re: Exploring the simulator (was Re: REPL image for simulation)
|
|
Hi Ben,
On Mon, May 30, 2016 at 10:09 AM, Ben Coman <[hidden email]> wrote:
Clément is such a good teacher. It might help to understand the structure of the class table and other structures in Spur to read the SpurMemoryManager class comment. If it contains anything that you don't understand, tell me and I can improve it. It is a design sketch for the entire memory manager; concise, but I hope illuminating. _,,,^..^,,,_ best, Eliot |
Re: Exploring the simulator (was Re: REPL image for simulation)
|
|
In reply to this post by Ben Coman
> On 30-05-2016, at 10:09 AM, Ben Coman <[hidden email]> wrote: > > > On Mon, May 30, 2016 at 11:35 PM, Clément Bera <[hidden email]> wrote: >> >> I did a post out of this thread: >> >> https://clementbera.wordpress.com/2016/05/30/simulating-the-cog-vm/ > > Nice article Clement, thanks. > One thing though, I can't think what the "dis" means in genAndDis: ? Ooh, ooh - I can answer that one! "generate and disassemble” as in generate the code and then disassemble it and display the nicely formatted string result so you can see where it all went horribly wrong. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim The less time planning, the more time programming. |
|
|
Is their some method I can call in the Image to cause the simulator to break into a debugger? I want to do this right before the end block bracket, so I can trace the termination of a forked block without needing to trace over the block's statements. cheers -ben On Tue, May 31, 2016 at 1:27 AM, tim Rowledge <[hidden email]> wrote: > > >> On 30-05-2016, at 10:09 AM, Ben Coman <[hidden email]> wrote: >> >> >> On Mon, May 30, 2016 at 11:35 PM, Clément Bera <[hidden email]> wrote: >>> >>> I did a post out of this thread: >>> >>> https://clementbera.wordpress.com/2016/05/30/simulating-the-cog-vm/ >> >> Nice article Clement, thanks. >> One thing though, I can't think what the "dis" means in genAndDis: ? > > Ooh, ooh - I can answer that one! "generate and disassemble” as in generate the code and then disassemble it and display the nicely formatted string result so you can see where it all went horribly wrong. > > > tim > -- > tim Rowledge; [hidden email]; http://www.rowledge.org/tim > The less time planning, the more time programming. > > |
Re: Exploring the simulator (was Re: REPL image for simulation)
|
|
Well you can call a method with a specific selector name and use the [break selector] feature in the simulator. Alternatively you call a specific primitive and put a halt in it in the simulator. On Fri, Jun 3, 2016 at 3:45 PM, Ben Coman <[hidden email]> wrote:
|
|
|
> On Fri, Jun 3, 2016 at 3:45 PM, Ben Coman <[hidden email]> wrote: >> >> >> Is their some method I can call in the Image to cause the simulator to >> break into a debugger? I want to do this right before the end block >> bracket, so I can trace the termination of a forked block without >> needing to trace over the block's statements. >> >> cheers -ben On Fri, Jun 3, 2016 at 11:50 PM, Clément Bera <[hidden email]> wrote: > > Well you can call a method with a specific selector name and use the [break selector] feature in the simulator. Alternatively you call a specific primitive and put a halt in it in the simulator. cool, thanks. cheers -ben |
Re: Exploring the simulator (was Re: REPL image for simulation)
|
|
> On 04.06.2016, at 04:17, Ben Coman <[hidden email]> wrote: > > >> On Fri, Jun 3, 2016 at 3:45 PM, Ben Coman <[hidden email]> wrote: >>> >>> >>> Is their some method I can call in the Image to cause the simulator to >>> break into a debugger? I want to do this right before the end block >>> bracket, so I can trace the termination of a forked block without >>> needing to trace over the block's statements. >>> >>> cheers -ben > > On Fri, Jun 3, 2016 at 11:50 PM, Clément Bera <[hidden email]> wrote: >> >> Well you can call a method with a specific selector name and use the [break selector] feature in the simulator. Alternatively you call a specific primitive and put a halt in it in the simulator. > > cool, thanks. > cheers -ben Smalltalk exitToDebugger calls primitive 114 which runs primitiveExitToDebugger which maybe the simulator should handle differently ... - Bert - |
smime.p7s (5K)
Re: Exploring the simulator (was Re: REPL image for simulation)
|
|
In reply to this post by Ben Coman
Hi Ben,
On Fri, Jun 3, 2016 at 6:45 AM, Ben Coman <[hidden email]> wrote:
There is a general facility for message selectors (setBreakSelector:), either when sent in the interpreter, compiled in the JIT or not understood (setBreakMNUSelector:). There is a generalised breakpoint facility using blocks. In the Cogit you can specify a block that will be run on each machine instruction, e.g. | vm | vm := CogVMSimulator newWithOptions: #(#ObjectMemory #Spur32BitCoMemoryManager). vm desiredNumStackPages: 8. vm setBreakSelector: #behaviorHashOf:. vm openOn: (FileDirectory default fullNameFor: 'startreader.image'). vm breakBlock: [:cogit | cogit processor pc = 446009 and: [cogit processor edx = 3871504]]. In the interpreter there's an atEachStep block run on each bytecode. You can have separate atEachStep: and breakBlock: blocks. cheers -ben _,,,^..^,,,_ best, Eliot |
|
|
In reply to this post by Ben Coman
I am stepping for the first time through the CogVM, having [set break selector...] forkAt: After stepping in a few times I get to #activateCoggedNewMethod. CogVMSimulatorLSB(CoInterpreter)>>dispatchOn:in: CogVMSimulatorLSB(CoInterpreter)>>sendLiteralSelector1ArgBytecode CogVMSimulatorLSB(CoInterpreter)>>commonSendOrdinary CogVMSimulatorLSB(CoInterpreter)>>insternalExecuteNewMethod CogVMSimulatorLSB(CoInterpreter)>>activateCoggedNewMethod Here from the code at the top. methodHeader := self rawHeaderOf: newMethod. self assert: (self isCogMethodReference: methodHeader). cogMethod := self cCoerceSimple: methodHeader to: #'CogMethod *'. methodHeader := cogMethod methodHeader. I guess methodHeader's double assignment above is related to the machine code frame having two addresses as Clement described... >> On Mon, May 30, 2016 at 4:12 PM, Clément Bera <[hidden email]> wrote: >>> Now that you've print the frame, you can see the method addresses in this line: >>> 16r103144: method: 16r51578 16r102BDD0 16r102BDD0: a(n) CompiledMethod. >>> This is a machine code frame, so the method has two addresses: >>> 16r51578 => in generated method, so you need to use [disassembleMethod/trampoline...] and write down the hex to see the disassembly. >>> 16r102BDD0 => in the heap. This is the bytecode version of the method. You can print it using [print oop...] This time... [print ext head frame] ==> 16r101214 M BlockClosure>forkAt: 16r2FC420: a(n) BlockClosure 16r101210: method: 16rBBF0 16rC4E948 16rC4E948: a(n) CompiledMethod self rawHeaderOf: newMethod ==> 16rBBF0 So the "raw header" is the cogged method. Looking at the output below, the space ship operator <-> seems to link between cogged method headers like a call stack, except #forkAt: calls #newProcess which calls #asContext [print cog method for...] 16rBBF0 ==> 16rBBF0 <-> 16rBC80: method: 16rC4E948 selector: 16r6CC798 forkAt: [print cog method for...] 16rBC80 ==> 16rBC80 <-> 16rBEA8: method: 16rC51970 prim 19 selector: 16r6D1620 newProcess [print cog method for...] 16rBEA8 ==> 16rBEA8 <-> 16rBF28: method: 16rC518C0 selector: 16r76A600 asContext However the links don't seem to go back up the call stack but forward, to statements to be executed in the future. So I am confused? ------------- Considering further [print cog method for...] 16rBBF0 ==> 16rBBF0 <-> 16rBC80: method: 16rC4E948 selector: 16r6CC798 forkAt: [print oop...] 16r6CC798 ==> a(n) ByteSymbol nbytes 7 forkAt: Clement early advised is the bytecode version of the method is this... [print oop...] 16rC4E948 ==> 16rC4E948: a(n) CompiledMethod nbytes 37 16rBBF0 is in generated methods 16r6D1620 #newProcess 16r6CC650 #priority: 16r6CC690 #resume 16r6CC798 #forkAt: 16rAE5490 a ClassBinding #BlockClosure -> 16r0088D618 16rC4E968: 70/112 D0/208 88/136 10/16 E1/225 87/135 D2/210 7C/124 16rC4E970: 28/40 AF/175 BA/186 F3/243 20/32 Now I've been a bit slow on the uptake and only just realised, but to confirm... the line 16r6CC798 is the one specifying the method as BlockClosure>>forkAt: For the last two lines, I notice the numbers before the slash (70, 88, 10...) are the method bytecode, but what are the numbers after the slash? ---------------- In #activeCoggedNewMethod: the second assignment to methodHeader ==> 16r208000B which matches the mthhdr field of the raw header [print cog method header for...] 16rBBF0 ==> BBF0 objhdr: 8000000A000035 nArgs: 1 type: 2 blksiz: 90 method: C4E948 mthhdr: 208000B selctr: 6CC798=#forkAt: blkentry: 0 stackCheckOffset: 5E/BC4E cmRefersToYoung: no cmIsFullBlock: no What is "type: 2" ? -------------------------- Stepping through to Cogit>>ceEnterCogCodePopReceiverReg I notice its protocol is "simulation only" and it calls "simulateEnilopmart:numArgs: ceEnterCogCodePopReceiverReg" but I don't see any other implementors of #ceEnterCogCodePopReceiverReg. Also there is a pragma <doNotGenerate>. Obviously the real non-simulated VM works differently, but I can't determine how. btw, I have noticed that ceEnterCogCodePopReceiverReg ==> 16r10B8 and [print cog method for...] 16r10B8 ==> trampoline ceEnterCogCodePopReceiverReg Is ceEnterCogCodePopReceiverReg provided by the platform C code? --------------------------- Stepping through to simulateCogCodeAt: it called processor singleStepIn:minimumAddress:readOnlyBelow: which called BochsIA32Alien>>primitiveSingleStepInMemory:minimumAddress:readOnlyBelow: <primitive: 'primitiveSingleStepInMemoryMinimumAddressReadWrite' module: 'BochsIA32Plugin' error: ec> ^ec == #'inappropriate operation' ifTrue: [self handleExecutionPrimitiveFailureIn: memoryArray minimumAddress: minimumAddress] ifFalse: [self reportPrimitiveFailure] and the debugger cursor was inside the ifTrue: statement. I found I didn't have bochs installed, but after installing bochs-2.6-2, I go the same result. So could I get some background around this.. Also I'm curious how the simulator seemed to be running a CogVM before bochs was installed. Perhaps since I was not debugging through it, the machine code ran for real rather than being simulated. cheers -ben |
Re: Exploring the simulator (was Re: REPL image for simulation)
|
|
Hi Ben,
I'm glad you're now looking into the JIT. If you have some blog or something, please write an experience report about you looking into the simulator. It's helpful for us to have noise around the VM. On Sun, Jun 12, 2016 at 8:35 AM, Ben Coman <[hidden email]> wrote:
Errr... I don't really fancy the way you say it but I think yes that's it. A method can have 2 addresses, the address of the bytecoded version in the heap and the address of its jitted version in the machine code zone. In the machine code frame printing, the simulator displays the 2 addresses. But the frame has a single pointer to the method. So what you're looking at is the dispatch logic from the bytecoded method to the jitted method. When the JIT compiles a bytecoded method to machine code, it replaces the bytecoded method compiled method header (first literal) by a pointer to the jitted version. The machine code version of the method keeps the compiled method header, so accessing it is different in methods compiled to machine code and methods not compiled to machine code. #rawHeaderOf: answers the first literal of the bytecoded method which is a pointer to the jitted version of the method if the method has a jitted version, else is the compiled method header. In the code you show, the VM ensures the method has a jitted version with the assertion, hence the compiled method header is fetched from the jitted version.
Yeah it's the jitted version of the method header address, then <->, then the jitted method entry point address, the bytecode version address, selector address. The cogMethod header is used to store the bytecoded compiled method header (because it was replaced with a pointer to the cogMethod) and various flags.
16r6CC798 is the address of the selector #forkAt:
The bytecode in decimal instead of hexa I think.
Haha. Well when you iterate over the machine code zone you need to know what the current element you iterate on is. In the machine code zone there can be: - cog method - closed PICS - open PICS - free space And now we're adding cog full block method but it's sharing the index with cog method and have a separated flag :-) The type tells you what it is. Look at the Literal variables CMFree, CMClosedPIC, CMOpenPIC, etc . 2 is CMMethod with is a constant. You can improve the printing there and commit the changes if you feel so. Ok I have to go I will look at the rest of your mail later.
|
Re: Exploring the simulator (was Re: REPL image for simulation)
|
|
Hi again,
On Sun, Jun 12, 2016 at 10:44 AM, Clément Bera <[hidden email]> wrote:
What did I write here I don't understand myself ? I mean CMMethod = 2, so type = 2 means the struct you're looking at in the machine code zone is a method and not free space or a PIC.
Let's do this...
Well it's in cogitIA32.c. I don't remember where it comes from. Basically in Cog you have specific machine code routines, called trampolines, that switch from machine code to C code. When trampoline is written backward (Enilopmart) it means that the routine is meant to switch from C code to machine code. The real VM calls in ceEnterCogCodePopReceiverReg a machine code routine that does the right thing (register remapped, maybe fp and sp saved, etc) to switch from the C runtime from the C compiler to the machine code runtime executing code generated by the JIT. In simulation, the C code is simulated by executing Slang as Smalltalk code and the machine code is simulated using the processor simulator (Bochs for IA32). So it has to be done differently as there is no C stack with register state and stuff. Both trampolines and enilmoparts are simulated with specific code.
No the machine code is always simulated. Bochs was working for sure if you successfully simulated the image on top of the cog simulator until the display was shown. If you have a VM from one of Eliot's build (from the Cog blog) the processor simulators are present as plugins by default. On Mac you can do [show package contents...] and then look at the file inside to check the Bochs Plugin is there. It's not the case on the Pharo VMs so don't use them for CogVM simulation. You don't need to install anything. On normal simulation the simulator goes often in the branch you've just shown. It means it reached a simulation trap. As for enilmopart that can't be properly simulated, trampolines can't be simulated. So to simulate a trampoline the processor simulator fails a call and the trampoline is done in the simulation code. Look at #handleCallOrJumpSimulationTrap: for example.
|
|
|
On Sun, Jun 12, 2016 at 10:59 PM, Clément Bera <[hidden email]> wrote: > > Hi again, > > On Sun, Jun 12, 2016 at 10:44 AM, Clément Bera <[hidden email]> wrote: >> >> Hi Ben, >> >> I'm glad you're now looking into the JIT. If you have some blog or something, please write an experience report about you looking into the simulator. It's helpful for us to have noise around the VM. Cool. I'll have a go. >> >> On Sun, Jun 12, 2016 at 8:35 AM, Ben Coman <[hidden email]> wrote: >>> >>> >>> I am stepping for the first time through the CogVM, having [set break >>> selector...] forkAt: >>> After stepping in a few times I get to #activateCoggedNewMethod. >>> CogVMSimulatorLSB(CoInterpreter)>>dispatchOn:in: >>> CogVMSimulatorLSB(CoInterpreter)>>sendLiteralSelector1ArgBytecode >>> CogVMSimulatorLSB(CoInterpreter)>>commonSendOrdinary >>> CogVMSimulatorLSB(CoInterpreter)>>insternalExecuteNewMethod >>> CogVMSimulatorLSB(CoInterpreter)>>activateCoggedNewMethod >>> >>> Here from the code at the top. >>> methodHeader := self rawHeaderOf: newMethod. >>> self assert: (self isCogMethodReference: methodHeader). >>> cogMethod := self cCoerceSimple: methodHeader to: #'CogMethod *'. >>> methodHeader := cogMethod methodHeader. >>> >>> I guess methodHeader's double assignment above is related to the >>> machine code frame having two addresses as Clement described... >> >> >> Errr... I don't really fancy the way you say it but I think yes that's it. >> >> A method can have 2 addresses, the address of the bytecoded version in the heap and the address of its jitted version in the machine code zone. In the machine code frame printing, the simulator displays the 2 addresses. But the frame has a single pointer to the method. >> >> So what you're looking at is the dispatch logic from the bytecoded method to the jitted method. When the JIT compiles a bytecoded method to machine code, it replaces the bytecoded method compiled method header (first literal) by a pointer to the jitted version. The machine code version of the method keeps the compiled method header, so accessing it is different in methods compiled to machine code and methods not compiled to machine code. >> >> #rawHeaderOf: answers the first literal of the bytecoded method which is a pointer to the jitted version of the method if the method has a jitted version, else is the compiled method header. In the code you show, the VM ensures the method has a jitted version with the assertion, hence the compiled method header is fetched from the jitted version. I think I've got it. So upon JITing, CompiledMethod and its literals and bytecodes don't move. Only its bytecodeHeader is manipulated and re-purposed. Before JIT... compiledMethod := { bytecodeHeader, literals, bytecodes }. byteCodeHeader := compiledMethod at: 1 After JIT something like... cogMethod := { cogMethodHeader, bytecodeHeader, machineCode } compiledMethod := { pointerTo_cogMethod, literals, bytecodes }. rawHeader := compiledMethod at: 1 cogMethodHeader := dereferenced(rawHeader) at: 1. >> >>> >>> >> On Mon, May 30, 2016 at 4:12 PM, Clément Bera <[hidden email]> wrote: >>> >>> Now that you've print the frame, you can see the method addresses in this line: >>> >>> 16r103144: method: 16r51578 16r102BDD0 16r102BDD0: a(n) CompiledMethod. >>> >>> This is a machine code frame, so the method has two addresses: >>> >>> 16r51578 => in generated method, so you need to use [disassembleMethod/trampoline...] and write down the hex to see the disassembly. >>> >>> 16r102BDD0 => in the heap. This is the bytecode version of the method. You can print it using [print oop...] >>> >>> This time... >>> [print ext head frame] ==> >>> 16r101214 M BlockClosure>forkAt: 16r2FC420: a(n) BlockClosure >>> 16r101210: method: 16rBBF0 16rC4E948 16rC4E948: a(n) CompiledMethod >>> >>> self rawHeaderOf: newMethod ==> 16rBBF0 >>> So the "raw header" is the cogged method. >>> >>> Looking at the output below, the space ship operator <-> seems to link >>> between cogged method headers like a call stack, except #forkAt: >>> calls #newProcess which calls #asContext >>> >>> [print cog method for...] 16rBBF0 ==> >>> 16rBBF0 <-> 16rBC80: method: 16rC4E948 selector: 16r6CC798 forkAt: >>> >>> [print cog method for...] 16rBC80 ==> >>> 16rBC80 <-> 16rBEA8: method: 16rC51970 prim 19 selector: 16r6D1620 newProcess >>> >>> [print cog method for...] 16rBEA8 ==> >>> 16rBEA8 <-> 16rBF28: method: 16rC518C0 selector: 16r76A600 asContext >>> >>> However the links don't seem to go back up the call stack but forward, >>> to statements to be executed in the future. So I am confused? >> >> >> Yeah it's the jitted version of the method header address, then <->, then the jitted method entry point address, the bytecode version address, selector address. >> >> The cogMethod header is used to store the bytecoded compiled method header (because it was replaced with a pointer to the cogMethod) and various flags. >> >>> >>> >>> ------------- >>> >>> Considering further [print cog method for...] 16rBBF0 ==> >>> 16rBBF0 <-> 16rBC80: method: 16rC4E948 selector: 16r6CC798 forkAt: >>> >>> [print oop...] 16r6CC798 ==> >>> a(n) ByteSymbol nbytes 7 forkAt: >>> >>> Clement early advised is the bytecode version of the method is this... >>> [print oop...] 16rC4E948 ==> >>> 16rC4E948: a(n) CompiledMethod nbytes 37 >>> 16rBBF0 is in generated methods >>> 16r6D1620 #newProcess 16r6CC650 #priority: 16r6CC690 #resume >>> 16r6CC798 #forkAt: 16rAE5490 a ClassBinding #BlockClosure -> 16r0088D618 >>> 16rC4E968: 70/112 D0/208 88/136 10/16 E1/225 87/135 D2/210 7C/124 >>> 16rC4E970: 28/40 AF/175 BA/186 F3/243 20/32 >>> >>> Now I've been a bit slow on the uptake and only just realised, but to confirm... >>> the line 16r6CC798 is the one specifying the method as BlockClosure>>forkAt: >> >> 16r6CC798 is the address of the selector #forkAt: Sorry I wasn't clear. I wasn't referring to the address itself of the selector - that was just a line reference. My insight I wanted to confirm was that the last oop before the bytecode was... a ClassBinding #BlockClosure -> 16r0088D618 and the next last before that was... #forkAt: 16rAE5490 indicating the output of [print oop...] was method BlockClosure>>forkAt: , while above that line are the methods called by #forkAt: and below it is the bytecode. Ahhh, actually I just saw this relevant comment in CompiledMethod... "The last literal in a CompiledMethod must be its methodClassAssociation, a binding whose value is the class the method is installed in. The methodClassAssociation is used to implement super sends. If a method contains no super send then its methodClassAssociation may be nil (as would be the case for example of methods providing a pool of inst var accessors). By convention the penultimate literal of a method is either its selector or an instance of AdditionalMethodState. " So it seems it won't always show the Class>>method, but often will. >> >>> >>> For the last two lines, I notice the numbers before the slash (70, 88, >>> 10...) are the method bytecode, but what are the numbers after the >>> slash? >> >> >> The bytecode in decimal instead of hexa I think. I checked. You are right. Obvious in hindsight. >>> ---------------- >>> >>> In #activeCoggedNewMethod: the second assignment to methodHeader >>> ==> 16r208000B >>> >>> which matches the mthhdr field of the raw header >>> [print cog method header for...] 16rBBF0 ==> >>> BBF0 >>> objhdr: 8000000A000035 >>> nArgs: 1 type: 2 >>> blksiz: 90 >>> method: C4E948 >>> mthhdr: 208000B >>> selctr: 6CC798=#forkAt: >>> blkentry: 0 >>> stackCheckOffset: 5E/BC4E >>> cmRefersToYoung: no cmIsFullBlock: no >>> >>> What is "type: 2" ? >> >> >> Haha. >> >> Well when you iterate over the machine code zone you need to know what the current element you iterate on is. In the machine code zone there can be: >> - cog method >> - closed PICS >> - open PICS >> - free space >> And now we're adding cog full block method but it's sharing the index with cog method and have a separated flag :-) >> >> The type tells you what it is. Look at the Literal variables CMFree, CMClosedPIC, CMOpenPIC, etc . >> >> 2 is CMMethod with is a constant. You can improve the printing there and commit the changes if you feel so. > > > What did I write here I don't understand myself ? I mean CMMethod = 2, so type = 2 means the struct you're looking at in the machine code zone is a method and not free space or a PIC. >> >> >> Ok I have to go I will look at the rest of your mail later. > > > Let's do this... >> >> >>> >>> >>> -------------------------- >>> >>> Stepping through to Cogit>>ceEnterCogCodePopReceiverReg >>> I notice its protocol is "simulation only" >>> and it calls "simulateEnilopmart:numArgs: ceEnterCogCodePopReceiverReg" >>> but I don't see any other implementors of #ceEnterCogCodePopReceiverReg. >>> Also there is a pragma <doNotGenerate>. >>> >>> Obviously the real non-simulated VM works differently, but I can't >>> determine how. >>> >>> btw, I have noticed that ceEnterCogCodePopReceiverReg >>> ==> 16r10B8 >>> and [print cog method for...] 16r10B8 >>> ==> trampoline ceEnterCogCodePopReceiverReg >>> >>> Is ceEnterCogCodePopReceiverReg provided by the platform C code? > > > Well it's in cogitIA32.c. I don't remember where it comes from. Cool. I had a peek. > > Basically in Cog you have specific machine code routines, called trampolines, that switch from machine code to C code. When trampoline is written backward (Enilopmart) it means that the routine is meant to switch from C code to machine code. > > The real VM calls in ceEnterCogCodePopReceiverReg a machine code routine that does the right thing (register remapped, maybe fp and sp saved, etc) to switch from the C runtime from the C compiler to the machine code runtime executing code generated by the JIT. I see its a function pointer... void (*ceEnterCogCodePopReceiverReg)(void) set by... ceEnterCogCodePopReceiverReg = genEnilopmartForandandforCallcalled(ReceiverResultReg, NoReg, NoReg, 0, "ceEnterCogCodePopReceiverReg"); which is beyond my current level need-to-know. Still useful to fill in the background architecture. This comment comparing trampoline/enilopmart to system-call-like transition was enlightening... /* An enilopmart (the reverse of a trampoline) is a piece of code that makes the system-call-like transition from the C runtime into generated machine code. The desired arguments and entry-point are pushed on a stackPage's stack. The enilopmart pops off the values to be loaded into registers and then executes a return instruction to pop off the entry-point and jump to it. BEFORE AFTER (stacks grow down) whatever stackPointer -> whatever target address => reg1 = reg1val, etc reg1val pc = target address reg2val stackPointer -> reg3val */ /* Cogit>>#genEnilopmartFor:and:and:forCall:called: */ > > In simulation, the C code is simulated by executing Slang as Smalltalk code and the machine code is simulated using the processor simulator (Bochs for IA32). So it has to be done differently as there is no C stack with register state and stuff. Both trampolines and enilmoparts are simulated with specific code. > >>> >>> >>> --------------------------- >>> Stepping through to simulateCogCodeAt: >>> it called processor singleStepIn:minimumAddress:readOnlyBelow: >>> which called BochsIA32Alien>>primitiveSingleStepInMemory:minimumAddress:readOnlyBelow: >>> <primitive: 'primitiveSingleStepInMemoryMinimumAddressReadWrite' >>> module: 'BochsIA32Plugin' >>> error: ec> >>> ^ec == #'inappropriate operation' >>> ifTrue: [self handleExecutionPrimitiveFailureIn: memoryArray >>> minimumAddress: minimumAddress] >>> ifFalse: [self reportPrimitiveFailure] >>> >>> and the debugger cursor was inside the ifTrue: statement. I found I >>> didn't have bochs installed, but after installing bochs-2.6-2, I go >>> the same result. So could I get some background around this.. >>> >>> Also I'm curious how the simulator seemed to be running a CogVM before >>> bochs was installed. Perhaps since I was not debugging through it, the >>> machine code ran for real rather than being simulated. >>> > > No the machine code is always simulated. Bochs was working for sure if you successfully simulated the image on top of the cog simulator until the display was shown. > > If you have a VM from one of Eliot's build (from the Cog blog) the processor simulators are present as plugins by default. On Mac you can do [show package contents...] and then look at the file inside to check the Bochs Plugin is there. It's not the case on the Pharo VMs so don't use them for CogVM simulation. You don't need to install anything. Ahhh... I see them now. ./lib/squeak/5.0-3692/BochsX64Plugin ./lib/squeak/5.0-3692/BochsIA32Plugin The clears my misconception - a lack of understanding the purpose of the primitive failure and a red herring when I saw the Boch's system package wasn't installed. > > On normal simulation the simulator goes often in the branch you've just shown. It means it reached a simulation trap. As for enilmopart that can't be properly simulated, trampolines can't be simulated. So to simulate a trampoline the processor simulator fails a call and the trampoline is done in the simulation code. Look at #handleCallOrJumpSimulationTrap: for example. Ah, so its an 'inappropriate operation' from Bochs' perspective, but from the Simulator's perspective the primitiveFail is a useful condition like the #ensure: "Primitive 198 always fails. The VM uses prim 198 in a context's method as the mark for an ensure:/ifCurtailed: activation." ? cheers -ben btw, I bumped into a bit of history... http://www.mirandabanda.org/cogblog/2008/12/12/simulate-out-of-the-bochs/ |
Re: Exploring the simulator (was Re: REPL image for simulation)
|
|
Hi Ben,
On Sun, Jun 12, 2016 at 10:36 AM, Ben Coman <[hidden email]> wrote:
Right. When a method is cogged (jitter) its header is set to point to the Cog method (machine code method), and the actual header is stashed inside the Cog method. This is invisible to the image because only the objectAt: primitive accesses CompiledMethod literals and this primitive checks. In the VM all points where methodHeader is accessed must check for a normal method (header is a SmallInteger) and a cogged method (header is not a SmallInteger).
Note that you can use the image[level byte code printing machinery on a method in the simulator by using Stackinterpreter>>symbolicMethod:. The text is output in the simulator window's transcript or the system transcript. See "toggle transcript" towards the top of the bottom right hand simulator window's menu.
Right. Trampolines are the machine code routines that call into the Smalltalk (simulator) / C (real VM) run-time support routines. They make a stack switch from the Smalltalk/Cog-machine-code stack to the actual C stack, and pass parameters. Enilopmarts do the reverse. They switch from the actual C stack back into the Smalltalk/Cog-machine-code stack, possibly popping values pushed onto that stack into specific registers, and then executing a return instruction to jump to some machine code address in the machine code zone to start or resume machine-code execution. > Not quite. The trampolines are simulated. The calls the trampolines make can't be simulated. These calls are to illegal addresses and cause traps. The trap handler maps the addresses into the appropriate Smalltalk blocks/methods and invokes them. The same goes for accessing variables in the simulator such as framePointer, stackPointer, instructionPointer etc. These are Smalltalk objects that are instanced variables of the CoInterpreter. They are mapped to illegal addresses in machine code and attempts to access them cause traps and the trap handler maps these to fetch/store the relevant inst var. In the real VM the actual addresses of the variables are used directly.
Right. Andreas realised specific primitives that did nothing could be used to mark methods for the VM's benefit, without needing a bit in the method header. Very clever, very economical. A nice idea. cheers -ben :-) -- _,,,^..^,,,_ best, Eliot |
Re: Exploring the simulator (was Re: REPL image for simulation)
|
|
In reply to this post by Ben Coman
On Sun, Jun 12, 2016 at 7:36 PM, Ben Coman <[hidden email]> wrote:
I guess we could say that. A compiled method is more like, before JIT: object's header (64 bits) compiled method header (1 word) literals (several words) bytecodes (several bytes) compiled method trailer (several bytes) After jitting, the compiled method header is replaced by a pointer to the cog method. The cog method has (assuming it has no blocks): - header - native instructions - map and the cog method header includes the compiled method header.
Well... By convention the bytecode compiler always put the class binding as the last literal except if there is not enough room in the literal frame, which in practice happens on average 1 methods out of 100.000 in my experience, and which will never happen once we've switched to the new bytecode set... But if the method has no super sends, it does not need the class as the last literal, and one could compile an image like that to save some memory. If one removes both the selector (last but one literal) and the class binding, one could save 150kb in the base Pharo image out of ~47Mb. Currently there is no setting to do that, but one could do it.
Err... I think it's a bit different. The processor simulator keeps running machine code until it traps, in which case the simulation figures out why it traps, and likely it trapped because it needed to switch from machine code to C code hence to the Smalltalk runtime in simulation. The normal behavior is that most of the time processor simulator primitives succeed, sometimes they fail. Primitive 198 and 199 always fail. If you want to try you can alternatively use the MIPS back-end to simulate machine code which is done fully in Smalltalk instead of Bochs. The back-ends for x86, x64 and ARM are simulated using external processor simulator frameworks, while the MIPS simulator is written entirely in Smalltalk. The settings to use MIPS is (ISA MIPSEL). Don't hesitate to use other back-ends, it's fun, (IA32 X64 ARMv5 MIPSEL) settings.
Yeah this is a good post. |
«
Return to Squeak VM
|
1 view|%1 views
| Free forum by Nabble | Edit this page |