Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/..VMMaker.oscog-eem.618.mcz ==================== Summary ==================== Name: ..VMMaker.oscog-eem.618 Author: eem Time: 13 February 2014, 11:51:10.104 am UUID: 3756532a-fdbf-4e96-8cc9-be0f35157864 Ancestors: ..VMMaker.oscog-eem.617 Some musings on Spur compaction. =============== Diff against ..VMMaker.oscog-eem.617 =============== Item was added: + ----- Method: SpurMemoryManager>>abstractCompaction (in category 'compaction - analysis') ----- + abstractCompaction + "This method answers a rough estimate of compactibility." + <doNotGenerate> + | lowestFree freeChunks used movable | + lowestFree := SmallInteger maxVal. + freeChunks := Set new. + used := Set new. + movable := Set new. + self allObjectsInFreeTreeDo: + [:f| + (self addressAfter: f) < endOfMemory ifTrue: + [freeChunks add: f. + f < lowestFree ifTrue: [lowestFree := f]]]. + self allOldSpaceObjectsFrom: lowestFree do: + [:o| | size delta best | + size := self bytesInObject: o. + delta := SmallInteger maxVal. + freeChunks do: [:f| | fs | + ((fs := self bytesInObject: f) >= size) ifTrue: + [delta > (fs - size) ifTrue: + [delta := fs - size. best := f]]]. + best ifNotNil: + [movable add: o. + used add: (freeChunks remove: best)]]. + ^{ totalFreeOldSpace. movable inject: 0 into: [:s :o| s + (self bytesInObject: o)]. used inject: 0 into: [:s :o| s + (self bytesInObject: o)] }! Item was added: + ----- Method: SpurMemoryManager>>abstractPigCompaction (in category 'compaction - analysis') ----- + abstractPigCompaction + "This method answers a rough estimate of compactibility using a pig (a large free chunk)." + <doNotGenerate> + | pig pork moved unmoved nmoved nunmoved | + pig := self findAPig. + pork := self bytesInObject: pig. + moved := unmoved := nmoved := nunmoved := 0. + self allOldSpaceObjectsFrom: pig do: + [:o| | bytes | + bytes := self bytesInObject: o. + bytes <= pork + ifTrue: + [moved := moved + bytes. + nmoved := nmoved + 1. + pork := pork - bytes] + ifFalse: + [unmoved := unmoved + bytes. + nunmoved := nunmoved + 1]]. + ^{ self bytesInObject: pig. pork. moved. nmoved. unmoved. nunmoved }! Item was added: + ----- Method: SpurMemoryManager>>biggies (in category 'compaction - analysis') ----- + biggies + "This method answers a sorted collection of the objects >= 1,000,000 bytes long, + above the lowest large free chunk, sandwiched between nilObj and the end of memory." + <doNotGenerate> + | lowestFree biggies | + lowestFree := SmallInteger maxVal. + self allObjectsInFreeTreeDo: + [:f| (self addressAfter: f) < endOfMemory ifTrue: [f < lowestFree ifTrue: [lowestFree := f]]]. + biggies := SortedCollection new. + self allOldSpaceObjectsFrom: lowestFree do: + [:f| + (self bytesInObject: f) >= 1000000 ifTrue: + [biggies add: f]]. + ^{{nilObj hex. #nil}}, (biggies collect: [:f| {f hex. self bytesInObject: f}]), {{endOfMemory hex. #endOfMemory}}! Item was added: + ----- Method: SpurMemoryManager>>compactionIssues (in category 'compaction - analysis') ----- + compactionIssues + <doNotGenerate> + "Compaction isn't working well. It rarely moves more than a few tens of kilobytes. Why? + Load an image and before you run it, or just before a GC, run these anaylsis routines. + e.g. + self abstractCompaction #(63230272 75456 63210648) + shows we can move 75456 bytes of objects, but that would use 63210648 of free space. + i.e. there are lots of big free chunks in play, not many small ones that fit the bill. + + self largeFreeChunkDistribution + #( #('16r31C788' #nil) + #('16r1423AC0' 2061864) + #('16r1B705E8' 1515200) + #('16r1D31D20' 2011152) + #('16r1F37818' 1491480) + #('16r2225968' 1450512) + #('16r24C92C8' 48575672) (16r24C92C8 + 48575672) hex '16r531C780' a free chunk + #('16r531C788' #endOfMemory)) + shows there's plenty of large free chunks. And the trailing 16-byte free chunk shows coallescing is not working properly. + + self biggies #(#('16r31C788' #nil) #('16r531C788' #endOfMemory)) + shows there are no large objects to be moved. + + So... looks like compaction should hold onto the lowest large chunk and preferentially move objects into that. + Let's call it a pig. Compaction needs to whittle away at the pig. + + e.g. + self abstractPigCompaction #(2061864 0 2061864 18759 2018224 34757) + shows we can move 18759 objects that will occupy 2018224 bytes into that + low pig of 2061864 bytes."! Item was added: + ----- Method: SpurMemoryManager>>findAPig (in category 'compaction - analysis') ----- + findAPig + "Answer a large low free chuink." + <doNotGenerate> + | pig | + self allObjectsInFreeTreeDo: + [:f| + (self bytesInObject: f) >= 1000000 ifTrue: + [(pig isNil or: [pig > f]) ifTrue: + [pig := f]]]. + ^pig! Item was added: + ----- Method: SpurMemoryManager>>largeFreeChunkDistribution (in category 'compaction - analysis') ----- + largeFreeChunkDistribution + "This method answers a sorted collection of the free chunks >= 1,000,000 bytes long, + sandwiched between nilObj and the end of memory (ignoring the large chunk often found at the end of the heap)." + <doNotGenerate> + | freeChunks | + freeChunks := SortedCollection new. + self allObjectsInFreeTreeDo: + [:f| + ((self addressAfter: f) < endOfMemory + and: [(self bytesInObject: f) >= 1000000]) ifTrue: + [freeChunks add: f]]. + ^{{nilObj hex. #nil}}, (freeChunks collect: [:f| {f hex. self bytesInObject: f}]), {{endOfMemory hex. #endOfMemory}}! |
Is it intentional that this package and the previous one are prefixed with dots? Levente On Thu, 13 Feb 2014, [hidden email] wrote: > > Eliot Miranda uploaded a new version of VMMaker to project VM Maker: > http://source.squeak.org/VMMaker/..VMMaker.oscog-eem.618.mcz > > ==================== Summary ==================== > > Name: ..VMMaker.oscog-eem.618 > Author: eem > Time: 13 February 2014, 11:51:10.104 am > UUID: 3756532a-fdbf-4e96-8cc9-be0f35157864 > Ancestors: ..VMMaker.oscog-eem.617 > > Some musings on Spur compaction. > > =============== Diff against ..VMMaker.oscog-eem.617 =============== > > Item was added: > + ----- Method: SpurMemoryManager>>abstractCompaction (in category 'compaction - analysis') ----- > + abstractCompaction > + "This method answers a rough estimate of compactibility." > + <doNotGenerate> > + | lowestFree freeChunks used movable | > + lowestFree := SmallInteger maxVal. > + freeChunks := Set new. > + used := Set new. > + movable := Set new. > + self allObjectsInFreeTreeDo: > + [:f| > + (self addressAfter: f) < endOfMemory ifTrue: > + [freeChunks add: f. > + f < lowestFree ifTrue: [lowestFree := f]]]. > + self allOldSpaceObjectsFrom: lowestFree do: > + [:o| | size delta best | > + size := self bytesInObject: o. > + delta := SmallInteger maxVal. > + freeChunks do: [:f| | fs | > + ((fs := self bytesInObject: f) >= size) ifTrue: > + [delta > (fs - size) ifTrue: > + [delta := fs - size. best := f]]]. > + best ifNotNil: > + [movable add: o. > + used add: (freeChunks remove: best)]]. > + ^{ totalFreeOldSpace. movable inject: 0 into: [:s :o| s + (self bytesInObject: o)]. used inject: 0 into: [:s :o| s + (self bytesInObject: o)] }! > > Item was added: > + ----- Method: SpurMemoryManager>>abstractPigCompaction (in category 'compaction - analysis') ----- > + abstractPigCompaction > + "This method answers a rough estimate of compactibility using a pig (a large free chunk)." > + <doNotGenerate> > + | pig pork moved unmoved nmoved nunmoved | > + pig := self findAPig. > + pork := self bytesInObject: pig. > + moved := unmoved := nmoved := nunmoved := 0. > + self allOldSpaceObjectsFrom: pig do: > + [:o| | bytes | > + bytes := self bytesInObject: o. > + bytes <= pork > + ifTrue: > + [moved := moved + bytes. > + nmoved := nmoved + 1. > + pork := pork - bytes] > + ifFalse: > + [unmoved := unmoved + bytes. > + nunmoved := nunmoved + 1]]. > + ^{ self bytesInObject: pig. pork. moved. nmoved. unmoved. nunmoved }! > > Item was added: > + ----- Method: SpurMemoryManager>>biggies (in category 'compaction - analysis') ----- > + biggies > + "This method answers a sorted collection of the objects >= 1,000,000 bytes long, > + above the lowest large free chunk, sandwiched between nilObj and the end of memory." > + <doNotGenerate> > + | lowestFree biggies | > + lowestFree := SmallInteger maxVal. > + self allObjectsInFreeTreeDo: > + [:f| (self addressAfter: f) < endOfMemory ifTrue: [f < lowestFree ifTrue: [lowestFree := f]]]. > + biggies := SortedCollection new. > + self allOldSpaceObjectsFrom: lowestFree do: > + [:f| > + (self bytesInObject: f) >= 1000000 ifTrue: > + [biggies add: f]]. > + ^{{nilObj hex. #nil}}, (biggies collect: [:f| {f hex. self bytesInObject: f}]), {{endOfMemory hex. #endOfMemory}}! > > Item was added: > + ----- Method: SpurMemoryManager>>compactionIssues (in category 'compaction - analysis') ----- > + compactionIssues > + <doNotGenerate> > + "Compaction isn't working well. It rarely moves more than a few tens of kilobytes. Why? > + Load an image and before you run it, or just before a GC, run these anaylsis routines. > + e.g. > + self abstractCompaction #(63230272 75456 63210648) > + shows we can move 75456 bytes of objects, but that would use 63210648 of free space. > + i.e. there are lots of big free chunks in play, not many small ones that fit the bill. > + > + self largeFreeChunkDistribution > + #( #('16r31C788' #nil) > + #('16r1423AC0' 2061864) > + #('16r1B705E8' 1515200) > + #('16r1D31D20' 2011152) > + #('16r1F37818' 1491480) > + #('16r2225968' 1450512) > + #('16r24C92C8' 48575672) (16r24C92C8 + 48575672) hex '16r531C780' a free chunk > + #('16r531C788' #endOfMemory)) > + shows there's plenty of large free chunks. And the trailing 16-byte free chunk shows coallescing is not working properly. > + > + self biggies #(#('16r31C788' #nil) #('16r531C788' #endOfMemory)) > + shows there are no large objects to be moved. > + > + So... looks like compaction should hold onto the lowest large chunk and preferentially move objects into that. > + Let's call it a pig. Compaction needs to whittle away at the pig. > + > + e.g. > + self abstractPigCompaction #(2061864 0 2061864 18759 2018224 34757) > + shows we can move 18759 objects that will occupy 2018224 bytes into that > + low pig of 2061864 bytes."! > > Item was added: > + ----- Method: SpurMemoryManager>>findAPig (in category 'compaction - analysis') ----- > + findAPig > + "Answer a large low free chuink." > + <doNotGenerate> > + | pig | > + self allObjectsInFreeTreeDo: > + [:f| > + (self bytesInObject: f) >= 1000000 ifTrue: > + [(pig isNil or: [pig > f]) ifTrue: > + [pig := f]]]. > + ^pig! > > Item was added: > + ----- Method: SpurMemoryManager>>largeFreeChunkDistribution (in category 'compaction - analysis') ----- > + largeFreeChunkDistribution > + "This method answers a sorted collection of the free chunks >= 1,000,000 bytes long, > + sandwiched between nilObj and the end of memory (ignoring the large chunk often found at the end of the heap)." > + <doNotGenerate> > + | freeChunks | > + freeChunks := SortedCollection new. > + self allObjectsInFreeTreeDo: > + [:f| > + ((self addressAfter: f) < endOfMemory > + and: [(self bytesInObject: f) >= 1000000]) ifTrue: > + [freeChunks add: f]]. > + ^{{nilObj hex. #nil}}, (freeChunks collect: [:f| {f hex. self bytesInObject: f}]), {{endOfMemory hex. #endOfMemory}}! > > |
The notes at Eliot's blog here: http://www.squeakvm.org/svn/squeak/branches/Cog/image/Workspace.text
imply the difference is important when building a Cog Development Image. i.e.
|
Hi Tty,
On Fri, Feb 14, 2014 at 8:31 AM, gettimothy <[hidden email]> wrote:
VMMaker is the trunk Interpreter VM. It doesn't contain Cog. VMMaker-oscog is an old misnamed line of versions.
VMMaker.oscog is my Cog branch. Pharo has their own branch from this.
best, Eliot
|
In reply to this post by tty
On Fri, 14 Feb 2014, gettimothy wrote: (no quote thanks to gmail's empty first attachment) My question is why is the file named "..VMMaker.oscog-eem.618.mcz" instead of "VMMaker.oscog-eem.618.mcz"? Levente |
On Fri, Feb 14, 2014 at 3:09 PM, Levente Uzonyi <[hidden email]> wrote:
best, Eliot
|
In reply to this post by Levente Uzonyi-2
Hi Levente,
On Fri, Feb 14, 2014 at 3:09 PM, Levente Uzonyi <[hidden email]> wrote: --
I'm confused. What's the context? best, Eliot
|
On Fri, 14 Feb 2014, Eliot Miranda wrote: (again no qoute..) This is the context: > Eliot Miranda uploaded a new version of VMMaker to project VM Maker: > http://source.squeak.org/VMMaker/..VMMaker.oscog-eem.618.mcz > > ==================== Summary ==================== > > Name: ..VMMaker.oscog-eem.618 The name of the file begins with two dots. Why? Levente |
In reply to this post by Eliot Miranda-2
On 14 Feb 2014, at 23:28, Eliot Miranda <[hidden email]> wrote:
well, not completely, this one is the branch we (pharo) are using… keeping it in track (as much as possible) with VMMaker.oscog Esteban
|
In reply to this post by Levente Uzonyi-2
On Fri, Feb 14, 2014 at 8:03 AM, Levente Uzonyi <[hidden email]> wrote:
oops. definitely not. i must have mistyped during the commit dialog. fixed this. David, can you delete the
.VMMaker.oscog-eem.615 ..VMMaker.oscog-eem.616 ..VMMaker.oscog-eem.617 ..VMMaker.oscog-eem.618 packages?
best, Eliot
|
On Mon, Feb 17, 2014 at 12:22:00PM -0800, Eliot Miranda wrote: > > On Fri, Feb 14, 2014 at 8:03 AM, Levente Uzonyi <[hidden email]> wrote: > > > > Is it intentional that this package and the previous one are prefixed with > > dots? > > oops. definitely not. i must have mistyped during the commit dialog. > fixed this. David, can you delete the > > .VMMaker.oscog-eem.615 > ..VMMaker.oscog-eem.616 > ..VMMaker.oscog-eem.617 > ..VMMaker.oscog-eem.618 > > packages? Done. |
Thanks and sorry for the make work mistake... Eliot (phone) On Feb 17, 2014, at 4:01 PM, "David T. Lewis" <[hidden email]> wrote: > > On Mon, Feb 17, 2014 at 12:22:00PM -0800, Eliot Miranda wrote: >> >> On Fri, Feb 14, 2014 at 8:03 AM, Levente Uzonyi <[hidden email]> wrote: >>> >>> Is it intentional that this package and the previous one are prefixed with >>> dots? >> >> oops. definitely not. i must have mistyped during the commit dialog. >> fixed this. David, can you delete the >> >> .VMMaker.oscog-eem.615 >> ..VMMaker.oscog-eem.616 >> ..VMMaker.oscog-eem.617 >> ..VMMaker.oscog-eem.618 >> >> packages? > > Done. > |
Free forum by Nabble | Edit this page |