Literal Filter

Folks,

Spurred on by Esteban's recent post I thought I'd take a look at the
literal filer idea. It seems that, without too much effort, I've
managed to get something working (complete with some tests) in classes
STLInFiler and STLOutFiler.

Here is the filer output (or at least the start of it) for the
ClassBrowserShell view.

#('!STL ' 3 788558 10 STBViewProxy 8 ShellView 98 27 0 0 98 2 27131905
131073 416 0 524550 ColorRef 8 4294967295 0 533 0 0 0 416 788230
BorderLayout 1 1 410 8 Toolbar 98 25 0 416 98 2 8 1140851524 131073 560
0
482 8 4278190080 0 517 0 263174 Font 0 16 459014 LOGFONT 8 #[243 255
255 255
0 0 0 0 0 0 0 0 0 0 0 0 144 1 0 0 0 0 0 0 3 2 1 34 65 114 105 97 108 0
159 4
28 198 63 1 0 0 204 53 63 1 3 0 20 59 0 0 0 0 247 0 5 86 111 1] 328198
Point
193 193 0 560 482 656 8 4294903435 234 256 98 6 410 8 ReferenceView 98
14 0
560 98 2 8 1140850688 131073 848 0 0 0 5 0 0 0 848 1180166
ResourceIdentifier 576 8 'Smalltalk tools' 0 983302 MessageSequence 202
208
98 1 721670 MessageSend 8 #createAt:extent: 98 2 754 63 1 754 523 51 848
983302 WINDOWPLACEMENT 8 #[44 0 0 0 0 0 0 0 1 0 0 0 255 255 255 255 255
255
255 255 255 255 255 255 255 255 255 255 31 0 0 0 0 0 0 0 36 1 0 0 25 0
0 0]
98 0 754 193 193 0 27 8 'smalltalkTools' 410 864 98 14 0 560 98 2 8
1140850688 131073 1232 0 0 0 5 0 0 0 1232 930 576 8 'Class browser
tools' 0
978 202 208 98 1 1042 1072 98 2 754 585 1 754 161 51 1232 1138 8 #[44 0
0 0
0 0 0 0 1 0 0 0 255 255 255 255 255 255 255 255 255 255 255 255 255 255
255
255 36 1 0 0 0 0 0 0 116 1 0 0 25 0 0 0] 1184 1200 0 27 8
'classBrowserTools' 410 864 98 14 0 560 98 2 8 1140850688 131073 1488 0
0 0
5 0 0 0 1488 930 576 8 'Image tools' 0 978 202 208 98 1 1042 1072 98 2
754 1
1 754 63 51 1488 1138 8 #[44 0 0 0 0 0 0 0 1 0 0 0 255 255 255 255 255
255
255 255 255 255 255 255 255 255 255 255 0 0 0 0 0 0 0 0 31 0 0 0 25 0 0
0]
1184 1200 0 27 8 'imageTools' 234 256 1184 202 208 1184 234 240 1184 0
1 0
754 33 33 754 45 45 0 656198 1 FlowLayout 1 1 1 978 202 208 98 2 1042
1072
98 2 754 1 1 754 1187 51 560 1042 8 #updateSize 1184 560 1138 8 #[44 0
0 0 0
0 0 0 1 0 0 0 255 255 255 255 255 255 255 255 255 255 255 255 255 255
255
255 0 0 0 0 0 0 0 0 81 2 0 0 25 0 0 0] 98 3 1488 848 1232 1200 0 27 410
8
StatusBar 98 18 0 416 98 2 8 1140852996 1 2048 0 482 656 0 7 0 674 0 16
706
8 #[245 255 255 255 0 0 0 0 0 0 0 0 0 0 0 0 144 1 0 0 0 0 0 0 3 2 1 34
77
105 99 114 111 115 111 102 116 32 83 97 110 115 32 83 101 114 105 102 0
59 0
0 0 0 247 0 5 86 111 1] 754 193 193 0 2048 0 8 4294903617 234 256 98 6
853766 StatusBarItem 1 37 2048 0 0 787814 3 BlockClosure 0 0 1180966
CompiledExpression 5 1 8 UndefinedObject 8 'doIt' 8 '[:modified |
modified
ifTrue: [Smalltalk developmentSystem changedIcon imageIndex]]' 8 #[33
105 17
122 45 159 160 161 106 60 106] 721414 Association 8 #Smalltalk 48 8
#developmentSystem 8 #changedIcon 8 #imageIndex 2304 7 257 0 0 8
'modified'
2258 1 501 2048 0 8 BasicListAbstract 8 IconicListAbstract 1114638
STBSingletonProxy 8 IconImageManager 8 #current 8 'packageOfSelection'
2258
1 -1 2048 0 2544 2560 2592 8 'message' 98 3 2528 2272 2656 1115142
StatusBarNullItem 513 1 2048 0 0 978 202 208 98 1 1042 1072 98 2 754 1
929
754 1187 41 2048 1138 8 #[44 0 0 0 0 0 0 0 1 0 0 0 255 255 255 255 255
255
255 255 255 255 255 255 255 255 255 255 0 0 0 0 208 1 0 0 81 2 0 0 228
1 0
0] ... etc ...

You'll notice that 8 is the prefix for a following literal. Also,
SmallIntegers and Characters continue to be output as part of the prefix
rather than being expressed as separate literals (which would perhaps be
more "readable" but somewhat larger).

The #storageSize for the above literal array (as reported by Chris
Uppal's method) is around 21K. This is around 2/3 the size of the byte
array for the original STB! The load time is also slightly faster at
17ms compared to 18.5ms although I'm surprised this isn't faster still.

If we *can* manage to store resources as class methods by compiling
these literal arrays up (which we have not yet done) then we'll be able
to solve a number of issue at one fell swoop. Notably:

1) Browsing of Class references in resources will work.
2) As will browsing of symbol/selector references in resources.
3) Rename refactorings will work in resources automagically.
4) Resources will also be tied into .cls files and not need any other
external storage like the current .vu files.

The format is still pretty much the same as STB, however. This means
that it is likewise fragile with respect to class layout changes.
However, since it will be easier (and faster) to spot class references
in resources it might be possible to allow the development system to
warn when a dangerous layout change is proposed.

I'm not promising this is going to make it into D6 or the beta but it
is interesting nevertheless.

Best regards,

Andy Bower
Dolphin Support
www.object-arts.com

Hello Andy, Folks,

Andy Bower escribió:
> Folks,
> Spurred on by Esteban's recent post I thought I'd take a look at the
> literal filer idea. It seems that, without too much effort, I've
> managed to get something working (complete with some tests) in classes
> STLInFiler and STLOutFiler.

Great news! refactoring resources is one of the points where the
Dolphin RB can't get into, being it (the RB) one of the best RB around.

> The format is still pretty much the same as STB, however.

Yes, and this is curious, also, because when I said "literal", I was
trying to say "not binary" :-), because as a colleague said to me:
the resources as binary objects is the most C++'ish thing Dolphin
Smalltalk has.

What I would like is an "smalltalk expression" responsible of the
view construction (addSubView: etc...), considering that
programatically construction of Views is not very hard in Dolphin.

 > This means
> that it is likewise fragile with respect to class layout changes.
> However, since it will be easier (and faster) to spot class references
> in resources it might be possible to allow the development system to
> warn when a dangerous layout change is proposed.

Yes... the advance would be substantial for development and
maintenance, and can open an small door to internationalization of
views at "base" level.

> I'm not promising this is going to make it into D6 or the beta but it
> is interesting nevertheless.

Yes, it is. Thanks Andy for the dedication. Let's see how it evolves.


Best regards.

--
Esteban.

"Andy Bower" <[hidden email]> wrote in message
news:[hidden email]...
> Spurred on by Esteban's recent post I thought I'd take a look at the
> literal filer idea. It seems that, without too much effort, I've
> managed to get something working (complete with some tests) in classes
> STLInFiler and STLOutFiler.
...

This is a neat idea.  I think I like the idea of resources being treated
more like Smalltalk code.  I think it may also be worth exploring the idea
of including instance variable names to better handle shape changes.  I
suppose this could be a performance issue.  Perhaps the resource could use
instance variable names in the IDE, but be compiled to use indexes when
deployed?  I think it could be similar to the way structures are handled.
Some IDE and even user specified resources could always be compiled to use
indexes if speed in the IDE was more important than gracefully handling
shape changes.

Another approach might be to support the option of including a separate
Filer section containing an index to variable name map.  The variable index
could be used and the map section ignored unless there was an index count
miss-match, then the map could be used to put the values in the correct
slots.  The map section could optionally be stripped out during deployment.

Chris

Andy,

> The format is still pretty much the same as STB, however. This means
> that it is likewise fragile with respect to class layout changes.
> However, since it will be easier (and faster) to spot class references
> in resources it might be possible to allow the development system to
> warn when a dangerous layout change is proposed.
>
> I'm not promising this is going to make it into D6 or the beta but it
> is interesting nevertheless.

Am I correct in thinking that you have created a new type of filer, so
the existing binary filer would continue to work as before?  Backward
compatibility is important to me, at least for a while.  Hopefully you
can maintain that w/o holding up progress.  If it is helpful, you might
create a generic filer that accepts policies to provide different behaviors.

Have a good one,

Bill


--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Chris,

> This is a neat idea.  I think I like the idea of resources being treated
> more like Smalltalk code.  I think it may also be worth exploring the idea
> of including instance variable names to better handle shape changes.  I
> suppose this could be a performance issue.  Perhaps the resource could use
> instance variable names in the IDE, but be compiled to use indexes when
> deployed?  I think it could be similar to the way structures are handled.
> Some IDE and even user specified resources could always be compiled to use
> indexes if speed in the IDE was more important than gracefully handling
> shape changes.

Sounds good, with one caveat: I sometimes introduce version changes even
when layouts are unchanged.  It is a great way to force conversions w/o
a lot of ugly tests on type or other lazy conversions.

I will have more to say on this at Smalltalk Solutions.

Have a good one,

Bill


--
Wilhelm K. Schwab, Ph.D.
[hidden email]

> What I would like is an "smalltalk expression" responsible of the view
> construction (addSubView: etc...), considering that programatically
> construction of Views is not very hard in Dolphin.

I would like the same.
[...]

I'd like to see more in this direction.


Shaping

Bill,

> Am I correct in thinking that you have created a new type of filer,
> so the existing binary filer would continue to work as before?
> Backward compatibility is important to me, at least for a while.
> Hopefully you can maintain that w/o holding up progress.  If it is
> helpful, you might create a generic filer that accepts policies to
> provide different behaviors.

The existing binary filer is still present and is not deprecated. By
its nature the literal filer is not suitable for externalizing objects
outside of the image (since it effectively produces an array of literal
objects and not a bytestream that can be written to a file).

So, both binary and literal filers are available in the base image.

Best regards,

Andy Bower
Dolphin Support
www.object-arts.com

Andy,

> The existing binary filer is still present and is not deprecated. By
> its nature the literal filer is not suitable for externalizing objects
> outside of the image (since it effectively produces an array of literal
> objects and not a bytestream that can be written to a file).
>
> So, both binary and literal filers are available in the base image.

Sounds great!

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]