Cryptographic Primitives

Hi J J,

the "real" about closures depends on whom you ask and where you ask. For  
an example see

- http://www.google.com/search?q=%22Smalltalk+blocks+are+closures%22

/Klaus

On Sat, 07 Oct 2006 14:21:04 +0200, J J wrote:

tim Rowledge writes:
 > Actually I don't think JIT is a sensible description at all; it isn't  
 > accurate and it doesn't really indicate the advantages that Exupery  
 > could have. For example, no extra startup costs, easier extension/
 > maintenance/optimising because it is written in a sensible language,  
 > access to all of the reflexive capabilities of the system, trivial  
 > fallback  if the system is run on an unsupported platform  (it just  
 > doesn't do the compiles and leaves everything to the interpreter),  
 > potential for using a spoon-like inter-image communication system to  
 > allow the compiles to run in a different image/machine, and... well I  
 > could probably go on.

Well, most people only really understand the static compilation model
of C/C++, and the dynamic compilation model of VisualWorks and Java.
They call any form of dynamic compilation a JIT. Many so called JITs
compile in the background. I think IBM's Jalapino does background
thread compilation.

Exupery is in some ways closer to a Lisp compiler written in Lisp than
a Deutsch-Schiffman influenced JIT. So, you could say it's drawing on
a tradition that goes back to at least the '70s.

Help is always appreciated. Exupery is a test heavy project, this
should help people get up to speed. Tests are required for everything.

If anyone would like to help please join the Exupery mailing list
and ask.

  http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery

There's a list of things to do before 1.0 here:

  http://minnow.cc.gatech.edu/squeak/5793

If you're thinking of writing hundreds of primitives for speed then
helping out with Exupery is probably the best use of your time. If
you're thinking about writing tens of primitives then it's
debatable. If you're only thinking of writing a handful of primitives
the just write the primitive unless working on a compiler is it's own
reward.

Going back to crypto. From what I understand about SSL writing the
streaming cyphers as primitives should make sense for the next few
years. Writing public key primitives would require some serious
profiling first as they're only used once per session. But we all vote
with our time. Just don't get upset if in 5 years time you're
rewriting cyphers into Smalltalk from primitives for speed.

Bryce

They are real in every Smalltalk but Squeak. That's why we have #fixTemps.

Philippe

2006/10/7, Klaus D. Witzel <[hidden email]>:

Philippe Marschall writes:
 > They are real in every Smalltalk but Squeak. That's why we have #fixTemps.

There are two separate issues:
1) Why Anthony's VI4 work was rejected?
2) Why we haven't yet incorporated the new compiler work that Anthony
did then Markus has been maintaining?

1) is of interest only if you're interested in how the community has
reacted to performance improvements that cheated but got caught in the
past.

2) is much more interesting now. What is stopping us from adding the
new compiler work in Squeak 3.10? It would be very nice to see proper
closures in Squeak. What's stopping us?

Bryce

2006/10/7, [hidden email] <[hidden email]>:
> Philippe Marschall writes:
>  > They are real in every Smalltalk but Squeak. That's why we have #fixTemps.
>
> There are two separate issues:
> 1) Why Anthony's VI4 work was rejected?

You may only ask if you don't fear the answer.

> 2) Why we haven't yet incorporated the new compiler work that Anthony
> did then Markus has been maintaining?

Because it's not a drop in replacement yet:
- Decompilation of blocks doesn't work yet.
- Doesn't work for DoIt's in the debugger
- Error messages are bad. There is only one error message: 'token not
excpected'. Mathieu is working on this.
- No bootstrapping, can't compile itself. Mathieu is working on this.
- ... (whatever I forgot)

Don't get me wrong it's really cool for research and if you want to
hack the compiler or language but has disadvantages in day to day
usage. However work is being done to close the gap.

>> There are two separate issues:
>> 1) Why Anthony's VI4 work was rejected?

I do not know in fact. There were some implications on the stack  
representations.
But this was log time ago.

ESUG Summertalk succeeded to get a really smart guy helping there.
Mathieu is regularly working and improving the new compiler. Thanks  
Mathieu
So ESUG spent 1500 Euros (this is not a lot but we do it) to go  
slowly and
make sure that we could get a new compiler in a better shape.

The squeakfoundation is supporting the Summertalk effort.

Of course, if the foundation really wanted to collect some money for  
improving the
system this would help... But you know why I left it.

Stef

Sorry for not being clear.  By "real" I mean downward *and* upward usable
closures.  I.e. no "fixTemps".

Now I have seen some people say something like "well we aparently don't need
them".  But that is always a dangerous path.  C thought they didn't need
automatic memory management.  BASIC thought they didn't need recursion.  
Paul Graham thinks he has never needed Object orientation.

stephane ducasse a écrit :
YW Smalltalk is a really pretty language to work with.

> So ESUG spent 1500 Euros (this is not a lot but we do it)

It is enough for me, Many thanks ESUG :)

> to go slowly and
> make sure that we could get a new compiler in a better shape.

Yes and I have some idea for better error message that will be quite fast to implement

Mathieu writes:
 > > to go slowly and
 > > make sure that we could get a new compiler in a better shape.
 >
 > Yes and I have some idea for better error message that will be quite fast to implement

Mattieu, any chance you could either write a quick introduction
to the project or point to one that exists? It's great that you're
working on it. This is something that would be very nice for Squeak.

A little effort spent on promoting the project might encourage other
people to help, or at least let us know that work's going on.

Bryce

[hidden email] a écrit :
You have this one:
http://www.iam.unibe.ch/~scg/Teaching/Smalltalk/Slides/11Bytecode.pdf

In fact we will provide a web site to document your project
You can also subscribe to the NewCompiler mailing list:
http://lists.squeakfoundation.org/mailman/listinfo/newcompiler

If you want to see the what is going on you can go here:
http://smallwiki.unibe.ch/NewCompiler

To get the source:
http://www.squeaksource.com/NewCompiler.html

I have also a repository where package waiting there for approval:
http://www.squeaksource.com/NewCompilerPragma.html

We start to make a small scripting language call Sapphir:
http://www.squeaksource.com/Saphir.html
It is not yet fully support by all environment(for writting in the shell) but you can test it and
give us feedback. See unit test to see the syntaxe or juste ask.

        Math

[hidden email] wrote:
> There are two separate issues:
> 1) Why Anthony's VI4 work was rejected?

Anthony's work moved away from the linked-frame representation and put
(overlapping) frames into stack segments, and pretty much everyone who
looked at it didn't like the exposure of stack segments (there are some
interesting issues with stack manipulation when you do that). The
general feeling was that if a VM wants to play these tricks they should
be hidden under the hoods and the "user model" should remain simple
linked frames.

> 2) Why we haven't yet incorporated the new compiler work that Anthony
> did then Markus has been maintaining?

It's not complete. The last time I tried using it I gave up after the
umphteenth "Token not expected" error message (which is apparently the
only error message that SmaCC knows). Also, decompilation was never
completed, the compiler is about 4x slower than the current one etc.

Seriously, if you wanted closures a better bet may be to take the
existing infrastructure and hack that support in the current
compiler[*]. If you know what you're doing this takes a couple of weeks
but then it's done. The trouble with that new compiler is that while
it's used for a lot of exciting research there doesn't seem to be anyone
who is willing to spend the time to address the "boring" issues that are
so important for the non-research users of a compiler.

[*] And I may yet do that for Croquet because we have plenty of non-lifo
blocks which may need #fixTemps and I don't really want to explain to
people when and where exactly they need to do it. That's why I looked at
the new compiler in the first place. Unfortunately, given the choice
between (the relatively rare) #fixTemps and the (very common) "Token not
expected" error message, I choose to live with the former every time.


Cheers,
   - Andreas

Andreas Raab a écrit :
Witch boring issues are you talking about?

Mathieu writes:
 > > Seriously, if you wanted closures a better bet may be to take the
 > > existing infrastructure and hack that support in the current
 > > compiler[*]. If you know what you're doing this takes a couple of weeks
 > > but then it's done. The trouble with that new compiler is that while
 > > it's used for a lot of exciting research there doesn't seem to be anyone
 > > who is willing to spend the time to address the "boring" issues that are
 > > so important for the non-research users of a compiler.
 > >
 >
 > Which boring issues are you talking about?

Probably the ones you said you were working on. Like getting sensible
error messages. Though the performance may need some work too from
what Andreas says. Though, I'd trade compilation speed for full block
closures it might annoy me when loading big packages.

Bryce

On Sat, Oct 07, 2006 at 01:19:15PM -0700, Andreas Raab wrote:
> Seriously, if you wanted closures a better bet may be to take the
> existing infrastructure and hack that support in the current
> compiler[*]. If you know what you're doing this takes a couple of weeks
> but then it's done. The trouble with that new compiler is that while
> it's used for a lot of exciting research there doesn't seem to be anyone
> who is willing to spend the time to address the "boring" issues that are
> so important for the non-research users of a compiler.

Is there an explanation somewhere (swiki, prior message on the list,
etc) that explains how this might be done? I'm asking only out of
curiosity. This issue has been around for a long time, and I assumed
that it must be a really hard problem to solve or someone would have
done so by now.

Dave
 

Mathieu wrote:
>> Seriously, if you wanted closures a better bet may be to take the
>> existing infrastructure and hack that support in the current
>> compiler[*]. If you know what you're doing this takes a couple of weeks
>> but then it's done. The trouble with that new compiler is that while
>> it's used for a lot of exciting research there doesn't seem to be anyone
>> who is willing to spend the time to address the "boring" issues that are
>> so important for the non-research users of a compiler.
>
> Witch boring issues are you talking about?

Error messages, decompilation, speed. I'm glad to see you're going to
work on error messages, though.

Cheers,
   - Andreas

David T. Lewis wrote:
> Is there an explanation somewhere (swiki, prior message on the list,
> etc) that explains how this might be done? I'm asking only out of
> curiosity. This issue has been around for a long time, and I assumed
> that it must be a really hard problem to solve or someone would have
> done so by now.

The main thing that got adopted from Anthony is the way he represented
proper closures. I don't remember the exact details but it has been
worked out and if you need inspiration just look at the new compiler ;-)

Cheers,
   - Andreas

> I'd trade compilation speed for full block closures [although] it
> might annoy me when loading big packages.

     Of course, if you can load packages without using the compiler at
all (as in Spoon[1]), then that issue goes away.


-C

[1] http://netjam.org/spoon

--
Craig Latta
http://netjam.org/resume

Andreas Raab a écrit :
For speed I have made some test with MessageTally and find out that NewCompiler spend a lote of time
in SqueakScanner>>scanForToken.

This depende on SmaCC so may be we should use a better algorithm for the scanner. May be we can use
automaton.

But this should be disscus on the NewCompiler mailing list.

        Math

On Sat, 07 Oct 2006 23:18:00 +0200, Andreas Raab wrote:
> Mathieu wrote:
...
>>  Witch boring issues are you talking about?
>
> Error messages, decompilation, speed. I'm glad to see you're going to  
> work on error messages, though.

Andreas, have you looked at the implementor of #createBlock: and the  
associated BlockClosure class (both are in the 3.9 image). The latter has  
a class comment (I know, that's a rarity ;-) On the instance side, there's  
a message category #evaluating, equally interesting.

I doubt that you will ever use BlockClosure in one of your performance  
sensitive projects, especially when the messages you need are in {#value.  
#value:} etc.

/Klaus

> Cheers,
>    - Andreas
>
>

Klaus D. Witzel wrote:
> Andreas, have you looked at the implementor of #createBlock: and the
> associated BlockClosure class (both are in the 3.9 image). The latter
> has a class comment (I know, that's a rarity ;-) On the instance side,
> there's a message category #evaluating, equally interesting.

I have not. Thanks for pointing it out.

> I doubt that you will ever use BlockClosure in one of your performance
> sensitive projects, especially when the messages you need are in
> {#value. #value:} etc.

Not sure about this. There are issues with our current contexts, BIG
issues. So big that I'd very seriously consider accepting a slowdown in
exchange to fixing these problems.

Cheers,
   - Andreas