Concurrent Futures

[hidden email] wrote:
> You'd need to serialise object creation and accessing the root table
> in the write barrier. That may be possible without too much work but
> there's likely to be some overhead.

Yes. Such is the price of shared state concurrency.

> Providing a parallel object memory as part of a garbage collector
> rewrite that speed up single CPU code should be possible. The major
> design change would be changing the write barrier from a remembered
> set to card marking. That unfortunately might make it necessary to
> separate pointer object space from byte storage space.

Given the sensitivity of GC algorithms in real-world situations,
rewriting the garbage collector is not my understanding of a practical
approach. The overhead of using an atomic allocator/GC mechanism can be
reduced to a single decrement and test for the default single-threaded
case, so that the price would only be paid when you run multiple
threads. That seems like more straight-forward approach in particular if
one can tweak the GC parameters to run GC less often (and incur the
overhead less often).

Cheers,
   - Andreas

On 01/11/2007, Jecel Assumpcao Jr <[hidden email]> wrote:
Yes, without barriers the following code simply will not work as desired:

| a b |
a := (1 to: 20) collect: [ :i | i * i ].
Interval removeSelector: #collect: .
b := (1 to: 20) collect: [ :x | x + 7 ].
^ a + b


--
Best regards,
Igor Stasenko AKA sig.

On 31-Oct-07, at 9:39 AM, Igor Stasenko wrote:


Hah!

Once upon a time we had platform code 'files' in the image. In  
particular, the Mac files. Everyone else had to go through irritating  
and time consuming gymnastics to build a vm.

You couldn't edit these 'files' sensibly because Squeaks text editing  
ad gc has conniptions when you try to edit large strings. The changes  
log got polluted by massive chunks of text with each edit.  It was not  
fun. People complained incessantly about not having  vm code in a  
sourcecode control system. The they still bitch about not having all  
the vm code that way, about how dreadfull it is to have to actually  
*run squeak* in order to build a vm.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: EIV: Erase IPL Volume

On 01/11/2007, tim Rowledge <[hidden email]> wrote:
Heh, then don't put large strings. We all know that such code stinks :)

> The changes
> log got polluted by massive chunks of text with each edit.  It was not
> fun. People complained incessantly about not having  vm code in a
> sourcecode control system.

Huh? What does MC for?
I could complain about keeping one plugin code divided by two
different source control systems. It seems very impractical.

> The they still bitch about not having all
> the vm code that way, about how dreadfull it is to have to actually
> *run squeak* in order to build a vm.
>
Yeah, and i'm the one of them. ;) I wrote a small plugin which
requires a platform dependent code.. To my experience its much easier
to code it in external editor and at some point, after debugging, put
it in some method as string and forget about it.
Then it can be shipped with plugin code rather than shipped in two
different parts with tons of instructions how and where to unpack
sources, what modify in makefiles e.t.c e.t.c..

Ohh, i really don't know.. Maybe we should step aside and use idst for
building new VM?
Ian mentioned previously that he will welcome an efforts in this way.
But this means refactoring all VM code from scratch..

I'm just asking myself, when we could finally get rid of burden of C?

--
Best regards,
Igor Stasenko AKA sig.

On 31-Oct-07, at 5:37 PM, Igor Stasenko wrote: Do feel free to travel back in time ten years and try to persuade SqC  
to change things.

>
>
>> The changes
>> log got polluted by massive chunks of text with each edit.  It was  
>> not
>> fun. People complained incessantly about not having  vm code in a
>> sourcecode control system.
>
> Huh? What does MC for?
And similarly, feel free to travel back in time so that MC can be  
available ten years ago.

>
>
> I'm just asking myself, when we could finally get rid of burden of C?

Never. Not going to happen. Probably against the lore.

Yes, we could probably rewrite a lot of code currently in C source  
files and put it into Slang methods. Yes, we could probably improve  
Slang (we tried to get some of that done at Interval but ran out of  
time) to be more friendly. Yes, we could do lots of things. Got time  
to do them? Or money to pay me/Craig/Andreas/Ian/John/Mike/Bryce/
whoever to do it fulltime? That's the kind of thing that would be  
required to be able to make any major changes


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: ESC: Emulate Small Child

tim Rowledge wrote:
> Yes, we could probably rewrite a lot of code currently in C source files
> and put it into Slang methods. Yes, we could probably improve Slang (we
> tried to get some of that done at Interval but ran out of time) to be
> more friendly. Yes, we could do lots of things. Got time to do them? Or
> money to pay me/Craig/Andreas/Ian/John/Mike/Bryce/whoever to do it
> fulltime? That's the kind of thing that would be required to be able to
> make any major changes

Well, but let's not throw out the baby with the bath water. Improvements
would be welcome, in particular if they are easy to review and to
integrate. It is probably unwise to start this as the
lets-rewrite-the-vm-and-its-tools-from-scratch approach but there are
plenty of things that we could do better. For example, I would welcome a
patch that enables the code generator to optionally build the entire VM
as an object. That'd be a very nice stepping stone towared a
multi-threaded VM and can probably be done in a fairly incremental way.

Cheers,
  - Andreas

A Slang to Squeak refactoring tool, perhaps...

On Oct 31, 2007, at 6:00 PM, Andreas Raab wrote:

Well building the interpreter.c file to use a single structure for  
globals and then setup a pointer to that allocated structure  
somewhere is trivial as compared to the other ideas kicked about.
However to complete this concept is you need to move the support code  
and plugins to use memory hung off the interpreter structure for  
their globals, that is a more time consuming
process, but certainly not difficult, unlike oh concurrent multi-
threaded GC algorithms.


--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

On Nov 1, 2007, at 2:10 , John M McIntosh wrote:

The idea was to use a GIL for that problem.

- Bert -

On Oct 31, 2007, at 6:00 PM, Andreas Raab wrote:


I had some code a while ago that generated the entire VM as an  
Objective-C object, with the actual class hierarchy (ObjectMemory ->  
Interpreter  ) still intact.  It interpreted a couple of thousand byte  
codes until it hit a function that had been generated from literal C  
code embedded in the Smalltalk code for the VM.  I gave up at that  
point because there turned out to be quite a bit of that literal code  
that would need to be rewritten, as it would probably need to be for  
any such effort unless these C snippets have since been replaced.

Now that we seem to  be seriously discussing turning the VM into an  
object, I would suggest at least having a very close look at  
leveraging Objective-C, and yes, I can hear the virtual groans ;-)  
Anyhow, here it goes:

First, unlike the C++ approach there are no problems in terms of  
linkage compatibility:  Objective-C behave just like C in that  
respect.  I also don't see any speed issues, because Objective-C  
methods are just C functions with some extra metadata and some  
implicit arguments.  So functions can still be called and inlined  
pretty much exactly the way they are now.  (This would entail either  
generating the functions and adding the metadata to make them known to  
the runtime or generating methods and getting function pointers from  
the runtime.  Either approach would work, with the former possibly  
retaining better C compatibility).

Speaking of compatibility:  despite the rumors, Objective-C is not an  
Apple/MacOS-X thing.  It comes with gcc and AFAIK most every 32 (and  
64?) bit platform gcc supports.  Furthermore, as I alluded to before,  
and unlike C++,  Objective-C isn't really a new language per-se, it  
really is just C with an object-runtime library that works just as  
well from C and largely optional syntactic sugar.

So if there's so little difference, what's the upside?   Well, the  
runtime does come with all or most of the niggly details you need for  
on OO system.  So on the purely practical side, you can instantiate  
multiple objects (interpreters, plugins), load classes (plugins  
again).   If you have multiple instances of the interpreter, you can  
also have subclasses of the interpreter, override methods on-the-fly,  
do introspection etc.  Plugins can be just objects, with the plugin  
interface regular messages sends.  Etc.

Then there's all that with syntactic and semantic similarity, passing  
messages between the VM and the programs it is running (nice  
metacircularity...) and so on and so forth.

Anyway, just my € 0.02

Marcel

Igor Stasenko wrote:
> Or as i proposed in earlier posts, the other way could be to schedule
> all primitive calls, which currently don't support multi-threading to
> single 'main' thread.

I thought about this a little more today and decided this is indeed the
better way to go. It preserves all the semantics that currently exist in
Squeak with regards of running in the UI thread etc. It is also trivial
to implement since we'll need some sort of thread identifier per VM
instance so the test can be done without any additional synchronization
overhead.

So the (updated) steps to the multi-threaded VM would then be:

1) Objectify the Interpreter by making the code generator generate the
appropriate code.

2) Implement the "failure mode" for calling primitives from non-primary
threads and possibly implement the first few plugins that are
multi-threaded (files, sockets, ffi come to mind).

And then, either:

3a) For message passing concurrency: Implement a "load image" primitive
which allows loading and running another image in parallel.

3b) For shared state concurrency: Implement allocation and GC sync points.

Any interest in actually doing this? I think I might even be able to
find some funding for a project like this if someone has a bit of spare
time to work on it. If you are interested, let me know off-list.

Cheers,
   - Andreas

"Michael Rueger" <[hidden email]> wrote:

> there is a new version of Yaxo up at
> http://source.impara.de/infrastructure/XML-Parser-mir.10.mcz

Thank you very much for your prompt fix!
 
> You need the two attached 3.8.2/3.10 fixes for the new package to work.

> Please test the new version. For now I only tried to verify against some
> examples I had readily available.
>
Here are my first test results:
1. Squeak 3.9 #7067 does not contain the classes
    CharacterSetComplement  and WideCharacterSet,
    therefore I could not immediately use your code with that
    version. To proceed, I dropped works with 3.9 for now
    and moved to 3.10beta #7158.

2. In Squeak 3.10beta #7158 I could try your code. The
    surprising result is, that a structured xml element with
    subelements can be parsed, but in the parse tree, all
    subelements are missing.  

Problem analysis:
1. During parsing, a new element is inserted into its parent
in one of the methods
XMLDOMParser>>startElement:attributeList:
XMLDOMParser>>startElement:namespaceURI:namespace:attributeList:

The relevant statement is.
   self top addElement: newElement

addElement:  is defined in XMLNodeWithElements
where we find:

addElement: element
   self elements add: element

Method #elements  is an accessor with lazy initialization
in XMLNodeWithElements, but in the subclass XMLElement
it is now a method that uses a #select: to construct a new
collection.

Conclusion:  We do not add into the instance variable
element, but in a collection that was constructed on the fly
and that is subsequently dropped.

Proposal of a work around.
As  #elements does not answer an lazily initialized
instance variable, we cannot use in XMLElement
the #addElements: method that we inherit from
XMLNodeWithElements.  

Rather, I use these instance methods in XMLElement:

addContent: contentString
 super elements add: contentString


addElement: element
 super elements add: element

The first method is a modification of your method,
the second one is added.


With this change, I can correctly build the parse
tree for xml elements with subelements - also for
xml elements that mix #PCDATA (that is, strings)
with structured elements.

Comments to these proposals are of cource
welcome. Perhaps you can find a better solution.

By the way: Do we have any tests for the XML parser?
At this moment I am deeply involved in xml, that would
be a good moment to write some tests. Any interest?

Greetings
Boris

Boris.Gaertner wrote:

> Here are my first test results:
> 1. Squeak 3.9 #7067 does not contain the classes
>    CharacterSetComplement  and WideCharacterSet,
>    therefore I could not immediately use your code with that
>    version. To proceed, I dropped works with 3.9 for now
>    and moved to 3.10beta #7158.

Yes, those are changes that came in in 3.10.
And I just realized that I need to include them in the 3.8.2 fixes as well.

> 2. In Squeak 3.10beta #7158 I could try your code. The
>    surprising result is, that a structured xml element with
>    subelements can be parsed, but in the parse tree, all
>    subelements are missing.
> Problem analysis:

Thanks for tracking this down. I will look at your proposed changes.

> By the way: Do we have any tests for the XML parser?
> At this moment I am deeply involved in xml, that would
> be a good moment to write some tests. Any interest?

There is a huge (generated) test library that I need to dig out again
and make available.

Michael

On 01/11/2007, Andreas Raab <[hidden email]> wrote:
Writing a generic threading framework comes in mind.
A brief description:
- each object in system should have a way how to get it's assigned
thread id (and how assign it , of course).
- if we see, that object having assigned thread, that means, that this
object is 'captured' by given thread for processing, and we need to
schedule new messages to that thread.

Early i proposed this approach only for smalltalk objects and their
active contexts, but this concept can be easily expanded to virtually
any objects in system (Interpreter/GC/primitives).

For instance, a concurrent GC mark phase can be implemented by following:
- we have a single thread which dedicated for marking
- for each object we want to mark we should send a 'mark' message to it.
If given object is not 'busy' (not having a thread id set) we simply
set it and begin marking its references.
if its already set, then we schedule a mark message to the thread
where it currently assigned and can continue with own queue (mark
other objects).
You can see that with this concept we can virtually mix an interpreter
'sends' with VM low-level 'sends'.
In that way i even thinking what if not having a dedicated thread for
marking, just start it in some single thread (when we have a GC
criteria met) and let it go until it converges.
Eventually if we push further, we can come up with parallel marking,
then we don't need to have a 'sync' points and never need a GIL.
I'm on board :)

> Cheers,
>    - Andreas
>
>


--
Best regards,
Igor Stasenko AKA sig.

----- Original Message -----
From: "Igor Stasenko" <[hidden email]>
To: "The general-purpose Squeak developers list"
<[hidden email]>
Sent: Thursday, November 01, 2007 8:21 AM
Subject: Re: Thoughts on a concurrent Squeak VM


+1.

Please expose the ability for a use to do a scheduled move of an object from
one thread to another thread.


> You can see that with this concept we can virtually mix an interpreter
> 'sends' with VM low-level 'sends'.

Yes, indeed.

On 01/11/2007, Rob Withers <[hidden email]> wrote: I now writing a small code snippet in ST to show how it will look like.

>
> > You can see that with this concept we can virtually mix an interpreter
> > 'sends' with VM low-level 'sends'.
>
> Yes, indeed.
>
>
>
>


--
Best regards,
Igor Stasenko AKA sig.

This report is for Squeak3.10beta #7158

I ran into problems when I tried to load code
into a fresh image. The method

CharacterSetComplement>> hash

sends the message bitXOr: which is nowhere
defined. I think the correct writing of the message
is  bitXor:

Greetings
Boris

El 11/1/07 2:46 PM, "Boris.Gaertner" <[hidden email]> escribió:


Very thanks, Boris.
I confirm method have four on 7130 , Integer, LargePositiveInteger,
SmallInteger, ThirtyTwoBitRegister so I fix tomorrow.
Edgar

Is on new blue http://ftp.squeak.org/3.10/Squeak3.10.1beta.7141.zip

Edgar

Boris.Gaertner a écrit :
Thanks!

Sure I clicked one bit off in correction menu...
Obviously, this class lacks testings, among its friends of
Collections-Support, only Association has an AssociationTest!

Too bad...
Because, WideCharacterSet also has a buggy hash.

Nicolas