Re: [Squeak-VMdev] GC improvements

Just curious: Did we ever come to a conclusion re: GC improvements? I'm
asking because I *will* require proper weak array support for Tweak in
the not-too-far future and I'd like to see those changes rolled into
standard VMs as soon as possible.

Cheers,
   - Andreas

Andreas Raab <[hidden email]> wrote:

> Just curious: Did we ever come to a conclusion re: GC improvements? I'm
> asking because I *will* require proper weak array support for Tweak in
> the not-too-far future and I'd like to see those changes rolled into
> standard VMs as soon as possible.
It's an issue of time. I tried testing out the changes (which seem to function
adequately from the limited testing I managed to do) but there appeared to be a
massive performance hit.

First issue of time is that whilst trying to characterise that and see what I
could do about it I discovered a machine dependant ultra-weirdness in my time
grabbing code which made timing unreliable. (see an email from me some time ago
withexplanation of crystal frequency drift and net time correction etc)  I'm
still trying to find a way out of that hole.

Second issue is _my_ time of course. I have fulltime work right now on a quite
different issue and plenty of 'spare time' things that have to be done, like
most of us.

Third issue is that Ian pretty much has the vm baton right now to do whatever
the next step of 64bitting is; I imagine getting some C type stuff in to make
an equivalent >2Gb address fix for Ned's earlier code is among the work. Now
you control that somewhat, being his leader. Once that is passed back I can try
to get on with dealing with the gc code changes etc. I was hoping to be with
you dealing with this right now but that damn passport still hasn't arrived.

And of course you could gain some control over my time by simple application of
an employment contract. :-)


tim
--
Tim Rowledge, [hidden email], http://sumeru.stanford.edu/tim
Useful random insult:- Can't find log base two of 65536 without a calculator.

Tim wrote (re: GC changes):
> It's an issue of time. I tried testing out the changes (which seem to function
> adequately from the limited testing I managed to do) but there appeared to be a
> massive performance hit.

I just verified it and like I suspected, the performance hit has
absolutely nothing to do with the GC changes. We lost a decent amount of
speed somewhere - anyone having ideas? Here are the micro benchmarks
from my machine:

3.7.1:  ~200,000,000 bytecodes/sec; 5,800,000 sends/sec'
3.8.1:  ~150,000,000 bytecodes/sec; 5,000,000 sends/sec'

(this is fairly consistent, regardless of whether the GC changes are
applied or not)

> Second issue is _my_ time of course. I have fulltime work right now on a quite
> different issue and plenty of 'spare time' things that have to be done, like
> most of us.

No problem. Just as long as we share a repository - it is only
problematic because things somehow "get to you" and then at some point
you point to a new version at your website and that process isn't very
transparent. If we'd use a shared repository (squeaksource?) we could
fold in changes as we see fit and then have a place where it's clear
what the latest version is.

> Third issue is that Ian pretty much has the vm baton right now to do whatever
> the next step of 64bitting is; I imagine getting some C type stuff in to make
> an equivalent >2Gb address fix for Ned's earlier code is among the work. Now
> you control that somewhat, being his leader. Once that is passed back I can try
> to get on with dealing with the gc code changes etc. I was hoping to be with
> you dealing with this right now but that damn passport still hasn't arrived.

I talked to Ian - as far as he is concerned the changes are finished.
There is still lots of stuff to be done but this falls into the
responsibility of either the VM or the plugin maintainer (like, for
example, making the LargeInt plugin work on 64bit).

Cheers,
   - Andreas

> I just verified it and like I suspected, the performance hit has
> absolutely nothing to do with the GC changes. We lost a decent amount of
> speed somewhere - anyone having ideas?

All right, found it. The gnuifier still matched against "int
interpret()" where it has to match against "sqInt interpret()" now. All
of you guys using awk-style gnuification might want to check that it's
doing the right thing.

So now speed is back on par with the 3.7 VMs (again, with or without GC
changes).

Cheers,
   - Andreas

In message <[hidden email]>
          Andreas Raab <[hidden email]> wrote:

> > I just verified it and like I suspected, the performance hit has
> > absolutely nothing to do with the GC changes. We lost a decent amount of
> > speed somewhere - anyone having ideas?
>
> All right, found it. The gnuifier still matched against "int
> interpret()" where it has to match against "sqInt interpret()" now. All
> of you guys using awk-style gnuification might want to check that it's
> doing the right thing.
>
Excellent news for those platforms using gnuifier. It's constantly amazing how
delicate performance is.

Now if only I can get my timing subsystem working properly so I can find out
where the issue is on my machine. :-(


tim
--
Tim Rowledge, [hidden email], http://sumeru.stanford.edu/tim
"My name is Inigo Montoya. You killed my parent process. Prepare to vi!"

In message <[hidden email]>
          Andreas Raab <[hidden email]> wrote:

>
> No problem. Just as long as we share a repository - it is only
> problematic because things somehow "get to you" and then at some point
> you point to a new version at your website and that process isn't very
> transparent. If we'd use a shared repository (squeaksource?) we could
> fold in changes as we see fit and then have a place where it's clear
> what the latest version is.
That's a good idea, though squeaksource doesn't seem to be very reliable right
now and the guys have mentioned completely reworking it. Perhaps a semi-public
MC repo on hplab?


> I talked to Ian - as far as he is concerned the changes are finished.
Hmm, I had the distinct impression he had things he wanted to do from various
emails. Not least was some mention of applying the equivalent fixes to make 2Gb
addresses not be a problem.

> There is still lots of stuff to be done but this falls into the
> responsibility of either the VM or the plugin maintainer (like, for
> example, making the LargeInt plugin work on 64bit).
And FloatArray.

Oh and since you used the new VMMakerTool UI layout, was it reasonably logical
for you? And nobody responded on the issue I raised a while ago about pushing
this out as the official 3.8 VM package, with the implied requirement to get
some of the stuff into the 3.8 image RSN.


tim
--
Tim Rowledge, [hidden email], http://sumeru.stanford.edu/tim
Useful Latin Phrases:- Noli me vocare, ego te vocabo = Don't call me, I'll call
you.

Hi -

> That's a good idea, though squeaksource doesn't seem to be very reliable right
> now and the guys have mentioned completely reworking it. Perhaps a semi-public
> MC repo on hplab?

How would we do this? Can we set up some WebDAV or so? Alternatively we
might be able to host it on source.impara.de but I would have to talk to
Michael about whether that's considered acceptable use of Impara resources.

>>I talked to Ian - as far as he is concerned the changes are finished.
>
> Hmm, I had the distinct impression he had things he wanted to do from various
> emails. Not least was some mention of applying the equivalent fixes to make 2Gb
> addresses not be a problem.

I don't know. Ian?

>>There is still lots of stuff to be done but this falls into the
>>responsibility of either the VM or the plugin maintainer (like, for
>>example, making the LargeInt plugin work on 64bit).
>
> And FloatArray.

And lots of others ;-) No doubt about it. This is an incremental process
which will need plenty of time to work out. One thing we might do though
is to comment on what plugins are supposed to be 64bit clean and which
one's aren't.

> Oh and since you used the new VMMakerTool UI layout, was it reasonably logical
> for you? And nobody responded on the issue I raised a while ago about pushing
> this out as the official 3.8 VM package, with the implied requirement to get
> some of the stuff into the 3.8 image RSN.

I certainly liked the VMMaker layout. For someone who's seen VMMaker
before it's pretty obvious. Except from the issue that changing forth
and back from/to 64bit does not seem to force regeneration of the VM nor
does it seem to recreate interp.h (possibly the same problem).

While integrating the GC changes, there were a few things I needed to
fix though, amongst which were:
- MiscPrimitivePlugin needs to refer to ByteString now
- MessageNode>>asTranslatorNode needs to use #isSymbol
- #(nil true false) in literal arrays
I can send you an MC with the changes if you'd like (or if we had a repo
I could upload it ;-)

Cheers,
   - Andreas

On Apr 22, 2005, at 9:34 PM, Andreas Raab wrote:

This is what I do for my wiresong repository. Apache2 with mod_dav
works quite well. You can do fine-grained access controls, and allow
read/write/both access by username, domain name, IP address etc. Also,
you can create repository-only accounts; no need for shell access.

I don't run SqueakSource, but I can host a basic repository if folks
would find that helpful.

Colin

I've a smalltalk version of the gnuifier that David P did.

On Apr 22, 2005, at 5:47 PM, Tim Rowledge wrote:
--
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

In message <[hidden email]>
          Andreas Raab <[hidden email]> wrote:

>
>
> I certainly liked the VMMaker layout. For someone who's seen VMMaker
> before it's pretty obvious. Except from the issue that changing forth
> and back from/to 64bit does not seem to force regeneration of the VM nor
> does it seem to recreate interp.h (possibly the same problem).
D'oh! Good catch. Yes, flipping the switch ought to do something to invalidate
the core vm (at least).

tim
--
Tim Rowledge, [hidden email], http://sumeru.stanford.edu/tim
Useful random insult:- Living proof that nature does not abhor a vacuum.

Am 23.04.2005 um 05:22 schrieb Colin Putney:

You do not even need full-blown WebDAV support, enabling HTTP PUT for
some directory is sufficient. You will want to restrict write access
using basic authorization:

         Alias /vmmaker /path/to/repository
         <Directory /path/to/repository >
             PUT_EnablePut On
             PUT_EnableDelete Off
             PUT_DirMask 777
             AuthType Basic
             AuthName vmmaker
             AuthUserFile /path/to/.htpasswd
             <Limit PUT>
                    require valid-user
             </Limit>
             Options +Indexes
         </Directory>

- Bert -

On Apr 23, 2005, at 02:08, Bert Freudenberg wrote:

> You do not even need full-blown WebDAV support, enabling HTTP PUT for
> some directory is sufficient.

Thanks for the config example.  I'm using Apache 2 (for SVN support) so
unfortunately it isn't quite what's needed.  OTOH...

PUT is now enabled for squeak.hpl.hp.com/vmmaker with user 'vmmaker',
password from me on request (by email).  Don't blame me (or expect
instantaneous repairs) if my put.cgi script is incompatible with your
client and/or if someone takes down the entire site via this hole.

It wasn't clear what you wanted to put in this repository or what you
wanted it called, so I called it 'vmmaker'.  If that's wrong, let me
know and I'll rename it.  If you want the contents included in our
daily backups, let me know.

(Why on Earth anyone would prefer http/PUT over FTP is a mystery to me,
but if that's what you want then that's what you've got. ;-)

Have fun!

Cheers,
Ian

Am 27.04.2005 um 00:03 schrieb Ian Piumarta:

> (Why on Earth anyone would prefer http/PUT over FTP is a mystery to
> me, but if that's what you want then that's what you've got. ;-)

Well, Monticello would have worked with FTP, too, as far as I know ...

- Bert -

> (Why on Earth anyone would prefer http/PUT over FTP is a mystery to me,
> but if that's what you want then that's what you've got. ;-)

(*jumping up and down eagerly*) Oh, I can answer this: It's called port
blocking!!! So unless you a) convince corporate IT to allow ftp over the
external wireless or b) convince corporate IT to plug my non-HP machine
into the internal HP network (for both of which the chances are exactly
zero) I'll take HTTP/PUT ;-)

Cheers,
   - Andreas