egg sucking alert

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

egg sucking alert

Eliot Miranda-2
 
Hi All,

    just a simple message about 64-bit code.  Don't use int.  Please use long, unsigned long, sqInt or usqInt for variables that are of the machine's natural word length (sqInt is a synonym for long and usqInt is a synonym for unsigned long).  On most, if not all 64-bit C compilers int is a 32-bit datatype.  Therefore code that tries to pass a pointer through an int parameter will end up truncating the pointer, with potentially disastrous results.  Here's an example I'm fixing now.

In the AsynchFilePlugin the standard header defines everything as int:

int asyncFileClose(AsyncFile *f);
int asyncFileOpen(AsyncFile *f, int fileNamePtr, int fileNameSize, int writeFlag, int semaIndex);
int asyncFileRecordSize();
int asyncFileReadResult(AsyncFile *f, int bufferPtr, int bufferSize);
int asyncFileReadStart(AsyncFile *f, int fPosition, int count);
int asyncFileWriteResult(AsyncFile *f);
int asyncFileWriteStart(AsyncFile *f, int fPosition, int bufferPtr, int bufferSize);

but the fileNamePtr and bufferPtr parameters are really pointers passed as integers.  These should be declared as "long", "sqInt", et al.  long is defines as an integral type large enough to hold a pointer.

If every variable had been declared as sqInt no change would be necessary.  As it is I'm changing just those two parameters to long, to avoid the noise.

Our code base is littered with such issues.  Hence before I can get a 64-bit Spur stack VM running I have to hack, hack, hack.
--
cheers,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

timrowledge


On 10-12-2014, at 3:13 PM, Eliot Miranda <[hidden email]> wrote:

> Hi All,
>
>     just a simple message about 64-bit code.  Don't use int.  Please use long, unsigned long, sqInt or usqInt for variables that are of the machine's natural word length

That’s stuff we dealt with a long time ago, surely? The entire idea of introducing the sqInt/usqInt etc type defines was to allow the old 64bit plain interp vm to build. Yeah, this is what we wrote about it -
http://squeakvm.org/squeak64/ Wholly Khao - ten years ago.

So the interesting question is how did ‘bad’ stuff get to be there still?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: SFA: Seek Financial Assistance


Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

Eliot Miranda-2
 
Hi Tim,

On Wed, Dec 10, 2014 at 3:22 PM, tim Rowledge <[hidden email]> wrote:


On 10-12-2014, at 3:13 PM, Eliot Miranda <[hidden email]> wrote:

> Hi All,
>
>     just a simple message about 64-bit code.  Don't use int.  Please use long, unsigned long, sqInt or usqInt for variables that are of the machine's natural word length

That’s stuff we dealt with a long time ago, surely? The entire idea of introducing the sqInt/usqInt etc type defines was to allow the old 64bit plain interp vm to build. Yeah, this is what we wrote about it -
http://squeakvm.org/squeak64/ Wholly Khao - ten years ago.

So the interesting question is how did ‘bad’ stuff get to be there still?

There are not too many occurrences.  I'm down to 55 pointer size cast issues in 7 plugins, at eats on linux.  That's not too bad, and shrinking as we speak ;-)


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: SFA: Seek Financial Assistance





--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

timrowledge


On 10-12-2014, at 3:31 PM, Eliot Miranda <[hidden email]> wrote:
>
> There are not too many occurrences.  I'm down to 55 pointer size cast issues in 7 plugins, at eats on linux.  That's not too bad, and shrinking as we speak ;-)

That’s good, but there really ought not have been any for ten years or so. Something got messed up at some point. That always worries me because it may indicate more serious backsliding that we just haven’t noticed yet.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Justify my text?  I'm sorry but it has no excuse.


Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

johnmci
In reply to this post by timrowledge
 
Gulity guilty.. but 9 years, 6 months ago... 

But I think you will find we migrated to the sqmacunixfileinterface.c about 8 years 10 months ago and abandon the older carbon file FSSPec and hfs+ file names. The macunixfile interface is a copy of the unix file interface but has the logic needed to deal with macintosh alias files, not to be confused with unix alias file nodes...  

So not sure you need this plugin, as I'd think the unix async one would be the target? 


On Wed, Dec 10, 2014 at 3:22 PM, tim Rowledge <[hidden email]> wrote:


On 10-12-2014, at 3:13 PM, Eliot Miranda <[hidden email]> wrote:

> Hi All,
>
>     just a simple message about 64-bit code.  Don't use int.  Please use long, unsigned long, sqInt or usqInt for variables that are of the machine's natural word length

That’s stuff we dealt with a long time ago, surely? The entire idea of introducing the sqInt/usqInt etc type defines was to allow the old 64bit plain interp vm to build. Yeah, this is what we wrote about it -
http://squeakvm.org/squeak64/ Wholly Khao - ten years ago.

So the interesting question is how did ‘bad’ stuff get to be there still?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: SFA: Seek Financial Assistance





--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================
Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

Eliot Miranda-2
 


On Wed, Dec 10, 2014 at 3:51 PM, John McIntosh <[hidden email]> wrote:
 
Gulity guilty.. but 9 years, 6 months ago... 

But I think you will find we migrated to the sqmacunixfileinterface.c about 8 years 10 months ago and abandon the older carbon file FSSPec and hfs+ file names. The macunixfile interface is a copy of the unix file interface but has the logic needed to deal with macintosh alias files, not to be confused with unix alias file nodes...  

So not sure you need this plugin, as I'd think the unix async one would be the target? 

I'm talking about platforms/Cross/plugins/AsynchFilePlugin/AsynchFilePlugin.h  (amongst other offenders).



On Wed, Dec 10, 2014 at 3:22 PM, tim Rowledge <[hidden email]> wrote:


On 10-12-2014, at 3:13 PM, Eliot Miranda <[hidden email]> wrote:

> Hi All,
>
>     just a simple message about 64-bit code.  Don't use int.  Please use long, unsigned long, sqInt or usqInt for variables that are of the machine's natural word length

That’s stuff we dealt with a long time ago, surely? The entire idea of introducing the sqInt/usqInt etc type defines was to allow the old 64bit plain interp vm to build. Yeah, this is what we wrote about it -
http://squeakvm.org/squeak64/ Wholly Khao - ten years ago.

So the interesting question is how did ‘bad’ stuff get to be there still?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: SFA: Seek Financial Assistance





--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================




--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

johnmci
 
Unloved, no owner? plugins don't get much love... "(9 years, 8 months ago) by piumarta "


On Wed, Dec 10, 2014 at 3:55 PM, Eliot Miranda <[hidden email]> wrote:
 


On Wed, Dec 10, 2014 at 3:51 PM, John McIntosh <[hidden email]> wrote:
 
Gulity guilty.. but 9 years, 6 months ago... 

But I think you will find we migrated to the sqmacunixfileinterface.c about 8 years 10 months ago and abandon the older carbon file FSSPec and hfs+ file names. The macunixfile interface is a copy of the unix file interface but has the logic needed to deal with macintosh alias files, not to be confused with unix alias file nodes...  

So not sure you need this plugin, as I'd think the unix async one would be the target? 

I'm talking about platforms/Cross/plugins/AsynchFilePlugin/AsynchFilePlugin.h  (amongst other offenders).



On Wed, Dec 10, 2014 at 3:22 PM, tim Rowledge <[hidden email]> wrote:


On 10-12-2014, at 3:13 PM, Eliot Miranda <[hidden email]> wrote:

> Hi All,
>
>     just a simple message about 64-bit code.  Don't use int.  Please use long, unsigned long, sqInt or usqInt for variables that are of the machine's natural word length

That’s stuff we dealt with a long time ago, surely? The entire idea of introducing the sqInt/usqInt etc type defines was to allow the old 64bit plain interp vm to build. Yeah, this is what we wrote about it -
http://squeakvm.org/squeak64/ Wholly Khao - ten years ago.

So the interesting question is how did ‘bad’ stuff get to be there still?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: SFA: Seek Financial Assistance





--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================




--
best,
Eliot




--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================
Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

David T. Lewis
In reply to this post by Eliot Miranda-2
 
On Wed, Dec 10, 2014 at 03:13:13PM -0800, Eliot Miranda wrote:
>  
> Hi All,
>
>     just a simple message about 64-bit code.  Don't use int.  Please use
> long, unsigned long, sqInt or usqInt for variables that are of the
> machine's natural word length (sqInt is a synonym for long and usqInt is a
> synonym for unsigned long).

No, sqInit is *NOT* a synonym for long.

In particular, it is a 64-bit data type in some circumstances, and a 32-bit
data type in others. Ditto for usqInt.

If you are working with something where word size matters, declare it as
such, and do not rely on sqInt and usqInt to do what you expect. They are
not 32 bit data types.

Some but not all of the plugins are 32/64 bit clean in trunk. But many,
including FFI, SSL, and plugins that rely on the SurfacePlugin stub, remain
to be sorted out. By far the worst problem is the assumption that C pointers
fit into 32-bit int data types, but there are various other issues remaining
to be addressed. At the risk of a broad generalization, they all relate to
missing type declarations and invalid assumptions about word size, such as
assuming that sqInt is 32-bits in size.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

Eliot Miranda-2
 


On Wed, Dec 10, 2014 at 5:48 PM, David T. Lewis <[hidden email]> wrote:

On Wed, Dec 10, 2014 at 03:13:13PM -0800, Eliot Miranda wrote:
>
> Hi All,
>
>     just a simple message about 64-bit code.  Don't use int.  Please use
> long, unsigned long, sqInt or usqInt for variables that are of the
> machine's natural word length (sqInt is a synonym for long and usqInt is a
> synonym for unsigned long).

No, sqInit is *NOT* a synonym for long.

Apart from on the weird "64-bit image on 32-bit platform" oddity that I see no value in maintaining, sqInt == long.

On 32-bit C compilers

        sizeof(int) == 4
        sizeof(long) == 4
        sizeof(sqInt) == 4

On 64-bit C compilers with a 64-bit image

        sizeof(int) == 4
        sizeof(long) == 8
        sizeof(sqInt) == 8

There's an oddity of a 32-bit VM compiled in 64-bit mode on a 64-bit machine.  I don't know of anyone using it.  But on that config

        sizeof(int) == 4
        sizeof(long) == 8
        sizeof(sqInt) == 4

I'm in the process of providing a real 64-bit system with 61-bit SmallIntegers, immediate floating point, a segmented memory, etc, etc.  I am /not/ going to be held up trying to maintain the old 64-bit VM oddities.  A 32-bit VM on a 32-bit platform and a 64-bit VM on a 64-bit platform are enough, and expensive enough to maintain.

So yes, use sqInt, usqInt, long and unsigned long.  Do not use int.  Preferably declare pointer parameters as pointer rather than sqInt/long.  But in fixing existing code the easiest thing is to change int to either sqInt or long, depending on whether that parameter receives an oop or not.


In particular, it is a 64-bit data type in some circumstances, and a 32-bit
data type in others. Ditto for usqInt.

If you are working with something where word size matters, declare it as
such, and do not rely on sqInt and usqInt to do what you expect. They are
not 32 bit data types.

Some but not all of the plugins are 32/64 bit clean in trunk. But many,
including FFI, SSL, and plugins that rely on the SurfacePlugin stub, remain
to be sorted out. By far the worst problem is the assumption that C pointers
fit into 32-bit int data types, but there are various other issues remaining
to be addressed. At the risk of a broad generalization, they all relate to
missing type declarations and invalid assumptions about word size, such as
assuming that sqInt is 32-bits in size.

Dave




--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

David T. Lewis
 
On Wed, Dec 10, 2014 at 06:01:02PM -0800, Eliot Miranda wrote:

>  
> On Wed, Dec 10, 2014 at 5:48 PM, David T. Lewis <[hidden email]> wrote:
>
> >
> > On Wed, Dec 10, 2014 at 03:13:13PM -0800, Eliot Miranda wrote:
> > >
> > > Hi All,
> > >
> > >     just a simple message about 64-bit code.  Don't use int.  Please use
> > > long, unsigned long, sqInt or usqInt for variables that are of the
> > > machine's natural word length (sqInt is a synonym for long and usqInt is
> > a
> > > synonym for unsigned long).
> >
> > No, sqInit is *NOT* a synonym for long.
> >
>
> Apart from on the weird "64-bit image on 32-bit platform" oddity that I see
> no value in maintaining, sqInt == long.
>
> On 32-bit C compilers
>
>         sizeof(int) == 4
>         sizeof(long) == 4
>         sizeof(sqInt) == 4
>
> On 64-bit C compilers with a 64-bit image
>
>         sizeof(int) == 4
>         sizeof(long) == 8
>         sizeof(sqInt) == 8
>
> There's an oddity of a 32-bit VM compiled in 64-bit mode on a 64-bit
> machine.  I don't know of anyone using it.  But on that config
>
>         sizeof(int) == 4
>         sizeof(long) == 8
>         sizeof(sqInt) == 4

There is nothing odd about this at all. A sqInt corresponds to the size of a
slot in the object memory, which only coincidentally has anything to do with
the size of C integer data types on the host platform.

>
> I'm in the process of providing a real 64-bit system with 61-bit
> SmallIntegers, immediate floating point, a segmented memory, etc, etc.  I
> am /not/ going to be held up trying to maintain the old 64-bit VM
> oddities.  A 32-bit VM on a 32-bit platform and a 64-bit VM on a 64-bit
> platform are enough, and expensive enough to maintain.

Nonsense. If you want a data type to represent object pointers, then define
one, and give it a meaningful name such as "sqOop". This is very much needed,
and would be a great improvement over the current state of affairs. Please
do not continue the past practice of guessing at what "sqInt" should mean,
particularly if your personal opinion as to what it should mean happens to
be different from the opinion of the last person who made the same mistake.

It would also be useful to explicitly define a data type that represents a
location in the object memory, which might be 32 or 64 bits in size, independent
of the size of object pointers, C pointers, and integer size on the host platform.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

Bert Freudenberg
In reply to this post by Eliot Miranda-2
 
On 11.12.2014, at 03:01, Eliot Miranda <[hidden email]> wrote:

There's an oddity of a 32-bit VM compiled in 64-bit mode on a 64-bit machine.  I don't know of anyone using it.

This actually is *the* single most requested VM people want on Linux. It's just not quite stable because Dave has been working on it all on his own.

  But on that config

        sizeof(int) == 4
        sizeof(long) == 8
        sizeof(sqInt) == 4

I'm in the process of providing a real 64-bit system with 61-bit SmallIntegers, immediate floating point, a segmented memory, etc, etc.  I am /not/ going to be held up trying to maintain the old 64-bit VM oddities.  A 32-bit VM on a 32-bit platform and a 64-bit VM on a 64-bit platform are enough, and expensive enough to maintain.

Dear VM. You have *one* job. Be virtual. Make my image run no matter what actual platform we are on.

Seriously, we need to be able to take an image from one platform and run it on another. If your particular VM cannot cover all platform-image combos, fine. You trade performance for cross-platformness. That's cool. But please don't make it harder for those of us who *do* value cross-platformness in futzing around with sqInt. It is *defined* as being a type of the image's word size, not the machines's word size.

Please do *not* use "long" in VM code. 

+1 to what Dave wrote. (*)

- Bert -

(*) The first word coming to my mind was the same one Dave used.

smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

David T. Lewis
 
On Thu, Dec 11, 2014 at 12:12:59PM +0100, Bert Freudenberg wrote:

>  
> On 11.12.2014, at 03:01, Eliot Miranda <[hidden email]> wrote:
> >
> > There's an oddity of a 32-bit VM compiled in 64-bit mode on a 64-bit machine.  I don't know of anyone using it.
>
> This actually is *the* single most requested VM people want on Linux. It's just not quite stable because Dave has been working on it all on his own.
>
> >   But on that config
> >
> >         sizeof(int) == 4
> >         sizeof(long) == 8
> >         sizeof(sqInt) == 4
> >
> > I'm in the process of providing a real 64-bit system with 61-bit SmallIntegers, immediate floating point, a segmented memory, etc, etc.  I am /not/ going to be held up trying to maintain the old 64-bit VM oddities.  A 32-bit VM on a 32-bit platform and a 64-bit VM on a 64-bit platform are enough, and expensive enough to maintain.
>
> Dear VM. You have *one* job. Be virtual. Make my image run no matter what actual platform we are on.
>
> Seriously, we need to be able to take an image from one platform and run it on another. If your particular VM cannot cover all platform-image combos, fine. You trade performance for cross-platformness. That's cool. But please don't make it harder for those of us who *do* value cross-platformness in futzing around with sqInt. It is *defined* as being a type of the image's word size, not the machines's word size.
>
> Please do *not* use "long" in VM code.
>
> +1 to what Dave wrote. (*)
>
> - Bert -
>
> (*) The first word coming to my mind was the same one Dave used.

Um. Actually, I think I should apologize to Eliot for the tone of my message.
While I do think that is is "nonsense" to assume sqInt must be 32 bits, the
way I said it makes it sound like a personal attack. That is not my intent,
and I apologize.

Sorry,
Dave

Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

Eliot Miranda-2
In reply to this post by David T. Lewis
 
Hi David, Hi Bert,

On Wed, Dec 10, 2014 at 6:52 PM, David T. Lewis <[hidden email]> wrote:

On Wed, Dec 10, 2014 at 06:01:02PM -0800, Eliot Miranda wrote:
>
> On Wed, Dec 10, 2014 at 5:48 PM, David T. Lewis <[hidden email]> wrote:
>
> >
> > On Wed, Dec 10, 2014 at 03:13:13PM -0800, Eliot Miranda wrote:
> > >
> > > Hi All,
> > >
> > >     just a simple message about 64-bit code.  Don't use int.  Please use
> > > long, unsigned long, sqInt or usqInt for variables that are of the
> > > machine's natural word length (sqInt is a synonym for long and usqInt is
> > a
> > > synonym for unsigned long).
> >
> > No, sqInit is *NOT* a synonym for long.
> >
>
> Apart from on the weird "64-bit image on 32-bit platform" oddity that I see
> no value in maintaining, sqInt == long.
>
> On 32-bit C compilers
>
>         sizeof(int) == 4
>         sizeof(long) == 4
>         sizeof(sqInt) == 4
>
> On 64-bit C compilers with a 64-bit image
>
>         sizeof(int) == 4
>         sizeof(long) == 8
>         sizeof(sqInt) == 8
>
> There's an oddity of a 32-bit VM compiled in 64-bit mode on a 64-bit
> machine.  I don't know of anyone using it.  But on that config
>
>         sizeof(int) == 4
>         sizeof(long) == 8
>         sizeof(sqInt) == 4

There is nothing odd about this at all. A sqInt corresponds to the size of a
slot in the object memory, which only coincidentally has anything to do with
the size of C integer data types on the host platform.

I don't want to fight, but let me tell you what I really think.  Having an oop be larger than the machine's pointer size is *absurd*.  One wastes half the available address space representing a pointer by using twice the space one needs.  That is insane.  I'm not going to expend one iota of effort making this work.


>
> I'm in the process of providing a real 64-bit system with 61-bit
> SmallIntegers, immediate floating point, a segmented memory, etc, etc.  I
> am /not/ going to be held up trying to maintain the old 64-bit VM
> oddities.  A 32-bit VM on a 32-bit platform and a 64-bit VM on a 64-bit
> platform are enough, and expensive enough to maintain.

Nonsense. If you want a data type to represent object pointers, then define
one, and give it a meaningful name such as "sqOop". This is very much needed,
and would be a great improvement over the current state of affairs. Please
do not continue the past practice of guessing at what "sqInt" should mean,
particularly if your personal opinion as to what it should mean happens to
be different from the opinion of the last person who made the same mistake.

I agree.  But that's much more effort than fixing the few places where int is used mistakenly and erroneously to define a paramter that can take an oop.  I'm not rewriting all of platforms to move to a more elegant solution now.  I don't have the time now.  Now that 32-bt Spur is ready for deployment my priority is to get 64-bit Spur working.


It would also be useful to explicitly define a data type that represents a
location in the object memory, which might be 32 or 64 bits in size, independent
of the size of object pointers, C pointers, and integer size on the host platform.

Again, the issue is to get a 64-bit Spur VM to compile and run, not to revamp VMMaker to do things right.  For that, using long and/or sqInt is a suitable solution.

--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

Eliot Miranda-2
In reply to this post by Bert Freudenberg
 


On Thu, Dec 11, 2014 at 3:12 AM, Bert Freudenberg <[hidden email]> wrote:
 
On 11.12.2014, at 03:01, Eliot Miranda <[hidden email]> wrote:

There's an oddity of a 32-bit VM compiled in 64-bit mode on a 64-bit machine.  I don't know of anyone using it.

Right.  But since the rationale for this VM is only to interface with 64-bit libraries, it depends on a 64-bit FFI, which we do not have.  So in the absence of a 64-bit FFI it is pointless.  Instead running the 32-bit VM on the 64-bit platform is a perfect replacement.  For me having a 64-bit VM/image working is a higher priority than having a 32-bit image/64-bit VM.  Both are dependent on the 64-bit FFI but the latter also lifts the address space limit, so it kills more birds.
 

This actually is *the* single most requested VM people want on Linux. It's just not quite stable because Dave has been working on it all on his own.

  But on that config

        sizeof(int) == 4
        sizeof(long) == 8
        sizeof(sqInt) == 4

I'm in the process of providing a real 64-bit system with 61-bit SmallIntegers, immediate floating point, a segmented memory, etc, etc.  I am /not/ going to be held up trying to maintain the old 64-bit VM oddities.  A 32-bit VM on a 32-bit platform and a 64-bit VM on a 64-bit platform are enough, and expensive enough to maintain.

Dear VM. You have *one* job. Be virtual. Make my image run no matter what actual platform we are on.

That's fine.  You break your back doing that.  My definition of a VM is different.  Provide a performant, safe, interoperative platform on which to run Smalltalk et al. I'm not interested in virtuality per se.  I'm interested in being able to get work done.  I will choose popular platforms and performance over slowness and portability.  I will choose FFI over plugins.  We differ.  Let's accept that difference.

 
Seriously, we need to be able to take an image from one platform and run it on another.

But that can be done with off-line tools.  It doesn't need to be done with an all-in-one VM that is slow, complex and difficult to maintain.
 
If your particular VM cannot cover all platform-image combos, fine. You trade performance for cross-platformness. That's cool. But please don't make it harder for those of us who *do* value cross-platformness in futzing around with sqInt. It is *defined* as being a type of the image's word size, not the machines's word size.

OK, so use long.
 

Please do *not* use "long" in VM code. 

Bollocks.  long is an integral type defined to be large enough to take a pointer.  If sqInt is out, as you stipulate above, and the existing code is written to take an integer parameter in which is passed a pointer then long is the only choice.  Rewriting all the platform code to take pointers where pointers are passed is work for the future, not busy work that advances nothing.

But if th absurdity of sizeof(sqInt) == 8 when sizeof(void *) == 4 is accepted then we can use sqInt.

Now, I don't ant to fight anymore.  I'm getting back to getting a 64-bit Spur stack VM running on linux.  You do what pleases you.



+1 to what Dave wrote. (*)

- Bert -

(*) The first word coming to my mind was the same one Dave used.



Whatever Bert...
--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

johnmci
In reply to this post by Eliot Miranda-2
 
Recall at the time Apple didn't have 64bit support for a program to run under Cocoa, plus only a limited number of machines support 64 bit kernels. The fact you could go with a a 64 bit image on an 32bit machine allowed us to debug the base VM.  

I agree the 32/64 64/32 is nice and symmetric, but I'm afraid it's a nice to have versus need now. If it's not seamless to write software and plugins that honour that agreement, then it's a world of trouble and hassle that it's not worth pursuing further. 

On Thu, Dec 11, 2014 at 10:18 AM, Eliot Miranda <[hidden email]> wrote:
 
Hi David, Hi Bert,

On Wed, Dec 10, 2014 at 6:52 PM, David T. Lewis <[hidden email]> wrote:

On Wed, Dec 10, 2014 at 06:01:02PM -0800, Eliot Miranda wrote:
>
> On Wed, Dec 10, 2014 at 5:48 PM, David T. Lewis <[hidden email]> wrote:
>
> >
> > On Wed, Dec 10, 2014 at 03:13:13PM -0800, Eliot Miranda wrote:
> > >
> > > Hi All,
> > >
> > >     just a simple message about 64-bit code.  Don't use int.  Please use
> > > long, unsigned long, sqInt or usqInt for variables that are of the
> > > machine's natural word length (sqInt is a synonym for long and usqInt is
> > a
> > > synonym for unsigned long).
> >
> > No, sqInit is *NOT* a synonym for long.
> >
>
> Apart from on the weird "64-bit image on 32-bit platform" oddity that I see
> no value in maintaining, sqInt == long.
>
> On 32-bit C compilers
>
>         sizeof(int) == 4
>         sizeof(long) == 4
>         sizeof(sqInt) == 4
>
> On 64-bit C compilers with a 64-bit image
>
>         sizeof(int) == 4
>         sizeof(long) == 8
>         sizeof(sqInt) == 8
>
> There's an oddity of a 32-bit VM compiled in 64-bit mode on a 64-bit
> machine.  I don't know of anyone using it.  But on that config
>
>         sizeof(int) == 4
>         sizeof(long) == 8
>         sizeof(sqInt) == 4

There is nothing odd about this at all. A sqInt corresponds to the size of a
slot in the object memory, which only coincidentally has anything to do with
the size of C integer data types on the host platform.

I don't want to fight, but let me tell you what I really think.  Having an oop be larger than the machine's pointer size is *absurd*.  One wastes half the available address space representing a pointer by using twice the space one needs.  That is insane.  I'm not going to expend one iota of effort making this work.


>
> I'm in the process of providing a real 64-bit system with 61-bit
> SmallIntegers, immediate floating point, a segmented memory, etc, etc.  I
> am /not/ going to be held up trying to maintain the old 64-bit VM
> oddities.  A 32-bit VM on a 32-bit platform and a 64-bit VM on a 64-bit
> platform are enough, and expensive enough to maintain.

Nonsense. If you want a data type to represent object pointers, then define
one, and give it a meaningful name such as "sqOop". This is very much needed,
and would be a great improvement over the current state of affairs. Please
do not continue the past practice of guessing at what "sqInt" should mean,
particularly if your personal opinion as to what it should mean happens to
be different from the opinion of the last person who made the same mistake.

I agree.  But that's much more effort than fixing the few places where int is used mistakenly and erroneously to define a paramter that can take an oop.  I'm not rewriting all of platforms to move to a more elegant solution now.  I don't have the time now.  Now that 32-bt Spur is ready for deployment my priority is to get 64-bit Spur working.


It would also be useful to explicitly define a data type that represents a
location in the object memory, which might be 32 or 64 bits in size, independent
of the size of object pointers, C pointers, and integer size on the host platform.

Again, the issue is to get a 64-bit Spur VM to compile and run, not to revamp VMMaker to do things right.  For that, using long and/or sqInt is a suitable solution.

--
best,
Eliot




--
===========================================================================
John M. McIntosh <[hidden email]https://www.linkedin.com/in/smalltalk
===========================================================================
Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

Bert Freudenberg
In reply to this post by Eliot Miranda-2
 
Eliot,

maybe this came across as if we wanted to somehow interfere with your 64 bit work. On the contrary! We simply think it wise to use clearly defined type declarations that allow all sorts of implementations, even ones you would consider silly. Nobody is expecting you to make the silly stuff work :)

On 11.12.2014, at 19:28, Eliot Miranda <[hidden email]> wrote:

On Thu, Dec 11, 2014 at 3:12 AM, Bert Freudenberg <[hidden email]> wrote:
 
On 11.12.2014, at 03:01, Eliot Miranda <[hidden email]> wrote:

There's an oddity of a 32-bit VM compiled in 64-bit mode on a 64-bit machine.  I don't know of anyone using it.
This actually is *the* single most requested VM people want on Linux. It's just not quite stable because Dave has been working on it all on his own.
 
Right.  But since the rationale for this VM is only to interface with 64-bit libraries, it depends on a 64-bit FFI, which we do not have.  So in the absence of a 64-bit FFI it is pointless.  Instead running the 32-bit VM on the 64-bit platform is a perfect replacement. 

Not for people who can't easily run a 32 bit VM on their 64 bit system. I'm with you that the Linux folks got it wrong, that their zealous goal of making a 64-bit-only system is misguided. Nevertheless, it is legitimate to want to compile a VM on these crippled 64-bit-only systems and have it run a normal (32-bit) image.


  But on that config

        sizeof(int) == 4
        sizeof(long) == 8
        sizeof(sqInt) == 4

I'm in the process of providing a real 64-bit system with 61-bit SmallIntegers, immediate floating point, a segmented memory, etc, etc.  I am /not/ going to be held up trying to maintain the old 64-bit VM oddities.  A 32-bit VM on a 32-bit platform and a 64-bit VM on a 64-bit platform are enough, and expensive enough to maintain.

Dear VM. You have *one* job. Be virtual. Make my image run no matter what actual platform we are on.

That's fine.  You break your back doing that.  My definition of a VM is different.  Provide a performant, safe, interoperative platform on which to run Smalltalk et al. I'm not interested in virtuality per se.  I'm interested in being able to get work done.  I will choose popular platforms and performance over slowness and portability.  I will choose FFI over plugins.  We differ.  Let's accept that difference.

Sure. That does not imply we need to make our respective goals harder to achieve, if that is simple to avoid. And using "sqInt" throughout is pretty simple IMHO.
 
Seriously, we need to be able to take an image from one platform and run it on another.

But that can be done with off-line tools.  It doesn't need to be done with an all-in-one VM that is slow, complex and difficult to maintain.

As I said, I don't particularly care how we make this work. If a child takes their image from the school computer and wants to run it at home, and one machine happens to be 64 bits and the other does not, then it would be dreadful if it had to run it through a converter. But we may be able to hide that in the VM startup script.

If your particular VM cannot cover all platform-image combos, fine. You trade performance for cross-platformness. That's cool. But please don't make it harder for those of us who *do* value cross-platformness in futzing around with sqInt. It is *defined* as being a type of the image's word size, not the machines's word size.

OK, so use long.

No. Maybe I miswrote, or you misread. We do want to use sqInt for oops, because that is defined by us, not by the compiler.


Please do *not* use "long" in VM code. 

Bollocks.

I meant "do *not* use 'long' for oops", with which I think you actually agree.

 long is an integral type defined to be large enough to take a pointer.  If sqInt is out, as you stipulate above,

I didn't. I advocated using sqInt.

and the existing code is written to take an integer parameter in which is passed a pointer then long is the only choice.  Rewriting all the platform code to take pointers where pointers are passed is work for the future, not busy work that advances nothing.

But if th absurdity of sizeof(sqInt) == 8 when sizeof(void *) == 4 is accepted then we can use sqInt.

That would only be the case for a 64-bit image running on a 32-bit platform. Which is very expensive, sure, and not something we would advise anyone to use in production.

Much more relevant is the case of sizeof(sqInt) == 4 when sizeof(void *) == 8, as I wrote above.

Now, I don't ant to fight anymore.

Yeah, fights are no fun if you can't scream at each other and have a beer after. But then again, apart from use of the n-word (for which there was an apology), we're not even really fighting :)

The only thing we questioned was that you said "long" was a fine replacement for "sqInt" in general. But as I understood it that was just an off-hand remark and not an actual advice to go and replace every use of "sqInt" with "long". So we just wanted to clarify that the right type to use for oops was in fact sqInt (or as David said a yet-to-be introduced "oop" type).

Peace?

- Bert -


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

Eliot Miranda-2
 


On Thu, Dec 11, 2014 at 12:25 PM, Bert Freudenberg <[hidden email]> wrote:
 
Eliot,

maybe this came across as if we wanted to somehow interfere with your 64 bit work. On the contrary! We simply think it wise to use clearly defined type declarations that allow all sorts of implementations, even ones you would consider silly. Nobody is expecting you to make the silly stuff work :)

On 11.12.2014, at 19:28, Eliot Miranda <[hidden email]> wrote:

On Thu, Dec 11, 2014 at 3:12 AM, Bert Freudenberg <[hidden email]> wrote:
 
On 11.12.2014, at 03:01, Eliot Miranda <[hidden email]> wrote:

There's an oddity of a 32-bit VM compiled in 64-bit mode on a 64-bit machine.  I don't know of anyone using it.
This actually is *the* single most requested VM people want on Linux. It's just not quite stable because Dave has been working on it all on his own.
 
Right.  But since the rationale for this VM is only to interface with 64-bit libraries, it depends on a 64-bit FFI, which we do not have.  So in the absence of a 64-bit FFI it is pointless.  Instead running the 32-bit VM on the 64-bit platform is a perfect replacement. 

Not for people who can't easily run a 32 bit VM on their 64 bit system. I'm with you that the Linux folks got it wrong, that their zealous goal of making a 64-bit-only system is misguided. Nevertheless, it is legitimate to want to compile a VM on these crippled 64-bit-only systems and have it run a normal (32-bit) image.


  But on that config

        sizeof(int) == 4
        sizeof(long) == 8
        sizeof(sqInt) == 4

I'm in the process of providing a real 64-bit system with 61-bit SmallIntegers, immediate floating point, a segmented memory, etc, etc.  I am /not/ going to be held up trying to maintain the old 64-bit VM oddities.  A 32-bit VM on a 32-bit platform and a 64-bit VM on a 64-bit platform are enough, and expensive enough to maintain.

Dear VM. You have *one* job. Be virtual. Make my image run no matter what actual platform we are on.

That's fine.  You break your back doing that.  My definition of a VM is different.  Provide a performant, safe, interoperative platform on which to run Smalltalk et al. I'm not interested in virtuality per se.  I'm interested in being able to get work done.  I will choose popular platforms and performance over slowness and portability.  I will choose FFI over plugins.  We differ.  Let's accept that difference.

Sure. That does not imply we need to make our respective goals harder to achieve, if that is simple to avoid. And using "sqInt" throughout is pretty simple IMHO.
 
Seriously, we need to be able to take an image from one platform and run it on another.

But that can be done with off-line tools.  It doesn't need to be done with an all-in-one VM that is slow, complex and difficult to maintain.

As I said, I don't particularly care how we make this work. If a child takes their image from the school computer and wants to run it at home, and one machine happens to be 64 bits and the other does not, then it would be dreadful if it had to run it through a converter. But we may be able to hide that in the VM startup script.

If your particular VM cannot cover all platform-image combos, fine. You trade performance for cross-platformness. That's cool. But please don't make it harder for those of us who *do* value cross-platformness in futzing around with sqInt. It is *defined* as being a type of the image's word size, not the machines's word size.

OK, so use long.

No. Maybe I miswrote, or you misread. We do want to use sqInt for oops, because that is defined by us, not by the compiler.


Please do *not* use "long" in VM code. 

Bollocks.

I meant "do *not* use 'long' for oops", with which I think you actually agree.

 long is an integral type defined to be large enough to take a pointer.  If sqInt is out, as you stipulate above,

I didn't. I advocated using sqInt.

and the existing code is written to take an integer parameter in which is passed a pointer then long is the only choice.  Rewriting all the platform code to take pointers where pointers are passed is work for the future, not busy work that advances nothing.

But if th absurdity of sizeof(sqInt) == 8 when sizeof(void *) == 4 is accepted then we can use sqInt.

That would only be the case for a 64-bit image running on a 32-bit platform. Which is very expensive, sure, and not something we would advise anyone to use in production.

Much more relevant is the case of sizeof(sqInt) == 4 when sizeof(void *) == 8, as I wrote above.

Now, I don't ant to fight anymore.

Yeah, fights are no fun if you can't scream at each other and have a beer after. But then again, apart from use of the n-word (for which there was an apology), we're not even really fighting :)

The only thing we questioned was that you said "long" was a fine replacement for "sqInt" in general. But as I understood it that was just an off-hand remark and not an actual advice to go and replace every use of "sqInt" with "long".

Right.  I suggest using long to declare parameters that take pointers encoded as integers.  As in the fileNamePtr argument in

int asyncFileOpen(AsyncFile *f, int fileNamePtr, int fileNameSize, int writeFlag, int semaIndex);

The simplest fix to this code is

int asyncFileOpen(AsyncFile *f, long fileNamePtr, int fileNameSize, int writeFlag, int semaIndex);

Of course it should be written

int asyncFileOpen(AsyncFile *f, char * fileNamePtr, int fileNameSize, int writeFlag, int semaIndex);

or void *, but that requires modification of the body of asyncFileOpen which is more work than I want to put in on this.

Of course it should /really/ be written

long asyncFileOpen(AsyncFile *f, char * fileNamePtr, long fileNameSize, long writeFlag, long semaIndex);

because on 64-bits there may actually be code generated to convert to the 32-bits tat int defines, whereas long is natural and requires no nonsense by the compiler to pass as a parameter, put in a register etc.

Hence my advice, don't use int.


 
So we just wanted to clarify that the right type to use for oops was in fact sqInt (or as David said a yet-to-be introduced "oop" type).

Peace?

Peace.

- Bert -

--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

32/64 bit VM and image combinations (was: egg sucking alert)

David T. Lewis
In reply to this post by johnmci
 
On Thu, Dec 11, 2014 at 10:30:41AM -0800, John McIntosh wrote:

>  
> Recall at the time Apple didn't have 64bit support for a program to run
> under Cocoa, plus only a limited number of machines support 64 bit kernels.
> The fact you could go with a a 64 bit image on an 32bit machine allowed us
> to debug the base VM.
>
> I agree the 32/64 64/32 is nice and symmetric, but I'm afraid it's a nice
> to have versus need now. If it's not seamless to write software and plugins
> that honour that agreement, then it's a world of trouble and hassle that
> it's not worth pursuing further.

For the record, and possibly to offer some perspective:

Getting the VM to work for all combinations of 32/64 bit image and VM was
not any extra work at all. The hard part was getting the 64-bit VM with
32-bit image to work. Once the type declarations were cleaned up well enough
to support this, the rest came along for free.

The 64-bit VMs work fine, and I have been using them for a long time (several
years I guess). Linux users can find one at http://squeakvm.org/unix/.

The 64-bit VM with 32-bit image is a particularly important configuration,
because it involves 64-bit machine pointers and 32-bit ints. This combination
will quickly expose the majority of problems related to pointer and word sizes.
If you want to make a plugin 32/64 bit clean (and there are several remaining
to be tidied up), this configuration is essential.

Dave

Reply | Threaded
Open this post in threaded view
|

64-bit FFI (was: egg sucking alert)

David T. Lewis
In reply to this post by Eliot Miranda-2
 
On Thu, Dec 11, 2014 at 10:28:04AM -0800, Eliot Miranda wrote:

>  
> On Thu, Dec 11, 2014 at 3:12 AM, Bert Freudenberg <[hidden email]> wrote:
> >
> > On 11.12.2014, at 03:01, Eliot Miranda <[hidden email]> wrote:
> >
> > There's an oddity of a 32-bit VM compiled in 64-bit mode on a 64-bit
> > machine.  I don't know of anyone using it.
> >
> Right.  But since the rationale for this VM is only to interface with
> 64-bit libraries, it depends on a 64-bit FFI, which we do not have.

I realize that this refers to the VMs and FFI implementations that are
actually available in released VMs, but just for the historical record:

A working 64-bit FFI has been available since 2008, but the patches were not
adopted. Source patches and issue status are here:

  http://bugs.squeak.org/view.php?id=7237

My intention had been to make this a top priority for the VM team circa 2009,
but other priorities took precedence.

Since then, the original FFI has been rendered obsolete by later development,
and as a practical matter there is no 64-bit FFI available today.

There were image side changes to the Mantis 7237 patches, and I'm not sure
if these will be relevant to Eliot's newer FFI implementation. I'll try
to look into this and report back if I find anything of interest.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: egg sucking alert

Stefan Marr-3
In reply to this post by Eliot Miranda-2

Hi:

> On 11 Dec 2014, at 19:28, Eliot Miranda <[hidden email]> wrote:
>
> Bollocks.  long is an integral type defined to be large enough to take a pointer.

Ignoring all the other issues in this thread, I would be very hesitant to take this statement as a general truth.
The last time I checked, the C standard defined only intptr_t and uintptr_t to be guaranteed to be a capable of holding pointers.

Best regards
Stefan

--
Stefan Marr
INRIA Lille - Nord Europe
http://stefan-marr.de/research/



12