Issues creating instance of SmallInteger

Hello,
I've got a problem creating objects via this sample of code:
(Smalltalk at: #AClassName) new
The problem is that, when trying to create an instance of SmallInteger, it raises an error :
"SmallIntegers can only be created by performing arithmetic"...
If somebody can tell me why it isn't possible to create a SmallInteger like this, and give me a way to do it, I'd appreciate...
And are there other classes behaving the same way? So that I can take them in account in my code.

Thank you for reading this!

Michel,

On 5/12/06, Michel Calonne <[hidden email]> wrote:
> "SmallIntegers can only be created by performing arithmetic"...
> If somebody can tell me why it isn't possible to create a SmallInteger like this, and give me a way to do it, I'd appreciate...

short answer: SmallIntegers aren't actually objects.

Long answer (of course I won't let you down): SmallIntegers are an
optimised representation for numbers in the range +-2^30 (I believe).
These numbers are used extremely often and deserve special attention.
This is actually a detail of how the VM is implemented to optimise
arithmetics and handling of such "small" numbers in general.

In Smalltalk, all objects are referred to by pointers. A pointer is a
32-bit word. What it refers to is a structure in memory, containing of
a header (32 bit) and some more words containing pointers to all the
instance members of the object in question.

Imagine what would happen if an ordinary number was represented in
that way. For a 32-bit integer, you'd have (1) the pointer, (2) the
header, and (3) the contents (i.e., the actual value of the number).
That's, simplified, at least 3*32 bits for representing 32 bits.
Doesn't make much sense.

So, there is this neat little trick of (mis)using the lowest bit of
every pointer to denote whether the pointer is an actual pointer or
not. If it is set to 0, it's a pointer. If it's set to 1, the
"pointer" actually represents an instance of SmallInteger; a so-called
immediate integer.

Of course that means that you've only got 31 bits available for
representing a two's complement number, but who uses larger numbers
anyway? ;-)

Doing arithmetics with such an object is very simple: shift away the
lowest bit, adjust the sign bit, and perform ordinary CPU-provided
arithmetic operations. Bingo. Should the result ever exceed the range,
a LargePositiveInteger (or corresponding instance) is created.

Hope that helps,

Michael

Hi again,

On 5/12/06, Michael Haupt <[hidden email]> wrote:
> > If somebody can tell me why it isn't possible to create a SmallInteger like this, and give me a way to do it, I'd appreciate...
> [...]
> Hope that helps,

well, sorry for forgetting to actually answer your question. :-P

So, these SmallIntegers are always created implicitly when performing
computations. No allocation is associated with creating one, it's
purely arithmetics. So, it doesn't make sense to send #new to
SmallInteger - if you need one, say, 42, just create it by typing 42
in a workspace and inspecting it.

Another neat feature of this representation is, by the way, that you
cannot possibly have two different instances of SmallInteger
representing the same number. SmallInteger equality is simply checked
by comparing the pointers. Very cheap.

Best,

Michael

On 5/12/06, Michel Calonne <[hidden email]> wrote:
> (Smalltalk at: #AClassName) new
> The problem is that, when trying to create an instance of SmallInteger, it raises an error :
> "SmallIntegers can only be created by performing arithmetic"...

That's because integers are pure values, they are immutable. It
wouldn't make sense to create several copies of 42... and if integers
had a working new, how would you choose which integer you get ? set
its value ? to what ?  :)  The only way to get hold of an integer is
via a literal value in the code or by combining existing integers by
arithmetic.

In the VM, some objects are represented in a compact way for
performance reasons: instead of a pointer to the object data, you have
directly the value (if you know eiffel, that's the same as the
expanded types). I'm not sure which classes behave this way but most
probably all the ones with small memory footprint like characters,
numbers, etc

(hopefully I didn't say too much stupidities... :)

--
 Damien Pollet
 type less, do more

> (hopefully I didn't say too much stupidities... :)
Bah, I need to type faster...

--
 Damien Pollet
 type less, do more

On May 12, 2006, at 11:32 AM, Michel Calonne wrote:

My guess is that #new is forbidden to SmallIntegers because it's an  
immediate - you can't just create an object with all it's instance  
variables set to nil. For a SmallInteger to exist it must have a  
specific integer value.

On the other hand, throwing an error doesn't seem like useful  
behavior, either. Changing the definition of #new to just return 0  
would be perfectly reasonable. Heck, it could return 7 for that matter.

Colin

On 5/12/06, Michel Calonne <[hidden email]> wrote:

> I've got a problem creating objects via this sample of code:
> (Smalltalk at: #AClassName) new
> The problem is that, when trying to create an instance of SmallInteger,
> it raises an error :
> "SmallIntegers can only be created by performing arithmetic"...
> If somebody can tell me why it isn't possible to create a SmallInteger
> like this, and give me a way to do it, I'd appreciate...
> And are there other classes behaving the same way? So that I can
> take them in account in my code.

SmallIntegers are implemented in a different way than most ordinary
objects, so they don't use #new. Some other classes don't create
instances with #new; Boolean is one.

As an alternative to #new, you could try #initializedInstance, which
should work with nearly any class.

Hope this helps!

--Tom Phoenix

Hi Michel

May be Object readFrom: '1'  is what you need. Because certain  
objects can't be created with new.

Stef

On 12 mai 06, at 17:32, Michel Calonne wrote:

Mmm let me clarify some things.

First object point to other objects via an object reference, one  
shouldn't call them pointers because
they technically aren't true pointers from the "C" sense of the word.  
For example earlier smalltalk used an
index into an object table, and why would you need more than 64K of  
objects in your image anyway?

When object references were thought up at some point someone had the  
clever idea if I'm holding onto
this data to refer to another object, why can't that data be the  
object. Thus the concept of an immediate object type
was born. In Squeak we use 2 bits of the 32 in 32bit Squeak, 1 bit  
for a tag for the GC logic, and the other bit to signify if this  
object is a
SmallInteger.

xxxxxxxx00 -> object reference (tag bit not set; divisible by four)
xxxxxxxx10 -> Unused (tag bit not set; but not a valid pointer)
xxxxxxxx01 -> SmallInteger (tag bit set)
xxxxxxxx11 -> SmallInteger (tag bit set)

That of course gives us a range of +_ 1 billionish
(0 - (2 raisedTo: 30)) to ((2 raisedTo: 30) - 1)

I'll note other Smalltalk might have different immediate data types  
and 64bit Smalltalks usually have lots more
immediate types, such as single precision float, small doubles, and  
character.

Lastly, I'm afraid we do create many copies of 42 since people do  
alter them. However there is a grain of truth in your comment since  
we don't create many copies of $A  If you look at the Class Character  
you'll see: "I represent a character by storing its associated  
Unicode. The first 256 characters are created uniquely, so that all  
instances of latin1 characters ($R, for example) are identical." So  
Characters are immutable.

On 12-May-06, at 9:11 AM, Damien Pollet wrote:

> On 5/12/06, Michel Calonne <[hidden email]> wrote:
>> (Smalltalk at: #AClassName) new
>> The problem is that, when trying to create an instance of  
>> SmallInteger, it raises an error :
>> "SmallIntegers can only be created by performing arithmetic"...
>
> That's because integers are pure values, they are immutable. It
> wouldn't make sense to create several copies of 42...
>

--
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

John,

On 5/12/06, John M McIntosh <[hidden email]> wrote:
> In Squeak we use 2 bits of the 32 in 32bit Squeak, 1 bit
> for a tag for the GC logic, and the other bit to signify if this
> object is a SmallInteger.
>
> xxxxxxxx00 -> object reference (tag bit not set; divisible by four)
> xxxxxxxx10 -> Unused (tag bit not set; but not a valid pointer)
> xxxxxxxx01 -> SmallInteger (tag bit set)
> xxxxxxxx11 -> SmallInteger (tag bit set)

I was not aware of that GC flag bit, but I had actually been wondering
why one bit was "wasted" when pointers/references are word-aligned...
Can you point me to a place in the image where that is made clear?

(I told some students of mine about this issue just yesterday and
omitted these GC details, but they should be updated, I think.)

> Lastly, I'm afraid we do create many copies of 42 since people do
> alter them.

I don't understand this. The oop representing 42 always has the same
shape, hasn't it? Just having it lying around in numerous places all
over the image does not mean creating a copy... after all, it's
identical with all the others.

And if 42 is altered, it's not 42 any more, but, say 23.

Am I missing something?

Best,

Michael

Many thanks to all of those who answered me !!
Even if I had trouble understanding some of the technical stuff about SmallIntegers (although I should know what immediates are now! :), it should help me get rid of this XML serializer project we've gotta do for 1 week from here!

For the purpose of generating a SmallInteger (from XML ya know..), Object readFrom:'42' works just fine!

As Stéphane Ducasse said, Smalltalk community seems to be truely helpfull and tolerant with newbies like me, and it will encourage me to post questions (less stupid I hope!) in the future!

Thank you again to the Smalltalk community!

Michel.

"Michel Calonne" <[hidden email]> asked:

> And are there other classes behaving the same way?
> So that I can take them in account in my code.
Yes, there are others, too.
You cannot use the template
  (Smalltalk at: <aClassName>) new
for <aClassName> in
   { True, False, UndefinedObject, Character, SmallInteger }

When you want to implement a persistence framework you
will have to figure out something special to reload
  true,  false, nil, the characters and small integers into your
image.

Hope this helps.
Boris

> For the purpose of generating a SmallInteger (from XML ya know..), Object readFrom:'42' works just fine!

Do _not_ do that. This is a _huge_ security hole. What it does it
evalutates the string. This string can be any Smalltalk code. This way
you have aribrary code execution in Smalltalk.

An exploit for this would look like this:
Object readFrom: 'SmalltalkImage current snapshot: false andQuit: true'

Do
Number readFrom: aString
instead. This has its own problems like that
Number readFrom: 'garbage'
returns 0 but this will be fixed and at leas it's safe.

Please not that also Boolean class >> #readFrom: is borken in the same
way. This is the reason why you can execute arbitrary Smalltalk code
in every Squeak image that uses SOAP either as client or server.
Combine that with FFI and X11 root exploits and you have a nightmare.

Cheers
Philippe

Wasn't aware of this. Code fixed ;) !

>An exploit for this would look like this:
>Object readFrom: 'SmalltalkImage current snapshot: false andQuit: true'

It is like turning off the computer...
e.g. I can do the same by hardware.
How you make the software "secure" to do not let my children
 turn off my computer when I am using Squeak?

In practice, use of free scripting has bring powerfull
 experiences for "power"users ussing commercial
 applications in small and big products.
Smalltalk let power users talk to your system's objects,
 and you can publish the hight level language you want
 to be used and expose the objects they need in the
 context of application.
The mechanism promoted with Parts (VS) was very interesting
 and not followed by any other dialect of smalltalk (imho because
 it's power requires experience in it's use to be observed).
Parts let power user's customize teh GUI of the product
 following the constrains emerging from the underlying
 model of the core system.
It was better than compilation because parts was loaded/saved
 in binary mode and do not requires compilation (a really
 slooow process)

best;
Ale.


----- Original Message -----
From: "Philippe Marschall" <[hidden email]>
To: "The general-purpose Squeak developers list"
<[hidden email]>
Sent: Saturday, May 13, 2006 7:20 AM
Subject: Re: Re: Issues creating instance of SmallInteger


> For the purpose of generating a SmallInteger (from XML ya know..), Object
readFrom:'42' works just fine!

Do _not_ do that. This is a _huge_ security hole. What it does it
evalutates the string. This string can be any Smalltalk code. This way
you have aribrary code execution in Smalltalk.

An exploit for this would look like this:
Object readFrom: 'SmalltalkImage current snapshot: false andQuit: true'

Do
Number readFrom: aString
instead. This has its own problems like that
Number readFrom: 'garbage'
returns 0 but this will be fixed and at leas it's safe.

Please not that also Boolean class >> #readFrom: is borken in the same
way. This is the reason why you can execute arbitrary Smalltalk code
in every Squeak image that uses SOAP either as client or server.
Combine that with FFI and X11 root exploits and you have a nightmare.

Cheers
Philippe

FYI ...

This would have been quite safe with the OOZE object swapping system done
for Smalltalk-76 at PARC. It guaranteed that no matter what happened, your
image would be consistent and safe and no more than 20 seconds out of date.
This was a very pretty design and implementation by Dan Ingalls and Ted
Kaehler.

Cheers,

Alan

At 06:27 AM 5/13/2006, Alejandro F. Reimondo wrote:

2006/5/13, Alejandro F. Reimondo <[hidden email]>:
>
> >An exploit for this would look like this:
> >Object readFrom: 'SmalltalkImage current snapshot: false andQuit: true'
>
> It is like turning off the computer...

Yes, it is also called DoS attack.

> In practice, use of free scripting has bring powerfull
>  experiences for "power"users ussing commercial
>  applications in small and big products.
> Smalltalk let power users talk to your system's objects,
>  and you can publish the hight level language you want
>  to be used and expose the objects they need in the
>  context of application.

Yes and by doing Object/Boolean class >> #readFrom: on data received
from the web you give the same power to balckhat hackers. No longer do
they need to write their exploits in hardware language. The can use
Smalltalk sourcecode which is highlevel and portable. The can also
make use of powerful Smalltalk features like:
- walk over all instances of all classes that have "user" in their
name and inspect them
- use the compiler (add or change classes or methods)
- change the compiler
- change to tools to not show code they added or changed
- use FFI
- send the whole image somewhere
- ...

Philippe

I told you that they were cool :)
And this is an excellent way to learn and have fun.


On 12 mai 06, at 20:24, Michel Calonne wrote:

> Many thanks to all of those who answered me !!
> Even if I had trouble understanding some of the technical stuff  
> about SmallIntegers (although I should know what immediates are  
> now! :), it should help me get rid of this XML serializer project  
> we've gotta do for 1 week from here!
>
> For the purpose of generating a SmallInteger (from XML ya know..),  
> Object readFrom:'42' works just fine!

:)
I thought that you did not provide enough context to get the real  
answer you wanted. But we all learned something :)

On 13 mai 06, at 12:20, Philippe Marschall wrote:

True!!!! Argh!
The problem is that it forces you to save in the file also the type  
of your field (except if we would use
magritte :) but this is for a later version.


Philippe do you have fix?
You know me :)

Stef


>
> Cheers
> Philippe
>
>

> The mechanism promoted with Parts (VS) was very interesting
>  and not followed by any other dialect of smalltalk (imho because
>  it's power requires experience in it's use to be observed).
> Parts let power user's customize teh GUI of the product
>  following the constrains emerging from the underlying
>  model of the core system.

Could you explain a bit more. Because this is still really cryptic  
for me :)

Stef