UTCDateAndTime updated for Squeak 4.6/5.0

UTCDateAndTime is a UTC based implementation of class DateAndTime with
one instance variable representing the magnitude of the point in time,
and another representing local time zone offset.

I recommend loading with the SAR from http://wiki.squeak.org/squeak/6197,
as maintaining this in Montecello is problematic.

After the 4.6/5.0 release, I would like to suggest moving Chronology out
of the Kernel package into its own package, so that changes in Chronology
can be maintained in Montecello without conflicting with the rest of Kernel.

Spur notes: The Spur image provides a huge performance when loading this
set of changes. There are two steps in the loading process in which all
DateAndTime instances need to be #becomed into new instances. This is
painfully slow in the traditional image (with or without Cog), and it is
amazing fast in Spur.

That said, there is a bug in the Spur VM (easily resolved I think) that
produces incorrect time values. I'll post note about that on vm-dev. I
would not recommend using UTCDateAndTime in Spur yet, other than just
to see how amazingly fast it is compared to loading the same thing in
a non-Spur image.

Dave

On Sun, 24 May 2015, David T. Lewis wrote:

Similar speed can be achieved in V3 images (on any VM), if all instances
are exchanged in one shot. In LX-2.2.cs:

| oldInstances newInstances |
oldInstances := DateAndTime allInstances, TimeStamp allInstances.
newInstances := oldInstances collect: [ :each |
  each class == DateAndTime
  ifTrue: [ each asLXDateAndTime ]
  ifFalse: [ each asLXTimeStamp ] ].
oldInstances elementsForwardIdentityTo: newInstances.

And in LX-4.1.cs:

| oldInstances newInstances |
oldInstances := LXDateAndTime allInstances, LXTimeStamp allInstances.
newInstances := oldInstances collect: [ :each |
  each class == LXDateAndTime
  ifTrue: [ each asDateAndTime ]
  ifFalse: [ each asTimeStamp ] ].
oldInstances elementsForwardIdentityTo: newInstances.


Levente

On Sun, May 24, 2015 at 11:22:48PM +0200, Levente Uzonyi wrote:
Levente,

Thank you! I was not aware of that.

Dave

On Mon, May 25, 2015 at 10:48:20AM -0400, David T. Lewis wrote:
I updated the SAR at http://wiki.squeak.org/squeak/6197. Indeed it now loads
very quickly in a V3 image.

Dave

On Mon, 25 May 2015, David T. Lewis wrote:

Cool. I've got another change set, which makes instance creation more than
twice as fast:
http://leves.web.elte.hu/squeak/DateAndTimeCreationSpeedup.1.cs

Levente

>
> Dave
>
>

Excellent, thank you Levente. I'll make the update as soon as I can (late
today).

FYI, I added you as developer on the SS3 repository
http://ss3.gemstone.com/ss/UTCDateAndTime.html (but that does not mean I
expect you to keep it updated, doing this in MC is a pain right now).

Dave

On Tue, May 26, 2015 at 03:39:01AM +0200, Levente Uzonyi wrote:
Thanks Levente,

Added to the SAR at http://wiki.squeak.org/squeak/6197 and to the repository
at http://ss3.gemstone.com/ss/UTCDateAndTime.html. I also updated #hasUtcPrim
for startup processing to use your #primPosixMicrosecondClockWithOffset change.

Dave

On Tue, 26 May 2015, David T. Lewis wrote:

Thanks Dave!

Levente

On Sun, May 24, 2015 at 12:36:02PM -0400, David T. Lewis wrote:
> UTCDateAndTime is a UTC based implementation of class DateAndTime with
> one instance variable representing the magnitude of the point in time,
> and another representing local time zone offset.

I have updated the UTCDateAndTime package to make it loadable in the latest
Squeak trunk and Spur.

A new Monticello repository is at http://www.squeaksource.com/UTCDateAndTime.
The home page (with a new SAR) is at http://wiki.squeak.org/squeak/6197.

Starting with an updated trunk image, you can load UTCDateAndTime in two ways:

1) Open the http://www.squeaksource.com/UTCDateAndTime repository, and load
the MCZ files in sequence beginning with Chronology-Core-dtl.3.

2) In a preferences browser, in category 'updates' set the 'Update URL'
preference to 'http://www.squeaksource.com/UTCDateAndTime', and do
world -> help... -> update code from server.

The main objective of UTCDateAndTime is to make DateAndTime conceptually
simpler, but a happy side effect is that it is also significantly faster
than the old implementation.

Dave

> The main objective of UTCDateAndTime is to make DateAndTime conceptually
> simpler, but a happy side effect is that it is also significantly faster
> than the old implementation.

Hi Dave, when you first introduced UTCDateAndTime, it was originally
significantly faster than legacy Chronology because legacy had a gross
inefficiency in it.  You work exposed that inefficiency, which was
then fixed, pretty much neutralizing the performance advantage of
UTCDateAndTime, of that time.

If you have made further improvements since then, I would be
interested to see your benchmarks again.

One thing I wondered was, due to greater SmallInteger range, whether
your implementation would be significantly faster in 64-bit image.  In
32-bit though, IIRC, it no longer is..

Hi Chris,

I did a quick check last night and it seemed to be about 2x faster overall
on 32 bit Spur, except for DateAndTime>>= which is now slower. The
performance check utility is in the squeaksource repository if you want to
take a look at it. YMMV.

Dave

That's great news, I'll check it out.  Thanks.

On Mon, Mar 7, 2016 at 12:17 PM, David T. Lewis <[hidden email]> wrote:

On Mon, Mar 07, 2016 at 12:21:38AM -0500, David T. Lewis wrote:
> On Sun, May 24, 2015 at 12:36:02PM -0400, David T. Lewis wrote:
> > UTCDateAndTime is a UTC based implementation of class DateAndTime with
> > one instance variable representing the magnitude of the point in time,
> > and another representing local time zone offset.
>
> I have updated the UTCDateAndTime package to make it loadable in the latest
> Squeak trunk and Spur.

Has anyone looked at this yet? Any interest?

Dave

Hi David,

I just loaded the code into the latest 64-bit Spur image and VM.
Everything works so far, and the overall performance is better as well.
For those who care about performance, the speedups are

#testNow->3.01.
#testEquals->1.83.
#testGreaterThan->1.86.
#testLessThan->1.92.
#testPrintString->0.85.
#testStringAsDateAndTime->1.75.

So, #testPrintString is slighly slower, but everything else is
significanlty quicker.
I haven't reviewed the code yet, but I'll do.

Levente

On Sat, 12 Mar 2016, David T. Lewis wrote:

Hi Levente,

I had not checked the 64-bit image yet, thanks. I did notice from the
32-bit image that the DateAndTime>>= test shows that we might have an
opportunity to improve #digitCompare: performance, or maybe think of
a better way to do the #= implementation. Your 64-bit results show that
it is not a problem for the 64-bit image (because we would be dealing
with immediate small integers there).

It's interesting to see that Spur is speeding up overall performance
to the point that some primitives are starting to appear relatively
slow in comparison. That is very nice problem to have :-)

Dave


On Sat, Mar 12, 2016 at 07:39:14PM +0100, Levente Uzonyi wrote:

I always forget to mention, even though I have had this idea since you had
introduced primitiveUtcWithOffset, that it would be better if the
primitive could take an optional argument to store the values in the first
two slots of it instead of creating a new Array. If it's too much burden
to accept any object, then the argument type can be limited to Array.
This change would make it possible to decrease the GC pressure when many
timestamps are created in a row.

Levente

On Sat, 12 Mar 2016, David T. Lewis wrote:

Hi Dave,

I ran the tests and found that there are three types of failures.
1. Some tests assume that the default offset is #localOffset. I don't
think it's correct, but that's how is it in Trunk.
2. Hashes have changed. This implies that HashedCollections containing
DateAndTime instances as key will have to be rehashed. In the base Trunk
image there are appear to be no such HashedCollections, but not rehashing
such collections will break images with custom packages in them after
update.
3. Loss of resolution and strict monotonicity. In Trunk [DateAndTime now <
DateAndTime now] is always true. This is possible because DateAndTime has
nanosecond resolution, but the source clock only has microsecond
resolution, and our machines are so slow that it takes more than a
nanosecond to execute the primitive. All these allow us to use the
sub-microsecond part of the timestamp to create intermediate strictly
monotonic values.

Levente

On Sat, 12 Mar 2016, David T. Lewis wrote:

I had the same idea, and I already implemented it in trunk interpreter
VMMaker, but it is probably not in oscog yet. I did not add it on the
image side, although we could do so if we update the VMs.

Dave

Apologies, this is not right. I was replying from a cell phone and my
recollection was wrong.

I did at one point implement a #primitiveUtcWithOffset that could receive
an array of size two as an argument into which the result values are stored,
instead if allocating the array in the primitive. However, I did *not* commit
it to VMMaker, and it is not in any currently available VMs.

I think I decided at the time that this approach was dangerous because it
would invite problems in cases involving multiple processes, where some
other process might call the primitive using the same array. In that case
the time value in the array would magically change without the first process
being aware.

I still think that it is a good idea, but maybe not worth the risk unless
there is a noticable affect of performance or GC activity. Of course, adding
the capability to the primitive would not force anyone to actually use it
that way.

Sorry for the misinformation.

Dave


On Sat, Mar 12, 2016 at 11:37:55PM -0500, David T. Lewis wrote:

On Sat, Mar 12, 2016 at 08:48:21PM +0100, Levente Uzonyi wrote:
> Hi Dave,
>
> I ran the tests and found that there are three types of failures.
> 1. Some tests assume that the default offset is #localOffset. I don't
> think it's correct, but that's how is it in Trunk.

There are 10 failing tests in my image. These are cases in which either
I was uncertain how to handle the issue, or where I thought that the
current behaviour in trunk may be wrong.

> 2. Hashes have changed. This implies that HashedCollections containing
> DateAndTime instances as key will have to be rehashed. In the base Trunk
> image there are appear to be no such HashedCollections, but not rehashing
> such collections will break images with custom packages in them after
> update.

I am not expert in this area at all. But part of the update process
(in the MC postscripts) finds all instances of the old style DateAndTime,
and becomes them into the new ones. Would the HashedCollections contianing
DateAndTime instances as key still require rehash in that case?

Related to DateAndTime>>hash, I would also like to ask for guidance as
to how DateAndTime>>= should be defined. Maybe this is an ANSI Smalltalk
question, but I was not sure if we should be thinking of DateAndTime as
a magnitude, such that two instances are equal if their utcMicroseconds
are the same? Or are two instances equal if they have the same magnitude
and also the same localOffsetSeconds?

I implemented the first approach (two instances are #= if they represent
the same UTC time, regardless of timezone). There may be use cases
where the other approach makes sense. But my thinking was that when
comparing two instances of DateAndTime, they should not be considered
the same unless they both represent the same point in time, hence UTC
time regardless of the time zone in which the instance was created.

> 3. Loss of resolution and strict monotonicity. In Trunk [DateAndTime now <
> DateAndTime now] is always true. This is possible because DateAndTime has
> nanosecond resolution, but the source clock only has microsecond
> resolution, and our machines are so slow that it takes more than a
> nanosecond to execute the primitive. All these allow us to use the
> sub-microsecond part of the timestamp to create intermediate strictly
> monotonic values.
>

To be honest, I do not fully recall the rationale for this. One
important case is that of the system clock being set back due to
ntp synchronization. Fiddling with the nanoseconds number to make
it appear as if the OS had reported an increase in time (when in
fact it did not) seems questionable to me, but I know that it was
done for some good reason. I do remember the discussions, so I
should look back through the squeak-dev archives and find out the
reason this was done.

> Levente
>

Thanks very much for looking at this. I added you and Chris Muller as
developers on the UTCDateAndTime repository on squeaksource.com in case
you want to make any changes.

Much appreciated,

Dave