UTCDateAndTime updated for Squeak 4.6/5.0

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
71 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

Levente Uzonyi
On Sun, 13 Mar 2016, David T. Lewis wrote:

> Apologies, this is not right. I was replying from a cell phone and my
> recollection was wrong.
>
> I did at one point implement a #primitiveUtcWithOffset that could receive
> an array of size two as an argument into which the result values are stored,
> instead if allocating the array in the primitive. However, I did *not* commit
> it to VMMaker, and it is not in any currently available VMs.
>
> I think I decided at the time that this approach was dangerous because it
> would invite problems in cases involving multiple processes, where some
> other process might call the primitive using the same array. In that case
> the time value in the array would magically change without the first process
> being aware.

That could only happen if the Array would be shared by multiple processes.
For example if there were a dedicated class variable to retrieve the
values from the VM, but that would be nothing but bad design.

Why I thought that the argument should be of any Object with two slots is
that this would allow us to use the DateAndTime instance itself to fetch
the values into. Or anyone could come up with their own class to store
timestamps and still be able to use this primitive.

>
> I still think that it is a good idea, but maybe not worth the risk unless
> there is a noticable affect of performance or GC activity. Of course, adding
> the capability to the primitive would not force anyone to actually use it
> that way.

We have a long running application, which most of the time generates a few
hundred timestamps per second, but sometimes is goes up to the thousands
range. That would definitely benefit from such feature.

Levente

>
> Sorry for the misinformation.
>
> Dave
>
>
> On Sat, Mar 12, 2016 at 11:37:55PM -0500, David T. Lewis wrote:
>> I had the same idea, and I already implemented it in trunk interpreter
>> VMMaker, but it is probably not in oscog yet. I did not add it on the
>> image side, although we could do so if we update the VMs.
>>
>> Dave
>>
>>> I always forget to mention, even though I have had this idea since you had
>>> introduced primitiveUtcWithOffset, that it would be better if the
>>> primitive could take an optional argument to store the values in the first
>>> two slots of it instead of creating a new Array. If it's too much burden
>>> to accept any object, then the argument type can be limited to Array.
>>> This change would make it possible to decrease the GC pressure when many
>>> timestamps are created in a row.
>>>
>>> Levente
>>>
>>> On Sat, 12 Mar 2016, David T. Lewis wrote:
>>>
>>>> On Mon, Mar 07, 2016 at 12:21:38AM -0500, David T. Lewis wrote:
>>>>> On Sun, May 24, 2015 at 12:36:02PM -0400, David T. Lewis wrote:
>>>>>> UTCDateAndTime is a UTC based implementation of class DateAndTime with
>>>>>> one instance variable representing the magnitude of the point in time,
>>>>>> and another representing local time zone offset.
>>>>>
>>>>> I have updated the UTCDateAndTime package to make it loadable in the
>>>>> latest
>>>>> Squeak trunk and Spur.
>>>>
>>>> Has anyone looked at this yet? Any interest?
>>>>
>>>> Dave
>>>>
>>>>>
>>>>> A new Monticello repository is at
>>>>> http://www.squeaksource.com/UTCDateAndTime.
>>>>> The home page (with a new SAR) is at
>>>>> http://wiki.squeak.org/squeak/6197.
>>>>>
>>>>> Starting with an updated trunk image, you can load UTCDateAndTime in
>>>>> two ways:
>>>>>
>>>>> 1) Open the http://www.squeaksource.com/UTCDateAndTime repository, and
>>>>> load
>>>>> the MCZ files in sequence beginning with Chronology-Core-dtl.3.
>>>>>
>>>>> 2) In a preferences browser, in category 'updates' set the 'Update URL'
>>>>> preference to 'http://www.squeaksource.com/UTCDateAndTime', and do
>>>>> world -> help... -> update code from server.
>>>>>
>>>>> The main objective of UTCDateAndTime is to make DateAndTime
>>>>> conceptually
>>>>> simpler, but a happy side effect is that it is also significantly
>>>>> faster
>>>>> than the old implementation.
>>>>>
>>>>> Dave
>>>>
>>>>
>>>
>>
>>
>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

David T. Lewis
On Sun, Mar 13, 2016 at 10:28:19PM +0100, Levente Uzonyi wrote:

> On Sun, 13 Mar 2016, David T. Lewis wrote:
>
> >Apologies, this is not right. I was replying from a cell phone and my
> >recollection was wrong.
> >
> >I did at one point implement a #primitiveUtcWithOffset that could receive
> >an array of size two as an argument into which the result values are
> >stored,
> >instead if allocating the array in the primitive. However, I did *not*
> >commit
> >it to VMMaker, and it is not in any currently available VMs.
> >
> >I think I decided at the time that this approach was dangerous because it
> >would invite problems in cases involving multiple processes, where some
> >other process might call the primitive using the same array. In that case
> >the time value in the array would magically change without the first
> >process
> >being aware.
>
> That could only happen if the Array would be shared by multiple processes.
> For example if there were a dedicated class variable to retrieve the
> values from the VM, but that would be nothing but bad design.

Well yes, but bad design is a very common failure mode ;-)

>
> Why I thought that the argument should be of any Object with two slots is
> that this would allow us to use the DateAndTime instance itself to fetch
> the values into. Or anyone could come up with their own class to store
> timestamps and still be able to use this primitive.

That is absolutely brilliant! What a great idea.

  DateAndTime class>>now
    ^self basicNew initializeFromPrimitive

Dave


Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

David T. Lewis
In reply to this post by David T. Lewis
Urk. Sorry for the confusion, but I was looking at the wrong image and I see
now that my original statement was correct:

   "I had the same idea, and I already implemented it in trunk interpreter
   VMMaker, but it is probably not in oscog yet. I did not add it on the
   image side, although we could do so if we update the VMs."

This is in the trunk VMM (interpreter VM), though not in Cog/Spur yet. Better
yet, it already works with Levente's idea (see below) of passing the DateAndTime
instance to the primitive. No further changes to the primitive are needed to
do this.

Therefore the following works in a Squeak 4.6 image with UTCDateAndTime loaded,
and running on an interpreter VM:

  DateAndTime>>initializeFromPrimitive
        ^Time primPosixMicrosecondClockWithOffset: self

  DateAndTime basicNew initializeFromPrimitive ==> 2016-03-14T21:38:49.81355-04:00

So I think this is a very good idea that just needs a couple of minor VM
updates to make it available.

Dave


On Sun, Mar 13, 2016 at 03:03:55PM -0400, David T. Lewis wrote:

> Apologies, this is not right. I was replying from a cell phone and my
> recollection was wrong.
>
> I did at one point implement a #primitiveUtcWithOffset that could receive
> an array of size two as an argument into which the result values are stored,
> instead if allocating the array in the primitive. However, I did *not* commit
> it to VMMaker, and it is not in any currently available VMs.
>
> I think I decided at the time that this approach was dangerous because it
> would invite problems in cases involving multiple processes, where some
> other process might call the primitive using the same array. In that case
> the time value in the array would magically change without the first process
> being aware.
>
> I still think that it is a good idea, but maybe not worth the risk unless
> there is a noticable affect of performance or GC activity. Of course, adding
> the capability to the primitive would not force anyone to actually use it
> that way.
>
> Sorry for the misinformation.
>
> Dave
>
>
> On Sat, Mar 12, 2016 at 11:37:55PM -0500, David T. Lewis wrote:
> > I had the same idea, and I already implemented it in trunk interpreter
> > VMMaker, but it is probably not in oscog yet. I did not add it on the
> > image side, although we could do so if we update the VMs.
> >
> > Dave
> >
> > > I always forget to mention, even though I have had this idea since you had
> > > introduced primitiveUtcWithOffset, that it would be better if the
> > > primitive could take an optional argument to store the values in the first
> > > two slots of it instead of creating a new Array. If it's too much burden
> > > to accept any object, then the argument type can be limited to Array.
> > > This change would make it possible to decrease the GC pressure when many
> > > timestamps are created in a row.
> > >
> > > Levente
> > >
> > > On Sat, 12 Mar 2016, David T. Lewis wrote:
> > >
> > >> On Mon, Mar 07, 2016 at 12:21:38AM -0500, David T. Lewis wrote:
> > >>> On Sun, May 24, 2015 at 12:36:02PM -0400, David T. Lewis wrote:
> > >>>> UTCDateAndTime is a UTC based implementation of class DateAndTime with
> > >>>> one instance variable representing the magnitude of the point in time,
> > >>>> and another representing local time zone offset.
> > >>>
> > >>> I have updated the UTCDateAndTime package to make it loadable in the
> > >>> latest
> > >>> Squeak trunk and Spur.
> > >>
> > >> Has anyone looked at this yet? Any interest?
> > >>
> > >> Dave
> > >>
> > >>>
> > >>> A new Monticello repository is at
> > >>> http://www.squeaksource.com/UTCDateAndTime.
> > >>> The home page (with a new SAR) is at
> > >>> http://wiki.squeak.org/squeak/6197.
> > >>>
> > >>> Starting with an updated trunk image, you can load UTCDateAndTime in
> > >>> two ways:
> > >>>
> > >>> 1) Open the http://www.squeaksource.com/UTCDateAndTime repository, and
> > >>> load
> > >>> the MCZ files in sequence beginning with Chronology-Core-dtl.3.
> > >>>
> > >>> 2) In a preferences browser, in category 'updates' set the 'Update URL'
> > >>> preference to 'http://www.squeaksource.com/UTCDateAndTime', and do
> > >>> world -> help... -> update code from server.
> > >>>
> > >>> The main objective of UTCDateAndTime is to make DateAndTime
> > >>> conceptually
> > >>> simpler, but a happy side effect is that it is also significantly
> > >>> faster
> > >>> than the old implementation.
> > >>>
> > >>> Dave
> > >>
> > >>
> > >
> >
> >
> >
> >

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

Bert Freudenberg
If we have to touch the VM for this anyway, is there any reason not to use the Smalltalk epoch for this primitive, as Eliot suggested? This would put it in line with the other primitives (and make it slightly cheaper to use the skewed VM time).

- Bert -

> On 15.03.2016, at 02:50, David T. Lewis <[hidden email]> wrote:
>
> Urk. Sorry for the confusion, but I was looking at the wrong image and I see
> now that my original statement was correct:
>
>   "I had the same idea, and I already implemented it in trunk interpreter
>   VMMaker, but it is probably not in oscog yet. I did not add it on the
>   image side, although we could do so if we update the VMs."
>
> This is in the trunk VMM (interpreter VM), though not in Cog/Spur yet. Better
> yet, it already works with Levente's idea (see below) of passing the DateAndTime
> instance to the primitive. No further changes to the primitive are needed to
> do this.
>
> Therefore the following works in a Squeak 4.6 image with UTCDateAndTime loaded,
> and running on an interpreter VM:
>
>  DateAndTime>>initializeFromPrimitive
> ^Time primPosixMicrosecondClockWithOffset: self
>
>  DateAndTime basicNew initializeFromPrimitive ==> 2016-03-14T21:38:49.81355-04:00
>
> So I think this is a very good idea that just needs a couple of minor VM
> updates to make it available.
>
> Dave
>
>
> On Sun, Mar 13, 2016 at 03:03:55PM -0400, David T. Lewis wrote:
>> Apologies, this is not right. I was replying from a cell phone and my
>> recollection was wrong.
>>
>> I did at one point implement a #primitiveUtcWithOffset that could receive
>> an array of size two as an argument into which the result values are stored,
>> instead if allocating the array in the primitive. However, I did *not* commit
>> it to VMMaker, and it is not in any currently available VMs.
>>
>> I think I decided at the time that this approach was dangerous because it
>> would invite problems in cases involving multiple processes, where some
>> other process might call the primitive using the same array. In that case
>> the time value in the array would magically change without the first process
>> being aware.
>>
>> I still think that it is a good idea, but maybe not worth the risk unless
>> there is a noticable affect of performance or GC activity. Of course, adding
>> the capability to the primitive would not force anyone to actually use it
>> that way.
>>
>> Sorry for the misinformation.
>>
>> Dave
>>
>>
>> On Sat, Mar 12, 2016 at 11:37:55PM -0500, David T. Lewis wrote:
>>> I had the same idea, and I already implemented it in trunk interpreter
>>> VMMaker, but it is probably not in oscog yet. I did not add it on the
>>> image side, although we could do so if we update the VMs.
>>>
>>> Dave
>>>
>>>> I always forget to mention, even though I have had this idea since you had
>>>> introduced primitiveUtcWithOffset, that it would be better if the
>>>> primitive could take an optional argument to store the values in the first
>>>> two slots of it instead of creating a new Array. If it's too much burden
>>>> to accept any object, then the argument type can be limited to Array.
>>>> This change would make it possible to decrease the GC pressure when many
>>>> timestamps are created in a row.
>>>>
>>>> Levente
>>>>
>>>> On Sat, 12 Mar 2016, David T. Lewis wrote:
>>>>
>>>>> On Mon, Mar 07, 2016 at 12:21:38AM -0500, David T. Lewis wrote:
>>>>>> On Sun, May 24, 2015 at 12:36:02PM -0400, David T. Lewis wrote:
>>>>>>> UTCDateAndTime is a UTC based implementation of class DateAndTime with
>>>>>>> one instance variable representing the magnitude of the point in time,
>>>>>>> and another representing local time zone offset.
>>>>>>
>>>>>> I have updated the UTCDateAndTime package to make it loadable in the
>>>>>> latest
>>>>>> Squeak trunk and Spur.
>>>>>
>>>>> Has anyone looked at this yet? Any interest?
>>>>>
>>>>> Dave
>>>>>
>>>>>>
>>>>>> A new Monticello repository is at
>>>>>> http://www.squeaksource.com/UTCDateAndTime.
>>>>>> The home page (with a new SAR) is at
>>>>>> http://wiki.squeak.org/squeak/6197.
>>>>>>
>>>>>> Starting with an updated trunk image, you can load UTCDateAndTime in
>>>>>> two ways:
>>>>>>
>>>>>> 1) Open the http://www.squeaksource.com/UTCDateAndTime repository, and
>>>>>> load
>>>>>> the MCZ files in sequence beginning with Chronology-Core-dtl.3.
>>>>>>
>>>>>> 2) In a preferences browser, in category 'updates' set the 'Update URL'
>>>>>> preference to 'http://www.squeaksource.com/UTCDateAndTime', and do
>>>>>> world -> help... -> update code from server.
>>>>>>
>>>>>> The main objective of UTCDateAndTime is to make DateAndTime
>>>>>> conceptually
>>>>>> simpler, but a happy side effect is that it is also significantly
>>>>>> faster
>>>>>> than the old implementation.
>>>>>>
>>>>>> Dave
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>>
>




smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

David T. Lewis
On Tue, Mar 15, 2016 at 10:02:20AM +0100, Bert Freudenberg wrote:
> If we have to touch the VM for this anyway, is there any reason not to use the Smalltalk epoch for this primitive, as Eliot suggested? This would put it in line with the other primitives (and make it slightly cheaper to use the skewed VM time).
>
> - Bert -

There is no requirement to touch the VM, the current implementation in Cog/Spur
is sufficient. Levente's suggestion is a nice enhancement but not a requirement.
I have already added this to the UTCDateAndTime package in such a way that
Levente's approach is used if the VM supports it, otherwise my original
implementation is used at the cost of one extra Array instantiation when
calling DateAndTime now.

I would not want to change the current primitive, but it would be easy to
add an additional one with a different name such that a transition could be
done at a later time if someone wanted to do so. This is not a numbered
primitive, so it is not a problem to add a different implementation with
a slightly different name.

>From the point of view of UTCDateAndTime, using the Smalltalk epoch would be
a disadvantage because Olson time zone tables are based on Posix epoch. This
means that you would be converting Posix -> Smalltalk in the VM, then converting
it back to the Posix time base in the image.

The background on this is that I would like to be able to do a clean integration
of TimeZoneDatabase (the Olson tz tables) with DateAndTime, and get rid of
the historical baggage associated with Squeak's original reliance on local
time in the VM. The overall result should be much easier to understand and
test, and will use concepts that are well documented on wikipedia.

Dave


> > On 15.03.2016, at 02:50, David T. Lewis <[hidden email]> wrote:
> >
> > Urk. Sorry for the confusion, but I was looking at the wrong image and I see
> > now that my original statement was correct:
> >
> >   "I had the same idea, and I already implemented it in trunk interpreter
> >   VMMaker, but it is probably not in oscog yet. I did not add it on the
> >   image side, although we could do so if we update the VMs."
> >
> > This is in the trunk VMM (interpreter VM), though not in Cog/Spur yet. Better
> > yet, it already works with Levente's idea (see below) of passing the DateAndTime
> > instance to the primitive. No further changes to the primitive are needed to
> > do this.
> >
> > Therefore the following works in a Squeak 4.6 image with UTCDateAndTime loaded,
> > and running on an interpreter VM:
> >
> >  DateAndTime>>initializeFromPrimitive
> > ^Time primPosixMicrosecondClockWithOffset: self
> >
> >  DateAndTime basicNew initializeFromPrimitive ==> 2016-03-14T21:38:49.81355-04:00
> >
> > So I think this is a very good idea that just needs a couple of minor VM
> > updates to make it available.
> >
> > Dave
> >
> >
> > On Sun, Mar 13, 2016 at 03:03:55PM -0400, David T. Lewis wrote:
> >> Apologies, this is not right. I was replying from a cell phone and my
> >> recollection was wrong.
> >>
> >> I did at one point implement a #primitiveUtcWithOffset that could receive
> >> an array of size two as an argument into which the result values are stored,
> >> instead if allocating the array in the primitive. However, I did *not* commit
> >> it to VMMaker, and it is not in any currently available VMs.
> >>
> >> I think I decided at the time that this approach was dangerous because it
> >> would invite problems in cases involving multiple processes, where some
> >> other process might call the primitive using the same array. In that case
> >> the time value in the array would magically change without the first process
> >> being aware.
> >>
> >> I still think that it is a good idea, but maybe not worth the risk unless
> >> there is a noticable affect of performance or GC activity. Of course, adding
> >> the capability to the primitive would not force anyone to actually use it
> >> that way.
> >>
> >> Sorry for the misinformation.
> >>
> >> Dave
> >>
> >>
> >> On Sat, Mar 12, 2016 at 11:37:55PM -0500, David T. Lewis wrote:
> >>> I had the same idea, and I already implemented it in trunk interpreter
> >>> VMMaker, but it is probably not in oscog yet. I did not add it on the
> >>> image side, although we could do so if we update the VMs.
> >>>
> >>> Dave
> >>>
> >>>> I always forget to mention, even though I have had this idea since you had
> >>>> introduced primitiveUtcWithOffset, that it would be better if the
> >>>> primitive could take an optional argument to store the values in the first
> >>>> two slots of it instead of creating a new Array. If it's too much burden
> >>>> to accept any object, then the argument type can be limited to Array.
> >>>> This change would make it possible to decrease the GC pressure when many
> >>>> timestamps are created in a row.
> >>>>
> >>>> Levente
> >>>>
> >>>> On Sat, 12 Mar 2016, David T. Lewis wrote:
> >>>>
> >>>>> On Mon, Mar 07, 2016 at 12:21:38AM -0500, David T. Lewis wrote:
> >>>>>> On Sun, May 24, 2015 at 12:36:02PM -0400, David T. Lewis wrote:
> >>>>>>> UTCDateAndTime is a UTC based implementation of class DateAndTime with
> >>>>>>> one instance variable representing the magnitude of the point in time,
> >>>>>>> and another representing local time zone offset.
> >>>>>>
> >>>>>> I have updated the UTCDateAndTime package to make it loadable in the
> >>>>>> latest
> >>>>>> Squeak trunk and Spur.
> >>>>>
> >>>>> Has anyone looked at this yet? Any interest?
> >>>>>
> >>>>> Dave
> >>>>>
> >>>>>>
> >>>>>> A new Monticello repository is at
> >>>>>> http://www.squeaksource.com/UTCDateAndTime.
> >>>>>> The home page (with a new SAR) is at
> >>>>>> http://wiki.squeak.org/squeak/6197.
> >>>>>>
> >>>>>> Starting with an updated trunk image, you can load UTCDateAndTime in
> >>>>>> two ways:
> >>>>>>
> >>>>>> 1) Open the http://www.squeaksource.com/UTCDateAndTime repository, and
> >>>>>> load
> >>>>>> the MCZ files in sequence beginning with Chronology-Core-dtl.3.
> >>>>>>
> >>>>>> 2) In a preferences browser, in category 'updates' set the 'Update URL'
> >>>>>> preference to 'http://www.squeaksource.com/UTCDateAndTime', and do
> >>>>>> world -> help... -> update code from server.
> >>>>>>
> >>>>>> The main objective of UTCDateAndTime is to make DateAndTime
> >>>>>> conceptually
> >>>>>> simpler, but a happy side effect is that it is also significantly
> >>>>>> faster
> >>>>>> than the old implementation.
> >>>>>>
> >>>>>> Dave
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >
>
>



>


Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

Levente Uzonyi
In reply to this post by David T. Lewis
Hi Dave,

I finally had some time to dig into the code and the tests.

On Sun, 13 Mar 2016, David T. Lewis wrote:

> On Sat, Mar 12, 2016 at 08:48:21PM +0100, Levente Uzonyi wrote:
>> Hi Dave,
>>
>> I ran the tests and found that there are three types of failures.
>> 1. Some tests assume that the default offset is #localOffset. I don't
>> think it's correct, but that's how is it in Trunk.
>
> There are 10 failing tests in my image. These are cases in which either
> I was uncertain how to handle the issue, or where I thought that the
> current behaviour in trunk may be wrong.
>
>> 2. Hashes have changed. This implies that HashedCollections containing
>> DateAndTime instances as key will have to be rehashed. In the base Trunk
>> image there are appear to be no such HashedCollections, but not rehashing
>> such collections will break images with custom packages in them after
>> update.
>
> I am not expert in this area at all. But part of the update process
> (in the MC postscripts) finds all instances of the old style DateAndTime,
> and becomes them into the new ones. Would the HashedCollections contianing
> DateAndTime instances as key still require rehash in that case?

Yes, it is. When their hash changes, these objects will not be found in
such collections.

>
> Related to DateAndTime>>hash, I would also like to ask for guidance as
> to how DateAndTime>>= should be defined. Maybe this is an ANSI Smalltalk
> question, but I was not sure if we should be thinking of DateAndTime as
> a magnitude, such that two instances are equal if their utcMicroseconds
> are the same? Or are two instances equal if they have the same magnitude
> and also the same localOffsetSeconds?
>
> I implemented the first approach (two instances are #= if they represent
> the same UTC time, regardless of timezone). There may be use cases
> where the other approach makes sense. But my thinking was that when
> comparing two instances of DateAndTime, they should not be considered
> the same unless they both represent the same point in time, hence UTC
> time regardless of the time zone in which the instance was created.

As we discussed this at the last board meeting, the right thing is to use
UTC time only both for #hash and #=, so this is correct.

>
>> 3. Loss of resolution and strict monotonicity. In Trunk [DateAndTime now <
>> DateAndTime now] is always true. This is possible because DateAndTime has
>> nanosecond resolution, but the source clock only has microsecond
>> resolution, and our machines are so slow that it takes more than a
>> nanosecond to execute the primitive. All these allow us to use the
>> sub-microsecond part of the timestamp to create intermediate strictly
>> monotonic values.
>>
>
> To be honest, I do not fully recall the rationale for this. One
> important case is that of the system clock being set back due to
> ntp synchronization. Fiddling with the nanoseconds number to make
> it appear as if the OS had reported an increase in time (when in
> fact it did not) seems questionable to me, but I know that it was
> done for some good reason. I do remember the discussions, so I
> should look back through the squeak-dev archives and find out the
> reason this was done.

It comes handy in many situations where uniqueness is a must. E.g.
creating file names, storing them in a database. I don't know why it was
added originally.

The loss of the nanosecond part has another side effect. utcMicroseconds
will be a Fraction when the resolution of the parsed input is too high.
This is somewhat compatible, but it makes things slower. E.g.:

'2002-05-16T17:20:45.000000009+01:01' asDateAndTime utcMicroseconds
"==> (1021565985000000009/1000)"

I found two other issues:

DateAndTime class >> #date:time: ignores the nanoseconds from the time
argument. This is present in the Trunk as well, but no test fails, because
in Trunk TimeStamps have their nanoseconds part cleared (in a very
inefficient way).

The other issue is that DateAndTime >> #getSeconds returns a fraction,
which contains the nanosecond part as well. This makes DateAndTime >>
#asTime return a Time instance which has its nanoseconds part doubled.
I simply changed #/ to #// in #getSeconds to work that around. This
also satisfied the expectations of all other senders of #getSeconds, but
then it turned out to be the same as #getWholeSeconds.
Since #getSeconds is a replacement for the original seconds variable (if
I'm not mistaken), I think it's safe to do this change and replace sends
of #getWholeSeconds to #getSeconds.

I found that I'm a developer on the project, so I have uploaded a new
version with the changes described above + some optimizations.

About some #FIXMEs:
#offset: is plain wrong according to its comment. It should simply create
a copy with the new offset and the same utcMicrosecond value. Just like
how #utcOffset: does. We should keep one or the other but not both.

Levente

>
>> Levente
>>
>
> Thanks very much for looking at this. I added you and Chris Muller as
> developers on the UTCDateAndTime repository on squeaksource.com in case
> you want to make any changes.
>
> Much appreciated,
>
> Dave
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

David T. Lewis
Hi Levente,

Thank you for the review and for your updates. Printing dates seems
to be almost 50% faster now with your updates.

Dave

On Sun, Mar 20, 2016 at 12:39:38AM +0100, Levente Uzonyi wrote:

> Hi Dave,
>
> I finally had some time to dig into the code and the tests.
>
> On Sun, 13 Mar 2016, David T. Lewis wrote:
>
> >On Sat, Mar 12, 2016 at 08:48:21PM +0100, Levente Uzonyi wrote:
> >>Hi Dave,
> >>
> >>I ran the tests and found that there are three types of failures.
> >>1. Some tests assume that the default offset is #localOffset. I don't
> >>think it's correct, but that's how is it in Trunk.
> >
> >There are 10 failing tests in my image. These are cases in which either
> >I was uncertain how to handle the issue, or where I thought that the
> >current behaviour in trunk may be wrong.
> >
> >>2. Hashes have changed. This implies that HashedCollections containing
> >>DateAndTime instances as key will have to be rehashed. In the base Trunk
> >>image there are appear to be no such HashedCollections, but not rehashing
> >>such collections will break images with custom packages in them after
> >>update.
> >
> >I am not expert in this area at all. But part of the update process
> >(in the MC postscripts) finds all instances of the old style DateAndTime,
> >and becomes them into the new ones. Would the HashedCollections contianing
> >DateAndTime instances as key still require rehash in that case?
>
> Yes, it is. When their hash changes, these objects will not be found in
> such collections.
>
> >
> >Related to DateAndTime>>hash, I would also like to ask for guidance as
> >to how DateAndTime>>= should be defined. Maybe this is an ANSI Smalltalk
> >question, but I was not sure if we should be thinking of DateAndTime as
> >a magnitude, such that two instances are equal if their utcMicroseconds
> >are the same? Or are two instances equal if they have the same magnitude
> >and also the same localOffsetSeconds?
> >
> >I implemented the first approach (two instances are #= if they represent
> >the same UTC time, regardless of timezone). There may be use cases
> >where the other approach makes sense. But my thinking was that when
> >comparing two instances of DateAndTime, they should not be considered
> >the same unless they both represent the same point in time, hence UTC
> >time regardless of the time zone in which the instance was created.
>
> As we discussed this at the last board meeting, the right thing is to use
> UTC time only both for #hash and #=, so this is correct.
>
> >
> >>3. Loss of resolution and strict monotonicity. In Trunk [DateAndTime now <
> >>DateAndTime now] is always true. This is possible because DateAndTime has
> >>nanosecond resolution, but the source clock only has microsecond
> >>resolution, and our machines are so slow that it takes more than a
> >>nanosecond to execute the primitive. All these allow us to use the
> >>sub-microsecond part of the timestamp to create intermediate strictly
> >>monotonic values.
> >>
> >
> >To be honest, I do not fully recall the rationale for this. One
> >important case is that of the system clock being set back due to
> >ntp synchronization. Fiddling with the nanoseconds number to make
> >it appear as if the OS had reported an increase in time (when in
> >fact it did not) seems questionable to me, but I know that it was
> >done for some good reason. I do remember the discussions, so I
> >should look back through the squeak-dev archives and find out the
> >reason this was done.
>
> It comes handy in many situations where uniqueness is a must. E.g.
> creating file names, storing them in a database. I don't know why it was
> added originally.
>
> The loss of the nanosecond part has another side effect. utcMicroseconds
> will be a Fraction when the resolution of the parsed input is too high.
> This is somewhat compatible, but it makes things slower. E.g.:
>
> '2002-05-16T17:20:45.000000009+01:01' asDateAndTime utcMicroseconds
> "==> (1021565985000000009/1000)"
>
> I found two other issues:
>
> DateAndTime class >> #date:time: ignores the nanoseconds from the time
> argument. This is present in the Trunk as well, but no test fails, because
> in Trunk TimeStamps have their nanoseconds part cleared (in a very
> inefficient way).
>
> The other issue is that DateAndTime >> #getSeconds returns a fraction,
> which contains the nanosecond part as well. This makes DateAndTime >>
> #asTime return a Time instance which has its nanoseconds part doubled.
> I simply changed #/ to #// in #getSeconds to work that around. This
> also satisfied the expectations of all other senders of #getSeconds, but
> then it turned out to be the same as #getWholeSeconds.
> Since #getSeconds is a replacement for the original seconds variable (if
> I'm not mistaken), I think it's safe to do this change and replace sends
> of #getWholeSeconds to #getSeconds.
>
> I found that I'm a developer on the project, so I have uploaded a new
> version with the changes described above + some optimizations.
>
> About some #FIXMEs:
> #offset: is plain wrong according to its comment. It should simply create
> a copy with the new offset and the same utcMicrosecond value. Just like
> how #utcOffset: does. We should keep one or the other but not both.
>
> Levente
>
> >
> >>Levente
> >>
> >
> >Thanks very much for looking at this. I added you and Chris Muller as
> >developers on the UTCDateAndTime repository on squeaksource.com in case
> >you want to make any changes.
> >
> >Much appreciated,
> >
> >Dave
> >
> >
> >

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

David T. Lewis
In reply to this post by Levente Uzonyi
On Sun, Mar 20, 2016 at 12:39:38AM +0100, Levente Uzonyi wrote:
>
> The loss of the nanosecond part has another side effect. utcMicroseconds
> will be a Fraction when the resolution of the parsed input is too high.
> This is somewhat compatible, but it makes things slower. E.g.:
>
> '2002-05-16T17:20:45.000000009+01:01' asDateAndTime utcMicroseconds
> "==> (1021565985000000009/1000)"
>

This is an interesting problem.

The fractional representation of utcMicroseconds seems reasonable to me in
this case, at least now that you have fixed the bugs that I introduced :-)

It might however cause issues for Magma and similar applications. I would
be interested to hear from Chris Muller is that is a problem.

My expectation would be that microseconds should be the unit of measure
for time magnitude, but that there should be no limit on precision. Or
to say it another way, clocks may have ticks, but time itself should be
thought of as continous. Thus I expect utcMicroseconds to be a Number, but
not necessarily an Integer.

Microseconds is a reasonable unit of measure because it is the highest
level of precision available from the clocks on the Linux, OS X, and
Windows systems that we use, and because it implies clock accuracy that
is well beyond the real time measurement capabilities of those platforms.

As far as I know, the practical use of nanosecond precision in DateAndTime
would be in the case of adding incremental nanoseconds to the time value
in order to create the illusion that repeated calls to the system clock
will always result in monotonically increasing values. If so, then I
suspect that the same practical result could be achieved by artificially
incrementing the value in units of milliseconds rather than nanoseconds,
which would ensure unique integer values.

After all, it does not seem likely that any application involving a
database would be querying the system clock at a rate anywhere close to
a million per second. But I have not actually tried this, so I may be
misunderstanding the requirement.

Dave

 

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

David T. Lewis
On Sun, Mar 20, 2016 at 01:16:45PM -0400, David T. Lewis wrote:

> On Sun, Mar 20, 2016 at 12:39:38AM +0100, Levente Uzonyi wrote:
> >
> > The loss of the nanosecond part has another side effect. utcMicroseconds
> > will be a Fraction when the resolution of the parsed input is too high.
> > This is somewhat compatible, but it makes things slower. E.g.:
> >
> > '2002-05-16T17:20:45.000000009+01:01' asDateAndTime utcMicroseconds
> > "==> (1021565985000000009/1000)"
> >
>
> This is an interesting problem.
>
> The fractional representation of utcMicroseconds seems reasonable to me in
> this case, at least now that you have fixed the bugs that I introduced :-)
>
> It might however cause issues for Magma and similar applications. I would
> be interested to hear from Chris Muller is that is a problem.
>
> My expectation would be that microseconds should be the unit of measure
> for time magnitude, but that there should be no limit on precision. Or
> to say it another way, clocks may have ticks, but time itself should be
> thought of as continous. Thus I expect utcMicroseconds to be a Number, but
> not necessarily an Integer.
>
> Microseconds is a reasonable unit of measure because it is the highest
> level of precision available from the clocks on the Linux, OS X, and
> Windows systems that we use, and because it implies clock accuracy that
> is well beyond the real time measurement capabilities of those platforms.
>
> As far as I know, the practical use of nanosecond precision in DateAndTime
> would be in the case of adding incremental nanoseconds to the time value
> in order to create the illusion that repeated calls to the system clock
> will always result in monotonically increasing values. If so, then I
> suspect that the same practical result could be achieved by artificially
> incrementing the value in units of milliseconds rather than nanoseconds,
> which would ensure unique integer values.
>
> After all, it does not seem likely that any application involving a
> database would be querying the system clock at a rate anywhere close to
> a million per second. But I have not actually tried this, so I may be
> misunderstanding the requirement.
>

I updated the package in Chronology-Core-dtl.14 to provide implementations
of various strategies for providing monotonically increasing DateAndTime now
with a utcMicrosecond time base. I am not sure which is the best approach,
but this shows how any of them can be supported.

The update comment describes the options:

Name: Chronology-Core-dtl.14
Author: dtl
Time: 20 March 2016, 9:42:49.417923 pm
UUID: 813fbf89-d8e0-4242-a5e5-9e665c9b8618
Ancestors: Chronology-Core-ul.13

Implement a variety of policies for incrementing the clock on repeated calls to DateAndTime class>>now. It is not clear which policy may be best, so provide four variants for experimentation and review. Presumably three of the four options can be removed at a later date.

See Time class>>clockPolicy to select a policy. The current clock policy is set to #monotonicForceNanosecondIncrement.

The four policies are:

#acceptPlatformTime - Accept the time value provided by the system platform, even in this allows time to appear to move backwards when the clock is adjusted. Simple, fast, and no hidden side effects.

#monotonicAllowDuplicates - Accept the time value provided by the system platform unless it is less that the last clock tick. This protects for system clock adjustments. Side effects unlikely. Does not ensure unique values on repeated calls.

#monotonicForceMicrosecondIncrement - If the time value is less than or equal to the value from the last call, force increment by one microsecond. This ensures integral values of UTC time, but increments the time value by a full microsecond.

#monotonicForceNanosecondIncrement - If the time value is less than or equal to the value from the last call, force increment by one nanosecond. This it functionally compatible with previous Squeak practice, but is computationally expensive and results in time values represented as fractions, which might be a problem for some database applications.


 

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

Levente Uzonyi
Hi Dave,

On Sun, 20 Mar 2016, David T. Lewis wrote:

>
> I updated the package in Chronology-Core-dtl.14 to provide implementations
> of various strategies for providing monotonically increasing DateAndTime now
> with a utcMicrosecond time base. I am not sure which is the best approach,
> but this shows how any of them can be supported.
>
> The update comment describes the options:
>
> Name: Chronology-Core-dtl.14
> Author: dtl
> Time: 20 March 2016, 9:42:49.417923 pm
> UUID: 813fbf89-d8e0-4242-a5e5-9e665c9b8618
> Ancestors: Chronology-Core-ul.13
>
> Implement a variety of policies for incrementing the clock on repeated calls to DateAndTime class>>now. It is not clear which policy may be best, so provide four variants for experimentation and review. Presumably three of the four options can be removed at a later date.
>
> See Time class>>clockPolicy to select a policy. The current clock policy is set to #monotonicForceNanosecondIncrement.
>
> The four policies are:
>
> #acceptPlatformTime - Accept the time value provided by the system platform, even in this allows time to appear to move backwards when the clock is adjusted. Simple, fast, and no hidden side effects.
>
> #monotonicAllowDuplicates - Accept the time value provided by the system platform unless it is less that the last clock tick. This protects for system clock adjustments. Side effects unlikely. Does not ensure unique values on repeated calls.
>
> #monotonicForceMicrosecondIncrement - If the time value is less than or equal to the value from the last call, force increment by one microsecond. This ensures integral values of UTC time, but increments the time value by a full microsecond.
>
> #monotonicForceNanosecondIncrement - If the time value is less than or equal to the value from the last call, force increment by one nanosecond. This it functionally compatible with previous Squeak practice, but is computationally expensive and results in time values represented as fractions, which might be a problem for some database applications.

I think this covers most possible use cases. What I was originally
thinking about was the addition of another field to hold the nanosecond
part, since that would have only been set by the image itself, so the
primitive could still have been used to initialize the rest. That we we
could have avoided the use of fractions at the cost of having another
field per object even when there's no need for high accuracy.

One more thing that I noticed is that this primitive still uses the "quick
but inaccurate" way to get the current time, which only updates every two
milliseconds (I think once per heartbeat). Eliot has changed primitive 240
a while ago to always provide the accurate time:

| start end |
start := DateAndTime now.
[ (end := DateAndTime now) = start ] whileTrue.
{ start. end. end - start }
  {2016-03-23T16:53:33.637835+01:00 . 2016-03-23T16:53:33.639844+01:00 .
0:00:00:00.002009}

| start end |
start := Time utcMicrosecondClock.
[ (end := Time utcMicrosecondClock) = start ] whileTrue.
{ start. end. end - start }
  #(3636201206461149 3636201206461150 1)

Levente

>
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

Bert Freudenberg
IMHO monotonicForceMicrosecondIncrement is good rough and should be the default (or only option).

And we could easily use Elliot's utcMicrosecondClock (as my latest trunk code does).

- Bert -


> On 23 Mar 2016, at 17:01, Levente Uzonyi <[hidden email]> wrote:
>
> Hi Dave,
>
>> On Sun, 20 Mar 2016, David T. Lewis wrote:
>>
>>
>> I updated the package in Chronology-Core-dtl.14 to provide implementations
>> of various strategies for providing monotonically increasing DateAndTime now
>> with a utcMicrosecond time base. I am not sure which is the best approach,
>> but this shows how any of them can be supported.
>>
>> The update comment describes the options:
>>
>> Name: Chronology-Core-dtl.14
>> Author: dtl
>> Time: 20 March 2016, 9:42:49.417923 pm
>> UUID: 813fbf89-d8e0-4242-a5e5-9e665c9b8618
>> Ancestors: Chronology-Core-ul.13
>>
>> Implement a variety of policies for incrementing the clock on repeated calls to DateAndTime class>>now. It is not clear which policy may be best, so provide four variants for experimentation and review. Presumably three of the four options can be removed at a later date.
>>
>> See Time class>>clockPolicy to select a policy. The current clock policy is set to #monotonicForceNanosecondIncrement.
>>
>> The four policies are:
>>
>> #acceptPlatformTime - Accept the time value provided by the system platform, even in this allows time to appear to move backwards when the clock is adjusted. Simple, fast, and no hidden side effects.
>>
>> #monotonicAllowDuplicates - Accept the time value provided by the system platform unless it is less that the last clock tick. This protects for system clock adjustments. Side effects unlikely. Does not ensure unique values on repeated calls.
>>
>> #monotonicForceMicrosecondIncrement - If the time value is less than or equal to the value from the last call, force increment by one microsecond. This ensures integral values of UTC time, but increments the time value by a full microsecond.
>>
>> #monotonicForceNanosecondIncrement - If the time value is less than or equal to the value from the last call, force increment by one nanosecond. This it functionally compatible with previous Squeak practice, but is computationally expensive and results in time values represented as fractions, which might be a problem for some database applications.
>
> I think this covers most possible use cases. What I was originally thinking about was the addition of another field to hold the nanosecond part, since that would have only been set by the image itself, so the primitive could still have been used to initialize the rest. That we we could have avoided the use of fractions at the cost of having another field per object even when there's no need for high accuracy.
>
> One more thing that I noticed is that this primitive still uses the "quick but inaccurate" way to get the current time, which only updates every two milliseconds (I think once per heartbeat). Eliot has changed primitive 240 a while ago to always provide the accurate time:
>
> | start end |
> start := DateAndTime now.
> [ (end := DateAndTime now) = start ] whileTrue.
> { start. end. end - start }
> {2016-03-23T16:53:33.637835+01:00 . 2016-03-23T16:53:33.639844+01:00 . 0:00:00:00.002009}
>
> | start end |
> start := Time utcMicrosecondClock.
> [ (end := Time utcMicrosecondClock) = start ] whileTrue.
> { start. end. end - start }
> #(3636201206461149 3636201206461150 1)
>
> Levente
>
>



smime.p7s (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

David T. Lewis
On Wed, Mar 23, 2016 at 05:22:49PM +0100, Bert Freudenberg wrote:
> IMHO monotonicForceMicrosecondIncrement is good rough and should be the default (or only option).
>
> And we could easily use Elliot's utcMicrosecondClock (as my latest trunk code does).
>
> - Bert -
>

I made a number of updates yesterday (but did not mention it on the list). I
think that everything is now up to date and reasonably compatible with trunk,
including your utcMicrosecondClock changes. The automaticTimezone preference is
working now too. I also added HashedCollection rehashAll to the package postscript
(thanks Levente).

I set the clock policy to monotonicAllowDuplicates, which is my own personal
preference. But I expect to be outvoted on that, and I would be quite happy
with your preferred choice of monotonicForceMicrosecondIncrement.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

Bert Freudenberg
> On 23.03.2016, at 23:48, David T. Lewis <[hidden email]> wrote:

>
>> On Wed, Mar 23, 2016 at 05:22:49PM +0100, Bert Freudenberg wrote:
>> IMHO monotonicForceMicrosecondIncrement is good rough and should be the default (or only option).
>>
>> And we could easily use Elliot's utcMicrosecondClock (as my latest trunk code does).
>>
>> - Bert -
>>
>
> I made a number of updates yesterday (but did not mention it on the list). I
> think that everything is now up to date and reasonably compatible with trunk,
> including your utcMicrosecondClock changes. The automaticTimezone preference is
> working now too. I also added HashedCollection rehashAll to the package postscript
> (thanks Levente).
>
> I set the clock policy to monotonicAllowDuplicates, which is my own personal
> preference. But I expect to be outvoted on that, and I would be quite happy
> with your preferred choice of monotonicForceMicrosecondIncrement.
Oh. Well I don't need #now to be unique, dupes would be fine. But someone needed it. Speak up! ;)

- Bert -




smime.p7s (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

David T. Lewis
In reply to this post by Levente Uzonyi
Hi Levente,

On Wed, Mar 23, 2016 at 05:01:35PM +0100, Levente Uzonyi wrote:

>
> On Sun, 20 Mar 2016, David T. Lewis wrote:
> >
> >The four policies are:
> >
> >#acceptPlatformTime - Accept the time value provided by the system
> >platform, even in this allows time to appear to move backwards when the
> >clock is adjusted. Simple, fast, and no hidden side effects.
> >
> >#monotonicAllowDuplicates - Accept the time value provided by the system
> >platform unless it is less that the last clock tick. This protects for
> >system clock adjustments. Side effects unlikely. Does not ensure unique
> >values on repeated calls.
> >
> >#monotonicForceMicrosecondIncrement - If the time value is less than or
> >equal to the value from the last call, force increment by one microsecond.
> >This ensures integral values of UTC time, but increments the time value by
> >a full microsecond.
> >
> >#monotonicForceNanosecondIncrement - If the time value is less than or
> >equal to the value from the last call, force increment by one nanosecond.
> >This it functionally compatible with previous Squeak practice, but is
> >computationally expensive and results in time values represented as
> >fractions, which might be a problem for some database applications.
>
> I think this covers most possible use cases. What I was originally
> thinking about was the addition of another field to hold the nanosecond
> part, since that would have only been set by the image itself, so the
> primitive could still have been used to initialize the rest. That we we
> could have avoided the use of fractions at the cost of having another
> field per object even when there's no need for high accuracy.

I'm actually quite happy with the idea of the value being a Number that
is an Integer for all reasonable values that could ever be supplied by
the VM, but that might be a Fraction if we want to represent time values
at some higher precision. Mainly I like it because it seems good conceptually
to think of time as continuous, and not as discreet ticks of a clock.

It also works well computationally, because the normal case uses integer
values (immediates in 64-bit Spur), and because Fractions involve two
integer values, which presumably would be no worse than a design that
added another field. But given that none of our computers or VMs can
report time to a precision greater than microseconds, I think that we
can safely ignore the issue for another 20 years or so ;-)

>
> One more thing that I noticed is that this primitive still uses the "quick
> but inaccurate" way to get the current time, which only updates every two
> milliseconds (I think once per heartbeat). Eliot has changed primitive 240
> a while ago to always provide the accurate time:
>
> | start end |
> start := DateAndTime now.
> [ (end := DateAndTime now) = start ] whileTrue.
> { start. end. end - start }
>  {2016-03-23T16:53:33.637835+01:00 . 2016-03-23T16:53:33.639844+01:00 .
> 0:00:00:00.002009}
>
> | start end |
> start := Time utcMicrosecondClock.
> [ (end := Time utcMicrosecondClock) = start ] whileTrue.
> { start. end. end - start }
>  #(3636201206461149 3636201206461150 1)
>

There is a good implementation of the primitive in the trunk SVN branch
that needs to be added to the oscog branch. I think that will take care
of the problem. I promised Eliot I would move the code over, but it might
be a couple of weeks before I can get to it. We need to add this to the
platform code to provide a true atomic primitive (from sq.h):

  sqInt ioUtcWithOffset(sqLong*, int*);   /* primitiveUtcWithOffset */

And then the primitive (slang) can be updated to match. I am afraid that
I am responsible for some of the confusion, because when this was added
to Cog a number of years ago, I did not want to touch the platform code,
and I implemented a rather kludgy workaround just to get it working. Sorry.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

Eliot Miranda-2
Hi Dave,

On Wed, Mar 23, 2016 at 4:12 PM, David T. Lewis <[hidden email]> wrote:
Hi Levente,

On Wed, Mar 23, 2016 at 05:01:35PM +0100, Levente Uzonyi wrote:
>
> On Sun, 20 Mar 2016, David T. Lewis wrote:
> >
> >The four policies are:
> >
> >#acceptPlatformTime - Accept the time value provided by the system
> >platform, even in this allows time to appear to move backwards when the
> >clock is adjusted. Simple, fast, and no hidden side effects.
> >
> >#monotonicAllowDuplicates - Accept the time value provided by the system
> >platform unless it is less that the last clock tick. This protects for
> >system clock adjustments. Side effects unlikely. Does not ensure unique
> >values on repeated calls.
> >
> >#monotonicForceMicrosecondIncrement - If the time value is less than or
> >equal to the value from the last call, force increment by one microsecond.
> >This ensures integral values of UTC time, but increments the time value by
> >a full microsecond.
> >
> >#monotonicForceNanosecondIncrement - If the time value is less than or
> >equal to the value from the last call, force increment by one nanosecond.
> >This it functionally compatible with previous Squeak practice, but is
> >computationally expensive and results in time values represented as
> >fractions, which might be a problem for some database applications.
>
> I think this covers most possible use cases. What I was originally
> thinking about was the addition of another field to hold the nanosecond
> part, since that would have only been set by the image itself, so the
> primitive could still have been used to initialize the rest. That we we
> could have avoided the use of fractions at the cost of having another
> field per object even when there's no need for high accuracy.

I'm actually quite happy with the idea of the value being a Number that
is an Integer for all reasonable values that could ever be supplied by
the VM, but that might be a Fraction if we want to represent time values
at some higher precision. Mainly I like it because it seems good conceptually
to think of time as continuous, and not as discreet ticks of a clock.

It also works well computationally, because the normal case uses integer
values (immediates in 64-bit Spur), and because Fractions involve two
integer values, which presumably would be no worse than a design that
added another field. But given that none of our computers or VMs can
report time to a precision greater than microseconds, I think that we
can safely ignore the issue for another 20 years or so ;-)

>
> One more thing that I noticed is that this primitive still uses the "quick
> but inaccurate" way to get the current time, which only updates every two
> milliseconds (I think once per heartbeat). Eliot has changed primitive 240
> a while ago to always provide the accurate time:
>
> | start end |
> start := DateAndTime now.
> [ (end := DateAndTime now) = start ] whileTrue.
> { start. end. end - start }
>  {2016-03-23T16:53:33.637835+01:00 . 2016-03-23T16:53:33.639844+01:00 .
> 0:00:00:00.002009}
>
> | start end |
> start := Time utcMicrosecondClock.
> [ (end := Time utcMicrosecondClock) = start ] whileTrue.
> { start. end. end - start }
>  #(3636201206461149 3636201206461150 1)
>

There is a good implementation of the primitive in the trunk SVN branch
that needs to be added to the oscog branch. I think that will take care
of the problem. I promised Eliot I would move the code over, but it might
be a couple of weeks before I can get to it. We need to add this to the
platform code to provide a true atomic primitive (from sq.h):

  sqInt ioUtcWithOffset(sqLong*, int*);   /* primitiveUtcWithOffset */

And then the primitive (slang) can be updated to match. I am afraid that
I am responsible for some of the confusion, because when this was added
to Cog a number of years ago, I did not want to touch the platform code,
and I implemented a rather kludgy workaround just to get it working. Sorry.

Turns out this isn't needed for Cog.  I have ioLocalSecondsOffset which answers a value determined at start-up and only changed via ioUpdateVMTimezone, which itself is controlled by primitiveUpdateTimezone, #243.  So ioUTCMicroseconds is all that's needed to get at the clock and timezone atomically.

If I had had half a brain when I implemented the 64-bit microsecond clock I would have done it as a primitive to answer UTC and a primitive to answer timezone offset, but instead I chose to hide timezone in local microseconds.  Live and learn.  But the existence of primitiveUpdateTimezone, #243, means I could make it answer the timezone offset if invoked with zero arguments.  And that way we'll have every possible combination of 64-bit clock access we could wish for :-) (not)


VMMaker.oscog-eem.1739
Author: eem
Time: 23 March 2016, 12:41:48.73939 pm
UUID: e9e980a3-1fc0-4425-acb8-f1f02f1977b0
Ancestors: VMMaker.oscog-eem.1738

Fix a slip in my correction of primitiveUtcWithOffset (wrong arg count check), and use ioLocalSecondsOffset to get the timezone, hence avoiding the need for the atomic accessor for both clock and timezone offset (the offset can only be changed from the system, and hence won't change while this primitive is executing).

Add a Smalltalk epoch version primitiveUtcAndTimezoneOffset and give it primitive #244.
 
Dave
 
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

David T. Lewis
On Wed, Mar 23, 2016 at 04:22:21PM -0700, Eliot Miranda wrote:
>
> Turns out this isn't needed for Cog.  I have ioLocalSecondsOffset which
> answers a value determined at start-up and only changed
> via ioUpdateVMTimezone, which itself is controlled by
> primitiveUpdateTimezone, #243.  So ioUTCMicroseconds is all that's needed
> to get at the clock and timezone atomically.

If it is updated at start-up, then it's wrong. Think of daylight savings
time transitions.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

Eliot Miranda-2


On Wed, Mar 23, 2016 at 4:51 PM, David T. Lewis <[hidden email]> wrote:
On Wed, Mar 23, 2016 at 04:22:21PM -0700, Eliot Miranda wrote:
>
> Turns out this isn't needed for Cog.  I have ioLocalSecondsOffset which
> answers a value determined at start-up and only changed
> via ioUpdateVMTimezone, which itself is controlled by
> primitiveUpdateTimezone, #243.  So ioUTCMicroseconds is all that's needed
> to get at the clock and timezone atomically.

If it is updated at start-up, then it's wrong. Think of daylight savings
time transitions.

So update it automatically once a second or some such?

_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

David T. Lewis
On Wed, Mar 23, 2016 at 05:50:19PM -0700, Eliot Miranda wrote:

> On Wed, Mar 23, 2016 at 4:51 PM, David T. Lewis <[hidden email]> wrote:
>
> > On Wed, Mar 23, 2016 at 04:22:21PM -0700, Eliot Miranda wrote:
> > >
> > > Turns out this isn't needed for Cog.  I have ioLocalSecondsOffset which
> > > answers a value determined at start-up and only changed
> > > via ioUpdateVMTimezone, which itself is controlled by
> > > primitiveUpdateTimezone, #243.  So ioUTCMicroseconds is all that's needed
> > > to get at the clock and timezone atomically.
> >
> > If it is updated at start-up, then it's wrong. Think of daylight savings
> > time transitions.
> >
>
> So update it automatically once a second or some such?
>

Are you joking, or is that a serious question?

Confused,
Dave


/* implementation of ioUtcWithOffset(), defined in config.h to
/* override default definition in src/vm/interp.h
 */
sqInt sqUnixUtcWithOffset(sqLong *microSeconds, int *offset)
{
  struct timeval timeval;
  if (gettimeofday(&timeval, NULL) == -1) return -1;
  time_t seconds= timeval.tv_sec;
  suseconds_t usec= timeval.tv_usec;
  *microSeconds= seconds * 1000000 + usec;
#if defined(HAVE_TM_GMTOFF)
  *offset= localtime(&seconds)->tm_gmtoff;
#else
  {
    struct tm *local= localtime(&seconds);
    struct tm *gmt= gmtime(&seconds);
    int d= local->tm_yday - gmt->tm_yday;
    int h= ((d < -1 ? 24 : 1 < d ? -24 : d * 24) + local->tm_hour - gmt->tm_hour);
    int m= h * 60 + local->tm_min - gmt->tm_min;
    *offset= m * 60;
  }
#endif
  return 0;
}



Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

Ben Coman
In reply to this post by David T. Lewis
On Thu, Mar 24, 2016 at 7:12 AM, David T. Lewis <[hidden email]> wrote:

> Hi Levente,
>
> On Wed, Mar 23, 2016 at 05:01:35PM +0100, Levente Uzonyi wrote:
>>
>> On Sun, 20 Mar 2016, David T. Lewis wrote:
>> >
>> >The four policies are:
>> >
>> >#acceptPlatformTime - Accept the time value provided by the system
>> >platform, even in this allows time to appear to move backwards when the
>> >clock is adjusted. Simple, fast, and no hidden side effects.
>> >
>> >#monotonicAllowDuplicates - Accept the time value provided by the system
>> >platform unless it is less that the last clock tick. This protects for
>> >system clock adjustments. Side effects unlikely. Does not ensure unique
>> >values on repeated calls.
>> >
>> >#monotonicForceMicrosecondIncrement - If the time value is less than or
>> >equal to the value from the last call, force increment by one microsecond.
>> >This ensures integral values of UTC time, but increments the time value by
>> >a full microsecond.
>> >
>> >#monotonicForceNanosecondIncrement - If the time value is less than or
>> >equal to the value from the last call, force increment by one nanosecond.
>> >This it functionally compatible with previous Squeak practice, but is
>> >computationally expensive and results in time values represented as
>> >fractions, which might be a problem for some database applications.
>>
>> I think this covers most possible use cases. What I was originally
>> thinking about was the addition of another field to hold the nanosecond
>> part, since that would have only been set by the image itself, so the
>> primitive could still have been used to initialize the rest. That we we
>> could have avoided the use of fractions at the cost of having another
>> field per object even when there's no need for high accuracy.
>
> I'm actually quite happy with the idea of the value being a Number that
> is an Integer for all reasonable values that could ever be supplied by
> the VM, but that might be a Fraction if we want to represent time values
> at some higher precision. Mainly I like it because it seems good conceptually
> to think of time as continuous, and not as discreet ticks of a clock.
>
> It also works well computationally, because the normal case uses integer
> values (immediates in 64-bit Spur), and because Fractions involve two
> integer values, which presumably would be no worse than a design that
> added another field. But given that none of our computers or VMs can
> report time to a precision greater than microseconds, I think that we
> can safely ignore the issue for another 20 years or so ;-)

Its not so far away. Three contemporary examples...

* Systems that synchronise time from GPS do so "typically well within
40 nanoseconds" [1]

* The distributed clock of EtherCAT networks synchronises nodes to
better than 100ns. See slide 23 [2] and figure 7 [3].

* IEEE 1588 provides synchronised network time "in the range of
+/-200ns between some nodes and even +/-50ns and less between the
better ones." [4]

This matters for applications like financial trading, motion control
and sequence-of-event logging on industrial plants.  Maybe not
Smalltalk's bread and butter, but the last two are areas I'd like to
use it at a co-ordinator level.

cheers -ben

[1] http://gpsinformation.net/main/gpstime.htm
[2] https://ece.uwaterloo.ca/~sfischme/rate/Joey-Stubbs-Ethercat.pdf
[3] http://www.iestcfa.org/bestpaper/wfcs10/wfcs10_bpa.pdf
[4] https://home.zhaw.ch/~wei/IEEE1588/embedded_World_05_Contribution_final.pdf

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime updated for Squeak trunk Chronology and Spur (was: UTCDateAndTime updated for Squeak 4.6/5.0)

David T. Lewis
On Fri, Mar 25, 2016 at 12:48:17AM +0800, Ben Coman wrote:

> On Thu, Mar 24, 2016 at 7:12 AM, David T. Lewis <[hidden email]> wrote:
> >
> > I'm actually quite happy with the idea of the value being a Number that
> > is an Integer for all reasonable values that could ever be supplied by
> > the VM, but that might be a Fraction if we want to represent time values
> > at some higher precision. Mainly I like it because it seems good conceptually
> > to think of time as continuous, and not as discreet ticks of a clock.
> >
> > It also works well computationally, because the normal case uses integer
> > values (immediates in 64-bit Spur), and because Fractions involve two
> > integer values, which presumably would be no worse than a design that
> > added another field. But given that none of our computers or VMs can
> > report time to a precision greater than microseconds, I think that we
> > can safely ignore the issue for another 20 years or so ;-)
>
> Its not so far away. Three contemporary examples...
>
> * Systems that synchronise time from GPS do so "typically well within
> 40 nanoseconds" [1]
>
> * The distributed clock of EtherCAT networks synchronises nodes to
> better than 100ns. See slide 23 [2] and figure 7 [3].
>
> * IEEE 1588 provides synchronised network time "in the range of
> +/-200ns between some nodes and even +/-50ns and less between the
> better ones." [4]
>
> This matters for applications like financial trading, motion control
> and sequence-of-event logging on industrial plants.  Maybe not
> Smalltalk's bread and butter, but the last two are areas I'd like to
> use it at a co-ordinator level.
>
> cheers -ben
>
> [1] http://gpsinformation.net/main/gpstime.htm
> [2] https://ece.uwaterloo.ca/~sfischme/rate/Joey-Stubbs-Ethercat.pdf
> [3] http://www.iestcfa.org/bestpaper/wfcs10/wfcs10_bpa.pdf
> [4] https://home.zhaw.ch/~wei/IEEE1588/embedded_World_05_Contribution_final.pdf

Indeed you are quite right. And thank you for those links.

Our current Squeak platforms are certainly not real time systems, but
our representation of DateAndTime should not be limited by that.

I guess that all I can add at this point is my basic premise that a
simple representation of a time scale is better than a complicated
one. For now, the idea of microseconds as the unit of measure seems
to work quite well. Maybe in the future some different representation
will be better.

At higher levels of precision, I suspect that Fraction (or ScaledDecimal,
which is more readable) will be a very efficient representation. And since
DateAndTime instances are likely to be important for databases, forcing
all of the values to be Fraction (i.e. two integers) might be a good
policy. But for now, microseconds since the epoch works well, yielding
Integer representations in all practical cases, and overflowing into
Fraction if nanosecond precision is needed.

Dave

1234