Smalltalk › Squeak › Squeak - Dev

efficient DateAndTime

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

14 messages Options

Chris Muller-3

efficient DateAndTime

I have a bunch of two-element Array log entries, where the first element is the timestamp, the second is the object being logged. But these entries may never be consumed, so I wish to minimize their cost. So, instead of storing the timestamps as DateAndTimes, I wanted to keep them as SmallIntegers. I thought I could use Time utcMicroseconds, now a SmallInteger in 64-bit Squeak.

However, when I went to instantiate the DateAndTime lazily from that number it's said it's year 2089...

DateAndTime utcMicroseconds: Time utcMicrosecondClock offset: 0 " 2089-04-29T00:49:26.3326+00:00"

I realized this is because a different epoch being used (1901 vs. 1971) between the two. This seems inconsistent and confusing, but that's less important to me at the moment than the efficiency I'm looking for. Is there another way?

- Chris

David T. Lewis

Re: efficient DateAndTime

On Tue, Apr 28, 2020 at 07:53:05PM -0500, Chris Muller wrote:

> I have a bunch of two-element Array log entries, where the first element is
> the timestamp, the second is the object being logged. But these entries
> may never be consumed, so I wish to minimize their cost. So, instead of
> storing the timestamps as DateAndTimes, I wanted to keep them as
> SmallIntegers. I thought I could use Time utcMicroseconds, now a
> SmallInteger in 64-bit Squeak.
>
> However, when I went to instantiate the DateAndTime lazily from that number
> it's said it's year 2089...
>
> DateAndTime utcMicroseconds: Time utcMicrosecondClock offset: 0
> " 2089-04-29T00:49:26.3326+00:00"
>
> I realized this is because a different epoch being used (1901 vs. 1971)
> between the two. This seems inconsistent and confusing, but that's less
> important to me at the moment than the efficiency I'm looking for. Is
> there another way?
>
> - Chris

Store the time in units of milliseconds or microseconds relative to the
Posix epoch. This is an integer value (small integer in 64 bit Spur), and
it a common time basis in many operating system and database libraries,
so it is well understood and well defined.

At higher levels of precision (e.g. nanosecond), the value is not an
integer, but that is perfectly all right, just think of it as a Number
that is usually an integer in common use cases. If you need to store
it in the database as an integer, just round it off to the nearest
millisecond or microsecond (whichever you have chosen).

Microsecond precision is probably a good choice. This would be measured
as microseconds elapsed since the Posix epoch. That gives you small
integer values on Spur 64 for reasonable time values, and it is a higher
level of precision than you will probably ever need.

Assuming that you want to use a measurement scale of seconds relative to
Posix epoch, and assuming also that you want to measure this in units
of microseconds, then:

| now uSecRelativeToPosixEpoch newInstance |
now := DateAndTime now.
uSecRelativeToPosixEpoch := now utcMicroseconds.
newInstance := DateAndTime utcMicroseconds: uSecRelativeToPosixEpoch offset: 0.
now = newInstance "==> true"

Dave

Chris Muller-4

Re: efficient DateAndTime

Log entries are created often, I actually want to avoid creation of an interim DateAndTime in the first place. I'm not even concerned about storage yet, but that will also benefit later.

Wait a second. The Squeak epoch is only one year different than the posix epoch, right? So why am I getting 2089 instead of something within 1 year of 2020?

On Tue, Apr 28, 2020 at 9:07 PM David T. Lewis <[hidden email]> wrote:

On Tue, Apr 28, 2020 at 07:53:05PM -0500, Chris Muller wrote:
> I have a bunch of two-element Array log entries, where the first element is
> the timestamp, the second is the object being logged. But these entries
> may never be consumed, so I wish to minimize their cost. So, instead of
> storing the timestamps as DateAndTimes, I wanted to keep them as
> SmallIntegers. I thought I could use Time utcMicroseconds, now a
> SmallInteger in 64-bit Squeak.
>
> However, when I went to instantiate the DateAndTime lazily from that number
> it's said it's year 2089...
>
> DateAndTime utcMicroseconds: Time utcMicrosecondClock offset: 0
> " 2089-04-29T00:49:26.3326+00:00"
>
> I realized this is because a different epoch being used (1901 vs. 1971)
> between the two. This seems inconsistent and confusing, but that's less
> important to me at the moment than the efficiency I'm looking for. Is
> there another way?
>
> - Chris

Store the time in units of milliseconds or microseconds relative to the
Posix epoch. This is an integer value (small integer in 64 bit Spur), and
it a common time basis in many operating system and database libraries,
so it is well understood and well defined.

At higher levels of precision (e.g. nanosecond), the value is not an
integer, but that is perfectly all right, just think of it as a Number
that is usually an integer in common use cases. If you need to store
it in the database as an integer, just round it off to the nearest
millisecond or microsecond (whichever you have chosen).

Microsecond precision is probably a good choice. This would be measured
as microseconds elapsed since the Posix epoch. That gives you small
integer values on Spur 64 for reasonable time values, and it is a higher
level of precision than you will probably ever need.

Assuming that you want to use a measurement scale of seconds relative to
Posix epoch, and assuming also that you want to measure this in units
of microseconds, then:

| now uSecRelativeToPosixEpoch newInstance |
now := DateAndTime now.
uSecRelativeToPosixEpoch := now utcMicroseconds.
newInstance := DateAndTime utcMicroseconds: uSecRelativeToPosixEpoch offset: 0.
now = newInstance "==> true"

Dave

timrowledge

Re: efficient DateAndTime

1901 -> 1970 = 69 years

> On 2020-04-28, at 8:01 PM, Chris Muller <[hidden email]> wrote:
>
> Log entries are created often, I actually want to avoid creation of an interim DateAndTime in the first place. I'm not even concerned about storage yet, but that will also benefit later.
>
> Wait a second. The Squeak epoch is only one year different than the posix epoch, right? So why am I getting 2089 instead of something within 1 year of 2020?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
..... REALITY.SYS Corrupted - Unable to recover Universe

Eliot Miranda-2

Re: efficient DateAndTime

In reply to this post by Chris Muller-3

Hi Chris, Hi Dave,

> On Apr 28, 2020, at 5:53 PM, Chris Muller <[hidden email]> wrote:
>
>
> I have a bunch of two-element Array log entries, where the first element is the timestamp, the second is the object being logged. But these entries may never be consumed, so I wish to minimize their cost. So, instead of storing the timestamps as DateAndTimes, I wanted to keep them as SmallIntegers. I thought I could use Time utcMicroseconds, now a SmallInteger in 64-bit Squeak.
>
> However, when I went to instantiate the DateAndTime lazily from that number it's said it's year 2089...
>
> DateAndTime utcMicroseconds: Time utcMicrosecondClock offset: 0 " 2089-04-29T00:49:26.3326+00:00"

I gave yo say that this is a bug. Dave, I know you have affection for the posix epoch but the system uses the Smalltalk epoch, the start of the 20th c, and the utcMucroseconds primitive answers microseconds from that epoch, not the posix epoch. And it’s good for some 58,000 years before it overflows 61-bit SmallIntegers (there are three tag bits) so there’s essentially nothing to be gained from using the posix epoch.

Now the posix epoch is important; and we should support it. But we should clearly indicate it in spud, eg by having the word posix in the selector somewhere.

So the method should be renamed either utcPosixMicroseconds:offset: or posixUTCMicroseconds:offset:, and utcMicroseconds:offset: should go what Chris expects. The principle of least astonishment says it must be so.

>
> I realized this is because a different epoch being used (1901 vs. 1971) between the two. This seems inconsistent and confusing, but that's less important to me at the moment than the efficiency I'm looking for. Is there another way?
>
> - Chris
>

David T. Lewis

Re: efficient DateAndTime

In reply to this post by Chris Muller-4

Hi Chris,

On Tue, Apr 28, 2020 at 10:01:23PM -0500, Chris Muller wrote:
> Log entries are created often, I actually want to avoid creation of an
> interim DateAndTime in the first place. I'm not even concerned about
> storage yet, but that will also benefit later.
>

You can bypass the DateAndTime creation like this:

(Time primPosixMicrosecondClockWithOffset: (Array new: 2)) first

The primitive is designed to accept either an uninitialized DateAndTime
with two instance variables or an Array of size 2. If you pass it a
two element array, it will store the posix microseconds in the first
slot, and the offset seconds in the second slot.

For database use, keep in mind that the primitive will not guarantee
unique values, and it may even give you time going "backwards" if
your system clock is being adjusted. So if you want fast, just do
what I suggest above, but if you need unique monotonically increasing
values, you will still want to use the logic in #posixMicrosecondClockWithOffset:

Also keep in mind that the posix microsecond value will be a SmallInteger
on a 64-bit image, and it will be a LargePositiveInteger on 32-bits, so
my may need to accomodate that difference when serializing it to the database.

Dave

Ron Teitelbaum

Re: efficient DateAndTime

In reply to this post by Eliot Miranda-2

Can I just say now that I think the year 60,000 will be a bad year for programmers!

(Sorry couldn't resist. We're gonna party like it's Fifty Nine Nine Ninety Nine!)

Ron

On Wed, Apr 29, 2020 at 5:18 AM Eliot Miranda <[hidden email]> wrote:

Hi Chris, Hi Dave,

> On Apr 28, 2020, at 5:53 PM, Chris Muller <[hidden email]> wrote:
>
>
> I have a bunch of two-element Array log entries, where the first element is the timestamp, the second is the object being logged. But these entries may never be consumed, so I wish to minimize their cost. So, instead of storing the timestamps as DateAndTimes, I wanted to keep them as SmallIntegers. I thought I could use Time utcMicroseconds, now a SmallInteger in 64-bit Squeak.
>
> However, when I went to instantiate the DateAndTime lazily from that number it's said it's year 2089...
>
> DateAndTime utcMicroseconds: Time utcMicrosecondClock offset: 0 " 2089-04-29T00:49:26.3326+00:00"

I gave yo say that this is a bug. Dave, I know you have affection for the posix epoch but the system uses the Smalltalk epoch, the start of the 20th c, and the utcMucroseconds primitive answers microseconds from that epoch, not the posix epoch. And it’s good for some 58,000 years before it overflows 61-bit SmallIntegers (there are three tag bits) so there’s essentially nothing to be gained from using the posix epoch.

Now the posix epoch is important; and we should support it. But we should clearly indicate it in spud, eg by having the word posix in the selector somewhere.

So the method should be renamed either utcPosixMicroseconds:offset: or posixUTCMicroseconds:offset:, and utcMicroseconds:offset: should go what Chris expects. The principle of least astonishment says it must be so.

>
> I realized this is because a different epoch being used (1901 vs. 1971) between the two. This seems inconsistent and confusing, but that's less important to me at the moment than the efficiency I'm looking for. Is there another way?
>
> - Chris
>

David T. Lewis

Re: efficient DateAndTime

In reply to this post by Eliot Miranda-2

On Wed, Apr 29, 2020 at 02:18:17AM -0700, Eliot Miranda wrote:

> Hi Chris, Hi Dave,
>
> > On Apr 28, 2020, at 5:53 PM, Chris Muller <[hidden email]> wrote:
> >
> > ???
> > I have a bunch of two-element Array log entries, where the first element is the timestamp, the second is the object being logged. But these entries may never be consumed, so I wish to minimize their cost. So, instead of storing the timestamps as DateAndTimes, I wanted to keep them as SmallIntegers. I thought I could use Time utcMicroseconds, now a SmallInteger in 64-bit Squeak.
> >
> > However, when I went to instantiate the DateAndTime lazily from that number it's said it's year 2089...
> >
> > DateAndTime utcMicroseconds: Time utcMicrosecondClock offset: 0 " 2089-04-29T00:49:26.3326+00:00"
>
> I gave yo say that this is a bug. Dave, I know you have affection for the posix epoch but the system uses the Smalltalk epoch, the start of the 20th c, and the utcMucroseconds primitive answers microseconds from that epoch, not the posix epoch. And it???s good for some 58,000 years before it overflows 61-bit SmallIntegers (there are three tag bits) so there???s essentially nothing to be gained from using the posix epoch.
>
> Now the posix epoch is important; and we should support it. But we should clearly indicate it in spud, eg by having the word posix in the selector somewhere.
>
> So the method should be renamed either utcPosixMicroseconds:offset: or posixUTCMicroseconds:offset:, and utcMicroseconds:offset: should go what Chris expects. The principle of least astonishment says it must be so.
>

Currently we have this:

DateAndTime class>>utcMicroseconds: microsecondsSincePosixEpoch offset: secondsFromGMT
^super new
utcMicroseconds: microsecondsSincePosixEpoch
offset: secondsFromGMT

No objection from me if you want to change the method name, and
I think posixUTCMicroseconds:offset: would be a good choice if you
want to do that. The original rationale for the name of this constructor
is simply that it matches the instance variable names.

If there is still a method called utcMicroseconds:offset: then please
do *not* give it different semantics.

Dave

>
> >
> > I realized this is because a different epoch being used (1901 vs. 1971) between the two. This seems inconsistent and confusing, but that's less important to me at the moment than the efficiency I'm looking for. Is there another way?
> >
> > - Chris
> >
>

David T. Lewis

Re: efficient DateAndTime

In reply to this post by Ron Teitelbaum

It should be a great party, I'll put it on my calendar!

But the year 60,000 shouldn't be a problem for programmers. All we need
to do is get everybody to start using sensible software such as Squeak:

'60000-01-01T00:00:00-00:00' asDateAndTime ==> 60000-01-01T00:00:00+00:00

Of course, from what I know about programmers, it will probably take
another 60,000 years to convince them to start using sensible software.

;-)
Dave

On Wed, Apr 29, 2020 at 11:02:17AM -0400, Ron Teitelbaum wrote:

> Can I just say now that I think the year 60,000 will be a bad year for
> programmers!
>
> (Sorry couldn't resist. We're gonna party like it's Fifty Nine Nine
> Ninety Nine!)
>
> Ron
>
> On Wed, Apr 29, 2020 at 5:18 AM Eliot Miranda <[hidden email]>
> wrote:
>
> > Hi Chris, Hi Dave,
> >
> > > On Apr 28, 2020, at 5:53 PM, Chris Muller <[hidden email]> wrote:
> > >
> > > ???
> > > I have a bunch of two-element Array log entries, where the first element
> > is the timestamp, the second is the object being logged. But these entries
> > may never be consumed, so I wish to minimize their cost. So, instead of
> > storing the timestamps as DateAndTimes, I wanted to keep them as
> > SmallIntegers. I thought I could use Time utcMicroseconds, now a
> > SmallInteger in 64-bit Squeak.
> > >
> > > However, when I went to instantiate the DateAndTime lazily from that
> > number it's said it's year 2089...
> > >
> > > DateAndTime utcMicroseconds: Time utcMicrosecondClock offset: 0
> > " 2089-04-29T00:49:26.3326+00:00"
> >
> > I gave yo say that this is a bug. Dave, I know you have affection for the
> > posix epoch but the system uses the Smalltalk epoch, the start of the 20th
> > c, and the utcMucroseconds primitive answers microseconds from that epoch,
> > not the posix epoch. And it???s good for some 58,000 years before it
> > overflows 61-bit SmallIntegers (there are three tag bits) so there???s
> > essentially nothing to be gained from using the posix epoch.
> >
> > Now the posix epoch is important; and we should support it. But we should
> > clearly indicate it in spud, eg by having the word posix in the selector
> > somewhere.
> >
> > So the method should be renamed either utcPosixMicroseconds:offset: or
> > posixUTCMicroseconds:offset:, and utcMicroseconds:offset: should go what
> > Chris expects. The principle of least astonishment says it must be so.
> >
> >
> > >
> > > I realized this is because a different epoch being used (1901 vs. 1971)
> > between the two. This seems inconsistent and confusing, but that's less
> > important to me at the moment than the efficiency I'm looking for. Is
> > there another way?
> > >
> > > - Chris
> > >
> >
> >

Nicolas Cellier

Re: efficient DateAndTime

That remind me in 1999, my company was distributing a software built on Visualworks.

The main client asked for an analysis of robustness to Y2K.

I gave an analysis explaining in details why there were no bug to be expected.

As a reward for delivering good software, I've got nothing, not even a thank you...

I later learned that my client had a budget for helping main software providers to fix their own bugs!

I learned that: if you want to earn money in this industry, do deliver crappy code, just above the deal-breaker level, but no more.

And let the client pay for upgrades ;)

Le mer. 29 avr. 2020 à 21:14, David T. Lewis <[hidden email]> a écrit :

It should be a great party, I'll put it on my calendar!

But the year 60,000 shouldn't be a problem for programmers. All we need
to do is get everybody to start using sensible software such as Squeak:

'60000-01-01T00:00:00-00:00' asDateAndTime ==> 60000-01-01T00:00:00+00:00

Of course, from what I know about programmers, it will probably take
another 60,000 years to convince them to start using sensible software.

;-)
Dave

On Wed, Apr 29, 2020 at 11:02:17AM -0400, Ron Teitelbaum wrote:
> Can I just say now that I think the year 60,000 will be a bad year for
> programmers!
>
> (Sorry couldn't resist. We're gonna party like it's Fifty Nine Nine
> Ninety Nine!)
>
> Ron
>
> On Wed, Apr 29, 2020 at 5:18 AM Eliot Miranda <[hidden email]>
> wrote:
>
> > Hi Chris, Hi Dave,
> >
> > > On Apr 28, 2020, at 5:53 PM, Chris Muller <[hidden email]> wrote:
> > >
> > > ???
> > > I have a bunch of two-element Array log entries, where the first element
> > is the timestamp, the second is the object being logged. But these entries
> > may never be consumed, so I wish to minimize their cost. So, instead of
> > storing the timestamps as DateAndTimes, I wanted to keep them as
> > SmallIntegers. I thought I could use Time utcMicroseconds, now a
> > SmallInteger in 64-bit Squeak.
> > >
> > > However, when I went to instantiate the DateAndTime lazily from that
> > number it's said it's year 2089...
> > >
> > > DateAndTime utcMicroseconds: Time utcMicrosecondClock offset: 0
> > " 2089-04-29T00:49:26.3326+00:00"
> >
> > I gave yo say that this is a bug. Dave, I know you have affection for the
> > posix epoch but the system uses the Smalltalk epoch, the start of the 20th
> > c, and the utcMucroseconds primitive answers microseconds from that epoch,
> > not the posix epoch. And it???s good for some 58,000 years before it
> > overflows 61-bit SmallIntegers (there are three tag bits) so there???s
> > essentially nothing to be gained from using the posix epoch.
> >
> > Now the posix epoch is important; and we should support it. But we should
> > clearly indicate it in spud, eg by having the word posix in the selector
> > somewhere.
> >
> > So the method should be renamed either utcPosixMicroseconds:offset: or
> > posixUTCMicroseconds:offset:, and utcMicroseconds:offset: should go what
> > Chris expects. The principle of least astonishment says it must be so.
> >
> >
> > >
> > > I realized this is because a different epoch being used (1901 vs. 1971)
> > between the two. This seems inconsistent and confusing, but that's less
> > important to me at the moment than the efficiency I'm looking for. Is
> > there another way?
> > >
> > > - Chris
> > >
> >
> >

>

Giorgio Ferraris

Re: efficient DateAndTime

giorgio

On Wed, Apr 29, 2020 at 9:26 PM Nicolas Cellier <[hidden email]> wrote:

That remind me in 1999, my company was distributing a software built on Visualworks.
The main client asked for an analysis of robustness to Y2K.
I gave an analysis explaining in details why there were no bug to be expected.
As a reward for delivering good software, I've got nothing, not even a thank you...
I later learned that my client had a budget for helping main software providers to fix their own bugs!

I learned that: if you want to earn money in this industry, do deliver crappy code, just above the deal-breaker level, but no more.
And let the client pay for upgrades ;)

Le mer. 29 avr. 2020 à 21:14, David T. Lewis <[hidden email]> a écrit :
It should be a great party, I'll put it on my calendar!

But the year 60,000 shouldn't be a problem for programmers. All we need
to do is get everybody to start using sensible software such as Squeak:

'60000-01-01T00:00:00-00:00' asDateAndTime ==> 60000-01-01T00:00:00+00:00

Of course, from what I know about programmers, it will probably take
another 60,000 years to convince them to start using sensible software.

;-)
Dave

On Wed, Apr 29, 2020 at 11:02:17AM -0400, Ron Teitelbaum wrote:
> Can I just say now that I think the year 60,000 will be a bad year for
> programmers!
>
> (Sorry couldn't resist. We're gonna party like it's Fifty Nine Nine
> Ninety Nine!)
>
> Ron
>
> On Wed, Apr 29, 2020 at 5:18 AM Eliot Miranda <[hidden email]>
> wrote:
>
> > Hi Chris, Hi Dave,
> >
> > > On Apr 28, 2020, at 5:53 PM, Chris Muller <[hidden email]> wrote:
> > >
> > > ???
> > > I have a bunch of two-element Array log entries, where the first element
> > is the timestamp, the second is the object being logged. But these entries
> > may never be consumed, so I wish to minimize their cost. So, instead of
> > storing the timestamps as DateAndTimes, I wanted to keep them as
> > SmallIntegers. I thought I could use Time utcMicroseconds, now a
> > SmallInteger in 64-bit Squeak.
> > >
> > > However, when I went to instantiate the DateAndTime lazily from that
> > number it's said it's year 2089...
> > >
> > > DateAndTime utcMicroseconds: Time utcMicrosecondClock offset: 0
> > " 2089-04-29T00:49:26.3326+00:00"
> >
> > I gave yo say that this is a bug. Dave, I know you have affection for the
> > posix epoch but the system uses the Smalltalk epoch, the start of the 20th
> > c, and the utcMucroseconds primitive answers microseconds from that epoch,
> > not the posix epoch. And it???s good for some 58,000 years before it
> > overflows 61-bit SmallIntegers (there are three tag bits) so there???s
> > essentially nothing to be gained from using the posix epoch.
> >
> > Now the posix epoch is important; and we should support it. But we should
> > clearly indicate it in spud, eg by having the word posix in the selector
> > somewhere.
> >
> > So the method should be renamed either utcPosixMicroseconds:offset: or
> > posixUTCMicroseconds:offset:, and utcMicroseconds:offset: should go what
> > Chris expects. The principle of least astonishment says it must be so.
> >
> >
> > >
> > > I realized this is because a different epoch being used (1901 vs. 1971)
> > between the two. This seems inconsistent and confusing, but that's less
> > important to me at the moment than the efficiency I'm looking for. Is
> > there another way?
> > >
> > > - Chris
> > >
> >
> >

>

timrowledge

Re: efficient DateAndTime

In reply to this post by Nicolas Cellier

> On 2020-04-29, at 12:26 PM, Nicolas Cellier <[hidden email]> wrote:
>
> I learned that: if you want to earn money in this industry, do deliver crappy code, just above the deal-breaker level, but no more.
> And let the client pay for upgrades ;)

I'd agree but you still haven't upgraded to the AgreeableTim package

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: NBRM: Unconditional No BRanch Multiple

David T. Lewis

Re: efficient DateAndTime

On Wed, Apr 29, 2020 at 02:53:29PM -0700, tim Rowledge wrote:
>
>
> > On 2020-04-29, at 12:26 PM, Nicolas Cellier <[hidden email]> wrote:
> >
> > I learned that: if you want to earn money in this industry, do deliver crappy code, just above the deal-breaker level, but no more.
> > And let the client pay for upgrades ;)
>
> I'd agree but you still haven't upgraded to the AgreeableTim package
>

Due to recent budget efficiencies, management has decided to stick with
the DisagreeableTim release for a few more years.

Dave

Squeak - Dev mailing list

Re: efficient DateAndTime

In reply to this post by Nicolas Cellier

That’s been true for ages. Now it’s called Agile. I delivered a product back in 1986 that’s still in use and I still get calls about changing the program when it’s not necessary. Of course, I don’t get paid to consult, so I’m rather short with the people that call me.

/—————————————————————/

For encrypted mail use [hidden email] - Free account at ProtonMail.com

Web: https://objectnets.net and https://objectnets.org

https://datascilv.com https://datascilv.org

On Apr 29, 2020, at 12:28, Nicolas Cellier <[hidden email]> wrote:

I learned that: if you want to earn money in this industry, do deliver crappy code, just above the deal-breaker level, but no more.
And let the client pay for upgrades ;)