UTCDateAndTime discussion + Q&A

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

UTCDateAndTime discussion + Q&A

Chris Muller-4
Hi Dave,

Thanks for making UTCDateAndTime available on SqueakMap that "just
works" -- not having to go research how to properly install it left me
some time to actually start looking at it today.

A cursory look reveals that, at least functionally, everything appears
to be essentially the same as original Chronology except the on-going
million-dollar question:

             What are the point-in-time endpoints of a timespan
identified by a particular "Date"?
             There are two reasonable answers to this question:

                 1) "local" e.g., a period that begins from 12am to
11:59:59.999999 of my LOCAL time,
             or 2) "global" e.g., a period that begins from 12am to
11:59:59.999999 of UTC (offset 0).

We already know that original Chronlology supports ability for
applications to represent their Dates either way.  What about
UTCDateAndTime?   When I do:

        DateAndTime now asDate = Date today

I get "true", even though MY 22-Oct-2018 begins and ends at a
different point-in-time than those in the UTC timezone..

I realize most applications want a canonical representation of Dates,
but where does this leave the timespan use-cases?  Are they even
possible at all?   What if I truly need the same local representation
of  22-Oct-2018 that everyone else recognizes in my local area?

Thanks,
  Chris

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime discussion + Q&A

David T. Lewis
Hi Chris,

On Mon, Oct 22, 2018 at 04:30:52PM -0500, Chris Muller wrote:

> Hi Dave,
>
> Thanks for making UTCDateAndTime available on SqueakMap that "just
> works" -- not having to go research how to properly install it left me
> some time to actually start looking at it today.
>
> A cursory look reveals that, at least functionally, everything appears
> to be essentially the same as original Chronology except the on-going
> million-dollar question:
>
>              What are the point-in-time endpoints of a timespan
> identified by a particular "Date"?
>              There are two reasonable answers to this question:
>
>                  1) "local" e.g., a period that begins from 12am to
> 11:59:59.999999 of my LOCAL time,
>              or 2) "global" e.g., a period that begins from 12am to
> 11:59:59.999999 of UTC (offset 0).
>
> We already know that original Chronlology supports ability for
> applications to represent their Dates either way.  What about
> UTCDateAndTime?   When I do:
>
>         DateAndTime now asDate = Date today
>
> I get "true", even though MY 22-Oct-2018 begins and ends at a
> different point-in-time than those in the UTC timezone..
>
> I realize most applications want a canonical representation of Dates,
> but where does this leave the timespan use-cases?  Are they even
> possible at all?   What if I truly need the same local representation
> of  22-Oct-2018 that everyone else recognizes in my local area?
>

I think the difference is when we get to Date class>>starting: which
sets the starting point to be aDateAndTime midnight.

In classic Chronology:
  DateAndTime now asDate start ==> 2018-10-22T00:00:00-04:00

In UTCDateAndTime Chronology:
  DateAndTime now asDate start ==> 2018-10-22T00:00:00+00:00

So I think you are right. Given that a date is modeled as a duration, the
start time of the date should align with the timezone of the DateAndTime
from which it was created.

The likely fix is to change DateAndTime>>midnight:

  DateAndTime>>midnight
        "Answer a DateAndTime starting at midnight of the same timezone offset as the receiver."
        ^ self class basicNew
                setJdn: self julianDayNumber
                seconds: 0
                nano: 0
                offset: (Duration seconds: localOffsetSeconds)

But fixing that triggers one other test failure so I need to sort that
out before posting anything.

Thanks for looking at this, much appreciated.

Dave
 

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime discussion + Q&A

David T. Lewis
On Mon, Oct 22, 2018 at 09:02:57PM -0400, David T. Lewis wrote:

> Hi Chris,
>
> On Mon, Oct 22, 2018 at 04:30:52PM -0500, Chris Muller wrote:
> > Hi Dave,
> >
> > Thanks for making UTCDateAndTime available on SqueakMap that "just
> > works" -- not having to go research how to properly install it left me
> > some time to actually start looking at it today.
> >
> > A cursory look reveals that, at least functionally, everything appears
> > to be essentially the same as original Chronology except the on-going
> > million-dollar question:
> >
> >              What are the point-in-time endpoints of a timespan
> > identified by a particular "Date"?
> >              There are two reasonable answers to this question:
> >
> >                  1) "local" e.g., a period that begins from 12am to
> > 11:59:59.999999 of my LOCAL time,
> >              or 2) "global" e.g., a period that begins from 12am to
> > 11:59:59.999999 of UTC (offset 0).
> >
> > We already know that original Chronlology supports ability for
> > applications to represent their Dates either way.  What about
> > UTCDateAndTime?   When I do:
> >
> >         DateAndTime now asDate = Date today
> >
> > I get "true", even though MY 22-Oct-2018 begins and ends at a
> > different point-in-time than those in the UTC timezone..
> >
> > I realize most applications want a canonical representation of Dates,
> > but where does this leave the timespan use-cases?  Are they even
> > possible at all?   What if I truly need the same local representation
> > of  22-Oct-2018 that everyone else recognizes in my local area?
> >
>
> I think the difference is when we get to Date class>>starting: which
> sets the starting point to be aDateAndTime midnight.
>
> In classic Chronology:
>   DateAndTime now asDate start ==> 2018-10-22T00:00:00-04:00
>
> In UTCDateAndTime Chronology:
>   DateAndTime now asDate start ==> 2018-10-22T00:00:00+00:00
>
> So I think you are right. Given that a date is modeled as a duration, the
> start time of the date should align with the timezone of the DateAndTime
> from which it was created.
>
> The likely fix is to change DateAndTime>>midnight:
>
>   DateAndTime>>midnight
> "Answer a DateAndTime starting at midnight of the same timezone offset as the receiver."
> ^ self class basicNew
> setJdn: self julianDayNumber
> seconds: 0
> nano: 0
> offset: (Duration seconds: localOffsetSeconds)
>
> But fixing that triggers one other test failure so I need to sort that
> out before posting anything.
>
> Thanks for looking at this, much appreciated.
>

I added a test to cover this case in Chronology-Tests in trunk, and a
fix in the UTCDateAndTime repository in Chronology-Core-dtl.30.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime discussion + Q&A

Sean P. DeNigris
Administrator
In reply to this post by Chris Muller-4
Chris Muller-4 wrote
>                  1) "local" e.g., a period that begins from 12am to
> 11:59:59.999999 of my LOCAL time,

Not to reignite the long and numerous debates on the topic, but IHMO the
seemingly unresolvable (in-practice) problem with appending offsets to
historical dates is the offset-changing effect of daylight savings time.
What we want to know is "the offset of my local time on the date in
question", but all we know is "the local offset today". This leads to subtle
bugs like:
"executed just before DST"
timestamp := '1/1/2016' asDate start.
"executed just after"
timestamp = '1/1/2016' asDate start "false - WTH!"

The only solution seems to be more seriously modeling timezones w.r.t. all
the dates effecting them via an external authority. This is a nasty bug and
again IMHO is probably better not to provide this kind of behavior at all
rather than provide it in a subtly wrong way.



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Squeak-Dev-f45488.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime discussion + Q&A

Chris Muller-3
On Tue, Oct 23, 2018 at 10:17 AM Sean P. DeNigris <[hidden email]> wrote:

>
> Chris Muller-4 wrote
> >                  1) "local" e.g., a period that begins from 12am to
> > 11:59:59.999999 of my LOCAL time,
>
> Not to reignite the long and numerous debates on the topic, but IHMO the
> seemingly unresolvable (in-practice) problem with appending offsets to
> historical dates is the offset-changing effect of daylight savings time.
> What we want to know is "the offset of my local time on the date in
> question", but all we know is "the local offset today". This leads to subtle
> bugs like:
> "executed just before DST"
> timestamp := '1/1/2016' asDate start.
> "executed just after"
> timestamp = '1/1/2016' asDate start "false - WTH!"

The above example was fixed in Squeak in 2011 by introducing Timespan
class>>#defaultOffset (which at the time was Duration zero, but has
since been changed to nil) for Timespan instances (incl. Date, Month,
Year), and letting all future created instances "inherit" their
TZ-context from whatever other source / calculation that created them.
Since Timespans (incl. Date, Month, Year) created via the class
constructors have no such TZ-context, they assume the #defaultOffset,
and any instances that emerge from calculations, etc. from those, will
assume that same no-TZ context, too.  Kinda like a virus...

> The only solution seems to be more seriously modeling timezones w.r.t. all
> the dates effecting them via an external authority. This is a nasty bug and
> again IMHO is probably better not to provide this kind of behavior at all
> rather than provide it in a subtly wrong way.

I agree with your sentiments about bug types, but I don't think this
case is that serious.  Yes, it can and has been confusing for
developers new to Squeak before they understand that Dates's can be
_local_, and indeed do inherit that property from
DateAndTime>>#asDate.  But that's pretty much the only confusing one
and my hope is that by adding #asCanonicalDate, it will be "seen" by
them and help accelerate that learning curve.   Once they know, it's
pretty easy to know, in any code context, where a #beCanonical
conversion might be needed.

Best,
  Chris

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime discussion + Q&A

Chris Muller-3
In reply to this post by David T. Lewis
Hi Dave, I took a look at Chronology-Core-dtl.30, it looks like a good change.

Now we just need something to obtain a canonical date from
DateAndTime's similar to Chronology's #beCanonical.  BUT, given the
confusion we've had about #asDate in the past, what if we changed
#asDate to do the "expected" behavior -- answer the canonical version
--, and added, say, #asLocalDate to preserve the not-so-common
version?

 - Chris
On Mon, Oct 22, 2018 at 10:22 PM David T. Lewis <[hidden email]> wrote:

>
> On Mon, Oct 22, 2018 at 09:02:57PM -0400, David T. Lewis wrote:
> > Hi Chris,
> >
> > On Mon, Oct 22, 2018 at 04:30:52PM -0500, Chris Muller wrote:
> > > Hi Dave,
> > >
> > > Thanks for making UTCDateAndTime available on SqueakMap that "just
> > > works" -- not having to go research how to properly install it left me
> > > some time to actually start looking at it today.
> > >
> > > A cursory look reveals that, at least functionally, everything appears
> > > to be essentially the same as original Chronology except the on-going
> > > million-dollar question:
> > >
> > >              What are the point-in-time endpoints of a timespan
> > > identified by a particular "Date"?
> > >              There are two reasonable answers to this question:
> > >
> > >                  1) "local" e.g., a period that begins from 12am to
> > > 11:59:59.999999 of my LOCAL time,
> > >              or 2) "global" e.g., a period that begins from 12am to
> > > 11:59:59.999999 of UTC (offset 0).
> > >
> > > We already know that original Chronlology supports ability for
> > > applications to represent their Dates either way.  What about
> > > UTCDateAndTime?   When I do:
> > >
> > >         DateAndTime now asDate = Date today
> > >
> > > I get "true", even though MY 22-Oct-2018 begins and ends at a
> > > different point-in-time than those in the UTC timezone..
> > >
> > > I realize most applications want a canonical representation of Dates,
> > > but where does this leave the timespan use-cases?  Are they even
> > > possible at all?   What if I truly need the same local representation
> > > of  22-Oct-2018 that everyone else recognizes in my local area?
> > >
> >
> > I think the difference is when we get to Date class>>starting: which
> > sets the starting point to be aDateAndTime midnight.
> >
> > In classic Chronology:
> >   DateAndTime now asDate start ==> 2018-10-22T00:00:00-04:00
> >
> > In UTCDateAndTime Chronology:
> >   DateAndTime now asDate start ==> 2018-10-22T00:00:00+00:00
> >
> > So I think you are right. Given that a date is modeled as a duration, the
> > start time of the date should align with the timezone of the DateAndTime
> > from which it was created.
> >
> > The likely fix is to change DateAndTime>>midnight:
> >
> >   DateAndTime>>midnight
> >       "Answer a DateAndTime starting at midnight of the same timezone offset as the receiver."
> >       ^ self class basicNew
> >               setJdn: self julianDayNumber
> >               seconds: 0
> >               nano: 0
> >               offset: (Duration seconds: localOffsetSeconds)
> >
> > But fixing that triggers one other test failure so I need to sort that
> > out before posting anything.
> >
> > Thanks for looking at this, much appreciated.
> >
>
> I added a test to cover this case in Chronology-Tests in trunk, and a
> fix in the UTCDateAndTime repository in Chronology-Core-dtl.30.
>
> Dave
>
>

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime discussion + Q&A

David T. Lewis
On Fri, Oct 26, 2018 at 03:54:28PM -0500, Chris Muller wrote:
> Hi Dave, I took a look at Chronology-Core-dtl.30, it looks like a good change.
>
> Now we just need something to obtain a canonical date from
> DateAndTime's similar to Chronology's #beCanonical.  BUT, given the
> confusion we've had about #asDate in the past, what if we changed
> #asDate to do the "expected" behavior -- answer the canonical version
> --, and added, say, #asLocalDate to preserve the not-so-common
> version?

Hi Chris,

That sounds reasonable enough to me, although I would defer to you and
others as to what is the preferred default because I really don't have
any applications that would be impacted one way or the other.

What's your feeling overall regarding UTCDateAndTime? Does it seem like
a worthwhile change from your point of view? And to my main worry, is
there anything about the different layout of instance variables that is
likely to cause problems for Magma, or for applications that use other
databases for persistence? I think that it will be OK, but it's better
to ask now than to be sorry later.

Thanks,
Dave


>
>  - Chris
> On Mon, Oct 22, 2018 at 10:22 PM David T. Lewis <[hidden email]> wrote:
> >
> > On Mon, Oct 22, 2018 at 09:02:57PM -0400, David T. Lewis wrote:
> > > Hi Chris,
> > >
> > > On Mon, Oct 22, 2018 at 04:30:52PM -0500, Chris Muller wrote:
> > > > Hi Dave,
> > > >
> > > > Thanks for making UTCDateAndTime available on SqueakMap that "just
> > > > works" -- not having to go research how to properly install it left me
> > > > some time to actually start looking at it today.
> > > >
> > > > A cursory look reveals that, at least functionally, everything appears
> > > > to be essentially the same as original Chronology except the on-going
> > > > million-dollar question:
> > > >
> > > >              What are the point-in-time endpoints of a timespan
> > > > identified by a particular "Date"?
> > > >              There are two reasonable answers to this question:
> > > >
> > > >                  1) "local" e.g., a period that begins from 12am to
> > > > 11:59:59.999999 of my LOCAL time,
> > > >              or 2) "global" e.g., a period that begins from 12am to
> > > > 11:59:59.999999 of UTC (offset 0).
> > > >
> > > > We already know that original Chronlology supports ability for
> > > > applications to represent their Dates either way.  What about
> > > > UTCDateAndTime?   When I do:
> > > >
> > > >         DateAndTime now asDate = Date today
> > > >
> > > > I get "true", even though MY 22-Oct-2018 begins and ends at a
> > > > different point-in-time than those in the UTC timezone..
> > > >
> > > > I realize most applications want a canonical representation of Dates,
> > > > but where does this leave the timespan use-cases?  Are they even
> > > > possible at all?   What if I truly need the same local representation
> > > > of  22-Oct-2018 that everyone else recognizes in my local area?
> > > >
> > >
> > > I think the difference is when we get to Date class>>starting: which
> > > sets the starting point to be aDateAndTime midnight.
> > >
> > > In classic Chronology:
> > >   DateAndTime now asDate start ==> 2018-10-22T00:00:00-04:00
> > >
> > > In UTCDateAndTime Chronology:
> > >   DateAndTime now asDate start ==> 2018-10-22T00:00:00+00:00
> > >
> > > So I think you are right. Given that a date is modeled as a duration, the
> > > start time of the date should align with the timezone of the DateAndTime
> > > from which it was created.
> > >
> > > The likely fix is to change DateAndTime>>midnight:
> > >
> > >   DateAndTime>>midnight
> > >       "Answer a DateAndTime starting at midnight of the same timezone offset as the receiver."
> > >       ^ self class basicNew
> > >               setJdn: self julianDayNumber
> > >               seconds: 0
> > >               nano: 0
> > >               offset: (Duration seconds: localOffsetSeconds)
> > >
> > > But fixing that triggers one other test failure so I need to sort that
> > > out before posting anything.
> > >
> > > Thanks for looking at this, much appreciated.
> > >
> >
> > I added a test to cover this case in Chronology-Tests in trunk, and a
> > fix in the UTCDateAndTime repository in Chronology-Core-dtl.30.
> >
> > Dave
> >
> >
>

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime discussion + Q&A

Chris Muller-4
> What's your feeling overall regarding UTCDateAndTime? Does it seem like
> a worthwhile change from your point of view? And to my main worry, is

I would expect the storage and performance advantage of UTCDateAndTime
to be muted in 32-bit Squeak.  Chronology breaking down the "point" of
a point-in-time into 3 SmallIntegers -- days, seconds, nanos   (days
a.k.a., 'jdn') means there's always just three object pointers for
that.  50-bits for #utcMicroseconds in a 32-bit image, however, may be
nearly as expensive.

But since SmallIntegers run well beyond 50-bits in 64-bit Squeak, I
would expect a slight performance and storage advantage.

There's also the claimed understandability advantage, which is
something, but somewhat offset by the conversion / compatibility
issues users must face and endure.  "Worthwhile" is definitely the key
question we need to decide.  For me, it seems worth it in theory
because it seems like it shouldn't be too hard to convert.  The only
way to find out is to attempt to convert one of my most date-intensive
applications (including database) see how it goes, see what issues
arise.

> there anything about the different layout of instance variables that is
> likely to cause problems for Magma, or for applications that use other
> databases for persistence? I think that it will be OK, but it's better
> to ask now than to be sorry later.

There is the layout, there is also the hash calculation.  Could you
make it the same as in Chronology?  It could ease migration, but we
want it to remain fast.

We need to think about how different migrations could and should work.
Bulk conversions?  Lazy migrations?  We may need some way for apps to
be able to (quickly) ask if they're running the new UTCDateAndTime or
the old Chronology, to help support those...

  - Chris


 - Chris

> Thanks,
> Dave
>
>
> >
> >  - Chris
> > On Mon, Oct 22, 2018 at 10:22 PM David T. Lewis <[hidden email]> wrote:
> > >
> > > On Mon, Oct 22, 2018 at 09:02:57PM -0400, David T. Lewis wrote:
> > > > Hi Chris,
> > > >
> > > > On Mon, Oct 22, 2018 at 04:30:52PM -0500, Chris Muller wrote:
> > > > > Hi Dave,
> > > > >
> > > > > Thanks for making UTCDateAndTime available on SqueakMap that "just
> > > > > works" -- not having to go research how to properly install it left me
> > > > > some time to actually start looking at it today.
> > > > >
> > > > > A cursory look reveals that, at least functionally, everything appears
> > > > > to be essentially the same as original Chronology except the on-going
> > > > > million-dollar question:
> > > > >
> > > > >              What are the point-in-time endpoints of a timespan
> > > > > identified by a particular "Date"?
> > > > >              There are two reasonable answers to this question:
> > > > >
> > > > >                  1) "local" e.g., a period that begins from 12am to
> > > > > 11:59:59.999999 of my LOCAL time,
> > > > >              or 2) "global" e.g., a period that begins from 12am to
> > > > > 11:59:59.999999 of UTC (offset 0).
> > > > >
> > > > > We already know that original Chronlology supports ability for
> > > > > applications to represent their Dates either way.  What about
> > > > > UTCDateAndTime?   When I do:
> > > > >
> > > > >         DateAndTime now asDate = Date today
> > > > >
> > > > > I get "true", even though MY 22-Oct-2018 begins and ends at a
> > > > > different point-in-time than those in the UTC timezone..
> > > > >
> > > > > I realize most applications want a canonical representation of Dates,
> > > > > but where does this leave the timespan use-cases?  Are they even
> > > > > possible at all?   What if I truly need the same local representation
> > > > > of  22-Oct-2018 that everyone else recognizes in my local area?
> > > > >
> > > >
> > > > I think the difference is when we get to Date class>>starting: which
> > > > sets the starting point to be aDateAndTime midnight.
> > > >
> > > > In classic Chronology:
> > > >   DateAndTime now asDate start ==> 2018-10-22T00:00:00-04:00
> > > >
> > > > In UTCDateAndTime Chronology:
> > > >   DateAndTime now asDate start ==> 2018-10-22T00:00:00+00:00
> > > >
> > > > So I think you are right. Given that a date is modeled as a duration, the
> > > > start time of the date should align with the timezone of the DateAndTime
> > > > from which it was created.
> > > >
> > > > The likely fix is to change DateAndTime>>midnight:
> > > >
> > > >   DateAndTime>>midnight
> > > >       "Answer a DateAndTime starting at midnight of the same timezone offset as the receiver."
> > > >       ^ self class basicNew
> > > >               setJdn: self julianDayNumber
> > > >               seconds: 0
> > > >               nano: 0
> > > >               offset: (Duration seconds: localOffsetSeconds)
> > > >
> > > > But fixing that triggers one other test failure so I need to sort that
> > > > out before posting anything.
> > > >
> > > > Thanks for looking at this, much appreciated.
> > > >
> > >
> > > I added a test to cover this case in Chronology-Tests in trunk, and a
> > > fix in the UTCDateAndTime repository in Chronology-Core-dtl.30.
> > >
> > > Dave
> > >
> > >
> >

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime discussion + Q&A

David T. Lewis
Hi Chris,

On Fri, Oct 26, 2018 at 10:26:20PM -0500, Chris Muller wrote:

> > What's your feeling overall regarding UTCDateAndTime? Does it seem like
> > a worthwhile change from your point of view? And to my main worry, is
>
> I would expect the storage and performance advantage of UTCDateAndTime
> to be muted in 32-bit Squeak.  Chronology breaking down the "point" of
> a point-in-time into 3 SmallIntegers -- days, seconds, nanos   (days
> a.k.a., 'jdn') means there's always just three object pointers for
> that.  50-bits for #utcMicroseconds in a 32-bit image, however, may be
> nearly as expensive.
>
> But since SmallIntegers run well beyond 50-bits in 64-bit Squeak, I
> would expect a slight performance and storage advantage.

To check performance, you can use the DateAndTimePerformance package
in the http://www.squeaksource.com/UTCDateAndTime repository. I think
you will be pleased with the results, although I have only run it on
Linux so it would be good to check Windows and OSX also.

>
> There's also the claimed understandability advantage, which is
> something, but somewhat offset by the conversion / compatibility
> issues users must face and endure.  "Worthwhile" is definitely the key
> question we need to decide.  For me, it seems worth it in theory
> because it seems like it shouldn't be too hard to convert.  The only
> way to find out is to attempt to convert one of my most date-intensive
> applications (including database) see how it goes, see what issues
> arise.
>
> > there anything about the different layout of instance variables that is
> > likely to cause problems for Magma, or for applications that use other
> > databases for persistence? I think that it will be OK, but it's better
> > to ask now than to be sorry later.
>
> There is the layout, there is also the hash calculation.  Could you
> make it the same as in Chronology?  It could ease migration, but we
> want it to remain fast.

The hash in the UTCDateAndTime version is:

  DateAndTime>>hash
      ^utcMicroseconds hash

I don't know if this is a good hash function, although it at least
has the virtue of simplicity.

I expect we could make it compatible with the existing function if
that is a better thing to do, although it might have some performance
impact.

>
> We need to think about how different migrations could and should work.
> Bulk conversions?  Lazy migrations?  We may need some way for apps to
> be able to (quickly) ask if they're running the new UTCDateAndTime or
> the old Chronology, to help support those...

Some kind of test like "DateAndTime instVarNames size" should work.

Dave


> >
> > >
> > > On Mon, Oct 22, 2018 at 10:22 PM David T. Lewis <[hidden email]> wrote:
> > > >
> > > > On Mon, Oct 22, 2018 at 09:02:57PM -0400, David T. Lewis wrote:
> > > > > Hi Chris,
> > > > >
> > > > > On Mon, Oct 22, 2018 at 04:30:52PM -0500, Chris Muller wrote:
> > > > > > Hi Dave,
> > > > > >
> > > > > > Thanks for making UTCDateAndTime available on SqueakMap that "just
> > > > > > works" -- not having to go research how to properly install it left me
> > > > > > some time to actually start looking at it today.
> > > > > >
> > > > > > A cursory look reveals that, at least functionally, everything appears
> > > > > > to be essentially the same as original Chronology except the on-going
> > > > > > million-dollar question:
> > > > > >
> > > > > >              What are the point-in-time endpoints of a timespan
> > > > > > identified by a particular "Date"?
> > > > > >              There are two reasonable answers to this question:
> > > > > >
> > > > > >                  1) "local" e.g., a period that begins from 12am to
> > > > > > 11:59:59.999999 of my LOCAL time,
> > > > > >              or 2) "global" e.g., a period that begins from 12am to
> > > > > > 11:59:59.999999 of UTC (offset 0).
> > > > > >
> > > > > > We already know that original Chronlology supports ability for
> > > > > > applications to represent their Dates either way.  What about
> > > > > > UTCDateAndTime?   When I do:
> > > > > >
> > > > > >         DateAndTime now asDate = Date today
> > > > > >
> > > > > > I get "true", even though MY 22-Oct-2018 begins and ends at a
> > > > > > different point-in-time than those in the UTC timezone..
> > > > > >
> > > > > > I realize most applications want a canonical representation of Dates,
> > > > > > but where does this leave the timespan use-cases?  Are they even
> > > > > > possible at all?   What if I truly need the same local representation
> > > > > > of  22-Oct-2018 that everyone else recognizes in my local area?
> > > > > >
> > > > >
> > > > > I think the difference is when we get to Date class>>starting: which
> > > > > sets the starting point to be aDateAndTime midnight.
> > > > >
> > > > > In classic Chronology:
> > > > >   DateAndTime now asDate start ==> 2018-10-22T00:00:00-04:00
> > > > >
> > > > > In UTCDateAndTime Chronology:
> > > > >   DateAndTime now asDate start ==> 2018-10-22T00:00:00+00:00
> > > > >
> > > > > So I think you are right. Given that a date is modeled as a duration, the
> > > > > start time of the date should align with the timezone of the DateAndTime
> > > > > from which it was created.
> > > > >
> > > > > The likely fix is to change DateAndTime>>midnight:
> > > > >
> > > > >   DateAndTime>>midnight
> > > > >       "Answer a DateAndTime starting at midnight of the same timezone offset as the receiver."
> > > > >       ^ self class basicNew
> > > > >               setJdn: self julianDayNumber
> > > > >               seconds: 0
> > > > >               nano: 0
> > > > >               offset: (Duration seconds: localOffsetSeconds)
> > > > >
> > > > > But fixing that triggers one other test failure so I need to sort that
> > > > > out before posting anything.
> > > > >
> > > > > Thanks for looking at this, much appreciated.
> > > > >
> > > >
> > > > I added a test to cover this case in Chronology-Tests in trunk, and a
> > > > fix in the UTCDateAndTime repository in Chronology-Core-dtl.30.
> > > >
> > > > Dave
> > > >
> > > >
> > >
>

Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime discussion + Q&A

David T. Lewis
In reply to this post by Chris Muller-4
On Mon, Oct 22, 2018 at 04:30:52PM -0500, Chris Muller wrote:
> Hi Dave,
>
> Thanks for making UTCDateAndTime available on SqueakMap that "just
> works" -- not having to go research how to properly install it left me
> some time to actually start looking at it today.

The SqueakMap entry for UTCDateAndTime has started to fail due to a bug
in Installer. A proposed fix is in the inbox as Installer-Core-dtl.425.

The Installer script started failing when I later published
'Chronology-Core-dtl.30' to the repository, which Installer is now
confusing with the 'Chronology-Core-dtl.3' version that it should
actually be loading. The fix in the inbox, or some similar fix, will
make the SqueakMap entry for UTCDateAndTime work properly again.

Thanks to Chris for noticing the problem.

Dave


cbc
Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime discussion + Q&A

cbc
In reply to this post by David T. Lewis
HI.


On Fri, Oct 26, 2018 at 4:39 PM David T. Lewis <[hidden email]> wrote:
On Fri, Oct 26, 2018 at 03:54:28PM -0500, Chris Muller wrote:
> Hi Dave, I took a look at Chronology-Core-dtl.30, it looks like a good change.
>
> Now we just need something to obtain a canonical date from
> DateAndTime's similar to Chronology's #beCanonical.  BUT, given the
> confusion we've had about #asDate in the past, what if we changed
> #asDate to do the "expected" behavior -- answer the canonical version
> --, and added, say, #asLocalDate to preserve the not-so-common
> version?

Hi Chris,

That sounds reasonable enough to me, although I would defer to you and
others as to what is the preferred default because I really don't have
any applications that would be impacted one way or the other.

I think I would prefer to have DateAndTime now asDate (the asDate part) return a canonical date - one with the starting time having no offset at all.  While I have had cases to use the local date, they are actually rare and far between. On the other hand, wanting a canonical date is very common, and apparently expected by most other users.
I definitely approve of adding a #asLocalDate for those cases where I want the local date.

Along these lines, I think we should make the default #asMonth, #asYear, #asWeek also return canonical instances.  These make even less sense to have in the 'local' time.
That said, the default Duration probably doesn't deserve the same canonical treatment - if I want 2-1/2 hours starting now, I definitely want it in the same timezone!

-cbc

ps - I strongly vote for integrating UTCDateAndTime into Trunk, having just been bitten by the daylight savings again - my long running image never switched over to the new offsets, since that apparently only happens when you restart the image.  So, wrong offset for a good week before I noticed.
 
What's your feeling overall regarding UTCDateAndTime? Does it seem like
a worthwhile change from your point of view? And to my main worry, is
there anything about the different layout of instance variables that is
likely to cause problems for Magma, or for applications that use other
databases for persistence? I think that it will be OK, but it's better
to ask now than to be sorry later.

Thanks,
Dave


>
>  - Chris
> On Mon, Oct 22, 2018 at 10:22 PM David T. Lewis <[hidden email]> wrote:
> >
> > On Mon, Oct 22, 2018 at 09:02:57PM -0400, David T. Lewis wrote:
> > > Hi Chris,
> > >
> > > On Mon, Oct 22, 2018 at 04:30:52PM -0500, Chris Muller wrote:
> > > > Hi Dave,
> > > >
> > > > Thanks for making UTCDateAndTime available on SqueakMap that "just
> > > > works" -- not having to go research how to properly install it left me
> > > > some time to actually start looking at it today.
> > > >
> > > > A cursory look reveals that, at least functionally, everything appears
> > > > to be essentially the same as original Chronology except the on-going
> > > > million-dollar question:
> > > >
> > > >              What are the point-in-time endpoints of a timespan
> > > > identified by a particular "Date"?
> > > >              There are two reasonable answers to this question:
> > > >
> > > >                  1) "local" e.g., a period that begins from 12am to
> > > > 11:59:59.999999 of my LOCAL time,
> > > >              or 2) "global" e.g., a period that begins from 12am to
> > > > 11:59:59.999999 of UTC (offset 0).
> > > >
> > > > We already know that original Chronlology supports ability for
> > > > applications to represent their Dates either way.  What about
> > > > UTCDateAndTime?   When I do:
> > > >
> > > >         DateAndTime now asDate = Date today
> > > >
> > > > I get "true", even though MY 22-Oct-2018 begins and ends at a
> > > > different point-in-time than those in the UTC timezone..
> > > >
> > > > I realize most applications want a canonical representation of Dates,
> > > > but where does this leave the timespan use-cases?  Are they even
> > > > possible at all?   What if I truly need the same local representation
> > > > of  22-Oct-2018 that everyone else recognizes in my local area?
> > > >
> > >
> > > I think the difference is when we get to Date class>>starting: which
> > > sets the starting point to be aDateAndTime midnight.
> > >
> > > In classic Chronology:
> > >   DateAndTime now asDate start ==> 2018-10-22T00:00:00-04:00
> > >
> > > In UTCDateAndTime Chronology:
> > >   DateAndTime now asDate start ==> 2018-10-22T00:00:00+00:00
> > >
> > > So I think you are right. Given that a date is modeled as a duration, the
> > > start time of the date should align with the timezone of the DateAndTime
> > > from which it was created.
> > >
> > > The likely fix is to change DateAndTime>>midnight:
> > >
> > >   DateAndTime>>midnight
> > >       "Answer a DateAndTime starting at midnight of the same timezone offset as the receiver."
> > >       ^ self class basicNew
> > >               setJdn: self julianDayNumber
> > >               seconds: 0
> > >               nano: 0
> > >               offset: (Duration seconds: localOffsetSeconds)
> > >
> > > But fixing that triggers one other test failure so I need to sort that
> > > out before posting anything.
> > >
> > > Thanks for looking at this, much appreciated.
> > >
> >
> > I added a test to cover this case in Chronology-Tests in trunk, and a
> > fix in the UTCDateAndTime repository in Chronology-Core-dtl.30.
> >
> > Dave
> >
> >
>



Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime discussion + Q&A

Eliot Miranda-2
In reply to this post by Chris Muller-4
Hi Chris,

On Fri, Oct 26, 2018 at 8:27 PM Chris Muller <[hidden email]> wrote:
> What's your feeling overall regarding UTCDateAndTime? Does it seem like
> a worthwhile change from your point of view? And to my main worry, is

I would expect the storage and performance advantage of UTCDateAndTime
to be muted in 32-bit Squeak.  Chronology breaking down the "point" of
a point-in-time into 3 SmallIntegers -- days, seconds, nanos   (days
a.k.a., 'jdn') means there's always just three object pointers for
that.  50-bits for #utcMicroseconds in a 32-bit image, however, may be
nearly as expensive.

But since SmallIntegers run well beyond 50-bits in 64-bit Squeak, I
would expect a slight performance and storage advantage.

There's also the claimed understandability advantage, which is
something, but somewhat offset by the conversion / compatibility
issues users must face and endure.  "Worthwhile" is definitely the key
question we need to decide.  For me, it seems worth it in theory
because it seems like it shouldn't be too hard to convert.  The only
way to find out is to attempt to convert one of my most date-intensive
applications (including database) see how it goes, see what issues
arise.

The simplicity of the large integer utcMicroseconds representation trumps all the nonsense of breaking it down into sub components.  In any case the 32-bit VM is communicating time up to the image as 64-bit large integer microseconds anyway.  Not decomposing gives much faster instantiation, and very simple arithmetic (simply compare the utcMicroseconds).

Note that it would be trivial to extend the representation with the decomposed elements, an d these could be nil initially and instantiated on demand.  Forcing the large integer arithmetic to decompose on every instantiation would kill any performance advantage one might expect to get from using immediate instead of there large integer.  And of course in 64-bits, utcMicroseconds is an immediate anyway.


> there anything about the different layout of instance variables that is
> likely to cause problems for Magma, or for applications that use other
> databases for persistence? I think that it will be OK, but it's better
> to ask now than to be sorry later.

There is the layout, there is also the hash calculation.  Could you
make it the same as in Chronology?  It could ease migration, but we
want it to remain fast.

We need to think about how different migrations could and should work.
Bulk conversions?  Lazy migrations?  We may need some way for apps to
be able to (quickly) ask if they're running the new UTCDateAndTime or
the old Chronology, to help support those...

  - Chris


 - Chris

> Thanks,
> Dave
>
>
> >
> >  - Chris
> > On Mon, Oct 22, 2018 at 10:22 PM David T. Lewis <[hidden email]> wrote:
> > >
> > > On Mon, Oct 22, 2018 at 09:02:57PM -0400, David T. Lewis wrote:
> > > > Hi Chris,
> > > >
> > > > On Mon, Oct 22, 2018 at 04:30:52PM -0500, Chris Muller wrote:
> > > > > Hi Dave,
> > > > >
> > > > > Thanks for making UTCDateAndTime available on SqueakMap that "just
> > > > > works" -- not having to go research how to properly install it left me
> > > > > some time to actually start looking at it today.
> > > > >
> > > > > A cursory look reveals that, at least functionally, everything appears
> > > > > to be essentially the same as original Chronology except the on-going
> > > > > million-dollar question:
> > > > >
> > > > >              What are the point-in-time endpoints of a timespan
> > > > > identified by a particular "Date"?
> > > > >              There are two reasonable answers to this question:
> > > > >
> > > > >                  1) "local" e.g., a period that begins from 12am to
> > > > > 11:59:59.999999 of my LOCAL time,
> > > > >              or 2) "global" e.g., a period that begins from 12am to
> > > > > 11:59:59.999999 of UTC (offset 0).
> > > > >
> > > > > We already know that original Chronlology supports ability for
> > > > > applications to represent their Dates either way.  What about
> > > > > UTCDateAndTime?   When I do:
> > > > >
> > > > >         DateAndTime now asDate = Date today
> > > > >
> > > > > I get "true", even though MY 22-Oct-2018 begins and ends at a
> > > > > different point-in-time than those in the UTC timezone..
> > > > >
> > > > > I realize most applications want a canonical representation of Dates,
> > > > > but where does this leave the timespan use-cases?  Are they even
> > > > > possible at all?   What if I truly need the same local representation
> > > > > of  22-Oct-2018 that everyone else recognizes in my local area?
> > > > >
> > > >
> > > > I think the difference is when we get to Date class>>starting: which
> > > > sets the starting point to be aDateAndTime midnight.
> > > >
> > > > In classic Chronology:
> > > >   DateAndTime now asDate start ==> 2018-10-22T00:00:00-04:00
> > > >
> > > > In UTCDateAndTime Chronology:
> > > >   DateAndTime now asDate start ==> 2018-10-22T00:00:00+00:00
> > > >
> > > > So I think you are right. Given that a date is modeled as a duration, the
> > > > start time of the date should align with the timezone of the DateAndTime
> > > > from which it was created.
> > > >
> > > > The likely fix is to change DateAndTime>>midnight:
> > > >
> > > >   DateAndTime>>midnight
> > > >       "Answer a DateAndTime starting at midnight of the same timezone offset as the receiver."
> > > >       ^ self class basicNew
> > > >               setJdn: self julianDayNumber
> > > >               seconds: 0
> > > >               nano: 0
> > > >               offset: (Duration seconds: localOffsetSeconds)
> > > >
> > > > But fixing that triggers one other test failure so I need to sort that
> > > > out before posting anything.
> > > >
> > > > Thanks for looking at this, much appreciated.
> > > >
> > >
> > > I added a test to cover this case in Chronology-Tests in trunk, and a
> > > fix in the UTCDateAndTime repository in Chronology-Core-dtl.30.
> > >
> > > Dave
> > >
> > >
> >



--
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: UTCDateAndTime discussion + Q&A

Chris Muller-3
HI Eliot,

>> There's also the claimed understandability advantage, which is
>> something, but somewhat offset by the conversion / compatibility
>> issues users must face and endure.  "Worthwhile" is definitely the key
>> question we need to decide.  For me, it seems worth it in theory
>> because it seems like it shouldn't be too hard to convert.  The only
>> way to find out is to attempt to convert one of my most date-intensive
>> applications (including database) see how it goes, see what issues
>> arise.
>
> The simplicity of the large integer utcMicroseconds representation trumps all the nonsense of breaking it down into sub components.  In any case the 32-bit VM is communicating time up to the image as 64-bit large integer microseconds anyway.  Not decomposing gives much faster instantiation, and very simple arithmetic (simply compare the utcMicroseconds).

I've been getting a good look at UTCDateAndTime and emailing Dave for
the last few days.  I like it for 64-bit and agree with your
sentiments except for the old way being "nonsense".  It was made in
the early 2000's when 32-bit was all we had.  It beat a LargeInteger
based competitor I tried to use for a while so I switched.

Dave has been patient helping me get the Ma Serializer upgraded.  I'm
trying to find a way to preserve forward-compatibility in legacy
systems, otherwise it means having to shut everything down and upgrade
every client image.  Simultaneously.  Big bang style.

> Note that it would be trivial to extend the representation with the decomposed elements, an d these could be nil initially and instantiated on demand.  Forcing the large integer arithmetic to decompose on every instantiation would kill any performance advantage one might expect to get from using immediate instead of there large integer.  And of course in 64-bits, utcMicroseconds is an immediate anyway.

That is an **amazing idea**!  The code should decompose the new
utcMicroseconds to the old smaller values on-demand but also the
reverse.  For UTC clients reading legacy data, the 'utcMicroseconds'
variable will be mapped to nil upon materialization, after which the
code would lazily calculate it on-demand.  Brilliant!

Will definitely have to look into this..

Best,
  Chris