DateAndTime Offset Bug Proposal

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

DateAndTime Offset Bug Proposal

Sean P. DeNigris
Administrator
I was bitten by this very annoying bug again. As most of us probably know due
to the steady stream of confused ML posts in the past, the bug in summary is
that we have an incomplete timezone implementation that doesn't properly
take into account historical DST changes. This flares up without warning
especially when DST toggles. I created a wiki page to document the
situation: https://github.com/seandenigris/pharo/wiki/Time-Zone-Fiasco

Here's an example blowup: at 11:59pm before DST changes, eval aDate :=
'1/1/1901' asDate. Now, wait two minutes and at 12:01am eval self assert:
'1/1/1901' asDate = aDate and… whammo, an exception! The "different" offsets
render equal dates unequal depending on when the objects were created.

The more I think about it, the more I think that we should just assume UTC
for all Date[AndTime]s that don't explicitly specify an offset, rather than
pretend to set an offset which is only sometimes correct. More advanced
users can use one of the available libraries to get full timezone support.
What do you think?



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Max Leske
Hi Sean,

On 8 April 2018 at 15:33:11, Sean P. DeNigris ([hidden email]) wrote:

I was bitten by this very annoying bug again. As most of us probably know due 
to the steady stream of confused ML posts in the past, the bug in summary is 
that we have an incomplete timezone implementation that doesn't properly 
take into account historical DST changes. This flares up without warning 
especially when DST toggles. I created a wiki page to document the 
situation: https://github.com/seandenigris/pharo/wiki/Time-Zone-Fiasco 

Here's an example blowup: at 11:59pm before DST changes, eval aDate := 
'1/1/1901' asDate. Now, wait two minutes and at 12:01am eval self assert: 
'1/1/1901' asDate = aDate and… whammo, an exception! The "different" offsets 
render equal dates unequal depending on when the objects were created. 

The more I think about it, the more I think that we should just assume UTC 
for all Date[AndTime]s that don't explicitly specify an offset, rather than 
pretend to set an offset which is only sometimes correct. More advanced 
users can use one of the available libraries to get full timezone support. 
What do you think? 

I first wanted to fully agree with you but then I had to think back on the times I've had to fight timestamp bugs when working with Pharo and relational databases. A database will make its own assumptions about the offset. From the PostgreSQL documentation (https://www.postgresql.org/docs/9.1/static/datatype-datetime.html):

"For timestamp with time zone, the internally stored value is always in UTC (Universal Coordinated Time, traditionally known as Greenwich Mean Time, GMT). An input value that has an explicit time zone specified is converted to UTC using the appropriate offset for that time zone. If no time zone is stated in the input string, then it is assumed to be in the time zone indicated by the system's timezone parameter, and is converted to UTC using the offset for the timezone zone."

Assuming UTC is probably just as wrong as assuming the local time zone. Maybe the problem is more about the interface of DateAndTime, where you don't really have to care about the time zone offset until it bites you. If the interface would force you to specify the time zone, then you'd be aware from the beginning that it's something to take into account.


Cheers,

Max

Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Sven Van Caekenberghe-2


> On 9 Apr 2018, at 08:51, Max Leske <[hidden email]> wrote:
>
> Hi Sean,
> On 8 April 2018 at 15:33:11, Sean P. DeNigris ([hidden email]) wrote:
>
>> I was bitten by this very annoying bug again. As most of us probably know due
>> to the steady stream of confused ML posts in the past, the bug in summary is
>> that we have an incomplete timezone implementation that doesn't properly
>> take into account historical DST changes. This flares up without warning
>> especially when DST toggles. I created a wiki page to document the
>> situation: https://github.com/seandenigris/pharo/wiki/Time-Zone-Fiasco 
>>
>> Here's an example blowup: at 11:59pm before DST changes, eval aDate :=
>> '1/1/1901' asDate. Now, wait two minutes and at 12:01am eval self assert:
>> '1/1/1901' asDate = aDate and… whammo, an exception! The "different" offsets
>> render equal dates unequal depending on when the objects were created.
>>
>> The more I think about it, the more I think that we should just assume UTC
>> for all Date[AndTime]s that don't explicitly specify an offset, rather than
>> pretend to set an offset which is only sometimes correct. More advanced
>> users can use one of the available libraries to get full timezone support.
>> What do you think?
> I first wanted to fully agree with you but then I had to think back on the times I've had to fight timestamp bugs when working with Pharo and relational databases. A database will make its own assumptions about the offset. From the PostgreSQL documentation (https://www.postgresql.org/docs/9.1/static/datatype-datetime.html):
>
> "For timestamp with time zone, the internally stored value is always in UTC (Universal Coordinated Time, traditionally known as Greenwich Mean Time, GMT). An input value that has an explicit time zone specified is converted to UTC using the appropriate offset for that time zone. If no time zone is stated in the input string, then it is assumed to be in the time zone indicated by the system's timezone parameter, and is converted to UTC using the offset for the timezone zone."
>
> Assuming UTC is probably just as wrong as assuming the local time zone. Maybe the problem is more about the interface of DateAndTime, where you don't really have to care about the time zone offset until it bites you. If the interface would force you to specify the time zone, then you'd be aware from the beginning that it's something to take into account.

Right now, a Date is defined as a timespan of 1 day starting from a precise point in time (a DateAndTime with an offset). The fact that it is more advanced is hidden behind a simpler (too simple) API.

I think most people think of a Date as a AbstractJulianDate with three components: year/month/day that they want to interpret in there own way (in there own timezone). I would not call that an UTCDate, although that would be one interpretation of the abstraction.

I don't think there is an easy/universal solution.

> Cheers,
>
> Max
>
>>
>>
>>
>>
>> -----
>> Cheers,
>> Sean
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html 
>>


Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

alistairgrant
In reply to this post by Sean P. DeNigris
Hi Sean,

On 8 April 2018 at 15:32, Sean P. DeNigris <[hidden email]> wrote:

> I was bitten by this very annoying bug again. As most of us probably know due
> to the steady stream of confused ML posts in the past, the bug in summary is
> that we have an incomplete timezone implementation that doesn't properly
> take into account historical DST changes. This flares up without warning
> especially when DST toggles. I created a wiki page to document the
> situation: https://github.com/seandenigris/pharo/wiki/Time-Zone-Fiasco
>
> Here's an example blowup: at 11:59pm before DST changes, eval aDate :=
> '1/1/1901' asDate. Now, wait two minutes and at 12:01am eval self assert:
> '1/1/1901' asDate = aDate and… whammo, an exception! The "different" offsets
> render equal dates unequal depending on when the objects were created.
>
> The more I think about it, the more I think that we should just assume UTC
> for all Date[AndTime]s that don't explicitly specify an offset, rather than
> pretend to set an offset which is only sometimes correct. More advanced
> users can use one of the available libraries to get full timezone support.
> What do you think?

I think using UTC would just move the problem by an hour.  As Sven
says, the real issue is Date as a timespan vs Date as a d/m/y.  What
you're doing here is comparing timespans, and one timespan has been
moved by 1 hour due to the start of DST.

If you did:

'1/1/1901' asDate equals: aDate

Then everything would be fine.

Cheers,
Alistair

Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Sean P. DeNigris
Administrator
Alistair Grant wrote
> I think using UTC would just move the problem by an hour.

I'm not sure I understand. How so? I thought UTC is a constant reference
point unaffected by UTC (https://stackoverflow.com/a/5495816/424245)



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Sean P. DeNigris
Administrator
In reply to this post by Max Leske
Max Leske wrote
> Assuming UTC is probably just as wrong as assuming the local time zone.

I'll modify your statement slightly. "Assuming UTC is */almost/* as wrong as
assuming the local time zone."

I've never seemed to be able to drive the essential point home when these
discussions have come up. Beyond all the wider issues, there is a bug, plain
and simple: The offset of `'1/1/1990' asDate`, considering that you mean
local time, is still not guaranteed to be the current local offset of the
image, which is how we set it by default. That only makes sense if the
historical date in question was in the same state of DST as the current
image. For example, the historical date above is in winter, so if I eval
that code in summer, it will /always/ give the objectively wrong offset.

What I'm proposing is not a cure all, but a slightly-better way that fixes
this bug by giving users consistent behavior that they may not want instead
of inconsistent and often wrong behavior that they may not want.



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Sven Van Caekenberghe-2


> On 9 Apr 2018, at 15:19, Sean P. DeNigris <[hidden email]> wrote:
>
> Max Leske wrote
>> Assuming UTC is probably just as wrong as assuming the local time zone.
>
> I'll modify your statement slightly. "Assuming UTC is */almost/* as wrong as
> assuming the local time zone."
>
> I've never seemed to be able to drive the essential point home when these
> discussions have come up. Beyond all the wider issues, there is a bug, plain
> and simple: The offset of `'1/1/1990' asDate`, considering that you mean
> local time, is still not guaranteed to be the current local offset of the
> image, which is how we set it by default. That only makes sense if the
> historical date in question was in the same state of DST as the current
> image. For example, the historical date above is in winter, so if I eval
> that code in summer, it will /always/ give the objectively wrong offset.
>
> What I'm proposing is not a cure all, but a slightly-better way that fixes
> this bug by giving users consistent behavior that they may not want instead
> of inconsistent and often wrong behavior that they may not want.

Sean,

You are right, the current system cannot be fixed. It only knows about the current timezone's offset (via the OS), not about historical offsets. And it wrongly uses that offset because it does not know better.

Neither

  '1/1/1990' asDate.

nor

  Date today.

can work without the context of a precise timezone. It is even relatively pointless to remember offsets without remembering timezones. You simply need a precise reference into the transitions database.

New York is 5 hours behind UTC in winter.

Question 1: When (in absolute UTC time) was the beginning of the 1st day of January in 1990 in New York's local time, when we express the date in UTC ?

(ZTimezone id: 'America/New_York') gmtToLocal: (ZTimestamp @ '1990/01/01').

  => "1989-12-31T19:00:00Z"

So when the UTC day of January 1st 1990 starts, New York local time is still 5 hours behind.

Question 2: When (in absolute UTC time) was the beginning of the 1st day of January in 1990 in UTC time, when we express the date locally ?

(ZTimezone id: 'America/New_York') localToGmt: (ZTimestamp @ '1990/01/01').

  => "1990-01-01T05:00:00Z"

So when the New York day of January 1st 1990 starts, UTC time is already 5 hours ahead.

Note that the question 'When does January 1st 1990 start in any timezone, when expressed in that timezone, is of course a constant, midnight'.


I think that making Date always UTC will probably not help, because you will want to be able to move between timezones. I guess the only solution is to add a class like ZTimezone (which has no dependencies).


Sven

> -----
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
>


Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Aliaksei Syrel
Must watch :)

The Problem with Time & Timezones - Computerphile
https://www.youtube.com/watch?v=-5wpm-gesOY

On 9 April 2018 at 17:26, Sven Van Caekenberghe <[hidden email]> wrote:


> On 9 Apr 2018, at 15:19, Sean P. DeNigris <[hidden email]> wrote:
>
> Max Leske wrote
>> Assuming UTC is probably just as wrong as assuming the local time zone.
>
> I'll modify your statement slightly. "Assuming UTC is */almost/* as wrong as
> assuming the local time zone."
>
> I've never seemed to be able to drive the essential point home when these
> discussions have come up. Beyond all the wider issues, there is a bug, plain
> and simple: The offset of `'1/1/1990' asDate`, considering that you mean
> local time, is still not guaranteed to be the current local offset of the
> image, which is how we set it by default. That only makes sense if the
> historical date in question was in the same state of DST as the current
> image. For example, the historical date above is in winter, so if I eval
> that code in summer, it will /always/ give the objectively wrong offset.
>
> What I'm proposing is not a cure all, but a slightly-better way that fixes
> this bug by giving users consistent behavior that they may not want instead
> of inconsistent and often wrong behavior that they may not want.

Sean,

You are right, the current system cannot be fixed. It only knows about the current timezone's offset (via the OS), not about historical offsets. And it wrongly uses that offset because it does not know better.

Neither

  '1/1/1990' asDate.

nor

  Date today.

can work without the context of a precise timezone. It is even relatively pointless to remember offsets without remembering timezones. You simply need a precise reference into the transitions database.

New York is 5 hours behind UTC in winter.

Question 1: When (in absolute UTC time) was the beginning of the 1st day of January in 1990 in New York's local time, when we express the date in UTC ?

(ZTimezone id: 'America/New_York') gmtToLocal: (ZTimestamp @ '1990/01/01').

  => "1989-12-31T19:00:00Z"

So when the UTC day of January 1st 1990 starts, New York local time is still 5 hours behind.

Question 2: When (in absolute UTC time) was the beginning of the 1st day of January in 1990 in UTC time, when we express the date locally ?

(ZTimezone id: 'America/New_York') localToGmt: (ZTimestamp @ '1990/01/01').

  => "1990-01-01T05:00:00Z"

So when the New York day of January 1st 1990 starts, UTC time is already 5 hours ahead.

Note that the question 'When does January 1st 1990 start in any timezone, when expressed in that timezone, is of course a constant, midnight'.


I think that making Date always UTC will probably not help, because you will want to be able to move between timezones. I guess the only solution is to add a class like ZTimezone (which has no dependencies).


Sven

> -----
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
>



Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Sven Van Caekenberghe-2


> On 9 Apr 2018, at 17:34, Aliaksei Syrel <[hidden email]> wrote:
>
> Must watch :)
>
> The Problem with Time & Timezones - Computerphile
> https://www.youtube.com/watch?v=-5wpm-gesOY

Haha, that is indeed hilarious.

I guess there is no complete, absolute 100% correct answer for all use cases, we can only try to be reasonably good.

> On 9 April 2018 at 17:26, Sven Van Caekenberghe <[hidden email]> wrote:
>
>
> > On 9 Apr 2018, at 15:19, Sean P. DeNigris <[hidden email]> wrote:
> >
> > Max Leske wrote
> >> Assuming UTC is probably just as wrong as assuming the local time zone.
> >
> > I'll modify your statement slightly. "Assuming UTC is */almost/* as wrong as
> > assuming the local time zone."
> >
> > I've never seemed to be able to drive the essential point home when these
> > discussions have come up. Beyond all the wider issues, there is a bug, plain
> > and simple: The offset of `'1/1/1990' asDate`, considering that you mean
> > local time, is still not guaranteed to be the current local offset of the
> > image, which is how we set it by default. That only makes sense if the
> > historical date in question was in the same state of DST as the current
> > image. For example, the historical date above is in winter, so if I eval
> > that code in summer, it will /always/ give the objectively wrong offset.
> >
> > What I'm proposing is not a cure all, but a slightly-better way that fixes
> > this bug by giving users consistent behavior that they may not want instead
> > of inconsistent and often wrong behavior that they may not want.
>
> Sean,
>
> You are right, the current system cannot be fixed. It only knows about the current timezone's offset (via the OS), not about historical offsets. And it wrongly uses that offset because it does not know better.
>
> Neither
>
>   '1/1/1990' asDate.
>
> nor
>
>   Date today.
>
> can work without the context of a precise timezone. It is even relatively pointless to remember offsets without remembering timezones. You simply need a precise reference into the transitions database.
>
> New York is 5 hours behind UTC in winter.
>
> Question 1: When (in absolute UTC time) was the beginning of the 1st day of January in 1990 in New York's local time, when we express the date in UTC ?
>
> (ZTimezone id: 'America/New_York') gmtToLocal: (ZTimestamp @ '1990/01/01').
>
>   => "1989-12-31T19:00:00Z"
>
> So when the UTC day of January 1st 1990 starts, New York local time is still 5 hours behind.
>
> Question 2: When (in absolute UTC time) was the beginning of the 1st day of January in 1990 in UTC time, when we express the date locally ?
>
> (ZTimezone id: 'America/New_York') localToGmt: (ZTimestamp @ '1990/01/01').
>
>   => "1990-01-01T05:00:00Z"
>
> So when the New York day of January 1st 1990 starts, UTC time is already 5 hours ahead.
>
> Note that the question 'When does January 1st 1990 start in any timezone, when expressed in that timezone, is of course a constant, midnight'.
>
>
> I think that making Date always UTC will probably not help, because you will want to be able to move between timezones. I guess the only solution is to add a class like ZTimezone (which has no dependencies).
>
>
> Sven
>
> > -----
> > Cheers,
> > Sean
> > --
> > Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
> >
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Sean P. DeNigris
Administrator
In reply to this post by Sven Van Caekenberghe-2
Sven Van Caekenberghe-2 wrote
> I think that making Date always UTC will probably not help, because you
> will want to be able to move between timezones

Two things:
- I don't think changing timezones will be affected, because my proposal is
only: "default to 0 offset for new Date[AndTime] instances where no offset
is specified". This would not prevent changing the offset later on (although
one would make the case that since guessing default offset is unfixable in
practice in the core, it should be removed altogether), and it also wouldn't
affect the offset of the image, so `Date today` would still use the local
offset.
- I'm not saying that it is a panacea, but it will avoid the bug in
question. It is less wrong than the current behavior and will allow those of
us on DST to keep our sanity while we come up with a more perfect long term
solution.

What is the downside to this change?



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Sven Van Caekenberghe-2


> On 9 Apr 2018, at 18:52, Sean P. DeNigris <[hidden email]> wrote:
>
> Sven Van Caekenberghe-2 wrote
>> I think that making Date always UTC will probably not help, because you
>> will want to be able to move between timezones
>
> Two things:
> - I don't think changing timezones will be affected, because my proposal is
> only: "default to 0 offset for new Date[AndTime] instances where no offset
> is specified". This would not prevent changing the offset later on (although
> one would make the case that since guessing default offset is unfixable in
> practice in the core, it should be removed altogether), and it also wouldn't
> affect the offset of the image, so `Date today` would still use the local
> offset.
> - I'm not saying that it is a panacea, but it will avoid the bug in
> question. It is less wrong than the current behavior and will allow those of
> us on DST to keep our sanity while we come up with a more perfect long term
> solution.
>
> What is the downside to this change?

So you want to change

'1/1/1990' asDate.

so that it does the equivalent of

(DateAndTime fromString: '1990-01-01T00:00:00Z') asDate.

While you would not want to change

Date today.

so that it does the equivalent of

DateAndTime now asUTC asDate.

?

Is that the key change ? And why not the second ?

> -----
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
>


Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Sean P. DeNigris
Administrator
Sven Van Caekenberghe-2 wrote
> So you want to change '1/1/1990' asDate so that it does the equivalent of
> (DateAndTime fromString: '1990-01-01T00:00:00Z') asDate.

Precisely.


Sven Van Caekenberghe-2 wrote
> While you would not want to change Date today so that it does the
> equivalent of DateAndTime now asUTC asDate.

I guess it would make sense for consistency and thinking further,
`yesterday` and friends suffer from the same problems we've been discussing.



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Ron Teitelbaum


On Mon, Apr 9, 2018 at 1:46 PM, Sean P. DeNigris <[hidden email]> wrote:
Sven Van Caekenberghe-2 wrote
> So you want to change '1/1/1990' asDate so that it does the equivalent of
> (DateAndTime fromString: '1990-01-01T00:00:00Z') asDate.

Precisely.

That won't work :)
 
'1/1/1990' asDate = 
(DateAndTime fromString: '1990-01-01T00:00:00Z') asDate ->  true

(DateAndTime fromString: '1990-01-01T00:00:00Z') asDate start ->  1990-01-01T00:00:00-04:00

The date doesn't change converted to local timezone.  So you could solve your problem by comparing just the date part yyyymmdd.  

Note that 

Timespan >> = comparand
^ self class = comparand class 
and: [ self start = comparand start
and: [ self duration = comparand duration ] ]

Maybe a good solution is to modify the start check for the Date class?  I don't seem to have any real control over the start so having that be part of my check and fail seems wrong on this class.

Date >> starting: aDateAndTime

^super starting: (aDateAndTime midnight) duration: (Duration days: 1)

Suggest the fix is 

Date>> = comparand
^ self class = comparand class 
and: [ "Converts to current tz for comparision"
                          self start midnight = comparand start midnight  
and: [ self duration = comparand duration ] ]

or

Date>> = comparand
^ self class = comparand class 
and: [ "Compare only date part" 
                          self start yyyymmdd = comparand start yyyymmdd
and: [ self duration = comparand duration ] ]

Seems to match the intent of the class without changing the timezone.

All the best,

Ron Teitelbaum 


Sven Van Caekenberghe-2 wrote
> While you would not want to change Date today so that it does the
> equivalent of DateAndTime now asUTC asDate.

I guess it would make sense for consistency and thinking further,
`yesterday` and friends suffer from the same problems we've been discussing.

Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Sean P. DeNigris
Administrator
Ron Teitelbaum wrote
> (DateAndTime fromString: '1990-01-01T00:00:00Z') asDate start ->
> 1990-01-01T00:00:00-04:00

I don't understand. On Pharo 6.1, `(DateAndTime fromString:
'1990-01-01T00:00:00Z') asDate start. "-> 1990-01-01T00:00:00+00:00"`

The start DateAndTime instVar is exactly what the patch changes.



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

alistairgrant
In reply to this post by Sean P. DeNigris
Hi Sean,

On 9 April 2018 at 15:13, Sean P. DeNigris <[hidden email]> wrote:
> Alistair Grant wrote
>> I think using UTC would just move the problem by an hour.
>
> I'm not sure I understand. How so? I thought UTC is a constant reference
> point unaffected by UTC (https://stackoverflow.com/a/5495816/424245)

The problem as you described it was that comparing times from just
before midnight and just after midnight local time weren't equal.  I
haven't looked at the code or tested it, but I expect that if times
were stored in UTC, instead of the problem happening just before and
after midnight local time, it would happen just before and after
midnight UTC time.

I.e. by itself storing times as UTC wouldn't solve the problem.

As has been mentioned by all the participants in this discussion there is also:

- Date as a timespan vs. Date as a d/m/y
- DST historical changes.  This covers more than the start and end
dates, in Australia there is a city that changed time zones altogether
- it decided to move from Australian Central time to Australian
Eastern time (from memory).
- Named time zones vs Time zone offsets, e.g. is UTC + 2 hours CET in
DST or Russia Time Zone 1 out of DST?
- etc.

So I'm not convinced that using UTC would solve the problem you
originally reported.

Having said that, I think that always storing UTC is the right thing
to do.  Which timezone to display in is up to the UI, and just having
the offset by itself is insufficient - the timezone name is needed.

Cheers,
Alistair

Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Paul DeBruicker
In reply to this post by Sean P. DeNigris
I  think #= is a bad selector for Date and should be avoided when determining
whether something happens on a date, or whether two dates are the same.   We
all know March 24th in London covers a different 24 hours than March 24th in
Hawaii but Date>>#= does not.  



I think whats needed are more descriptive selectors like


Date>>#isSameOnDateAs: aDateOrDateAndTime
Date>>#overlapsWithDate: aDate
DateAndTime>>#occursOnDate: aDate
DateAndTime>>#sameHMSButDifferentUTCIn: aTimeZoneKey
DateAndTime>>#sameUTCButDifferentHMSIn: aTimeZoneKey

and change Date>>#= to #shouldNotImplement.


FWIW I also don't like #offset: as before you send it you know the timezone
and after you may let that knowledge be forgotten. Real offsets can change
as laws change.



I think people are aware of this but if you have need for comparing dates &
times then you must use a library that accesses the regularly updated Olson
timezone database on your system and classes that respect time zones.  Time
zones are political, and legal definitions of offsets can change hours
before the DST transition dates & times.  


I don't think it matters which default timezone you pick for the image if
you're not going to take them into account when doing comparisons.  


Unfortunately there isn't a way to avoid this complexity until DST goes
away.  


There's certainly flaws to how we currently do it and I think
TimeZoneDatabase and Chronos make good attempts to fix it.  I haven't looked
at Chalten but would guess its good too.  








Sean P. DeNigris wrote

> I was bitten by this very annoying bug again. As most of us probably know
> due
> to the steady stream of confused ML posts in the past, the bug in summary
> is
> that we have an incomplete timezone implementation that doesn't properly
> take into account historical DST changes. This flares up without warning
> especially when DST toggles. I created a wiki page to document the
> situation: https://github.com/seandenigris/pharo/wiki/Time-Zone-Fiasco
>
> Here's an example blowup: at 11:59pm before DST changes, eval aDate :=
> '1/1/1901' asDate. Now, wait two minutes and at 12:01am eval self assert:
> '1/1/1901' asDate = aDate and… whammo, an exception! The "different"
> offsets
> render equal dates unequal depending on when the objects were created.
>
> The more I think about it, the more I think that we should just assume UTC
> for all Date[AndTime]s that don't explicitly specify an offset, rather
> than
> pretend to set an offset which is only sometimes correct. More advanced
> users can use one of the available libraries to get full timezone support.
> What do you think?
>
>
>
> -----
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html





--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Ron Teitelbaum
In reply to this post by Sean P. DeNigris
Sorry for jumping in. I wasn't aware of a patch.  Changing tz on Date seems like a mistake but I haven't been paying close attention to the reasoning behind the change.  I defer.  

On Mon, Apr 9, 2018 at 2:20 PM, Sean P. DeNigris <[hidden email]> wrote:
Ron Teitelbaum wrote
> (DateAndTime fromString: '1990-01-01T00:00:00Z') asDate start ->
> 1990-01-01T00:00:00-04:00

I don't understand. On Pharo 6.1, `(DateAndTime fromString:
'1990-01-01T00:00:00Z') asDate start. "-> 1990-01-01T00:00:00+00:00"`

The start DateAndTime instVar is exactly what the patch changes.

Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Stephane Ducasse-3
In reply to this post by Sven Van Caekenberghe-2
Hi Sven

I'm totally stupid on this topic now I was retrospectively thinking
about the situation
already some years ago.
We tried to enhance the old core back when we were still in Squeak and now
I would do the opposite.

I would make sure that the core is only working for the pharo logic
(and use the minimal set of abstractions
covering only the minimum for pharo core) and provide a good library
for other cases.

What you think?

Alternatively we could have Date -> SimpleDate
and add ZTimezone.

Last time I played with Date and Span I got confused (It was for
automatically issuing bills each month) and I realised that
the abstractions we have are clunky or may be I did not use them. I
wanted to have a month in a year (for example to get
the 29 or 28 automatically and I had to do all kind of tricks.

Stef





On Mon, Apr 9, 2018 at 5:26 PM, Sven Van Caekenberghe <[hidden email]> wrote:

>
>
>> On 9 Apr 2018, at 15:19, Sean P. DeNigris <[hidden email]> wrote:
>>
>> Max Leske wrote
>>> Assuming UTC is probably just as wrong as assuming the local time zone.
>>
>> I'll modify your statement slightly. "Assuming UTC is */almost/* as wrong as
>> assuming the local time zone."
>>
>> I've never seemed to be able to drive the essential point home when these
>> discussions have come up. Beyond all the wider issues, there is a bug, plain
>> and simple: The offset of `'1/1/1990' asDate`, considering that you mean
>> local time, is still not guaranteed to be the current local offset of the
>> image, which is how we set it by default. That only makes sense if the
>> historical date in question was in the same state of DST as the current
>> image. For example, the historical date above is in winter, so if I eval
>> that code in summer, it will /always/ give the objectively wrong offset.
>>
>> What I'm proposing is not a cure all, but a slightly-better way that fixes
>> this bug by giving users consistent behavior that they may not want instead
>> of inconsistent and often wrong behavior that they may not want.
>
> Sean,
>
> You are right, the current system cannot be fixed. It only knows about the current timezone's offset (via the OS), not about historical offsets. And it wrongly uses that offset because it does not know better.
>
> Neither
>
>   '1/1/1990' asDate.
>
> nor
>
>   Date today.
>
> can work without the context of a precise timezone. It is even relatively pointless to remember offsets without remembering timezones. You simply need a precise reference into the transitions database.
>
> New York is 5 hours behind UTC in winter.
>
> Question 1: When (in absolute UTC time) was the beginning of the 1st day of January in 1990 in New York's local time, when we express the date in UTC ?
>
> (ZTimezone id: 'America/New_York') gmtToLocal: (ZTimestamp @ '1990/01/01').
>
>   => "1989-12-31T19:00:00Z"
>
> So when the UTC day of January 1st 1990 starts, New York local time is still 5 hours behind.
>
> Question 2: When (in absolute UTC time) was the beginning of the 1st day of January in 1990 in UTC time, when we express the date locally ?
>
> (ZTimezone id: 'America/New_York') localToGmt: (ZTimestamp @ '1990/01/01').
>
>   => "1990-01-01T05:00:00Z"
>
> So when the New York day of January 1st 1990 starts, UTC time is already 5 hours ahead.
>
> Note that the question 'When does January 1st 1990 start in any timezone, when expressed in that timezone, is of course a constant, midnight'.
>
>
> I think that making Date always UTC will probably not help, because you will want to be able to move between timezones. I guess the only solution is to add a class like ZTimezone (which has no dependencies).
>
>
> Sven
>
>> -----
>> Cheers,
>> Sean
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Stephane Ducasse-3
In reply to this post by Paul DeBruicker
Hi Paul

I agree and instead of patching the current system I would start using
TDD to design
a new Date package.

stef

On Mon, Apr 9, 2018 at 8:42 PM, Paul DeBruicker <[hidden email]> wrote:

> I  think #= is a bad selector for Date and should be avoided when determining
> whether something happens on a date, or whether two dates are the same.   We
> all know March 24th in London covers a different 24 hours than March 24th in
> Hawaii but Date>>#= does not.
>
>
>
> I think whats needed are more descriptive selectors like
>
>
> Date>>#isSameOnDateAs: aDateOrDateAndTime
> Date>>#overlapsWithDate: aDate
> DateAndTime>>#occursOnDate: aDate
> DateAndTime>>#sameHMSButDifferentUTCIn: aTimeZoneKey
> DateAndTime>>#sameUTCButDifferentHMSIn: aTimeZoneKey
>
> and change Date>>#= to #shouldNotImplement.
>
>
> FWIW I also don't like #offset: as before you send it you know the timezone
> and after you may let that knowledge be forgotten. Real offsets can change
> as laws change.
>
>
>
> I think people are aware of this but if you have need for comparing dates &
> times then you must use a library that accesses the regularly updated Olson
> timezone database on your system and classes that respect time zones.  Time
> zones are political, and legal definitions of offsets can change hours
> before the DST transition dates & times.
>
>
> I don't think it matters which default timezone you pick for the image if
> you're not going to take them into account when doing comparisons.
>
>
> Unfortunately there isn't a way to avoid this complexity until DST goes
> away.
>
>
> There's certainly flaws to how we currently do it and I think
> TimeZoneDatabase and Chronos make good attempts to fix it.  I haven't looked
> at Chalten but would guess its good too.
>
>
>
>
>
>
>
>
> Sean P. DeNigris wrote
>> I was bitten by this very annoying bug again. As most of us probably know
>> due
>> to the steady stream of confused ML posts in the past, the bug in summary
>> is
>> that we have an incomplete timezone implementation that doesn't properly
>> take into account historical DST changes. This flares up without warning
>> especially when DST toggles. I created a wiki page to document the
>> situation: https://github.com/seandenigris/pharo/wiki/Time-Zone-Fiasco
>>
>> Here's an example blowup: at 11:59pm before DST changes, eval aDate :=
>> '1/1/1901' asDate. Now, wait two minutes and at 12:01am eval self assert:
>> '1/1/1901' asDate = aDate and… whammo, an exception! The "different"
>> offsets
>> render equal dates unequal depending on when the objects were created.
>>
>> The more I think about it, the more I think that we should just assume UTC
>> for all Date[AndTime]s that don't explicitly specify an offset, rather
>> than
>> pretend to set an offset which is only sometimes correct. More advanced
>> users can use one of the available libraries to get full timezone support.
>> What do you think?
>>
>>
>>
>> -----
>> Cheers,
>> Sean
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
>
>
>
>
>
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
>

Reply | Threaded
Open this post in threaded view
|

Re: DateAndTime Offset Bug Proposal

Jaroslaw Podgajny
In reply to this post by Sean P. DeNigris
It has always been my intuition that Date (and Time) model has to contain the time zone the Date is supposed to represent, e.g. like in y/m/d+timezone. This way we would always have an object that precisely represents an absolute point in time. Very many problems, if not all, discussed here only arise when two instances of time magnitude are being compared. This seems to me much better addressed if taken into consideration at the time of comparison.
Some conventions would have to be introduced to create deterministic response for arithmetics but at least the exposure to the complex semantics and computational overhead go away until it cannot any longer be avoided: when ultimate point in time has to be converted in line with the current legal stance of the definitions. This also hints towards the requirement for having a “time of arithmetic operation” play a role in the comparison in order to allow programmatic reasoning about date/time arithmetics as taken in the past, e.g. before a particular regulation adjusted particular semantics.
In practical terms comparing a Date expressed in EST with a date expressed in Zulu could be resolved by first coercing the units both instances are expressed in, say to UTC, and then performing arithmetics. 
Such view also resonates with Sean’s observation below that offset of 0 seems reasonable...



Regards, Jaroslaw


On Mon, 9 Apr 2018 at 17:53, Sean P. DeNigris <[hidden email]> wrote:
Sven Van Caekenberghe-2 wrote
> I think that making Date always UTC will probably not help, because you
> will want to be able to move between timezones

Two things:
- I don't think changing timezones will be affected, because my proposal is
only: "default to 0 offset for new Date[AndTime] instances where no offset
is specified". This would not prevent changing the offset later on (although
one would make the case that since guessing default offset is unfixable in
practice in the core, it should be removed altogether), and it also wouldn't
affect the offset of the image, so `Date today` would still use the local
offset.
- I'm not saying that it is a panacea, but it will avoid the bug in
question. It is less wrong than the current behavior and will allow those of
us on DST to keep our sanity while we come up with a more perfect long term
solution.

What is the downside to this change?



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

12