[squeak-dev] Time zone help

I have an app that needs to track a local time and the different from
UTC.  If I do this:

    DateAndTime localTimeZone:
       (TimeZone offset: -5 hours name: 'EST5EDT' abbreviation: 'EST5EDT').

Then it is easy for me to make conversions to UTC using #asUTC.  The
problem is this offset needs to be adjusted twice a year.  I've looked
at the time zone plugin but it seems pretty heavyweight for something
that is /almost/ workable in the image.  Does anyone have a solution for
this?

I'm using Squeak3.9.1 if it makes a difference.

David

On Thu, May 28, 2009 at 11:27:17PM -0400, C. David Shaffer wrote:
Use TimeZoneDatabase. No plugin is required.
  http://wiki.squeak.org/squeak/1076

Latest source is always here:
  http://wiki.squeak.org/squeak/1076

Summary:
Complete implementation of Olson zoneinfo time zones for Smalltalk. The
time zone database may be loaded from compiled zoneinfo files, widely
available on unix and other operating systems. On Squeak, the time zones
are integrated with Chronology to permit zoneinfo time zones to be used
instead of the default TimeZone implementation.

Dave
 

David T. Lewis wrote:
>
> Use TimeZoneDatabase. No plugin is required.
>   http://wiki.squeak.org/squeak/1076
>
> Latest source is always here:
>   http://wiki.squeak.org/squeak/1076
>
>  
Thanks Dave.  I loaded the released version 1-2-5-dtl under Gentoo Linux
I end up in an endless stream of warnings "should be at end of stream"
during parsing of the time zone files.  Example stack trace attached.
Disabling the warning dialog gets me through to another warning "nil is
not a valid abbreviation for a time zone".  Stack trace attached.
Proceeding through that the load completes but I don't think my time
zones loaded properly:

  TimeZoneDatabase grepFor: 'Detroit'   ---> #()

I also tried New_York.  I verified that these files are in my
/usr/share/zoneinfo directory.  I seem to recall going through this
process before.  Is there a patch?

David

Oops, forgot to attach traces...sorry :-[

Warning 29 May 2009 11:58:10 am VM: unix - a SmalltalkImage Image: Squeak3.9.1 [latest update: #7075] SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir /home/shaffer/Source/Squeak/ProfitPortal Trusted Dir /home/shaffer/Source/Squeak/ProfitPortal/secure Untrusted Dir /home/shaffer/Source/Squeak/ProfitPortal/My Squeak Warning>>defaultAction Receiver: Warning: zoneinfo parse error, should be at end of file Arguments and temporary variables: Receiver's instance variables: messageText: 'zoneinfo parse error, should be at end of file' tag: nil signalContext: Warning(Exception)>>signal handlerContext: nil outerContext: nil UndefinedObject>>handleSignal: Receiver: nil Arguments and temporary variables: exception: Warning: zoneinfo parse error, should be at end of file Receiver's instance variables: nil MethodContext(ContextPart)>>handleSignal: Receiver: BlockContext>>on:do: Arguments and temporary variables: exception: Warning: zoneinfo parse error, should be at end of file val: nil Receiver's instance variables: sender: PasteUpMorph>>becomeActiveDuring: pc: 17 stackp: 3 method: a CompiledMethod (790) receiverMap: nil receiver: [] in PasteUpMorph>>becomeActiveDuring: {[aBlock value]} MethodContext(ContextPart)>>handleSignal: Receiver: BlockContext>>on:do: Arguments and temporary variables: exception: Warning: zoneinfo parse error, should be at end of file val: nil Receiver's instance variables: sender: [] in MultiByteBinaryOrTextStream(PositionableStream)>>fileInAnnouncing...etc... pc: 17 stackp: 3 method: a CompiledMethod (790) receiverMap: nil receiver: [] in MultiByteBinaryOrTextStream(PositionableStream)>>fileInAnnounci...etc... --- The full stack --- Warning>>defaultAction UndefinedObject>>handleSignal: MethodContext(ContextPart)>>handleSignal: MethodContext(ContextPart)>>handleSignal: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Warning(Exception)>>signal Warning(Exception)>>signal: Warning class(Exception class)>>signal: TzFileLoader(Object)>>notify: TzFileLoader>>load TimeZoneDatabase>>loadDataFrom:prefixPath:pathSeparator:condenseDuplicates: [] in TimeZoneDatabase class>>fromFiles:prefixPath:pathSeparator: {[:e |  idx := idx + 1.  bar value: idx.  db   loadDataFrom: e   prefixPath: ...]} OrderedCollection>>do: [] in TimeZoneDatabase class>>fromFiles:prefixPath:pathSeparator: {[:bar | aCollectionOfNames   do: [:e |     idx := idx + 1.    bar value: idx...]} [] in ProgressInitiationException>>defaultMorphicAction {[result := workBlock value: progress]} BlockContext>>ensure: ProgressInitiationException>>defaultMorphicAction ProgressInitiationException>>defaultAction UndefinedObject>>handleSignal: MethodContext(ContextPart)>>handleSignal: MethodContext(ContextPart)>>handleSignal: ProgressInitiationException(Exception)>>signal ProgressInitiationException>>display:at:from:to:during: ProgressInitiationException class>>display:at:from:to:during: ByteString(String)>>displayProgressAt:from:to:during: TimeZoneDatabase class>>showProgressFrom:to:withMessage:forBlock: TimeZoneDatabase class>>fromFiles:prefixPath:pathSeparator: TimeZoneDatabase class>>fromFiles:prefixPath: TimeZoneDatabase class>>fromZoneinfoDirectory: TimeZoneDatabase class>>buildSystemDatabase TimeZoneDatabase class>>buildSystemDatabaseForSqueak TimeZoneDatabase class>>initializeForSqueak TimeZoneDatabase class>>initialize UndefinedObject>>DoIt Compiler>>evaluate:in:to:notifying:ifFail:logged: Compiler class>>evaluate:for:notifying:logged: Compiler class>>evaluate:for:logged: Compiler class>>evaluate:logged: [] in MultiByteBinaryOrTextStream(PositionableStream)>>fileInAnnouncing: {[val := (self peekFor: $!)     ifTrue: [(Compiler evaluate: self nextChunk l...]} BlockContext>>on:do: [] in MultiByteBinaryOrTextStream(PositionableStream)>>fileInAnnouncing: {[:bar |  [self atEnd]   whileFalse: [bar value: self position.    self skipS...]} [] in ProgressInitiationException>>defaultMorphicAction {[result := workBlock value: progress]} BlockContext>>ensure: ProgressInitiationException>>defaultMorphicAction ProgressInitiationException>>defaultAction UndefinedObject>>handleSignal: MethodContext(ContextPart)>>handleSignal: ProgressInitiationException(Exception)>>signal ProgressInitiationException>>display:at:from:to:during: ...etc...
Warning 29 May 2009 11:58:42 am VM: unix - a SmalltalkImage Image: Squeak3.9.1 [latest update: #7075] SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir /home/shaffer/Source/Squeak/ProfitPortal Trusted Dir /home/shaffer/Source/Squeak/ProfitPortal/secure Untrusted Dir /home/shaffer/Source/Squeak/ProfitPortal/My Squeak Warning>>defaultAction Receiver: Warning: "nil" is not a valid abbreviation for this time zone Arguments and temporary variables: Receiver's instance variables: messageText: '"nil" is not a valid abbreviation for this time zone' tag: nil signalContext: Warning(Exception)>>signal handlerContext: nil outerContext: nil UndefinedObject>>handleSignal: Receiver: nil Arguments and temporary variables: exception: Warning: "nil" is not a valid abbreviation for this time zone Receiver's instance variables: nil MethodContext(ContextPart)>>handleSignal: Receiver: BlockContext>>on:do: Arguments and temporary variables: exception: Warning: "nil" is not a valid abbreviation for this time zone val: nil Receiver's instance variables: sender: PasteUpMorph>>becomeActiveDuring: pc: 17 stackp: 3 method: a CompiledMethod (790) receiverMap: nil receiver: [] in PasteUpMorph>>becomeActiveDuring: {[aBlock value]} MethodContext(ContextPart)>>handleSignal: Receiver: BlockContext>>on:do: Arguments and temporary variables: exception: Warning: "nil" is not a valid abbreviation for this time zone val: nil Receiver's instance variables: sender: [] in MultiByteBinaryOrTextStream(PositionableStream)>>fileInAnnouncing...etc... pc: 17 stackp: 3 method: a CompiledMethod (790) receiverMap: nil receiver: [] in MultiByteBinaryOrTextStream(PositionableStream)>>fileInAnnounci...etc... --- The full stack --- Warning>>defaultAction UndefinedObject>>handleSignal: MethodContext(ContextPart)>>handleSignal: MethodContext(ContextPart)>>handleSignal: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Warning(Exception)>>signal Warning(Exception)>>signal: Warning class(Exception class)>>signal: TimeZoneProxy(Object)>>notify: [] in TimeZoneProxy>>abbreviation: {[self notify: '"' , newAbbrev asString , '" is not a valid abbreviation for ...]} TimeZoneProxy>>abbreviation:ifNotValid: TimeZoneProxy>>abbreviation: [] in TimeZoneProxy>>update: {[self abbreviation: self transform abbreviation]} TimeZoneProxy>>abbreviation:ifNotValid: TimeZoneProxy>>update: [] in TimeZoneDatabase class(Object)>>changed: {[:aDependent | aDependent update: aParameter]} DependentsArray>>do: TimeZoneDatabase class(Object)>>changed: TimeZoneDatabase class>>thisSystemDatabase: TimeZoneDatabase class>>buildSystemDatabase TimeZoneDatabase class>>buildSystemDatabaseForSqueak TimeZoneDatabase class>>initializeForSqueak TimeZoneDatabase class>>initialize UndefinedObject>>DoIt Compiler>>evaluate:in:to:notifying:ifFail:logged: Compiler class>>evaluate:for:notifying:logged: Compiler class>>evaluate:for:logged: Compiler class>>evaluate:logged: [] in MultiByteBinaryOrTextStream(PositionableStream)>>fileInAnnouncing: {[val := (self peekFor: $!)     ifTrue: [(Compiler evaluate: self nextChunk l...]} BlockContext>>on:do: [] in MultiByteBinaryOrTextStream(PositionableStream)>>fileInAnnouncing: {[:bar |  [self atEnd]   whileFalse: [bar value: self position.    self skipS...]} [] in ProgressInitiationException>>defaultMorphicAction {[result := workBlock value: progress]} BlockContext>>ensure: ProgressInitiationException>>defaultMorphicAction ProgressInitiationException>>defaultAction UndefinedObject>>handleSignal: MethodContext(ContextPart)>>handleSignal: ProgressInitiationException(Exception)>>signal ProgressInitiationException>>display:at:from:to:during: ProgressInitiationException class>>display:at:from:to:during: ByteString(String)>>displayProgressAt:from:to:during: MultiByteBinaryOrTextStream(PositionableStream)>>fileInAnnouncing: [] in ChangeSet class>>newChangesFromStream:named: {[newSet   ifNotNil: [(aStream respondsTo: #converter:)     ifTrue: [newStrea...]} BlockContext>>ensure: ChangeSet class>>newChangesFromStream:named: ChangeSorter class>>newChangesFromStream:named: GZipReadStream class>>fileIntoNewChangeSet: SimpleServiceEntry>>performServiceFor: PluggableButtonMorph>>performAction [] in PluggableButtonMorph>>mouseUp: {[:m | (m containsPoint: evt cursorPoint)   ifTrue: [m performAction]]} ...etc...

Here's a zoneinfo file if that helps.

David

On Fri, May 29, 2009 at 12:01:41PM -0400, C. David Shaffer wrote: Hi David,

For reasons that I do not understand, the New_York zoneinfo file from
your Gentoo distribution is larger than I expected, and contains additional
binary data following the time zone table data. My file loader checks
for this, and throws up the warning that you are seeing. I don't know
if this is some Gentoo-specific thing, or if there is some alternate
file format that I am not aware of.

In any case, ignoring the additional data in the Gentoo zoneinfo files
appears to have no ill effects. I have attached a patch that disables
my syntax check, and that hopefully will enable you to load the zoneinfo
files on your Gentoo system when you evaluate "TimeZoneDatabase initialize".

Please let me know if this works for you.

Dave

David T. Lewis wrote: sure I downloaded the Olson database (Gentoo uses tzdata-2009c currenly,
although I see newer versions).  I downloaded tzcode-2009b, compiled the
code, used it to build a zoneinfo database and the New_York file
produced from this is exactly the same as the one I have in my system.  
What version of zic are you using?  Which data set?

With your patch and the attached patch which I had to add in order to
ignore some time zones that caused other problems.  I can't recall which
ones they were but I know I don't need them :-)

After that I did a:
    TimeZoneDatabase buildSystemDatabaseForSqueak

Running the tests gives 14 failures most with names that hint at DST
transition issues (testDateAndTimeAtDstTransition,
testDateAndTimeImmediatelyAfter...).  Attached is a stack dump for one
of the failed tests in case it provides a clue.

David


'From Squeak3.9.1 of 2 March 2008 [latest update: #7075] on 30 May 2009 at 11:41 am'!
!TimeZoneDatabase class methodsFor: 'instance creation' stamp: 'cds 5/29/2009 21:49'!
fromFiles: aCollectionOfNames prefixPath: aPathName pathSeparator: aCharacter
        "Build a full timezone database from a set of tzfile files. The path names have
        aPathName prepended in order to permit the names (without aPathName) to be
        used as the names of the TimeZoneRuleSets. Attempt to save space by avoiding
        duplicate entries for leap second rules and time zone offset rule sets.

        The path separator (aCharacter) is always replaced by self>>indexSeparator for
        purposes of naming the time zones in the database. This enforces a consistent
        time zone path name independent of the system platform."

        "TimeZoneDatabase fromFiles: TimeZoneDatabase exampleTzFiles prefixPath: TimeZoneDatabase tzExamplesPrefixPath pathSeparator: $- "

        | db idx |
        db := super new.
        idx := 0.
        self showProgressFrom: 1 to: aCollectionOfNames size withMessage: 'loading timezone data files'
                forBlock: [ :bar |
                        aCollectionOfNames do: [:e |
                                [idx := idx + 1.
                                bar value: idx.
                                db loadDataFrom: e
                                        prefixPath: aPathName
                                        pathSeparator: aCharacter
                                        condenseDuplicates: false] on: Error do: [:ex | ]]].
        ^ db normalize
! !

TestFailure: Assertion failed 30 May 2009 11:43:47 am VM: unix - a SmalltalkImage Image: Squeak3.9.1 [latest update: #7075] SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir /home/shaffer/Source/Squeak/ProfitPortal Trusted Dir /home/shaffer/Source/Squeak/ProfitPortal/secure Untrusted Dir /home/shaffer/Source/Squeak/ProfitPortal/My Squeak TimeTransformTest(TestCase)>>signalFailure: Receiver: TimeTransformTest>>#testAbsoluteTimeToDateAndTimeSeconds {testing-PointInTime seconds} Arguments and temporary variables: aString: 'Assertion failed' Receiver's instance variables: testSelector: #testAbsoluteTimeToDateAndTimeSeconds saveLocalTimeZone: a TimeZone(EDT) saveLocalTransform: a TimeZoneRuleSet for UTC saveLastTransitionTime: nil dstTransitionAbsoluteSeconds: 1099202422 dstTransitionPosixSeconds: 1099202400 leapSeconds: 22 TimeTransformTest(TestCase)>>assert: Receiver: TimeTransformTest>>#testAbsoluteTimeToDateAndTimeSeconds {testing-PointInTime seconds} Arguments and temporary variables: aBooleanOrBlock: false Receiver's instance variables: testSelector: #testAbsoluteTimeToDateAndTimeSeconds saveLocalTimeZone: a TimeZone(EDT) saveLocalTransform: a TimeZoneRuleSet for UTC saveLastTransitionTime: nil dstTransitionAbsoluteSeconds: 1099202422 dstTransitionPosixSeconds: 1099202400 leapSeconds: 22 TimeTransformTest>>testAbsoluteTimeToDateAndTimeSeconds Receiver: TimeTransformTest>>#testAbsoluteTimeToDateAndTimeSeconds {testing-PointInTime seconds} Arguments and temporary variables: interval: (1099202420 to: 1099202423) expectedLocalSmalltalkSeconds: #(3276640798 3276640799 3276637200 3276637201) dateAndTimes: an Array(2004-10-31T01:59:58-04:00 2004-10-31T01:59:59-04:00 2004...etc... dateAndTimeSeconds: #(3276640798 3276640799 3276640800 3276640801) abs: 1099202423 dt: 2004-10-31T01:00:01-05:00 Receiver's instance variables: testSelector: #testAbsoluteTimeToDateAndTimeSeconds saveLocalTimeZone: a TimeZone(EDT) saveLocalTransform: a TimeZoneRuleSet for UTC saveLastTransitionTime: nil dstTransitionAbsoluteSeconds: 1099202422 dstTransitionPosixSeconds: 1099202400 leapSeconds: 22 TimeTransformTest(TestCase)>>performTest Receiver: TimeTransformTest>>#testAbsoluteTimeToDateAndTimeSeconds {testing-PointInTime seconds} Arguments and temporary variables: Receiver's instance variables: testSelector: #testAbsoluteTimeToDateAndTimeSeconds saveLocalTimeZone: a TimeZone(EDT) saveLocalTransform: a TimeZoneRuleSet for UTC saveLastTransitionTime: nil dstTransitionAbsoluteSeconds: 1099202422 dstTransitionPosixSeconds: 1099202400 leapSeconds: 22 --- The full stack --- TimeTransformTest(TestCase)>>signalFailure: TimeTransformTest(TestCase)>>assert: TimeTransformTest>>testAbsoluteTimeToDateAndTimeSeconds TimeTransformTest(TestCase)>>performTest - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - [] in TimeTransformTest(TestCase)>>runCase {[self setUp.  self performTest]} BlockContext>>ensure: [] in TimeTransformTest(TestCase)>>runCase {[[self setUp.  self performTest]   ensure: [self tearDown]]} TimeTransformTest(TestCase)>>runCaseWith: TimeTransformTest(TestCase)>>runCase [] in TimeTransformTest(TestCase)>>debug {[(self class selector: testSelector) runCase]} BlockContext>>on:do: [] in TimeTransformTest(TestCase)>>debug {[[(self class selector: testSelector) runCase]   on: Error   do: [:ex | Unha...]} BlockContext>>ensure: TimeTransformTest(TestCase)>>debug [] in TestRunner>>debugSuite: {[:each | each debug]} Set>>do: [] in TestRunner>>basicRunSuite:do: {[aTestSuite name isEmptyOrNil   ifTrue: [aTestSuite tests do: aBlock]   ifFa...]} BlockContext>>ensure: TestRunner>>basicRunSuite:do: TestRunner>>debugSuite: TestRunner>>debug: TestRunner>>failedSelected: PluggableListMorphPlus(PluggableListMorph)>>changeModelSelection: PluggableListMorphPlus(PluggableListMorph)>>mouseUp: PluggableListMorphPlus(Morph)>>handleMouseUp: MouseButtonEvent>>sentTo: PluggableListMorphPlus(Morph)>>handleEvent: PluggableListMorphPlus(Morph)>>handleFocusEvent: [] in HandMorph>>sendFocusEvent:to:clear: {[ActiveHand := self.  ActiveEvent := anEvent.  e := anEvent     transformedB...]} [] in PasteUpMorph>>becomeActiveDuring: {[aBlock value]} BlockContext>>on:do: PasteUpMorph>>becomeActiveDuring: ...etc...

On Sat, May 30, 2009 at 11:48:14AM -0400, C. David Shaffer wrote:
Right you are. It looks like there is now a new version of the tzfile format.
I'm using a SuSE Linux system and a FreeBSD system, neither of which has
this new version. According to the new tzfile man page, the new file format
is backward compatible, and the patch that I sent to you should be safe
to use.

I'll look into this more carefully when I get some time, and update the
TimeZoneDatabase package as needed. Note, I wrote this back in 1999, and
the tzfile parser has not needed any attention until now. Thanks for
catching this and letting me know about the issue.

> With your patch and the attached patch which I had to add in order to
> ignore some time zones that caused other problems.  I can't recall which
> ones they were but I know I don't need them :-)

Thanks, I'll look into that also.

> After that I did a:
>    TimeZoneDatabase buildSystemDatabaseForSqueak
>
> Running the tests gives 14 failures most with names that hint at DST
> transition issues (testDateAndTimeAtDstTransition,
> testDateAndTimeImmediatelyAfter...).  Attached is a stack dump for one
> of the failed tests in case it provides a clue.

I'm seeing the same thing on my system. I believe that the errors are in
my test code, which looks for DST transitions at certain times and dates.
The Michigan legislature changed the law about when daylight savings time
is in effect here, so my tests may be out of date. I'll check this also
when I get time.

Thanks for your help and feedback,
Dave

David T. Lewis wrote:
>
> I'm seeing the same thing on my system. I believe that the errors are in
> my test code, which looks for DST transitions at certain times and dates.
> The Michigan legislature changed the law about when daylight savings time
> is in effect here, so my tests may be out of date. I'll check this also
> when I get time.
>
>  
That's reassuring.  I was concerned that it was the code itself.  I
guess I'll know if it's working on Nov. 1 :-)

Thanks for all of your help.

David

On Sat, May 30, 2009 at 03:40:15PM -0400, C. David Shaffer wrote:
If you are watching this closely on Nov 1, then there is one more thing
I'll mention just FYI. The class TimeZoneDstTransitionWatcher maintains
a process that watches for DST transitions and makes sure Squeak
handles them correctly. Details are in the class comment, but I mention
it because the watcher process must be kept running. In other words, if
you notice that process in your process browser, don't kill it ;-)

Dave

On Sat, May 30, 2009 at 11:48:14AM -0400, C. David Shaffer wrote:
> >  
> Thanks for the patch.  It is definitely not Gentoo-specific.  To make
> sure I downloaded the Olson database (Gentoo uses tzdata-2009c currenly,
> although I see newer versions).  I downloaded tzcode-2009b, compiled the
> code, used it to build a zoneinfo database and the New_York file
> produced from this is exactly the same as the one I have in my system.  
> What version of zic are you using?  Which data set?

I have added support for the new version 2 tzinfo files that are distributed
with your Gentoo system. The update is on SqueakSource at
http://www.squeaksource.com/TimeZoneDatabase/

    Name: Time-dtl.28
    Author: dtl
    Time: 31 May 2009, 1:50:19 pm
    UUID: dcd6f516-8303-4d0d-a842-4d34ec8398f4
    Ancestors: Time-dtl.27
   
    TimeZoneDatabase version 1.3
   
    Add support for version 2 tzinfo files. A TimeZoneRuleSet created from a
    version 2 tzinfo file has time values supplied from 64 bit data fields,
    and has a posixTzEnvString instance variable intended for use in handling
    instants after the last transition time stored in the tzinfo file (provided
    here for informational purposes only).

> With your patch and the attached patch which I had to add in order to
> ignore some time zones that caused other problems.  I can't recall which
> ones they were but I know I don't need them :-)

Hopefully all of your system tzinfo files will load properly now. If you can
confirm this on your Gentoo system, I'd appreciate it.

Note, I have not yet updated the broken sunit tests discussed previously.

- Dave