Smalltalk › Gemtalk › GLASS

DateAndTime ... all pretty strange

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

8 messages Options

GLASS mailing list

DateAndTime ... all pretty strange

If I execute "DateAndTime now printString" I get (e.g.) '2017-01-11T20:37:32.28995895385742+01:00', which looks pretty ok. It's a little bit strange about the many digits in the fractional part of seconds - but it seems, that this is allowed.

If you give this string to e.g. Python it will throw an exception (e.g. via json) - due to the many digits. For this purposes you have to cut the number of digits to about 3 to 5 - then its work.

Now try the reverse with "(DateAndTime fromString: '2017-01-11T20:37:32.28995895385742+01:00') printString" and you get: "'2017-01-11T20:37:32+06:00'". That feels very strange ......

Some other tests:

(DateAndTime fromString: '2017-01-11T20:37:32.289+01:00') printString -> '2017-01-11T20:37:32+01:00'

(DateAndTime fromString: '2017-01-11T20:37:32.28995+01:00') printString -> '2017-01-11T20:37:32+03:00'

For me this seems to be badly broken (Gemstone/S 3.3.3) ... any other solution in Gemstone to parse more difficult DateTime stuff ???

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

GLASS mailing list

Re: DateAndTime ... all pretty strange

Ok, because I only needed RFC3339 support I wrote three little methods:

From a DateAndTime instance to a string:

DateAndTimeANSI>>asRFC3339String
   "
       (DateAndTime fromRFC3339String: '2017-01-17T12:01:02Z') asRFC3339String   '2017-01-17T12:01:02Z'
       (DateAndTime fromRFC3339String: '2017-01-17T12:01:02+01:00') asRFC3339String '2017-01-17T12:01:02+01:00'
       (DateAndTime fromRFC3339String: '2017-01-17T12:01:02-01:00') asRFC3339String   '2017-01-17T12:01:02-01:00'

       (DateAndTime fromRFC3339String: '2017-01-17T12:01:02.123Z') asRFC3339String '2017-01-17T12:01:02.123Z'

   "
   | writeStream tmp truncatedValue |

   writeStream := WriteStream on: String new.

   self printYMDOn: writeStream withLeadingSpace: false.
   writeStream nextPut: $T.
   (tmp := self hour24 abs) < 10 ifTrue: [writeStream nextPut: $0].
   tmp printOn: writeStream.
   writeStream nextPut: $:.
   (tmp := self minute abs) < 10 ifTrue: [writeStream nextPut: $0].
   tmp printOn: writeStream.
   writeStream nextPut: $:.
   (tmp := self second abs truncated) < 10 ifTrue: [writeStream nextPut: $0].
   tmp printOn: writeStream.
   truncatedValue := ((self second - self second truncated) * 1000) truncated.
   (truncatedValue > 0)
       ifTrue: [
           writeStream nextPut: $..
           truncatedValue printOn: writeStream
       ].

   self offset isZero
       ifTrue:[   writeStream nextPut: $Z ]
       ifFalse:[    self offset printOnHoursMinutes: writeStream    ].

   ^writeStream contents

From a string to a DateAndTime object:

DateAndTimeANSI class>>fromRFC3339String: aString
   "YYYY-MM-DDTHH:MM:SS.ssss+/-0000

       DateAndTime fromRFC3339String: '2017-01-17T12:01:02Z'
       DateAndTime fromRFC3339String: '2017-01-17T12:01:02+01:00'
       DateAndTime fromRFC3339String: '2017-01-17T12:01:02-01:00'

       DateAndTime fromRFC3339String: '2017-01-17T12:01:02.123Z'
       (DateAndTime fromRFC3339String: '2017-01-17T12:01:02.123Z') seconds

       Duration days: 0 hours: 1 minutes: 1 seconds: 0

   "

   | year month dayOfMonth hours minute seconds offset readStream plusMinusCharacter |

   readStream := ReadStream on: aString.

   year := self _mskNumberWithDigits: 4 from: readStream.
   (readStream next ~= $-) ifTrue:[ self _errIncorrectFormat: aString ].
   month := self _mskNumberWithDigits: 2 from: readStream.
   (readStream next ~= $-) ifTrue:[ self _errIncorrectFormat: aString ].
   dayOfMonth := self _mskNumberWithDigits: 2 from: readStream.

   (readStream next ~= $T) ifTrue:[ self _errIncorrectFormat: aString ].

   hours := self _mskNumberWithDigits: 2 from: readStream.
   (readStream next ~= $:) ifTrue:[ self _errIncorrectFormat: aString ].
   minute := self _mskNumberWithDigits: 2 from: readStream.
   (readStream next ~= $:) ifTrue:[ self _errIncorrectFormat: aString ].
   seconds := self _mskNumberWithDigits: 2 from: readStream.

   "fractionall part"
   readStream peek = $.
       ifTrue:[
           readStream next.
           seconds := seconds + (self _mskNumberUpToNonDigitFrom: readStream).
       ].

   "get the +/- character or Z character"
   plusMinusCharacter := readStream next.
   plusMinusCharacter = $Z
       ifTrue:[ offset := Duration zero ]
       ifFalse:[
           | hOffset mOffset |
           hOffset := self _mskNumberWithDigits: 2 from: readStream.
           (readStream next ~= $:) ifTrue:[ self _errIncorrectFormat: aString ].
           mOffset := self _mskNumberWithDigits: 2 from: readStream.
           offset := Duration days: 0 hours: hOffset minutes: mOffset seconds: 0.
           plusMinusCharacter = $- ifTrue:[ offset := offset negated ]
       ].

   ^self year: year month: month day: dayOfMonth hour: hours minute: minute second: seconds offset: offset

And the last method needed two little helpers:

DateAndTimeANSI class>>_mskNumberWithDigits: numberOfDigits from: aStream
   "
           DateAndTime _mskNumberWithDigits: 4 from: (ReadStream on: '1234')
   "

   | number factor digitString |

   number := 0.
   factor := 1.

   digitString := aStream next: numberOfDigits.

   numberOfDigits to: 1 by: -1 do: [ :anIndex |
       | character |
       (character := digitString at: anIndex) isDigit
           ifFalse:[].

       number := number + (character numericValue * factor).
       factor := factor * 10.
   ].

   ^number

and

DateAndTimeANSI class>>_mskNumberUpToNonDigitFrom: readStream " DateAndTime _mskNumberUpToNonDigitFrom: (ReadStream on: '123456') " | numberString | numberString := WriteStream on: String new. numberString nextPut: $0 ; nextPut: $. . [ readStream atEnd not and:[ readStream peek isDigit ]] whileTrue:[ numberString nextPut: readStream next ]. ^(Float fromStringLocaleC: numberString contents)

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Richard Sargent

Re: DateAndTime ... all pretty strange

Administrator

In reply to this post by GLASS mailing list

GLASS mailing list wrote

<!DOCTYPE html>

If I execute "DateAndTime now printString" I get (e.g.) '2017-01-11T20:37:32.28995895385742+01:00', which looks pretty ok. It's a little bit strange about the many digits in the fractional part of seconds - but it seems, that this is allowed. If you give this string to e.g. Python it will throw an exception (e.g. via json) - due to the many digits. For this purposes you have to cut the number of digits to about 3 to 5 - then its work. Now try the reverse with "(DateAndTime fromString: '2017-01-11T20:37:32.28995895385742+01:00') printString" and you get: "'2017-01-11T20:37:32+06:00'". That feels very strange ...... Some other tests: (DateAndTime fromString: '2017-01-11T20:37:32.289+01:00') printString -> '2017-01-11T20:37:32+01:00' (DateAndTime fromString: '2017-01-11T20:37:32.28995+01:00') printString -> '2017-01-11T20:37:32+03:00' For me this seems to be badly broken (Gemstone/S 3.3.3) ... any other solution in Gemstone to parse more difficult DateTime stuff ???

Using a virgin 3.3.3 database, I get the following:
topaz 1> printit
{
(DateAndTime fromString: '2017-01-11T20:37:32.289+01:00') printString.
(DateAndTime fromString: '2017-01-11T20:37:32.28995895385742+01:00') printString.
(DateAndTime fromString: '2017-01-11T20:37:32.289+01:00') printString.
(DateAndTime fromString: '2017-01-11T20:37:32.28995+01:00') printString
}
%
a Array
#1 2017-01-11T20:37:32.28899997472763+01:00
#2 2017-01-11T20:37:32.28995895385742+01:00
#3 2017-01-11T20:37:32.28899997472763+01:00
#4 2017-01-11T20:37:32.28995001316071+01:00
topaz 1>

Using a 3.3.1 database with Seaside and all loaded, I get this:
topaz 1> printit
{
(DateAndTime fromString: '2017-01-11T20:37:32.289+01:00') printString.
(DateAndTime fromString: '2017-01-11T20:37:32.28995895385742+01:00') printString.
(DateAndTime fromString: '2017-01-11T20:37:32.289+01:00') printString.
(DateAndTime fromString: '2017-01-11T20:37:32.28995+01:00') printString
}
%
a Array
#1 2017-01-11T20:37:32.289+01:00
#2 2017-01-11T20:37:32.28995895385742+01:00
#3 2017-01-11T20:37:32.289+01:00
#4 2017-01-11T20:37:32.28995+01:00
topaz 1>

I do not see the wildly wandering offsets; I do see a difference in how the precision is parsed.

Can you provide more information about your configuration?

GLASS mailing list

Re: DateAndTime ... all pretty strange

In reply to this post by GLASS mailing list

Marten,

I agree that what you are seeing is "pretty strange", but I am not able to reproduce your results.

The #fromString implementation uses a ScaledDecimal instance for the express purpose of preserving the number decimal points so that the #printString and #fromString: results match ...

If I run as SystemUser, where the #fromString: does not use a ScaledDecimal, the number of decimal places is not preserved, but in my case:

topaz 1> run
(DateAndTime fromString: '2017-01-11T20:37:32.28995+01:00') printString
%
2017-01-11T20:37:32.28995001316071+01:00
and

topaz 1> run
(DateAndTime fromString: '2017-01-11T20:37:32.28995+01:00') printString
%
2017-01-11T20:37:32.28995001316071+01:00

But these results are consistent with using converting the seconds to a Float ....

So I don't see the truncation of seconds that you see with this expression:

(DateAndTime fromString: '2017-01-11T20:37:32.289+01:00') printString -> '2017-01-11T20:37:32+01:00'

and I don't see truncation of seconds plus time zone skew in these results:

(DateAndTime fromString: '2017-01-11T20:37:32.28995+01:00') printString -> '2017-01-11T20:37:32+03:00'

So I'm not exactly sure how to proceed ...

If you were seeing consistent time zone skew, I would suspect that the GemStone stone's timezone and the timezone of your machine might disagree (by default GemStone assumes PST/PDT), but given the fact that you seem to be getting random generation of the UTC offset it is hard to come up with a rational explanation ...

... Just a second, If I use #printStringRoundedSeconds, I do see results similar to what you are reporting:

(DateAndTime fromString: '2017-01-11T20:37:32.289+01:00') printStringWithRoundedSeconds -> '2017-01-11T20:37:32+01:00'

but again, I am not seeing inconsistent results ... #printStringWithRoundedSeconds always rounds ... and I can't explain the random transformation of the UTC offset.

Dale

On 1/11/17 11:54 AM, Marten Feldtmann via Glass wrote:

If I execute "DateAndTime now printString" I get (e.g.) '2017-01-11T20:37:32.28995895385742+01:00', which looks pretty ok. It's a little bit strange about the many digits in the fractional part of seconds - but it seems, that this is allowed.

If you give this string to e.g. Python it will throw an exception (e.g. via json) - due to the many digits. For this purposes you have to cut the number of digits to about 3 to 5 - then its work.

Now try the reverse with "(DateAndTime fromString: '2017-01-11T20:37:32.28995895385742+01:00') printString" and you get: "'2017-01-11T20:37:32+06:00'". That feels very strange ......

Some other tests:

(DateAndTime fromString: '2017-01-11T20:37:32.289+01:00') printString -> '2017-01-11T20:37:32+01:00'

(DateAndTime fromString: '2017-01-11T20:37:32.28995+01:00') printString -> '2017-01-11T20:37:32+03:00'

For me this seems to be badly broken (Gemstone/S 3.3.3) ... any other solution in Gemstone to parse more difficult DateTime stuff ???
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

GLASS mailing list

Re: DateAndTime ... all pretty strange

I have the answer for this strange behaviour - all the method comes from Squeak and they are Locale oriented - and my decimalPoint is $, and not $. - this of course is a pretty critical stuff. It works for all the US persons, but not for us here in Germany.

Marten

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

GLASS mailing list

Re: DateAndTime ... all pretty strange

Okay, so when you said:

"DateAndTime now printString" I get (e.g.) '2017-01-11T20:37:32.28995895385742+01:00' which looks pretty ok.

It really doesn't "look okay" because the printString should have been displayed as:

'2017-01-11T20:37:32,28995895385742+01:00'

using a $, instead of a $. ... is that what you are saying? Or perhaps the format using $. is correct because it conforms to the spec?

I see that a year ago there was an attempt to address a Locale-based issue[1], and it might be that the fix was not quite correct as I think that a $. is still being used here[2]:

[ aStream atEnd | (ch = $.) | (ch = $+) | (ch = $-) ]

and perhaps that line should look like this:

[ aStream atEnd | (ch = decimalPointChr) | (ch = $+) | (ch = $-) ]

Could you give this patch a try and see if it fixes your problem?

Dale

[1] https://github.com/GsDevKit/GsDevKit/issues/82
[2] https://github.com/glassdb/glass/commit/ed2a944e7eb5095f633c4a2d2c06ad68a7c20211#diff-b6b7c7433911f46cc6441cd69d260015R25

On 1/11/17 3:30 PM, Marten Feldtmann wrote:

I have the answer for this strange behaviour - all the method comes from Squeak and they are Locale oriented - and my decimalPoint is $, and not $. - this of course is a pretty critical stuff. It works for all the US persons, but not for us here in Germany.

Marten

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

GLASS mailing list

Re: DateAndTime ... all pretty strange

Well, printString and fromString introduce some kind of a date/time representation. Whatever this is, the output should be useable via the parsing method.

If I use printString it looks like ISO8601 and/or RFC3339 - but the corresponding method "fromString" introduces locales and this breaks the whole stuff. And I do not find any "," in the syntax specification of ISO8601 and RFC3339.

Actually I would NOT like to see a locale introduced there. I prefere method with names showing me, that locales are used by that method (e.g asLocaleC) - then one knows, what to expect.

And I also do not see a need for that large number of digits in the fractional part of the seconds - as I said it may results in problems with other languages (though it is allowed to have so many digits) and the resolution shown here seems to be not true - there seems to be the natural single precision/double conversion precision problem.

But these are only my personal ideas

Marten

Dale Henrichs <[hidden email]> hat am 12. Januar 2017 um 00:56 geschrieben:

Okay, so when you said:
"DateAndTime now printString" I get (e.g.) '2017-01-11T20:37:32.28995895385742+01:00' which looks pretty ok.
It really doesn't "look okay" because the printString should have been displayed as:
'2017-01-11T20:37:32,28995895385742+01:00'
using a $, instead of a $. ... is that what you are saying? Or perhaps the format using $. is correct because it conforms to the spec?

I see that a year ago there was an attempt to address a Locale-based issue[1], and it might be that the fix was not quite correct as I think that a $. is still being used here[2]:

[ aStream atEnd | (ch = $.) | (ch = $+) | (ch = $-) ]

and perhaps that line should look like this:

[ aStream atEnd | (ch = decimalPointChr) | (ch = $+) | (ch = $-) ]

Could you give this patch a try and see if it fixes your problem?

Dale

[1] https://github.com/GsDevKit/GsDevKit/issues/82
[2] https://github.com/glassdb/glass/commit/ed2a944e7eb5095f633c4a2d2c06ad68a7c20211#diff-b6b7c7433911f46cc6441cd69d260015R25

On 1/11/17 3:30 PM, Marten Feldtmann wrote:
I have the answer for this strange behaviour - all the method comes from Squeak and they are Locale oriented - and my decimalPoint is $, and not $. - this of course is a pretty critical stuff. It works for all the US persons, but not for us here in Germany.

Marten

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

GLASS mailing list

Re: DateAndTime ... all pretty strange

On 1/11/17 10:32 PM, Marten Feldtmann wrote:
>
> Well, printString and fromString introduce some kind of a date/time
> representation. Whatever this is, the output should be useable via the
> parsing method.
>
I am not sure what you mean by this ... if the output of printString for
DateAndTime cannot be used to produce the same object suing #fromString:
then I think that it is a bug ...

And the results that you were seeing indicates that there was a bug and
I gave you a patch to try.

Did you try the patch and if you did, I would appreciate it if you let
me know ... if you don't let me know, I will not publish the patch ...
>
> If I use printString it looks like ISO8601 and/or RFC3339 - but the
> corresponding method "fromString" introduces locales and this breaks
> the whole stuff. And I do not find any "," in the syntax specification
> of ISO8601 and RFC3339.
>
I guess I'll see if the ANSI standard description describes which format
they are using, but your guess is probably right and it is good to know
that a Locale-specific decimal point is not supposed to be used here.
>
> Actually I would NOT like to see a locale introduced there. I prefere
> method with names showing me, that locales are used by that method
> (e.g asLocaleC) - then one knows, what to expect.
>
That makes sense ... in this particular case the Locale is used by the
printString for ScaledDecimal and the printString code for DateAndTime
was parsing the printString of the ScaledDecimal expecting a $. instead
of a Locale-specific decimal point.

Again, I am hoping that you let me know whether or not my proposed patch
actually fixes the problem.
>
> And I also do not see a need for that large number of digits in the
> fractional part of the seconds - as I said it may results in problems
> with other languages (though it is allowed to have so many digits) and
> the resolution shown here seems to be not true - there seems to be the
> natural single precision/double conversion precision problem.
>
By default, DateAndTime class>>now uses System class>>_timeGmtFloat
which returns a Float with up to microsecond resolution.

If you use one of the other DateAndTime factory methods:

year:day:hour:minute:second:
year:day:hour:minute:second:offset:
year:month:day:hour:minute:second:
year:month:day:hour:minute:second:offset:

The object in the second argument will be used to control how many
decimal points are displayed by the printString method.

If you are content with 1 second resolution you can use
#printStringWithRoundedSeconds or printRoundedOn:. Or you can override
DateAndTimeANSI class>>now and/or send #beRounded to the DateAndTime
instance before using it ...

If you want to have a specific number of decimal points of resolution
for seconds, then the best bet is to use a ScaledDecimal for seconds by
adding and using a #beScaledDecimal: and/or override DateAndTimeANSI
class>>now.

Again, I am hoping that you let me know whether or not my proposed patch
actually fixes your original problem.

Dale
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass