Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

Wayne Johnston
I want to get a file's timestamp in UTC/GMT.  The problem relates Daylight Savings Time (DST, which is like British Summer Time).  I am on Windows 7 or Windows 2008 R2 with NTFS.

Say I have a file which was modified in the summer at 5pm DST.  And I am in the Eastern time zone of the US which is 5 hours behind UTC during the winter, and 4 hours behind during DST.  So that file modified at 5pm Eastern DST is equivalent to 9pm UTC.  During the summer, CfsStat>>#stat: gives me an stMtime of 5pm.  That is obtained via #fileModifiedDateAndTime which uses FileTimeToLocalFileTime and FileTimeToSystemTime.

However, now in the winter, CfsStat>>#stat: gives me an stMtime of 4pm.  And when I convert that to UTC via tzSpecificLocalTimeToSystemTime:lpUniversalTime: (TzSpecificLocalTimeToSystemTime), that yields 8pm.  That isn't right.  You could argue that the 4pm is correct if DST wasn't in effect (if we hadn't "sprung ahead" an hour) - that is, 4pm is a DST-unaware answer.  But TzSpecificLocalTimeToSystemTime is DST-aware, knowing of the 4-hour difference between Eastern time and UTC in the summer.  So the problem is mixing these DST-unaware and DST-aware APIs.

I really think stMtime should answer 5pm on the file no matter what day of the year it is now.  Is it a bug?  Or do I just need to live with the APIs and do my own conversion ("stMtime answered 4pm, but since DST was in effect then and isn't now, add an hour")?

Actually what would be better is for stMtime to answer UTC.  This would do it (instead of the current VA code which uses FileTimeToLocalFileTime and FileTimeToSystemTime, this just uses FileTimeToSystemTime).

fileModifiedDateAndTime
"Answer an Array of size two containing the date and time the file was last modified."

  | st |

##PR = ##'1ET6RFP'.

  st := CfsOSSystemtime new.
  (self ftLastWriteTime fileTimeToSystemTime: st)
   ifTrue: [
  ^Array
 with: (Date new year: st wYear month: st wMonth day: st wDay)
 with: (Time new hours: st wHour
 minutes: st wMinute
 seconds: st wSecond)].
  ^Array with: (Date fromDays: 0) with: (Time fromSeconds: 0)  

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

dmacq
Hi Wayne,

On Tuesday, January 7, 2014 4:48:45 AM UTC-5, Wayne Johnston wrote:
I want to get a file's timestamp in UTC/GMT.  The problem relates Daylight Savings Time (DST, which is like British Summer Time).  I am on Windows 7 or Windows 2008 R2 with NTFS.

Say I have a file which was modified in the summer at 5pm DST.  And I am in the Eastern time zone of the US which is 5 hours behind UTC during the winter, and 4 hours behind during DST.  So that file modified at 5pm Eastern DST is equivalent to 9pm UTC.  

I disagree here. Eastern DST is UTC-4:00.
 
During the summer, CfsStat>>#stat: gives me an stMtime of 5pm.  That is obtained via #fileModifiedDateAndTime which uses FileTimeToLocalFileTime and FileTimeToSystemTime.

However, now in the winter, CfsStat>>#stat: gives me an stMtime of 4pm.  And when I convert that to UTC via tzSpecificLocalTimeToSystemTime:lpUniversalTime: (TzSpecificLocalTimeToSystemTime), that yields 8pm.  That isn't right.  You could argue that the 4pm is correct if DST wasn't in effect (if we hadn't "sprung ahead" an hour) - that is, 4pm is a DST-unaware answer.  But TzSpecificLocalTimeToSystemTime is DST-aware, knowing of the 4-hour difference between Eastern time and UTC in the summer.  

This seems backwards to me. Summer should be -4 and winter should be -5. 
 
So the problem is mixing these DST-unaware and DST-aware APIs.

Probably. I will check into it an let you know.

I really think stMtime should answer 5pm on the file no matter what day of the year it is now.  Is it a bug?  Or do I just need to live with the APIs and do my own conversion ("stMtime answered 4pm, but since DST was in effect then and isn't now, add an hour")?

Actually what would be better is for stMtime to answer UTC.  This would do it (instead of the current VA code which uses FileTimeToLocalFileTime and FileTimeToSystemTime, this just uses FileTimeToSystemTime).

fileModifiedDateAndTime
"Answer an Array of size two containing the date and time the file was last modified."

  | st |

##PR = ##'1ET6RFP'.

  st := CfsOSSystemtime new.
  (self ftLastWriteTime fileTimeToSystemTime: st)
   ifTrue: [
  ^Array
 with: (Date new year: st wYear month: st wMonth day: st wDay)
 with: (Time new hours: st wHour
 minutes: st wMinute
 seconds: st wSecond)].
  ^Array with: (Date fromDays: 0) with: (Time fromSeconds: 0)  

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

Wayne Johnston
Donald - bottom line is thanks for looking into this.

But to talk about the above details - Eastern time is UTC minus 5.  That's what my computer tells me, and indeed right now it's about 2:20pm here and 7:20pm UTC.  So "normally" we are 5 hours behind, but during DST when we spring ahead, we essentially spring an hour east, an hour closer to UTC, so minus 4.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

dmacq
Agreed.

What version of VAST are you using? We introduced timezone support in 8.6.

On Thursday, January 9, 2014 2:22:48 PM UTC-5, Wayne Johnston wrote:
Donald - bottom line is thanks for looking into this.

But to talk about the above details - Eastern time is UTC minus 5.  That's what my computer tells me, and indeed right now it's about 2:20pm here and 7:20pm UTC.  So "normally" we are 5 hours behind, but during DST when we spring ahead, we essentially spring an hour east, an hour closer to UTC, so minus 4.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

dmacq
Wayne,

Something like this will get you stMtime as UTC -if- you are using Vast with timezone support:

CfsStat>>stMtimeAsUtc
        "Answers  the DateAndTime that the receiver's file was last modified."

    ^stMtime notNil ifTrue: [DateAndTime basicNew
        fromMilliseconds: (stMtime at: 1) asMilliseconds
        + (stMtime at: 2)  asMilliseconds]
        ifFalse: [nil]

The problem with using the OS time zone classes is that Windows 'knows' only about the two most recent DST changes. It has no historical knowledge.

I have a file modified 8/7/2013 at 7:51:14 PM.  stMtime from CfsStat stat: run today shows the modified time at 6:51:14 PM which is correct because DST is not in effect. Both are the same UTC time of 8/7/2013 at 11:51:14 PM.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

dmacq
Whoops! Should be


CfsStat>>stMtimeAsUtc
        "Answers  the DateAndTime that the receiver's file was last modified in UTC."

    ^stMtime notNil ifTrue: [(DateAndTime basicNew
        fromMilliseconds: (stMtime at: 1) asMilliseconds
        + (stMtime at: 2)  asMilliseconds) asUTC]
        ifFalse: [nil]

Stupid keyboard. ;-)

On Friday, January 10, 2014 5:01:49 PM UTC-5, Donald MacQueen wrote:
Wayne,

Something like this will get you stMtime as UTC -if- you are using Vast with timezone support:

CfsStat>>stMtimeAsUtc
        "Answers  the DateAndTime that the receiver's file was last modified."

    ^stMtime notNil ifTrue: [DateAndTime basicNew
        fromMilliseconds: (stMtime at: 1) asMilliseconds
        + (stMtime at: 2)  asMilliseconds]
        ifFalse: [nil]

The problem with using the OS time zone classes is that Windows 'knows' only about the two most recent DST changes. It has no historical knowledge.

I have a file modified 8/7/2013 at 7:51:14 PM.  stMtime from CfsStat stat: run today shows the modified time at 6:51:14 PM which is correct because DST is not in effect. Both are the same UTC time of 8/7/2013 at 11:51:14 PM.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

Wayne Johnston
Donald, first again I'd like to thank you for looking into this.  I thought someone from the community might pipe in with some wisdom, otherwise I would have sent this to support as a bug, since I still think it's a bug.

My main argument is the code you specify above give me a UTC DateAndTime which varies depending on the time of the year I ask for it.  But it should not vary.  Eastern US DST time of 8/7/2013 at 7:51:14 PM was UTC time of 8/7/2013 at 11:51:14 PM.  I should get that answer whether it is now DST or not, from any time zone in the world.

You had written "I have a file modified 8/7/2013 at 7:51:14 PM.  stMtime from CfsStat stat: run today shows the modified time at 6:51:14 PM'.  I would say that is also wrong.  A file modified at 8/7/2013 at 7:51:14 PM should give me a modified time of 8/7/2013 at 7:51:14 PM whether it is DST now or not.

There are surely many people smarter than me that have considered this, and apparently they don't mind the stMtime etc. behavior.  So maybe I can be convinced it's OK.

My interest is a client and server in perhaps two different time zones which want a local file with the same timestamp as each other, so that in future comparisons we see the timestamp is the same and is up to date and the file need not be copied again.  Say the server and client are in the same time zone, Eastern US, and I want the client to get a copy of the server's file.  I was taking what stMtime told me, the 8/7/2013 at 6:51:14 PM timestamp, and running it through code of our own which says that truly was 8/7/2013 at 10:51:14 PM GMT/UTC.  So when the file contents are copied to the client via SST let's say, I'd go through MSKOperatingServices coded using CfsOSSystemtime and ultimately <c: bool 'kernel32.dll':SetFileTime uint32 pointer pointer pointer> to set the UTC timestamp to 8/7/2013 at 10:51:14 PM.  That results in the file looking in Windows Explorer like it has a local time of 8/7/2013 at 6:51:14 PM (I would say that is correct given the input).  But that is different from how the original looks in Windows explorer, still 7:51:14 PM.  So the next time I compare the timestamps on the server and client, they give me a different stMtime so I think the client's copy is out of date so I need to re-copy it.  Bad for performance.

I have gotten around the problem by using my aforementioned code (in a separate method) which gives me what I believe is the correct UTC timestamp.  Even if I am proved wrong that my thinking is wrong, both the client and server answer the same UTC time so I think the timestamps match.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

dmacq
Wayne,

We may be talking past each other.  But before we go any further, please tell me what version of VAST you are using. Time support was added in 8.5.2.

Thanks,

Donald

On Saturday, January 11, 2014 6:46:02 PM UTC-5, Wayne Johnston wrote:
Donald, first again I'd like to thank you for looking into this.  I thought someone from the community might pipe in with some wisdom, otherwise I would have sent this to support as a bug, since I still think it's a bug.

My main argument is the code you specify above give me a UTC DateAndTime which varies depending on the time of the year I ask for it.  But it should not vary.  Eastern US DST time of 8/7/2013 at 7:51:14 PM was UTC time of 8/7/2013 at 11:51:14 PM.  I should get that answer whether it is now DST or not, from any time zone in the world.

You had written "I have a file modified 8/7/2013 at 7:51:14 PM.  stMtime from CfsStat stat: run today shows the modified time at 6:51:14 PM'.  I would say that is also wrong.  A file modified at 8/7/2013 at 7:51:14 PM should give me a modified time of 8/7/2013 at 7:51:14 PM whether it is DST now or not.

There are surely many people smarter than me that have considered this, and apparently they don't mind the stMtime etc. behavior.  So maybe I can be convinced it's OK.

My interest is a client and server in perhaps two different time zones which want a local file with the same timestamp as each other, so that in future comparisons we see the timestamp is the same and is up to date and the file need not be copied again.  Say the server and client are in the same time zone, Eastern US, and I want the client to get a copy of the server's file.  I was taking what stMtime told me, the 8/7/2013 at 6:51:14 PM timestamp, and running it through code of our own which says that truly was 8/7/2013 at 10:51:14 PM GMT/UTC.  So when the file contents are copied to the client via SST let's say, I'd go through MSKOperatingServices coded using CfsOSSystemtime and ultimately <c: bool 'kernel32.dll':SetFileTime uint32 pointer pointer pointer> to set the UTC timestamp to 8/7/2013 at 10:51:14 PM.  That results in the file looking in Windows Explorer like it has a local time of 8/7/2013 at 6:51:14 PM (I would say that is correct given the input).  But that is different from how the original looks in Windows explorer, still 7:51:14 PM.  So the next time I compare the timestamps on the server and client, they give me a different stMtime so I think the client's copy is out of date so I need to re-copy it.  Bad for performance.

I have gotten around the problem by using my aforementioned code (in a separate method) which gives me what I believe is the correct UTC timestamp.  Even if I am proved wrong that my thinking is wrong, both the client and server answer the same UTC time so I think the timestamps match.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

Wayne Johnston
Sorry.  I'm on VA 8.6 now, though I would complain about the stMtime inconsistency back in 8.0.0 as well.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

dmacq
Hi Wayne,

I think I now understand that you are saying that stMtime is not DST aware.  Consider this example:

Set the PC date to 2013 08 01.  DateAndTime now ----> 2013-08-01T15:23:11.991-04:00
Inspect (CfsStat stat: 'c:\avast\mingw.bat') stMtime  ---> (6/11/2013 6:02:54 PM) which is what Windows Explorer says
Create and Inspect a DateAndTime from stMtime ----> 2013-06-11T18:02:54-04:00
                                                                                       3548440974000 <------------------ milliseconds
                                                                                       America/New_York EDT -14400 Dst=true

Set the PC date to today. DateAndTime now -----> 2014-01-13T10:55:58.381-05:00
Inspect (CfsStat stat: 'c:\avast\mingw.bat') stMtime  ---> (6/11/2013 5:02:54 PM) <------------------------- different by 1 hour
Create and Inspect a DateAndTime from stMtime ----> 2013-06-11T17:02:54-04:00 <------------------------- different by 1 hour
                                                                                      3548437374000 <--- milliseconds differ by 3600000 or 1 hour
                                                                                      America/New_York EDT -14400 Dst=true

Does this example agree with what you are saying? IOW, CfsStat stat: is not 'DST aware' and gives incorrect values for DST dates if called when DST is not in effect.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

Wayne Johnston
Yes.

I want a way to get a file's timestamp which won't vary depending on the time of the year I ask for it.  Maybe it's OK for stMtime to not do that, so I was wanting a way to get the UTC time, which I maintain should definitely not change, and is accomplished by my code above.

Thanks Donald.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

dmacq
Wayne,

Would you please elaborate on how you are getting around this problem? Are you resetting the file's modified time stamp in Windows?

On Monday, January 13, 2014 3:04:33 PM UTC-5, Wayne Johnston wrote:
Yes.

I want a way to get a file's timestamp which won't vary depending on the time of the year I ask for it.  Maybe it's OK for stMtime to not do that, so I was wanting a way to get the UTC time, which I maintain should definitely not change, and is accomplished by my code above.

Thanks Donald.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Windows file last modified timestamp in UTC {stat:, FileTimeToLocalFileTime, FileTimeToSystemTime, TzSpecificLocalTimeToSystemTime}

Wayne Johnston
Time to admit for the benefit of any of my teammates reading this that our application says "GMT" instead of "UTC".  

The UTC-getting method #fileModifiedDateAndTime as I wrote in my first post above - I have renamed my version to fileModifiedDateAndTimeGMT below.  Similar methods for access time and create time.  And a #basicStatGMT: that call them, and a #statGMT:.  So when I want the UTC modified time on a file, I call the new #statGMT: of mine.

I wasn't bold enough to go and fix #fileModifiedDateAndTime to answer what I would call the correct local timestamp.  That is, in case anyone depends on what I would call its faulty behavior.  Note that method takes the UTC time and converts it to local time, where my method below answers the UTC time without a further conversion.

And to complete the picture, if on a copy of the file I pass that UTC time to SetFileTime, I get a file that looks right in Windows Explorer, and both files answer the same UTC timestamp from then on - unless of course one is modified.  At one point above you mentioned how this code doesn't know about the DST dates before the 2007 change, but that is OK - both files answer the same faulty UTC timestamp perhaps on old files - the sameness is what I need.

fileModifiedDateAndTimeGMT
"Like fileModifiedDateAndTime, but GMT"

  | st |

  st := CfsOSSystemtime new.
  (self ftLastWriteTime fileTimeToSystemTime: st)
   ifTrue: [
  ^Array
 with: (Date new year: st wYear month: st wMonth day: st wDay)
 with: (Time new hours: st wHour
 minutes: st wMinute
 seconds: st wSecond)].
  ^Array with: (Date fromDays: 0) with: (Time fromSeconds: 0)  

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/groups/opt_out.