[vwnc] Anyone having problems with Delays?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Anyone having problems with Delays?

Chris Winemiller
cstb wrote:
> So, if we add the obvious interface
>         Delay class>>until: aTimestamp
> then we should expect this variant to provide wallTime scheduling
#until: would be a convenience method, but there already exist a family
of methods that should do this: #untilSeconds:, #untilMilliseconds:, and
#untilMicroseconds:.

Chris
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Anyone having problems with Delays?

Eliot Miranda-2
In reply to this post by jas


On Thu, Aug 14, 2008 at 4:49 PM, cstb <[hidden email]> wrote:
[snip]

 You actually need 3 distinct clocks
 to always get the right results:

 1) upTime
 2) wallTime
 3) elapsedWallTime

 The first is required for internal-duration based scheduling,
 and MUST NOT be affected by changing values of systemWallTime.
 This includes all timeout scheduling; most, but not all,
 periodic scheduling; and most, but not all, sensor/controller
 scheduling.  Note that upTime CAN NOT be adjusted by anything
 except a periodic increment, nor can said period be modified.
 And upTime must either count "forever", without overflowing,
 or ALL deltas must be calculated modulo the container size.

 The second is required for (most) timeOfDay based scheduling,
 and MUST BE affected by any changes to the system's timeOfDay.
 This is rarely the correct clock to use for scheduling, because
 of susceptibility to time dilation, a common error.  Very low
 resolution schedules - mostly for dates, rather than times.

 Note that a single clock CANNOT perform both functions correctly.
 You can fake one, using the other, but you'll have to maintain
 multiple, distinct offsets and skews, and use them in every
 calculation. Merging of values (such as the biased upTime of VW2.5)
 destroys necessary information - guaranteed to result in heisenbugs.

 The third of these is hard to obtain, and not often considered
 but required for (most) external-duration based scheduling.
 Most applications that care about it use an estimated version.
 The simpler versions rely on an external counter surviving a reboot.
 Some also save a snapshot of (wallTime at system boot) somewhere
 in persistant store, and compare it with the previous version
 to figure out how much wallTime has elapsed while the system
 was down. This variation is important in fault-tolerant contexts,
 where processes in the 'scheduled environment' continue even
 while the system is down.

I think this analysis is almost right.  I think you need two clocks.  I think I'm right that UTC does not change except for leap seconds and then only increases, so there are occasionally second duration jumps forward; there are never two adjacent seconds that have the same second count in UTC.  So at least terrestrially UTC can be used for wall time, and a non-adjusting clock that starts with current UTC and doesn't adjust with leap seconds can provide up-time and duration.

Hence the vw7.x clock which is UTC microseconds from 1901 in 64 bits, which wraps after 57k years.

Further, if the system can adjust UTC with leap-seconds without missing delays (i.e. spotting delays which expire within a leap second) you only need one clock.  Since there is only ever zero or one delay in effect when adjusting for leap seconds this can't be that hard top do.

This applies at least terrestrially where UTC provides a locally absolute clock.

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
jas
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Anyone having problems with Delays?

jas
At 05:54 PM 8/15/2008, Eliot Miranda wrote:


On Thu, Aug 14, 2008 at 4:49 PM, cstb <[hidden email]> wrote:
[snip]

 You actually need 3 distinct clocks
 to always get the right results:

 1) upTime
 2) wallTime
 3) elapsedWallTime

[snip]

I think this analysis is almost right.


Thank you.  I've worked hard to become so.


I think you need two clocks.


Then why are you smiling?


I think I'm right that UTC does...


You are using UTC against me, ah?


...that UTC does not change
except for leap seconds and then only increases,
so there are occasionally second duration jumps forward;
there are never two adjacent seconds
that have the same second count in UTC.

So at least terrestrially
UTC can be used for wall time,


Naturally you must suspect me to attack with monotonicity?


and a non-adjusting clock
that starts with current UTC
and doesn't adjust with leap seconds
can provide up-time and duration.


Unless the enemy has studied his minkowski space-time manifolds
with applications in the special and general relativity of
Dolphin, GNUST, GS, Squeak, ST/X, VA, VW235, and vw7.w clocks,
which I have.


Hence the vw7.x clock


And what is that?


which is UTC microseconds from 1901 in 64 bits which wraps after 57k years.
Further, if the system can adjust UTC with leap-seconds without missing delays
(i.e. spotting delays which expire within a leap second) you only need one clock.


You're amazing!


Since there is only ever zero or one delay in effect
when adjusting for leap seconds this can't be that hard to do.

This applies at least terrestrially where UTC provides a locally absolute clock.


There's something I ought to tell you.


Tell me!


I am not right-handed either ;-)


Cheers,

-Jim







_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Anyone having problems with Delays?

Chris Winemiller
In reply to this post by Eliot Miranda-2
Eliot Miranda wrote:

> I think this analysis is almost right.  I think you need two clocks.
>  I think I'm right that UTC does not change except for leap seconds
> and then only increases, so there are occasionally second duration
> jumps forward; there are never two adjacent seconds that have the same
> second count in UTC.  So at least terrestrially UTC can be used for
> wall time, and a non-adjusting clock that starts with current UTC and
> doesn't adjust with leap seconds can provide up-time and duration.
>
> Hence the vw7.x clock which is UTC microseconds from 1901 in 64 bits,
> which wraps after 57k years.
>
> Further, if the system can adjust UTC with leap-seconds without
> missing delays (i.e. spotting delays which expire within a leap
> second) you only need one clock.  Since there is only ever zero or one
> delay in effect when adjusting for leap seconds this can't be that
> hard top do.
>
> This applies at least terrestrially where UTC provides a locally
> absolute clock.
Nice in theory, but not in practice.  This is the fundamental flaw of
7.x's implementation.  Computer times *do* drift, and are periodically
corrected forward or backward by a command from some kind of time
provider (may or may not be NTP).  By tying duration Delays to the
computer's "absolute" (but drifting) time, 7.x completely broke duration
Delays.  They break when the time corrections occur.
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Anyone having problems with Delays?

Cesar Rabak
Chris Winemiller escreveu:
[snipped]

> Nice in theory, but not in practice.  This is the fundamental flaw of
> 7.x's implementation.  Computer times *do* drift, and are periodically
> corrected forward or backward by a command from some kind of time
> provider (may or may not be NTP).  By tying duration Delays to the
> computer's "absolute" (but drifting) time, 7.x completely broke duration
> Delays.  They break when the time corrections occur.
Chris,

I'm a bit confused right now.

For what time span are you considering the delays you need to implement?

--
Cesar Rabak
GNU/Linux User 52247.
Get counted: http://counter.li.org/
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Anyone having problems with Delays?

Chris Winemiller
Cesar Rabak wrote:
Chris Winemiller escreveu:
[snipped]

  
Nice in theory, but not in practice.  This is the fundamental flaw of 
7.x's implementation.  Computer times *do* drift, and are periodically 
corrected forward or backward by a command from some kind of time 
provider (may or may not be NTP).  By tying duration Delays to the 
computer's "absolute" (but drifting) time, 7.x completely broke duration 
Delays.  They break when the time corrections occur.
    
Chris,

I'm a bit confused right now.

For what time span are you considering the delays you need to implement?
  
The spans do not matter but, for the record, anywhere from a few seconds to perhaps a few minutes.  And the spans are not predetermined; they are dictated by customer-created "recipes" that the application must execute.

Why do you think the span matters?  If the Delay governs the length of time of a recipe step, altering that Delay's duration by even a few seconds can change the step time by 20%, 50%, 75%, or whatever.  And that is enough to ruin a product whose current value can be as much as a million dollars.

Chris
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Anyone having problems with Delays?

Eliot Miranda-2


On Wed, Aug 20, 2008 at 6:31 AM, Chris Winemiller <[hidden email]> wrote:
Cesar Rabak wrote:
Chris Winemiller escreveu:
[snipped]

  
Nice in theory, but not in practice.  This is the fundamental flaw of 
7.x's implementation.  Computer times *do* drift, and are periodically 
corrected forward or backward by a command from some kind of time 
provider (may or may not be NTP).  By tying duration Delays to the 
computer's "absolute" (but drifting) time, 7.x completely broke duration 
Delays.  They break when the time corrections occur.
    
Chris,

I'm a bit confused right now.

For what time span are you considering the delays you need to implement?
  
The spans do not matter but, for the record, anywhere from a few seconds to perhaps a few minutes.  And the spans are not predetermined; they are dictated by customer-created "recipes" that the application must execute.

Why do you think the span matters?  If the Delay governs the length of time of a recipe step, altering that Delay's duration by even a few seconds can change the step time by 20%, 50%, 75%, or whatever.  And that is enough to ruin a product whose current value can be as much as a million dollars.

Chris, are you running on windows?  If so, what happens if you run with -walltime?  As has been pointed out to me by Andreas Raab the high-performance counter on modern CPUs is no longer constant frequency.  On-chip cpu power-management will alter the frequency of the HPC and make it entirely unsuitable as the basis for the clock, something that didn't happen back in the day.




Chris

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc



_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Anyone having problems with Delays?

Chris Winemiller
Eliot Miranda wrote:


On Wed, Aug 20, 2008 at 6:31 AM, Chris Winemiller <[hidden email]> wrote:
Cesar Rabak wrote:
Chris Winemiller escreveu:
[snipped]

  
Nice in theory, but not in practice.  This is the fundamental flaw of 
7.x's implementation.  Computer times *do* drift, and are periodically 
corrected forward or backward by a command from some kind of time 
provider (may or may not be NTP).  By tying duration Delays to the 
computer's "absolute" (but drifting) time, 7.x completely broke duration 
Delays.  They break when the time corrections occur.
    
Chris,

I'm a bit confused right now.

For what time span are you considering the delays you need to implement?
  
The spans do not matter but, for the record, anywhere from a few seconds to perhaps a few minutes.  And the spans are not predetermined; they are dictated by customer-created "recipes" that the application must execute.

Why do you think the span matters?  If the Delay governs the length of time of a recipe step, altering that Delay's duration by even a few seconds can change the step time by 20%, 50%, 75%, or whatever.  And that is enough to ruin a product whose current value can be as much as a million dollars.

Chris, are you running on windows?  If so, what happens if you run with -walltime?  As has been pointed out to me by Andreas Raab the high-performance counter on modern CPUs is no longer constant frequency.  On-chip cpu power-management will alter the frequency of the HPC and make it entirely unsuitable as the basis for the clock, something that didn't happen back in the day.
I have not tried -walltime.  But it won't be adequate anyway because our product is a mult-image application, and only some of those images run on Windows.  We also have other VW-based products that customers typically run on a Unix flavor.  We need a cross-platform solution.

Chris

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Anyone having problems with Delays?

Dennis smith-4
Sorry to jump into the middle of this at this late point,
but I have been on vacation.
Just a note -- no solution needed.
We had a problem with Delays running on Windows on Xen on Linux.
Seems that as it switched processors the time we got shifted a tiny (both directions)
amount -- enough to cause VW to appear to freeze.  We solved that specific
issue by binding a VW image to on processor.  We never determined what
was exactly at fault.  -walltime did not appear to solve that particular problem.

Chris Winemiller wrote:
Eliot Miranda wrote:


On Wed, Aug 20, 2008 at 6:31 AM, Chris Winemiller <[hidden email]> wrote:
Cesar Rabak wrote:
Chris Winemiller escreveu:
[snipped]

  
Nice in theory, but not in practice.  This is the fundamental flaw of 
7.x's implementation.  Computer times *do* drift, and are periodically 
corrected forward or backward by a command from some kind of time 
provider (may or may not be NTP).  By tying duration Delays to the 
computer's "absolute" (but drifting) time, 7.x completely broke duration 
Delays.  They break when the time corrections occur.
    
Chris,

I'm a bit confused right now.

For what time span are you considering the delays you need to implement?
  
The spans do not matter but, for the record, anywhere from a few seconds to perhaps a few minutes.  And the spans are not predetermined; they are dictated by customer-created "recipes" that the application must execute.

Why do you think the span matters?  If the Delay governs the length of time of a recipe step, altering that Delay's duration by even a few seconds can change the step time by 20%, 50%, 75%, or whatever.  And that is enough to ruin a product whose current value can be as much as a million dollars.

Chris, are you running on windows?  If so, what happens if you run with -walltime?  As has been pointed out to me by Andreas Raab the high-performance counter on modern CPUs is no longer constant frequency.  On-chip cpu power-management will alter the frequency of the HPC and make it entirely unsuitable as the basis for the clock, something that didn't happen back in the day.
I have not tried -walltime.  But it won't be adequate anyway because our product is a mult-image application, and only some of those images run on Windows.  We also have other VW-based products that customers typically run on a Unix flavor.  We need a cross-platform solution.

Chris

_______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

-- 
Dennis Smith                 		         +1 416.798.7948
Cherniak Software Development Corporation   Fax: +1 416.798.0948
509-2001 Sheppard Avenue East        [hidden email]
Toronto, ON M2J 4Z8              <a class="moz-txt-link-freetext" href="sip:dennis@CherniakSoftware.com">sip:dennis@...
Canada			         http://www.CherniakSoftware.com
Entrance off Yorkland Blvd south of Sheppard Ave east of the DVP

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Anyone having problems with Delays?

Chris Winemiller
Dennis,

We've recently noticed this problem on an HP-UX system (i.e. that each processor has its own time base).  However, that is not the source of the bug I've been describing. The bug happens on many (all?) supported platforms; it occurs whenever the OS time is changed (by command or otherwise) forward or backward.; and Cincom has known about it for at least a year (when we told them a year ago), and I think they also recognize it as "critical".

Chris

Dennis Smith wrote:
Sorry to jump into the middle of this at this late point,
but I have been on vacation.
Just a note -- no solution needed.
We had a problem with Delays running on Windows on Xen on Linux.
Seems that as it switched processors the time we got shifted a tiny (both directions)
amount -- enough to cause VW to appear to freeze.  We solved that specific
issue by binding a VW image to on processor.  We never determined what
was exactly at fault.  -walltime did not appear to solve that particular problem.

Chris Winemiller wrote:
Eliot Miranda wrote:


On Wed, Aug 20, 2008 at 6:31 AM, Chris Winemiller <[hidden email]> wrote:
Cesar Rabak wrote:
Chris Winemiller escreveu:
[snipped]

  
Nice in theory, but not in practice.  This is the fundamental flaw of 
7.x's implementation.  Computer times *do* drift, and are periodically 
corrected forward or backward by a command from some kind of time 
provider (may or may not be NTP).  By tying duration Delays to the 
computer's "absolute" (but drifting) time, 7.x completely broke duration 
Delays.  They break when the time corrections occur.
    
Chris,

I'm a bit confused right now.

For what time span are you considering the delays you need to implement?
  
The spans do not matter but, for the record, anywhere from a few seconds to perhaps a few minutes.  And the spans are not predetermined; they are dictated by customer-created "recipes" that the application must execute.

Why do you think the span matters?  If the Delay governs the length of time of a recipe step, altering that Delay's duration by even a few seconds can change the step time by 20%, 50%, 75%, or whatever.  And that is enough to ruin a product whose current value can be as much as a million dollars.

Chris, are you running on windows?  If so, what happens if you run with -walltime?  As has been pointed out to me by Andreas Raab the high-performance counter on modern CPUs is no longer constant frequency.  On-chip cpu power-management will alter the frequency of the HPC and make it entirely unsuitable as the basis for the clock, something that didn't happen back in the day.
I have not tried -walltime.  But it won't be adequate anyway because our product is a mult-image application, and only some of those images run on Windows.  We also have other VW-based products that customers typically run on a Unix flavor.  We need a cross-platform solution.

Chris

_______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
jas
Reply | Threaded
Open this post in threaded view
|

Re: [vwnc] Anyone having problems with Delays?

jas
At 02:00 PM 8/20/2008, Chris Winemiller wrote:
Dennis,

We've recently noticed this problem on an HP-UX system (i.e. that each processor has its own time base).  However, that is not the source of the bug I've been describing. The bug happens on many (all?) supported platforms; it occurs whenever the OS time is changed (by command or otherwise) forward or backward.


The crux of the issue is right here, as Chris points out.

I can change the wallTime of my system whenever I want,
using the GUI thing on the bottom right, or using the command prompt,
or whatever.  And sure, if I'm getting time updates from the net,
it may autocorrect, but that only means there is yet another source,
each of which might cause wallTime to fail the monotonically increasing
constraint - and invalidate one or more delays in the process.

On the other hand, if I am offline, and change the system time,
I do want the Smalltalk image to use the modified wallTime
for anything having to do with dates and/or timeOfDay.

Makes two clocks the lower bound,
as far as I can tell.


Cheers,

-cstb

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
12