[R] timezone conversion difficulties with the new US daylight saving time switch over

Tue Oct 30 20:18:05 CET 2007

I'm having difficulties with daylight saving times in US time zones.
(Apologies for the long post, but the problem seems subtle and complex,
unless I'm doing something completely wrong, in which case it should
be evident from the first 10 lines below.)

This is what I see, using a (slightly modified) example from ?as.POSIXlt :

 > as.POSIXlt((d <- Sys.time()), "EST5EDT")  # the current time in New York
[1] "2007-10-30 12:38:47 EST"
 > d
[1] "2007-10-30 11:38:47 Mountain Daylight Time"
 >

The problem is that Mountain Time is 2 hours behind Eastern Time and
the US is still on Daylight Saving Time - the current time in New York
should be reported as "13:38:47 EDT", not "12:38:47 EST".

This is running on Windows XP 64 bit (SP2), and I see the same behavior
on Windows XP 32 bit and on Windows 2000 Server (SP4).

AFAICS, this problem only occurs this week, and this week is unusual in that
it is the first year that Daylight Saving Time in the US ends on the first
Sunday in November rather than the last Sunday in October (I don't know
whether this is the cause of the problem, but it seems likely).

I see the same problem around the same week last year, but before and after
this week in both years, the conversions are fine:

 > # *** problem in 2007
 > as.POSIXlt(as.POSIXct("2007-10-30 11:38:47"), "EST5EDT")
[1] "2007-10-30 12:38:47 EST"
 > # before the problem week 2007
 > as.POSIXlt(as.POSIXct("2007-10-20 11:38:47"), "EST5EDT")
[1] "2007-10-20 13:38:47 EDT"
 > # after the problem week 2007
 > as.POSIXlt(as.POSIXct("2007-11-05 11:38:47"), "EST5EDT")
[1] "2007-11-05 13:38:47 EST"
 > # *** problem in 2006
 > as.POSIXlt(as.POSIXct("2006-10-30 11:38:47"), "EST5EDT")
[1] "2006-10-30 12:38:47 EST"
 > # before the problem week 2006
 > as.POSIXlt(as.POSIXct("2006-10-27 11:38:47"), "EST5EDT")
[1] "2006-10-27 13:38:47 EDT"
 > # after the problem week 2006
 > as.POSIXlt(as.POSIXct("2006-11-07 11:38:47"), "EST5EDT")
[1] "2006-11-07 13:38:47 EST"
 >

My computer is set to the Mountain Daylight timezone, and is set to automatically
adjust for Daylight Saving Time changes.

 > sessionInfo()
R version 2.6.0 Patched (2007-10-11 r43143)
i386-pc-mingw32

locale:
LC_COLLATE=English_United States.1252;LC_CTYPE=English_United States.1252;LC_MONETARY=English_United States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base
 > Sys.timezone()
[1] "Mountain Daylight Time"
 >

If I explicitly set env var TZ, the conversion problems go away,
but the time reported by Sys.time() is inappropriately not in daylight
saving time:

 > Sys.time()
[1] "2007-10-30 13:14:38 Mountain Daylight Time"
 > Sys.setenv(TZ="MST7MDT")
 > Sys.time()
[1] "2007-10-30 12:14:51 MST"
 >

If I set my system timezone to Eastern Daylight Time, and
restart R, I also get problematic behavior (as.POSIXlt
inappropriately adjusting a time by an hour on the day
after Daylight saving time ends):

 > Sys.timezone()
[1] "Eastern Daylight Time"
 > as.POSIXlt((d <- Sys.time()), "EST5EDT")
[1] "2007-10-30 12:57:40 EST"
 > d
[1] "2007-10-30 13:57:40 Eastern Daylight Time"
 >
 > # *** problem week 2007
 > as.POSIXlt(as.POSIXct("2007-10-30 11:38:47"), "EST5EDT")
[1] "2007-10-30 10:38:47 EST"
 > # before the problem week 2007
 > as.POSIXlt(as.POSIXct("2007-10-20 11:38:47"), "EST5EDT")
[1] "2007-10-20 11:38:47 EDT"
 > # after the problem week 2007
 > as.POSIXlt(as.POSIXct("2007-11-05 11:38:47"), "EST5EDT")
[1] "2007-11-05 11:38:47 EST"
 > # *** problem week 2006 - the day is after the switch, but
 > # the time gets adjusted by one hour
 > as.POSIXlt(as.POSIXct("2006-10-30 11:38:47"), "EST5EDT")
[1] "2006-10-30 10:38:47 EST"
 > # before the problem week 2006
 > as.POSIXlt(as.POSIXct("2006-10-27 11:38:47"), "EST5EDT")
[1] "2006-10-27 11:38:47 EDT"
 > # after the problem week 2006
 > as.POSIXlt(as.POSIXct("2006-11-10 11:38:47"), "EST5EDT")
[1] "2006-11-10 11:38:47 EST"
 >

The problem in 2006 goes away if I set TZ="EST5EDT":
 > Sys.setenv(TZ = "EST5EDT")
 > Sys.timezone()
[1] "EST"
 > as.POSIXlt(as.POSIXct("2006-10-30 11:38:47"), "EST5EDT")
[1] "2006-10-30 11:38:47 EST"
 >

Questions:
(1) should I be using a different way to convert times between time
     zones?
(2) is there a problem in how R is interacting with the system time
     conversion facilities?
(3) if the answer to (2) is yes, does anyone have any good hints for
     where to start looking for a solution? It looks to me like
     unclass(as.POSIXlt(as.POSIXct("2007-10-30 11:38:47"), "EST5EDT"))$isdst
     is inappropriately zero.  I tried following the code in
     src/main/datetime.c:do_asPOSIXlt, and it looks like this isdst value
     comes from a call to localtime() from localtime0() ...

thanks in advance for any help,

Tony Plate

PS. I do not see the same problem in an old version of R (2.2.1)
running under Ubuntu Linux:

 > Sys.timezone()
[1] ""
 > as.POSIXlt((d <- Sys.time()), "EST5EDT")
[1] "2007-10-29 14:56:49 EDT"
 > d
[1] "2007-10-29 12:56:49 MDT"
 >
 > # ok in 2007
 > as.POSIXlt(as.POSIXct("2007-10-30 11:38:47"), "EST5EDT")
[1] "2007-10-30 13:38:47 EDT"
 > # before the problem week 2007
 > as.POSIXlt(as.POSIXct("2007-10-20 11:38:47"), "EST5EDT")
[1] "2007-10-20 13:38:47 EDT"
 > # after the problem week 2007
 > as.POSIXlt(as.POSIXct("2007-11-05 11:38:47"), "EST5EDT")
[1] "2007-11-05 13:38:47 EST"
 > # ok in 2006
 > as.POSIXlt(as.POSIXct("2006-10-30 11:38:47"), "EST5EDT")
[1] "2006-10-30 13:38:47 EST"
 > # before the problem week 2006
 > as.POSIXlt(as.POSIXct("2006-10-27 11:38:47"), "EST5EDT")
[1] "2006-10-27 13:38:47 EDT"
 > # after the problem week 2006
 > as.POSIXlt(as.POSIXct("2006-11-07 11:38:47"), "EST5EDT")
[1] "2006-11-07 13:38:47 EST"
 >
 > sessionInfo()
R version 2.2.1, 2005-12-20, i486-pc-linux-gnu

attached base packages:
[1] "methods"   "stats"     "graphics"  "grDevices" "utils"     "datasets"
[7] "base"
 >