@@ -2129,11 +2129,13 @@ These can easily be converted to a ``PeriodIndex``:
2129
2129
Time Zone Handling
2130
2130
------------------
2131
2131
2132
- Pandas provides rich support for working with timestamps in different time
2133
- zones using ``pytz `` and ``dateutil `` libraries. ``dateutil `` currently is only
2134
- supported for fixed offset and tzfile zones. The default library is ``pytz ``.
2135
- Support for ``dateutil `` is provided for compatibility with other
2136
- applications e.g. if you use ``dateutil `` in other Python packages.
2132
+ pandas provides rich support for working with timestamps in different time
2133
+ zones using the ``pytz `` and ``dateutil `` libraries.
2134
+
2135
+ .. note ::
2136
+
2137
+ pandas does not yet support ``datetime.timezone `` objects from the standard
2138
+ library.
2137
2139
2138
2140
Working with Time Zones
2139
2141
~~~~~~~~~~~~~~~~~~~~~~~
@@ -2145,94 +2147,87 @@ By default, pandas objects are time zone unaware:
2145
2147
rng = pd.date_range(' 3/6/2012 00:00' , periods = 15 , freq = ' D' )
2146
2148
rng.tz is None
2147
2149
2148
- To supply the time zone, you can use the ``tz `` keyword to ``date_range `` and
2149
- other functions. Dateutil time zone strings are distinguished from ``pytz ``
2150
- time zones by starting with ``dateutil/ ``.
2150
+ To localize these dates to a time zone (assign a particular time zone to a naive date),
2151
+ you can use the ``tz_localize `` method or the ``tz `` keyword argument in
2152
+ :func: `date_range `, :class: `Timestamp `, or :class: `DatetimeIndex `.
2153
+ You can either pass ``pytz `` or ``dateutil `` time zone objects or Olson time zone database strings.
2154
+ Olson time zone strings will return ``pytz `` time zone objects by default.
2155
+ To return ``dateutil `` time zone objects, append ``dateutil/ `` before the string.
2151
2156
2152
2157
* In ``pytz `` you can find a list of common (and less common) time zones using
2153
2158
``from pytz import common_timezones, all_timezones ``.
2154
- * ``dateutil `` uses the OS timezones so there isn't a fixed list available. For
2159
+ * ``dateutil `` uses the OS time zones so there isn't a fixed list available. For
2155
2160
common zones, the names are the same as ``pytz ``.
2156
2161
2157
2162
.. ipython :: python
2158
2163
2159
2164
import dateutil
2160
2165
2161
2166
# pytz
2162
- rng_pytz = pd.date_range(' 3/6/2012 00:00' , periods = 10 , freq = ' D' ,
2167
+ rng_pytz = pd.date_range(' 3/6/2012 00:00' , periods = 3 , freq = ' D' ,
2163
2168
tz = ' Europe/London' )
2164
2169
rng_pytz.tz
2165
2170
2166
2171
# dateutil
2167
- rng_dateutil = pd.date_range(' 3/6/2012 00:00' , periods = 10 , freq = ' D' ,
2168
- tz = ' dateutil/Europe/London' )
2172
+ rng_dateutil = pd.date_range(' 3/6/2012 00:00' , periods = 3 , freq = ' D' )
2173
+ rng_dateutil = rng_dateutil.tz_localize( ' dateutil/Europe/London' )
2169
2174
rng_dateutil.tz
2170
2175
2171
2176
# dateutil - utc special case
2172
- rng_utc = pd.date_range(' 3/6/2012 00:00' , periods = 10 , freq = ' D' ,
2177
+ rng_utc = pd.date_range(' 3/6/2012 00:00' , periods = 3 , freq = ' D' ,
2173
2178
tz = dateutil.tz.tzutc())
2174
2179
rng_utc.tz
2175
2180
2176
- Note that the ``UTC `` timezone is a special case in ``dateutil `` and should be constructed explicitly
2177
- as an instance of ``dateutil.tz.tzutc ``. You can also construct other timezones explicitly first,
2178
- which gives you more control over which time zone is used:
2181
+ Note that the ``UTC `` time zone is a special case in ``dateutil `` and should be constructed explicitly
2182
+ as an instance of ``dateutil.tz.tzutc ``. You can also construct other time
2183
+ zones objects explicitly first.
2179
2184
2180
2185
.. ipython :: python
2181
2186
2182
2187
import pytz
2183
2188
2184
2189
# pytz
2185
2190
tz_pytz = pytz.timezone(' Europe/London' )
2186
- rng_pytz = pd.date_range(' 3/6/2012 00:00' , periods = 10 , freq = ' D' ,
2187
- tz = tz_pytz)
2191
+ rng_pytz = pd.date_range(' 3/6/2012 00:00' , periods = 3 , freq = ' D' )
2192
+ rng_pytz = rng_pytz.tz_localize( tz_pytz)
2188
2193
rng_pytz.tz == tz_pytz
2189
2194
2190
2195
# dateutil
2191
2196
tz_dateutil = dateutil.tz.gettz(' Europe/London' )
2192
- rng_dateutil = pd.date_range(' 3/6/2012 00:00' , periods = 10 , freq = ' D' ,
2197
+ rng_dateutil = pd.date_range(' 3/6/2012 00:00' , periods = 3 , freq = ' D' ,
2193
2198
tz = tz_dateutil)
2194
2199
rng_dateutil.tz == tz_dateutil
2195
2200
2196
- Timestamps, like Python's ``datetime.datetime `` object can be either time zone
2197
- naive or time zone aware. Naive time series and ``DatetimeIndex `` objects can be
2198
- *localized * using ``tz_localize ``:
2199
-
2200
- .. ipython :: python
2201
-
2202
- ts = pd.Series(np.random.randn(len (rng)), rng)
2203
-
2204
- ts_utc = ts.tz_localize(' UTC' )
2205
- ts_utc
2206
-
2207
- Again, you can explicitly construct the timezone object first.
2208
- You can use the ``tz_convert `` method to convert pandas objects to convert
2209
- tz-aware data to another time zone:
2201
+ To convert a time zone aware pandas object from one time zone to another,
2202
+ you can use the ``tz_convert `` method.
2210
2203
2211
2204
.. ipython :: python
2212
2205
2213
- ts_utc .tz_convert(' US/Eastern' )
2206
+ rng_pytz .tz_convert(' US/Eastern' )
2214
2207
2215
2208
.. warning ::
2216
2209
2217
- Be wary of conversions between libraries. For some zones ``pytz `` and ``dateutil `` have different
2218
- definitions of the zone. This is more of a problem for unusual timezones than for
2210
+ Be wary of conversions between libraries. For some time zones, ``pytz `` and ``dateutil `` have different
2211
+ definitions of the zone. This is more of a problem for unusual time zones than for
2219
2212
'standard' zones like ``US/Eastern ``.
2220
2213
2221
2214
.. warning ::
2222
2215
2223
- Be aware that a timezone definition across versions of timezone libraries may not
2224
- be considered equal. This may cause problems when working with stored data that
2225
- is localized using one version and operated on with a different version.
2226
- See :ref: `here<io.hdf5-notes> ` for how to handle such a situation.
2216
+ Be aware that a time zone definition across versions of time zone libraries may not
2217
+ be considered equal. This may cause problems when working with stored data that
2218
+ is localized using one version and operated on with a different version.
2219
+ See :ref: `here<io.hdf5-notes> ` for how to handle such a situation.
2227
2220
2228
2221
.. warning ::
2229
2222
2230
- It is incorrect to pass a timezone directly into the ``datetime.datetime `` constructor (e.g.,
2231
- ``datetime.datetime(2011, 1, 1, tz=timezone('US/Eastern')) ``. Instead, the datetime
2232
- needs to be localized using the localize method on the timezone.
2223
+ For ``pytz `` time zones, it is incorrect to pass a time zone object directly into
2224
+ the ``datetime.datetime `` constructor
2225
+ (e.g., ``datetime.datetime(2011, 1, 1, tz=pytz.timezone('US/Eastern')) ``.
2226
+ Instead, the datetime needs to be localized using the ``localize `` method
2227
+ on the ``pytz `` time zone object.
2233
2228
2234
- Under the hood, all timestamps are stored in UTC. Scalar values from a
2235
- `` DatetimeIndex `` with a time zone will have their fields (day, hour, minute)
2229
+ Under the hood, all timestamps are stored in UTC. Values from a time zone aware
2230
+ :class: ` DatetimeIndex ` or :class: ` Timestamp ` will have their fields (day, hour, minute, etc. )
2236
2231
localized to the time zone. However, timestamps with the same UTC value are
2237
2232
still considered to be equal even if they are in different time zones:
2238
2233
@@ -2241,114 +2236,78 @@ still considered to be equal even if they are in different time zones:
2241
2236
rng_eastern = rng_utc.tz_convert(' US/Eastern' )
2242
2237
rng_berlin = rng_utc.tz_convert(' Europe/Berlin' )
2243
2238
2244
- rng_eastern[5 ]
2245
- rng_berlin[5 ]
2246
- rng_eastern[5 ] == rng_berlin[5 ]
2247
-
2248
- Like ``Series ``, ``DataFrame ``, and ``DatetimeIndex ``; ``Timestamp `` objects
2249
- can be converted to other time zones using ``tz_convert ``:
2250
-
2251
- .. ipython :: python
2252
-
2253
- rng_eastern[5 ]
2254
- rng_berlin[5 ]
2255
- rng_eastern[5 ].tz_convert(' Europe/Berlin' )
2256
-
2257
- Localization of ``Timestamp `` functions just like ``DatetimeIndex `` and ``Series ``:
2258
-
2259
- .. ipython :: python
2260
-
2261
- rng[5 ]
2262
- rng[5 ].tz_localize(' Asia/Shanghai' )
2263
-
2239
+ rng_eastern[2 ]
2240
+ rng_berlin[2 ]
2241
+ rng_eastern[2 ] == rng_berlin[2 ]
2264
2242
2265
- Operations between `` Series ` ` in different time zones will yield UTC
2266
- `` Series ` `, aligning the data on the UTC timestamps:
2243
+ Operations between :class: ` Series ` in different time zones will yield UTC
2244
+ :class: ` Series `, aligning the data on the UTC timestamps:
2267
2245
2268
2246
.. ipython :: python
2269
2247
2248
+ ts_utc = pd.Series(range (3 ), pd.date_range(' 20130101' , periods = 3 , tz = ' UTC' ))
2270
2249
eastern = ts_utc.tz_convert(' US/Eastern' )
2271
2250
berlin = ts_utc.tz_convert(' Europe/Berlin' )
2272
2251
result = eastern + berlin
2273
2252
result
2274
2253
result.index
2275
2254
2276
- To remove timezone from tz-aware `` DatetimeIndex `` , use ``tz_localize(None) `` or ``tz_convert(None) ``.
2277
- ``tz_localize(None) `` will remove timezone holding local time representations .
2278
- ``tz_convert(None) `` will remove timezone after converting to UTC time.
2255
+ To remove time zone information , use ``tz_localize(None) `` or ``tz_convert(None) ``.
2256
+ ``tz_localize(None) `` will remove the time zone yielding the local time representation .
2257
+ ``tz_convert(None) `` will remove the time zone after converting to UTC time.
2279
2258
2280
2259
.. ipython :: python
2281
2260
2282
2261
didx = pd.date_range(start = ' 2014-08-01 09:00' , freq = ' H' ,
2283
- periods = 10 , tz = ' US/Eastern' )
2262
+ periods = 3 , tz = ' US/Eastern' )
2284
2263
didx
2285
2264
didx.tz_localize(None )
2286
2265
didx.tz_convert(None )
2287
2266
2288
- # tz_convert(None) is identical with tz_convert('UTC').tz_localize(None)
2267
+ # tz_convert(None) is identical to tz_convert('UTC').tz_localize(None)
2289
2268
didx.tz_convert(' UTC' ).tz_localize(None )
2290
2269
2291
2270
.. _timeseries.timezone_ambiguous :
2292
2271
2293
2272
Ambiguous Times when Localizing
2294
2273
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2295
2274
2296
- In some cases, localize cannot determine the DST and non-DST hours when there are
2297
- duplicates. This often happens when reading files or database records that simply
2298
- duplicate the hours. Passing ``ambiguous='infer' `` into ``tz_localize `` will
2299
- attempt to determine the right offset. Below the top example will fail as it
2300
- contains ambiguous times and the bottom will infer the right offset.
2275
+ ``tz_localize `` may not be able to determine the UTC offset of a timestamp
2276
+ because daylight savings time (DST) in a local time zone causes some times to occur
2277
+ twice within one day ("clocks fall back"). The following options are available:
2278
+
2279
+ * ``'raise' ``: Raises a ``pytz.AmbiguousTimeError `` (the default behavior)
2280
+ * ``'infer' ``: Attempt to determine the correct offset base on the monotonicity of the timestamps
2281
+ * ``'NaT' ``: Replaces ambiguous times with ``NaT ``
2282
+ * ``bool ``: ``True `` represents a DST time, ``False `` represents non-DST time. An array-like of ``bool `` values is supported for a sequence of times.
2301
2283
2302
2284
.. ipython :: python
2303
2285
2304
2286
rng_hourly = pd.DatetimeIndex([' 11/06/2011 00:00' , ' 11/06/2011 01:00' ,
2305
- ' 11/06/2011 01:00' , ' 11/06/2011 02:00' ,
2306
- ' 11/06/2011 03:00' ])
2287
+ ' 11/06/2011 01:00' , ' 11/06/2011 02:00' ])
2307
2288
2308
- This will fail as there are ambiguous times
2289
+ This will fail as there are ambiguous times (`` '11/06/2011 01:00' ``)
2309
2290
2310
2291
.. code-block :: ipython
2311
2292
2312
2293
In [2]: rng_hourly.tz_localize('US/Eastern')
2313
2294
AmbiguousTimeError: Cannot infer dst time from Timestamp('2011-11-06 01:00:00'), try using the 'ambiguous' argument
2314
2295
2315
- Infer the ambiguous times
2316
-
2317
- .. ipython :: python
2318
-
2319
- rng_hourly_eastern = rng_hourly.tz_localize(' US/Eastern' , ambiguous = ' infer' )
2320
- rng_hourly_eastern.to_list()
2321
-
2322
- In addition to 'infer', there are several other arguments supported. Passing
2323
- an array-like of bools or 0s/1s where True represents a DST hour and False a
2324
- non-DST hour, allows for distinguishing more than one DST
2325
- transition (e.g., if you have multiple records in a database each with their
2326
- own DST transition). Or passing 'NaT' will fill in transition times
2327
- with not-a-time values. These methods are available in the ``DatetimeIndex ``
2328
- constructor as well as ``tz_localize ``.
2296
+ Handle these ambiguous times by specifying the following.
2329
2297
2330
2298
.. ipython :: python
2331
2299
2332
- rng_hourly_dst = np.array([1 , 1 , 0 , 0 , 0 ])
2333
- rng_hourly.tz_localize(' US/Eastern' , ambiguous = rng_hourly_dst).to_list()
2334
- rng_hourly.tz_localize(' US/Eastern' , ambiguous = ' NaT' ).to_list()
2335
-
2336
- didx = pd.date_range(start = ' 2014-08-01 09:00' , freq = ' H' ,
2337
- periods = 10 , tz = ' US/Eastern' )
2338
- didx
2339
- didx.tz_localize(None )
2340
- didx.tz_convert(None )
2341
-
2342
- # tz_convert(None) is identical with tz_convert('UTC').tz_localize(None)
2343
- didx.tz_convert(' UCT' ).tz_localize(None )
2300
+ rng_hourly.tz_localize(' US/Eastern' , ambiguous = ' infer' )
2301
+ rng_hourly.tz_localize(' US/Eastern' , ambiguous = ' NaT' )
2302
+ rng_hourly.tz_localize(' US/Eastern' , ambiguous = [True , True , False , False ])
2344
2303
2345
2304
.. _timeseries.timezone_nonexistent :
2346
2305
2347
2306
Nonexistent Times when Localizing
2348
2307
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2349
2308
2350
2309
A DST transition may also shift the local time ahead by 1 hour creating nonexistent
2351
- local times. The behavior of localizing a timeseries with nonexistent times
2310
+ local times ("clocks spring forward") . The behavior of localizing a timeseries with nonexistent times
2352
2311
can be controlled by the ``nonexistent `` argument. The following options are available:
2353
2312
2354
2313
* ``'raise' ``: Raises a ``pytz.NonExistentTimeError `` (the default behavior)
@@ -2382,58 +2341,61 @@ Transform nonexistent times to ``NaT`` or shift the times.
2382
2341
2383
2342
.. _timeseries.timezone_series :
2384
2343
2385
- TZ Aware Dtypes
2386
- ~~~~~~~~~~~~~~~
2344
+ Time Zone Series Operations
2345
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
2387
2346
2388
- ``Series/DatetimeIndex `` with a timezone **naive ** value are represented with a dtype of ``datetime64[ns] ``.
2347
+ A :class: `Series ` with time zone **naive ** values is
2348
+ represented with a dtype of ``datetime64[ns] ``.
2389
2349
2390
2350
.. ipython :: python
2391
2351
2392
2352
s_naive = pd.Series(pd.date_range(' 20130101' , periods = 3 ))
2393
2353
s_naive
2394
2354
2395
- ``Series/DatetimeIndex `` with a timezone **aware ** value are represented with a dtype of ``datetime64[ns, tz] ``.
2355
+ A :class: `Series ` with a time zone **aware ** values is
2356
+ represented with a dtype of ``datetime64[ns, tz] `` where ``tz `` is the time zone
2396
2357
2397
2358
.. ipython :: python
2398
2359
2399
2360
s_aware = pd.Series(pd.date_range(' 20130101' , periods = 3 , tz = ' US/Eastern' ))
2400
2361
s_aware
2401
2362
2402
- Both of these ``Series `` can be manipulated via the ``.dt `` accessor, see :ref: `here <basics.dt_accessors >`.
2363
+ Both of these :class: `Series ` time zone information
2364
+ can be manipulated via the ``.dt `` accessor, see :ref: `the dt accessor section <basics.dt_accessors >`.
2403
2365
2404
- For example, to localize and convert a naive stamp to timezone aware.
2366
+ For example, to localize and convert a naive stamp to time zone aware.
2405
2367
2406
2368
.. ipython :: python
2407
2369
2408
2370
s_naive.dt.tz_localize(' UTC' ).dt.tz_convert(' US/Eastern' )
2409
2371
2410
-
2411
- Further more you can `` .astype(...) `` timezone aware ( and naive). This operation is effectively a localize AND convert on a naive stamp, and
2412
- a convert on an aware stamp .
2372
+ Time zone information can also be manipulated using the `` astype `` method.
2373
+ This method can localize and convert time zone naive timestamps or
2374
+ convert time zone aware timestamps .
2413
2375
2414
2376
.. ipython :: python
2415
2377
2416
- # localize and convert a naive timezone
2378
+ # localize and convert a naive time zone
2417
2379
s_naive.astype(' datetime64[ns, US/Eastern]' )
2418
2380
2419
2381
# make an aware tz naive
2420
2382
s_aware.astype(' datetime64[ns]' )
2421
2383
2422
- # convert to a new timezone
2384
+ # convert to a new time zone
2423
2385
s_aware.astype(' datetime64[ns, CET]' )
2424
2386
2425
2387
.. note ::
2426
2388
2427
2389
Using :meth: `Series.to_numpy ` on a ``Series ``, returns a NumPy array of the data.
2428
- NumPy does not currently support timezones (even though it is *printing * in the local timezone !),
2429
- therefore an object array of Timestamps is returned for timezone aware data:
2390
+ NumPy does not currently support time zones (even though it is *printing * in the local time zone !),
2391
+ therefore an object array of Timestamps is returned for time zone aware data:
2430
2392
2431
2393
.. ipython :: python
2432
2394
2433
2395
s_naive.to_numpy()
2434
2396
s_aware.to_numpy()
2435
2397
2436
- By converting to an object array of Timestamps, it preserves the timezone
2398
+ By converting to an object array of Timestamps, it preserves the time zone
2437
2399
information. For example, when converting back to a Series:
2438
2400
2439
2401
.. ipython :: python
0 commit comments