Skip to content

Adding a list of timedelta to a column does not work for tzlocal() timezones #14523

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ekipmanager opened this issue Oct 27, 2016 · 1 comment
Labels
Bug Timedelta Timedelta data type Timezones Timezone data dtype

Comments

@ekipmanager
Copy link

A small, complete example of the issue

When you try to add a list of timedelta to a column that is datetime the behavior changes if the tzinfo is coming from datetime.tz.tzlocal(). In the example below I make two columns of timestamps, on with tz.tzlocal() and the other tz.tzutc() and then try to add a list of timedelta to it. The tzutc() ones behave as expected but the ones with tzloca() only the first row does the addition properly and for the rest the addition is done to unix epoch of 0!

# Your code here
import datetime
from dateutil import tz
import pandas as pd
dates1 = [datetime.datetime(2013, 9, 20,13,00, tzinfo=tz.tzlocal()) + datetime.timedelta(minutes=100*i) for i in range(10)]
dates2 = [datetime.datetime(2013, 9, 20,13,00, tzinfo=tz.tzutc()) + datetime.timedelta(minutes=100*i) for i in range(10)]
df = pd.DataFrame({"start1": dates1, "start2": dates2})
delta = [datetime.timedelta(seconds=x) for x in range(10)]
df['end1'] = df['start1'] + delta
df['end2'] = df['start2'] + delta
df

Expected Output

                 start1                    start2  \

0 2013-09-20 13:00:00-07:00 2013-09-20 13:00:00+00:00
1 2013-09-20 14:40:00-07:00 2013-09-20 14:40:00+00:00
2 2013-09-20 16:20:00-07:00 2013-09-20 16:20:00+00:00
3 2013-09-20 18:00:00-07:00 2013-09-20 18:00:00+00:00
4 2013-09-20 19:40:00-07:00 2013-09-20 19:40:00+00:00
5 2013-09-20 21:20:00-07:00 2013-09-20 21:20:00+00:00
6 2013-09-20 23:00:00-07:00 2013-09-20 23:00:00+00:00
7 2013-09-21 00:40:00-07:00 2013-09-21 00:40:00+00:00
8 2013-09-21 02:20:00-07:00 2013-09-21 02:20:00+00:00
9 2013-09-21 04:00:00-07:00 2013-09-21 04:00:00+00:00

                   end1                      end2  

0 2013-09-20 13:00:00-07:00 2013-09-20 13:00:00+00:00
1 2013-09-20 13:00:01-07:00 2013-09-20 14:40:01+00:00
2 2013-09-20 13:00:02-07:00 2013-09-20 16:20:02+00:00
3 2013-09-20 13:00:03-07:00 2013-09-20 18:00:03+00:00
4 2013-09-20 13:00:04-07:00 2013-09-20 19:40:04+00:00
5 2013-09-20 13:00:05-07:00 2013-09-20 21:20:05+00:00
6 2013-09-20 13:00:06-07:00 2013-09-20 23:00:06+00:00
7 2013-09-20 13:00:07-07:00 2013-09-21 00:40:07+00:00
8 2013-09-20 13:00:08-07:00 2013-09-21 02:20:08+00:00
9 2013-09-20 13:00:09-07:00 2013-09-21 04:00:09+00:00

Output of pd.show_versions()

# Paste the output here ## INSTALLED VERSIONS

commit: None
python: 2.7.11.final.0
python-bits: 64
OS: Linux
OS-release: 4.7.4-100.fc23.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.utf8

pandas: 0.18.1
nose: 1.3.7
pip: 8.1.2
setuptools: 18.0.1
Cython: 0.23.4
numpy: 1.11.1
scipy: 0.17.1
statsmodels: 0.6.1
xarray: None
IPython: 4.1.1
sphinx: 1.2.3
patsy: 0.4.1
dateutil: 2.5.3
pytz: 2016.6.1
blosc: None
bottleneck: 0.6.0
tables: None
numexpr: 2.4.6
matplotlib: 1.5.1
openpyxl: 1.8.6
xlrd: None
xlwt: None
xlsxwriter: None
lxml: 3.4.4
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.12
pymysql: None
psycopg2: 2.6.1 (dt dec pq3 ext lo64)
jinja2: 2.8
boto: 2.41.0
pandas_datareader: None

@jreback jreback added Timedelta Timedelta data type Timezones Timezone data dtype Bug Duplicate Report Duplicate issue or pull request labels Oct 28, 2016
@jreback jreback added this to the No action milestone Oct 28, 2016
@jreback
Copy link
Contributor

jreback commented Oct 28, 2016

fixed by #13583, in 0.19.0

In [1]: pd.__version__
Out[1]: '0.19.0'

In [2]: import datetime
   ...: from dateutil import tz
   ...: import pandas as pd
   ...: dates1 = [datetime.datetime(2013, 9, 20,13,00, tzinfo=tz.tzlocal()) + datetime.timedelta(minutes=100*i) for i in range(10)]
   ...: dates2 = [datetime.datetime(2013, 9, 20,13,00, tzinfo=tz.tzutc()) + datetime.timedelta(minutes=100*i) for i in range(10)]
   ...: df = pd.DataFrame({"start1": dates1, "start2": dates2})
   ...: delta = [datetime.timedelta(seconds=x) for x in range(10)]
   ...: df['end1'] = df['start1'] + delta
   ...: df['end2'] = df['start2'] + delta
   ...: df
   ...: 
Out[2]: 
                     start1                    start2                      end1                      end2
0 2013-09-20 13:00:00-04:00 2013-09-20 13:00:00+00:00 2013-09-20 13:00:00-04:00 2013-09-20 13:00:00+00:00
1 2013-09-20 14:40:00-04:00 2013-09-20 14:40:00+00:00 2013-09-20 14:40:01-04:00 2013-09-20 14:40:01+00:00
2 2013-09-20 16:20:00-04:00 2013-09-20 16:20:00+00:00 2013-09-20 16:20:02-04:00 2013-09-20 16:20:02+00:00
3 2013-09-20 18:00:00-04:00 2013-09-20 18:00:00+00:00 2013-09-20 18:00:03-04:00 2013-09-20 18:00:03+00:00
4 2013-09-20 19:40:00-04:00 2013-09-20 19:40:00+00:00 2013-09-20 19:40:04-04:00 2013-09-20 19:40:04+00:00
5 2013-09-20 21:20:00-04:00 2013-09-20 21:20:00+00:00 2013-09-20 21:20:05-04:00 2013-09-20 21:20:05+00:00
6 2013-09-20 23:00:00-04:00 2013-09-20 23:00:00+00:00 2013-09-20 23:00:06-04:00 2013-09-20 23:00:06+00:00
7 2013-09-21 00:40:00-04:00 2013-09-21 00:40:00+00:00 2013-09-21 00:40:07-04:00 2013-09-21 00:40:07+00:00
8 2013-09-21 02:20:00-04:00 2013-09-21 02:20:00+00:00 2013-09-21 02:20:08-04:00 2013-09-21 02:20:08+00:00
9 2013-09-21 04:00:00-04:00 2013-09-21 04:00:00+00:00 2013-09-21 04:00:09-04:00 2013-09-21 04:00:09+00:00

@jreback jreback closed this as completed Oct 28, 2016
@jreback jreback removed the Duplicate Report Duplicate issue or pull request label Oct 28, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Timedelta Timedelta data type Timezones Timezone data dtype
Projects
None yet
Development

No branches or pull requests

2 participants