Skip to content

Rolling.cov with offset window is expanding not rolling #16058

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hgilde opened this issue Apr 19, 2017 · 3 comments · Fixed by #16244
Closed

Rolling.cov with offset window is expanding not rolling #16058

hgilde opened this issue Apr 19, 2017 · 3 comments · Fixed by #16244
Labels
Bug Datetime Datetime data dtype Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Milestone

Comments

@hgilde
Copy link

hgilde commented Apr 19, 2017

Code Sample, a copy-pastable example if possible

import pandas as pd
import numpy as np

idx = pd.date_range('2017-01-01', periods=24, freq='1h')
ss = pd.Series(np.arange(len(idx)), index=idx)

(ss.rolling('2h').cov() == ss.expanding().cov()).iloc[2:].any() # this should be False!!

Problem description

When using Rolling.cov with an offset window like '2h' the window is expanding and not rolling

Expected Output

Rolling should not be equal to expanding cov - but it is equal.

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.5.2.final.0 python-bits: 64 OS: Windows OS-release: 7 machine: AMD64 processor: Intel64 Family 6 Model 45 Stepping 7, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None

pandas: 0.19.2
nose: 1.3.7
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.25.1
numpy: 1.11.3
scipy: 0.18.1
statsmodels: 0.8.0
xarray: None
IPython: 5.1.0
sphinx: 1.4.1
patsy: 0.4.1
dateutil: 2.5.3
pytz: 2016.6.1
blosc: None
bottleneck: None
tables: 3.2.2
numexpr: 2.6.1
matplotlib: 1.5.1
openpyxl: None
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.2
lxml: None
bs4: 4.5.1
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.13
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.42.0
pandas_datareader: None

@jreback
Copy link
Contributor

jreback commented Apr 27, 2017

yeah this is a bug. I don't think we are passing the time-aware window parms thru. instead it looks like we are passing the integer (already converted) ns, so it uses the entire window.

@jreback jreback added Bug Difficulty Intermediate Reshaping Concat, Merge/Join, Stack/Unstack, Explode Datetime Datetime data dtype labels Apr 27, 2017
@jreback jreback added this to the Next Major Release milestone Apr 27, 2017
@keitakurita
Copy link
Contributor

Is passing the converted integer instead of the time-aware window parameters through a deliberate design choice (such as for the sake of avoiding recalculation of the indexes)?

@jreback
Copy link
Contributor

jreback commented May 4, 2017

no, .cov and .corr are implemented in terms of lower moments so the original park should be preserved to be passed

this is an oversight in the impl

@jreback jreback modified the milestones: 0.20.2, Next Major Release May 6, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Datetime Datetime data dtype Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants