Skip to content

ENH: Checking gaps for time series #59542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1 of 3 tasks
ncooder opened this issue Aug 17, 2024 · 2 comments
Closed
1 of 3 tasks

ENH: Checking gaps for time series #59542

ncooder opened this issue Aug 17, 2024 · 2 comments
Labels
Closing Candidate May be closeable, needs more eyeballs Datetime Datetime data dtype Enhancement

Comments

@ncooder
Copy link

ncooder commented Aug 17, 2024

Feature Type

  • Adding new functionality to pandas

  • Changing existing functionality in pandas

  • Removing existing functionality in pandas

Problem Description

Add built-in schema validation capabilities to pandas, similar to those provided by pandera. It should check time series gaps based on datestamp gaps.

Feature Description

Time series data typically requires continuous timestamps without gaps. Currently, users have to implement their own custom solutions to check for missing periods. A built-in method would be useful to verify the continuity of timestamps. For example, if you have a time series with daily data from 2010-01-01 to 2020-01-01, there shouldn't be any gaps, such as a missing entry for 2019-01-02 or any other date. This method should work for various frequencies, not just daily data.

Alternative Solutions

When gaps are detected, the function should automatically fill them using various interpolation methods or give an error.

Additional Context

No response

@ncooder ncooder added Enhancement Needs Triage Issue that has not been reviewed by a pandas team member labels Aug 17, 2024
@rhshadrach
Copy link
Member

Thanks for the request!

To determine if there are missing values, you can generate a DateTimeIndex that is not missing any and then use .difference(your_index) to determine if any are missing (see this SO question).

When gaps are detected, the function should automatically fill them using various interpolation methods or give an error.

This can be achieved using reindex and interpolate.

This feature request seems duplicative of methods already available in pandas.

@rhshadrach rhshadrach added Closing Candidate May be closeable, needs more eyeballs Datetime Datetime data dtype and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Aug 25, 2024
@mroeschke
Copy link
Member

Agreed, since this can be accomplished with existing APIs, closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Closing Candidate May be closeable, needs more eyeballs Datetime Datetime data dtype Enhancement
Projects
None yet
Development

No branches or pull requests

3 participants