Skip to content

BUG: pd.concat does not drop DatetimeIndex.freq when result is monotonic with even spacing #25796

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 26, 2019

Conversation

mroeschke
Copy link
Member

@codecov
Copy link

codecov bot commented Mar 20, 2019

Codecov Report

Merging #25796 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #25796      +/-   ##
==========================================
- Coverage   91.26%   91.26%   -0.01%     
==========================================
  Files         173      173              
  Lines       52982    52983       +1     
==========================================
  Hits        48356    48356              
- Misses       4626     4627       +1
Flag Coverage Δ
#multiple 89.83% <100%> (ø) ⬆️
#single 41.76% <100%> (ø) ⬆️
Impacted Files Coverage Δ
pandas/core/indexes/datetimelike.py 98.52% <100%> (ø) ⬆️
pandas/util/testing.py 89.3% <0%> (-0.11%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4663951...f7e087d. Read the comment docs.

@codecov
Copy link

codecov bot commented Mar 20, 2019

Codecov Report

Merging #25796 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #25796      +/-   ##
==========================================
+ Coverage   91.48%   91.48%   +<.01%     
==========================================
  Files         175      175              
  Lines       52877    52879       +2     
==========================================
+ Hits        48372    48374       +2     
  Misses       4505     4505
Flag Coverage Δ
#multiple 90.04% <100%> (ø) ⬆️
#single 41.83% <100%> (ø) ⬆️
Impacted Files Coverage Δ
pandas/core/indexes/datetimelike.py 98.53% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d37c531...da62bde. Read the comment docs.

@jreback jreback added the Reshaping Concat, Merge/Join, Stack/Unstack, Explode label Mar 20, 2019

# GH 3232: If the concat result is evenly spaced, we can retain the
# original frequency
is_diff_evenly_spaced = len(np.unique(np.diff(new_data))) == 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have something that does this in tseries.frequencies, can you see / use it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked in tseries.frequencies but didn't find anything. We have is_monotonic functions but it doesn't check for even spacing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can use unique_1d here (should be faster), though not a big deal. worth moving this actually to tseries.frequencies as a named function?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks a bit like unique_deltas

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jbrockmendel. Exactly what I needed.

@jreback jreback added this to the 0.25.0 milestone Mar 22, 2019

# GH 3232: If the concat result is evenly spaced, we can retain the
# original frequency
is_diff_evenly_spaced = len(np.unique(np.diff(new_data))) == 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can use unique_1d here (should be faster), though not a big deal. worth moving this actually to tseries.frequencies as a named function?

@jreback
Copy link
Contributor

jreback commented Mar 25, 2019

lgtm. merge on green.

@jreback jreback merged commit 48d76d6 into pandas-dev:master Mar 26, 2019
@jreback
Copy link
Contributor

jreback commented Mar 26, 2019

thanks @mroeschke

@mroeschke mroeschke deleted the freq_concat branch March 26, 2019 00:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pd.concat loses frequency attribute for 'continuous' DataFrame appends
3 participants