Skip to content

ENH: do not convert mixed-integer type indexes to datetimeindex #3878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 13, 2013
Merged

ENH: do not convert mixed-integer type indexes to datetimeindex #3878

merged 2 commits into from
Jun 13, 2013

Conversation

cpcloud
Copy link
Member

@cpcloud cpcloud commented Jun 13, 2013

closes #3877.

@cpcloud
Copy link
Member Author

cpcloud commented Jun 13, 2013

@jreback any thoughts/opinions here? i'm not sure if i really want this because of eval or because it makes sense, feel free to tell me that i'm too gung-ho :) on the one hand just because datetimeindexes have join precedence over integers they shouldn't then assume that the joined index of integers refer to dates. on the other hand how often (or not) does one have an object index containing mixed integers and datetimes?

@jreback
Copy link
Contributor

jreback commented Jun 13, 2013

hmm....maybe this should be changed a bit to only allow certain types of indicies on a time-series join.

e.g. string, integer (only if time-series convertible)

I would raise I think if inferred_type != object or the TypeError is triggered

of course see what this breaks

@cpcloud
Copy link
Member Author

cpcloud commented Jun 13, 2013

one problem with the second point is that i believe all integers except for reserved ones like nan/inf are time-series convertible. i think mixed-integer datetimeindexes are too ambiguous to just assume that the ints are datetimes thus only this change...i will play around to see if being more strict with joins breaks the entirety of pandas

@cpcloud
Copy link
Member Author

cpcloud commented Jun 13, 2013

sweet only breaks 4 tests 2 of which are trying to convert, easy enough to change, need to look at others

@cpcloud
Copy link
Member Author

cpcloud commented Jun 13, 2013

@jreback legacy support is broken if you raise and only allow strings to be converted. currently it's really only an issue with converting mixed-integer type indexes. unless one of the goals is to raise whenever an object index is being used...thus discouraging users from using it. my change here preserves legacy support and fixes the issue of conversion i can do the raise if u want but is it too soon for htat?

@jreback
Copy link
Contributor

jreback commented Jun 13, 2013

that's fine then
I always want to have general solutions rather that specific related to types but
sometimes unavoidable

@cpcloud
Copy link
Member Author

cpcloud commented Jun 13, 2013

i agree that it's annoying to have to special case this but the int<->datetime mapping makes it necessary when u already have an object index :(

@cpcloud
Copy link
Member Author

cpcloud commented Jun 13, 2013

@jreback anything else u think i should add here? if not should i merge?

@jreback
Copy link
Contributor

jreback commented Jun 13, 2013

go ahead

cpcloud added a commit that referenced this pull request Jun 13, 2013
ENH: do not convert mixed-integer type indexes to datetimeindex
@cpcloud cpcloud merged commit 89e62c0 into pandas-dev:master Jun 13, 2013
@cpcloud cpcloud deleted the index-join-date-casting-fix-3877 branch June 13, 2013 17:44
@cpcloud
Copy link
Member Author

cpcloud commented Jun 13, 2013

@jreback u were right after all, all these issues are related to this ....

@jreback
Copy link
Contributor

jreback commented Jun 13, 2013

the fix or the symtom?

@cpcloud
Copy link
Member Author

cpcloud commented Jun 13, 2013

symptom

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

unexpected join behavior for datetime indexes with integers
2 participants