Skip to content

TypeError in Index.get_indexer_non_unique if Index.is_all_dates #19187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
toobaz opened this issue Jan 11, 2018 · 8 comments
Closed

TypeError in Index.get_indexer_non_unique if Index.is_all_dates #19187

toobaz opened this issue Jan 11, 2018 · 8 comments
Labels
good first issue Indexing Related to indexing on series/frames, not to indexes themselves Needs Tests Unit test(s) needed to prevent regressions

Comments

@toobaz
Copy link
Member

toobaz commented Jan 11, 2018

Code Sample, a copy-pastable example if possible

In [2]: idx = pd.DatetimeIndex(['2012-02-01', '2012-02-02']).append(pd.Index(['not date']))[:2]

In [3]: idx.get_indexer_non_unique(['2012-02-01'])
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-df568b5df150> in <module>()
----> 1 idx.get_indexer_non_unique(['2012-02-01'])

/home/pietro/nobackup/repo/pandas/pandas/core/indexes/base.py in get_indexer_non_unique(self, target)
   2797 
   2798         if self.is_all_dates:
-> 2799             self = Index(self.asi8)
   2800             tgt_values = target.asi8
   2801         else:

/home/pietro/nobackup/repo/pandas/pandas/core/indexes/base.py in __new__(cls, data, dtype, copy, name, fastpath, tupleize_cols, **kwargs)
    351                          **kwargs)
    352         elif data is None or is_scalar(data):
--> 353             cls._scalar_data_error(data)
    354         else:
    355             if tupleize_cols and is_list_like(data) and data:

/home/pietro/nobackup/repo/pandas/pandas/core/indexes/base.py in _scalar_data_error(cls, data)
    702         raise TypeError('{0}(...) must be called with a collection of some '
    703                         'kind, {1} was passed'.format(cls.__name__,
--> 704                                                       repr(data)))
    705 
    706     @classmethod

TypeError: Index(...) must be called with a collection of some kind, None was passed

Problem description

Not a great example, but this can bite more reasonable code. It is due to the fact that asi8 is used on a non-DatetimeIndex. In fact, asi8 should really disappear in the non-datetime-specific parts of the code base.

Expected Output

(array([0]), array([]))

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.5.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.0-5-amd64
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: en_GB.UTF-8

pandas: 0.22.0.dev0+448.gdbec3c92e
pytest: 3.0.6
pip: 9.0.1
setuptools: 33.1.1
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.18.1
pyarrow: None
xarray: None
IPython: 5.2.2
sphinx: None
patsy: 0.4.1+dev
dateutil: 2.6.0
pytz: 2016.10
blosc: None
bottleneck: 1.2.0
tables: 3.3.0
numexpr: 2.6.1
feather: 0.3.1
matplotlib: 2.0.0
openpyxl: 2.3.0
xlrd: 1.0.0
xlwt: 1.2.0
xlsxwriter: None
lxml: 3.7.1
bs4: 4.5.3
html5lib: 0.999999999
sqlalchemy: 1.0.15
pymysql: None
psycopg2: None
jinja2: 2.8
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@toobaz toobaz changed the title TypeError in Index.get_indexer_non_unique if Index.all_dates TypeError in Index.get_indexer_non_unique if Index.is_all_dates Jan 13, 2018
@jreback
Copy link
Contributor

jreback commented Jan 13, 2018

hmm, I think we could remove the check from is_all_dates if its object dtype entirely (it should just return False). if we just use inferred type then this is not a problem (not sure how much this might break though).

@jreback
Copy link
Contributor

jreback commented Jan 13, 2018

this is pretty much a result of #19204

@jreback
Copy link
Contributor

jreback commented Jan 13, 2018

so will leave this open, but it is going to be tied to fixing that one.

@jreback jreback added Indexing Related to indexing on series/frames, not to indexes themselves Dtype Conversions Unexpected or buggy dtype conversions Duplicate Report Duplicate issue or pull request labels Jan 13, 2018
@jreback jreback added this to the Next Major Release milestone Jan 13, 2018
@toobaz
Copy link
Member Author

toobaz commented Jan 14, 2018

this is pretty much a result of #19204

Not really (unless I misunderstood your comment there): #19204 affects Timedelta and Period, while this affects only Timestamp. If #19204 is fixed, then this will affect all of them.

@jreback
Copy link
Contributor

jreback commented Jan 14, 2018

my point is we do not want to infer object type and simply use the inferred property (and include for period timedelta and timestamp)

then this issue is moot

@jbrockmendel
Copy link
Member

This appears fixed on master, needs test.

@jbrockmendel jbrockmendel added the Needs Tests Unit test(s) needed to prevent regressions label Feb 12, 2020
@mroeschke mroeschke added good first issue and removed Dtype Conversions Unexpected or buggy dtype conversions Duplicate Report Duplicate issue or pull request Indexing Related to indexing on series/frames, not to indexes themselves labels Apr 3, 2020
@jbrockmendel jbrockmendel added the Indexing Related to indexing on series/frames, not to indexes themselves label Sep 21, 2020
@jbrockmendel
Copy link
Member

is_all_dates has been deprecated, so if this isnt closed by then, we can close it when is_all_dates is removed

@mroeschke
Copy link
Member

Since it looks like is_all_dates will not undergo un-deprecation, I think it should be safe to close this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Indexing Related to indexing on series/frames, not to indexes themselves Needs Tests Unit test(s) needed to prevent regressions
Projects
None yet
Development

No branches or pull requests

4 participants