Skip to content

Using tuples as dataframe index, containing single entry makes rows impossible to select #20991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
zilion22 opened this issue May 9, 2018 · 2 comments · Fixed by #29448
Closed
Labels
Bug good first issue Indexing Related to indexing on series/frames, not to indexes themselves Needs Tests Unit test(s) needed to prevent regressions
Milestone

Comments

@zilion22
Copy link

zilion22 commented May 9, 2018

Code Sample

print "-- Good example --"
df=pd.DataFrame([[tuple([1,2])],[tuple([1])]], columns=['a']).set_index('a')
z=df.index[0]
df.loc[[z]]

print "-- Buggy example --"
z=df.index[1]
df.loc[[z]]

Results in:

-- Good example --
a
(1, 2)

-- Buggy example --

KeyErrorTraceback (most recent call last)
in ()
10 print "-- Bug example --"
11 z=df.index[1]
---> 12 df.loc[[z]]

/usr/lib64/python2.7/site-packages/pandas/core/indexing.pyc in getitem(self, key)
1371
1372 maybe_callable = com._apply_if_callable(key, self.obj)
-> 1373 return self._getitem_axis(maybe_callable, axis=axis)
1374
1375 def _is_scalar_access(self, key):

/usr/lib64/python2.7/site-packages/pandas/core/indexing.pyc in _getitem_axis(self, key, axis)
1614 raise ValueError('Cannot index with multidimensional key')
1615
-> 1616 return self._getitem_iterable(key, axis=axis)
1617
1618 # nested tuple slicing

/usr/lib64/python2.7/site-packages/pandas/core/indexing.pyc in _getitem_iterable(self, key, axis)
1113
1114 if self._should_validate_iterable(axis):
-> 1115 self._has_valid_type(key, axis)
1116
1117 labels = self.obj._get_axis(axis)

/usr/lib64/python2.7/site-packages/pandas/core/indexing.pyc in _has_valid_type(self, key, axis)
1470 raise KeyError(
1471 u"None of [{key}] are in the [{axis}]".format(
-> 1472 key=key, axis=self.obj._get_axis_name(axis)))
1473 else:
1474

KeyError: u'None of [[(1,)]] are in the [index]'

Problem description

I wanted to use tuples as an indexer.
However if the tuple contains only one entry, it makes lines to be impossible to select since it brakes pandas.

Expected Output

-- Good example --
a
(1, 2)

-- Buggy example --
a
(1, )

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line]

INSTALLED VERSIONS

commit: None
python: 2.7.5.final.0
python-bits: 64
OS: Linux
OS-release: 3.10.0-693.5.2.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.22.0
pytest: 3.1.2
pip: 10.0.1
setuptools: None
Cython: 0.24
numpy: 1.14.0
scipy: 1.0.0
pyarrow: None
xarray: None
IPython: 5.6.0
sphinx: 1.6.2
patsy: 0.4.1
dateutil: 2.7.2
pytz: 2017.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.0.0
openpyxl: 2.5.0
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.2
lxml: 4.1.1
bs4: None
html5lib: 1.0.1
sqlalchemy: 1.0.13
pymysql: None
psycopg2: 2.6.1 (dt dec pq3 ext)
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@chris-b1
Copy link
Contributor

If it can be avoided you probably don't want to be using tuples as indexes - it won't be as performant and isn't idiomatic - inference causes odd stuff like this to happen.

That said you are welcome to have a look and if you can fix this case without impacting existing tests, would certainly take a PR.

@chris-b1 chris-b1 added Bug Indexing Related to indexing on series/frames, not to indexes themselves Difficulty Intermediate labels May 17, 2018
@chris-b1 chris-b1 added this to the Next Major Release milestone May 17, 2018
@toobaz
Copy link
Member

toobaz commented Jun 29, 2019

This works correctly now, but probably isn't tested

@toobaz toobaz added good first issue Needs Tests Unit test(s) needed to prevent regressions and removed Difficulty Intermediate labels Jun 29, 2019
@gfyoung gfyoung modified the milestones: Contributions Welcome, 1.0 Nov 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug good first issue Indexing Related to indexing on series/frames, not to indexes themselves Needs Tests Unit test(s) needed to prevent regressions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants