Skip to content

Unexpected output using brackets vs. parents in dataframe.loc #12411

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
AbeHandler opened this issue Feb 22, 2016 · 16 comments
Closed

Unexpected output using brackets vs. parents in dataframe.loc #12411

AbeHandler opened this issue Feb 22, 2016 · 16 comments
Labels
Error Reporting Incorrect or improved errors from pandas Indexing Related to indexing on series/frames, not to indexes themselves
Milestone

Comments

@AbeHandler
Copy link

I have a dataframe, a_dataframe. I want to access the value at index=a, column=0. I can do this successfully with a_dataframe.loc["a", "0"]. However, a_dataframe.loc("a", "0") returns an object! Note: parens, no brackets.

I am new to Pandas, so this was very unexpected behavior in the API. I would think the API should throw an exception or issue a warning. This is a very, very subtle difference in syntax and I think it makes it very easy for new users to get confused.

Code Sample, a copy-pastable example if possible

`self.assertEqual(a_dataframe.loc["a", "0"], a_dataframe.loc("a", "0"))` 

Expected Output

true

output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.10.final.0
python-bits: 64
OS: Darwin
OS-release: 14.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.16.2
nose: 1.3.7
Cython: 0.23.2
numpy: 1.8.0rc1
scipy: 0.13.0b1
statsmodels: 0.6.1
IPython: 4.0.0
sphinx: 1.3.1
patsy: 0.4.1
dateutil: 2.2
pytz: 2013.7
bottleneck: None
tables: None
numexpr: 2.4.6
matplotlib: 1.3.1
openpyxl: 2.2.0-b1
xlrd: 0.9.4
xlwt: None
xlsxwriter: None
lxml: 3.5.0
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.8
pymysql: None
psycopg2: 2.6.1 (dt dec pq3 ext lo64)

@jreback
Copy link
Contributor

jreback commented Feb 22, 2016

This is a standard Python and pandas way of indexing, there is much documentation : http://pandas.pydata.org/pandas-docs/stable/indexing.htm

@jreback jreback added Indexing Related to indexing on series/frames, not to indexes themselves Usage Question labels Feb 22, 2016
@jreback jreback closed this as completed Feb 22, 2016
@nickeubank
Copy link
Contributor

Is that something we can put an exception on? Or is that being used internally?

@jreback
Copy link
Contributor

jreback commented Feb 22, 2016

no

@max-sixty
Copy link
Contributor

To back that up, here are some failures you get when you remove the __call__ method from _NDFrameIndexer, albeit being called from the tests:

======================================================================
ERROR: test_loc_arguments (pandas.tests.test_indexing.TestIndexing)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/workspace/pandas/pandas/tests/test_indexing.py", line 2474, in test_loc_arguments
    result = df.loc(axis=0)['A1':'A3', :, ['C1', 'C3']]
TypeError: '_LocIndexer' object is not callable

======================================================================
ERROR: test_per_axis_per_level_doc_examples (pandas.tests.test_indexing.TestIndexing)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/workspace/pandas/pandas/tests/test_indexing.py", line 2459, in test_per_axis_per_level_doc_examples
    df.loc(axis=0)[:, :, ['C1', 'C3']] = -10
TypeError: '_LocIndexer' object is not callable

======================================================================
ERROR: test_per_axis_per_level_setitem (pandas.tests.test_indexing.TestIndexing)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/workspace/pandas/pandas/tests/test_indexing.py", line 2538, in test_per_axis_per_level_setitem
    df.loc(axis=0)[:, :] = 100
TypeError: '_LocIndexer' object is not callable

@jorisvandenbossche
Copy link
Member

The __call__ method takes *args, **kwargs. It is not possible to only remove the args (it seems they are not used)? This would already raise a error message in this case I think?

@nickeubank
Copy link
Contributor

Or, if *args is never used by the internals, the presence of an *args could give an error suggesting user is confusing parentheses and square brackets.

@max-sixty
Copy link
Contributor

It's not clear that the internals are using __call__ rather than the tests, if anyone wants to dig around further...

@jorisvandenbossche
Copy link
Member

@MaximilianR The use-case you see in the tests that use call like df.loc(axis=0)[:, :, ['C1', 'C3']] is AFAIK perfectly allowed as user code (although not really well documented. EDIT: it is mentioned here: http://pandas.pydata.org/pandas-docs/stable/advanced.html#using-slicers)

@jreback
Copy link
Contributor

jreback commented Feb 22, 2016

yeh, this needs to stay like it is. We may need to pass more arguments

e.g.

df.loc(strict=False)[....] for strict-reindexing vs raising behavior and such

@max-sixty
Copy link
Contributor

@jorisvandenbossche thanks

@jorisvandenbossche
Copy link
Member

@jreback yes, the kwargs are certainly needed, but the args as well?

@jreback
Copy link
Contributor

jreback commented Feb 22, 2016

@jorisvandenbossche yes I suppose we could raise on non-zero len *args, might provide some kind of an error check. ok will reopen

@jreback jreback reopened this Feb 22, 2016
@jreback jreback added Error Reporting Incorrect or improved errors from pandas and removed Usage Question labels Feb 22, 2016
@jreback jreback added this to the 0.18.1 milestone Feb 22, 2016
@jreback
Copy link
Contributor

jreback commented Feb 22, 2016

@AbeHandler want to do a pull-request?

@max-sixty
Copy link
Contributor

yes I suppose we could raise on non-zero len *args, might provide some kind of an error check

Could also just remove the *args kw, although less helpful error message

@jreback
Copy link
Contributor

jreback commented Feb 22, 2016

@MaximilianR right I think in this case, since we are going out-of-the-way to be helpful, a nice error message pointing to the correct syntax would be useful.

@AbeHandler
Copy link
Author

@jreback happy to do a pull request!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Error Reporting Incorrect or improved errors from pandas Indexing Related to indexing on series/frames, not to indexes themselves
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants