Skip to content

Error in indexing Series and DataFrames with a list: valid index cannot be found #10256

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
cousin333 opened this issue Jun 3, 2015 · 3 comments
Labels
Duplicate Report Duplicate issue or pull request Indexing Related to indexing on series/frames, not to indexes themselves

Comments

@cousin333
Copy link

(Hopefully I did not find this issue in the list, because there was none...) I have experienced the following issue for a while (wonder if it ever worked actually):

I create a Series or a DataFrame with well-defined, floating-point indices. Then I try to slice some rows using a list. For some valid index values I get NaN instead of the value itself. An example:

>>> import numpy as np
>>> import pandas as pd
>>> i = np.linspace(0, 10, 101)    # create a floating-point (i)ndex-range
>>> d = np.random.random(101)      # add some random numbers as (d)ata
>>> s = pd.Series(d, index=i)     # create a Series from the generated data
>>> type(s)
<class 'pandas.core.series.Series'>
>>> s.head()
0.0    0.969683
0.1    0.463655
0.2    0.879788
0.3    0.122202
0.4    0.466166
dtype: float64
>>> xs = [0.8, 1.1, 1.9, 2.2, 3.3]  # we make a list with a few indexes of interest
>>> s[xs]   # try to list Series-values for those indices, but for some we see only NaN!
0.8    0.996011
1.1    0.699921
1.9         NaN
2.2    0.919059
3.3         NaN
dtype: float64
R:\WinPython\python-3.4.3.amd64\lib\site-packages\pandas\core\format.py:2012: RuntimeWarning: invalid value encountered in greater
  has_large_values = (abs_vals > 1e8).any()
R:\WinPython\python-3.4.3.amd64\lib\site-packages\pandas\core\format.py:2013: RuntimeWarning: invalid value encountered in less
  has_small_values = ((abs_vals < 10 ** (-self.digits+1)) &
R:\WinPython\python-3.4.3.amd64\lib\site-packages\pandas\core\format.py:2014: RuntimeWarning: invalid value encountered in greater
  (abs_vals > 0)).any()
>>> s[1:2]   # we can see, that the values are really there, also at index 1.9
1.0    0.905714
1.1    0.699921
1.2    0.105968
1.3    0.809368
1.4    0.227002
1.5    0.825698
1.6    0.911150
1.7    0.027182
1.8    0.046001
1.9    0.456927
2.0    0.592820
dtype: float64
>>> 

The warnings are actually came from the console, the IPython Notebook that I normally use does not even issue them. Same result using e.g. s.loc[xs]. The problem also exists for DataFrames, which can be reproduced in the same manner.

I'm using the latest 64-bit WinPython with Python 3.4.3 and Pandas 0.16.1:

>>> pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 3.4.3.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 42 Stepping 7, GenuineIntel
byteorder: little
LC_ALL: None
LANG: hu_HU

pandas: 0.16.1
nose: 1.3.6
Cython: 0.22
numpy: 1.9.2
scipy: 0.15.1
statsmodels: 0.6.1
IPython: 3.1.0
sphinx: 1.3.1
patsy: 0.3.0
dateutil: 2.4.2
pytz: 2015.2
bottleneck: None
tables: 3.2.0
numexpr: 2.4.3
matplotlib: 1.4.3
openpyxl: None
xlrd: 0.9.3
xlwt: None
xlsxwriter: 0.7.2
lxml: None
bs4: 4.3.2
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.4
pymysql: None
psycopg2: None
@cousin333 cousin333 changed the title Error in indexing Series and DataFrames with a list: valid index did not find Error in indexing Series and DataFrames with a list: valid index cannot be found Jun 3, 2015
@jorisvandenbossche
Copy link
Member

This is a floating point issue:

In [72]: s.index[19]
Out[72]: 1.9000000000000001

In [74]: xs[2]
Out[74]: 1.9

In [76]: s.index[19] == xs[2]
Out[76]: False

See #9817.

@jorisvandenbossche jorisvandenbossche added Indexing Related to indexing on series/frames, not to indexes themselves Duplicate Report Duplicate issue or pull request labels Jun 3, 2015
@jorisvandenbossche jorisvandenbossche added this to the No action milestone Jun 3, 2015
@jorisvandenbossche
Copy link
Member

Closing as a duplicate of #9817

@cousin333
Copy link
Author

I also suspected, that the issue is about floating point numbers. Thanks for linking it to a seemingly unconnected issue, I would never find that one. Actually they really have the same root.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Duplicate Report Duplicate issue or pull request Indexing Related to indexing on series/frames, not to indexes themselves
Projects
None yet
Development

No branches or pull requests

2 participants