Duplicate index with FloatIndex giving 'ValueError: Length mismatch'

[jorisvandenbossche]

Not certain if this is a bug or defined behaviour (but then the error message is not clear in any case).

In 0.13.1:

In [28]: df = pd.DataFrame(np.random.randn(9).reshape(3,3), index=[0.1,0.2,0.2],
 columns=['a','b','c'])
In [29]: df
Out[29]:
            a         b         c
0.1  1.711117  1.218853 -1.322363
0.2  0.956266  0.230374 -1.005935
0.2 -0.137729 -0.993931 -0.902793

In [30]: df.ix[0.2,'a']
Out[30]: array([ 0.95626607, -0.13772877])

In [31]: df.ix[0.2]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
...
ValueError: Length mismatch: Expected axis has 0 elements, new values have 2 ele
ments

In master, both (df.loc[0.2] and df.loc[0.2, 'a']) give this error message. Wile for integer index, this works.

[jreback]

hmm...would say that's a bug (also your first example should return a Series).

[jorisvandenbossche]

about the Series, that is indeed how it is with integer indices

[cpcloud]

hm duplicate fun i'll take this ... @jreback is this a blocker?

[cpcloud]

oh nvm i c u marked as 0.14.1

[jreback]

sure

[cpcloud]

this has been an enlightening experience. @jreback you are a hero for squashing all of these duplicate indexing bugs.

couple of things:

1. i've fixed the ValueError bug, just needed an annoying copypaste of the IndexEngine._maybe_get_bool_indexer with a type change and a subtle slicing KeyError because of duplicates. This should be fused-typed at some point to eliminate the copypasting between the different engine types on this method

2. doesn't look like ix really ever returned a Series for duplicate indices, and that's because it calls self.obj.get_value(*key) which is designed for single element access and directly pulls the ndarray from the underlying index engine.

   BUT

if you remove that line then _getitem_tuple does upcasting and the aptly named TestDataFrame.test_single_element_ix_dont_upcast breaks 😞

i see two ways to deal with this:

1. break the test (bad)
2. don't upcast (deeper into the rabbit hole)

I'll submit a PR for the ValueError and open a separate issue for the ix insanity.