Bug: Pandas indexing with at/iat fails with multi-index

[FedericoV]

Split off from: #9250

In [6]:

# Dataframe now:

cols = [('a_col', chr(i+65)) for i in range(5)]
cols.extend([('b_col', chr(i+65)) for i in range(5, 10)])

idx = [('a_row', chr(i+65)) for i in range(5)]
idx.extend([('b_row', chr(i+65)) for i in range(5, 10)])

df = pd.DataFrame(np.linspace(1, 100, 100).reshape(10, 10),
                  index = pd.MultiIndex.from_tuples(idx),
                  columns = pd.MultiIndex.from_tuples(cols))

In [7]:

df
Out[7]:
a_col   b_col
A   B   C   D   E   F   G   H   I   J
a_row   A   1   2   3   4   5   6   7   8   9   10
B   11  12  13  14  15  16  17  18  19  20
C   21  22  23  24  25  26  27  28  29  30
D   31  32  33  34  35  36  37  38  39  40
E   41  42  43  44  45  46  47  48  49  50
b_row   F   51  52  53  54  55  56  57  58  59  60
G   61  62  63  64  65  66  67  68  69  70
H   71  72  73  74  75  76  77  78  79  80
I   81  82  83  84  85  86  87  88  89  90
J   91  92  93  94  95  96  97  98  99  100
In [8]:

df.loc[('a_row', 'A')] # Works
df.loc[:, ('a_col', 'A')] # Works
df.ix[('a_row', 'B')] # Works
Out[8]:
a_col  A    11
       B    12
       C    13
       D    14
       E    15
b_col  F    16
       G    17
       H    18
       I    19
       J    20
Name: (a_row, B), dtype: float64
In [9]:

df.at[('a_row', 'A')] # fails
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-9-6b9462dd2fc9> in <module>()
----> 1 df.at[('a_row', 'A')] # fails

/home/federico/Python_Libraries/pandas/pandas/core/indexing.pyc in __getitem__(self, key)
   1505 
   1506         key = self._convert_key(key)
-> 1507         return self.obj.get_value(*key, takeable=self._takeable)
   1508 
   1509     def __setitem__(self, key, value):

/home/federico/Python_Libraries/pandas/pandas/core/frame.pyc in get_value(self, index, col, takeable)
   1640             return _maybe_box_datetimelike(series.values[index])
   1641 
-> 1642         series = self._get_item_cache(col)
   1643         engine = self.index._engine
   1644         return engine.get_value(series.get_values(), index)

/home/federico/Python_Libraries/pandas/pandas/core/generic.pyc in _get_item_cache(self, item)
   1066         res = cache.get(item)
   1067         if res is None:
-> 1068             values = self._data.get(item)
   1069             res = self._box_item_values(item, values)
   1070             cache[item] = res

/home/federico/Python_Libraries/pandas/pandas/core/internals.pyc in get(self, item, fastpath)
   2847 
   2848             if not isnull(item):
-> 2849                 loc = self.items.get_loc(item)
   2850             else:
   2851                 indexer = np.arange(len(self.items))[isnull(self.items)]

/home/federico/Python_Libraries/pandas/pandas/core/index.pyc in get_loc(self, key)
   4123 
   4124         if not isinstance(key, tuple):
-> 4125             loc = self._get_level_indexer(key, level=0)
   4126             return _maybe_to_slice(loc)
   4127 

/home/federico/Python_Libraries/pandas/pandas/core/index.pyc in _get_level_indexer(self, key, level)
   4359         else:
   4360 
-> 4361             loc = level_index.get_loc(key)
   4362             if level > 0 or self.lexsort_depth == 0:
   4363                 return np.array(labels == loc,dtype=bool)

/home/federico/Python_Libraries/pandas/pandas/core/index.pyc in get_loc(self, key)
   1405         loc : int if unique index, possibly slice or mask if not
   1406         """
-> 1407         return self._engine.get_loc(_values_from_object(key))
   1408 
   1409     def get_value(self, series, key):

/home/federico/Python_Libraries/pandas/pandas/index.so in pandas.index.IndexEngine.get_loc (pandas/index.c:3812)()

/home/federico/Python_Libraries/pandas/pandas/index.so in pandas.index.IndexEngine.get_loc (pandas/index.c:3692)()

/home/federico/Python_Libraries/pandas/pandas/hashtable.so in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12318)()

/home/federico/Python_Libraries/pandas/pandas/hashtable.so in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12269)()

KeyError: 'A'

It also fails on Series with multi-indexing.

[jreback]

I think this is by definition not allowed

the point of iat/at is to provide a fast exec path
however with a multi index its not simple and a direct path is not possible
better to have a nicer error message

[FedericoV]

Ah, fair enough. I guess if I want to directly do 'fast' multi-indexing, I
have to use some of the specific internal private methods.

On Thu, Jan 15, 2015 at 11:29 AM, jreback [email protected] wrote:

I think this is by definition not allowed

the point of iat/at is to provide a fast exec path
however with a multi index its not simple and a direct path is not possible
better to have a nicer error message

—
Reply to this email directly or view it on GitHub
#9259 (comment).

[jreback]

that said, I think this could give a better error message, e.g. the at/iat could be wrapped in a try:except: and then if its a multi-index it could then raise a more informative message?

[mroeschke]

Seems like there hasn't been much interest in this feature over the years so closing