BUG: numericlike set ops on unsupported Indexes

[jreback]

closes #10038
closes #10039

• CategoricalIndex now raise TypeError on ops like : idx - idx (we are moving all non-numeric Indexes to this, but just deprecated for now on existing index types)
• Index ops with lists will now show the same warning (e.g. idx - ['a','b'])

[shoyer]

This isn't right -- this makes other - index equal to index.difference(other), when it really should be other.difference(index). Maybe __rsub__ should just always raise TypeError? Or I guess it could do _ensure_index(other).difference(index).

[jreback]

oh, I just added __rsub__ for compat, i'll take it out (yes they should prob just always raise though)

[jorisvandenbossche]

this is not correct I think, as now a list will show the warning, but not perform a set (union) operation (as a list is not an index)?

Should it not be like:

if com.is_list_like(other):
    warnings.warn(...)
    return self.union(other)

[jreback]

fixed

[jorisvandenbossche]

@jreback I don't really see how this is fixed? It is still

if com.is_list_like(other):
    warnings.warn(...)
if isinstance(other, Index):
    return self.union(other)

So a list will trigger the warning, but not do the union set operation.

[jorisvandenbossche]

There is also an inconsistency between the behaviour of __add__ and __sub__ (I see now, this was already before this PR).

def __add__(self, other):
    if is_list_like(other):
        warn ...
        return self.union(other)
    return Index(np.array(self) + other)

while

def __sub__(self, other):
    warn ...
    return self.union(other)

Shouldn't this last one also not do the list_like check (so you don't do a set operation when subtracting a scalar)?

I can do a PR if this seems OK

[jreback]

I guess technically the warning is showing up when using __sub__, but then it immediately raises anyhow as you cannot use a non-iterable (while for __add__ you can).

This is concatenation, and IS convenient

In [1]: i = Index(['foo','bar','baz'])

In [2]: i + ' 2'
Out[2]: Index([u'foo 2', u'bar 2', u'baz 2'], dtype='object')

In [3]: i - ' 2'
pandas/core/index.py:1193: FutureWarning: using '-' to provide set differences with Indexes is deprecated, use .difference()
  "use .difference()",FutureWarning)
TypeError: Input must be iterable!

If you want to change __sub__ to not show the warning (even though it raises), ok by me

[jorisvandenbossche]

yes, that one as well.
But the inline comment I made was about this:

In [1]: idx = pd.Index(['a', 'b', 'c', 'd'])

In [2]: idx + ['a', 'e']
pandas\core\index.py:1184: FutureWarning: using '+' to provide set union with In
dexes is deprecated, use '|' or .union()
  "use '|' or .union()",FutureWarning)
---------------------------------------------------------------------------
...
ValueError: operands could not be broadcast together with shapes (4,) (2,)

In [3]: idx + pd.Index(['a', 'e'])
Out[3]: Index([u'a', u'b', u'c', u'd', u'e'], dtype='object')

So for the list, it displays both a warning and an error, and those two contradict each other (and I think it is also more logical to have the same behaviour as with an Index)

[jreback]

@jorisvandenbossche ahh, ok, I see. I think the example I used broadcast correctly, so yes this needs cleaning up. be my guest!