would prefer to check for duplicates rather than catch an exception

I it worth factoring out the this check from here and _setop to a helper routine

May I add the decorator for the check when more setops in IntervalIndex's own form is implemented?

if you can do it here would be great; this is pretty common code

this is really complex can u split to a searate function (each case)

_​intersection_unique and _intersection_non_unique works in an opposite way.

_intersecton_unique gathers the matches on both hand sides. _intersection_non_unique eliminates the unmatched on the left hand side first and then the unmatched on the right hand side.

Looks like git merge conflict cruft

I know you just copied and pasted, but I think is a great opportunity to move from an OOP class and go functional (and smaller tests) instead per pytest idiom.

@gfyoung Would you give me more guidelines or examples? I don't know what you want.

Absolutely: pytest prefers that you define test functions directly instead of via test classes (less overhead). Notice that in a lot of your tests, you don't really use the self parameter. And here's why: you have create_index as an instance method, but it's really a static method (you don't use self anywhere there).

So the first thing you do is move create_index outside (just like your name fixture0 and drop the self parameter from the function definition:

def create_index(closed="right"):
    return IntervalIndex.from_breaks(range(11), closed=closed)

Now let's take a look at test_union. That's a massive test! There's actually about five tests in that one test alone. But we'll get to that step later on (separate comment). Let's focus first making this test "functional". Essentially, what that means is simply moving the test out of your class, and dropping the self parameter from the definition.

@pytest.mark.parametrize(...)
def test_union(closed, sort):
    index = create_index(closed=closed)
    ...   # rest of test as before

This is a functional test because it's just a test function, no separate wrapping by a class definition (it doesn't have a self parameter). Notice that I leverage the create_index function that we just moved out of our test class right above.

BTW, for more info on pytest to get a better feel for how it works:

https://docs.pytest.org/en/latest/
https://docs.pytest.org/en/latest/example/

@gfyoung Thanks for your detail reply.

if you can do it here would be great; this is pretty common code

imports at the top of the module

could move this to a fixture as well (or we may already have this defined in a higher level conftest)

There might be an issue with this approach when dupes are present in self and other. For other index types, such a scenario can result in more dupes being present in the intersection than in self. This behavior looks a bit buggy and inconsistent though, so I'm not sure if we actually want IntervalIndex to be consistent with it.

Some examples of the buggy and inconsistent behavior with Index:

In [2]: idx2 = pd.Index(list('aa')) 
   ...: idx3 = pd.Index(list('aaa')) 
   ...: idx3b = pd.Index(list('baaa'))

In [3]: idx2.intersection(idx3)
Out[3]: Index(['a', 'a', 'a', 'a'], dtype='object')

In [4]: idx3.intersection(idx3)
Out[4]: Index(['a', 'a', 'a'], dtype='object')

In [5]: idx2.intersection(idx3)
Out[5]: Index(['a', 'a', 'a', 'a'], dtype='object')

In [6]: idx2.intersection(idx3b)
Out[6]: Index(['a', 'a', 'a'], dtype='object')

It seems strange that [3] has more dupes present than in either original index but [4] does not. Similarly, it seems like [5] and [6] should be identical, as the presence of a non-intersecting element shouldn't impact the number of dupes returned.

@jreback : Do you know what the expected behavior for intersection with dupes should be? Or if there are any dependencies on the behavior of intersection that would dictate this?

If we treat indexes like multisets, then the intersection should contain the minimum multiplicity of dupes, e.g. idx2.intersection(idx3) and idx3.intersection(idx2) should both have length 2, so you maintain the property of the intersection being a subset of the original indexes.

yeah this is weird as these are set ops

what happens (meaning how much breakage) if

• raise if left and right are not unique
• uniquify left and right

prob need to do this for all set ops

Haven't had time to extensively test this out but I made the two changes you suggested in indexes/base.py for intersection and both resulted in some breakage. Aside some from breakage in the index set ops tests, there was also some breakage in tests/reshape/test_merge.py.