BUG: raise on RangeIndex.array

[jbrockmendel]

• closes #xxxx
• tests added / passed
• Ensure all linting tests pass, see here for how to run them
• whatsnew entry

xerf #37277 cc @TomAugspurger this is failing one of the dask tests locally, looks like dask is calling index.array. Thoughts on how to handle this?

Might be worth folding the RangeIndex check into a keyword in extract_array.

[jreback]

Might be worth folding the RangeIndex check into a keyword in extract_array.

yes please as we already special case MI there.

[jreback]

yeah let's fold this into extract_array

[TomAugspurger]

This can and should go through a deprecation warning IMO.

[jorisvandenbossche]

This can and should go through a deprecation warning IMO.

Agreed.

In addition, are we sure that we actually want to raise an error? Eg .values also works. And if for some reason you need an array, it seems a bit annoying to need to check for RangeIndex.

(the public .array API can be different than extract_array)

[jbrockmendel]

This can and should go through a deprecation warning IMO.

Agreed.

In addition, are we sure that we actually want to raise an error? Eg .values also works. And if for some reason you need an array, it seems a bit annoying to need to check for RangeIndex.

The only strong opinion i have on this is that RangeIndex behavior should match MultiIndex behavior.

[TomAugspurger]

In addition, are we sure that we actually want to raise an error?

I'm not sure. I think it's surprising that RangeIndex.array allocates memory though. I suppose we could implement a RangeArray extension array? And it wouldn't implement __setitem__, so we can support the use case where a user wants read-only access to the array, but defer raising till they do an unsupported operation?

[jorisvandenbossche]

There are many operations on RangeIndex that do allocate memory though. For me, I see RangeIndex a bit like Int64Index but without materializing "as long as possible". And when asking for its array representation, it is no longer possible.

The only strong opinion i have on this is that RangeIndex behavior should match MultiIndex behavior.

@jbrockmendel Can you explain why you want this? A practical reason? Or conceptually since both are not directly backed by a single array? (on which I would say: that's true, but RangeIndex still "represents" a single array, while MultiIndex does not. So I think there are also reasons to give them different behaviour)

[jbrockmendel]

I suppose we could implement a RangeArray extension array?

id like to see this. among other things it would allows set_index(rangecol).reset_index() to round-trip nicely

The only strong opinion i have on this is that RangeIndex behavior should match MultiIndex behavior.

@jbrockmendel Can you explain why you want this? A practical reason? Or conceptually since both are not directly backed by a single array? (on which I would say: that's true, but RangeIndex still "represents" a single array, while MultiIndex does not. So I think there are also reasons to give them different behaviour)

(At least until there is a RangeArray) RangeIndex doesn't represent a single array. As you mentioned above, it has .values, but if that is the standard, then we should do the same for MultiIndex.

(I actually dont like the .array property in most cases, pretty much always find ._values more useful)

I am more concerned with extract_array than with .array, and extract_array definitely shouldn't be allocating an entirely new array.

[jorisvandenbossche]

I am more concerned with extract_array than with .array, and extract_array definitely shouldn't be allocating an entirely new array.

Yes, as mentioned above, IMO the two don't necessarily need to do the same. I certainly follow that extract_array shouldn't allocate a new array.

(I actually dont like the .array property in most cases, pretty much always find ._values more useful)

The .array us public API, and being able to use array-like values (eg to circumvent alignment) reliably is nice. For this use case, having RangeIndex raise seems more annoying than useful? (of course, you still have the memory allocation you might not expect)

I suppose we could implement a RangeArray extension array?

id like to see this. among other things it would allows set_index(rangecol).reset_index() to round-trip nicely

Let's open a separate issue to discuss this if people would like to see this.
(I am personally not convinced that we should add it: if we have it, it needs to be a proper dtype and allowed in columns, and IMO it is not needed to support a "range" dtype)

[pep8speaks]

Hello @jbrockmendel! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-04-07 17:00:09 UTC

[jbrockmendel]

Should be ready for another look

[jorisvandenbossche]

Apart from #40022 (comment), not further comments from my side

[jreback]

can you merge master. i will look at this soon.

[jbrockmendel]

rebased + greenish

[jreback]

can you rebase one more time, ping on green

[jreback]

hmm if you use extract_range on L376 i think you can just fall thru no? (or I guess need L380 to change), anyhow for the future

[jreback]

also do you need a whatnew? e.g. what exactly is the bug

[jbrockmendel]

also do you need a whatnew? e.g. what exactly is the bug

there shouldnt be anything user-facing