Add array object specification

[kgryte]

This PR

• adds specifications for array object methods and attributes.

• updates language to be more verbose (and explicit) for special cases in several element-wise functions.

• only includes magic methods and not non-magic methods (e.g., max, argmax, etc) in deference to the functional API counterparts.

  In general, array object methods are inconsistently supported. Some array libraries (e.g., Torch, MXNet) expose array objects will many, many methods; other array libraries (e.g., TensorFlow) expose array objects with only magic methods; and yet other array libraries expose array objects with an inconsistent grab bag of methods. For now, this proposal sides with exposing a minimal array object API surface, nudging array library authors and consumers toward functional APIs.

• a number of reflective magic methods are inconsistently implemented across array libraries. For completeness and as determined in consortium meetings, this proposal requires that they are implemented.

• currently, the PR includes a number of TODOs. These mainly stem from addressing particular potentially thorny edge cases. For example, shape, ndim, and size attributes introduce issues for graph tensors whose attributes may be unknown/dynamic. Another example is __setitem__ which may be problematic for immutable arrays and is related to previous discussions regarding the out keyword argument.

• Here is a comparative summary of operator support across array libraries. If an array library is listed, this means that I was unable to confirm operator support.

__abs__
__add__
__and__ (MX)
__eq__
__floordiv__ (MX)
__ge__
__getitem__
__gt__
__invert__ (MX)
__le__
__len__
__lshift__ (TF, MX)
__lt__
__matmul__ (MX)
__mod__
__mul__
__ne__
__neg__
__or__ (MX)
__pos__ (TF, PT, MX)
__pow__
__radd__ (CuPy)
__rand__ (PT, MX, CuPy)
__rfloordiv__ (CuPy, MX)
__rlshift__ (TF, CuPy, MX, PT)
__rmod__ (PT, CuPy)
__rmul__ (CuPy)
__ror__ (PT, MX, CuPy)
__rpow__ (CuPy)
__rrshift__ (TF, PT, MX, CuPy)
__rshift__ (TF, MX)
__rsub__ (CuPy)
__rtruediv__ (CuPy)
__rxor__ (PT, CuPy, MX)
__sizeof__ (CuPy)
__sub__
__truediv__
__xor__ (MX)

[rgommers]

As just discussed, this PR will add a paragraph on Python operator support, to make explicit which operators need supporting.

Other operators such as __matmul__ have not been included, as their functional counterpart specifications are still being worked out.

I'd say just include it, and add a TODO for matmul (functional equivalent) if needed.

[kgryte]

Re: operators. I've added a section to the spec.

Re: __matmul__. I've added a placeholder and a TODO.

Re: status. This should be ready for review. Still some TODOs, but these are either related to concerns regarding shape or dependent on the indexing proposal. These TODOs can be resolved in a follow-up PR.

[rgommers]

LGTM modulo the few comments I made.

I'd be inclined to not have separate specifications for __r<op>__ and __i<op>__, but just tables specifying what needs to be supported. The rest is just duplication without much added value.

[rgommers]

Actually I noticed that __ror__, __rrshift__ etc. are included - those are fairly useless, and not currently supported by most array libraries. Given that we do not assume that different types of arrays should be able to interact or that there are such things as "duck arrays", the only things that make sense are scalar <op> array. Ops like __ror__ will never be useful for most libraries, because Python scalars don't make sense there.

EDIT: I can't remember a discussion on this - was there?

EDIT 2: maybe that statement is too strong, scalar <op> array is what's relevant here.

EDIT 3: some operators aren't symmetrical, but still it's completely obvious what the results and behaviour should be from referencing the left-hand dunder method. Removing all content and adding a table will be much easier. For example, that showed quickly that __rrshift__ was in the spec, but __rlshift__ was missing.

[rgommers]

@kgryte I pushed a set of commits. I'll wait with merging to make it easier to revert one or more in case I made a mistake somewhere.

[rgommers]

Applied last fixes as just discussed @kgryte, and merged. Thanks! Almost there:)