feat: ARRAY_API_TESTS_MODULE for runtime-defined xp

[lucascolley]

No description provided.

[lucascolley]

looks like this works, https://github.com/lucascolley/pint-array/actions/runs/12550145694/job/34992428347

[tool.pixi.feature.xp-tests.tasks]
# clean array-api-tests dir
clean-xp-tests = { cwd = ".", cmd = "rm -rf array-api-tests" }
# clone array-api-tests
clone-xp-tests.cmd = "git clone https://github.com/lucascolley/array-api-tests.git"
clone-xp-tests.cwd = "."
clone-xp-tests.depends-on = ["clean-xp-tests"]
# checkout array-api-tests commit
checkout-xp-tests.cmd = [
    "git",
    "reset",
    "--hard",
    "4244965fe1cdf1394143f089beac587307f375fa",
    "&&",
    "git",
    "submodule",
    "update",
    "--init",
]
checkout-xp-tests.cwd = "array-api-tests"
checkout-xp-tests.depends-on = ["clone-xp-tests"]
# run tests
xp-tests.cmd = [
    "pytest",
    "-v",
    "-rxXfE",
    "-W",
    # https://github.com/data-apis/array-api-tests/issues/284
    "ignore::UserWarning",
    # https://github.com/data-apis/array-api-tests/issues/329
    "--disable-extension",
    "fft",
    "--disable-extension",
    "linalg",
    "--xfails-file",
    "../xp-tests-xfails.txt",
    "--max-examples=100",
    "--derandomize",
    "--disable-deadline",
    "array_api_tests/",
]
xp-tests.env.ARRAY_API_TESTS_MODULE = "exec('import pint_array, array_api_strict; xp = pint_array.pint_namespace(array_api_strict)')"
xp-tests.cwd = "array-api-tests"
xp-tests.depends-on = ["checkout-xp-tests"]

[ev-br]

Looks nice, clean and self-contained. Addresses a real need. Usual reservations apply to exec-ing arbitrary code on import, but since array-api-tests is not meant to be user-visible anyway, the only user is a developer themselves, so all harm will be self-inflicted.

Using -tests with varying backends is cumbersome enough ATM; I've been told that previously actual usage was heavily relying on setting PYTHONAPTHs. So I suppose this same problem can be solved with dropping a relevant script somewhere, setting a PYTHONPATH so that it's found, and adding a hook to execute to __init__.py. Which will be pretty much the same as what's here now, only more cumbersome still.
Let's give it a whirl and see if the ergonomics is enough or if we'll want something else in the end.

Thanks Lucas!

[lucascolley]

FYI, @mdhaber came up with the idea of making namespaces available via e.g. from marray import numpy, which works with ARRAY_API_TESTS_MODULE=marray.numpy, so this PR isn't actually needed to test marray and pint-array. But it should still be useful if a library does not want to make the namespaces importable like that.

[ev-br]

Well, great to hear. Having an importable module sounds like a right plan TBH. Now that there's no immediate use case, I'd suggest to put this idea on a backburner until an actual use case arrives. Then we'll be able to reassess.

[lucascolley]

well, how about pint_namespace(marray_namespace(numpy))? Can we do that with just imports @mdhaber?

[mdhaber]

I think this would work:

from marray import numpy
from pint import marray.numpy
# pint.marray.numpy is now a module

I'm not at home but you could probably test it with nested marray imports.

[lucascolley]

No luck:

In [1]: from marray import array_api_strict as mxp

In [2]: mxp
Out[2]: <module 'marray.array_api_strict'>

In [3]: from pint_array import array_api_strict as pxp

In [4]: pxp
Out[4]: <module 'pint(array_api_strict)'>

In [5]: from pint_array import marray.array_api_strict as pmxp
  Cell In[5], line 1
    from pint_array import marray.array_api_strict as pmxp
                                 ^
SyntaxError: invalid syntax


In [6]: from pint_array.marray import array_api_strict as pmxp
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[6], line 1
----> 1 from pint_array.marray import array_api_strict as pmxp

ModuleNotFoundError: No module named 'pint_array.marray'

EDIT: we get a bit further the other way around but still not what we want:

In [8]: from marray import pint_array as mp

In [9]: mp
Out[9]: <module 'marray.pint_array'>

In [10]: from marray.pint_array import array_api_strict as mpxp
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[10], line 1
----> 1 from marray.pint_array import array_api_strict as mpxp

ImportError: cannot import name 'array_api_strict' from 'marray.pint_array' (unknown location)

[mdhaber]

Ok. I think we can make something work with a code adjustment.

from x import y calls the __getattr__ function of module x with the argument "y" (as a string). We need the code to take the string "y" and resolve it as a reference to a module so it can be passed to pint_namespace or whatever.

This is currently done with importlib assuming 1) import_module("y") works and 2) "y" satisfies the syntax requirements of from x import y (e.g. cannot include a .).

Since from x import y.z is a syntax violation:

from scipy import optimize.elementwise
#   Cell In[16], line 1
#     from scipy import optimize.elementwise
#                               ^
# SyntaxError: invalid syntax

it will probably need to look like:

from marray import array_api_strict as mxp
from pint_array import mxp as pmxp  # not from `pint_array import marray.array_api_strict...`

The current implementation will try importlib.importmodule("mxp"), and I think that will fail. When it does, we can try resolving "mxp" as an existing variable , e.g. vars()['mxp'], then passing this to pint_namespace.

I will probably not work on it immediately, though, so I'd welcome someone else investigating, and I'd be happy to review a PR to marray to make the getattr implementation compatible with this scheme.

[lucascolley]

Right. In the meantime, I'd suggest merging this PR to make it doable now.

[lucascolley]

In the meantime, I'd suggest merging this PR to make it doable mow.

Going ahead with this!

[lucascolley]

do the build checks need approval from someone with admin rights @rgommers ?

[rgommers]

do the build checks need approval from someone with admin rights @rgommers ?

No, they were misconfigured, the jobs don't actually exist anymore but they were explicitly required:

I changed it to this, let's see if that is more future-proof: