Skip to content

pytorch fails signature tests with odd error message #159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
asmeurer opened this issue Jan 10, 2023 · 2 comments · Fixed by #164
Closed

pytorch fails signature tests with odd error message #159

asmeurer opened this issue Jan 10, 2023 · 2 comments · Fixed by #164
Assignees

Comments

@asmeurer
Copy link
Member

This was discussed on slack. There are errors like

FAILED array_api_tests/test_signatures.py::test_func_signature[max] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.uint8 with fill value 0
FAILED array_api_tests/test_signatures.py::test_func_signature[mean] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.float32 with fill value 0.0
FAILED array_api_tests/test_signatures.py::test_func_signature[min] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.uint8 with fill value 0
FAILED array_api_tests/test_signatures.py::test_func_signature[prod] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.uint8 with fill value 0
FAILED array_api_tests/test_signatures.py::test_func_signature[std] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.float32 with fill value 0.0
FAILED array_api_tests/test_signatures.py::test_func_signature[sum] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.uint8 with fill value 0
FAILED array_api_tests/test_signatures.py::test_func_signature[var] - hypothesis.errors.InvalidArgument: Could not create full array of dtype=torch.float32 with fill value 0.0

which are not easy to read

The problem is that hypothesis tries to pass an integer to full, but torch only supports a tuple. I think a simple fix would be if hypothesis only tried passing a tuple to full to avoid this issue.

@asmeurer
Copy link
Member Author

This error actually occurs throughout the test suite, not just in the signature tests.

@honno
Copy link
Member

honno commented Jan 11, 2023

I think a simple fix would be if hypothesis only tried passing a tuple to full to avoid this issue.

Not a big fan of changes like this, where hypothesis.extra.array_api is nominally built with the mindset to happily do whatever the spec allows (such as passing ints as the shape arg). But given this is probably not going to change anytime soon for pytorch, most likely a worthy change beyond just for this test suite, so I'll submit a patch.


For onlookers, ideally we could fallback to using inspect, but alas

so we instead pass arrays for un-inspectable functions. They could be manual examples, but at the time I found it convenient to use Hypothesis, as it could easily translate a functions valid input dtypes and create an array from that... using manual examples could be revisited too, esp. if there's some blocker to changing Hypothesis behaviour.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants