Skip to content

BUG: NDArrayBacked2DTests compliance tests from pandas are failing #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tswast opened this issue Jan 27, 2022 · 0 comments · Fixed by #64
Closed

BUG: NDArrayBacked2DTests compliance tests from pandas are failing #65

tswast opened this issue Jan 27, 2022 · 0 comments · Fixed by #64
Assignees
Labels
api: bigquery Issues related to the googleapis/python-db-dtypes-pandas API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@tswast
Copy link
Collaborator

tswast commented Jan 27, 2022

(pip-3.10-prerelease) ➜  python-db-dtypes-pandas git:(issue28-NDArrayBacked2DTests) ✗ pytest tests/unit/test_date_compliance.py
======================================================= test session starts ========================================================
platform darwin -- Python 3.10.0, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /Users/swast/src/github.com/googleapis/python-db-dtypes-pandas
collected 32 items                                                                                                                 

tests/unit/test_date_compliance.py ....................F.....F.....                                                          [100%]

============================================================= FAILURES =============================================================
__________________________________________ Test2DCompat.test_reductions_2d_axis0[median] ___________________________________________

self = <test_date_compliance.Test2DCompat object at 0x11ec56740>
data = <DateArray>
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date(1...1),  datetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
Length: 5620, dtype: dbdate
method = 'median'

    @pytest.mark.parametrize("method", ["mean", "median", "var", "std", "sum", "prod"])
    def test_reductions_2d_axis0(self, data, method):
        arr2d = data.reshape(1, -1)
    
        kwargs = {}
        if method == "std":
            # pass ddof=0 so we get all-zero std instead of all-NA std
            kwargs["ddof"] = 0
    
        try:
>           result = getattr(arr2d, method)(axis=0, **kwargs)

/usr/local/Caskroom/miniconda/base/envs/pip-3.10-prerelease/lib/python3.10/site-packages/pandas/tests/extension/base/dim2.py:194: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <DateArray>
[
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date...atetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
]
Shape: (1, 5620), dtype: dbdate

    def median(
        self,
        *,
        axis: Optional[int] = None,
        out=None,
        overwrite_input: bool = False,
        keepdims: bool = False,
        skipna: bool = True,
    ):
        pandas_backports.numpy_validate_median(
            (),
            {"out": out, "overwrite_input": overwrite_input, "keepdims": keepdims},
        )
        result = pandas_backports.nanmedian(self._ndarray, axis=axis, skipna=skipna)
>       return self._box_func(result)

db_dtypes/core.py:163: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <DateArray>
[
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date...atetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
]
Shape: (1, 5620), dtype: dbdate
x = array(['1900-01-01T00:00:00.000000000', '1900-01-14T00:00:00.000000000',
       '1900-01-27T00:00:00.000000000', ...,
...T00:00:00.000000000', '2099-12-17T00:00:00.000000000',
       '2099-12-30T00:00:00.000000000'], dtype='datetime64[ns]')

    def _box_func(self, x):
>       if pandas.isnull(x):
E       ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

db_dtypes/__init__.py:253: ValueError

During handling of the above exception, another exception occurred:

self = <test_date_compliance.Test2DCompat object at 0x11ec56740>
data = <DateArray>
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date(1...1),  datetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
Length: 5620, dtype: dbdate
method = 'median'

    @pytest.mark.parametrize("method", ["mean", "median", "var", "std", "sum", "prod"])
    def test_reductions_2d_axis0(self, data, method):
        arr2d = data.reshape(1, -1)
    
        kwargs = {}
        if method == "std":
            # pass ddof=0 so we get all-zero std instead of all-NA std
            kwargs["ddof"] = 0
    
        try:
            result = getattr(arr2d, method)(axis=0, **kwargs)
        except Exception as err:
            try:
                getattr(data, method)()
            except Exception as err2:
                assert type(err) == type(err2)
                return
            else:
>               raise AssertionError("Both reductions should raise or neither")
E               AssertionError: Both reductions should raise or neither

/usr/local/Caskroom/miniconda/base/envs/pip-3.10-prerelease/lib/python3.10/site-packages/pandas/tests/extension/base/dim2.py:202: AssertionError
__________________________________________ Test2DCompat.test_reductions_2d_axis1[median] ___________________________________________

self = <DateArray>
[
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date...atetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
]
Shape: (1, 5620), dtype: dbdate
x = array(['1999-12-31T12:00:00.000000000'], dtype='datetime64[ns]')

    def _box_func(self, x):
        if pandas.isnull(x):
            return None
        try:
>           return x.astype("<M8[us]").astype(datetime.datetime).date()
E           AttributeError: 'numpy.ndarray' object has no attribute 'date'

db_dtypes/__init__.py:256: AttributeError

During handling of the above exception, another exception occurred:

self = <test_date_compliance.Test2DCompat object at 0x11f1b3be0>
data = <DateArray>
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date(1...1),  datetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
Length: 5620, dtype: dbdate
method = 'median'

    @pytest.mark.parametrize("method", ["mean", "median", "var", "std", "sum", "prod"])
    def test_reductions_2d_axis1(self, data, method):
        arr2d = data.reshape(1, -1)
    
        try:
>           result = getattr(arr2d, method)(axis=1)

/usr/local/Caskroom/miniconda/base/envs/pip-3.10-prerelease/lib/python3.10/site-packages/pandas/tests/extension/base/dim2.py:239: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <DateArray>
[
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date...atetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
]
Shape: (1, 5620), dtype: dbdate

    def median(
        self,
        *,
        axis: Optional[int] = None,
        out=None,
        overwrite_input: bool = False,
        keepdims: bool = False,
        skipna: bool = True,
    ):
        pandas_backports.numpy_validate_median(
            (),
            {"out": out, "overwrite_input": overwrite_input, "keepdims": keepdims},
        )
        result = pandas_backports.nanmedian(self._ndarray, axis=axis, skipna=skipna)
>       return self._box_func(result)

db_dtypes/core.py:163: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <DateArray>
[
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date...atetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
]
Shape: (1, 5620), dtype: dbdate
x = array(['1999-12-31T12:00:00.000000000'], dtype='datetime64[ns]')

    def _box_func(self, x):
        if pandas.isnull(x):
            return None
        try:
            return x.astype("<M8[us]").astype(datetime.datetime).date()
        except AttributeError:
>           x = numpy.datetime64(x)
E           ValueError: Could not convert object to NumPy datetime

db_dtypes/__init__.py:258: ValueError

During handling of the above exception, another exception occurred:

self = <test_date_compliance.Test2DCompat object at 0x11f1b3be0>
data = <DateArray>
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date(1...1),  datetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
Length: 5620, dtype: dbdate
method = 'median'

    @pytest.mark.parametrize("method", ["mean", "median", "var", "std", "sum", "prod"])
    def test_reductions_2d_axis1(self, data, method):
        arr2d = data.reshape(1, -1)
    
        try:
            result = getattr(arr2d, method)(axis=1)
        except Exception as err:
            try:
                getattr(data, method)()
            except Exception as err2:
                assert type(err) == type(err2)
                return
            else:
>               raise AssertionError("Both reductions should raise or neither")
E               AssertionError: Both reductions should raise or neither

/usr/local/Caskroom/miniconda/base/envs/pip-3.10-prerelease/lib/python3.10/site-packages/pandas/tests/extension/base/dim2.py:247: AssertionError
===================================================== short test summary info ======================================================
FAILED tests/unit/test_date_compliance.py::Test2DCompat::test_reductions_2d_axis0[median] - AssertionError: Both reductions shoul...
FAILED tests/unit/test_date_compliance.py::Test2DCompat::test_reductions_2d_axis1[median] - AssertionError: Both reductions shoul...
=================================================== 2 failed, 30 passed in 2.31s ===================================================
@tswast tswast added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. priority: p2 Moderately-important priority. Fix may not be included in next release. api: bigquery Issues related to the googleapis/python-db-dtypes-pandas API. labels Jan 27, 2022
@tswast tswast self-assigned this Jan 27, 2022
@tswast tswast changed the title BUG: compliance tests from pandas are failing BUG: NDArrayBacked2DTests compliance tests from pandas are failing Mar 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-db-dtypes-pandas API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant