Skip to content

TST 56 tests in pandas/tests/test_generic.py aren't being tested #50380

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MarcoGorelli opened this issue Dec 21, 2022 · 7 comments · Fixed by #50636
Closed

TST 56 tests in pandas/tests/test_generic.py aren't being tested #50380

MarcoGorelli opened this issue Dec 21, 2022 · 7 comments · Fixed by #50636
Assignees
Labels
Needs Tests Unit test(s) needed to prevent regressions

Comments

@MarcoGorelli
Copy link
Member

MarcoGorelli commented Dec 21, 2022

All these tests aren't actually being run:

class Generic:
@pytest.mark.parametrize(
"func",
[
str.lower,
{x: x.lower() for x in list("ABCD")},
Series({x: x.lower() for x in list("ABCD")}),
],
)
def test_rename(self, frame_or_series, func):
# single axis
idx = list("ABCD")
for axis in frame_or_series._AXIS_ORDERS:
kwargs = {axis: idx}
obj = construct(4, **kwargs)
# rename a single axis
result = obj.rename(**{axis: func})
expected = obj.copy()
setattr(expected, axis, list("abcd"))
tm.assert_equal(result, expected)
def test_get_numeric_data(self, frame_or_series):
n = 4
kwargs = {
frame_or_series._get_axis_name(i): list(range(n))
for i in range(frame_or_series._AXIS_LEN)
}
# get the numeric data
o = construct(n, **kwargs)
result = o._get_numeric_data()
tm.assert_equal(result, o)
# non-inclusion
result = o._get_bool_data()
expected = construct(n, value="empty", **kwargs)
if isinstance(o, DataFrame):
# preserve columns dtype
expected.columns = o.columns[:0]
tm.assert_equal(result, expected)
# get the bool data
arr = np.array([True, True, False, True])
o = construct(n, value=arr, **kwargs)
result = o._get_numeric_data()
tm.assert_equal(result, o)
def test_nonzero(self, frame_or_series):
# GH 4633
# look at the boolean/nonzero behavior for objects
obj = construct(frame_or_series, shape=4)
msg = f"The truth value of a {frame_or_series.__name__} is ambiguous"
with pytest.raises(ValueError, match=msg):
bool(obj == 0)
with pytest.raises(ValueError, match=msg):
bool(obj == 1)
with pytest.raises(ValueError, match=msg):
bool(obj)
obj = construct(frame_or_series, shape=4, value=1)
with pytest.raises(ValueError, match=msg):
bool(obj == 0)
with pytest.raises(ValueError, match=msg):
bool(obj == 1)
with pytest.raises(ValueError, match=msg):
bool(obj)
obj = construct(frame_or_series, shape=4, value=np.nan)
with pytest.raises(ValueError, match=msg):
bool(obj == 0)
with pytest.raises(ValueError, match=msg):
bool(obj == 1)
with pytest.raises(ValueError, match=msg):
bool(obj)
# empty
obj = construct(frame_or_series, shape=0)
with pytest.raises(ValueError, match=msg):
bool(obj)
# invalid behaviors
obj1 = construct(frame_or_series, shape=4, value=1)
obj2 = construct(frame_or_series, shape=4, value=1)
with pytest.raises(ValueError, match=msg):
if obj1:
pass
with pytest.raises(ValueError, match=msg):
obj1 and obj2
with pytest.raises(ValueError, match=msg):
obj1 or obj2
with pytest.raises(ValueError, match=msg):
not obj1
def test_frame_or_series_compound_dtypes(self, frame_or_series):
# see gh-5191
# Compound dtypes should raise NotImplementedError.
def f(dtype):
return construct(frame_or_series, shape=3, value=1, dtype=dtype)
msg = (
"compound dtypes are not implemented "
f"in the {frame_or_series.__name__} frame_or_series"
)
with pytest.raises(NotImplementedError, match=msg):
f([("A", "datetime64[h]"), ("B", "str"), ("C", "int32")])
# these work (though results may be unexpected)
f("int64")
f("float64")
f("M8[ns]")
def test_metadata_propagation(self, frame_or_series):
# check that the metadata matches up on the resulting ops
o = construct(frame_or_series, shape=3)
o.name = "foo"
o2 = construct(frame_or_series, shape=3)
o2.name = "bar"
# ----------
# preserving
# ----------
# simple ops with scalars
for op in ["__add__", "__sub__", "__truediv__", "__mul__"]:
result = getattr(o, op)(1)
tm.assert_metadata_equivalent(o, result)
# ops with like
for op in ["__add__", "__sub__", "__truediv__", "__mul__"]:
result = getattr(o, op)(o)
tm.assert_metadata_equivalent(o, result)
# simple boolean
for op in ["__eq__", "__le__", "__ge__"]:
v1 = getattr(o, op)(o)
tm.assert_metadata_equivalent(o, v1)
tm.assert_metadata_equivalent(o, v1 & v1)
tm.assert_metadata_equivalent(o, v1 | v1)
# combine_first
result = o.combine_first(o2)
tm.assert_metadata_equivalent(o, result)
# ---------------------------
# non-preserving (by default)
# ---------------------------
# add non-like
result = o + o2
tm.assert_metadata_equivalent(result)
# simple boolean
for op in ["__eq__", "__le__", "__ge__"]:
# this is a name matching op
v1 = getattr(o, op)(o)
v2 = getattr(o, op)(o2)
tm.assert_metadata_equivalent(v2)
tm.assert_metadata_equivalent(v1 & v2)
tm.assert_metadata_equivalent(v1 | v2)
def test_size_compat(self, frame_or_series):
# GH8846
# size property should be defined
o = construct(frame_or_series, shape=10)
assert o.size == np.prod(o.shape)
assert o.size == 10 ** len(o.axes)
def test_split_compat(self, frame_or_series):
# xref GH8846
o = construct(frame_or_series, shape=10)
assert len(np.array_split(o, 5)) == 5
assert len(np.array_split(o, 2)) == 2
# See gh-12301
def test_stat_unexpected_keyword(self, frame_or_series):
obj = construct(frame_or_series, 5)
starwars = "Star Wars"
errmsg = "unexpected keyword"
with pytest.raises(TypeError, match=errmsg):
obj.max(epic=starwars) # stat_function
with pytest.raises(TypeError, match=errmsg):
obj.var(epic=starwars) # stat_function_ddof
with pytest.raises(TypeError, match=errmsg):
obj.sum(epic=starwars) # cum_function
with pytest.raises(TypeError, match=errmsg):
obj.any(epic=starwars) # logical_function
@pytest.mark.parametrize("func", ["sum", "cumsum", "any", "var"])
def test_api_compat(self, func, frame_or_series):
# GH 12021
# compat for __name__, __qualname__
obj = (frame_or_series, 5)
f = getattr(obj, func)
assert f.__name__ == func
assert f.__qualname__.endswith(func)
def test_stat_non_defaults_args(self, frame_or_series):
obj = construct(frame_or_series, 5)
out = np.array([0])
errmsg = "the 'out' parameter is not supported"
with pytest.raises(ValueError, match=errmsg):
obj.max(out=out) # stat_function
with pytest.raises(ValueError, match=errmsg):
obj.var(out=out) # stat_function_ddof
with pytest.raises(ValueError, match=errmsg):
obj.sum(out=out) # cum_function
with pytest.raises(ValueError, match=errmsg):
obj.any(out=out) # logical_function
def test_truncate_out_of_bounds(self, frame_or_series):
# GH11382
# small
shape = [2000] + ([1] * (frame_or_series._AXIS_LEN - 1))
small = construct(frame_or_series, shape, dtype="int8", value=1)
tm.assert_equal(small.truncate(), small)
tm.assert_equal(small.truncate(before=0, after=3e3), small)
tm.assert_equal(small.truncate(before=-1, after=2e3), small)
# big
shape = [2_000_000] + ([1] * (frame_or_series._AXIS_LEN - 1))
big = construct(frame_or_series, shape, dtype="int8", value=1)
tm.assert_equal(big.truncate(), big)
tm.assert_equal(big.truncate(before=0, after=3e6), big)
tm.assert_equal(big.truncate(before=-1, after=2e6), big)
@pytest.mark.parametrize(
"func",
[copy, deepcopy, lambda x: x.copy(deep=False), lambda x: x.copy(deep=True)],
)
@pytest.mark.parametrize("shape", [0, 1, 2])
def test_copy_and_deepcopy(self, frame_or_series, shape, func):
# GH 15444
obj = construct(frame_or_series, shape)
obj_copy = func(obj)
assert obj_copy is not obj
tm.assert_equal(obj_copy, obj)

that's because the class name is Generic, rather than TestGeneric

They're all uncovered in the coverage report: https://app.codecov.io/gh/pandas-dev/pandas/blob/main/pandas/tests/generic/test_generic.py

Similar to #50378, the class needs renaming, and the tests fixing up

@MarcoGorelli MarcoGorelli added the Needs Tests Unit test(s) needed to prevent regressions label Dec 21, 2022
@MarcoGorelli MarcoGorelli changed the title TST 300 lines of tests in pandas/tests/test_generic.py aren't being tested TST 56 tests in pandas/tests/test_generic.py aren't being tested Dec 21, 2022
@phershbe
Copy link
Contributor

take

@phershbe
Copy link
Contributor

@MarcoGorelli I tried take but I didn't realize that that was exclusively for good first issue. I'm interested in working on this. I've changed the class name and run the tests and got 20 failing, but many of the error messages are similar to each other. I'm digging in now to understand what they do, the error messages are helpful obviously.

@MarcoGorelli
Copy link
Member Author

🤔 I thought take worked for any issue, not sure what happened here - consider it yours anyway

and thanks for working on this!

@phershbe
Copy link
Contributor

@MarcoGorelli I'm working on it and learning a lot. I wasn't sure that I knew enough programming to get it, but after some digging, things are starting to make sense. I learned what the parametrize decorator does, where frame_or_series was coming from and then what conftest.py is what a fixture does, et cetera. I'm going to work on it a lot today too, if I start getting really stuck I'll let you know.

@phershbe
Copy link
Contributor

phershbe commented Jan 3, 2023

@MarcoGorelli I've spent a long time on this and I've learned a ton and can read the code in the tests much better now, and understand much more about Pandas and even NumPy.

I was writing here to ask some questions and to say that I know that I can't spend forever on this and that I would unassign soon if I was still working through things, but then I got test_rename to pass by adding frame_or_series as the box parameter in construct on line 70. I'm learning a lot about how to understand unfamiliar code by breaking it down into many small pieces, which is much harder than the tutorials that I was doing and simple projects that I was making, obviously.

I don't know if the test is correct now, but I'm very encouraged that I at least got it to pass. If it's okay, let me spend the rest of the day to try to get the other tests to pass, and I'll get a pull request in tomorrow morning or leave a message and somebody else can do it if necessary. Thank you.

@MarcoGorelli
Copy link
Member Author

hey @phershbe

thanks for the update! there's no hurry here, and it's OK to submit a PR which only fix some of the tests and xfails others

@phershbe
Copy link
Contributor

phershbe commented Jan 4, 2023

@MarcoGorelli Awesome, thank you. I got the first and second of the five tests to past. I'm on the third now. I should be done soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Tests Unit test(s) needed to prevent regressions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants