TST: corrwith and tshift in groupby/groupby.transform

[ryankarlos]

This PR tests corrwith and tshift, so not sure if it fully closes the original issue or not, but the other functionsbackfill, pad and cumcount don't seem to be supported in groupby.transform as reported in issues #27472 and #31269.

• closes Functions allowed in DataFrameGroupby.transform are not all tested #27905
• tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff

[simonjayhawkins]

@ryankarlos is there more to add. if not I don't think this closes the issue?

[ryankarlos]

@simonjayhawkins yes Im thinking of adding for tshift and corrwith - anymore ?

[simonjayhawkins]

@simonjayhawkins yes Im thinking of adding for tshift and corrwith

those are explicitly mentioned in the issue, so could add here or seperate PRs and change the OP of this PR to xref instead of closes.

• anymore ?

not sure. @charlesdong1991 ?

[WillAyd]

So there is already a fixture called transformation_func which has all of these; I think should be using that in tests where there is a gap today instead of adding this

[ryankarlos]

Thanlks @WillAyd , Ive now added new tests with the transformation_func fixture.

[ryankarlos]

@jreback @WillAyd thanks for the comments - addressed them now.

[simonjayhawkins]

@ryankarlos there is a test, test_transform_transformation_func on L321 that was added recently. Should these be included there or at least co-located with that test?

[simonjayhawkins]

@simonjayhawkins yes Im thinking of adding for tshift and corrwith - anymore ?

looking at

pandas/pandas/tests/groupby/test_transform.py

Lines 330 to 332 in 53ece70

	if transformation_func in ["pad", "backfill", "tshift", "corrwith", "cumcount"]:
	# These transformation functions are not yet covered in this test
	pytest.xfail("See GH 31269 and GH 31270")

it appears that pad and cumcount need to be tested also.

[ryankarlos]

@simonjayhawkins yes Im thinking of adding for tshift and corrwith - anymore ?

looking at

pandas/pandas/tests/groupby/test_transform.py

Lines 330 to 332 in 53ece70

	if transformation_func in ["pad", "backfill", "tshift", "corrwith", "cumcount"]:
	# These transformation functions are not yet covered in this test
	pytest.xfail("See GH 31269 and GH 31270")

it appears that pad and cumcount need to be tested also.

So these other functions were throwing AttributeError when used with transform which are also mentioned in issues #31269 and #27472

[simonjayhawkins]

this is g.corrwith(df).

should we be testing g.transform("corrwith", df) to address #27905? (which raises AttributeError: 'Series' object has no attribute 'corrwith')

g.corrwith(df) is tested in pandas\tests\groupby\test_groupby.py

[ryankarlos]

so the only instance using corrwith in test_groupby.py is in test_dup_labels_output_shape which doesn't seem to be fully testing the output of corrwith as is done here

[simonjayhawkins]

should this test be testing g.transform("tshift", ...)?

[ryankarlos]

The output of g.transform("tshift", ...) seems to be buggy - ive just raised an issue #32344. Ive added a case for this and marked as pytest.xfail for now.

[simonjayhawkins]

A quick glance, it appears that with the latest changes, the tests are now not testing for expected behavior, since there are bugs that need to be fixed first. Is that correct @charlesdong1991 ?

[ryankarlos]

@simonjayhawkins to be honest, think this original issue may be out of date and superseded by #32293 , #31269 , #27472 and #32344 reporting the associated bugs which need to be fixed first for cumcount, pad, backfill, tshift, corrwith, as seems rest of transform funcs are already tested.

If you want I could just leave the tests for groupby and put them in test_groupby.py - otherwise feel free to close this PR if not relevant anymore ?

[simonjayhawkins]

otherwise feel free to close this PR if not relevant anymore ?

Thanks @ryankarlos for the work here, much appreciated, but I'll close this PR and close the issue too.