TST: GroupBy.ffill on a multi-index dataframe #55408
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This test checks whether the ffill method works correctly on a multi-index dataframe. In it, we define a dataframe with indexes [0, 1, 2, 0, 1, 2] and column "a" with values [1, 2, NaN, 3, 4, 5]. We group this dataframe by columns (level = 0) and shift it by one, so we have: 0: [1, 3] -> [NaN, 1] 1: [2, 4] -> [NaN, 2] 2: [NaN, 5] -> [NaN, NaN] Then, since index order remain the same, if we apply ffill method, it should give us a dataframe with column "a" equal to [NaN, NaN, NaN, 1, 2, 2]. Co-authored-by: José Lucas Silva Mayer <[email protected]> Co-authored-by: Willian Wang <[email protected]>
mroeschke
reviewed
Oct 4, 2023
pandas/tests/groupby/test_groupby.py
Outdated
| # GH#43412 | ||
| df = DataFrame({"a": [1, 2, np.nan, 3, 4, 5]}, index=[0, 1, 2, 0, 1, 2]) | ||
|
|
||
| result = df.groupby(level=0).shift().ffill() |
There was a problem hiding this comment.
I think the point of the original issue was to test groupby.ffill
There was a problem hiding this comment.
Sure!
I'm testing this and when I use groupby(level=0) and ffill() methods of dataframe, it gives me an error of different types:
def test_groupby_ffill_with_duplicated_index():
# GH#43412
df = DataFrame({"a": [1, 2, 3, 4, np.nan, np.nan]}, index=[0, 1, 2, 0, 1, 2])
result = df.groupby(level=0).ffill()
expected = DataFrame(
{"a": [1, 2, 3, 4, 2, 3]}, index=[0, 1, 2, 0, 1, 2]
)> tm.assert_frame_equal(result, expected)
E AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="a") are different
E
E Attribute "dtype" are different
E [left]: float64
E [right]: int64Is this behaviour expected?
Anyways, I will write the test with a float64 type dataframe.
Signed-off-by: José Lucas Silva Mayer <[email protected]>
…h-duplicated-index
mroeschke
reviewed
Oct 6, 2023
pandas/tests/groupby/test_groupby.py
Outdated
| expected = DataFrame( | ||
| {"a": [1, 2, 3, 4, 2, 3]}, index=[0, 1, 2, 0, 1, 2], dtype=float | ||
| ) | ||
| tm.assert_frame_equal(result, expected) |
There was a problem hiding this comment.
You can specify check_dtype=False here instead of needing to specify float above. i think the result should be int but can be investigated later
Signed-off-by: José Lucas Silva Mayer <[email protected]>
…h-duplicated-index
mroeschke
approved these changes
Oct 9, 2023
|
Thanks @josemayer |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For additional information, we have explained how this new automated test work on first commit description.