Skip to content

Commit 2172b0f

Browse files
YikunHyukjinKwon
authored andcommitted
[SPARK-38982][PYTHON][PS][TESTS] Skip categories setter test
### What changes were proposed in this pull request? Since pandas-dev/pandas@126a19d, pandas changes behavior. Before pandas 1.4, the pandas will refresh dtypes according to categories, since panda 1.4, `categories.setter` dtype refresh will not work. According to pandas-dev/pandas#46820 , the complete support of `categories.setter` will never back. And also only categories is refreshed (but dtype not) is useless behavior so we'd better to only fix test and keep current PS behavior, then remove this setter support when we remove all deprecated methods. ### Why are the changes needed? Make CI passed with pandas 1.4.x ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? test_categories_setter passed with 1.3.X and also 1.4.x Closes #36355 from Yikun/SPARK-38982. Authored-by: Yikun Jiang <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent 4f567f4 commit 2172b0f

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

python/pyspark/pandas/tests/indexes/test_category.py

+5-1
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,11 @@ def test_categories_setter(self):
8282

8383
pidx.categories = ["z", "y", "x"]
8484
psidx.categories = ["z", "y", "x"]
85-
if LooseVersion(pd.__version__) >= LooseVersion("1.1"):
85+
# Pandas deprecated all the in-place category-setting behaviors, dtypes also not be
86+
# refreshed in categories.setter since Pandas 1.4+, we should also consider to clean up
87+
# this test when in-place category-setting removed:
88+
# https://github.com/pandas-dev/pandas/issues/46820
89+
if LooseVersion("1.4") >= LooseVersion(pd.__version__) >= LooseVersion("1.1"):
8690
self.assert_eq(pidx, psidx)
8791
self.assert_eq(pdf, psdf)
8892
else:

0 commit comments

Comments
 (0)