Skip to content

Commit a6df8dd

Browse files
bjornjorgensenHyukjinKwon
authored andcommitted
[SPARK-43239][PS] Remove null_counts from info()
### What changes were proposed in this pull request? Remove `null_counts` from info() ### Why are the changes needed? Pandas 2.0 _Removed deprecated null_counts argument in [DataFrame.info()](https://pandas.pydata.org/pandas-docs/version/2.0/reference/api/pandas.DataFrame.info.html#pandas.DataFrame.info). Use show_counts instead ([GH37999](pandas-dev/pandas#37999 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Tested local ### Before this PR `F05.info()` ``` TypeError Traceback (most recent call last) Cell In[12], line 1 ----> 1 F05.info() File /opt/spark/python/pyspark/pandas/frame.py:12167, in DataFrame.info(self, verbose, buf, max_cols, null_counts) 12163 count_func = self.count 12164 self.count = ( # type: ignore[assignment] 12165 lambda: count_func()._to_pandas() # type: ignore[assignment, misc, union-attr] 12166 ) > 12167 return pd.DataFrame.info( 12168 self, # type: ignore[arg-type] 12169 verbose=verbose, 12170 buf=buf, 12171 max_cols=max_cols, 12172 memory_usage=False, 12173 null_counts=null_counts, 12174 ) 12175 finally: 12176 del self._data TypeError: DataFrame.info() got an unexpected keyword argument 'null_counts' ``` ### With this PR `F05.info()` ``` <class 'pyspark.pandas.frame.DataFrame'> Int64Index: 5257 entries, 0 to 5256 Data columns (total 203 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 DOFFIN_APPENDIX:EXPRESSION_OF_INTEREST_URL 471 non-null object (...) ``` Closes #40913 from bjornjorgensen/remove-null_counts. Authored-by: bjornjorgensen <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent 84d9461 commit a6df8dd

File tree

1 file changed

+0
-2
lines changed

1 file changed

+0
-2
lines changed

python/pyspark/pandas/frame.py

-2
Original file line numberDiff line numberDiff line change
@@ -12057,7 +12057,6 @@ def info(
1205712057
verbose: Optional[bool] = None,
1205812058
buf: Optional[IO[str]] = None,
1205912059
max_cols: Optional[int] = None,
12060-
null_counts: Optional[bool] = None,
1206112060
) -> None:
1206212061
"""
1206312062
Print a concise summary of a DataFrame.
@@ -12170,7 +12169,6 @@ def info(
1217012169
buf=buf,
1217112170
max_cols=max_cols,
1217212171
memory_usage=False,
12173-
null_counts=null_counts,
1217412172
)
1217512173
finally:
1217612174
del self._data

0 commit comments

Comments
 (0)