pd.NA TypeError in drop_duplicates with object dtype

[WillAyd]

Code Sample, a copy-pastable example if possible

>>> pd.DataFrame([[1, pd.NA], [2, "a"]]).drop_duplicates()
Traceback (most recent call last):
   ...
  File "/Users/williamayd/miniconda3/envs/sitka/lib/python3.8/site-packages/pandas/core/frame.py", line 4859, in f
    labels, shape = algorithms.factorize(
  File "/Users/williamayd/miniconda3/envs/sitka/lib/python3.8/site-packages/pandas/core/algorithms.py", line 629, in factorize
    codes, uniques = _factorize_array(
  File "/Users/williamayd/miniconda3/envs/sitka/lib/python3.8/site-packages/pandas/core/algorithms.py", line 478, in _factorize_array
    uniques, codes = table.factorize(values, na_sentinel=na_sentinel, na_value=na_value)
  File "pandas/_libs/hashtable_class_helper.pxi", line 1806, in pandas._libs.hashtable.PyObjectHashTable.factorize
  File "pandas/_libs/hashtable_class_helper.pxi", line 1728, in pandas._libs.hashtable.PyObjectHashTable._unique
  File "pandas/_libs/missing.pyx", line 360, in pandas._libs.missing.NAType.__bool__
TypeError: boolean value of NA is ambiguous

This same failure isn't present when using an extension type:

>>> df = pd.DataFrame([[1, pd.NA], [2, "a"]], columns=list("ab"))
>>> df["b"] = df["b"].astype("string")
>>> df.drop_duplicates()
   a     b
0  1  <NA>
1  2     a

[AnnaDaglis]

I cannot reproduce the error. Has it been fixed already?

[WillAyd]

Thanks @AnnaDaglis and well spotted. @jbrockmendel any idea where this might have been fixed?

[jbrockmendel]

any idea where this might have been fixed?

no idea off the top of my head.

[JochenFromm]

I can not reproduce it either, but it could be related to #15752

[simonjayhawkins]

Thanks @AnnaDaglis and well spotted. @jbrockmendel any idea where this might have been fixed?

fixed in #31939 (i.e. 1.0.2)

41bc226 is the first new commit
commit 41bc226
Author: Daniel Saxton [email protected]
Date: Sun Feb 23 08:57:07 2020 -0600

BUG: Fix construction of Categorical from pd.NA (#31939)