Skip to content

Commit 2778c3e

Browse files
ROADMAP: add consistent missing values for all dtypes to the roadmap (pandas-dev#35208)
1 parent 073048f commit 2778c3e

File tree

1 file changed

+26
-0
lines changed

1 file changed

+26
-0
lines changed

doc/source/development/roadmap.rst

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,32 @@ need to implement certain operations expected by pandas users (for example
5353
the algorithm used in, ``Series.str.upper``). That work may be done outside of
5454
pandas.
5555

56+
Consistent missing value handling
57+
---------------------------------
58+
59+
Currently, pandas handles missing data differently for different data types. We
60+
use different types to indicate that a value is missing (``np.nan`` for
61+
floating-point data, ``np.nan`` or ``None`` for object-dtype data -- typically
62+
strings or booleans -- with missing values, and ``pd.NaT`` for datetimelike
63+
data). Integer data cannot store missing data or are cast to float. In addition,
64+
pandas 1.0 introduced a new missing value sentinel, ``pd.NA``, which is being
65+
used for the experimental nullable integer, boolean, and string data types.
66+
67+
These different missing values have different behaviors in user-facing
68+
operations. Specifically, we introduced different semantics for the nullable
69+
data types for certain operations (e.g. propagating in comparison operations
70+
instead of comparing as False).
71+
72+
Long term, we want to introduce consistent missing data handling for all data
73+
types. This includes consistent behavior in all operations (indexing, arithmetic
74+
operations, comparisons, etc.). We want to eventually make the new semantics the
75+
default.
76+
77+
This has been discussed at
78+
`github #28095 <https://github.com/pandas-dev/pandas/issues/28095>`__ (and
79+
linked issues), and described in more detail in this
80+
`design doc <https://hackmd.io/@jorisvandenbossche/Sk0wMeAmB>`__.
81+
5682
Apache Arrow interoperability
5783
-----------------------------
5884

0 commit comments

Comments
 (0)