[skip ci] First revision

Author: MarcoGorelli

web/pandas/pdeps/0005-no-default-index-mode.md

@@ -62,7 +62,7 @@ Then:
   dtype: int64
   ```
 
-If a user didn't want to think about row labels (which they may have ended up after slicing some other ``DataFrame``/``Series``),
+If a user didn't want to think about row labels (which they may have ended up after slicing / concatenating operations),
 then ``NoRowIndex`` would enable the above to work in a more intuitive
 manner (details and examples to follow below).
 
@@ -107,27 +107,28 @@ result in a ``DataFrame`` which still has ``NoRowIndex``. To do this, the follow
 Example:
 
 ```python
-In [8]: df = pd.DataFrame({'a': [1,  2], 'b': [4, 5]}, index=NoRowIndex(2))
+In [7]: df1 = pd.DataFrame({'a': [1,  2], 'b': [4, 5]}, index=NoRowIndex(2))
 
-In [9]: df
+In [8]: df2 = pd.DataFrame({'a': [4], 'b': [0]}, index=NoRowIndex(1))
+
+In [9]: df1
 Out[9]:
  a  b
  1  4
  2  5
 
-In [10]: pd.concat([df, df])
+In [10]: pd.concat([df1, df2])
 Out[10]:
  a  b
  1  4
  2  5
- 1  4
- 2  5
+ 4  0
 
-In [11]: pd.concat([df, df]).index
-Out[11]: NoRowIndex(len=4)
+In [11]: pd.concat([df1, df2]).index
+Out[11]: NoRowIndex(len=3)
 ```
 
-Appending anything index other than another ``NoRowIndex`` would raise.
+Appending anything other than another ``NoRowIndex`` would raise.
 
 ### Slicing a ``NoRowIndex``
 
@@ -158,13 +159,14 @@ In [15]: df.loc[0, 'b']
 ---------------------------------------------------------------------------
 IndexError: Cannot use label-based indexing on NoRowIndex!
 ```
+Note that other uses of ``.loc``, such as boolean masks, would still be allowed (see F.A.Q).
 
 ### Aligning ``NoRowIndex``s
 
 To minimise surprises, the rule would be:
 
-  A ``NoRowIndex`` can only be aligned with another ``NoRowIndex`` of the same length.
-  Attempting to align it with anything else would raise.
+> A ``NoRowIndex`` can only be aligned with another ``NoRowIndex`` of the same length.
+> Attempting to align it with anything else would raise.
 
 Example:
 ```python
@@ -192,13 +194,13 @@ row labels. There's no suggestion to remove the column labels.
 In particular, calling ``transpose`` on a ``NoRowIndex`` ``DataFrame``
 would error. The error would come with a helpful error message, informing
 users that they should first set an index. E.g.:
-```
+```python
 In [4]: df = pd.DataFrame({'a': [1,  2, 3], 'b': [4, 5, 6]}, index=NoRowIndex(3))
 
 In [5]: df.transpose()
 ---------------------------------------------------------------------------
 ValueError: Columns cannot be NoRowIndex.
-If you got here via `transpose` or an `axis=1` operation, then you should first set an index, e.g.: `df.pipe(lambda _df: _df.set_axis(pd.RangeIndex(len(df))))`
+If you got here via `transpose` or an `axis=1` operation, then you should first set an index, e.g.: `df.pipe(lambda _df: _df.set_axis(pd.RangeIndex(len(_df))))`
 ```
 
 ### DataFrameFormatter and SeriesFormatter changes
@@ -222,21 +224,40 @@ Of the above changes, this may be the only one that would need implementing with
 ## Usage and Impact
 
 By itself, ``NoRowIndex`` would be of limited use. To become useful and user-friendly,
-a mode ``no_default_index`` could be introduced which, if enabled, would change
+a ``no_default_index`` mode could be introduced which, if enabled, would change
 the ``default_index`` function to return a ``NoRowIndex`` of the appropriate length.
 In particular, ``.reset_index()`` would result in a ``DataFrame`` with a ``NoRowIndex``.
 Likewise, a ``DataFrame`` constructed without explicitly specifying ``index=``.
 
-Then, if a user doesn't want to think about row labels, then with ``pd.set_option('no_default_index')``
-set, they wouldn't need to (barring methods such as `.pivot_table` which introduce an index).
-Discussion of such a mode is out-of-scope for this proposal.
+Furthermore, it could be useful to add ``as_index`` options to methods which currently
+set an index, and then allow for that mode to control the ``as_index`` default.
+
+Discussion of such a mode is out-of-scope for this proposal. A ``NoRowIndex`` would
+just be a first step towards getting there.
 
 ## Implementation
 
 Draft pull request showing proof of concept: https://github.com/pandas-dev/pandas/pull/49693.
 
+Note that implementation details could well change even if this PDEP were
+accepted. For example, ``NoRowIndex`` wouldn't necessarily need to subclass
+``RangeIndex``, and it wouldn't necessarily need to be accessible to the user
+(``df.index`` could well return ``None``)
+
 ## Likely FAQ
 
+**Q: Couldn't users just use ``RangeIndex``? Why do we need a new class?**
+
+**A**: ``RangeIndex`` isn't preserved under slicing and appending, e.g.:
+  ```python
+  In [1]: ser = pd.Series([1,2,3])
+
+  In [2]: ser[ser!=2].index
+  Out[2]: Int64Index([0, 2], dtype='int64')
+  ```
+  If someone doesn't want to think about row labels and starts off
+  with a ``RangeIndex``, they'll very quickly lose it.
+
 **Q: Aren't indices really powerful?**
 
 **A:** Yes! And they're also confusing to many users, even experienced developers.
@@ -245,16 +266,13 @@ Draft pull request showing proof of concept: https://github.com/pandas-dev/panda
   and alignment. Indices would be here to stay, and ``NoRowIndex`` would not be the
   default.
 
-**Q: In this mode, could users still get an ``Index`` if they really wanted to?**
+**Q: How could one switch a ``NoRowIndex`` ``DataFrame`` back to one with an index?**
 
-**A:** Yes! For example with
+**A:** The simplest way would probably be:
   ```python
-  df.set_index(Index(range(len(df))))
-  ```
-  or, if they don't have a column named ``'index'``:
-  ```python
-  df.reset_index().set_index('index')
+  df.set_axis(pd.RangeIndex(len(df)))
   ```
+  There's probably no need to introduce a new method for this.
 
 **Q: Why not let transpose switch ``NoRowIndex`` to ``RangeIndex`` under the hood before swapping index and columns?**
 
@@ -264,6 +282,37 @@ Draft pull request showing proof of concept: https://github.com/pandas-dev/panda
   to be intentional about what they want and end up with fewer surprises later
   on.
 
+**Q: What would df.sum(), and other methods which introduce an index, return?**
+
+**A:** Such methods would still set an index and would work the same way they
+  do now. There may be some way to change that (e.g. introducing ``as_index``
+  arguments and introducing a mode to set its default) but that's out of scope
+  for this particular PDEP.
+
+**Q: How would a user opt-in to a ``NoRowIndex`` DataFrame?**
+
+**A:** This PDEP would only allow it via the constructor, passing
+  ``index=NoRowIndex(len(df))``. A mode could be introduced to toggle
+  making that the default, but would be out-of-scope for the current PDEP.
+
+**Q: Would ``.loc`` stop working?**
+
+**A:** No. It would only raise if used for label-based selection. Other uses
+  of ``.loc``, such as ``df.loc[:, col_1]`` or ``df.loc[mask, col_1]``, would
+  continue working.
+
+**Q: What's unintuitive about ``Series`` aligning indices when summing?**
+
+**A:** Not sure, but I once asked a group of experienced developers what the
+  output of
+  ```python
+  ser1 = pd.Series([1,1,1], index=[1,2,3])
+  ser2 = pd.Series([1,1,1], index=[3,4,5])
+  print(ser1 + ser2)
+  ```
+  would be, and _nobody_ got it right.
+
 ## PDEP History
 
 - 14 November: Initial draft
+- 18 November: First revision