DOC: Data Editing Samples/Guide

[achapkowski]

Location of the documentation

N/A

Documentation problem

There seems to be general community confusion over when the use iloc, loc, at, iat or the other methods that allow you to update rows and columns. What is the best way to add a single new row? Multiple rows pd.concat something else?

It would be nice if a guide or doc was created to point to the best practices for data editing on a DataFrame.

[rhshadrach]

I think the relevant docs are:

https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html
https://pandas.pydata.org/pandas-docs/stable/user_guide/merging.html

Is there something missing there?

[achapkowski]

These two sets of docs are relevant to the case, but if I need to do this:

df1 = pd.DataFrame({'A': ['A0', 'A1', 'A2', 'A3'],
                     'B': ['B0', 'B1', 'B2', 'B3'],
                     'C': ['C0', 'C1', 'C2', 'C3'],
                     'D': ['D0', 'D1', 'D2', 'D3']},
                    index=[0, 1, 2, 3])
# add this single row to df1
row = ['A4', 'B4', 'C4', 'D4']

What is the most efficient way to do this?

Do I create a DataFrame for the single row and concat it? Do I use append? One of the loc/iloc methods?

[rhshadrach]

I think adding a section in the documentation discussing this operation makes sense; I couldn't find one on it and hopefully didn't miss anything. Two suggestions:

• There has been much support for deprecating append (Deprecate Series / DataFrame.append #35407).
• Regardless of the method used, it is more efficient to gather data up front (e.g. as a dictionary/list of dictionaries) and then create a single DataFrame rather than having many appends/concats/locs.

[achapkowski]

@rhshadrach thanks for pointing out the deprecation! There are so many issues, it's hard to keep track of them all.

I know the best practice is to have all your data in hand, but sometimes your dataframe needs a few more rows of data and especially for large datasets (millions of rows) recreating the whole dataframe is overkill.