DOC: Expand View vs copy in the docs

[hayd]

At the moment it's a common source of confusion as to what returns a view vs a copy. The docs are fairly light on this atm.
Need to expand this section with lots of example, and update with what now returns a view.

I've said before to @jreback I'd do this but tbh, am not fully confident on the ins and outs so perhaps we can post a few examples here to get started.

[hayd]

for lists row and columns this produces a view, same for iloc, ix:

df.loc[rows, columns] 

as do these

df[columns]
df.T.loc[columns, rows]

order matters example

use update for boolean indexing

[jreback]

df.loc[rows,columns] = value is always safe, it does produce a view, but the setitem handles this so it doesn't actually matter

you only really get in trouble with a chained setter e.g.

df.loc[:,['A','B']].iloc[0] = value DOES not work, nor should it

the key point is that the assignment should in one call

[hayd]

Do you mean copy?

I wonder if this also warrants a cookbook section (I guess mainly for the update thing).

[jreback]

no...what I mean is that when you do a setter it sets the correct object, it doesn't matter if its a view/copy because you never see a returned object anyways.

df.loc[rows,columns] will in general return a copy (except or the special case of C-order and rows = :, I think. It doesn't matter though. The only methods pandas supports for setting are direct methods, anything else is not really supported anyhow (e.g. the chained method of setting, will sometimes work but is definitly not supported nor expected to work)

[hayd]

wowza, so boolean indexing does work now provided you use that:

df[mask] = value

[jreback]

In [3]: df = DataFrame(np.arange(10).reshape(5,2),index=range(5),columns=['A','B'],dtype='float64')

In [4]: df
Out[4]: 
   A  B
0  0  1
1  2  3
2  4  5
3  6  7
4  8  9

In [10]: df.loc[df.A>4,['A']] = -df

In [11]: df
Out[11]: 
   A  B
0  0  1
1  2  3
2  4  5
3 -6  7
4 -8  9

In [12]: df = DataFrame(np.arange(10).reshape(5,2),index=range(5),columns=['A','B'],dtype='float64')

In [13]: df[df.A>4] = -df

In [14]: df
Out[14]: 
   A  B
0  0  1
1  2  3
2  4  5
3 -6 -7
4 -8 -9

[jreback]

boolean indexing with df[mask] = value works, but it is usually not 'precise' enough....almost always want to limit the columns and

df[['columns']][mask] = value is a chained expression so DOESN't work

but you CAN (and should) use .loc for this purpose

[hayd]

In [10]: df[df == 2] = 99

In [11]: df
Out[11]:
    A  B
0   0  1
1  99  3
2   4  5
3   6  7
4   8  9

[hayd]

I thought you couldn't use loc with a mask ? Oh wait I see, you use boolean index/column.

Am I missing something:

In [12]: df.loc[[0], df == 2]
Out[12]:
   A  B
0  0  1

[jreback]

the columns are aligned to that df == 2 is just ['A','B'] there....equiv to
df.loc[[0]]

iloc with a mask is a problem, in this case the integer labels are just that, labels

[hayd]

So to use with loc they should be column_mask and row_mask, if that makes sense.

I remember now (about iloc and boolean indexing).

Also creating link to #4192

[jreback]

yep (mask or labels are ok)

[jreback]

this is pretty good now