Strange behavior assigning values to elements

[phobson]

Basically, if you create a Data

import pandas as pd

df = pd.DataFrame({ "aa":range(5), "bb":[2.2]*5})

df["cc"] = 0.0 # remove this line, it works

ck = [True]*len(df)

df["bb"].iloc[0] = .13
df_tmp = df.iloc[ck] # or remove this line and it'll work
df["bb"].iloc[0] = .15 # doesn't happen

print df

which gives:

   aa    bb  cc
0   0  0.13   0
1   1  2.20   0
2   2  2.20   0
3   3  2.20   0
4   4  2.20   0

Very strange.

Specs:

pd.version.version
Out[5]: '0.11.0'

np.version.version
Out[7]: '1.7.1'

[jreback]

I responded to you e-mail..repro here

this is not a bug; you are modifying a copy (which is caused by the dtype change)
use loc like this

http://pandas.pydata.org/pandas-docs/dev/indexing.html#indexing-view-versus-copy

In [11]: df.loc[0,'bb'] = .13

In [12]: df_tmp = df[ck]

In [13]: df.loc[0,'bb'] = .15

In [14]: df
Out[14]: 
   aa    bb
0   0  0.15
1   1  2.20
2   2  2.20
3   3  2.20
4   4  2.20
5   5  2.20
6   6  2.20
7   7  2.20
8   8  2.20
9   9  2.20

[phobson]

I'm confused. Where does the dtype change?

I do agree that this behaves as expected

import pandas as pd
df = pd.DataFrame({ "aa":range(5), "bb":[2.2]*5})
df["cc"] = 0.0 
ck = [True]*len(df)
df.loc[0, 'bb'] = .13
df_tmp = df.iloc[ck] 
df.loc[0, 'bb'] = .15 
print df

But how is this not inconsistent behavior?

import pandas as pd
df = pd.DataFrame({ "aa":range(5), "bb":[2.2]*5})
df["cc"] = 0.0 
ck = [True]*len(df)
df["bb"].loc[0] = .13 # works
df_tmp = df.iloc[ck] 
df["bb"].loc[0] = .15 # doesn't work
print df

In other words, why can I change an element once, but not a second time?

[jreback]

sorry was thinking about another issue, dtype is not a problem here

this
df["bb"].loc[0] = .13

creates a copy (so you assign to the copy rather than the frame)
note that this does not always create a copy (has to do whether this is view or not,
and unfortunately that is numpy defined)

this is why a multi-axes assignment should use all axes in a single loc/iloc

[jreback]

df_tmp = df.iloc[ck] this triggers a copy of the underlying data (and it is no longer a view), not exactly sure why

[cpcloud]

doesn't indexing with a sequence (or any other non slice, non tuple object that is a valid index) trigger a copy? it does with numpy

[jreback]

the old advanced/fancy vs basic, could be

[cpcloud]

arg there's no explanation for why fancy indexing requires a copy. maybe it's obvious but i don't see it

[jreback]

I think its really related to how memory is laid out, what you are doing, etc. can it be done in an efficient manner and so for, basically implementation dependent.

[cpcloud]

probably is because in general fancy indexing must follow the rule that any operation leading to an irregularly strided array must return a copy. in come cases (ones that are equivalent to slicing you could have views, but there's the overhead of checking whether there's an equivalent slice to the passed numpy array)

[wesm]

Looks buggy to me, marked as such and labeled for 0.11.1

[jreback]

ill take a look

[johannh-zz]

Here's another good one. If I look at df["bb"] after this, I see the change (on 0.11):

In [21]: print df
   aa    bb  cc
0   0  0.13   0
1   1  2.20   0
2   2  2.20   0
...

In [22]: df["bb"]
Out[22]:
0    0.15
1    2.20
2    2.20
3    2.20
...

[jreback]

closed by #4077