BUG?: _is_view == False for DataFrame from DataFrame

[nickeubank]

Constructing off a dataframe creates a de facto view, but ._is_view reports false. Seems like a bug.

In [1]:
    df = pd.DataFrame({'col1':[1,2], 'col2':[3,4]})
    df2 = pd.DataFrame(df)
    df2._is_view     
Out[1]:
    False

In [2]:
    df.loc[0,'col1'] = -88
    df2
Out[2]:
    col1    col2
    0   -88 3
    1   2   4

Suggestions on how best to fix?

[jreback]

Your example is false. A dictionary NEVER creates a view. (I have a branch that fixes this by not consolidating, but thats a separate issue).

Views are created by being passed numpy arrays (the view is to the array itself)

In [49]: df = pd.DataFrame(np.array([[1,3],[2,4]]))

In [50]: df
Out[50]: 
   0  1
0  1  3
1  2  4

In [51]: df = pd.DataFrame(np.array([[1,3],[2,4]]),columns=list('AB'))

In [52]: df
Out[52]: 
   A  B
0  1  3
1  2  4

In [53]: df._data.blocks[0].is_view
Out[53]: True

In [59]: df._data.blocks[0].values.base
Out[59]: 
array([[1, 3],
       [2, 4]])

In [54]: df = DataFrame({'A':[1,2], 'B':[3,4]})

In [55]: df
Out[55]: 
   A  B
0  1  3
1  2  4

In [56]: df._data.blocks[0].is_view
Out[56]: False

In [57]: df._data.blocks[0].values.base

In [58]: df = pd.DataFrame(np.array([[1,3],[2,4]]),columns=list('AB'))

Note that this is NOT true for object dtypes (otherwise we can have aliasing issues).

In [60]: df = DataFrame(np.array([['foo','bar'],['baz','bah']]),columns=list('AB'))

In [61]: df
Out[61]: 
     A    B
0  foo  bar
1  baz  bah

In [62]: df._data.blocks[0].values.base

In [63]: df._data.blocks[0].is_view    
Out[63]: False

[nickeubank]

Hi @jreback

Think my example / issue was mis-understood. I agree creating a DataFrame from a dictionary never creates a view. The case I'm noting is that if you create a DataFrame from another DataFrame, then those are effectively views of one another, even if they reports _is_view == False.

In my example, if you change df, then df2 is also changed. Yet df2._is_view is False.

Indeed, view behavior documented in test modules (test_frame.py, line 2637):

def test_constructor_dtype_nocast_view(self):
    df = DataFrame([[1, 2]])
    should_be_view = DataFrame(df, dtype=df[0].dtype)
    should_be_view[0][0] = 99
    self.assertEqual(df.values[0, 0], 99)

    should_be_view = DataFrame(df.values, dtype=df[0].dtype)
    should_be_view[0][0] = 97
    self.assertEqual(df.values[0, 0], 97)

[jreback]

@nickeubank

so this is actually right.

._is_view tests whether something IS a view

e.g.

In [1]: arr = np.arange(5) 

In [2]: arr.base is None
Out[2]: True

In [3]: arr.view().base is None
Out[3]: False

but this happens to fail if the object IS THE SAME OBJECT.

E.g. in the case _data is assigned directly to the ._data in the df2. Not a view, not a copy, but the same object.

In [6]: df._data is df2._data
Out[6]: True

So this is technically 'wrong', but its more semantics than anything else. Sharing the same object or a view of it are from pandas point of view de-facto the same. numpy makes this distinction, but not pandas.

I suppose you could actually create a new block manager, where everything undering is created not with copy but with .view then this would be true. from a practical point of view I don't really know if this would make a difference though.

[jreback]

If you want to change this (independent of your other PR) and see if this actually breaks anything (except the test), I would accept a PR for that.

[nickeubank]

Great! Will do -- need it for the other PR anyway. And thanks for the explanation!

[nickeubank]

huh -- empty indexers do the same thing.

df = pd.DataFrame({'col1':range(10,20),
                   'col2':range(20,30)})

df2 = df.loc[:,:]

df2._data is df._data
Out[3]: True

df2._is_view
Out[4]: False

That ok to tweak too?

[manubhatt3]

PR means Personal Request?