Skip to content

DataFrame constructor produces NaN values when passing df as data param and an index containing values that are not present in passed in df #9951

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alan-wong opened this issue Apr 20, 2015 · 1 comment
Labels
API Design Reshaping Concat, Merge/Join, Stack/Unstack, Explode Usage Question

Comments

@alan-wong
Copy link

Whilst answering this SO question I discovered that the DataFrame constructor is ignoring the fact that I'm passing an index explicitly resulting in a df with all NaNs:

In [6]:
df = pd.DataFrame({'AAA':[4,5,6,7],'BBB':[10,20,30,40],'CCC':[100,50,-30,-50]})
df1 = pd.DataFrame(data=df,index=(['a','b','c','d']))
df1
Out[6]:
   AAA  BBB  CCC
a  NaN  NaN  NaN
b  NaN  NaN  NaN
c  NaN  NaN  NaN
d  NaN  NaN  NaN

Stepping through the code I see that it reaches here in frame.py:

            if isinstance(data, BlockManager):
            mgr = self._init_mgr(data, axes=dict(index=index, columns=columns),
                                 dtype=dtype, copy=copy)

and then ends up here in generic.py:

119         def _init_mgr(self, mgr, axes=None, dtype=None, copy=False):
120             """ passed a manager and a axes dict """
121             for a, axe in axes.items():
122                 if axe is not None:
123                     mgr = mgr.reindex_axis(
124  ->                     axe, axis=self._get_block_manager_axis(a), copy=False)

The behaviour can be reproduced simply like this:

In [46]:

data1 = pd.DataFrame({'AAA':[4,5,6,7],'BBB':[10,20,30,40],'CCC':[100,50,-30,-50]})
data1.reindex_axis(list('abcd'))
Out[46]:
   AAA  BBB  CCC
a  NaN  NaN  NaN
b  NaN  NaN  NaN
c  NaN  NaN  NaN
d  NaN  NaN  NaN
@jreback
Copy link
Contributor

jreback commented Apr 20, 2015

xref here: #9237

This is the defined behavior, to reindex the provided input to the passed index and/or columns .

@jreback jreback closed this as completed Apr 20, 2015
@jreback jreback added Reshaping Concat, Merge/Join, Stack/Unstack, Explode Usage Question API Design labels Apr 20, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Design Reshaping Concat, Merge/Join, Stack/Unstack, Explode Usage Question
Projects
None yet
Development

No branches or pull requests

2 participants