BUG: bug in construction using a tuple indexer with embedded None

[kyleabeauchamp]

In the following example, I was surprised that the None in the index somehow converts the value 1.0 to be stored as a nan in the values. Feel free to close if this is obviously user error :)

Code Sample, a copy-pastable example if possible

import pandas as pd

d = {(1, 1, None):-1.0, (1, 1, 1):-2.0, (1, 1, 2):-3.0}
x = pd.Series(d)
x
#### Naive Expected Output
1  1  None    -1.0
      1     -2.0
      2     -3.0
dtype: float64

#### Actual observed output: 

1  1  NaN    NaN
      1     -2.0
      2     -3.0
dtype: float64
#### output of ``pd.show_versions()``
pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.11.final.0
python-bits: 64
OS: Darwin
OS-release: 14.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.18.0
nose: 1.3.7
pip: 8.0.3
setuptools: 20.1.1
Cython: 0.23.4
numpy: 1.10.4
scipy: 0.17.0
statsmodels: 0.6.1
xarray: None
IPython: 4.1.1
sphinx: 1.3.5
patsy: 0.4.0
dateutil: 2.4.2
pytz: 2015.7
blosc: None
bottleneck: 1.0.0
tables: 3.2.2
numexpr: 2.5
matplotlib: 1.5.1
openpyxl: 2.3.2
xlrd: 0.9.4
xlwt: 1.0.0
xlsxwriter: 0.8.4
lxml: 3.5.0
bs4: 4.4.1
html5lib: 0.999
httplib2: None
apiclient: None
sqlalchemy: 1.0.11
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.39.0

[jreback]

None is always converted to a missing value indicator of np.nan (except in some very limited circumstances) as None is a python object and cannot be efficiently coded internally.

see: http://pandas.pydata.org/pandas-docs/stable/missing_data.html#values-considered-missing

[kyleabeauchamp]

I actually know that part already for storing the index. What was confusing to me was that the presence of a missing value in the index tuple also converted the corresponding value from -1.0 to nan.

Is there somewhere that documents this behavior? E.g. "a missing value in a MultiIndex tuple forces any Series value associated with this index tuple to be missing (nan)".

[kyleabeauchamp]

d = {}
d[(1, 1, None)] = -1.0
x = pd.Series(d)


In [23]: x
Out[23]: 
1  1  NaN   NaN
dtype: float64

[jreback]

The problem is you are creating it in a very odd way. if you use (1,1,np.nan) or do it in a more proper way it will work.

In [19]: Series(-1,index=pd.MultiIndex.from_tuples([(1,1,None)]))
Out[19]: 
1  1  NaN   -1
dtype: int64

I suppose this is a bug. you can have a look thru if you'd like.

[mroeschke]

This looks to work on master. Could use a test.

In [3]: d = {}
   ...: d[(1, 1, None)] = -1.0
   ...: x = pd.Series(d)

In [4]: x
Out[4]:
1  1  NaN   -1.0
dtype: float64

In [5]: pd.__version__
Out[5]: '0.26.0.dev0+593.g9d45934af'