BUG: na_values with a dict of scalars

[jseabold]

I can't get na_values to work with a dictionary.

from StringIO import StringIO
import pandas as pd

dta = pd.read_csv(StringIO("""var1,var2,var3,var4
1,2,3,MISSING
2,3,4,4.5
"""), na_values={3 : 'MISSING'})

dta = pd.read_csv(StringIO("""var1,var2,var3,var4
1,2,3,MISSING
2,3,4,4.5
"""), na_values={'var4' : 'MISSING'})

dta = pd.read_csv(StringIO("""var1,var2,var3,var4
1,2,3,MISSING
2,3,4,4.5
"""), na_values=['MISSING'])

On

[~/]
[10]: pd.version.version
[10]: '0.13.1-753-g4614ac8'

[jreback]

try na_values={'var4' : ['MISSING'] }

a bit non-intuitive, but that's what we ar testing on...prob a but not to accept a scalar

[jreback]

In [9]: read_csv(StringIO(data),na_values=dict(var4 = ['MISSING']))
Out[9]: 
   var1  var2  var3  var4
0     1     2     3   NaN
1     2     3     4   4.5

[2 rows x 4 columns]

[jseabold]

There was similar wart-y code very early on that required lists even for single variables. E.g., in dropna, etc. though I don't remember where exactly. Probably worth a patch.

[jreback]

docs say list of strings but not hard to accept a dict of scalars/lists for a particular column (as a single scalar is accepted too)

[gfyoung]

@jreback : #14056 resolves the second example. However, the behaviour for the first one is not entirely clear from documentation. Should we be able to accept column indices? At the moment, the answer appears to be no:

>>> from pandas.compat import StringIO
>>> from pandas import read_csv
>>> data = 'a\nfoo\n1'
>>>
>>> read_csv(StringIO(data), na_values={0: 'foo'}, engine='c')
...
TypeError: Expected list, got set
>>> read_csv(StringIO(data), na_values={0: 'foo'}, engine='python')
     a
0  foo
1    1

[jreback]

@gfyoung I think column indices are ok, except if a header is passed (in which case they must be the names; easiest to avoid conflict here).

[gfyoung]

@jreback : Hmm...okay. In any case, I think this issue should be closed in favor of another one that specifically states that we can correctly process column indices.

[jreback]

ok can u make one?

[gfyoung]

@jreback : Done --> #14203.