Series apply() and map() may produce a Series with different dtype than the original

[lbeltrame]

I'm unsure if it's by design or not, so I'm leaving this here.

In [84]: original = pandas.Series([1,2,3, np.nan], dtype="object")

In [85]: def convert(value):
    return np.nan if pandas.isnull(value) else int(value)
   ....: 

In [86]: original.apply(convert).dtype
Out[86]: dtype('float64')

[lodagro]

Would you prefer that once dtype is object it stays object?
It is by design that the ouput of apply func is evaluated to determine the dtype of return series.

In [46]: s
Out[46]:
0    1
1    2
2    3
3    4
4    5

In [47]: def divby3(x):
   ....:     return x/3.0
   ....:

In [48]: s.dtype
Out[48]: dtype('int64')

In [49]: s.apply(divby3).dtype
Out[49]: dtype('float64')

In [50]: s.apply(divby3)
Out[50]:
0    0.333333
1    0.666667
2    1.000000
3    1.333333
4    1.666667

In [51]: s.apply(str).dtype
Out[51]: dtype('object')

[lbeltrame]

In data mercoledì 6 giugno 2012 05:33:20, hai scritto:

Would you prefer that once dtype is object it stays object?

Personally yes, because otherwise to achieve the same result as the function
posted in the bug report:

1. I have to create a temporary list
2. I have to add all the items from the Series to the new list doing the work
   the function does
3. I have to create a Series with the desired dtype at the end, constructed
   with the list and the index of the original one.

Of course if there are technical reasons for this I'll just keep this hack
around.

[lodagro]

Note that you can use astype:

In [15]: original.apply(convert)
Out[15]:
0     1
1     2
2     3
3   NaN

In [16]: original.apply(convert).astype('object')
Out[16]:
0      1
1      2
2      3
3    NaN

In [17]: original.apply(convert).astype('object').dtype
Out[17]: dtype('object')

[lbeltrame]

It doesn't however keep the elements as ints, but instead as floats (so all ".0" etc). So "convert" doesn't do actually anything.

In [191]: type(original.apply(convert).astype("object")[0])
Out[191]: float

[lodagro]

int and na don`t match and the proposed workaround to use object has its limits also, or is it astype that does not do a proper job?

[lbeltrame]

I don't think it's astype. I think this is a side effect of int being promoted to float. For now I guess I'll keep a temporary list hack, even though this is pretty inefficient for large Series.

[wesm]

This is by design. I will see about adding an option to leave the result as dtype=object (the type inference helps much more than it hurts)

[wesm]

Added a convert_dtype option so you can pass convert_dtype=False to leave it as object