ERR: _shallow_copy should assert dtype

[jreback]

xref #13288

In [2]: i = Index([0,'A'])

In [3]: i._shallow_copy([0])
Out[3]: Index([0], dtype='int64')

We should assert that we have a valid dtype in the constructed object (so should do this in _simple_new to avoid these guarantee violations)

[pijucha]

An example for the same issue: np.array inside _simple_new converts nan to a string

i = pd.Index([0, 'A'])

i._shallow_copy(['A', np.nan])
Out[7]: Index(['A', 'nan'], dtype='<U3')

The output should be as follows

pd.Index(['A', np.nan])
Out[8]: Index(['A', nan], dtype='object')

---

And another issue in _simple_new: _shallow_copy returning an array

dti = pd.DatetimeIndex([0])

dti._shallow_copy([pd.Timestamp(0)])
Out[12]: array(['1970-01-01T00:00:00.000000000'], dtype='datetime64[ns]')

The expected output is as in the following:

dti._shallow_copy([0])
Out[13]: DatetimeIndex(['1970-01-01'], dtype='datetime64[ns]', freq=None)

---

INSTALLED VERSIONS
------------------
commit: 91617085c0696db3b457b524fe7273ecc928fedc
python: 3.5.1.final.0
python-bits: 64
OS: Linux
OS-release: 4.1.20-1
machine: x86_64
processor: Intel(R)_Core(TM)_i5-2520M_CPU_@_2.50GHz
byteorder: little

pandas: 0.18.1
numpy: 1.11.0

[jreback]

these violate guarantees of _shallow_copy

the data must already be an index

[pijucha]

This is probably something I misunderstood (misunderstand) about _shallow_copy and this issue.

Does it mean that all those examples (including [0] in the first one) is rather a misuse of _shallow_copy, which failed to be detected/asserted, and not a real bug?

And the only valid values in the expression

idx._shallow_copy(values)

is an index or an np.array with dtype compatible with idx.dtype (and not just a plain list)?

[jreback]

yep - this issue is about

a) making sure that we are an Index (or values is an ndarray)
b) validating that the dtype is compat for the class

or raise

[pijucha]

Thanks!

[jbrockmendel]

A lot of thought has gone into related topics for PeriodArray etc. I think the conclusion is pretty much that the caller is responsible for checking the inputs, since shallowxopy is private. Assertions in simplenew might be cleaner.

[mroeschke]

I think this asserts correctly as Brock mentions. Closing