DOC: diff() when using unsigned ints

[jjlkant]

Code Sample, a copy-pastable example if possible

>>> import numpy as np
>>> import pandas as pd
>>> 
>>> s = pd.Series([1, 0], dtype=np.uint8)
>>> s.diff()
0      NaN
1    255.0
dtype: float64

Problem description

The current version of the docs don't reflect on what happens when a Series or DataFrame containing unsigned ints are used with the diff() function, especially for cases where integer overflow occurs.

NumPy handles this in their docs for numpy.diff(), stating:

For unsigned integer arrays, the results will also be unsigned. This should not be surprising, as the result is consistent with calculating the difference directly:

>>> u8_arr = np.array([1, 0], dtype=np.uint8)
>>> np.diff(u8_arr)
array([255], dtype=uint8)
>>> u8_arr[1,...] - u8_arr[0,...]
255

The behaviour of pandas.Series.diff() and pandas.DataFrame.diff() during calculation is the same as the NumPy equivalent, but the actual return type is no longer an unsigned integer; it will always return with a float64 type.

Expected Output

The current output can be expected when the user is perfectly aware that integer overflow might possibly occur during the calculation of the diff() output, even though it returns as a float64 dtype. I do think it is desired to mention this in docs similarly to the NumPy equivalent though.

However, since the output dtype of diff() will always be float64, this also allows to handle the case differently: Instead of storing the output of the calculation:

pandas/pandas/core/algorithms.py

Line 1964 in 0c4404b

out_arr[res_indexer] = arr[res_indexer] - arr[lag_indexer]

into a float64 dtyped array, it would also be possible to process the values as float instead of integers? This would generate the output:

0    NaN
1   -1.0
dtype: float64

I understand that this might go against the absolute difference of the elements as they are defined inside of the Series or DataFrame, but it would better fit the output type? Maybe just a warning for the overflow is the best middle ground instead? Would love to hear your thoughts on the case...

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.4.final.0
python-bits : 64
OS : Windows
OS-release : 10
machine : AMD64
processor : Intel64 Family 6 Model 158 Stepping 10, GenuineIntel
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : None.None

pandas : 0.25.1
numpy : 1.17.2
pytz : 2019.2
dateutil : 2.8.0
pip : 19.2.3
setuptools : 41.2.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : 3.1.1
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : 1.3.1
sqlalchemy : None
tables : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None