-
-
Notifications
You must be signed in to change notification settings - Fork 18.4k
BUG: dtypes cast when reading JSON #44693
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
try on master as #42819 was recently merged |
I'm seeing the same behavior at commit d7eadde (current master) |
This looks deliberate when looking at the implementation |
It seems that dtype does not work here. See func _parse_no_numpy, dtype only works when orient == "table", for other scenarios, dtype is None. It means that pandas will infer the type of values. |
I too have this issue when reading a json lines with read_json pandas is ignoring my dtypes and casting some columns incorrectly |
Reproducible Example
Issue Description
Using
dtype
withpd.read_json
:str
"something" to afloat
Int64
when casting aint
field with missing valuesExpected Behavior
str
"something" to afloat
Int64
when casting aint
field with missing valuesInstalled Versions
INSTALLED VERSIONS
commit : 945c9ed
python : 3.8.11.final.0
python-bits : 64
OS : Darwin
OS-release : 20.6.0
Version : Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8
pandas : 1.3.4
numpy : 1.21.2
pytz : 2021.1
dateutil : 2.8.2
pip : 21.1.3
setuptools : 57.4.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.6.3
html5lib : None
pymysql : 1.0.2
psycopg2 : None
jinja2 : 3.0.1
IPython : 7.27.0
pandas_datareader: None
bs4 : 4.10.0
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.4.3
numexpr : None
odfpy : None
openpyxl : 3.0.8
pandas_gbq : None
pyarrow : 5.0.0
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : 0.8.9
xarray : None
xlrd : None
xlwt : None
numba : None
The text was updated successfully, but these errors were encountered: