BUG: Pandas should not swallow exceptions when close()ing a write handle #47136
Labels
Bug
Error Reporting
Incorrect or improved errors from pandas
IO Data
IO issues that don't fit into a more specific label
Milestone
Pandas version checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
Issue Description
IOHandles.close
(https://github.com/pandas-dev/pandas/blob/v1.4.2/pandas/io/common.py#L119) throws away any exceptions raised by calling close() on the handle it is managing. When writing to certain file systems (in particular NFS mounts), write errors caused by e.g. runnning out of disk space may not be raised at the point you actually do awrite()
but rather get deferred to theclose()
. Therefore, by ignoring exceptions raised byclose()
it's possible for a Pandas IO utility function to silently write a truncated file.Expected Behavior
Exceptions raised by
close()
should be propagated to the user for appropriate handling. You may optionally want to continue to ignore exceptions raised by handles open in read mode.Installed Versions
pandas : 1.3.5
numpy : 1.20.3
pytz : 2021.3
dateutil : 2.8.2
pip : 21.2.4
setuptools : 58.0.4
Cython : None
pytest : 6.2.5
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.5.1
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 3.0.2
IPython : 7.31.1
pandas_datareader: None
bs4 : 4.10.0
bottleneck : 1.3.2
fsspec : 2022.01.0
fastparquet : None
gcsfs : None
matplotlib : 3.5.0
numexpr : 2.8.1
odfpy : None
openpyxl : 3.0.9
pandas_gbq : None
pyarrow : 4.0.1
pyxlsb : None
s3fs : 2022.01.0
scipy : 1.7.3
sqlalchemy : None
tables : None
tabulate : None
xarray : 0.20.1
xlrd : 2.0.1
xlwt : None
numba : 0.54.1
The text was updated successfully, but these errors were encountered: