You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas (theoretically affected as well but my example doesn't work for <1.2 (need binary file handle) other examples should trigger this bug in <1.2).
(optional) I have confirmed this bug exists on the master branch of pandas.
Code Sample, a copy-pastable example
importosimportfsspecimportpandasaspd# create a 'normal' file handlewithopen("abc.test", mode="w") asopen_obj:
assertnotisinstance(open_obj, os.PathLike) # is not converted to a stringposition=open_obj.tell()
# let to_csv write to the opened filepd.DataFrame({"a": [1, 2, 3]}).to_csv(open_obj)
# the position of the file buffer should have changed if to_csv used itassertopen_obj.tell() !=position# create a file handle that also implements os.PathLike/has __fspath__fsspec_obj=fsspec.open("file://abc.test", mode="wb").open()
withfsspec_obj:
assertisinstance(fsspec_obj, os.PathLike) # is converted to a stringposition=fsspec_obj.tell()
# let to_csv write to the opened filepd.DataFrame({"a": [1, 2, 3]}).to_csv(fsspec_obj)
# the position of the file buffer should have changed if to_csv used itassertfsspec_obj.tell() !=position# fails
Problem description
get_filepath_or_buffer (<1.2) or get_handle (1.2) call stringify_path to convert pathlib.Path and other os.PathLike to a string. This string is then later opened. It seems that there is at least one file object that implements os.PathLike but at the same time already opens the file. In this case case, all to/read_* that use get_handle (or get_filepath_or_buffer in <1.2) extract the string and then open the file (even though the user already opened it).
I'm not sure whether there are other examples. I will look into how to fix this.
The text was updated successfully, but these errors were encountered:
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas (theoretically affected as well but my example doesn't work for <1.2 (need binary file handle) other examples should trigger this bug in <1.2).
(optional) I have confirmed this bug exists on the master branch of pandas.
Code Sample, a copy-pastable example
Problem description
get_filepath_or_buffer
(<1.2) orget_handle
(1.2) callstringify_path
to convertpathlib.Path
and otheros.PathLike
to a string. This string is then later opened. It seems that there is at least one file object that implementsos.PathLike
but at the same time already opens the file. In this case case, allto/read_*
that useget_handle
(orget_filepath_or_buffer
in <1.2) extract the string and then open the file (even though the user already opened it).I'm not sure whether there are other examples. I will look into how to fix this.
The text was updated successfully, but these errors were encountered: