Skip to content

Commit 1a18031

Browse files
committed
Added AWS_S3_HOST environment variable workaround to user_guide for issue pandas-dev#26195
1 parent 406abd9 commit 1a18031

File tree

1 file changed

+21
-4
lines changed

1 file changed

+21
-4
lines changed

doc/source/user_guide/io.rst

+21-4
Original file line numberDiff line numberDiff line change
@@ -1718,9 +1718,26 @@ at `fsimpl1`_ for implementations built into ``fsspec`` and `fsimpl2`_
17181718
for those not included in the main ``fsspec``
17191719
distribution.
17201720

1721-
You can also pass parameters directly to the backend driver. For example,
1722-
if you do *not* have S3 credentials, you can still access public data by
1723-
specifying an anonymous connection, such as
1721+
You can also pass parameters directly to the backend driver. Because we can no
1722+
longer utilize the ``AWS_S3_HOST`` environment variable, we can directly define a
1723+
dictionary containing the endpoint_url and pass the object into the storage
1724+
option parameter:
1725+
1726+
.. code-block:: python
1727+
1728+
def s3so(worker_id):
1729+
worker_id = "5" if worker_id == "master" else worker_id.lstrip("gw")
1730+
return dict(client_kwargs={"endpoint_url": f"http://127.0.0.1:555{worker_id}/"})
1731+
1732+
roundtripped_df = pd.read_json(
1733+
"s3://pandas-test/test-1", compression=compression, storage_options=s3so,
1734+
)
1735+
1736+
More sample configurations and documentation can be found at `S3Fs documentation
1737+
<https://s3fs.readthedocs.io/en/latest/index.html?highlight=host#s3-compatible-storage>`__.
1738+
1739+
If you do *not* have S3 credentials, you can still access public
1740+
data by specifying an anonymous connection, such as
17241741

17251742
.. versionadded:: 1.2.0
17261743

@@ -2992,7 +3009,7 @@ Read a URL with no options:
29923009
Read in the content of the "books.xml" file and pass it to ``read_xml``
29933010
as a string:
29943011

2995-
.. ipython:: python
3012+
.. ipython:: python :okexcept:
29963013

29973014
file_path = "books.xml"
29983015
with open(file_path, "w") as f:

0 commit comments

Comments
 (0)