@@ -1718,9 +1718,26 @@ at `fsimpl1`_ for implementations built into ``fsspec`` and `fsimpl2`_
1718
1718
for those not included in the main ``fsspec ``
1719
1719
distribution.
1720
1720
1721
- You can also pass parameters directly to the backend driver. For example,
1722
- if you do *not * have S3 credentials, you can still access public data by
1723
- specifying an anonymous connection, such as
1721
+ You can also pass parameters directly to the backend driver. Because we can no
1722
+ longer utilize the ``AWS_S3_HOST `` environment variable, we can directly define a
1723
+ dictionary containing the endpoint_url and pass the object into the storage
1724
+ option parameter:
1725
+
1726
+ .. code-block :: python
1727
+
1728
+ def s3so (worker_id ):
1729
+ worker_id = " 5" if worker_id == " master" else worker_id.lstrip(" gw" )
1730
+ return dict (client_kwargs = {" endpoint_url" : f " http://127.0.0.1:555 { worker_id} / " })
1731
+
1732
+ roundtripped_df = pd.read_json(
1733
+ " s3://pandas-test/test-1" , compression = compression, storage_options = s3so,
1734
+ )
1735
+
1736
+ More sample configurations and documentation can be found at `S3Fs documentation
1737
+ <https://s3fs.readthedocs.io/en/latest/index.html?highlight=host#s3-compatible-storage> `__.
1738
+
1739
+ If you do *not * have S3 credentials, you can still access public
1740
+ data by specifying an anonymous connection, such as
1724
1741
1725
1742
.. versionadded :: 1.2.0
1726
1743
@@ -2992,7 +3009,7 @@ Read a URL with no options:
2992
3009
Read in the content of the "books.xml" file and pass it to ``read_xml ``
2993
3010
as a string:
2994
3011
2995
- .. ipython :: python
3012
+ .. ipython :: python :okexcept:
2996
3013
2997
3014
file_path = "books.xml"
2998
3015
with open(file_path, "w") as f:
0 commit comments