Skip to content

Commit 73cc3c8

Browse files
author
TomAugspurger
committed
BUG: Fix read_csv on S3 files for python 3
Needed to pass along encoding parameter. Bump boto version. Enable test on py3
1 parent fc843d3 commit 73cc3c8

File tree

4 files changed

+7
-5
lines changed

4 files changed

+7
-5
lines changed

ci/requirements-2.7.txt

+1-1
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ lxml=3.2.1
1313
scipy
1414
xlsxwriter=0.4.6
1515
statsmodels
16-
boto=2.26.1
16+
boto=2.36.0
1717
bottleneck=0.8.0
1818
psycopg2=2.5.2
1919
patsy

doc/source/whatsnew/v0.16.0.txt

+4
Original file line numberDiff line numberDiff line change
@@ -344,3 +344,7 @@ Bug Fixes
344344
- Bug in groupby MultiIndex with missing pair (:issue:`9049`, :issue:`9344`)
345345
- Fixed bug in ``Series.groupby`` where grouping on ``MultiIndex`` levels would ignore the sort argument (:issue:`9444`)
346346
- Fix bug in ``DataFrame.Groupby`` where sort=False is ignored in case of Categorical columns. (:issue:`8868`)
347+
348+
349+
350+
- Fixed bug with reading CSV files from Amazon S3 on python 3 raising a TypeError (:issue:`9452`)

pandas/io/common.py

+2-1
Original file line numberDiff line numberDiff line change
@@ -154,7 +154,8 @@ def get_filepath_or_buffer(filepath_or_buffer, encoding=None):
154154
b = conn.get_bucket(parsed_url.netloc)
155155
k = boto.s3.key.Key(b)
156156
k.key = parsed_url.path
157-
filepath_or_buffer = BytesIO(k.get_contents_as_string())
157+
filepath_or_buffer = BytesIO(k.get_contents_as_string(
158+
encoding=encoding))
158159
return filepath_or_buffer, None
159160

160161

pandas/io/tests/test_parsers.py

-3
Original file line numberDiff line numberDiff line change
@@ -3802,9 +3802,6 @@ def setUp(self):
38023802
except ImportError:
38033803
raise nose.SkipTest("boto not installed")
38043804

3805-
if compat.PY3:
3806-
raise nose.SkipTest("boto incompatible with Python 3")
3807-
38083805
@tm.network
38093806
def test_parse_public_s3_bucket(self):
38103807
import nose.tools as nt

0 commit comments

Comments
 (0)