Skip to content

Commit 5d0b2fb

Browse files
authored
upgrade gcs_to_bq_util to work with pandas 1.4.x (#1652)
- pandas.read_json needs to take in a decoded string - see pandas-dev/pandas#46935
1 parent f165e94 commit 5d0b2fb

File tree

2 files changed

+2
-1
lines changed

2 files changed

+2
-1
lines changed

python/ingestion/gcs_to_bq_util.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,7 @@ def load_values_blob_as_df(blob):
167167
168168
blob: google.cloud.storage.blob.Blob object"""
169169
json_string = blob.download_as_string()
170+
json_string = json_string.decode('utf-8')
170171
return values_json_to_df(json_string)
171172

172173

python/tests/test_gcs_to_bq.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ def testLoadValuesBlobAsDataframe(self):
2222
"""Tests that data in json list format is loaded into a
2323
pandas.DataFrame object using the first row as a header."""
2424
mock_attrs = {
25-
'download_as_string.return_value': json.dumps(self._test_data)}
25+
'download_as_string.return_value': json.dumps(self._test_data).encode('utf-8')}
2626
mock_blob = Mock(**mock_attrs)
2727
frame = gcs_to_bq_util.load_values_blob_as_df(mock_blob)
2828

0 commit comments

Comments
 (0)