Unicode Decode Error when parsing DataFrame results using query_data_frame method

[bstanway22]

We have a basic query which generates a table of results which is queried using the query_data_frame influxdb-client method.

Example query:

from(bucket: "mybucket") |> range(start: -300s) |> filter(fn: (r) => r["_measurement"] == "data") |> filter(fn: (r) => r["_field"] == "cost") |> filter(fn: (r) => r["app"] == "app1") |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") |> group(columns: ["path", "instrument", "eng_units"]) |> top(n: 1, columns: ["_time"]) |> keep(columns: ["cost"]) |> rename(columns: {}) |> yield(name: "tuning")

In a recent update from version 1.32 to 1.33, the query_data_frame method intermittently results in a utf-8 decoding error when the 'read' method of the _StreamReaderToWithAsyncRead method is called. Comparing versions, this appears to be a recent addition to the flux_parser_csv.py module (see image attachment). Running the above query and processing the result in version 1.32 runs fine with no issues.

Here is the output of the traceback, where 'q' is the above query:

[06:59] Ben Stanway UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc2 in position 2047: unexpected end of data 2022-10-12 09:13:13,141 ERROR _handle_get_app_table(209) Unable to generate table. Traceback (most recent call last): File "/home/tango_api/tango_api_venv/lib/python3.7/site-packages/tango_api/influx/influx_connection.py", line 59, in query_async return await c.query_api().query_data_frame(q, data_frame_index=data_frame_index) File "/home/tango_api/tango_api_venv/lib/python3.7/site-packages/influxdb_client/client/query_api_async.py", line 160, in query_data_frame async for dataframe in _generator: File "/home/tango_api/tango_api_venv/lib/python3.7/site-packages/influxdb_client/client/flux_csv_parser.py", line 139, in _parse_flux_response_async async for csv in self._reader: File "/home/tango_api/tango_api_venv/lib/python3.7/site-packages/aiocsv/readers.py", line 37, in __anext__ return await self._parser.__anext__() File "aiocsv/_parser.pyx", line 216, in parser File "/home/tango_api/tango_api_venv/lib/python3.7/site-packages/influxdb_client/client/flux_csv_parser.py", line 389, in read return (await self.response.read(size)).decode(_UTF_8_encoding)

See example of the

Thanks for taking the time to look into this and keep up the great work!

Ben.

[bednar]

Hi @bstanway22,

thanks for using our client.

Is there a way how I can reproduce this error locally? Do you think that the error depends on the amount of data?

Regards

[bednar]

Hi @bstanway22,

I have prepared a fixed version within #518. Development version of the client will be available after #518 acceptance, stable version in regular release cycle.

You could use a fixed version by:

pip install git+https://github.com/influxdata/influxdb-client-python.git@async-utf8-two-bytes-character

Regards

[bstanway22]

Hi,

Apologies for the late reply. Thank you so much for resolving this issue in such a short time frame. Your efforts are invaluable to our work and we really appreciate the care and attention you put into your product.

Keep up the great work!

Ben.