String column changed in DataFrame Write

[josefmtd]

Steps to reproduce:
List the minimal actions needed to reproduce the behavior.

1. Create a dataframe that has a string column with space or comma
2. Write the dataframe to an InfluxDB measurement
3. An escape ('\') is added before every spaces (and/or) comma

in code:

now = datetime.datetime.now()
an_hour_ago = now - datetime.timedelta(hours = 1)

test = [{'a': an_hour_ago, 'b': 'hello world', 'c': 1}, 
        {'a': now, 'b': 'goodbye cruel world', 'c': 2}]

df_test = pandas.DataFrame(test)
df_test = df_test.set_index('a')

upload_data(df_test, 'test', []) # some wrapper function

Expected behavior:

select * from test
name: test
time b c

---

1599921346848720000 hello world 1
1599924946848720000 goodbye cruel world 2

Actual behavior:
Describe What actually happened.

> select * from test
name: test
time                b                     c
----                -                     -
1599921346848720000 hello\ world          1
1599924946848720000 goodbye\ cruel\ world 2

Specifications:

• Client Version: 1.10.0
• InfluxDB Version: 1.8.2
• Platform: Python Client API on Anaconda 3 Windows 10, InfluxDB running on Docker inside Ubuntu 20.04 VM (x64)

[josefmtd]

Additional information:

Escape character '\' only appears in field values, not in tag values.

Code:

import datetime
import pandas

from influxdb_client import InfluxDBClient
from influxdb_client.client.write_api import SYNCHRONOUS, PointSettings

now = datetime.datetime.now()
an_hour_ago = now - datetime.timedelta(hours = 1)

test = [{'a': an_hour_ago, 'b': 'hello world', 'c': 1, 'd': 'foo bar'}, 
        {'a': now, 'b': 'goodbye cruel world', 'c': 2, 'd': 'bar foo'}]

df_test = pandas.DataFrame(test)
df_test = df_test.set_index('a')

url = 'http://192.168.0.148:8086'

# InfluxDB Database
database = 'earthquake'
retention_policy = 'autogen'
bucket = f'{database}/{retention_policy}'

# InfluxDB Admin Username
username = 'admin'
password = 'admin'
token = f'{username}:{password}'

# InfluxDB API Organization
org = ''
    
# InfluxDB Client
client = InfluxDBClient(url = url,
                           token = token,
                           org = org)
point_settings = PointSettings()
    
write_api = client.write_api(write_options = SYNCHRONOUS,
                                 point_settings = point_settings)

write_api.write(bucket = bucket,
                           record = df_test,
                           data_frame_measurement_name='test',
                           data_frame_tag_columns=['d'])

Result:

> select * from test
name: test
time                b                     c d
----                -                     - -
1599921346848720000 hello\ world          1
1599924946848720000 goodbye\ cruel\ world 2
1600079584293173000 hello\ world          1 foo bar
1600083184293173000 goodbye\ cruel\ world 2 bar foo
>

[bednar]

Hi @josefmtd,

thanks for detail info. The issue will be fixed by #154.

If you would like to use a dev version then install client via:

pip install git+https://github.com/influxdata/influxdb-client-python.git@fix/dataframe-field-string

Regards

[josefmtd]

Cheers @bednar for the quick response and fix