Timezone parsing issue with plotly.py JSON serialization

[actioncrypto]

Hello,

This is a code snippet that shows how even when explicitly defining timezones, and stripping out any possible confusion, plot.ly will refuse to display the X axis in the local timezone, instead converting and showing UTC only.

My working code is slightly different, but the incorrect end behavior is the same in this case as well when originating from strings and simple DataFrames

inspecting the fig immediately before being called to draw graph indeed shows the correct US/Eastern timezone, which happens as expected when the x value of the trace is defined.

import plotly.plotly as py
from plotly.graph_objs import *
import pandas as pd
import sys, datetime
from datetime import timedelta

incoming = {
    7887966:    [0.601345, 152.21, 253.12, '2015-04-04 19:31:30'],
    7887967:    [0.075392, 19.08, 253.12, '2015-04-04 19:31:34'],
    7887968:    [0.385279, 97.39, 252.77, '2015-04-04 19:32:13'],
    7887969:    [0.330650, 83.69, 253.12, '2015-04-04 19:32:14'],
    7887970:    [0.110700, 28.02, 253.13, '2015-04-04 19:32:14']}

df = pd.DataFrame.from_dict(incoming, orient='index')
df.columns = ['a','b','y','time']
df['time'] = pd.to_datetime(df['time'])
df = df.set_index('time')

trace = Box(
    x=df.index.tz_localize('UTC').tz_convert('US/Eastern'),
    y=df['y'].values,
    name='timezone issues',
    fillcolor='#a5b404',
    opacity=0.66,
    marker=Marker(
        color='#a5b404'
    )
)

data = Data([trace])

axis_style = dict(
    zeroline=True,
    zerolinecolor='#444',
    zerolinewidth=1,
    gridcolor='#eee',
    autotick=True,
    showline=False
)

layout = Layout(
    title='Timezone offset issue',
    plot_bgcolor='#fff',
    hovermode='closest',
    autosize=True,
    height=610,
    width=1296,
    showlegend=False,
    xaxis=XAxis(
        axis_style,
        autorange=True,
        rangemode='normal',
        tickangle='auto',
        title='Time (note the UTC, when should be US Eastern)',
    ),
    yaxis=YAxis(
        axis_style,
        title='Y axis (no issues here)'
    )
)
fig = Figure(data=data, layout=layout)
py.plot(fig, filename='timezone-issues', world_readable=False)

[theengineear]

@actioncrypto , thanks for reporting. Is the time correct in UTC, or completely off?

[actioncrypto]

As far as plot.ly seems to be concerned, all time is UTC and that's final. In the example above, if you run it, you will see that the incoming 'hours' value of 19 is shown on the actual graph, when it should be displayed as 15 after it gets converted to US/Eastern (ideally 3pm & not 24hr style)

[theengineear]

Ok, on the same page now. You're right, there's not much to be done from the python-api side of things right now. I'm looking into it. I'll get back to you when I have more information.

[huixian]

Hi, can I check if there's more information on this? Thanks much!

[theengineear]

@huixian no more information at the moment, the python client continues to force things into being UTC. Apologies for the inconvenience for now.

[huixian]

@theengineear no worries; will work around it. :) thanks much for replying.

[actioncrypto]

Has there been any update for this forced UTC conversion on the x-axis request? Barring a bug-fix patch, is there anything we can do to hack the x-axis into local timezone?

[panthertrader]

Any updates on that`?
I get the same issues with :

fig = FF.create_ohlc(df.Open, df.High, df.Low, df.Close, dates=df.index.tz_convert('US/Eastern'))
    # Update the fig - all options here: https://plot.ly/python/reference/#Layout
fig['layout'].update({
    'title':  instrument ,
    'yaxis': {'title':  instrument} 

}) 
plotly.offline.iplot(fig  )

[mckelvin]

A workaround for this: to convert the datetime from aware datetime to naive datetime, by removing the tzinfo of datetime.

If you're using pandas, you can use Series.tz_localize(None):

print df.index[0]
# 2017-02-06 21:00:00+08:00
df_index = df.index.tz_localize(None)
print df_index[0]
# 2017-02-06 21:00:00

[KieranWynn]

I would also like to give plotly tz-aware datetimes and have them shown in the provided timezone. Maybe have an API-configurable option to select the viewing timezone?

[pohmelie]

It looks like conversion to utc timezone happens here. I have comment this conversion and got normal looking local time with proper timezone for x-axis.

[pubudug]

Is there a workaround for this if I'm not using pandas?

[tinkerjs]

I tried what @pohmelie mentioned and that worked for me. I did notice that when i used cufflinks to create charts the timezone is honored, when i use the graph object it is not. Its seems as though plotly can check if a TZ is set and if so then ignore otherwise convert to UTC.

[medihack]

Why does Plotly convert the datetime to UTC at all, and not handle the timezone as it was set by the user? Is this somehow internally important?

[jayvoy]

This is a solution to McKlevin's idea for those not using pandas. I used the localize function from pytz and solved the plotly hours conversion problem.

[JermellBeane]

the fix is to localize to UTC, convert to your user's tz, then remove the tz_info

worked for me something like:
from datetime import timezone
import pandas as pd
df['formatted_time']=pd.to_datetime(df["time"],unit='ms').dt.tz_localize('UTC').dt.tz_convert(timezone(user_tz)).dt.tz_localize(None)

Side note:
My timestamps are already in UTC in a time column formatted as millis since epoch and are timezone naive before I do this, and I'm operating on a column in my df which is why you need .dt.tz_localize/tz_convert

[hnazkani]

For financial timeseries data looking at plots in local timezone is very critical.
And It doesn't seem to be fixed even after 3 years. Is there a timeline to fix this ?

[VldmrB]

For a quick temp fix, the suggestion from @pohmelie in #209 (comment) works great. The code has changed slightly, but commenting out the following should get the job done:

plotly.py/plotly/utils.py

Lines 294 to 306 in 663a386

	if not (PY36 and (isinstance(obj, datetime.datetime) and
	obj.tzinfo is None)):
	try:
	obj = obj.astimezone(pytz.utc)
	except ValueError:
	# we'll get a value error if trying to convert with naive datetime
	pass
	except TypeError:
	# pandas throws a typeerror here instead of a value error, it's OK
	pass
	except AttributeError:
	# we'll get an attribute error if astimezone DNE
	raise NotEncodable

Alternatively, after having manipulated the datetime objects however needed, they can be converted to strings, and then passed to graphs as data: https://stackoverflow.com/a/54638376/7432972

[patricksheehan]

Checking in on this. When I make a plot with a tz-aware datetime (DateTimeIndex with a timezone in pandas) the correct times are displayed. When I export to html or an image, the timezone is SILENTLY converted to UTC. I'm sure the workaround provided works, but is someone still working on this?

[shkramer13]

I'm having issues with this as well in Jupyter Lab. If I create a plot with a tz-aware datetime, the plot will display in local time if I simply call the name of the figure object at the bottom of a code cell. But if I use iplot(fig) instead of fig, the figure displays in UTC.

[JermellBeane]

See my comment for an ugly hack. @shkramer13

…

________________________________ From: shkramer13 <[email protected]> Sent: Tuesday, March 26, 2019 1:17:56 PM To: plotly/plotly.py Cc: Beane, Jermell [COMRES/WR/US]; Comment Subject: [EXTERNAL] Re: [plotly/plotly.py] Timezone parsing issue with python plot.ly API (#209) I'm having issues with this as well in Jupyter Lab. If I create a plot with a tz-aware datetime, the plot will display in local time if I simply call the name of the figure object at the bottom of a code cell. But if I use iplot(fig) instead of fig, the figure displays in UTC. — You are receiving this because you commented. Reply to this email directly, view it on GitHub [github.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_plotly_plotly.py_issues_209-23issuecomment-2D476782799&d=DwMFaQ&c=jOURTkCZzT8tVB5xPEYIm3YJGoxoTaQsQPzPKJGaWbo&r=KVI6-geanVBy3OgJ41M5unmKFYQlg1Fy7gYHnQgR2Q4&m=AB_wO6FO6vKzNZs-NMO0lkaxnjmtPwk9GHgMCfMKwo0&s=V3hiEtAw-t61SzJjM1DIrTaelWoWcr3mw6BBqw3QuF8&e=>, or mute the thread [github.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_Amht6yp7TrRQuP5LczvZ9E4TeAlMFzs3ks5vamRUgaJpZM4D-2DH1F&d=DwMFaQ&c=jOURTkCZzT8tVB5xPEYIm3YJGoxoTaQsQPzPKJGaWbo&r=KVI6-geanVBy3OgJ41M5unmKFYQlg1Fy7gYHnQgR2Q4&m=AB_wO6FO6vKzNZs-NMO0lkaxnjmtPwk9GHgMCfMKwo0&s=aogUGZWoF1XDl9ur2TLDDI8SlkIR-aZGezJrsBkX1N4&e=>.

[jonmmease]

Hi all, I haven't really dug into this yet but it's definitely important and something that we plan to take care of as a part of version 4. Thanks for helping each other out with workarounds in the meantime!

[jonmmease]

See version 4 proposal at #1581.

For dev-build installation instructions, see #1581 (comment).