ValueError: Invalid value for superimpose: Upsampling not supported.

[sedna16]

This error start appearing a day ago when I was testing several strategies.
I didnt changed anything in my codes before this happened.

The backtest doesnt continue and and the chart does not appear.

C:\Users\admin\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\backtesting\backtesting.py:1042: FutureWarning: Index.is_all_dates is deprecated, will be removed in a future version.  check index.inferred_type instead
  if (not data.index.is_all_dates and
C:\Users\admin\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\backtesting\backtesting.py:1057: FutureWarning: Index.__and__ operating as a set operation is deprecated, in the future this will be a logical operation matching Series.__and__.  Use index.intersection(other) instead
  if len(data.columns & {'Open', 'High', 'Low', 'Close', 'Volume'}) != 5:
ValueError: Invalid value for `superimpose`: Upsampling not supported.

[sedna16]

Ok I think I found why this happened.

doenst work
stockcode - NASDAQ:TSLA
date format - strftime('%Y-%m-%d')

works here
stockcode - NASDAQ:TSLA
date format - strftime('%Y-%m-%d %H:%M:%S')

[kernc]

This is the relevant code part.

backtesting.py/backtesting/_plotting.py

Lines 423 to 445 in 1ee5670

	time_resolution = pd.DatetimeIndex(df['datetime']).resolution
	resample_rule = (superimpose if isinstance(superimpose, str) else
	dict(day='M',
	hour='D',
	minute='H',
	second='T',
	millisecond='S').get(time_resolution))
	if not resample_rule:
	warnings.warn(
	f"'Can't superimpose OHLC data with rule '{resample_rule}'"
	f"(index datetime resolution: '{time_resolution}'). Skipping.",
	stacklevel=4)
	return
	df2 = (df.assign(_width=1).set_index('datetime')
	.resample(resample_rule, label='left')
	.agg(dict(OHLCV_AGG, _width='count')))
	# Check if resampling was downsampling; error on upsampling
	orig_freq = _data_period(df['datetime'])
	resample_freq = _data_period(df2.index)
	if resample_freq < orig_freq:
	raise ValueError('Invalid value for `superimpose`: Upsampling not supported.')

I wonder what

import pandas as pd

df: pd.DataFrame  # your OHLC data

pd.DatetimeIndex(df.index).resolution  # ?

returns for your data?

[kernc]

@sedna16 Can you show a small, couple-lines example dataframe/index where this fails? I don't quite get this part:

doenst work
stockcode - NASDAQ:TSLA
date format - strftime('%Y-%m-%d')

[sedna16]

ok, I'll try to recreate this.

[sedna16]

@kernc - here's the whole print message

Backtest: Retreiving results with chart.
mySQL: rows from trading_data_api has been fetched.
                           open        high         low       close    volume
date
2017-01-30 14:30:00   50.506001   51.057999   49.419998   50.125999  19005500
2017-01-31 14:30:00   49.848000   51.178001   49.540001   50.386002  20580500
2017-02-01 14:30:00   50.610001   50.639999   49.810001   49.848000  19794000
2017-02-02 14:30:00   49.667999   50.484001   49.542000   50.310001  12499000
2017-02-03 14:30:00   50.382000   50.436001   49.936001   50.265999  10933500
...                         ...         ...         ...         ...       ...
2021-01-25 14:30:00  855.000000  900.400024  838.820007  880.799988  41173400
2021-01-26 14:30:00  891.380005  895.900024  871.599976  883.090027  23131600
2021-01-27 14:30:00  870.349976  891.500000  858.659973  864.159973  27334000
2021-01-28 14:30:00  820.000000  848.000000  801.000000  835.429993  26378000
2021-01-29 21:00:02  830.000305  842.409973  780.099976  793.530029  33999137

[1008 rows x 5 columns]
C:\Users\admin\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\backtesting\backtesting.py:1042: FutureWarning: Index.is_all_dates is deprecated, will be removed in a future version.  check index.inferred_type instead
  if (not data.index.is_all_dates and
C:\Users\admin\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\backtesting\backtesting.py:1057: FutureWarning: Index.__and__ operating as a set operation is deprecated, in the future this will be a logical operation matching Series.__and__.  Use index.intersection(other) instead
  if len(data.columns & {'Open', 'High', 'Low', 'Close', 'Volume'}) != 5:
Invalid value for `superimpose`: Upsampling not supported.

[sedna16]

this is from yahoo, 7days 1min

[kernc]

Ok, the bug is due to weird timestamps:

...
2021-01-27 14:30:00
2021-01-28 14:30:00
2021-01-29 21:00:02  # <-- second change

for which pd.DatetimeIndex(...).resolution returns second, whereas the data period is computed as the median difference between successive values:

backtesting.py/backtesting/_util.py

Lines 37 to 40 in 1ee5670

	def _data_period(index) -> Union[pd.Timedelta, Number]:
	"""Return data index period as pd.Timedelta"""
	values = pd.Series(index[-100:])
	return values.diff().dropna().median()

which is more than T (one minute) that second resolution downsamples to.

[sedna16]

this only happens on intra yahoo finance data. on finnhub, its ok

[ruipinghe]

I ran into this problem as well, should the timeseries be equal length? mine is like this:
'2011-06-05 14:20:32', '2011-06-05 14:21:21',
'2011-06-05 14:23:47', '2011-06-05 14:26:29', after I resample this issue solved, but I don't want it to be resampled, any else ways to solve this?

[stephanebruckert]

In my case ignoring the last row from the dataset fixed the issue:

2021-10-06 19:30:00+01:00
2021-10-06 19:45:00+01:00
2021-10-06 20:06:02+01:00

df = df.iloc[:-1]

[rigidlab]

I also use a dataframe returned from yfinance download. It looks like those weird timestamps have 0 volume, so I filter it out and rather than resample. Seems to work okay

df=df[df.Volume!=0]

[AgarwalPragy]

Getting the same issue.

It seems like this is because of the closing time not being at 00:00:00 AM.

The datetimes I have look like 2000-01-03 18:30:00, which is why pd.DatetimeIndex(df['datetime']).resolution returns minute, even though the orig_freq of data is 1 days.

[bravegag]

Getting the same issue with 1h candlesticks while attempting to predict 1h movements.