Skip to content

Invalid or erroneous drawdown calculations in equity plot for resampled datasets #162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
eervin123 opened this issue Oct 27, 2020 · 3 comments
Labels
bug Something isn't working

Comments

@eervin123
Copy link
Contributor

Expected Behavior

Plots, where data was resampled due to >10,000 bars, should calculate max drawdowns and drawdown periods similar to those computed in the _compute_stats method

Actual Behavior

It appears when plots are downsampled in the Backtest.plot(resample=True) the drawdown calculations and possibly drawdown duration (beginning and ending periods) are miscalculated.

Here are two examples of such charts:
https://www.realitysharesadvisors.com/fund_dash/fix_this_eth.html
https://www.realitysharesadvisors.com/fund_dash/fix_this.html

Note the legend in one of the charts showing 0 days of max drawdown
image

Here is an example where the red line is misplaced on an incorrect drawdown.
bokeh_plot (1)

Steps to Reproduce

  1. I was using 5-minute candles on multiple years of data.
  2. Run Backtest.plot() on a large dataset and it should reproduce the result...Hopefully, if not, I am happy to privately share my datafiles and strategy class for you to reproduce on your system.

Additional info

This was also referenced in a different issue #156 (comment)

  • Backtesting version: 0.2.3
  • bokeh 2.2.3
  • python 3.8.5
  • pandas 1.1.3
@kernc
Copy link
Owner

kernc commented Oct 27, 2020

Plots, where data was resampled due to >10,000 bars, should calculate max drawdowns and drawdown periods similar to those computed in the _compute_stats method

Resampling of equity curve is computed this way:

equity_data = equity_data.resample(freq, label='right').agg(_EQUITY_AGG).dropna(how='all')

_EQUITY_AGG = {
'Equity': 'mean',
'DrawdownPct': 'max',
'DrawdownDuration': 'max',
}

('Equity' should maybe aggregate with 'last'. 🤔)


Interestingly, the whole thing works correctly (conjecture) on this toy example:

from backtesting.test._test import SmaCross
bt = Backtest(GOOG, SmaCross)
bt.run(fast=3, slow=8)
bt.plot()

Screenshot_2020-10-27_15-45-18
Resampling daily to bi-monthly:

bt.plot(resample='2M', superimpose=False)

Screenshot_2020-10-27_15-49-09

Max DD Duration seems to be rounded to round units (10 2-months).

Therefore, it must be something specific to your dataset and something in the way drawdown intersections are computed:

def _plot_equity_section():
"""Equity section"""
# Max DD Dur. line
equity = equity_data['Equity'].copy()
dd_end = equity_data['DrawdownDuration'].idxmax()
if np.isnan(dd_end):
dd_start = dd_end = equity.index[0]
else:
dd_start = equity[:dd_end].idxmax()
# If DD not extending into the future, get exact point of intersection with equity
if dd_end != equity.index[-1]:
dd_end = np.interp(equity[dd_start],
(equity[dd_end - 1], equity[dd_end]),
(dd_end - 1, dd_end))

dd_timedelta_label = df['datetime'].iloc[int(round(dd_end))] - df['datetime'].iloc[dd_start]
fig.line([dd_start, dd_end], equity.iloc[dd_start],
line_color='red', line_width=2,
legend_label='Max Dd Dur. ({})'.format(dd_timedelta_label)
.replace(' 00:00:00', '')
.replace('(0 days ', '('))

I don't suppose something strikes you as odd at first glance? 😅

@kernc kernc added the bug Something isn't working label Oct 27, 2020
@eervin123
Copy link
Contributor Author

backtesting.py/backtesting/lib.py

Lines 62 to 66 in 05329de

 _EQUITY_AGG = { 
     'Equity': 'mean', 
     'DrawdownPct': 'max', 
     'DrawdownDuration': 'max', 
 } 

('Equity' should maybe aggregate with 'last'. 🤔)

That did the trick. Here is the before picture
image

with a zoom in on the problem period.
image
As you can see above, this is very close and likely the culprit from some downsampling throwing an erroneous peak.

Here is an after picture after I set the 'Equity' : 'last' in the _EQUITY_AGG dict.
image

The Max Drawdown Duration perfectly matches what is expected from the _compute_stats calculations.

@kernc
Copy link
Owner

kernc commented Oct 27, 2020

Pleasure to work with you.

@kernc kernc closed this as completed in dfadfd7 Oct 27, 2020
Goblincomet pushed a commit to Goblincomet/forex-trading-backtest that referenced this issue Jul 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants