Converting heatmap series to dataframe

[riodda]

Expected Behavior

stats should be a dataframe with all the parameters and all the results with properly named columns

Actual Behavior

stats is pandas series of different lenght, and the optimizing paramenter(in the example Equity) is missing label.

Steps to Reproduce

1. Do a strategy optimization

Additional info

I think that the the heatmap should be a full pandas dataframe with properly named columns and with run number as index. Currently it's quite tricky to get the best parameters for a simulation with aloto of paramers, haveing a proper dataframe as heatmap would really help.

• Backtesting version:
  Latest

[kernc]

stats is pandas series of different lenght, and the optimizing paramenter(in the example Equity) is missing label

Can you show an example of an inadequate and a proposed adequate result? Not sure I get it.

When Backtest.optimize(..., return_heatmap=True), as per docs, a tuple of two series is returned:

stats, heatmap = bt.optimize(..., return_heatmap=True)

You can easily convert a pandas series to a dataframe:

heatmap = heatmap.to_frame()

[riodda]

The scenario is that i run an iteration to find the best parameters for a strategy, currently if i convert heatmap to frame i have a dataframe with an uneven number of rows between the params and also the target of the optimization column is label less. this format of dataframe makes difficould (At least for me) to get immediately the best params of the optimization runs.
For example in the image attached thersold_buy is 2 items, thersold_sell is 4 items, thersold_long_close is 7 items etc. Also as you can see the equity column also is label less.

My workaround is to use stats._strategy convert into a string and decompose to get the params,

best_params = str(stats._strategy)
best_params_array=best_params[17:-2].split(',')
parameters = []    
for parameter in best_params_array:
        parameters.append(parameter.split('='))
    pars = pd.DataFrame(parameters,columns=['Parameter',ticker])

But if the dataframe will be output with a consistent number of rows for each columns (in bold what is now missing):

param1 param2 param3 **target**
0            3           6            1200
**0**            **3**            7           1300

Will make much easier to get the best params (or the worst, or the average) as the target columns is labelled and each column has the same number of rows.
Then my python is not really great therfore i might miss the way to get the best params.
Thanks for your hardwork.

[kernc]

Then my python is not really great therfore i might miss the way to get the best params.

Again, to get a frame, have you tried running:

heatmap = heatmap.to_frame('My Equity')

?

One simple way to get the best combinations is to run:

heatmap.sort_values()

---

Indeed, assigning heatmap a name=maximize if isinstance(maximize, str) else None might be the default. 👍

backtesting.py/backtesting/backtesting.py

Lines 816 to 818 in 0739f4b

	heatmap = pd.Series(np.nan,
	index=pd.MultiIndex.from_tuples([p.values() for p in param_combos],
	names=next(iter(param_combos)).keys()))

---

Additionally, stats._strategy is a strategy instance. You can query its attributes directly, e.g.

>>> stats._strategy.thersold_buy
2

[riodda]

heatmap = heatmap.to_frame('My Equity')

Yes i did, and the dataframe that is output has different row numbers between columns.

heatmap.sort_values()
Traceback (most recent call last):

  File "<ipython-input-36-221809f51301>", line 1, in <module>
    heatmap.sort_values()

TypeError: sort_values() missing 1 required positional argument: 'by'

here comes my lack of python knowledge 'by' the column i want to sort for has no name....

Regarding the method of using stats._strategy.thersold_buy i forgot that the optimize reruns the strategy with the best values (but it's a method that is rigid with the parameter names).

Many thanks for your help.

[kernc]

TypeError: sort_values() missing 1 required positional argument: 'by'

This should work on initial heatmap multiindex series as returned by .optimize(), i.e. before .to_frame(). (After .to_frame(), you need to specify a df column, e.g. 'My Equity' passed to to_frame. All df columns have names.)

[kernc]

Ok, having run it, I understand. Merely:

heatmap.to_frame()

retains the MultiIndex.

Instead try:

heatmap.reset_index(name='Equity')

[PilotGFX]

example:
optiheatmap_df = optiheatmap.to_frame(name='Result').reset_index().sort_values(by="Result",ascending=False)