Extend multicategory axes to more than 2 levels

[amellnik]

This is a request to add nested categorical axes. These are often used in "variability charts", examples of which can be seen here.

There's a few ways this could be implemented in practice. Rather than passing an array of values for y the user could pass an object like:

{
  "popcorn": ["gourmet", "gourmet", "plain", ...]
  "batch": ["large", "small", "large", ...]
  ...
}

or simply pass a category name / number (1 or "gourmet-large-little") and pass the nesting information for the axis in the layout section.

(I was surprised not to find an open issue for this -- it's possible that there is one but I didn't search using the right terms).

Edit: There is some limited support for this as shown here but it involves hacking the axis together by hand and would be difficult to use in an automated manner.

[jackparmer]

If your team is used to JMP, this axis style is definitely a must-have.

I think using annotations with subplots for the lower axes labels (like in the example you linked) is fairly robust though. This plot for example would be 3 subplots with 3 annotation labels:

The popcorn example would be 4 subplots with 6 annotations:

There isn't another issue for this ASFAIK. To make JMP-style labels part of the JSON schema, a company would probably have to sponsor this feature to get it on the roadmap: https://plot.ly/products/consulting-and-oem/

[amellnik]

I've been trying to figure out if there's a way to generate similar plots using the current plotly capabilities. One option is to just concatenate the keys and use that for a categorical axis, but that's hard to read.

I also tried to set this up using multiple axes (along the lines of this demo) but I haven't had much success, mostly because the position property doesn't accept negative values. There might be a way to get these to look right by fiddling with layout, but I haven't figured it out yet. (I'm automatically generating the positions of the labels on each axis using some julia code that's not shown, but this step could easily be rewritten in javascript).

I also saw a demo that used a normal categorical axis for the first level and annotations for the 2nd, but it was not clear if it could be extended to more than two levels.

Was there some automated process that generated the annotations shown here or was that all done by hand?

[etpinard]

Some fondation work has been done in: #3300

Extending axis.type: 'multicategory' to support more than 2 levels, should allow us to graph the above examples.

[concaho]

Hi @etpinard ,
Do you have any update of this enhancement please ?
I would like to do this example but in three levels:

import plotly.graph_objects as go
x = [
["BB+", "BB+", "BB+", "BB", "BB", "BB"],
[16, 17, 18, 16, 17, 18,]
]
fig = go.Figure()
fig.add_bar(x=x,y=[1,2,3,4,5,6])
fig.add_bar(x=x,y=[6,5,4,3,2,1])
fig.update_layout(barmode="relative")
fig.show()

I imagine just change x for
x = [
["BB+", "BB+", "BB+", "BB", "BB", "BB"],
[16, 17, 18, 16, 17, 18],
[level3t,level3a,level3t,level3a,level3t,level3a]
]

Thanks a lot,
Hadrien

[emmanuelle]

Mentioned in the community forum https://community.plotly.com/t/multicategory-axis-how-many-categories-are-allowed/39563/2

[JanghyunJK]

+1

[aaron-kyo]

Can someone please comment on the status of this feature or at least tell us where in the source code we need to look to try and add this capability ourselves?

@etpinard, can you give a high level overview of what we would need to change to extend multicategory axis capability? Any help is greatly appreciated.

Thanks

[navykoo]

I'm extremly expecting this feature (multil categories over 2) being available or the official team can provide an extend approach for developers to use in short term.

[richardnm-2]

Did anybody find at least where the slicing occurs? As no error is thrown when 3 columns are inserted as x argument, and only the first 2 get their labels plotted, I tried at least to find the slicing, without success.

Any help on this matter is appreciated.

[richardnm-2]

I’ve decided to take a closer look at this. Initially I was trying to find out where plotly.py was slicing the dataframe, as I was trying to do this in Python.
The slicing actually occurs in plotly.js/src/plots/cartesian/set_convert.js, at ax.setupMultiCategory

for(j = 0; j < len; j++) {
        var v0 = arrayIn[0][j];
        var v1 = arrayIn[1][j];

        if(isValidCategory(v0) && isValidCategory(v1)) {
                list.push([v0, v1]);

                if(!(v0 in seen[0][1])) {
                        seen[0][1][v0] = seen[0][0]++;
                }
                if(!(v1 in seen[1][1])) {
                        seen[1][1][v1] = seen[1][0]++;
                }
        }
}

If I understood it correctly, here it also tries to sort the columns, after it collects the combinations.

list.sort(function(a, b) {
        var ind0 = seen[0][1];
        var d = ind0[a[0]] - ind0[b[0]];
        if(d) return d;

        var ind1 = seen[1][1];
        return ind1[a[1]] - ind1[b[1]];
});

The sorting does not account for all the levels, as per #3723 and #3908. I solved the sorting problem, implemented it inside ax.setupMultiCategory, but it seems there is another place where it also tries to order the categories. Note that even though sorting inside the modified function correctly sorts the x axis labels, the sequence of the line connections is still problematic. The sorting is already capable of dealing with multiple (more than 2) categories. Plot data:

x: [
        ["EE", "EE", "DD", "DD", "DD", "DD", "BB", "BB"],
        ["HH", "AA", "ZZ", "BB", "AA", "VV", "MM", "AA"],
]
y: [8, 9.5, 14, 17, 8, 10, 10, 9]

Original sorting changes only the second level, and not all of them.

Edited sorting algorithm:

Slightly editing the x data, and increasing the dimension:

x: [
        ["EE", "EE", "EE", "DD", "DD", "DD", "BB", "BB"],
        ["HH", "HH", "HH", "BB", "AA", "VV", "MM", "AA"],
        ["5", "1", "3", "4", "1", "2", "3", "4"],
];

After solving the dimension and sorting issues, I tried to implement some foundation for the multicategory with n-levels, in formatMultiCategory and getSecondaryLabelVals.

function formatMultiCategory(ax, out, hover) {
    var v = Math.round(out.x);
    // copy categories to prevent hover instability
-  var cats = ax._categories[v] || [];
+ var cats = ax._categories[v].map(function(cat) {return cat;}) || [];
+ var texts = cats.reverse().map(function(cat) {
+        return cat === undefined ? '' : String(cat);
+ });
+    ax.levels = cats.length;

    if(hover) {
        // TODO is this what we want?
-      out.text = tt2 + ' - ' + tt;
+     out.text = texts.at(-1) + ' - ' + texts.at(-2);
    } else {
        // setup for secondary labels
-      out.text = tt;
-      out.text2 = tt2;
+     out.texts = texts
    }
}

-function getSecondaryLabelVals(ax, vals) {
+function getSecondaryLabelVals(ax, vals, level) {
    var out = [];
    var lookup = {};

    for(var i = 0; i < vals.length; i++) {
        var d = vals[i];
-        if(lookup[d.text2]) {
-        lookup[d.text2].push(d.x);
+       var text = d.texts[level];
+       if(lookup[text]) {
+           lookup[text].push(d.x);
        } else {
-             lookup[text2] = [d.x];
+            lookup[text] = [d.x];
        }
    }

    for(var k in lookup) {
        out.push(tickTextObj(ax, Lib.interp(lookup[k], 0.5), k));
    }

    return out;
}

At this point, the rough data is ready for the multicategory, but I could not advance in the actual formatting and plotting of the relabeled data. Instead, as I can sort this before plotting, I removed the x axis and rendered a simple html table. Using the popcorn example commented above and the boxplot, the result is enough to get the base layout. This hack is pretty much useless on Python, and far from ideal regarding component widths, but with a frontend and dynamic width tables I guess I can get close enough with the current functionality.

Maybe with a little help I could try to further improve on the code and get the expected results.

[richardnm-2]

I've decided to try and implement this in Plotly.js, and the result turned up pretty good. Available in https://codepen.io/richardnm-2/pen/RwyjMrB

I also replaced the minified version inside plotly.py, and the results were also pretty promising. JMP like styles applied.

and the popcorn example from before, connecting cell means also available.

Couple of things I saw that went wrong:

• had to comment dividers.exit().remove() at src/plots/cartesian/axes.js drawDividers() in order to successfully draw the ticks in a loop, at /axes.js drawOne(), for all the tick levels to appear. Not sure how to better address the problem. The main effect is that the ticks get stuck to the side when the window is rolled.

• Sometimes labels depend on bottom margin to be correctly displayed. Maybe something wrong with some label naming. Increase it if necessary. (saw this plotting with plotly.py).

Other than the dividers.exit().remove() intervention that I see as a little abusive, had to directly manipulate gd._fullData at src\plots\cartesian\set_convert.js setupMultiCategory(). In order to correctly display the data, it was not enough to pass the sorted list (matrix) to setCategoryIndex(). As mentioned before, this should also solve #3908.

Other then these problems, I would be happy to receive a code review, maybe work for a PR. https://github.com/richardnm-2/plotly.js.

If anyone finds another bug except those already mentioned, feel free to post it, I can try and work on them.

[alexcjohnson]

@richardnm-2 fantastic, nice work!

Perhaps the easiest would be to create a PR into the main repo, that would be the easiest way for us to look at what you have and see if we can help resolve those two issues, as well as figure out what else might be needed to get it merged.

[richardnm-2]

Ok, just opened a draft PR. This is my first contribution, so please let me know if I have to do anything differently.
Also, solved the stacking dividers, calling the remove only on the first pass (last/closest to plot axis). Updated the codepen with this fix with the data from #3723, to show the sorting also working as expected.

I hope this is integrated, please keep me posted on the development and next steps!

[PhilippaTreacy]

Apologies if I'm not using this properly - first time to comment. Is it possible to use this for a heatmap in plotly.js? I have doing it with the same sequence for the trace but it doesnt appear to work I just get y1 labels e.g. { x: [x1, x2], y: [y1, y2, y3 ], z: channelArray[channelList[parseInt(i)]], type: 'heatmap', xaxis: x${parseInt(i)+1}, yaxis: y${parseInt(i)+1}, name: channelList[i], showscale: parseInt(i)==1? true: false, })