Skip to content

Transform react #2577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Apr 26, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
bbe3533
remove no longer needed hack for finance in findArrayAttributes
alexcjohnson Apr 18, 2018
ed24765
_commonLength -> _length
alexcjohnson Apr 18, 2018
fcc459d
fix #2508, fix #2470 - problems with Plotly.react and aggregate trans…
alexcjohnson Apr 18, 2018
7044a13
standardize transforms handling of _length
alexcjohnson Apr 19, 2018
88b7b43
:racehorse: refactor sort transform from O(n^2) to O(n)
alexcjohnson Apr 19, 2018
cf4c9c3
heatmap&carpet/has_columns -> Lib.is1D
alexcjohnson Apr 20, 2018
e84d4b9
close #1410 - yes, stop pruning in nestedProperty
alexcjohnson Apr 20, 2018
b436d52
ensure every trace defines _length in supplyDefaults, and abort trans…
alexcjohnson Apr 22, 2018
2a41f9e
react+transforms PR review edits
alexcjohnson Apr 24, 2018
965bcfb
fixes and test for checklist in #2508
alexcjohnson Apr 24, 2018
6fae229
some fixes and tests for empty data arrays
alexcjohnson Apr 24, 2018
79295f1
rename violins mock that doesn't contain violin traces
alexcjohnson Apr 25, 2018
5e9aa65
transforms mock
alexcjohnson Apr 26, 2018
690eb95
clean up keyedContainer for possibly missing array
alexcjohnson Apr 26, 2018
a244cec
violin should not explicitly set whiskerwidth
alexcjohnson Apr 26, 2018
92bd5d2
add _length and stop slicing in scattercarpet
alexcjohnson Apr 26, 2018
dc6de2f
clean up handling of colorscale defaults
alexcjohnson Apr 26, 2018
03956e1
include count aggregates in _arrayAttrs - so they remap correctly
alexcjohnson Apr 26, 2018
f439e41
in diffData I had _fullInput in the new trace, but not the old!
alexcjohnson Apr 26, 2018
e47e6a9
_compareAsJSON for groupby styles
alexcjohnson Apr 26, 2018
aa30ad6
update plot_api_test to :lock: recent changes in react/transforms etc
alexcjohnson Apr 26, 2018
4c70826
lint
alexcjohnson Apr 26, 2018
7279b55
+shapes & annotations3d in react-noop test
alexcjohnson Apr 26, 2018
3e250df
tweak docs & remove commented-out code
alexcjohnson Apr 26, 2018
cd08479
reactWith -> reactTo
alexcjohnson Apr 26, 2018
0414147
separate svg and gl traces in react-noop tests
alexcjohnson Apr 26, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion src/plot_api/plot_api.js
Original file line number Diff line number Diff line change
Expand Up @@ -2267,7 +2267,10 @@ exports.react = function(gd, data, layout, config) {
gd.layout = layout || {};
helpers.cleanLayout(gd.layout);

Plots.supplyDefaults(gd);
// "true" skips updating calcdata and remapping arrays from calcTransforms,
// which supplyDefaults usually does at the end, but we may need to NOT do
// if the diff (which we haven't determined yet) says we'll recalc
Plots.supplyDefaults(gd, true);
Copy link
Contributor

@etpinard etpinard Apr 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a big fan of this pattern.

At the very least we should turn this into Plots.supplyDefaults(gd, {skipDefaultsUpdateCalc: 1}) to make it more readable and extendable.

Personally, I'd vote for removing the newly added Plots.supplyDefaultsUpdateCalc() from Plots.supplyDefaults and call Plotly.supplyDefaultsUpdateCalc after Plots.supplyDefaults in the places that new it.

On master, we have:

image

where the calls in validate.js and rangeslider/draw.js (and maybe more) don't need to update calc.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps the name supplyDefaultsUpdateCalc isn't the best - though you could argue the same about calcTransform, which happens during the calc step but updates the data arrays in _fullData. supplyDefaultsUpdateCalc has one simple function when there are no transforms, to attach the new traces to the old calcdata in case we're not going to run a recalc, but when there are transforms it also (first) pulls the transformed array attributes back from the old fullTrace to the new one. So I think in fact we do need this in most instances, and there are some external callers (like streambed) that depend on this as well.

So I'm going to keep it as part of the default supplyDefaults behavior, but I'm happy to make it into an options object for clarity.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cleaner supplyDefaults API -> 2a41f9e


var newFullData = gd._fullData;
var newFullLayout = gd._fullLayout;
Expand All @@ -2289,6 +2292,9 @@ exports.react = function(gd, data, layout, config) {

// clear calcdata if required
if(restyleFlags.calc || relayoutFlags.calc) gd.calcdata = undefined;
// otherwise do the calcdata updates and calcTransform array remaps that we skipped earlier
else Plots.supplyDefaultsUpdateCalc(gd.calcdata, newFullData);

if(relayoutFlags.margins) helpers.clearAxisAutomargins(gd);

// Note: what restyle/relayout use impliedEdits and clearAxisTypes for
Expand Down
45 changes: 33 additions & 12 deletions src/plot_api/plot_schema.js
Original file line number Diff line number Diff line change
Expand Up @@ -171,9 +171,12 @@ exports.isValObject = function(obj) {
exports.findArrayAttributes = function(trace) {
var arrayAttributes = [];
var stack = [];
var isArrayStack = [];
var baseContainer, baseAttrName;

function callback(attr, attrName, attrs, level) {
stack = stack.slice(0, level).concat([attrName]);
isArrayStack = isArrayStack.slice(0, level).concat([attr && attr._isLinkedToArray]);

var splittableAttr = (
attr &&
Expand All @@ -190,33 +193,51 @@ exports.findArrayAttributes = function(trace) {

if(!splittableAttr) return;

var astr = toAttrString(stack);
var val = Lib.nestedProperty(trace, astr).get();
if(!Lib.isArrayOrTypedArray(val)) return;

arrayAttributes.push(astr);
crawlIntoTrace(baseContainer, 0, '');
}

function toAttrString(stack) {
return stack.join('.');
function crawlIntoTrace(container, i, astrPartial) {
var item = container[stack[i]];
var newAstrPartial = astrPartial + stack[i];
if(i === stack.length - 1) {
if(Lib.isArrayOrTypedArray(item)) {
arrayAttributes.push(baseAttrName + newAstrPartial);
}
}
else {
if(isArrayStack[i]) {
if(Array.isArray(item)) {
for(var j = 0; j < item.length; j++) {
if(Lib.isPlainObject(item[j])) {
crawlIntoTrace(item[j], i + 1, newAstrPartial + '[' + j + '].');
}
}
}
}
else if(Lib.isPlainObject(item)) {
crawlIntoTrace(item, i + 1, newAstrPartial + '.');
}
}
}

baseContainer = trace;
baseAttrName = '';
exports.crawl(baseAttributes, callback);
if(trace._module && trace._module.attributes) {
exports.crawl(trace._module.attributes, callback);
}

if(trace.transforms) {
var transforms = trace.transforms;

var transforms = trace.transforms;
if(transforms) {
for(var i = 0; i < transforms.length; i++) {
var transform = transforms[i];
var module = transform._module;

if(module) {
stack = ['transforms[' + i + ']'];
baseAttrName = 'transforms[' + i + '].';
baseContainer = transform;

exports.crawl(module.attributes, callback, 1);
exports.crawl(module.attributes, callback);
}
}
}
Expand Down
30 changes: 17 additions & 13 deletions src/plots/plots.js
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@ var extraFormatKeys = [
// gd._fullLayout._transformModules
// is a list of all the transform modules invoked.
//
plots.supplyDefaults = function(gd) {
plots.supplyDefaults = function(gd, skipCalcUpdate) {
var oldFullLayout = gd._fullLayout || {};

if(oldFullLayout._skipDefaults) {
Expand Down Expand Up @@ -458,24 +458,28 @@ plots.supplyDefaults = function(gd) {
}

// update object references in calcdata
if(oldCalcdata.length === newFullData.length) {
for(i = 0; i < newFullData.length; i++) {
var newTrace = newFullData[i];
var cd0 = oldCalcdata[i][0];
if(cd0 && cd0.trace) {
if(cd0.trace._hasCalcTransform) {
remapTransformedArrays(cd0, newTrace);
} else {
cd0.trace = newTrace;
}
}
}
if(!skipCalcUpdate && oldCalcdata.length === newFullData.length) {
plots.supplyDefaultsUpdateCalc(oldCalcdata, newFullData);
}

// sort base plot modules for consistent ordering
newFullLayout._basePlotModules.sort(sortBasePlotModules);
};

plots.supplyDefaultsUpdateCalc = function(oldCalcdata, newFullData) {
for(var i = 0; i < newFullData.length; i++) {
var newTrace = newFullData[i];
var cd0 = oldCalcdata[i][0];
if(cd0 && cd0.trace) {
if(cd0.trace._hasCalcTransform) {
remapTransformedArrays(cd0, newTrace);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remapTransformedArrays is only called from here, maybe we could merge it in here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merged and 🐎 remapTransformedArrays -> 2a41f9e

} else {
cd0.trace = newTrace;
}
}
}
};

/**
* Make a container for collecting subplots we need to display.
*
Expand Down
2 changes: 2 additions & 0 deletions src/transforms/aggregate.js
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,8 @@ exports.calcTransform = function(gd, trace, opts) {
enabled: true
});
}

trace._length = groupings.length;
};

function aggregateOneArray(gd, trace, groupings, aggregation) {
Expand Down
184 changes: 184 additions & 0 deletions test/jasmine/tests/plot_api_test.js
Original file line number Diff line number Diff line change
Expand Up @@ -2955,6 +2955,190 @@ describe('Test plot api', function() {
.then(done);
});

function aggregatedPie(i) {
var labels = i <= 1 ?
['A', 'B', 'A', 'C', 'A', 'B', 'C', 'A', 'B', 'C', 'A'] :
['X', 'Y', 'Z', 'Z', 'Y', 'Z', 'X', 'Z', 'Y', 'Z', 'X'];
var trace = {
type: 'pie',
values: [4, 1, 4, 4, 1, 4, 4, 2, 1, 1, 15],
labels: labels,
transforms: [{
type: 'aggregate',
groups: labels,
aggregations: [{target: 'values', func: 'sum'}]
}]
};
return {
data: [trace],
layout: {
datarevision: i,
colorway: ['red', 'orange', 'yellow', 'green', 'blue', 'violet']
}
};
}

var aggPie1CD = [[
{v: 26, label: 'A', color: 'red', i: 0},
{v: 9, label: 'C', color: 'orange', i: 2},
{v: 6, label: 'B', color: 'yellow', i: 1}
]];

var aggPie2CD = [[
{v: 23, label: 'X', color: 'red', i: 0},
{v: 15, label: 'Z', color: 'orange', i: 2},
{v: 3, label: 'Y', color: 'yellow', i: 1}
]];

function aggregatedScatter(i) {
return {
data: [{
x: [1, 2, 3, 4, 6, 5],
y: [2, 1, 3, 5, 6, 4],
transforms: [{
type: 'aggregate',
groups: [1, -1, 1, -1, 1, -1],
aggregations: i > 1 ? [{func: 'last', target: 'x'}] : []
}]
}],
layout: {daterevision: i + 10}
};
}

var aggScatter1CD = [[
{x: 1, y: 2, i: 0},
{x: 2, y: 1, i: 1}
]];

var aggScatter2CD = [[
{x: 6, y: 2, i: 0},
{x: 5, y: 1, i: 1}
]];

function aggregatedParcoords(i) {
return {
data: [{
type: 'parcoords',
dimensions: [
{label: 'A', values: [1, 2, 3, 4]},
{label: 'B', values: [4, 3, 2, 1]}
],
transforms: i ? [{
type: 'aggregate',
groups: [1, 2, 1, 2],
aggregations: [
{target: 'dimensions[0].values', func: i > 1 ? 'avg' : 'first'},
{target: 'dimensions[1].values', func: i > 1 ? 'first' : 'avg'}
]
}] :
[]
}]
};
}

var aggParcoords0Vals = [[1, 2, 3, 4], [4, 3, 2, 1]];
var aggParcoords1Vals = [[1, 2], [3, 2]];
var aggParcoords2Vals = [[2, 3], [4, 3]];

function checkCalcData(expectedCD) {
return function() {
expect(gd.calcdata.length).toBe(expectedCD.length);
expectedCD.forEach(function(expectedCDi, i) {
var cdi = gd.calcdata[i];
expect(cdi.length).toBe(expectedCDi.length, i);
expectedCDi.forEach(function(expectedij, j) {
expect(cdi[j]).toEqual(jasmine.objectContaining(expectedij));
});
});
};
}

function checkValues(expectedVals) {
return function() {
expect(gd._fullData.length).toBe(1);
var dims = gd._fullData[0].dimensions;
expect(dims.length).toBe(expectedVals.length);
expectedVals.forEach(function(expected, i) {
expect(dims[i].values).toEqual(expected);
});
};
}

function reactWith(fig) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cute tests, especially this function name 💟

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... and I'm impressed that bbe3533 and standardizing _length was all we needed to get transforms to work with splom and parcoords 🎉

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would actually name this reactTo personally, in both senses of the word 'to' (with and towards) :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call @nicolaskruchten -> reactTo in cd08479

return function() { return Plotly.react(gd, fig); };
}

it('can change pie aggregations', function(done) {
Plotly.newPlot(gd, aggregatedPie(1))
.then(checkCalcData(aggPie1CD))

.then(reactWith(aggregatedPie(2)))
.then(checkCalcData(aggPie2CD))

.then(reactWith(aggregatedPie(1)))
.then(checkCalcData(aggPie1CD))
.catch(failTest)
.then(done);
});

it('can change scatter aggregations', function(done) {
Plotly.newPlot(gd, aggregatedScatter(1))
.then(checkCalcData(aggScatter1CD))

.then(reactWith(aggregatedScatter(2)))
.then(checkCalcData(aggScatter2CD))

.then(reactWith(aggregatedScatter(1)))
.then(checkCalcData(aggScatter1CD))
.catch(failTest)
.then(done);
});

it('can change parcoords aggregations', function(done) {
Plotly.newPlot(gd, aggregatedParcoords(0))
.then(checkValues(aggParcoords0Vals))

.then(reactWith(aggregatedParcoords(1)))
.then(checkValues(aggParcoords1Vals))

.then(reactWith(aggregatedParcoords(2)))
.then(checkValues(aggParcoords2Vals))

.then(reactWith(aggregatedParcoords(0)))
.then(checkValues(aggParcoords0Vals))

.catch(failTest)
.then(done);
});

it('can change type with aggregations', function(done) {
Plotly.newPlot(gd, aggregatedScatter(1))
.then(checkCalcData(aggScatter1CD))

.then(reactWith(aggregatedPie(1)))
.then(checkCalcData(aggPie1CD))

.then(reactWith(aggregatedParcoords(1)))
.then(checkValues(aggParcoords1Vals))

.then(reactWith(aggregatedScatter(1)))
.then(checkCalcData(aggScatter1CD))

.then(reactWith(aggregatedParcoords(2)))
.then(checkValues(aggParcoords2Vals))

.then(reactWith(aggregatedPie(2)))
.then(checkCalcData(aggPie2CD))

.then(reactWith(aggregatedScatter(2)))
.then(checkCalcData(aggScatter2CD))

.then(reactWith(aggregatedParcoords(0)))
.then(checkValues(aggParcoords0Vals))
.catch(failTest)
.then(done);
});

it('can change frames without redrawing', function(done) {
var data = [{y: [1, 2, 3]}];
var layout = {};
Expand Down