Skip to content

Implement strict autotypenumbers #5240

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Nov 4, 2020
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
cbac3ae
issue 5195 - implement convertnumeric for axes and inherit from layout
archmoj Oct 29, 2020
c4224f4
adjust autoType return when convertNumeric is disabled - add tests
archmoj Oct 30, 2020
3b1dd0f
fixups and add polar tests
archmoj Oct 30, 2020
48fabdd
add gl3d tests
archmoj Oct 30, 2020
5aca807
add box plot tests an improve gl3d tests
archmoj Oct 30, 2020
84907dc
add tests for carpet
archmoj Oct 30, 2020
04da8a1
Update src/plots/cartesian/layout_attributes.js
archmoj Nov 2, 2020
01115cd
Update src/plots/layout_attributes.js
archmoj Nov 2, 2020
fb67c82
revise new attribute names and val types
archmoj Nov 3, 2020
f1c0e31
add early return for empty arrays to test category case and a bit of …
archmoj Nov 4, 2020
837761c
maintain detecting dates using strict mode
archmoj Nov 4, 2020
b73d395
pass convertNumeric to linearOK and category
archmoj Nov 4, 2020
0224fa0
store type of
archmoj Nov 4, 2020
d218bab
add early return for dates - remove unnecessary early return in linea…
archmoj Nov 4, 2020
b800a65
add early returns - array of arrays is not relevant to date, category…
archmoj Nov 4, 2020
f6d8c39
flat input array before passing to autoType functions
archmoj Nov 4, 2020
560405c
revert description change
archmoj Nov 4, 2020
4697f9c
flat 2d arrays before passing to autotypes
archmoj Nov 4, 2020
688438c
handle typed arrays inside a 2d array - plus a bit of optimization
archmoj Nov 4, 2020
e02deaf
add tests for autotype - case of 2d arrays
archmoj Nov 4, 2020
4b5bfcc
correct keys in description
archmoj Nov 4, 2020
08aa300
drop unused option in ternary
archmoj Nov 4, 2020
1198e56
add noMultiCategory option for polar
archmoj Nov 4, 2020
99a2015
apply layout.calnedar and layout.autotypenumbers in box defaults
archmoj Nov 4, 2020
6fdf559
fixup box plot test to have a layout object
archmoj Nov 4, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/plots/cartesian/axes.js
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,9 @@ var getDataConversions = axes.getDataConversions = function(gd, trace, target, t
// setup the data-to-calc method.
if(Array.isArray(d2cTarget)) {
ax = {
type: autoType(targetArray),
type: autoType(targetArray, undefined, {
convertNumeric: gd._fullLayout.axesconvertnumeric
}),
_categories: []
};
axes.setConvert(ax);
Expand Down
16 changes: 10 additions & 6 deletions src/plots/cartesian/axis_autotype.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,22 +15,26 @@ var Lib = require('../../lib');
var BADNUM = require('../../constants/numerical').BADNUM;

module.exports = function autoType(array, calendar, opts) {
opts = opts || {};
var convertNumeric = opts.convertNumeric;

if(!opts.noMultiCategory && multiCategory(array)) return 'multicategory';
if(moreDates(array, calendar)) return 'date';
if(convertNumeric && moreDates(array, calendar)) return 'date';
if(category(array)) return 'category';
if(linearOK(array)) return 'linear';
else return '-';
if(linearOK(array, convertNumeric)) return 'linear';
else return convertNumeric ? '-' : 'category';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to leave else return '-', for empty arrays and arrays with only null entries, and instead include convertNumeric in the category(array) call, ie replace:

else if(Lib.cleanNumber(ai) !== BADNUM) curvenums++;

with something like:

else if(convertNumeric ? Lib.cleanNumber(ai) !== BADNUM : typeof ai === 'number') curvenums++;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Done in b73d395.

};

function hasTypeNumber(v, convertNumeric) {
return convertNumeric ? isNumeric(v) : typeof v === 'number';
}

// is there at least one number in array? If not, we should leave
// ax.type empty so it can be autoset later
function linearOK(array) {
function linearOK(array, convertNumeric) {
if(!array) return false;

for(var i = 0; i < array.length; i++) {
if(isNumeric(array[i])) return true;
if(hasTypeNumber(array[i], convertNumeric)) return true;
}

return false;
Expand Down
12 changes: 11 additions & 1 deletion src/plots/cartesian/layout_attributes.js
Original file line number Diff line number Diff line change
Expand Up @@ -97,11 +97,21 @@ module.exports = {
_noTemplating: true,
description: [
'Sets the axis type.',
'By default, plotly attempts to determined the axis type',
'By default, plotly.js attempts to determined the axis type',
'by looking into the data of the traces that referenced',
'the axis in question.'
].join(' ')
},
convertnumeric: {
valType: 'boolean',
role: 'info',
editType: 'calc',
description: [
'Determines whether or not a numeric string in this axis may be',
'treated as a number during automatic axis `type` detection.',
'Defaults to layout.axesconvertnumeric.'
].join(' ')
},
autorange: {
valType: 'enumerated',
values: [true, false, 'reversed'],
Expand Down
4 changes: 4 additions & 0 deletions src/plots/cartesian/layout_defaults.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ function appendList(cont, k, item) {
}

module.exports = function supplyLayoutDefaults(layoutIn, layoutOut, fullData) {
var convertnumericDflt = layoutOut.axesconvertnumeric;

var ax2traces = {};
var xaMayHide = {};
var yaMayHide = {};
Expand Down Expand Up @@ -246,6 +248,7 @@ module.exports = function supplyLayoutDefaults(layoutIn, layoutOut, fullData) {
automargin: true,
visibleDflt: visibleDflt,
reverseDflt: reverseDflt,
convertnumericDflt: convertnumericDflt,
splomStash: ((layoutOut._splomAxes || {})[axLetter] || {})[axId]
};

Expand Down Expand Up @@ -310,6 +313,7 @@ module.exports = function supplyLayoutDefaults(layoutIn, layoutOut, fullData) {
automargin: true,
visibleDflt: false,
reverseDflt: false,
convertnumericDflt: convertnumericDflt,
splomStash: ((layoutOut._splomAxes || {})[axLetter] || {})[axId]
};

Expand Down
3 changes: 3 additions & 0 deletions src/plots/cartesian/type_defaults.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ var autoType = require('./axis_autotype');
* name: axis object name (ie 'xaxis') if one should be stored
*/
module.exports = function handleTypeDefaults(containerIn, containerOut, coerce, options) {
coerce('convertnumeric', options.convertnumericDflt);
var axType = coerce('type', (options.splomStash || {}).type);

if(axType === '-') {
Expand Down Expand Up @@ -68,6 +69,8 @@ function setAutoType(ax, data) {
opts.noMultiCategory = true;
}

opts.convertNumeric = ax.convertnumeric;

// check all boxes on this x axis to see
// if they're dates, numbers, or categories
if(isBoxWithoutPositionCoords(d0, axLetter)) {
Expand Down
1 change: 1 addition & 0 deletions src/plots/gl3d/layout/axis_attributes.js
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ module.exports = overrideAll({
type: extendFlat({}, axesAttrs.type, {
values: ['-', 'linear', 'log', 'date', 'category']
}),
convertnumeric: axesAttrs.convertnumeric,
autorange: axesAttrs.autorange,
rangemode: axesAttrs.rangemode,
range: extendFlat({}, axesAttrs.range, {
Expand Down
2 changes: 2 additions & 0 deletions src/plots/gl3d/layout/defaults.js
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ module.exports = function supplyLayoutDefaults(layoutIn, layoutOut, fullData) {
font: layoutOut.font,
fullData: fullData,
getDfltFromLayout: getDfltFromLayout,
convertnumericDflt: layoutOut.axesconvertnumeric,
paper_bgcolor: layoutOut.paper_bgcolor,
calendar: layoutOut.calendar
});
Expand Down Expand Up @@ -109,6 +110,7 @@ function handleGl3dDefaults(sceneLayoutIn, sceneLayoutOut, coerce, opts) {
data: fullGl3dData,
bgColor: bgColorCombined,
calendar: opts.calendar,
convertnumericDflt: opts.convertnumericDflt,
fullLayout: opts.fullLayout
});

Expand Down
10 changes: 10 additions & 0 deletions src/plots/layout_attributes.js
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,16 @@ module.exports = {
'Sets the background color of the plotting area in-between x and y axes.'
].join(' ')
},
axesconvertnumeric: {
valType: 'boolean',
dflt: true,
role: 'info',
editType: 'calc',
description: [
'Determines whether or not a numeric string in axes of the plot may be',
'treated as a number during automatic axis `type` detection.'
].join(' ')
},
separators: {
valType: 'string',
role: 'style',
Expand Down
2 changes: 2 additions & 0 deletions src/plots/plots.js
Original file line number Diff line number Diff line change
Expand Up @@ -1476,6 +1476,8 @@ plots.supplyLayoutGlobalDefaults = function(layoutIn, layoutOut, formatObj) {
layoutOut._dataTemplate = template.data;
}

coerce('axesconvertnumeric');

var globalFont = Lib.coerceFont(coerce, 'font');

coerce('title.text', layoutOut._dfltTitle.plot);
Expand Down
2 changes: 2 additions & 0 deletions src/plots/polar/layout_attributes.js
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ var radialAxisAttrs = {
type: extendFlat({}, axesAttrs.type, {
values: ['-', 'linear', 'log', 'date', 'category']
}),
convertnumeric: axesAttrs.convertnumeric,

autorange: extendFlat({}, axesAttrs.autorange, {editType: 'plot'}),
rangemode: {
Expand Down Expand Up @@ -179,6 +180,7 @@ var angularAxisAttrs = {
'If *category, use `period` to set the number of integer coordinates around polar axis.'
].join(' ')
},
convertnumeric: axesAttrs.convertnumeric,

categoryorder: axesAttrs.categoryorder,
categoryarray: axesAttrs.categoryarray,
Expand Down
10 changes: 7 additions & 3 deletions src/plots/polar/layout_defaults.js
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ function handleDefaults(contIn, contOut, coerce, opts) {
axOut._traceIndices = subplotData.map(function(t) { return t._expandedIndex; });

var dataAttr = constants.axisName2dataArray[axName];
var axType = handleAxisTypeDefaults(axIn, axOut, coerceAxis, subplotData, dataAttr);
var axType = handleAxisTypeDefaults(axIn, axOut, coerceAxis, subplotData, dataAttr, opts);

handleCategoryOrderDefaults(axIn, axOut, coerceAxis, {
axData: subplotData,
Expand Down Expand Up @@ -187,7 +187,8 @@ function handleDefaults(contIn, contOut, coerce, opts) {
}
}

function handleAxisTypeDefaults(axIn, axOut, coerce, subplotData, dataAttr) {
function handleAxisTypeDefaults(axIn, axOut, coerce, subplotData, dataAttr, options) {
var convertnumeric = coerce('convertnumeric', options.convertnumericDflt);
var axType = coerce('type');

if(axType === '-') {
Expand All @@ -201,7 +202,9 @@ function handleAxisTypeDefaults(axIn, axOut, coerce, subplotData, dataAttr) {
}

if(trace && trace[dataAttr]) {
axOut.type = autoType(trace[dataAttr], 'gregorian');
axOut.type = autoType(trace[dataAttr], 'gregorian', {
convertNumeric: convertnumeric
});
}

if(axOut.type === '-') {
Expand All @@ -224,6 +227,7 @@ module.exports = function supplyLayoutDefaults(layoutIn, layoutOut, fullData) {
attributes: layoutAttributes,
handleDefaults: handleDefaults,
font: layoutOut.font,
convertnumericDflt: layoutOut.axesconvertnumeric,
paper_bgcolor: layoutOut.paper_bgcolor,
fullData: fullData,
layoutOut: layoutOut
Expand Down
1 change: 1 addition & 0 deletions src/plots/ternary/layout_defaults.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ module.exports = function supplyLayoutDefaults(layoutIn, layoutOut, fullData) {
attributes: layoutAttributes,
handleDefaults: handleTernaryDefaults,
font: layoutOut.font,
convertnumericDflt: layoutOut.axesconvertnumeric,
paper_bgcolor: layoutOut.paper_bgcolor
});
};
Expand Down
9 changes: 7 additions & 2 deletions src/traces/box/defaults.js
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,11 @@ function handleSampleDefaults(traceIn, traceOut, coerce, layout) {
var yLen = yDims && Lib.minRowLength(y);
var xLen = xDims && Lib.minRowLength(x);

var calendar = 'gregorian'; // TODO: should we use ax.calendar here?
var opts = {
convertNumeric: true // TODO: should we use ax.convertnumeric here?
};

var defaultOrientation, len;
if(traceOut._hasPreCompStats) {
switch(String(xDims) + String(yDims)) {
Expand Down Expand Up @@ -160,7 +165,7 @@ function handleSampleDefaults(traceIn, traceOut, coerce, layout) {
var hasCategories = false;
var i;
for(i = 0; i < x.length; i++) {
if(autoType(x[i]) === 'category') {
if(autoType(x[i], calendar, opts) === 'category') {
hasCategories = true;
break;
}
Expand All @@ -171,7 +176,7 @@ function handleSampleDefaults(traceIn, traceOut, coerce, layout) {
len = Math.min(sLen, xLen, y.length);
} else {
for(i = 0; i < y.length; i++) {
if(autoType(y[i]) === 'category') {
if(autoType(y[i], calendar, opts) === 'category') {
hasCategories = true;
break;
}
Expand Down
3 changes: 2 additions & 1 deletion src/traces/carpet/axis_attributes.js
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,12 @@ module.exports = {
editType: 'calc',
description: [
'Sets the axis type.',
'By default, plotly attempts to determined the axis type',
'By default, plotly.js attempts to determined the axis type',
'by looking into the data of the traces that referenced',
'the axis in question.'
].join(' ')
},
convertnumeric: axesAttrs.convertnumeric,
autorange: {
valType: 'enumerated',
values: [true, false, 'reversed'],
Expand Down
5 changes: 4 additions & 1 deletion src/traces/carpet/axis_defaults.js
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ module.exports = function handleAxisDefaults(containerIn, containerOut, options)
}

// now figure out type and do some more initialization
coerce('convertnumeric', options.convertnumericDflt);
var axType = coerce('type');
if(axType === '-') {
if(options.data) setAutoType(containerOut, options.data);
Expand Down Expand Up @@ -219,5 +220,7 @@ function setAutoType(ax, data) {
var calAttr = axLetter + 'calendar';
var calendar = ax[calAttr];

ax.type = autoType(data, calendar);
ax.type = autoType(data, calendar, {
convertNumeric: ax.convertnumeric
});
}
4 changes: 2 additions & 2 deletions src/traces/sunburst/attributes.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ module.exports = {
'Empty string items \'\' are understood to reference',
'the root node in the hierarchy.',
'If `ids` is filled, `parents` items are understood to be "ids" themselves.',
'When `ids` is not set, plotly attempts to find matching items in `labels`,',
'When `ids` is not set, plotly.js attempts to find matching items in `labels`,',
'but beware they must be unique.'
].join(' ')
},
Expand Down Expand Up @@ -83,7 +83,7 @@ module.exports = {
description: [
'Sets the level from which this trace hierarchy is rendered.',
'Set `level` to `\'\'` to start from the root node in the hierarchy.',
'Must be an "id" if `ids` is filled in, otherwise plotly attempts to find a matching',
'Must be an "id" if `ids` is filled in, otherwise plotly.js attempts to find a matching',
'item in `labels`.'
].join(' ')
},
Expand Down
45 changes: 45 additions & 0 deletions test/jasmine/tests/axes_test.js
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ describe('Test axes', function() {

beforeEach(function() {
layoutOut = {
axesconvertnumeric: true,
_has: Plots._hasPlotType,
_basePlotModules: [],
_dfltTitle: {x: 'x', y: 'y'},
Expand Down Expand Up @@ -335,6 +336,50 @@ describe('Test axes', function() {
});
});

describe('autotype disable/enable converting numeric strings', function() {
it('should disable converting numeric strings using axis.convertnumeric', function() {
layoutIn = {
xaxis: {},
yaxis: { convertnumeric: false }
};

supplyLayoutDefaults(layoutIn, layoutOut, [{
type: 'scatter',
xaxis: 'x',
yaxis: 'y',
x: ['0', '1', '1970', '2000'],
y: ['0', '1', '1970', '2000']
}]);

expect(layoutOut.xaxis.convertnumeric).toBe(true);
expect(layoutOut.yaxis.convertnumeric).toBe(false);
expect(layoutOut.xaxis.type).toBe('linear');
expect(layoutOut.yaxis.type).toBe('category');
});

it('should enable converting numeric strings using axis.convertnumeric and inherit defaults from layout.axesconvertnumeric', function() {
layoutOut.axesconvertnumeric = false;

layoutIn = {
xaxis: { convertnumeric: true },
yaxis: {}
};

supplyLayoutDefaults(layoutIn, layoutOut, [{
type: 'scatter',
xaxis: 'x',
yaxis: 'y',
x: ['0', '1', '1970', '2000'],
y: ['0', '1', '1970', '2000']
}]);

expect(layoutOut.xaxis.convertnumeric).toBe(true);
expect(layoutOut.yaxis.convertnumeric).toBe(false);
expect(layoutOut.xaxis.type).toBe('linear');
expect(layoutOut.yaxis.type).toBe('category');
});
});

it('should set undefined linewidth/linecolor if linewidth, linecolor or showline is not supplied', function() {
layoutIn = {
xaxis: {},
Expand Down
Loading