Skip to content

ENH: Validation to only allow positive integers for options #27382

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Aug 5, 2019
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
def4005
Created a new validation function (is_pos_int) to allow options to be
Adam-Klaum Jul 12, 2019
e50961d
Added positive and negative tests for is_pos_int in test_validation.
Adam-Klaum Jul 13, 2019
32e8c67
Updating whatsnew
Adam-Klaum Jul 13, 2019
9618877
Fixing order as per isort
Adam-Klaum Jul 13, 2019
7a5593b
Simplified the is_pos_int function to be used standalone rather than
Adam-Klaum Jul 15, 2019
d6fa258
Updated is_pos_int docstring with Returns and Raises
Adam-Klaum Jul 16, 2019
eb09813
Clarifying documentation
Adam-Klaum Jul 18, 2019
8f46673
Merge branch 'master' into 23348_max_rows
Adam-Klaum Jul 24, 2019
21bf599
Removed Returns section from docstring. Set dash length to be the same
Adam-Klaum Jul 24, 2019
ea09cc2
Moved whatsnew documentation from v25 to v26
Adam-Klaum Jul 24, 2019
6c2ff43
Added documentation to Other section
Adam-Klaum Jul 24, 2019
25e0247
Merge remote-tracking branch 'origin/master' into 23348_max_rows
Adam-Klaum Jul 27, 2019
d18a01a
Moving comments to v1.0.0 file
Adam-Klaum Jul 27, 2019
4e76640
Added double ticks around display parameters in whatsnew.
Adam-Klaum Jul 29, 2019
80d2f1a
Merge remote-tracking branch 'origin/master' into 23348_max_rows
Adam-Klaum Jul 29, 2019
2d783ef
Corrected is_pos_int ValueError message in test suite
Adam-Klaum Jul 30, 2019
7163990
Merge remote-tracking branch 'upstream/master' into 23348_max_rows
Adam-Klaum Jul 30, 2019
033339f
Refactored is_pos_int to is_nonnegative_int and updated whatsnew.
Adam-Klaum Aug 4, 2019
42d844e
Merge remote-tracking branch 'upstream/master' into 23348_max_rows
Adam-Klaum Aug 4, 2019
1a25d06
Fixing import sorting issue
Adam-Klaum Aug 5, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
201 changes: 201 additions & 0 deletions doc/source/whatsnew/v0.26.0.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
.. _whatsnew_0260:

What's new in 0.26.0 (??)
------------------------

.. warning::

Starting with the 0.25.x series of releases, pandas only supports Python 3.5.3 and higher.
See :ref:`install.dropping-27` for more details.

.. warning::

The minimum supported Python version will be bumped to 3.6 in a future release.

{{ header }}

These are the changes in pandas 0.26.0. See :ref:`release` for a full changelog
including other versions of pandas.


Enhancements
~~~~~~~~~~~~

.. _whatsnew_0260.enhancements.other:

-
-

Other enhancements
^^^^^^^^^^^^^^^^^^

.. _whatsnew_0260.api_breaking:

-
-

Backwards incompatible API changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. _whatsnew_0260.api.other:

-
-

Other API changes
^^^^^^^^^^^^^^^^^

-
-

.. _whatsnew_0260.deprecations:

Deprecations
~~~~~~~~~~~~

-
-

.. _whatsnew_0260.prior_deprecations:

Removal of prior version deprecations/changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

-
-

.. _whatsnew_0260.performance:

Performance improvements
~~~~~~~~~~~~~~~~~~~~~~~~

-
-

.. _whatsnew_0260.bug_fixes:

Bug fixes
~~~~~~~~~


Categorical
^^^^^^^^^^^

-
-


Datetimelike
^^^^^^^^^^^^

-
-


Timedelta
^^^^^^^^^

-
-

Timezones
^^^^^^^^^

-
-


Numeric
^^^^^^^

-
-

Conversion
^^^^^^^^^^

-
-

Strings
^^^^^^^

-
-


Interval
^^^^^^^^

-
-

Indexing
^^^^^^^^

-
-

Missing
^^^^^^^

-
-

MultiIndex
^^^^^^^^^^

-
-

I/O
^^^

-
-

Plotting
^^^^^^^^

-
-

Groupby/resample/rolling
^^^^^^^^^^^^^^^^^^^^^^^^

-
-

Reshaping
^^^^^^^^^

-
-

Sparse
^^^^^^

-
-


Build Changes
^^^^^^^^^^^^^


ExtensionArray
^^^^^^^^^^^^^^

-
-


Other
^^^^^
- Trying to set the display.precision, display.max_rows or display.max_columns using set_option to anything but a None or a positive int will raise a ``ValueError`` (:issue:`23348`)


.. _whatsnew_0260.contributors:

Contributors
~~~~~~~~~~~~
26 changes: 26 additions & 0 deletions pandas/_config/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -787,6 +787,7 @@ def is_instance_factory(_type):
ValueError if x is not an instance of `_type`

"""

if isinstance(_type, (tuple, list)):
_type = tuple(_type)
type_repr = "|".join(map(str, _type))
Expand Down Expand Up @@ -820,6 +821,31 @@ def inner(x):
return inner


def is_pos_int(value):
"""
Verifies that value is None or a positive int

Parameters
----------
value - the value to be checked
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jreback comment for returns section is lost in diff so re-commenting here but I'm surprised this isn't failing the CI either

@datapythonista do we ignore private docstrings in the validation? Vaguely recall some comment around that but didn't see code to confirm

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, shall I add the following?

Returns
----------
Nothing 

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WillAyd yes, we don't validate docstrings of private functions. Ideally we should validate everything, but with all the pending work in the public API that has an impact in the web docs, I don't think we should spend time in fixing the private docstrings yet (and if we don't fix the existing cases, we can't easily validate the new ones).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification - do you know where that exclusion happens? Didn't see it in code_checks and couldn't find on quick glance in validate_docstrings.py

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ups sorry, was going to mention and I forgot. It's not that we exclude them, it's that we get the docstrings to consider from the .rst files of the API docs. The code here is a good starting point to understand that: https://github.com/pandas-dev/pandas/blob/master/scripts/validate_docstrings.py#L873

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Adam-Klaum these need to be updated to follow the doctoring stander. See https://dev.pandas.io/development/contributing.html#updating-a-pandas-docstring


Raises
------
ValueError if value is not equal to None or int, or if value is not
positive in the case of a int type.

"""

if not (value is None or isinstance(value, int)):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use isistance(values, numbers.Integral)

Can you restructure this a bit?

if value is None:
    return value

elif isinstance(value, numbers.Integral):
    if value < 0:
         raise
    return value

raise ValueError(...)

msg = "Value must be an instance of <class 'NoneType'>|<class 'int'>"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think most people won't be familiar with NoneType. Should probably say like 'value' must be a positive integer or None.

raise ValueError(msg)

elif isinstance(value, int):
if value < 0:
msg = "int values must be positive for this option"
raise ValueError(msg)


# common type validators, for convenience
# usage: register_option(... , validator = is_int)
is_int = is_type_factory(int)
Expand Down
17 changes: 4 additions & 13 deletions pandas/core/config_init.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
is_instance_factory,
is_int,
is_one_of_factory,
is_pos_int,
is_text,
)

Expand Down Expand Up @@ -319,7 +320,7 @@ def is_terminal():


with cf.config_prefix("display"):
cf.register_option("precision", 6, pc_precision_doc, validator=is_int)
cf.register_option("precision", 6, pc_precision_doc, validator=is_pos_int)
cf.register_option(
"float_format",
None,
Expand All @@ -333,12 +334,7 @@ def is_terminal():
pc_max_info_rows_doc,
validator=is_instance_factory((int, type(None))),
)
cf.register_option(
"max_rows",
60,
pc_max_rows_doc,
validator=is_instance_factory([type(None), int]),
)
cf.register_option("max_rows", 60, pc_max_rows_doc, validator=is_pos_int)
cf.register_option(
"min_rows",
10,
Expand All @@ -351,12 +347,7 @@ def is_terminal():
max_cols = 0 # automatically determine optimal number of columns
else:
max_cols = 20 # cannot determine optimal number of columns
cf.register_option(
"max_columns",
max_cols,
pc_max_cols_doc,
validator=is_instance_factory([type(None), int]),
)
cf.register_option("max_columns", max_cols, pc_max_cols_doc, validator=is_pos_int)
cf.register_option(
"large_repr",
"truncate",
Expand Down
13 changes: 13 additions & 0 deletions pandas/tests/config/test_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -208,20 +208,33 @@ def test_set_option_multiple(self):

def test_validation(self):
self.cf.register_option("a", 1, "doc", validator=self.cf.is_int)
self.cf.register_option("d", 1, "doc", validator=self.cf.is_pos_int)
self.cf.register_option("b.c", "hullo", "doc2", validator=self.cf.is_text)

msg = "Value must have type '<class 'int'>'"
with pytest.raises(ValueError, match=msg):
self.cf.register_option("a.b.c.d2", "NO", "doc", validator=self.cf.is_int)

self.cf.set_option("a", 2) # int is_int
self.cf.set_option("b.c", "wurld") # str is_str
self.cf.set_option("d", 2)

# None not is_int
with pytest.raises(ValueError, match=msg):
self.cf.set_option("a", None)
with pytest.raises(ValueError, match=msg):
self.cf.set_option("a", "ab")

msg = r"Value must be an instance of <class 'NoneType'>\|<class 'int'>"
with pytest.raises(ValueError, match=msg):
self.cf.register_option(
"a.b.c.d3", "NO", "doc", validator=self.cf.is_pos_int
)

msg = "int values must be positive for this option"
with pytest.raises(ValueError, match=msg):
self.cf.register_option("a.b.c.d3", -2, "doc", validator=self.cf.is_pos_int)

msg = r"Value must be an instance of <class 'str'>\|<class 'bytes'>"
with pytest.raises(ValueError, match=msg):
self.cf.set_option("b.c", 1)
Expand Down