Skip to content

DOC: Updating capitalization of doc/source/development #33121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
Apr 3, 2020
Merged

DOC: Updating capitalization of doc/source/development #33121

Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
194d5a7
Updating capitalization in folder doc/source/reference
cleconte987 Mar 19, 2020
3d1960c
Merge remote-tracking branch 'upstream_main_pandas/master' into chang…
cleconte987 Mar 20, 2020
9da1e8a
Add folder doc/source/reference in ci code checking in ci/code_check…
cleconte987 Mar 20, 2020
55587f7
Modification of validate_rst_title_capitalization.py script
cleconte987 Mar 20, 2020
048fb78
Modify validate_rst_title_capitalization.py script to take into accou…
cleconte987 Mar 20, 2020
d1ae9ea
remove stylistic errors in comment section L:77,78,79
cleconte987 Mar 20, 2020
83280b7
Modify validate_rst_title_capitalization.py script
cleconte987 Mar 22, 2020
4314d3c
Merge remote-tracking branch 'upstream_main_pandas/master' into scrip…
cleconte987 Mar 22, 2020
937b5df
modify script following tests
cleconte987 Mar 22, 2020
589b0fe
Modify second time following style checking
cleconte987 Mar 22, 2020
f09689c
Update branch with pandas repository commits
cleconte987 Mar 23, 2020
e71cffd
Commit to update capitalization in folder doc/source/reference and to…
cleconte987 Mar 23, 2020
dc839c5
Merge branches to update PR
cleconte987 Mar 23, 2020
f354797
Merge remote-tracking branch 'upstream_main_pandas/master' into scrip…
cleconte987 Mar 23, 2020
62d5164
Merge remote-tracking branch 'temporary_repo/changes_to_pandas_remote…
cleconte987 Mar 23, 2020
f8b496c
Remove wrong lines of code
cleconte987 Mar 23, 2020
5d01189
Formatting
cleconte987 Mar 23, 2020
1685076
Update branch
cleconte987 Mar 24, 2020
06dc1cd
Remove exception "GroupBy" and lower "GroupBy" in doc
cleconte987 Mar 24, 2020
a11dc2b
modifying for PR
cleconte987 Mar 25, 2020
e55064d
Merge remote-tracking branch 'upstream_main_pandas/master' into chang…
cleconte987 Mar 25, 2020
fc1009d
Update "groupby" to "GroupBy"
cleconte987 Mar 25, 2020
eba0d13
Merge remote-tracking branch 'upstream_main_pandas/master' into chang…
cleconte987 Mar 27, 2020
726bd4c
Merge remote-tracking branch 'upstream_main_pandas/master' into chang…
cleconte987 Mar 29, 2020
0beeda1
Commit changes to folder doc/source/development
cleconte987 Mar 29, 2020
f48b2dd
Add keywords to CAPITALIZATION_EXCEPTIONS
cleconte987 Mar 29, 2020
5ba464c
Merge remote-tracking branch 'upstream_main_pandas/master' into chang…
cleconte987 Apr 1, 2020
7c09610
Commit and updating documentation syntax and capitalization for folde…
cleconte987 Apr 1, 2020
940109d
Remove the 'ExtensionDtype' and 'ExtensionArray' exceptions from 'CAP…
cleconte987 Apr 1, 2020
35b1a37
Style formatting
cleconte987 Apr 1, 2020
5f0512e
Correct error by removing wrong lines of code in validate_rst_title_c…
cleconte987 Apr 1, 2020
7767d3c
Weird behaviour file 'pandas/tests/frame/indexing/test_setitem.py' no…
cleconte987 Apr 2, 2020
888aa10
merging remote pandas-dev/pandas/master
cleconte987 Apr 2, 2020
5fc4dcc
Commit modifying script 'validate_rst_title_capitalization.py' to ch…
cleconte987 Apr 2, 2020
0ad8143
Correct style errors
cleconte987 Apr 2, 2020
3ea43a5
Merge remote-tracking branch 'upstream_main_pandas/master' into chang…
cleconte987 Apr 2, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/code_checks.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,7 +337,7 @@ if [[ -z "$CHECK" || "$CHECK" == "docstrings" ]]; then
RET=$(($RET + $?)) ; echo $MSG "DONE"

MSG='Validate correct capitalization among titles in documentation' ; echo $MSG
$BASE_DIR/scripts/validate_rst_title_capitalization.py $BASE_DIR/doc/source/development/contributing.rst
$BASE_DIR/scripts/validate_rst_title_capitalization.py $BASE_DIR/doc/source/development/contributing.rst $BASE_DIR/doc/source/reference
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If everything in development/ is fixed, can you change this in a new PR, so the whole directory is validated in the CI? Thanks!

Suggested change
$BASE_DIR/scripts/validate_rst_title_capitalization.py $BASE_DIR/doc/source/development/contributing.rst $BASE_DIR/doc/source/reference
$BASE_DIR/scripts/validate_rst_title_capitalization.py $BASE_DIR/doc/source/development $BASE_DIR/doc/source/reference

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure!

RET=$(($RET + $?)) ; echo $MSG "DONE"

fi
Expand Down
8 changes: 4 additions & 4 deletions doc/source/development/code_style.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ consistent code format throughout the project. For details see the
Patterns
========

foo.__class__
-------------
Using foo.__class__
-------------------


pandas uses 'type(foo)' instead 'foo.__class__' as it is making the code more
Expand Down Expand Up @@ -47,8 +47,8 @@ String formatting
Concatenated strings
--------------------

f-strings
~~~~~~~~~
Using f-strings
~~~~~~~~~~~~~~~

pandas uses f-strings formatting instead of '%' and '.format()' string formatters.

Expand Down
2 changes: 1 addition & 1 deletion doc/source/development/extending.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ and comments contain guidance for properly implementing the interface.

.. _extending.extension.operator:

:class:`~pandas.api.extensions.ExtensionArray` Operator Support
:class:`~pandas.api.extensions.ExtensionArray` operator support
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. versionadded:: 0.24.0
Expand Down
4 changes: 2 additions & 2 deletions doc/source/development/policies.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Policies

.. _policies.version:

Version Policy
Version policy
~~~~~~~~~~~~~~

.. versionchanged:: 1.0.0
Expand Down Expand Up @@ -48,7 +48,7 @@ deprecation removed in the next next major release (2.0.0).
These policies do not apply to features marked as **experimental** in the documentation.
pandas may change the behavior of experimental features at any time.

Python Support
Python support
~~~~~~~~~~~~~~

pandas will only drop support for specific Python versions (e.g. 3.6.x, 3.7.x) in
Expand Down
2 changes: 1 addition & 1 deletion doc/source/development/roadmap.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ We'd like to fund improvements and maintenance of these tools to

.. _roadmap.evolution:

Roadmap Evolution
Roadmap evolution
-----------------

pandas continues to evolve. The direction is primarily determined by community
Expand Down
2 changes: 1 addition & 1 deletion doc/source/reference/frame.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ Combining / joining / merging
DataFrame.merge
DataFrame.update

Time series-related
Time Series-related
~~~~~~~~~~~~~~~~~~~
.. autosummary::
:toctree: api/
Expand Down
2 changes: 1 addition & 1 deletion doc/source/reference/indexing.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,7 @@ DatetimeIndex

DatetimeIndex

Time/Date components
Time/date components
~~~~~~~~~~~~~~~~~~~~
.. autosummary::
:toctree: api/
Expand Down
4 changes: 2 additions & 2 deletions doc/source/reference/series.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ Binary operator functions
Series.product
Series.dot

Function application, groupby & window
Function application, GroupBy & window
--------------------------------------
.. autosummary::
:toctree: api/
Expand Down Expand Up @@ -249,7 +249,7 @@ Combining / joining / merging
Series.replace
Series.update

Time series-related
Time Series-related
-------------------
.. autosummary::
:toctree: api/
Expand Down
65 changes: 61 additions & 4 deletions scripts/validate_rst_title_capitalization.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@

"""
import argparse
import sys
import re
import os
from typing import Tuple, Generator, List
import glob
import os
import re
import sys
from typing import Generator, List, Tuple


CAPITALIZATION_EXCEPTIONS = {
Expand Down Expand Up @@ -46,6 +46,60 @@
"NumFOCUS",
"sklearn",
"Docker",
"PeriodIndex",
"NA",
"NaN",
"ValueError",
"BooleanArray",
"KeyError",
"API",
"FAQ",
"IO",
"TimedeltaIndex",
"DatetimeIndex",
"IntervalIndex",
"CategoricalIndex",
"GroupBy",
"SPSS",
"ORC",
"R",
"HDF5",
"HDFStore",
"CDay",
"CBMonthBegin",
"CBMonthEnd",
"BMonthBegin",
"BMonthEnd",
"BDay",
"FY5253Quarter",
"FY5253",
"YearBegin",
"YearEnd",
"BYearBegin",
"BYearEnd",
"YearOffset",
"QuarterBegin",
"QuarterEnd",
"BQuarterBegin",
"BQuarterEnd",
"QuarterOffset",
"LastWeekOfMonth",
"WeekOfMonth",
"SemiMonthBegin",
"SemiMonthEnd",
"SemiMonthOffset",
"CustomBusinessMonthBegin",
"CustomBusinessMonthEnd",
"BusinessMonthBegin",
"BusinessMonthEnd",
"MonthBegin",
"MonthEnd",
"MonthOffset",
"CustomBusinessHour",
"CustomBusinessDay",
"BusinessHour",
"BusinessDay",
"DateOffset",
}

CAP_EXCEPTIONS_DICT = {word.lower(): word for word in CAPITALIZATION_EXCEPTIONS}
Expand Down Expand Up @@ -121,6 +175,7 @@ def find_titles(rst_file: str) -> Generator[Tuple[str, int], None, None]:
len(line_chars) == 1
and line_chars.pop() in symbols
and len(line) == len(previous_line)
and previous_line[0] != ":"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I think I suggested this approach myself, but I see a problem now. This will skip the whole line from validation, and I see we use a mix of these :class: directives and regular text in some titles.

Instead of skipping the line here, I think we should update the condition in line 139. So, we check that word is not one of the exceptions, and also, it doesn't start with :. We could use a regex probably, but let's start simple, and see if it's enough.

What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that:

  • First of all this part of code line 180

    yield re.sub(r"[`*_]", "", previous_line), i

    removes "`" from the title. So when function checks if title is correct, it will already be modified. So this is the main problem.

  • Secondly, line 129:

    correct_title: str = re.sub(r"^\W*", "", title).capitalize()

    removes all non word characters at the beginning from the desired correct title that we want to keep, so ":" is not kept as being part of the correct title

  • Thirdly, this line, number 136

    word_list = re.split(r"\W", removed_https_title)

    removes all non word characters from the list to analyse. So ":" don't appear anymore.

By the way, code in line 180 causes problems because it also removes "_" which is present in documentation like > code_style.rst line 21

Using foo.__class__

So what I suggest is to modify in script "validate_rst_title_capitalization.py" from line 180 on:

NB: I replace "`" by {backtick} as it causes trouble to display it as code

yield re.sub(r"[{backtick}\*_]", "", previous_line), i

to:

if previous_line[0] != ":": yield re.sub(r"[{backtick}\*_]", "", previous_line), i else: yield re.sub(r"[\*_]", "", previous_line), i

in order to keep the "`" if title begins by a ":"
And to add in "validate_rst_title_capitalization.py" line 126:

if title[0] == ":": return title

So that if title begins by ":" it considers title to be valid no matter what.

Or to simply add in "validate_rst_title_capitalization.py" line 126:

if title[0] == ":": return title

for the same reason.

Note that there is still a problem with:

yield re.sub(r"[`*_]", "", previous_line), i

That doesn't keep "_" in the titles

):
yield re.sub(r"[`\*_]", "", previous_line), i
previous_line = line
Expand Down Expand Up @@ -177,6 +232,8 @@ def main(source_paths: List[str], output_format: str) -> bool:

for filename in find_rst_files(source_paths):
for title, line_number in find_titles(filename):
if ":class" in title:
print(title)
Copy link
Member

@ShaharNaveh ShaharNaveh Apr 1, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be:

if ":class:" in title:
   continue

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Horrible... I forgot to remove it. Correcting it right away!

Copy link
Contributor Author

@cleconte987 cleconte987 Apr 1, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm Im not so much used to git yet.. Apart from that mistake, I didn't roll back previous changes that I made, I thought it will be ok. I don't know if the changes i didn't remove will fail

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or if it conflicts with previous PR or something

if title != correct_title_capitalization(title):
print(
f"""{filename}:{line_number}:{err_msg} "{title}" to "{
Expand Down