DOC/BLD: add custom sphinx autodoc AccessorAttributeDocumenter

Accessor attributes/methods like Series.str.match or Series.dt.hour
cannot be processed by the default autosummary/autodoc machinery.
So added:

- autosummary template
- autodoc custom Documenter that fixes the recognition of the
  correct module and object

Related to GH9322 (reimplement Series delegates/accessors using descriptors)
to be able to document the accessors as eg pandas.Series.str.match and not
pandas.core.strings.StringMethods.match

Author: jorisvandenbossche

doc/_templates/autosummary/accessor_attribute.rst

@@ -0,0 +1,6 @@
+{{ fullname }}
+{{ underline }}
+
+.. currentmodule:: {{ module.split('.')[0] }}
+
+.. autoaccessorattribute:: {{ [module.split('.')[1], objname]|join('.') }}

doc/_templates/autosummary/accessor_method.rst

@@ -0,0 +1,6 @@
+{{ fullname }}
+{{ underline }}
+
+.. currentmodule:: {{ module.split('.')[0] }}
+
+.. autoaccessormethod:: {{ [module.split('.')[1], objname]|join('.') }}

doc/source/api.rst

@@ -449,114 +449,113 @@ Datetimelike Properties
 
 ``Series.dt`` can be used to access the values of the series as
 datetimelike and return several properties.
-Due to implementation details the methods show up here as methods of the
-``DatetimeProperties/PeriodProperties/TimedeltaProperties`` classes. These can be accessed like ``Series.dt.<property>``.
-
-.. currentmodule:: pandas.tseries.common
+These can be accessed like ``Series.dt.<property>``.
 
 **Datetime Properties**
 
 .. autosummary::
    :toctree: generated/
-
-   DatetimeProperties.date
-   DatetimeProperties.time
-   DatetimeProperties.year
-   DatetimeProperties.month
-   DatetimeProperties.day
-   DatetimeProperties.hour
-   DatetimeProperties.minute
-   DatetimeProperties.second
-   DatetimeProperties.microsecond
-   DatetimeProperties.nanosecond
-   DatetimeProperties.second
-   DatetimeProperties.weekofyear
-   DatetimeProperties.dayofweek
-   DatetimeProperties.weekday
-   DatetimeProperties.dayofyear
-   DatetimeProperties.quarter
-   DatetimeProperties.is_month_start
-   DatetimeProperties.is_month_end
-   DatetimeProperties.is_quarter_start
-   DatetimeProperties.is_quarter_end
-   DatetimeProperties.is_year_start
-   DatetimeProperties.is_year_end
+   :template: autosummary/accessor_attribute.rst
+
+   Series.dt.date
+   Series.dt.time
+   Series.dt.year
+   Series.dt.month
+   Series.dt.day
+   Series.dt.hour
+   Series.dt.minute
+   Series.dt.second
+   Series.dt.microsecond
+   Series.dt.nanosecond
+   Series.dt.second
+   Series.dt.weekofyear
+   Series.dt.dayofweek
+   Series.dt.weekday
+   Series.dt.dayofyear
+   Series.dt.quarter
+   Series.dt.is_month_start
+   Series.dt.is_month_end
+   Series.dt.is_quarter_start
+   Series.dt.is_quarter_end
+   Series.dt.is_year_start
+   Series.dt.is_year_end
 
 **Datetime Methods**
 
 .. autosummary::
    :toctree: generated/
+   :template: autosummary/accessor_method.rst
 
-   DatetimeProperties.to_period
-   DatetimeProperties.to_pydatetime
-   DatetimeProperties.tz_localize
-   DatetimeProperties.tz_convert
+   Series.dt.to_period
+   Series.dt.to_pydatetime
+   Series.dt.tz_localize
+   Series.dt.tz_convert
 
 **Timedelta Properties**
 
 .. autosummary::
    :toctree: generated/
+   :template: autosummary/accessor_attribute.rst
 
-   TimedeltaProperties.days
-   TimedeltaProperties.seconds
-   TimedeltaProperties.microseconds
-   TimedeltaProperties.nanoseconds
-   TimedeltaProperties.components
+   Series.dt.days
+   Series.dt.seconds
+   Series.dt.microseconds
+   Series.dt.nanoseconds
+   Series.dt.components
 
 **Timedelta Methods**
 
 .. autosummary::
    :toctree: generated/
+   :template: autosummary/accessor_method.rst
 
-   TimedeltaProperties.to_pytimedelta
+   Series.dt.to_pytimedelta
 
 String handling
 ~~~~~~~~~~~~~~~
 ``Series.str`` can be used to access the values of the series as
-strings and apply several methods to it. Due to implementation
-details the methods show up here as methods of the
-``StringMethods`` class. These can be acccessed like ``Series.str.<function/property>``.
+strings and apply several methods to it. These can be acccessed like
+``Series.str.<function/property>``.
 
 .. currentmodule:: pandas.core.strings
 
 .. autosummary::
    :toctree: generated/
-
-   StringMethods.cat
-   StringMethods.center
-   StringMethods.contains
-   StringMethods.count
-   StringMethods.decode
-   StringMethods.encode
-   StringMethods.endswith
-   StringMethods.extract
-   StringMethods.findall
-   StringMethods.get
-   StringMethods.join
-   StringMethods.len
-   StringMethods.lower
-   StringMethods.lstrip
-   StringMethods.match
-   StringMethods.pad
-   StringMethods.repeat
-   StringMethods.replace
-   StringMethods.rstrip
-   StringMethods.slice
-   StringMethods.slice_replace
-   StringMethods.split
-   StringMethods.startswith
-   StringMethods.strip
-   StringMethods.title
-   StringMethods.upper
-   StringMethods.get_dummies
+   :template: autosummary/accessor_method.rst
+
+   Series.str.cat
+   Series.str.center
+   Series.str.contains
+   Series.str.count
+   Series.str.decode
+   Series.str.encode
+   Series.str.endswith
+   Series.str.extract
+   Series.str.findall
+   Series.str.get
+   Series.str.join
+   Series.str.len
+   Series.str.lower
+   Series.str.lstrip
+   Series.str.match
+   Series.str.pad
+   Series.str.repeat
+   Series.str.replace
+   Series.str.rstrip
+   Series.str.slice
+   Series.str.slice_replace
+   Series.str.split
+   Series.str.startswith
+   Series.str.strip
+   Series.str.title
+   Series.str.upper
+   Series.str.get_dummies
 
 .. _api.categorical:
 
 Categorical
 ~~~~~~~~~~~
 
-.. currentmodule:: pandas.core.categorical
-
 If the Series is of dtype ``category``, ``Series.cat`` can be used to change the the categorical
 data. This accessor is similar to the ``Series.dt`` or ``Series.str`` and has the
 following usable methods and properties (all available as ``Series.cat.<method_or_property>``).
@@ -595,7 +594,6 @@ the Categorical back to a numpy array, so levels and order information is not pr
 
 Plotting
 ~~~~~~~~
-.. currentmodule:: pandas
 
 .. autosummary::
    :toctree: generated/

doc/source/conf.py

@@ -297,6 +297,73 @@
     'pd.options.display.encoding="utf8"'
     ]
 
+
+# Add custom Documenter to handle attributes/methods of an AccessorProperty
+# eg pandas.Series.str and pandas.Series.dt (see GH9322)
+
+from sphinx.util import rpartition
+from sphinx.ext.autodoc import Documenter, MethodDocumenter, AttributeDocumenter
+
+
+class AccessorLevelDocumenter(Documenter):
+    """
+    Specialized Documenter subclass for objects on accessor level (methods,
+    attributes).
+    """
+
+    # This is the simple straightforward version
+    # modname is None, base the last elements (eg 'hour')
+    # and path the part before (eg 'Series.dt')
+    # def resolve_name(self, modname, parents, path, base):
+    #     modname = 'pandas'
+    #     mod_cls = path.rstrip('.')
+    #     mod_cls = mod_cls.split('.')
+    #
+    #     return modname, mod_cls + [base]
+
+    def resolve_name(self, modname, parents, path, base):
+        if modname is None:
+            if path:
+                mod_cls = path.rstrip('.')
+            else:
+                mod_cls = None
+                # if documenting a class-level object without path,
+                # there must be a current class, either from a parent
+                # auto directive ...
+                mod_cls = self.env.temp_data.get('autodoc:class')
+                # ... or from a class directive
+                if mod_cls is None:
+                    mod_cls = self.env.temp_data.get('py:class')
+                # ... if still None, there's no way to know
+                if mod_cls is None:
+                    return None, []
+            # HACK: this is added in comparison to ClassLevelDocumenter
+            # mod_cls still exists of class.accessor, so an extra
+            # rpartition is needed
+            modname, accessor = rpartition(mod_cls, '.')
+            modname, cls = rpartition(modname, '.')
+            parents = [cls, accessor]
+            # if the module name is still missing, get it like above
+            if not modname:
+                modname = self.env.temp_data.get('autodoc:module')
+            if not modname:
+                modname = self.env.temp_data.get('py:module')
+            # ... else, it stays None, which means invalid
+        return modname, parents + [base]
+
+
+class AccessorAttributeDocumenter(AccessorLevelDocumenter, AttributeDocumenter):
+
+    objtype = 'accessorattribute'
+    directivetype = 'attribute'
+
+
+class AccessorMethodDocumenter(AccessorLevelDocumenter, MethodDocumenter):
+
+    objtype = 'accessormethod'
+    directivetype = 'method'
+
+
 # remove the docstring of the flags attribute (inherited from numpy ndarray)
 # because these give doc build errors (see GH issue 5331)
 def remove_flags_docstring(app, what, name, obj, options, lines):
@@ -305,3 +372,5 @@ def remove_flags_docstring(app, what, name, obj, options, lines):
 
 def setup(app):
     app.connect("autodoc-process-docstring", remove_flags_docstring)
+    app.add_autodocumenter(AccessorAttributeDocumenter)
+    app.add_autodocumenter(AccessorMethodDocumenter)

doc/source/reshaping.rst

@@ -478,7 +478,7 @@ This function is often used along with discretization functions like ``cut``:
 
    get_dummies(cut(values, bins))
 
-See also :func:`Series.str.get_dummies <pandas.core.strings.StringMethods.get_dummies>`.
+See also :func:`Series.str.get_dummies <pandas.Series.str.get_dummies>`.
 
 .. versionadded:: 0.15.0
 

doc/source/text.rst

@@ -204,27 +204,27 @@ Method Summary
     :header: "Method", "Description"
     :widths: 20, 80
 
-    :meth:`~core.strings.StringMethods.cat`,Concatenate strings
-    :meth:`~core.strings.StringMethods.split`,Split strings on delimiter
-    :meth:`~core.strings.StringMethods.get`,Index into each element (retrieve i-th element)
-    :meth:`~core.strings.StringMethods.join`,Join strings in each element of the Series with passed separator
-    :meth:`~core.strings.StringMethods.contains`,Return boolean array if each string contains pattern/regex
-    :meth:`~core.strings.StringMethods.replace`,Replace occurrences of pattern/regex with some other string
-    :meth:`~core.strings.StringMethods.repeat`,Duplicate values (``s.str.repeat(3)`` equivalent to ``x * 3``)
-    :meth:`~core.strings.StringMethods.pad`,"Add whitespace to left, right, or both sides of strings"
-    :meth:`~core.strings.StringMethods.center`,Equivalent to ``pad(side='both')``
-    :meth:`~core.strings.StringMethods.wrap`,Split long strings into lines with length less than a given width
-    :meth:`~core.strings.StringMethods.slice`,Slice each string in the Series
-    :meth:`~core.strings.StringMethods.slice_replace`,Replace slice in each string with passed value
-    :meth:`~core.strings.StringMethods.count`,Count occurrences of pattern
-    :meth:`~core.strings.StringMethods.startswith`,Equivalent to ``str.startswith(pat)`` for each element
-    :meth:`~core.strings.StringMethods.endswith`,Equivalent to ``str.endswith(pat)`` for each element
-    :meth:`~core.strings.StringMethods.findall`,Compute list of all occurrences of pattern/regex for each string
-    :meth:`~core.strings.StringMethods.match`,"Call ``re.match`` on each element, returning matched groups as list"
-    :meth:`~core.strings.StringMethods.extract`,"Call ``re.match`` on each element, as ``match`` does, but return matched groups as strings for convenience."
-    :meth:`~core.strings.StringMethods.len`,Compute string lengths
-    :meth:`~core.strings.StringMethods.strip`,Equivalent to ``str.strip``
-    :meth:`~core.strings.StringMethods.rstrip`,Equivalent to ``str.rstrip``
-    :meth:`~core.strings.StringMethods.lstrip`,Equivalent to ``str.lstrip``
-    :meth:`~core.strings.StringMethods.lower`,Equivalent to ``str.lower``
-    :meth:`~core.strings.StringMethods.upper`,Equivalent to ``str.upper``
+    :meth:`~Series.str.cat`,Concatenate strings
+    :meth:`~Series.str.split`,Split strings on delimiter
+    :meth:`~Series.str.get`,Index into each element (retrieve i-th element)
+    :meth:`~Series.str.join`,Join strings in each element of the Series with passed separator
+    :meth:`~Series.str.contains`,Return boolean array if each string contains pattern/regex
+    :meth:`~Series.str.replace`,Replace occurrences of pattern/regex with some other string
+    :meth:`~Series.str.repeat`,Duplicate values (``s.str.repeat(3)`` equivalent to ``x * 3``)
+    :meth:`~Series.str.pad`,"Add whitespace to left, right, or both sides of strings"
+    :meth:`~Series.str.center`,Equivalent to ``pad(side='both')``
+    :meth:`~Series.str.wrap`,Split long strings into lines with length less than a given width
+    :meth:`~Series.str.slice`,Slice each string in the Series
+    :meth:`~Series.str.slice_replace`,Replace slice in each string with passed value
+    :meth:`~Series.str.count`,Count occurrences of pattern
+    :meth:`~Series.str.startswith`,Equivalent to ``str.startswith(pat)`` for each element
+    :meth:`~Series.str.endswith`,Equivalent to ``str.endswith(pat)`` for each element
+    :meth:`~Series.str.findall`,Compute list of all occurrences of pattern/regex for each string
+    :meth:`~Series.str.match`,"Call ``re.match`` on each element, returning matched groups as list"
+    :meth:`~Series.str.extract`,"Call ``re.match`` on each element, as ``match`` does, but return matched groups as strings for convenience."
+    :meth:`~Series.str.len`,Compute string lengths
+    :meth:`~Series.str.strip`,Equivalent to ``str.strip``
+    :meth:`~Series.str.rstrip`,Equivalent to ``str.rstrip``
+    :meth:`~Series.str.lstrip`,Equivalent to ``str.lstrip``
+    :meth:`~Series.str.lower`,Equivalent to ``str.lower``
+    :meth:`~Series.str.upper`,Equivalent to ``str.upper``