chore: add types (#225)

* chore: add mypy testing

Scaffolding in the testing, and will not complete succssfully yet.

Will be followed by code changes to pass.

Signed-off-by: Mike Fiedler <[email protected]>

* lint: ignore empty dict

Surfaced via `mypy`, in that an empty dict could not infer what types
could be there.

Instead of importing `typing` and annotating the empty Dict, I opted
to ignore the line, as we do not expect to populate the dict at all,
and are using it to **prevent** additions to the value.

Signed-off-by: Mike Fiedler <[email protected]>

* chore: remove unused styles parameter

Surfaced via `mypy`, recommended adding a type to the empty list.

The list was originally empty back in 0.1.0.

Instead of adding a type, remove the constant, and the code that uses it
from `clean()` - as it was partially reverted in #121.

The default `ALLOWED_STYLES` in the underlying library is `[]`.

Related: #114 (comment)

Signed-off-by: Mike Fiedler <[email protected]>

* fix: correct import for unescape

Surfaced via `mypy`, in that the `html.parser` module does not have
a direct implementation of `unescape`.
Refs: https://docs.python.org/3.6/library/html.html#html.unescape

In #192 support for Python 2.7 was removed, the import path changed.

This works due to imports placing the imported code in the local scope.
If the `html.parser` module ever stopped importing `unescape`,
this import would break as a result.

Signed-off-by: Mike Fiedler <[email protected]>

* chore: update pytest markers cli flag

Currently emits a warning:

    PytestRemovedIn8Warning: The --strict option is deprecated, use --strict-markers instead.

Signed-off-by: Mike Fiedler <[email protected]>

* chore(types): add types to clean module

Surfaced by running mypy in strict mode, and added types where relevant.

Signed-off-by: Mike Fiedler <[email protected]>

* chore(types): add types to txt module

Signed-off-by: Mike Fiedler <[email protected]>

* chore(types): add types to markdown module

Signed-off-by: Mike Fiedler <[email protected]>

* chore: add types to rst module

The types-docutils hints are still incomplete,
good progress is being made.
See: python/typeshed#7256

I've had to use an ignore on the class inheritance, and a couple of
`typing.Any` annotations until that package implements more type hints.

Signed-off-by: Mike Fiedler <[email protected]>

* chore: ignore distutils module from types

`mypy` strict mode is having a hard time with the `distutils`
imports, since they are wrapped in `setuptools` right now as a private
package. This pacakge's distutils integration will need to be reworked
anyhow. Left a comment with details at the top of the file.

Signed-off-by: Mike Fiedler <[email protected]>

* test: use strict mode for mypy

Prevent new things from creeping in during development.

Signed-off-by: Mike Fiedler <[email protected]>

* chore: tell the world we've got types

Include a blank `py.typed` file in the package to inform `mypy` that
there's types to be found in this package.

Signed-off-by: Mike Fiedler <[email protected]>

* chore: move strict flag to config

Allows other tools to ebenfit from a consistent configuration.

Signed-off-by: Mike Fiedler <[email protected]>

* refactor: change imports to be consistent with others

Signed-off-by: Mike Fiedler <[email protected]>

* docs: add inline details for specific ignores

Signed-off-by: Mike Fiedler <[email protected]>

* lint: apply more specific type

Signed-off-by: Mike Fiedler <[email protected]>

* docs: add comment to why source is Any

Signed-off-by: Mike Fiedler <[email protected]>

* lint: replace typing imports with relative ones

Signed-off-by: Mike Fiedler <[email protected]>

Author: miketheman

.github/workflows/ci.yml

@@ -32,6 +32,8 @@ jobs:
         run: python -m pip install tox
       - name: Run linting
         run: python -m tox -e pep8
+      - name: Run mypy
+        run: python -m tox -e mypy
   packaging:
     name: Packaging
     runs-on: ubuntu-latest

MANIFEST.in

@@ -1,5 +1,6 @@
 include LICENSE README.rst CHANGES.rst
 include tox.ini .coveragerc pytest.ini
+include readme_renderer/py.typed
 
 recursive-include tests *.html
 recursive-include tests *.py

pyproject.toml

@@ -1,3 +1,13 @@
 [build-system]
 requires = ["setuptools>=40.8.0", "wheel", "bleach>=2.1.0", "docutils>=0.13.1", "Pygments>=2.5.1"]
 build-backend = "setuptools.build_meta:__legacy__"
+
+[tool.mypy]
+strict = true
+warn_unused_configs = true
+[[tool.mypy.overrides]]
+# These modules do not yet have types available.
+module = [
+    "cmarkgfm.*"
+]
+ignore_missing_imports = true

readme_renderer/clean.py

@@ -14,6 +14,7 @@
 from __future__ import absolute_import, division, print_function
 
 import functools
+from typing import Any, Dict, Iterator, List, Optional
 
 import bleach
 import bleach.callbacks
@@ -59,15 +60,14 @@
     "input": ["type", "checked", "disabled"],
 }
 
-ALLOWED_STYLES = [
-]
-
 
 class DisabledCheckboxInputsFilter:
-    def __init__(self, source):
+    # The typeshed for bleach (html5lib) filters is incomplete, use `typing.Any`
+    # See https://github.com/python/typeshed/blob/505ea726415016e53638c8b584b8fdc9c722cac1/stubs/bleach/bleach/html5lib_shim.pyi#L7-L8 # noqa E501
+    def __init__(self, source: Any) -> None:
         self.source = source
 
-    def __iter__(self):
+    def __iter__(self) -> Iterator[Dict[str, Optional[str]]]:
         for token in self.source:
             if token.get("name") == "input":
                 # only allow disabled checkbox inputs
@@ -85,23 +85,24 @@ def __iter__(self):
             else:
                 yield token
 
-    def __getattr__(self, name):
+    def __getattr__(self, name: str) -> Any:
         return getattr(self.source, name)
 
 
-def clean(html, tags=None, attributes=None, styles=None):
+def clean(
+    html: str,
+    tags: Optional[List[str]] = None,
+    attributes: Optional[Dict[str, List[str]]] = None
+) -> Optional[str]:
     if tags is None:
         tags = ALLOWED_TAGS
     if attributes is None:
         attributes = ALLOWED_ATTRIBUTES
-    if styles is None:
-        styles = ALLOWED_STYLES
 
     # Clean the output using Bleach
     cleaner = bleach.sanitizer.Cleaner(
         tags=tags,
         attributes=attributes,
-        styles=styles,
         filters=[
             # Bleach Linkify makes it easy to modify links, however, we will
             # not be using it to create additional links.

readme_renderer/integration/distutils.py

@@ -11,6 +11,13 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+
+# The `distutils` integration is going to need to get updated to `setuptools``
+# soon enough, as `distutils` is deprecated and will be removed in Python 3.12.
+# There's currently some pass-through imports that allow this to work, but is
+# challenging for `mypy` in `strict` mode, so let's skip this file for now.
+# See https://peps.python.org/pep-0632/
+# mypy: ignore-errors
 from __future__ import absolute_import, division, print_function
 
 import cgi

readme_renderer/markdown.py

@@ -15,8 +15,9 @@
 
 import re
 import warnings
+from typing import Any, Match, Optional
 
-from html.parser import unescape
+from html import unescape
 
 import pygments
 import pygments.lexers
@@ -51,7 +52,11 @@
 }
 
 
-def render(raw, variant="GFM", **kwargs):
+def render(
+    raw: str,
+    variant: str = "GFM",
+    **kwargs: Any
+) -> Optional[str]:
     if not variants:
         warnings.warn(_EXTRA_WARNING)
         return None
@@ -61,7 +66,8 @@ def render(raw, variant="GFM", **kwargs):
     if not renderer:
         return None
 
-    rendered = renderer(raw)
+    # The renderer is a lambda function, and mypy fails lambdas right now.
+    rendered = renderer(raw)  # type: ignore
 
     if not rendered:
         return None
@@ -71,7 +77,7 @@ def render(raw, variant="GFM", **kwargs):
     return cleaned
 
 
-def _highlight(html):
+def _highlight(html: str) -> str:
     """Syntax-highlights HTML-rendered Markdown.
 
     Plucks sections to highlight that conform the the GitHub fenced code info
@@ -94,7 +100,7 @@ def _highlight(html):
         '(?(in_code)|<code>)(?P<code>.+?)'
         r'</code></pre>', re.DOTALL)
 
-    def replacer(match):
+    def replacer(match: Match[Any]) -> str:
         try:
             lang = match.group('lang')
             lang = _LANG_ALIASES.get(lang, lang)

readme_renderer/py.typed

@@ -14,20 +14,28 @@
 from __future__ import absolute_import, division, print_function
 
 import io
+from typing import Any, Dict, IO, Optional, Union
 
 from docutils.core import publish_parts
+from docutils.nodes import colspec, image
 from docutils.writers.html4css1 import HTMLTranslator, Writer
 from docutils.utils import SystemMessage
 
 from .clean import clean
 
 
-class ReadMeHTMLTranslator(HTMLTranslator):
+class ReadMeHTMLTranslator(HTMLTranslator):  # type: ignore[misc] # docutils is incomplete, returns `Any` python/typeshed#7256 # noqa E501
 
     # Overrides base class not to output `<object>` tag for SVG images.
-    object_image_types = {}
-
-    def emptytag(self, node, tagname, suffix="\n", **attributes):
+    object_image_types: Dict[str, str] = {}
+
+    def emptytag(
+        self,
+        node: Union[colspec, image],
+        tagname: str,
+        suffix: str = "\n",
+        **attributes: Any
+    ) -> Any:
         """Override this to add back the width/height attributes."""
         if tagname == "img":
             if "width" in node:
@@ -95,7 +103,11 @@ def emptytag(self, node, tagname, suffix="\n", **attributes):
 }
 
 
-def render(raw, stream=None, **kwargs):
+def render(
+    raw: str,
+    stream: Optional[IO[str]] = None,
+    **kwargs: Any
+) -> Optional[str]:
     if stream is None:
         # Use a io.StringIO as the warning stream to prevent warnings from
         # being printed to sys.stderr.

readme_renderer/rst.py

@@ -14,6 +14,7 @@
 from __future__ import absolute_import, division, print_function
 
 import sys
+from typing import Any, Optional
 
 from .clean import clean
 
@@ -26,6 +27,6 @@ def html_escape(s):
         return escape(s, quote=True).replace("'", ''')
 
 
-def render(raw, **kwargs):
+def render(raw: str, **kwargs: Any) -> Optional[str]:
     rendered = html_escape(raw).replace("\n", "<br>")
     return clean(rendered, tags=["br"])

readme_renderer/txt.py

@@ -1,13 +1,22 @@
 [tox]
-envlist = py36,py37,py38,py39,py310,pep8,packaging,noextra
+envlist = py36,py37,py38,py39,py310,pep8,packaging,noextra,mypy
 
 [testenv]
 deps =
     pytest
 commands =
-    pytest --strict {posargs}
+    pytest --strict-markers {posargs}
 extras = md
 
+[testenv:mypy]
+basepython = python3
+deps =
+    mypy
+    types-bleach
+    types-docutils
+    types-Pygments
+commands = mypy readme_renderer
+
 [testenv:pep8]
 basepython = python3
 deps =