♻️ REFACTOR: package code (#55)

Essentially re-writes the most of the code, to implement a number of improvements and bug fixes, but the output LaTeX is shown (via tests) to be essentially the same.

- move event functions to separate file (and rename)
- add sphinx based tests, which provide comparisons of the doctrees create with/without this extension, before/after post-transforms have been applied. This also shows how the package can be used as a "standalone" sphinx extension.
- use `importlib.resources` for accessing `jupyterBook.cls`
- Remove enforcement of `myst-parser` `amsmath` extension
- make `myst_nb` requirement optional, and check for compatibility
- convert class/function nameing from pascalCase to camel_case (and enforce with pep8-naming linter):
  - replace `getFilenameWithSubpath` with `is_root_document`
  - replace `removeExtension` with `remove_suffix`
- replace `H2Node` and `H3Node` by `RootHeader` node, which applies to any sub-section of the root file (i.e. also H4 etc)
- added `app.add_config_value("jblatex_load_imgconverter", True, "env")` to enable loading of sphinx.ext.imgconverter to be turned off
- remove reliance on _toc.yml, instead adding `app.add_config_value("jblatex_captions_to_parts", None, "env", (type(None), bool))`
  - If `None` and `app.env.external_site_map` is present (from sphinx-external-toc), then this will be used to "infer" `latex_toplevel_sectioning` and whether to convert captions to parts.
- Move all doctree restructuring to the post-transform, so that the stored doctrees are builder agnostic (as should be the case). The transform (which is now called for all builders) only adds extra node attributes for use by the post-transform.
- a lot more code documentation and added to README

Author: chrisjsewell

.gitignore

@@ -1,5 +1,6 @@
 # Byte-compiled / optimized / DLL files
 __pycache__/
+*.pyc
 
 build/
 dist/

.pre-commit-config.yaml

@@ -39,14 +39,17 @@ repos:
     rev: 3.8.4
     hooks:
     - id: flake8
-      additional_dependencies: [flake8-bugbear==21.3.1]
+      additional_dependencies:
+      - flake8-bugbear~=21.3.1
+      - flake8-comprehensions~=3.4.0
+      - pep8-naming~=0.11.1
 
   - repo: https://github.com/pre-commit/mirrors-mypy
     rev: v0.790
     hooks:
     - id: mypy
       additional_dependencies: ["sphinx>=3,<4"]
-      exclude: tests/roots/test-codeCellTransforms/conf.py
+      exclude: tests/roots/.*py
 
   - repo: https://github.com/asottile/setup-cfg-fmt
     rev: v1.16.0

MANIFEST.in

@@ -14,4 +14,4 @@ include README.md
 include CHANGELOG.md
 
 include jupyterbook_latex/py.typed
-recursive-include jupyterbook_latex/theme *
+include jupyterbook_latex/theme/jupyterBook.cls

README.md

@@ -1,6 +1,6 @@
-# jupyterbook-latex
+# jupyterbook-latex [IN DEVELOPMENT]
 
-Supporting LaTeX infrastructure for Jupyter Book
+Sphinx extension to support LaTeX infrastructure for Jupyter Book.
 
 This repository is a **development** project to improve LaTeX support
 in `Jupyter Book`.
@@ -13,14 +13,47 @@ To get started with `jupyterbook-latex`, first install it through `pip`:
 pip install jupyterbook-latex
 ```
 
-then, add `jupyterbook_latex` to the `config.yml` file in your jupyterbook projects:
+then, add `jupyterbook_latex` to your extensions,
+in a Sphinx `conf.py`:
 
+```python
+extensions = ["jupyterbook_latex"]
+
+# autoload the sphinx.ext.imgconverter extension, optional (default is True)
+# jblatex_load_imgconverter = True
+# turn root level toctree captions into top-level `part` headings, optional (default is to auto-infer)
+#  jblatex_captions_to_parts = True
 ```
+
+OR in the jupyterbook `config.yml`:
+
+```yaml
 sphinx:
     extra_extensions:
-        - jupyterbook_latex
+    - jupyterbook_latex
+    # config:
+    #   jblatex_load_imgconverter: true
+    #   jblatex_captions_to_parts: true
 ```
 
+## Extension Details
+
+This extension does not provide an actual Sphinx LaTeX theme,
+instead it instantiates a number of transforms (for LaTeX builders only) that manipulate the AST into the required format:
+
+1. Overrides some configuration:
+
+- ``latex_engine`` -> ``xelatex``
+- ``latex_theme`` -> ``jupyterBook``
+- appends necessary LaTeX commands to the preamble
+
+2. When a latex builder is specified:
+
+- Set's up `sphinx.ext.imgconverter` (if `jblatex_load_imgconverter`)
+- Replace sub-headers in the root document
+- Create headings from the root-level toctree captions (if `jblatex_captions_to_parts`)
+- Move bibliographies to the bottom of the document
+
 Issues
 ------
 

docs/source/conf.py renamed to docs/conf.py

@@ -23,7 +23,7 @@
 # This pattern also affects html_static_path and html_extra_path.
 exclude_patterns = ["_build"]
 
-html_static_path = ["_static"]
+# html_static_path = ["_static"]
 
 # -- Options for HTML output -------------------------------------------------
 

docs/source/contributing.md renamed to docs/contributing.md

@@ -1,118 +1,32 @@
-import os
-from pathlib import Path
+""" """
 
-from docutils import nodes as docnodes
-from sphinx import builders
-from sphinx.util import logging
-from sphinx.util.fileutil import copy_asset_file
 
-from .nodes import (
-    H2Node,
-    H3Node,
-    HiddenCellNode,
-    depart_H2Node,
-    depart_H3Node,
-    visit_H2Node,
-    visit_H3Node,
-    visit_HiddenCellNode,
-)
-from .transforms import (
-    LatexMasterDocTransforms,
-    ToctreeTransforms,
-    codeCellTransforms,
-    handleSubSections,
-)
+from typing import TYPE_CHECKING
 
-__version__ = "0.2.0"
-"""jupyterbook-latex version"""
-
-logger = logging.getLogger(__name__)
-
-
-# Helper node
-def skip(self, node):
-    raise docnodes.SkipNode
-
-
-def build_init_handler(app):
-    from sphinx.util.console import bold
-
-    # only allow latex builder to access rest of the features
-    if isinstance(app.builder, builders.latex.LaTeXBuilder):
-        app.add_post_transform(codeCellTransforms)
-        copy_static_files(app)
-        TOC_PATH = Path(app.confdir).joinpath("_toc.yml")
-        if not os.path.exists(TOC_PATH):
-            logger.info(
-                "Some features of this exetension will work only with a jupyter-book application"  # noqa: E501
-            )
-            return
-        app.config["myst_amsmath_enable"] = True
-        app.setup_extension("sphinx.ext.imgconverter")
-        app.add_transform(LatexMasterDocTransforms)
-        app.add_post_transform(ToctreeTransforms)
-        app.add_post_transform(handleSubSections)
-        logger.info(
-            bold("jupyterbook-latex v%s:") + "(latex_engine='%s')",
-            __version__,
-            app.config["latex_engine"],
-        )
+if TYPE_CHECKING:
+    from sphinx.application import Sphinx
 
+__version__ = "0.2.1a1"
+"""jupyterbook-latex version"""
 
-def add_necessary_config(app, config):
-    # only allow latex builder to access rest of the features
-    config["latex_engine"] = "xelatex"
-    config["latex_theme"] = "jupyterBook"
 
-    # preamble to overwrite things from sphinx latex writer
-    configPreamble = ""
-    if "preamble" in config["latex_elements"]:
-        configPreamble = config["latex_elements"]["preamble"]
+def setup(app: "Sphinx") -> None:
+    """The sphinx entry-point for the extension."""
 
-    config["latex_elements"]["preamble"] = (
-        configPreamble
-        + r"""
-         \usepackage[Latin,Greek]{ucharclasses}
-        \usepackage{unicode-math}
-        % fixing title of the toc
-        \addto\captionsenglish{\renewcommand{\contentsname}{Contents}}
-        """
-    )
+    from .events import override_latex_config, setup_latex_transforms
+    from .nodes import HiddenCellNode, RootHeader
+    from .transforms import LatexRootDocTransforms
 
+    # autoload the sphinx.ext.imgconverter extension
+    app.add_config_value("jblatex_load_imgconverter", True, "env")
+    # turn root level toctree captions into top-level `part` headings
+    # If None, auto-infer whether to do this, or specifically specify
+    app.add_config_value("jblatex_captions_to_parts", None, "env", (type(None), bool))
 
-def copy_static_files(app):
-    themePath = Path(__file__).parent.joinpath("theme")
-    clsFile = themePath.joinpath("jupyterBook.cls")
-    copy_asset_file(str(clsFile), app.outdir)
+    HiddenCellNode.add_node(app)
+    RootHeader.add_node(app)
 
+    app.add_transform(LatexRootDocTransforms)
 
-def setup(app):
-    app.add_node(
-        HiddenCellNode,
-        override=True,
-        html=(visit_HiddenCellNode, None),
-        latex=(visit_HiddenCellNode, None),
-        textinfo=(visit_HiddenCellNode, None),
-        text=(visit_HiddenCellNode, None),
-        man=(visit_HiddenCellNode, None),
-    )
-    app.add_node(
-        H2Node,
-        override=True,
-        latex=(visit_H2Node, depart_H2Node),
-        html=(visit_H2Node, depart_H2Node),
-        textinfo=(skip, None),
-        text=(skip, None),
-        man=(skip, None),
-    )
-    app.add_node(
-        H3Node,
-        override=True,
-        latex=(visit_H3Node, depart_H3Node),
-        html=(visit_H3Node, depart_H3Node),
-        textinfo=(skip, None),
-        text=(skip, None),
-        man=(skip, None),
-    )
-    app.connect("config-inited", add_necessary_config)
-    app.connect("builder-inited", build_init_handler)
+    app.connect("config-inited", override_latex_config)
+    app.connect("builder-inited", setup_latex_transforms)

docs/source/intro.md renamed to docs/intro.md

@@ -0,0 +1,98 @@
+import sys
+from typing import cast
+
+from sphinx.application import Sphinx
+from sphinx.builders.latex import LaTeXBuilder
+from sphinx.config import Config
+from sphinx.util import logging
+from sphinx.util.fileutil import copy_asset_file
+
+from . import __version__, theme
+from .transforms import LatexRootDocPostTransforms, MystNbPostTransform
+
+if sys.version_info < (3, 9):
+    import importlib_resources as resources
+else:
+    import importlib.resources as resources
+
+logger = logging.getLogger(__name__)
+
+
+def override_latex_config(app: Sphinx, config: Config) -> None:
+    """This ``config-inited`` event overrides aspects of the sphinx latex config.
+
+    - ``latex_engine`` -> ``xelatex``
+    - ``latex_theme`` -> ``jupyterBook``
+    - appends necessary LaTeX commands to the preamble
+
+    """
+    # only allow latex builder to access rest of the features
+    config["latex_engine"] = "xelatex"
+    config["latex_theme"] = "jupyterBook"
+
+    latex_elements = cast(dict, config["latex_elements"])
+
+    # preamble to overwrite things from sphinx latex writer
+    config_preamble = (
+        latex_elements["preamble"] if "preamble" in config["latex_elements"] else ""
+    )
+
+    latex_elements["preamble"] = (
+        config_preamble
+        + r"""
+         \usepackage[Latin,Greek]{ucharclasses}
+        \usepackage{unicode-math}
+        % fixing title of the toc
+        \addto\captionsenglish{\renewcommand{\contentsname}{Contents}}
+        """
+    )
+
+
+def setup_latex_transforms(app: Sphinx) -> None:
+    """This ``builder-inited`` event sets up aspects of the extension,
+    reserved only for when a LaTeX builder is specified.
+    """
+
+    if not isinstance(app.builder, LaTeXBuilder):
+        return
+
+    # note: bold is a dynamically created function
+    from sphinx.util.console import bold  # type: ignore[attr-defined]
+
+    # decide whether we will convert top-level toctree captions to parts
+    app.env.jblatex_captions_to_parts = False  # type: ignore[attr-defined]
+    if app.config["jblatex_captions_to_parts"] is True:  # type: ignore[comparison-overlap]
+        app.config["latex_toplevel_sectioning"] = "part"
+        app.env.jblatex_captions_to_parts = True
+    elif app.config["jblatex_captions_to_parts"] is None:
+        # if using the sphinx-external-toc, we can look if parts are being specified
+        # TODO this should probably be made more robust
+        sitemap = getattr(app.config, "external_site_map", None)
+        if sitemap is not None:
+            if sitemap.file_format == "jb-book" and len(sitemap.root.subtrees) > 1:
+                app.config["latex_toplevel_sectioning"] = "part"
+                app.env.jblatex_captions_to_parts = True
+            elif sitemap.file_format == "jb-book":
+                app.config["latex_toplevel_sectioning"] = "chapter"
+            elif sitemap.file_format == "jb-article":
+                app.config["latex_toplevel_sectioning"] = "section"
+
+    logger.info(
+        bold("jupyterbook-latex v%s:") + "engine='%s', toplevel_section='%s'",
+        __version__,
+        app.config["latex_engine"],
+        app.config["latex_toplevel_sectioning"],
+    )
+
+    # Copy the class theme to the output directory.
+    # note: importlib.resources is the formal method to access files within packages
+    with resources.as_file(resources.files(theme).joinpath("jupyterBook.cls")) as path:
+        copy_asset_file(str(path), app.outdir)
+
+    # only load when myst-nb is present
+    if MystNbPostTransform.check_dependency():
+        app.add_post_transform(MystNbPostTransform)
+
+    if app.config["jblatex_load_imgconverter"]:
+        app.setup_extension("sphinx.ext.imgconverter")
+    app.add_post_transform(LatexRootDocPostTransforms)