Skip to content

docs: adding translation stats to docs #511

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions TRANSLATING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ This guide will help you get started contributing to the translation of the Pyth

The process of contributing to the translation of the guide is similar to the process of contributing to the guide itself, except that instead of working on the guide source files directly, you will be working on the translation files.

# Translation Status

```{translation-graph}
```

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe here would be a good spot to include the link to the site

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like we would want both directions? from the contributing page here and vice versa?

## Overview of the Translation Process

The process of adapting software to different languages is called internationalization, or i18n for short. Internationalization makes sure that translation can happen without having to modify the source code, or in our case, the original English source files of the guide.
Expand Down
85 changes: 85 additions & 0 deletions _ext/translation_graph.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
from pathlib import Path
import json
from typing import TypeAlias, TypedDict, Annotated as A

from docutils import nodes
from docutils.parsers.rst import Directive
import plotly.graph_objects as go
from plotly.offline import plot

Comment on lines +8 to +9
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
from plotly.offline import plot
from plotly.offline import plot
import numpy as np

oops, forgot that i added this (though this is just trying to follow the docs where it says it must be an array, but i bet it would work fine just as a list of lists)


class ModuleStats(TypedDict):
total: int
translated: int
fuzzy: int
untranslated: int
percentage: float

TranslationStats: TypeAlias = dict[A[str, "locale"], dict[A[str, "module"], ModuleStats]]


class TranslationGraph(Directive):
# Tells Sphinx that this directive can be used in the document body
# and has no content
has_content = False

def run(self):
# Read the JSON file containing translation statistics
json_path = Path(__file__).parent.parent / "_static" / "translation_stats.json"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i missed the PR that added the translation_stats script, but i think that this will go out of date almost immediately and become a misleading indicator if we don't generate this during the build process so that at the time the docs are generated the plot and the stats both reflect the same state of the repo.

I personally avoid committing generated data files that only need to exist at deployment time because they make PRs noisy and tempt us to treat them as files we can edit, but if we are to keep it there, we should add it and trigger it from some early build event like builder-inited - see the _post_build and setup functions at the bottom of conf.py

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @sneakers-the-rat! Yeah that makes sense - I could move the generation of the JSON file into a build_event. That sounds reasonable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it took me many years and many attempts of trying before the hook pattern of sphinx sunk in for me, the way you did it was completely understandable

with json_path.open("r") as f:
data: TranslationStats = json.load(f)

# Collect all module names -- iterates over the JSON data in 2 levels
all_modules = {module for stats in data.values() for module in stats}
all_modules = sorted(all_modules)

# Build one trace per locale with full hover info
traces = []

for locale, modules in data.items():
y_vals = []
hover_texts = []

for module in all_modules:
stats = modules.get(module)
y_vals.append(stats["percentage"])

hover_text = (
f"<b>{module}</b><br>"
f"Translated: {stats['translated']}<br>"
f"Fuzzy: {stats['fuzzy']}<br>"
f"Untranslated: {stats['untranslated']}<br>"
f"Total: {stats['total']}<br>"
f"Completed: {stats['percentage']}%"
)
hover_texts.append(hover_text)

traces.append(go.Bar(
name=locale,
x=all_modules,
y=y_vals,
hovertext=hover_texts,
hoverinfo="text"
))

# Create figure
fig = go.Figure(data=traces)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RobPasMue could we create a grid of plots - one for each language?

then each plot could have 3 bars - one for fuzzy, one for complete and one for incomplete (or it could be stacked bars too.

What you have now is awesome but if we add more languages it will get complex over time. And a static version of the plot would be nice too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure sounds good!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a good idea would be a heat map, which is condensed enough we can add many more languages.

Copy link
Contributor

@sneakers-the-rat sneakers-the-rat May 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed on the heatmap. i would expect it oriented with languages as rows and pages as columns (which satisfies the need to expandability to future languages)

fig.update_layout(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the plot please use our pyOS colors?

Dark Purple: #33205c
Light Purple: #735fab
Pale Purple: #bab3d4
Magenta: #bb82b0
Sea Green: #81c0aa

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most def! =) I'll look into the colors as soon as I can

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh whoops. just saw this comment. probably would be good to use the css variables directly when we can to avoid having them hardcoded in multiple places. i couldn't find a rhyme or reason to when i was able to use css vars in the plotly values and when i needed to declare them in the stylesheet, but ya some examples in this comment

barmode="group",
title="Translation Coverage by Module and Locale",
xaxis_title="Module",
yaxis_title="Percentage Translated",
height=600,
margin=dict(l=40, r=40, t=40, b=40)
)

div = plot(fig, output_type="div", include_plotlyjs=True)
return [nodes.raw("", div, format="html")]

def setup(app):
app.add_directive("translation-graph", TranslationGraph)
return {
"version": "0.1",
"parallel_read_safe": True,
"parallel_write_safe": True,
}
1 change: 1 addition & 0 deletions conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@
"sphinxext.opengraph",
"sphinx_favicon",
"sphinxcontrib.bibtex",
"_ext.translation_graph",
]

# colon fence for card support in md
Expand Down
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ dependencies = [
"sphinx-inline-tabs",
# for project cards
"matplotlib",
# for translation graphs
"plotly",
# for license page bibliography
"sphinxcontrib-bibtex",
]
Expand Down
Loading