Error in latexmk run doesn't cause build failure

[mgeier]

Details

• Build URL: https://readthedocs.org/projects/splines/builds/12895356/

Expected Result

I'm not sure if that's intended, but I would expect to see some indication of failure if the call to latexmk fails.

I guess it would still be useful to be able to access the generated HTML pages (which are built before the PDF), but some visible cue (e.g. in the build list) that there has been an error would be helpful.

Actual Result

At the end of the build, I see a nice green "Build completed" badge on the build page and a "Passed" message in the list of builds, even though there were errors:

Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 256
Latexmk: Errors, in force_mode: so I tried finishing targets

[humitos]

Hrm... This is because we found multiple cases of pdflatex failing with an exit code != 0 but creating a 100% correct PDF. So, years ago this code was added

readthedocs.org/readthedocs/doc_builder/backends/sphinx.py

Lines 458 to 461 in 30c0636

	# Force LaTeX exit code to be a little more optimistic. If LaTeX
	# reports an output file, let's just assume we're fine.
	if PDF_RE.search(self.output):
	self.exit_code = 0

It checks for Output written on (.*?) regex and if it's found, we mark the PDF as success.

Is the PDF generated for you project broken in some way?

[mgeier]

Is the PDF generated for you project broken in some way?

Good question! Yes.

I've fixed the error in the meantime, but I've created a new branch named pdf-error that reproduces the error (or non-error): https://readthedocs.org/projects/splines/builds/12911002/

The generated PDF is: https://splines.readthedocs.io/_/downloads/en/pdf-error/pdf/

On page 60 in code cell number 22, the greek letter alpha is missing (this is what the LaTeX error talks about).
This is shown in the PDF:

plot_alpha(0, ' = 0 (uniform)')

But it should be:

plot_alpha(0, 'α = 0 (uniform)')

As shown on the HTML page: https://splines.readthedocs.io/en/pdf-error/euclidean/catmull-rom-properties.html#Parameterized-Parameterization

[stsewd]

I think we can communicate better those warnings #4226

[humitos]

@mgeier excellent report! very detailed!

This is definitely something we have to improve. I'm not sure we want to fail the build (Latex is hard to make it perfect and most users would probably give up loosing the whole PDF just for a small error). However, I agree with @stsewd that we could communicate this better.

Also, we could add something similar to Sphinx's -W and --keep-going (or share the same values for the PDF build as well)

[astrojuanlu]

This affected Sphinx recently, see sphinx-doc/sphinx#9475 (thanks @mgeier for the pointer!)

[humitos]

I'd like to see a config that allows users to decide whether or not the build has to fail if the PDF (or any other format) fails.