YAML config: errors are not clear enough for users

[jnikula]

Details

I configured .readthedocs.yaml with:

build:
  tools:
    python: 3.11

Expected Result

Nothing unusual.

Actual Result

I got the error message:

Problem in your project's configuration. Invalid "build.tools.python":
expected one of (2.7, 3.6, 3.7, 3.8, 3.9, 3.10, 3.11, pypy3.7, pypy3.8,
pypy3.9, miniconda3-4.7, mambaforge-4.10, 3), got 3.11

Despite the confusing message "expected 3.11, got 3.11", I realized the latter was probably a float instead of a string.

The workaround was simple:

build:
  tools:
    python: "3.11"

But it can potentially be a huge time waster for someone not familiar with the pitfalls of yaml parsing. It would be nice if the parsing was more robust against this kind of type issues. Or the documentation explicitly told to use quotes.

Oh, Norway problem, you ask? See https://hitchdev.com/strictyaml/why/implicit-typing-removed/

[humitos]

Yes. We have to improve the error message we expose to users to be more friendly 😄 . We have a few issues about this already. This is the code where we should give the users more context about what has happened:

readthedocs.org/readthedocs/config/validation.py

Lines 15 to 28 in 1b32d18

	class ValidationError(Exception):
	"""Base error for validations."""
	messages = {
	INVALID_BOOL: 'expected one of (0, 1, true, false), got {value}',
	INVALID_CHOICE: 'expected one of ({choices}), got {value}',
	INVALID_DICT: '{value} is not a dictionary',
	INVALID_PATH: 'path {value} does not exist',
	INVALID_PATH_PATTERN: '{value} isn\'t a valid path pattern',
	INVALID_STRING: 'expected string',
	INVALID_LIST: 'expected list',
	VALUE_NOT_FOUND: '{value} not found',
	}

[humitos]

We should make the error of using pip.install.extra_requirements: [doc] clearer. See #10717

[stsewd]

Another error is that when using node, we require users to specify a python version too. The error isn't clear about this.

[humitos]

Another related issue: #6569

[humitos]

... and another one: #7707

There are a bunch of issues related to this 😞 -- probably worth prioritizing this work.

[humitos]

... another #8482 ...

[humitos]

🙃 , #8352

[humitos]

hrm... #9458

[humitos]

wait, there are more issues #9713 and #9365

[stsewd]

An error isn't clear when a user is using an old version of the config file (v1).

Ref #10824

[stsewd]

Looks like we lose the message about the validation of the version of the config file.

Example: https://github.com/readthedocs/test-builds/blob/config-file-with-wrong-version-number/.readthedocs.yaml#L1
Build: https://readthedocs.org/projects/test-builds/builds/23531844/

[stsewd]

Another one when the build option is missing https://github.com/pfusik/ONE/blob/55dc7c6e4d388a85c0deb49c8da40edbfdba72e7/.readthedocs.yml

[agjohnson]

These last two are a slightly different issue, and are a change/bug where project notifications are only visible for the maintainer users.

The beta instances doesn't have this change yet, the error is clearer there:
https://beta.readthedocs.org/projects/test-builds/builds/23531844/

[humitos]

These last two are a slightly different issue, and are a change/bug where project notifications are only visible for the maintainer users.

I opened #11146 to track this issue.

[humitos]

We made our error messages a clearer. They are not perfect still, tho, but I'm tempted to close this issue at this point since there are no clear directions here and the issue from the description was already fixed. Besides, we have created an issue per problem for those that are more important. See the list at #10985 (comment)

[agjohnson]

Agreed. I feel like we've addressed a few issues here in a few ways. If there are any more issues with the configuration file, we will address this individually at this point.