Strict validation in configuration file (v2 only) #4607

stsewd · 2018-09-05T22:38:35Z

~~I only need to add some docs. And maybe refactor a little~~.

stsewd · 2018-09-06T17:39:02Z

Ok, I think this is done!

stsewd · 2018-09-06T21:57:23Z

readthedocs/config/config.py

+                     after the key, if no value is passed,
+                     it raises an exception when the key is not found.
+        """
+        raise_ex = not bool(args)


So... I don't like much this hack, but I didn't find a better way to handle this. We can't put something like default=None, because we may need to default some values to None sometimes too. And I didn't want to add another parameter like default=None, raise_ex.

Why can't the method just be:

def pop_config(self, key, default=None, raise_ex=True): return self.pop(key.split('.'), self.raw_config, default, raise_ex

This still works if normal usage is:

self.pop_config('formats', [])

I didn't want to have another param here, but looks better than my hacky solution p:

raise_ex should be False here by default, so when we call self.pop_config('key', 'default') we don't need to pass another value like self.pop_config('key', 'default', raise_ex=False). But when we don't provide a default value, we need to called it like self.pop_config('key', raise_ex=False).

I'm trying to replicate the same behaviour from https://docs.python.org/3/library/stdtypes.html#dict.pop. Looks like the implementation is inside the c code.

It seems like raise_ex=False would be the default value then. Does than not solve your use case?

Already implemented that in the last commit!

raise_ex = not bool(args) and the rest of this method still seem like a non-standard way of doing a standard thing to me:

def pop_config(self, key, default=None, raise_ex=False): if default is None: raise_ex = True return self.pop(key.split('.'), self.raw_config, default, raise_ex)

This seems to be the same logic without having to manually handle *args and hacking raise_ex

Actually, we have more cases where default=None and shouldn't raise an exception, we only have one use of default=None and raise_ex=True. I don't think is a good idea to modify args based on other args.

This is the current implementation

https://github.com/stsewd/readthedocs.org/blob/c17e81c975a0639d56fc6028e90f80c5598bd3e7/readthedocs/config/config.py#L196-L204

agjohnson

👍 This is a nice way of implementing a check for left over keys. Noted some minor cleanup

readthedocs/config/config.py

agjohnson · 2018-09-07T17:15:41Z

readthedocs/config/config.py

+                     after the key, if no value is passed,
+                     it raises an exception when the key is not found.
+        """
+        raise_ex = not bool(args)


Why can't the method just be:

def pop_config(self, key, default=None, raise_ex=True): return self.pop(key.split('.'), self.raw_config, default, raise_ex

This still works if normal usage is:

self.pop_config('formats', [])

agjohnson · 2018-09-07T17:26:53Z

readthedocs/config/config.py

+        if wrong_key:
+            self.error(
+                wrong_key,
+                'Unsuported configuration: {}. Maybe a typo?'.format(wrong_key),


Typo at Unsuported.

Also, perhaps Unsupported configuration option: {} for more specificity.

Maybe a typo? might not be clear for non-english speakers as well, so maybe something more formal here?

That was the most formal sentence that I came up 😞

you honestly probably don't need the second portion of the error message. I think "unsupported configuration option: {x}" says enough

Already changed in the last commit

humitos

I think this PR works, but I'd like to see some more tests and improve the docstring because I think the flow is confusing when you read the code for the first time.

I'm marking as "Comment" my review though, to not be that strict for these changes.

humitos · 2018-10-01T08:54:23Z

readthedocs/config/tests/test_config.py

+            'build.extra'
+        )
+    ])
+    def test_strict_validation(self, value, key):


I may worth to have some unit test for the specific methods that do the trick: _get_extra_key and validate_keys.

In particular the one that is recursive with different start/stop conditions.

validate_keys is already called when running validate. _get_extra_key is validated here https://github.com/stsewd/readthedocs.org/blob/strict-validation/readthedocs/config/tests/test_config.py#L1718-L1743

I have added one more test, but we are already covered anyway with the other tests (they don't throw an exception).

Not throwing an exception doesn't test that _get_extra_key returns the correct keys. That's the test I wanted.

The same for validate_keys to check that if there is extra keys, it fails.

actually test_strict_validation test that, an error is thrown and key and code are the correct in the error. We test with validate as entrypoint, as that's the way to use the config module (all tests were changed to test like that)

I understand what you say. Although, I'd like to have a unit test for the methods I mentioned before for these cases:

_get_extra_key

returns no extra keys

a couple of cases where it returns extra keys

validate_keys

a couple of cases where it raises

the same for no raising the exception

The test_strict_validation is testing only one case and using the complete flow. I'd like to test the inner pieces, in particular the than that it's recursive (it's very easy to make a mistake on initial conditions)

humitos · 2018-10-01T08:55:59Z

readthedocs/config/config.py

+        """
+        Checks that we don't have extra keys.
+
+        This should be called after all the validations are done.


Why? Can you expand a little more here?

If we know that we have an extra key and the build will fail because of this, why we are going to validate all the values first if we already know that it will fail?

I think I got it now, but it's hard to read.

I suppose that when validating the keys we pop (remove) these keys from the raw_config and finally if there are some leftovers it means that we have invalid keys.

If this is the way it works, it would be good to clarify in the docstring.

Yep, that is

humitos · 2018-10-01T08:56:21Z

readthedocs/config/config.py

            submodules['recursive'] = validate_bool(recursive)

        return submodules

+    def validate_keys(self):
+        """
+        Checks that we don't have extra keys.


I'd say Checks that we don't have invalid keys.

humitos · 2018-10-01T08:56:56Z

readthedocs/config/config.py

+        This should be called after all the validations are done.
+        """
+        msg = (
+            'Unsupported configuration option: {}. '


I'd say Invalid configuration option:

That makes more sense here

humitos · 2018-10-01T09:00:05Z

readthedocs/config/config.py

+            'Unsupported configuration option: {}. '
+            'Make sure the key name is correct.'
+        )
+        self.pop_config('version', None)


What's this for?

We always pass the version key, but that's is already validated from outside

agjohnson · 2018-10-02T17:16:51Z

Changes look good to me, when @humitos gives the 👍 here, feel free to rebase/fix conflicts and merge!

stsewd · 2018-10-03T16:43:49Z

@humitos I have added more tests https://github.com/rtfd/readthedocs.org/pull/4607/files#diff-7bfb3976afdb676b247dad2ffe16295bR1755

humitos · 2018-10-04T09:58:00Z

readthedocs/config/config.py

+        """
+        Get the extra keyname (list form) of a dict object.
+
+        If there are more the one key, the first one is returned.


"more than one extra key"

humitos

Excellent! Thanks for the tests 😍

I left a comment for a typo :)

stsewd added PR: work in progress Pull request is not ready for full review and removed PR: work in progress Pull request is not ready for full review labels Sep 6, 2018

stsewd requested a review from a team September 6, 2018 17:46

stsewd commented Sep 6, 2018

View reviewed changes

agjohnson reviewed Sep 7, 2018

View reviewed changes

agjohnson added this to the YAML File Completion milestone Sep 19, 2018

humitos reviewed Oct 1, 2018

View reviewed changes

agjohnson modified the milestones: YAML File Completion, 2.7 Oct 2, 2018

stsewd added 11 commits October 3, 2018 11:34

Tests

0bbbc6a

Validation

573c878

Refactor

2fc02e7

Better docs

28ccce3

Refactor

06b40f8

Isort

1187efb

One more test

67d0420

Better docstrings

c60f59a

More tests

a2c6c9d

And more tests

7ce0202

Isort

d33854a

stsewd force-pushed the strict-validation branch from aef1b9f to d33854a Compare October 3, 2018 16:40

humitos reviewed Oct 4, 2018

View reviewed changes

humitos approved these changes Oct 4, 2018

View reviewed changes

Typo

997b5e2

stsewd merged commit 35695d1 into readthedocs:master Oct 4, 2018

stsewd deleted the strict-validation branch October 4, 2018 15:57

Uh oh!

Strict validation in configuration file (v2 only) #4607

Strict validation in configuration file (v2 only) #4607

Uh oh!

Conversation

stsewd commented Sep 5, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stsewd commented Sep 6, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agjohnson left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

humitos left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agjohnson commented Oct 2, 2018

Uh oh!

stsewd commented Oct 3, 2018

Uh oh!

humitos Oct 4, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

stsewd commented Sep 5, 2018 •

edited

Loading

humitos Oct 4, 2018 •

edited

Loading