Skip to content

ValidationError -- better detail #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ghost opened this issue Mar 22, 2012 · 4 comments
Closed

ValidationError -- better detail #5

ghost opened this issue Mar 22, 2012 · 4 comments
Labels
Enhancement Some new desired functionality

Comments

@ghost
Copy link

ghost commented Mar 22, 2012

Julian,

I modified your code to have ValidationError provide more detail

In [15]: try:
   ....:        jsonschema.validate(message, EVENT_TYPE_CREATE, stop_on_error=False)
   ....: except jsonschema.ValidationError as err:
   ....:        print err.detail
   ....: 
{'organized_flag': [u"'yes' is not of type 'boolean'"], 'parent_id': [u'-30 is less than the minimum of 0'], 'name': [u"'name' is a required property"]}

ie.,
https://gist.github.com/2158863

would there be a reason not to do this?

Cheers,
-K

@Julian
Copy link
Member

Julian commented Mar 22, 2012

Hi.

That looks ok for objects just from taking a quick peek, but how about when
you're validating, say, an array?

@ghost
Copy link
Author

ghost commented Mar 22, 2012

Good point.

    def validate_minItems(self, mI, instance, schema, property):
        if self._is_type(instance, "array") and len(instance) < mI:
            self._error(u"%r is too short" % (instance,), property)

    def validate_maxItems(self, mI, instance, schema, property):
        if self._is_type(instance, "array") and len(instance) > mI:
            self._error(u"%r is too long" % (instance,), property)

Is there anywhere that traverses inside the array to check the correct value types? I think we'd want to make sure the property name is still passed through.

How do I run your tests again the code update?

How would you approach it? The reason we're developing this is so not only is there human readable error messages, but we can also machine read and know what 'property' has errors, etc.

Cheers

@Julian
Copy link
Member

Julian commented Mar 22, 2012

Is there anywhere that traverses inside the array to check the correct value types? I think we'd want to make sure the property name is still passed through.

That validation would happen inside validate_items (the validation is
basically just like walking a tree, similar to ast.NodeVisitor in case
you're familiar). But, what I meant was, what if your entire instance is
just an array, then you'd have no key / property to store at all. How would
details behave there?

How do I run your tests again the code update?

If you've got tox installed you can just run tox in the root of the
checkout. Otherwise just run your favorite test runner, say trial tests
or nosetests also from the checkout.

How would you approach it? The reason we're developing this is so not only is there human readable error messages, but we can also machine read and know what 'property' has errors, etc.

Yeah, I can definitely appreciate making the errors more structured here
once we can just iron out the behavior for all the cases.

@ghost
Copy link
Author

ghost commented Mar 26, 2012

Julian,

Can you drop me an email at kevin at mapmy fitness dotcom? Would like to explore something with you...

Cheers,

-Kevin

@Julian Julian mentioned this issue Jun 7, 2012
@Julian Julian closed this as completed in b1d7464 Jun 18, 2012
Julian added a commit that referenced this issue Dec 25, 2012
Squashed commit of the following:

commit b255d27
Author: Julian Berman <[email protected]>
Date:   Mon Dec 24 19:41:09 2012 -0500

    Add tests for ignoring irrelevant types.

commit 71a0d30
Merge: 2262a91 082e844
Author: Julian Berman <[email protected]>
Date:   Sun Dec 23 21:14:38 2012 -0500

    Tag version 1.0.0.

    Closes #13

commit 082e844
Author: Julian Berman <[email protected]>
Date:   Sun Dec 23 00:28:48 2012 -0500

    Cleanup

    * Make errors in the tests be failures
    * Removed use of ErrorTree
    * Added another helper fn
    * Use fnmatch instead of glob

commit e39d537
Author: Julian Berman <[email protected]>
Date:   Fri Dec 21 14:52:16 2012 -0500

    Added tests differentiating between lexical and mathematical integers.

    Integer literals should be considered integers, while mathematical integers can
    be specified with divisibleBy.

    See: json-schema/json-schema#27

    Closes #2

commit 44540b7
Author: Julian Berman <[email protected]>
Date:   Fri Dec 21 14:47:26 2012 -0500

    Added name to license.

commit 646a981
Author: Julian Berman <[email protected]>
Date:   Fri Dec 21 14:31:47 2012 -0500

    Add a test for BigNum integers.

    Closes #12

commit d22c24e
Merge: e8bcd5a c0dbbb5
Author: Julian Berman <[email protected]>
Date:   Wed Dec 19 10:57:03 2012 -0800

    Merge pull request #17 from gazpachoking/validate_suite_content

    Add sanity checks for actual format of test suites

commit c0dbbb5
Author: Chase Sterling <[email protected]>
Date:   Wed Dec 19 10:53:57 2012 -0500

    Add back executable bit to suite_sanity_check

commit 86f3486
Author: Chase Sterling <[email protected]>
Date:   Wed Dec 19 00:49:20 2012 -0500

    Add sanity check to validate all suites against suite schema.
    Make sure optional test suites don't skip the sanity checks.

commit e8bcd5a
Author: Nick Lombard <[email protected]>
Date:   Wed Dec 19 06:38:00 2012 +0200

    Update README.md

    Travis status for develop branch

commit ca8f107
Author: Nick Lombard <[email protected]>
Date:   Wed Dec 19 06:36:03 2012 +0200

    Update README.md

    Added travis status image

commit cf4597e
Author: Julian Berman <[email protected]>
Date:   Tue Dec 18 23:27:13 2012 -0500

    Minor formatting (and get travis to build)

commit 5cb630c
Author: Julian Berman <[email protected]>
Date:   Tue Dec 18 23:12:42 2012 -0500

    Show number of tests found

commit 7baea07
Author: Julian Berman <[email protected]>
Date:   Tue Dec 18 23:11:33 2012 -0500

    Travis

commit 2f320bb
Author: Julian Berman <[email protected]>
Date:   Tue Dec 18 23:07:23 2012 -0500

    Add sanity checker.

commit 2262a91
Merge: 9cda89e a1ec588
Author: Julian Berman <[email protected]>
Date:   Tue Dec 18 13:22:23 2012 -0800

    Merge pull request #15 from tdegrunt/patch-1

    Update README.md

commit a1ec588
Author: Tom de Grunt <[email protected]>
Date:   Tue Dec 18 22:10:48 2012 +0100

    Update README.md

    Added node.js module jsonschema.

commit 9cda89e
Author: Julian Berman <[email protected]>
Date:   Tue Dec 18 14:35:19 2012 -0500

    Update README.md

commit b22d795
Author: Julian Berman <[email protected]>
Date:   Sat Nov 17 19:26:21 2012 -0500

    Ignore TODO

commit 330fd48
Author: Julian Berman <[email protected]>
Date:   Sat Nov 17 19:25:27 2012 -0500

    Merge into properties tests.

commit c7c2338
Author: Julian Berman <[email protected]>
Date:   Sat Nov 17 19:20:25 2012 -0500

    Remove duplicate tests.

commit 281c28b
Merge: 6f55ff5 6c28d7c
Author: Julian Berman <[email protected]>
Date:   Sat Nov 17 19:14:19 2012 -0500

    Merge git://github.com/IreneKnapp/JSON-Schema-Test-Suite

commit 6c28d7c
Author: Irene Knapp <[email protected]>
Date:   Fri Nov 16 08:33:30 2012 -0500

    Fix a test that should actually be invalid.  Remove some unnecessary tests.

commit b7858cc
Author: Irene Knapp <[email protected]>
Date:   Thu Nov 15 22:48:09 2012 -0500

    Port over the rest of the stuff from jsonschema (the Python project).

commit fa66623
Author: Irene Knapp <[email protected]>
Date:   Thu Nov 15 16:37:56 2012 -0500

    Port roughly the first third of jsonschema (the Python project)'s tests.py.

commit 6f55ff5
Author: Julian Berman <[email protected]>
Date:   Thu Nov 15 14:46:48 2012 -0500

    Add the haskell thing that was using us.

commit 756c825
Merge: d735a8a 459dd98
Author: Julian Berman <[email protected]>
Date:   Thu Nov 15 11:44:08 2012 -0800

    Merge pull request #6 from IreneKnapp/master

    Add direct-schema to list of clients

commit 459dd98
Author: Irene Knapp <[email protected]>
Date:   Thu Nov 15 14:40:41 2012 -0500

    Adding direct-schema to list of clients.

commit d735a8a
Author: Julian Berman <[email protected]>
Date:   Thu Nov 15 09:37:03 2012 -0500

    Add call to action.

commit ca78ea8
Author: Julian Berman <[email protected]>
Date:   Thu Nov 1 08:58:50 2012 -0400

    Add heterogeneous types unique test.

commit 4ab01de
Author: Julian Berman <[email protected]>
Date:   Wed Oct 31 20:12:53 2012 -0400

    Add specific test for bool and int.

commit 6a52026
Author: Julian Berman <[email protected]>
Date:   Wed Oct 31 20:12:38 2012 -0400

    Simplify test to remove also testing integers and booleans.

commit 3871dea
Merge: fefa4dc 6f18358
Author: Julian Berman <[email protected]>
Date:   Wed Oct 10 17:39:19 2012 -0400

    Merge branch 'render'

commit 6f18358
Author: Julian Berman <[email protected]>
Date:   Wed Oct 10 17:37:54 2012 -0400

    Fix README example.

commit fefa4dc
Merge: 189e231 295d43a
Author: Julian Berman <[email protected]>
Date:   Wed Oct 10 12:47:28 2012 -0700

    Merge pull request #5 from timjb/master

    Added tests for divisibleBy, format: "regex"

commit 295d43a
Author: Tim Baumann <[email protected]>
Date:   Wed Oct 10 21:27:32 2012 +0200

    Move format.json to folder for optional tests

commit 8f5d9e2
Author: Tim Baumann <[email protected]>
Date:   Wed Oct 10 21:06:08 2012 +0200

    regex validation

commit f11ddc3
Author: Tim Baumann <[email protected]>
Date:   Wed Oct 10 20:59:55 2012 +0200

    divisibleBy tests

commit 189e231
Author: Julian Berman <[email protected]>
Date:   Wed Oct 10 10:00:42 2012 -0400

    Fix the additionalItems tests.

    Closes #4

commit a0a6e71
Author: Julian Berman <[email protected]>
Date:   Wed Oct 10 09:54:52 2012 -0400

    Fix nil.

    Closes #3

commit 8dbaf21
Author: Julian Berman <[email protected]>
Date:   Tue Oct 9 20:28:09 2012 -0400

    Fix line endings and add invalid test case.

commit 7829853
Author: Julian Berman <[email protected]>
Date:   Sun Oct 7 10:33:14 2012 -0400

    required

commit 91e57ae
Author: Julian Berman <[email protected]>
Date:   Sun Oct 7 10:24:35 2012 -0400

    Pattern

commit 49af2eb
Author: Julian Berman <[email protected]>
Date:   Sat Oct 6 20:50:59 2012 -0400

    uniqueItems

commit 089ed8e
Author: Julian Berman <[email protected]>
Date:   Sat Oct 6 20:50:23 2012 -0400

    Enum

commit 7e4b937
Author: Julian Berman <[email protected]>
Date:   Sat Oct 6 20:40:53 2012 -0400

    max/min Items/Length

commit 5c27f85
Author: Julian Berman <[email protected]>
Date:   Sat Oct 6 20:34:21 2012 -0400

    Minimum and maximum

commit 9e24adb
Merge: 3649b58 3498f34
Author: Julian Berman <[email protected]>
Date:   Wed Sep 26 17:16:27 2012 -0700

    Merge pull request #1 from gazpachoking/clean

    Add some more tests

commit 3498f34
Author: Chase Sterling <[email protected]>
Date:   Wed Sep 26 20:02:58 2012 -0400

    Fix an improper additionalItems test.

commit 26a3873
Merge: baf0502 3649b58
Author: Chase Sterling <[email protected]>
Date:   Tue Sep 25 16:48:39 2012 -0400

    Merge branch 'master' into clean

commit baf0502
Author: Chase Sterling <[email protected]>
Date:   Tue Sep 25 16:48:30 2012 -0400

    Added items tests.

commit cb6dfb2
Author: Chase Sterling <[email protected]>
Date:   Tue Sep 25 16:11:21 2012 -0400

    Add cleaned up additionalItems

commit 3649b58
Author: Julian Berman <[email protected]>
Date:   Tue Sep 25 15:28:13 2012 -0400

    Would be nice if I'd have properly run lint.

commit e6a270f
Author: Julian Berman <[email protected]>
Date:   Tue Sep 25 15:00:57 2012 -0400

    Properties.

commit 2192a63
Author: Julian Berman <[email protected]>
Date:   Tue Sep 25 12:40:31 2012 -0400

    Clarify test reqs.

commit ff746d0
Author: Julian Berman <[email protected]>
Date:   Tue Sep 25 12:34:22 2012 -0400

    Add MIT License.

commit 9271ce5
Author: Julian Berman <[email protected]>
Date:   Tue Sep 25 12:29:00 2012 -0400

    A basic readme.

commit 530a0f3
Author: Julian Berman <[email protected]>
Date:   Tue Sep 25 12:12:43 2012 -0400

    Finish type tests.

commit 4f9cd46
Author: Julian Berman <[email protected]>
Date:   Tue Sep 25 11:49:34 2012 -0400

    Initial commit. Basic type tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Some new desired functionality
Projects
None yet
Development

No branches or pull requests

1 participant