absolute local URI gives RefResolutionError

[brikeats]

I have a schema file feature.schema with the following contents:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "title": "image",
    "type": "object",
    "required": ["type", "geometry", "properties"],
    "properties": {
        "type": { "enum": [ "Feature" ] },
        "geometry": {"$ref": "file:///Users/bkeating/temp_scripts/ds-vision-model/geojson_geometry.schema#"},
        "properties": {"type": "object"}
    }
}

The referenced schema file exists (and is in the same directory as feature.schema):

>>> os.path.exists('/Users/bkeating/temp_scripts/ds-vision-model/geojson_geometry.schema')
True

Yet, running validate gives RefResolutionError: No JSON object could be decoded.

[Julian]

Would need a full traceback at least.

But that error would likely be saying the file exists but isn't valid JSON.

[brikeats]

Sorry, full traceback:

---------------------------------------------------------------------------
RefResolutionError                        Traceback (most recent call last)
<ipython-input-1-4f65eb27a40a> in <module>()
     22 try:
     23 #     validate(json_obj, schema, resolver=resolver)
---> 24     validate(json_obj, schema)
     25     print '%s passed validation' % json_fn
     26 except ValidationError, e:

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in validate(instance, schema, cls, *args, **kwargs)
    539         cls = validator_for(schema)
    540     cls.check_schema(schema)
--> 541     cls(schema, *args, **kwargs).validate(instance)

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in validate(self, *args, **kwargs)
    127 
    128         def validate(self, *args, **kwargs):
--> 129             for error in self.iter_errors(*args, **kwargs):
    130                 raise error
    131 

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in iter_errors(self, instance, _schema)
    103 
    104                     errors = validator(self, v, instance, _schema) or ()
--> 105                     for error in errors:
    106                         # set details if not already set by the called fn
    107                         error._set(

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/_validators.pyc in properties_draft4(validator, properties, instance, schema)
    302                 subschema,
    303                 path=property,
--> 304                 schema_path=property,
    305             ):
    306                 yield error

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in descend(self, instance, schema, path, schema_path)
    119 
    120         def descend(self, instance, schema, path=None, schema_path=None):
--> 121             for error in self.iter_errors(instance, schema):
    122                 if path is not None:
    123                     error.path.appendleft(path)

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in iter_errors(self, instance, _schema)
    103 
    104                     errors = validator(self, v, instance, _schema) or ()
--> 105                     for error in errors:
    106                         # set details if not already set by the called fn
    107                         error._set(

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/_validators.pyc in ref(validator, ref, instance, schema)
    214 
    215         try:
--> 216             for error in validator.descend(instance, resolved):
    217                 yield error
    218         finally:

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in descend(self, instance, schema, path, schema_path)
    119 
    120         def descend(self, instance, schema, path=None, schema_path=None):
--> 121             for error in self.iter_errors(instance, schema):
    122                 if path is not None:
    123                     error.path.appendleft(path)

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in iter_errors(self, instance, _schema)
    103 
    104                     errors = validator(self, v, instance, _schema) or ()
--> 105                     for error in errors:
    106                         # set details if not already set by the called fn
    107                         error._set(

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/_validators.pyc in oneOf_draft4(validator, oneOf, instance, schema)
    339     all_errors = []
    340     for index, subschema in subschemas:
--> 341         errs = list(validator.descend(instance, subschema, schema_path=index))
    342         if not errs:
    343             first_valid = subschema

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in descend(self, instance, schema, path, schema_path)
    119 
    120         def descend(self, instance, schema, path=None, schema_path=None):
--> 121             for error in self.iter_errors(instance, schema):
    122                 if path is not None:
    123                     error.path.appendleft(path)

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in iter_errors(self, instance, _schema)
    103 
    104                     errors = validator(self, v, instance, _schema) or ()
--> 105                     for error in errors:
    106                         # set details if not already set by the called fn
    107                         error._set(

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/_validators.pyc in properties_draft4(validator, properties, instance, schema)
    302                 subschema,
    303                 path=property,
--> 304                 schema_path=property,
    305             ):
    306                 yield error

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in descend(self, instance, schema, path, schema_path)
    119 
    120         def descend(self, instance, schema, path=None, schema_path=None):
--> 121             for error in self.iter_errors(instance, schema):
    122                 if path is not None:
    123                     error.path.appendleft(path)

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in iter_errors(self, instance, _schema)
    103 
    104                     errors = validator(self, v, instance, _schema) or ()
--> 105                     for error in errors:
    106                         # set details if not already set by the called fn
    107                         error._set(

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/_validators.pyc in ref(validator, ref, instance, schema)
    210                 yield error
    211     else:
--> 212         scope, resolved = validator.resolver.resolve(ref)
    213         validator.resolver.push_scope(scope)
    214 

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in resolve(self, ref)
    373     def resolve(self, ref):
    374         url = self._urljoin_cache(self.resolution_scope, ref)
--> 375         return url, self._remote_cache(url)
    376 
    377     def resolve_from_url(self, url):

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/functools32/functools32.pyc in wrapper(*args, **kwds)
    398                     except KeyError:
    399                         pass
--> 400                 result = user_function(*args, **kwds)
    401                 with lock:
    402                     cache[key] = result     # record recent use of this key

/Users/bkeating/anaconda/envs/uptakecv/lib/python2.7/site-packages/jsonschema/validators.pyc in resolve_from_url(self, url)
    383                 document = self.resolve_remote(url)
    384             except Exception as exc:
--> 385                 raise RefResolutionError(exc)
    386 
    387         return self.resolve_fragment(document, fragment)

RefResolutionError: No JSON object could be decoded

The referenced schema (geojson_geometry.schema) is valid JSON -- I can load it with the json python package, and I've used it to validate a few test files. The issue only arises when I try to reference it in another schema.

In case it matters, I'm on OSX and using jsonschema version 2.5.0.

[Julian]

Still not enough to reproduce the issue here I'm afraid. Can you include a full example? The error message is definitely saying it found your file, but that it isn't valid JSON, so will need to see enough to be able to reproduce locally.

[brikeats]

Sorry, here is a repo with all the files. feature.schema has a reference to the geojson_geometry.schema. To reproduce on your machine, you'll have to change line 8 of feature.schema to point to the repo directory, and then run test_schema.py. On my machine, python test_schema.py prints "Geometry validation succeeded" followed by the "invalid JSON" error above.

geojson_geometry.schema seems okay -- the geometry object in feature.json validates correctly against geojson_geometry.schema. It's only when I reference it in another schema that I get the invalid JSON error.

[Julian]

Ah, cool! Thanks!

So, to be honest, $ref support is in a bit of an embarrassing state, so it's possible that there's still a bug here, but I am pretty sure from a first look that what's happening here is related to your id in the geojson_geometry.schema file -- it's pointing at a 404 URL on the json-schema.org website, and that's what's being requested in your $refs (and those pages do not return valid JSON, hence the errors above).

If you remove that, things work I think?

I'll have to dig in a bit to see whether there's a bug here, I'm in the process of revamping the way ref resolution works (unfortunately at the same time as trying to catch up on Draft 6, performance testing, and the rest of a somewhat busy life these days)

[brikeats]

"If you remove that, things work I think?" Yes, that's correct.

I'll leave up the repo for a while in case it's useful to you for debugging. Thanks for all your work maintaining this package!

[Julian]

:) thank you!

…

On Oct 3, 2017 10:29, "Brian Keating" ***@***.***> wrote: "If you remove that, things work I think?" Yes, that's correct. I'll leave up the repo for a while in case it's useful to you for debugging. Thanks for all your work maintaining this package! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#362 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAUIXjudRaQME4MysmFtN1s2SpMrWuVOks5sokS6gaJpZM4Pfl3o> .