Skip to content

Introduce failFast option #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Apr 18, 2020
1 change: 1 addition & 0 deletions docs/options.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ $RefParser.dereference("my-schema.yaml", {
withCredentials: true, // Include auth credentials when resolving HTTP references
}
},
failFast: true, // Abort upon first exception
dereference: {
circular: false // Don't allow circular $refs
}
Expand Down
8 changes: 8 additions & 0 deletions docs/ref-parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ This is the default export of JSON Schema $Ref Parser. You can creates instance
##### Properties
- [`schema`](#schema)
- [`$refs`](#refs)
- [`errors`](#errors)

##### Methods
- [`dereference()`](#dereferenceschema-options-callback)
Expand Down Expand Up @@ -42,6 +43,13 @@ await parser.dereference("my-schema.json");
parser.$refs.paths(); // => ["my-schema.json"]
```

### `errors`
The `errors` property contains all list of errors that occurred during the bundling/resolving/dereferencing process.
All errors share error properties:
- path - json path to the document property
- message
- source - the uri of document where the faulty document was referenced


### `dereference(schema, [options], [callback])`

Expand Down
6 changes: 5 additions & 1 deletion lib/bundle.js
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,11 @@ function crawl (parent, key, path, pathFromRoot, indirections, inventory, $refs,
function inventory$Ref ($refParent, $refKey, path, pathFromRoot, indirections, inventory, $refs, options) {
let $ref = $refKey === null ? $refParent : $refParent[$refKey];
let $refPath = url.resolve(path, $ref.$ref);
let pointer = $refs._resolve($refPath, options);
let pointer = $refs._resolve($refPath, pathFromRoot, options);
if (pointer === null) {
return;
}

let depth = Pointer.parse(pathFromRoot).length;
let file = url.stripHash(pointer.path);
let hash = url.getHash(pointer.path);
Expand Down
9 changes: 8 additions & 1 deletion lib/dereference.js
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,14 @@ function dereference$Ref ($ref, path, pathFromRoot, parents, $refs, options) {
// console.log('Dereferencing $ref pointer "%s" at %s', $ref.$ref, path);

let $refPath = url.resolve(path, $ref.$ref);
let pointer = $refs._resolve($refPath, options);
let pointer = $refs._resolve($refPath, pathFromRoot, options);

if (pointer === null) {
return {
circular: false,
value: null,
};
}

// Check for circular references
let directCircular = pointer.circular;
Expand Down
48 changes: 48 additions & 0 deletions lib/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,13 @@ declare class $RefParser {
*/
$refs: $RefParser.$Refs

/**
* List of all errors
*
* See https://github.com/APIDevTools/json-schema-ref-parser/blob/master/docs/ref-parser.md#errors
*/
errors: Array<$RefParser.JSONParserError | $RefParser.InvalidPointerError | $RefParser.ResolverError | $RefParser.ParserError | $RefParser.MissingPointerError | $RefParser.UnmatchedParserError | $RefParser.UnmatchedResolverError>;

/**
* Dereferences all `$ref` pointers in the JSON Schema, replacing each reference with its resolved value. This results in a schema object that does not contain any `$ref` pointers. Instead, it's a normal JavaScript object tree that can easily be crawled and used just like any other JavaScript object. This is great for programmatic usage, especially when using tools that don't understand JSON references.
*
Expand Down Expand Up @@ -210,6 +217,12 @@ declare namespace $RefParser {
[key: string]: Partial<ResolverOptions>
}

/**
* Determines how lenient the processing should be.
* If this option is enable, the processing will be performed in a bail mode - will abort upon the first exception.
*/
failFast?: boolean;

/**
* The `dereference` options control how JSON Schema `$Ref` Parser will dereference `$ref` pointers within the JSON schema.
*/
Expand Down Expand Up @@ -398,4 +411,39 @@ declare namespace $RefParser {
set($ref: string, value: JSONSchema4Type | JSONSchema6Type): void
}

export type JSONParserErrorType = "EUNKNOWN" | "EPARSER" | "EUNMATCHEDPARSER" | "ERESOLVER" | "EUNMATCHEDRESOLVER" | "EMISSINGPOINTER" | "EINVALIDPOINTER";

export class JSONParserError extends Error {
readonly name: string;
readonly message: string;
readonly path: Array<string | number>;
readonly source: string;
readonly code: JSONParserErrorType;
}

export class ParserError extends JSONParserError {
readonly name = "ParserError";
readonly code = "EPARSER";
}
export class UnmatchedParserError extends JSONParserError {
readonly name = "UnmatchedParserError";
readonly code ="EUNMATCHEDPARSER";
}
export class ResolverError extends JSONParserError {
readonly name = "ResolverError";
readonly code ="ERESOLVER";
readonly ioErrorCode?: string;
}
export class UnmatchedResolverError extends JSONParserError {
readonly name = "UnmatchedResolverError";
readonly code ="EUNMATCHEDRESOLVER";
}
export class MissingPointerError extends JSONParserError {
readonly name = "MissingPointerError";
readonly code ="EMISSINGPOINTER";
}
export class InvalidPointerError extends JSONParserError {
readonly name = "InvalidPointerError";
readonly code ="EINVALIDPOINTER";
}
}
51 changes: 45 additions & 6 deletions lib/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,19 @@ const resolveExternal = require("./resolve-external");
const bundle = require("./bundle");
const dereference = require("./dereference");
const url = require("./util/url");
const { JSONParserError, InvalidPointerError, MissingPointerError, ResolverError, ParserError, UnmatchedParserError, UnmatchedResolverError, isHandledError } = require("./util/errors");
const maybe = require("call-me-maybe");
const { ono } = require("@jsdevtools/ono");

module.exports = $RefParser;
module.exports.YAML = require("./util/yaml");
module.exports.JSONParserError = JSONParserError;
module.exports.InvalidPointerError = InvalidPointerError;
module.exports.MissingPointerError = MissingPointerError;
module.exports.ResolverError = ResolverError;
module.exports.ParserError = ParserError;
module.exports.UnmatchedParserError = UnmatchedParserError;
module.exports.UnmatchedResolverError = UnmatchedResolverError;

/**
* This class parses a JSON schema, builds a map of its JSON references and their resolved values,
Expand All @@ -38,6 +46,25 @@ function $RefParser () {
this.$refs = new $Refs();
}

/**
* List of all errors
* @type {Array<JSONParserError | ResolverError | ParserError | MissingPointerError | UnmatchedResolverError | UnmatchedResolverError>}
*/
Object.defineProperty($RefParser.prototype, "errors", {
get () {
const errors = [];

for (const $ref of Object.values(this.$refs._$refs)) {
if ($ref.errors) {
errors.push(...$ref.errors);
}
}

return errors;
},
enumerable: true,
});

/**
* Parses the given JSON schema.
* This method does not resolve any JSON references.
Expand Down Expand Up @@ -111,16 +138,28 @@ $RefParser.prototype.parse = async function (path, schema, options, callback) {
try {
let result = await promise;

if (!result || typeof result !== "object" || Buffer.isBuffer(result)) {
throw ono.syntax(`"${me.$refs._root$Ref.path || result}" is not a valid JSON Schema`);
}
else {
if (result !== null && typeof result === "object" && !Buffer.isBuffer(result)) {
me.schema = result;
return maybe(args.callback, Promise.resolve(me.schema));
}
else if (!args.options.failFast) {
me.schema = null; // it's already set to null at line 79, but let's set it again for the sake of readability
return maybe(args.callback, Promise.resolve(me.schema));
}
else {
throw ono.syntax(`"${me.$refs._root$Ref.path || result}" is not a valid JSON Schema`);
}
}
catch (e) {
return maybe(args.callback, Promise.reject(e));
catch (err) {
if (args.options.failFast || !isHandledError(err)) {
return maybe(args.callback, Promise.reject(err));
}

if (this.$refs._$refs[url.stripHash(args.path)]) {
this.$refs._$refs[url.stripHash(args.path)].addError(err);
}

return maybe(args.callback, Promise.resolve(null));
}
};

Expand Down
10 changes: 8 additions & 2 deletions lib/options.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ $RefParserOptions.defaults = {
* Determines how different types of files will be parsed.
*
* You can add additional parsers of your own, replace an existing one with
* your own implemenation, or disable any parser by setting it to false.
* your own implementation, or disable any parser by setting it to false.
*/
parse: {
json: jsonParser,
Expand All @@ -39,7 +39,7 @@ $RefParserOptions.defaults = {
* Determines how JSON References will be resolved.
*
* You can add additional resolvers of your own, replace an existing one with
* your own implemenation, or disable any resolver by setting it to false.
* your own implementation, or disable any resolver by setting it to false.
*/
resolve: {
file: fileResolver,
Expand All @@ -55,6 +55,12 @@ $RefParserOptions.defaults = {
external: true,
},

/**
* Determines how lenient the processing should be.
* If this option is enable, the processing will be performed in a bail mode - will abort upon the first exception.
*/
failFast: true,

/**
* Determines the types of JSON references that are allowed.
*/
Expand Down
64 changes: 41 additions & 23 deletions lib/parse.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
const { ono } = require("@jsdevtools/ono");
const url = require("./util/url");
const plugins = require("./util/plugins");
const { StoplightParserError, ResolverError, ParserError, UnmatchedParserError, UnmatchedResolverError, isHandledError } = require("./util/errors");

module.exports = parse;

Expand All @@ -17,21 +18,21 @@ module.exports = parse;
* The promise resolves with the parsed file contents, NOT the raw (Buffer) contents.
*/
async function parse (path, $refs, options) {
try {
// Remove the URL fragment, if any
path = url.stripHash(path);
// Remove the URL fragment, if any
path = url.stripHash(path);

// Add a new $Ref for this file, even though we don't have the value yet.
// This ensures that we don't simultaneously read & parse the same file multiple times
let $ref = $refs._add(path);
// Add a new $Ref for this file, even though we don't have the value yet.
// This ensures that we don't simultaneously read & parse the same file multiple times
let $ref = $refs._add(path);

// This "file object" will be passed to all resolvers and parsers.
let file = {
url: path,
extension: url.getExtension(path),
};
// This "file object" will be passed to all resolvers and parsers.
let file = {
url: path,
extension: url.getExtension(path),
};

// Read the file and then parse the data
// Read the file and then parse the data
try {
const resolver = await readFile(file, options, $refs);
$ref.pathType = resolver.plugin.name;
file.data = resolver.result;
Expand All @@ -41,8 +42,12 @@ async function parse (path, $refs, options) {

return parser.result;
}
catch (e) {
return Promise.reject(e);
catch (err) {
if (isHandledError(err)) {
$ref.value = err;
}

throw err;
}
}

Expand Down Expand Up @@ -71,13 +76,20 @@ function readFile (file, options, $refs) {
.then(resolve, onError);

function onError (err) {
if (!err && !options.failFast) {
// No resolver could be matched
reject(new UnmatchedResolverError(file.url));
}
else if (!err || !("error" in err)) {
// Throw a generic, friendly error.
reject(ono.syntax(`Unable to resolve $ref pointer "${file.url}"`));
}
// Throw the original error, if it's one of our own (user-friendly) errors.
// Otherwise, throw a generic, friendly error.
if (err && !(err instanceof SyntaxError)) {
reject(err);
else if (err.error instanceof ResolverError) {
reject(err.error);
}
else {
reject(ono.syntax(`Unable to resolve $ref pointer "${file.url}"`));
reject(new ResolverError(err, file.url));
}
}
}));
Expand Down Expand Up @@ -112,7 +124,7 @@ function parseFile (file, options, $refs) {
.then(onParsed, onError);

function onParsed (parser) {
if (!parser.plugin.allowEmpty && isEmpty(parser.result)) {
if ((!options.failFast || !parser.plugin.allowEmpty) && isEmpty(parser.result)) {
reject(ono.syntax(`Error parsing "${file.url}" as ${parser.plugin.name}. \nParsed value is empty`));
}
else {
Expand All @@ -121,13 +133,19 @@ function parseFile (file, options, $refs) {
}

function onError (err) {
if (err) {
err = err instanceof Error ? err : new Error(err);
reject(ono.syntax(err, `Error parsing ${file.url}`));
if (!err && !options.failFast) {
// No resolver could be matched
reject(new UnmatchedParserError(file.url));
}
else {
else if (!err || !("error" in err)) {
reject(ono.syntax(`Unable to parse ${file.url}`));
}
else if (err.error instanceof ParserError || err.error instanceof StoplightParserError) {
reject(err.error);
}
else {
reject(new ParserError(err.error.message, file.url));
}
}
}));
}
Expand Down
2 changes: 1 addition & 1 deletion lib/parsers/binary.js
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ module.exports = {
* @param {string} file.url - The full URL of the referenced file
* @param {string} file.extension - The lowercased file extension (e.g. ".txt", ".html", etc.)
* @param {*} file.data - The file contents. This will be whatever data type was returned by the resolver
* @returns {Promise<Buffer>}
* @returns {Buffer}
*/
parse (file) {
if (Buffer.isBuffer(file.data)) {
Expand Down
Loading