Test the ambiguous ampersand state

[zcorpan]

Follows whatwg/html#2731

---

I couldn't figure out how to run the tokenizer tests with html5lib. Just nosetests doesn't seem to run them?

[inikulin]

I believe it makes sense to use new error format for this

[zcorpan]

I took unknown-named-character-reference from the spec and otherwise made it consistent with surrounding tests. I figured we may want to sweep all tests to update to the new format. But happy to make the new tests added here use the new format.

What should the error lines say, exactly?

[zcorpan]

Sorry, that's what #92 does. So, I can rebase this on 92 I suppose.

[zcorpan]

Or vice versa; if this looks OK we can land this first and rebase 92 and convert these tests there. WDYT?

[inikulin]

@zcorpan Both works for me. Do whatever will take less efforts from you.

[zcorpan]

OK, I think I prefer landing this one first.

[gsnedders]

@zcorpan You need to update the submodule, make sure you have dependencies installed (pip install -r requirements-test.txt), and run pytest. For just the tokenizer tests, pytest html5lib/tests/testdata/tokenizer or whatever the path is. :)

[inikulin]

We don't report error in this case due to:

If the character reference was consumed as part of an attribute (return state is either attribute value (double-quoted) state, attribute value (single-quoted) state or attribute value (unquoted) state), and the last character matched is not a U+003B SEMICOLON character (;), and the next input character is either a U+003D EQUALS SIGN character (=) or an ASCII alphanumeric, then, for historical reasons, switch to the character reference end state.

https://html.spec.whatwg.org/multipage/parsing.html#named-character-reference-state

[zcorpan]

Hmmmmm. That should probably switch to the ambiguous ampersand state. https://html.spec.whatwg.org/#syntax-attribute-value says

Attribute values are a mixture of text and character references, except with the additional restriction that the text cannot contain an ambiguous ampersand.

[zcorpan]

whatwg/html#2776

[zcorpan]

Rebased and fixed the &noti; in attribute case to not be a parse error.

[inikulin]

It should be unknown-named-character-reference

[inikulin]

Same here

[zcorpan]

Fixed.