Make _LxmlFrameParser more extensible

[cancan101]

Allow creating a _LiberalLxmlFrameParser (see conversation on: #4469 (comment))

For now I will factor out:

parser = HTMLParser(recover=False)

so that subclass can change the behavior.

Would there be an issue with adding _LiberalLxmlFrameParser to the Pandas codebase?

[cpcloud]

no issues... would be happy to have your contrib... i think you could have a parameter that controls this like strict=True

[cpcloud]

marking for 0.14

[cpcloud]

@cancan101 FYI i'm going to assign myself to this issue to follow it so that it doens't get lost in the mix

[cpcloud]

not sure if that's visible to non repo collabs ... just letting you know

[cancan101]

@cpcloud That's fine. Yes it does show up that it was assigned to you.

[cancan101]

@cpcloud That reason to make another class (as opposed to just a parameter) is the valid parsers are specified as classes:

_valid_parsers = {'lxml': _LxmlFrameParser, None: _LxmlFrameParser,
                  'html5lib': _BeautifulSoupHtml5LibFrameParser,
                  'bs4': _BeautifulSoupHtml5LibFrameParser}

I suppose I could use a closure / partial (http://docs.python.org/2/library/functools.html#functools.partial) but that seems a bit messy.

[cpcloud]

class is fine ... i was thinking about the user facing API

[cancan101]

ah. Actually I could create a method with signature func(io, match, attrs):

def _LiberalLxmlFrameParser(io, match, attrs):
   return _LxmlFrameParser(io, match, attrs, strict=False)

and then:

_valid_parsers = {'lxml': _LxmlFrameParser, None: _LxmlFrameParser,
                  'html5lib': _BeautifulSoupHtml5LibFrameParser,
                  'bs4': _BeautifulSoupHtml5LibFrameParser,
                  'lxml-liberal': _LiberalLxmlFrameParser,}

[cpcloud]

i would prefer if you didn't mix my class naming convention with function names

[jtratner]

Much simpler: 'lxml-liberal': lambda io, match, attrs: _LxmlFrameParser(io, match, attrs, strict=False)

[cancan101]

@jtratner I had initially justed used a method rather than a class (albeit it was not anonymous). That being said, having the class allows me to put notes / docs. See: cancan101@91e7e5c#diff-a768e0e688020658ae7fb0dc23609b78L576

[jtratner]

These are all internal classes, nobody will see the docstring or documentation. Moreover, you can really just summarize it as:

lxml-liberal - uses lxml parser but allows errors to pass silently and then returns what it can from the parsed tables that lxml is able to find.

Also, I just learned you can set kewyord argument defaults on lambdas, so can do:

'lxml-liberal': lambda io, match, attrs, strict=False, **kwargs: _LxmlFrameParser(io, match, attrs, strict=strict, **kwargs)

[jtratner]

heck, you could even do this:

'lxml-liberal': partial(_LxmlFrameParser, strict=True)

[cpcloud]

i think @cancan101 was going to do that originally...

[cpcloud]

that lambda expression is a bit large ... :)

[cpcloud]

that technique is useful for passing closures around too

[cancan101]

Yep! #5130 (comment)

[cancan101]

Ok how about the partial approach and then a note under flavor? Should I copy what I have now for notes or should I strip it down? Most of what I have came directly from lxml docs.

[cpcloud]

Closing ... getting a bit stale ... and again should be for maybe a more in-depth html-pandas-parsing library.