Handle escape, unicode, and hex in tokenize_escaped_single_quoted_string

[JasonLi-cn]

Related to apache/datafusion#9268

[coveralls]

Pull Request Test Coverage Report for Build 8098472754

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 151 of 158 (95.57%) changed or added relevant lines in 2 files are covered.
• 310 unchanged lines in 3 files lost coverage.
• Overall coverage increased (+0.09%) to 87.881%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
tests/sqlparser_postgres.rs	30	31	96.77%
src/tokenizer.rs	121	127	95.28%

Files with Coverage Reduction	New Missed Lines	%
tests/sqlparser_common.rs	42	97.1%
src/ast/query.rs	67	83.27%
src/parser/mod.rs	201	83.22%

Totals
Change from base Build 8096073891:	0.09%
Covered Lines:	20203
Relevant Lines:	22989

---

💛 - Coveralls

[alamb]

Thank you @JasonLi-cn -- sorry for the delay in review. I think this PR does more than refactor the function, so I changed the name of the PR to reflect what I think is different

I also changed the API a little more to avoid some unecessary pub (crate)

Any chance you can take a look before I merge it to make sure I got it right?

[JasonLi-cn]

Thank you @JasonLi-cn -- sorry for the delay in review. I think this PR does more than refactor the function, so I changed the name of the PR to reflect what I think is different

I also changed the API a little more to avoid some unecessary pub (crate)

Any chance you can take a look before I merge it to make sure I got it right?

Thank you for reviewing this PR, and I think your modification is necessary.

[alamb]

Thanks again @JasonLi-cn