Skip to content

fix: parsing JsonOperator #913

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 17, 2023
Merged

fix: parsing JsonOperator #913

merged 2 commits into from
Jul 17, 2023

Conversation

izveigor
Copy link
Contributor

@izveigor izveigor commented Jul 6, 2023

Follow on to apache/datafusion#6885.
The main problem is the following: input string starting with characters like @, # in many dialects can have both identifiers and operators.
Because of this, there is ambiguity in grammar.

@izveigor izveigor marked this pull request as draft July 6, 2023 19:42
@izveigor izveigor changed the title fix: parsing AtArrow fix: parsing JsonOperator Jul 10, 2023
@coveralls
Copy link

Pull Request Test Coverage Report for Build 5507745151

  • 58 of 70 (82.86%) changed or added relevant lines in 2 files are covered.
  • 3 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.03%) to 86.927%

Changes Missing Coverage Covered Lines Changed/Added Lines %
tests/sqlparser_common.rs 20 21 95.24%
src/tokenizer.rs 38 49 77.55%
Files with Coverage Reduction New Missed Lines %
src/tokenizer.rs 3 88.1%
Totals Coverage Status
Change from base Build 5476034295: -0.03%
Covered Lines: 15194
Relevant Lines: 17479

💛 - Coveralls

@izveigor izveigor marked this pull request as ready for review July 10, 2023 11:58
@izveigor
Copy link
Contributor Author

cc @alamb

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM @izveigor . Thank you very much for the contribution and I apologize for the delay in review

I have some suggested style changes in 38f1864 that I have made in a follow on PR #923


#[test]
fn parse_json_ops_without_colon() {
use self::JsonOperator;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified this test covers the changes in this PR by running this test without the code in this PR and it failed (as expected):

---- parse_json_ops_without_colon stdout ----
thread 'parse_json_ops_without_colon' panicked at 'assertion failed: `(left == right)`
  left: `Ok([Query(Query { with: None, body: Select(Select { distinct: None, top: None, projection: [UnnamedExpr(JsonAccess { left: Identifier(Ident { value: "a", quote_style: None }), operator: HashArrow, right: Identifier(Ident { value: "b", quote_style: None }) })], into: None, from: [], lateral_views: [], selection: None, group_by: [], cluster_by: [], distribute_by: [], sort_by: [], having: None, named_window: [], qualify: None }), order_by: [], limit: None, offset: None, fetch: None, locks: [] })])`,
 right: `Err(ParserError("Expected end of statement, found: >"))`: Parse results with PostgreSqlDialect are different from GenericDialect', src/test_utils.rs:60:21
stack backtrace:

@alamb
Copy link
Contributor

alamb commented Jul 20, 2023

This PR introduced a regression - #926

Here is a PR to fix the regression: #927

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants