Skip to content
This repository was archived by the owner on Apr 14, 2022. It is now read-only.

PCRE matching of string fields #74

Merged
merged 3 commits into from
Mar 13, 2018
Merged

PCRE matching of string fields #74

merged 3 commits into from
Mar 13, 2018

Conversation

Totktonada
Copy link
Member

@Totktonada Totktonada commented Mar 10, 2018

Related to #73.

@Totktonada Totktonada added enhancement New feature or request prio1 labels Mar 10, 2018
@Totktonada Totktonada force-pushed the gh-73-support-pcre branch 2 times, most recently from 5951f73 to df60de0 Compare March 12, 2018 11:05
@Totktonada Totktonada self-assigned this Mar 13, 2018
if obj[field_name] == nil then
return false
end
assert(rex ~= nil, 'we should not pass over :compile() ' ..
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment or assert msg?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Propose one better or I’ll leave it as is.

@@ -696,6 +702,28 @@ local function validate_collections(collections, schemas)
end
end

--- XXX
local function match_using_re(obj, pcre)
if pcre == nil then return true end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this situation normal?
I suppose assert is more suitable here and this function is called only if necessary. But it is up to you...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pcre is like filter. You maybe confuse it with rex — module variable.

@@ -741,9 +769,11 @@ local function process_tuple(state, tuple, opts)
qstats.fetched_object_cnt, fetched_object_cnt_max))
assert(qcontext.deadline_clock > clock.monotonic64(),
'query execution timeout exceeded, use `timeout_ms` to increase it')
local collection_name = opts.collection_name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not like creating variables for one use.

Copy link
Member Author

@Totktonada Totktonada Mar 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not like unpacking some opts, but don’t unpacking others. Debatable and non-important. Going to leave as is.

@@ -696,6 +702,28 @@ local function validate_collections(collections, schemas)
end
end

--- XXX
local function match_using_re(obj, pcre)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer regexp name over pcre.
Pcre dis

  1. can be confusing for newcomers
  2. emphasizes implementation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I don’t know a developer that don’t know PCRE term.
  2. It holds a standard, not an implementation. Consider comment with link to the alternative lua PCRE implementation.

I’m tentative about your proposal. Maybe.

@@ -828,6 +859,8 @@ local function select_internal(self, collection_name, from, filter, args, extra)
-- XXX: save type at parsing and check here
--assert(args.offset == nil or type(args.offset) == 'number',
-- 'args.offset must be a number of nil, got ' .. type(args.offset))
assert(args.pcre == nil or type(args.pcre) == 'table',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not like, that we loose structure of filters.
I suggest:

  1. Create single attribute filter
  2. filter is a table which contains
    • regexp
    • brute (i do not insist on the naming, I insist on sense)
    • index

I believe that this approach

  1. much more extensible
  2. would speed up future coding
  3. do not ban such names of fields like pcre

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not in the scope of this issue.

local collection = self.collections[collection_name]
local schema = self.schemas[collection.schema_name]
for _, field in ipairs(schema.fields) do
if field.name == field_name then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is about rested records?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

XXX mark above. Will file an issue re nested records and 1:1 connections (nested filters).

@@ -976,6 +1010,87 @@ local function init_qcontext(accessor, qcontext)
settings.timeout_ms * 1000 * 1000
end

--- XXX
local function get_primary_key_type(self, collection_name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename pk_avro_type

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Here all types are avro.
  2. I prefer to omit abbreviations where possible.

Debatable, non-important. Going to leave as is.

--- connection
local function get_pcre_argument_type(self, collection_name)
local collection = self.collections[collection_name]
assert(collection ~= nil, 'cannot found collection ' ..
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not write long messages to errors which should not occur. It just makes code-base huge and increases maintenance costs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cannot found collection %s

This error message is not long. You are about some other assert I guess. Propose your wording.

local pcre_field
if rex ~= nil then
local pcre_type = get_pcre_argument_type(self, collection_name)
pcre_field = {name = 'pcre', type = pcre_type}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have wrote above. I do not lile pcre name. Moreover in user-visible part.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m going to think about this bunch of problems in the scope of #39. Not now.

}
]]

utils.show_trace(function()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of this diff?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you read 215c1df comment?

@Totktonada Totktonada changed the title WIP: PoC of PCRE match of string fields PCRE matching of string fields Mar 13, 2018
The criteria of correctness is that the following queries should produce
the same results.

```
local gql = ...
local query = [[
    query foo_list($offset: foo_offset) {
        foo(offset: $offset) {
            ...
        }
    }
]]
local variables = {
    offset = {
        bar = ...,
        baz = ...,
    }
}
print(yaml.encode(gql:compile(query):execute(variables)))
```

```
local gql = ...
local query = [[
    query foo_list($bar: String, $baz: Long) {
        foo(offset: {bar: $bar, baz: $baz}) {
            ...
        }
    }
]]
local variables = {
    bar = ...,
    baz = ...,
}
print(yaml.encode(gql:compile(query):execute(variables)))
```
It is for make the function general and don't have fears of corner
cases. It will loop in case of x = { ofType = x }, but it is anyway
incorrect input.
@Totktonada Totktonada merged commit f555d70 into master Mar 13, 2018
@Totktonada Totktonada deleted the gh-73-support-pcre branch March 13, 2018 21:16
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request prio1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants