Skip to content
This repository was archived by the owner on Apr 14, 2022. It is now read-only.

[WIP] Sb/avro arr #48

Closed
wants to merge 29 commits into from
Closed
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
589778a
add comments on gql queries assert and compile functions
SudoBobo Feb 19, 2018
417c83e
do small refactor; add comments
SudoBobo Feb 20, 2018
24ec6c4
add support for array avro type (only with scalars); change args
SudoBobo Feb 21, 2018
9a43096
simplify conditional statements
SudoBobo Feb 22, 2018
6bd1ca2
add mised simple test for array support
SudoBobo Feb 22, 2018
31897d6
add simple test for array avro type usage
SudoBobo Feb 23, 2018
f0fb9f4
move nullable function to utils as it is will be
SudoBobo Feb 23, 2018
ac9f0ed
readability refactor; clean up comments;
SudoBobo Feb 23, 2018
8110208
Update tarantool_graphql.lua
SudoBobo Feb 23, 2018
f50f9fc
Merge remote-tracking branch 'origin/sb/avro_arr' into sb/avro_arr
SudoBobo Feb 24, 2018
043d6f9
small style fixes
SudoBobo Feb 25, 2018
7714393
readability and comments fixes
SudoBobo Feb 26, 2018
470e8e1
remove excess test
SudoBobo Feb 26, 2018
0160c69
remove redundant 'map' test cases
SudoBobo Feb 26, 2018
19b8d8b
fix small error; now general convert_scalar_type accepts nil as opt
SudoBobo Feb 26, 2018
1377c75
add test for avro-array usage (with space accessor);
SudoBobo Feb 26, 2018
5af08d8
move nullable back to tarantool_graphql from utils
SudoBobo Feb 26, 2018
be8457c
add support of avro map and test for simple case with scalar map values
SudoBobo Feb 27, 2018
fec3b31
add test for map with space accessor
SudoBobo Feb 27, 2018
0423ec4
add gql support of avro map/array with records as items/values
SudoBobo Feb 28, 2018
a936771
style/readability fixes after review
SudoBobo Feb 28, 2018
c8abf95
remove executable bit
SudoBobo Feb 28, 2018
951e7ea
add info about Map
SudoBobo Feb 28, 2018
91bd8bf
typo fix
SudoBobo Feb 28, 2018
148f69f
Merge branch 'master' into sb/avro_arr
SudoBobo Feb 28, 2018
d9a7623
remove whitespaces at the end of lines
SudoBobo Feb 28, 2018
31abefa
Merge remote-tracking branch 'origin/sb/avro_arr' into sb/avro_arr
SudoBobo Feb 28, 2018
1f2583a
remove all debug prints
SudoBobo Feb 28, 2018
e406a31
make Map and Object explanation more specific
SudoBobo Feb 28, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 109 additions & 21 deletions graphql/tarantool_graphql.lua
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,14 @@ local function avro_type(avro_schema)
return 'record*'
elseif utils.is_array(avro_schema) then
return 'union'
elseif avro_schema.type == 'array' then
return 'array'
elseif avro_schema.type == 'array*' then
return 'array*'
elseif avro_schema.type == 'map' then
return 'map'
elseif avro_schema.type == 'map*' then
return 'map*'
end
elseif type(avro_schema) == 'string' then
if avro_schema == 'int' then
Expand Down Expand Up @@ -79,7 +87,7 @@ local function convert_scalar_type(avro_schema, opts)
assert(type(opts) == 'table', 'opts must be nil or table, got ' ..
type(opts))
local raise = opts.raise or false
assert(type(opts.raise) == 'boolean', 'opts.raise must be boolean, got ' ..
assert(type(raise) == 'boolean', 'opts.raise must be boolean, got ' ..
type(opts.raise))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace opts.raise with raise here too for consistency.


local avro_t = avro_type(avro_schema)
Expand All @@ -96,15 +104,21 @@ local function convert_scalar_type(avro_schema, opts)
elseif avro_t == 'string*' then
return types.string
end

if raise then
error('unrecognized avro-schema scalar type: ' ..
json.encode(avro_schema))
json.encode(avro_schema))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. Cannot get you. Why do you remove the indent? Now it feels as broken.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping?

end

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't use python-style double empty lines (moreover, here they are within a function, so don't fit PEP8 too). Consider our lua style guidelines: https://tarantool.org/en/doc/1.7/dev_guide/lua_style_guide.html

return nil
end

--- Non-recursive version of the @{gql_type} function that returns
--- InputObject instead of Object.
--- An error will be raised if avro_schema type is 'record'
--- and its' fields are not scalar type because currently
--- triple nesting level (record with record as a field - ok,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently triple nesting level... what? Is not supported?

--- record with record wich has inside another level - not ok)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wich → which

local function gql_argument_type(state, avro_schema)
assert(type(state) == 'table',
'state must be a table, got ' .. type(state))
Expand All @@ -115,17 +129,20 @@ local function gql_argument_type(state, avro_schema)
assert(type(avro_schema.name) == 'string',
('avro_schema.name must be a string, got %s (avro_schema %s)')
:format(type(avro_schema.name), json.encode(avro_schema)))

assert(type(avro_schema.fields) == 'table',
('avro_schema.fields must be a table, got %s (avro_schema %s)')
:format(type(avro_schema.fields), json.encode(avro_schema)))

local fields = {}
for _, field in ipairs(fields) do
for _, field in ipairs(avro_schema.fields) do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad. Thanks for pointing it out! Filed #53 for that.

assert(type(field.name) == 'string',
('field.name must be a string, got %s (schema %s)')
:format(type(field.name), json.encode(field)))

local gql_field_type = convert_scalar_type(
field.type, {raise = true})

fields[field.name] = {
name = field.name,
kind = types.nonNull(gql_field_type),
Expand All @@ -149,44 +166,69 @@ local function gql_argument_type(state, avro_schema)
end
end


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra empty line.

-- Convert list of fields in the avro-schema format to list of GraphQL types
-- with intention to use it as GraphQL arguments later.
-- It uses the @{gql_argument_type} function to convert each field, then skips
-- fields of array and map types and gives the resulting list of converted fields.
---
--- @tparam table state for read state.accessor and previously filled
--- state.types
--- @tparam table fields fields part from an avro-schema
local function convert_record_fields_to_args(state, fields)
local args = {}
for _, field in ipairs(fields) do
assert(type(field.name) == 'string',
('field.name must be a string, got %s (schema %s)')
:format(type(field.name), json.encode(field)))

local gql_class = gql_argument_type(state, field.type)
args[field.name] = nullable(gql_class)

-- arrays (gql lists) and maps can't be arguments
-- so these graphql types are to be skipped
if nullable(gql_class) ~= 'List' and nullable(gql_class) ~= 'Map' then
args[field.name] = nullable(gql_class)
end
end
return args
end

--- Convert each field of an avro-schema to a graphql type and corresponding
--- argument for an upper graphql type.
--- Recursively convert each field of an avro-schema to a graphql type and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that such implementation details is not for the header string of the function. You can write details below, like so:

--- What the function does.
---
--- Any details here.
---
--- ...params and return values...
local function...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping?

--- corresponding argument for an upper graphql type.
---
--- @tparam table state for read state.accessor and previously filled
--- state.types
--- @tparam table fields fields part from an avro-schema
local function convert_record_fields(state, fields)
--- @tparam table opts include is_for_args flag to specify
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter is redundant. You always set it to true when use the object_args result. Just do true-case action always.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping?

--- case when the function is used to collect arguments
local function convert_record_fields(state, fields, opts)
local res = {}
local object_args = {}

for _, field in ipairs(fields) do
assert(type(field.name) == 'string',
('field.name must be a string, got %s (schema %s)')
:format(type(field.name), json.encode(field)))

res[field.name] = {
name = field.name,
kind = gql_type(state, field.type),
}
object_args[field.name] = nullable(res[field.name].kind)

-- arrays (gql lists) and maps can't be arguments
local avro_t = avro_type(field.type)
if avro_t ~= 'array' and avro_t ~= 'array*' and avro_t ~= 'map'
and avro_t ~= 'map*' then
object_args[field.name] = nullable(res[field.name].kind)
end
end
return res, object_args
end

--- The function recursively converts passed avro-schema to a graphql type.
--- The function converts passed avro-schema to a GraphQL type.
---
--- @tparam table state for read state.accessor and previously filled
--- state.types
--- state.types (state.types are gql types)
--- @tparam table avro_schema input avro-schema
--- @tparam[opt] table collection table with schema_name, connections fields
--- described a collection (e.g. tarantool's spaces)
Expand All @@ -197,6 +239,10 @@ end
--- automatically generate corresponding decucible fields.
--- 2. The collection name will be used as the resulting graphql type name
--- instead of the avro-schema name.
---
--- XXX As it is not clear now what to do with complex types inside arrays
---(just pass to results or allow to use filters), only scalar arrays
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgotten to add space after ---.

--- is allowed for now.
gql_type = function(state, avro_schema, collection, collection_name)
assert(type(state) == 'table',
'state must be a table, got ' .. type(state))
Expand All @@ -212,12 +258,14 @@ gql_type = function(state, avro_schema, collection, collection_name)
('collection and collection_name must be nils or ' ..
'non-nils simultaneously, got: %s and %s'):format(type(collection),
type(collection_name)))

local accessor = state.accessor
assert(accessor ~= nil, 'state.accessor must not be nil')
assert(accessor.select ~= nil, 'state.accessor.select must not be nil')
assert(accessor.list_args ~= nil,
'state.accessor.list_args must not be nil')

-- type of the top element in the avro-schema
local avro_t = avro_type(avro_schema)

if avro_t == 'record' or avro_t == 'record*' then
Expand All @@ -230,6 +278,7 @@ gql_type = function(state, avro_schema, collection, collection_name)

local fields, _ = convert_record_fields(state, avro_schema.fields)

-- if collection param is passed then go over all connections
for _, c in ipairs((collection or {}).connections or {}) do
assert(type(c.type) == 'string',
'connection.type must be a string, got ' .. type(c.type))
Expand All @@ -243,6 +292,7 @@ gql_type = function(state, avro_schema, collection, collection_name)
assert(type(c.parts) == 'table',
'connection.parts must be a string, got ' .. type(c.parts))

-- gql type of connection field
local destination_type =
state.types[c.destination_collection]
assert(destination_type ~= nil,
Expand All @@ -268,18 +318,21 @@ gql_type = function(state, avro_schema, collection, collection_name)
resolve = function(parent, args_instance, info)
local destination_args_names = {}
local destination_args_values = {}

for _, part in ipairs(c.parts) do
assert(type(part.source_field) == 'string',
'part.source_field must be a string, got ' ..
type(part.destination_field))
assert(type(part.destination_field) == 'string',
'part.destination_field must be a string, got ' ..
type(part.destination_field))

destination_args_names[#destination_args_names + 1] =
part.destination_field
destination_args_values[#destination_args_values + 1] =
parent[part.source_field]
end

local from = {
collection_name = collection_name,
connection_name = c.name,
Expand Down Expand Up @@ -318,6 +371,7 @@ gql_type = function(state, avro_schema, collection, collection_name)
}
end

-- create gql type
local res = types.object({
name = collection ~= nil and collection.name or avro_schema.name,
description = 'generated from avro-schema for ' ..
Expand All @@ -327,6 +381,19 @@ gql_type = function(state, avro_schema, collection, collection_name)
return avro_t == 'enum' and types.nonNull(res) or res
elseif avro_t == 'enum' then
error('enums not implemented yet') -- XXX
elseif avro_t == 'array' or avro_t == 'array*' then
assert(avro_schema.items ~= nil,
'items field must not be nil in array avro schema')
assert(type(avro_schema.items) == 'string',
'avro_schema.items must be a string, got '
.. type(avro_schema.item))

local gql_items_type = convert_scalar_type(avro_schema.items)

assert(gql_items_type, "only scalars are supported as array items for now "
.. avro_type(avro_schema.items) .. " is not a scalar")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is good to being consistent in using single / double quotes and in .. at the end / at the start of a line.

local gql_array = types.list(gql_items_type)
return avro_t == 'array' and types.nonNull(gql_array) or gql_array
else
local res = convert_scalar_type(avro_schema, {raise = false})
if res == nil then
Expand Down Expand Up @@ -356,32 +423,41 @@ local function parse_cfg(cfg)

local fields = {}

for name, collection in pairs(state.collections) do
collection.name = name
for collection_name, collection in pairs(state.collections) do
collection.name = collection_name
assert(collection.schema_name ~= nil,
'collection.schema_name must not be nil')

local schema = cfg.schemas[collection.schema_name]
assert(schema ~= nil, ('cfg.schemas[%s] must not be nil'):format(
tostring(collection.schema_name)))
assert(schema.name == nil or schema.name == collection.schema_name,
('top-level schema name does not match the name in ' ..
'the schema itself: "%s" vs "%s"'):format(collection.schema_name,
schema.name))
state.types[name] = gql_type(state, schema, collection, name)

-- recursively converts all avro types into gql types in the given schema
assert(schema.type == 'record',
'top-level schema must have record avro type, not'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Space after 'not'. I write 'got' in such cases, though.

.. schema.type)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to use tostring(schema.type) to correctly show assert in case of lack of the type field.

state.types[collection_name] = gql_type(state, schema, collection,
collection_name)

-- prepare arguments' types
local _, object_args = convert_record_fields(state,
schema.fields)
local list_args = convert_record_fields_to_args(
state, accessor:list_args(name))
state, accessor:list_args(collection_name))
local args = utils.merge_tables(object_args, list_args)
state.object_arguments[name] = object_args
state.list_arguments[name] = list_args
state.all_arguments[name] = args

state.object_arguments[collection_name] = object_args
state.list_arguments[collection_name] = list_args
state.all_arguments[collection_name] = args

-- create entry points from collection names
fields[name] = {
kind = types.nonNull(types.list(state.types[name])),
arguments = state.all_arguments[name],
fields[collection_name] = {
kind = types.nonNull(types.list(state.types[collection_name])),
arguments = state.all_arguments[collection_name],
resolve = function(rootValue, args_instance, info)
local object_args_instance = {} -- passed to 'filter'
local list_args_instance = {} -- passed to 'args'
Expand All @@ -397,7 +473,7 @@ local function parse_cfg(cfg)
end
end
local from = nil
return accessor:select(rootValue, name, from,
return accessor:select(rootValue, collection_name, from,
object_args_instance, list_args_instance)
end,
}
Expand All @@ -414,6 +490,9 @@ local function parse_cfg(cfg)
return state
end

--- The function checks that one and only one GraphQL operation
--- (query/mutation/subscription) is defined in the AST and it's type
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and it's type is 'query' as mutations and subscriptions are not supported yet

Where do you see this check?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assert(ast.definitions[1].operation == 'query',
func_name .. ': expected a query operation')

and it actually falls if we try to make something like:

mutation userFavs($user_id: String) {
        user_collection(user_id: $user_id) {
            user_id
            favorite_food
        }
    }

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ouch, sorry.

--- is 'query' as mutations and subscriptions are not supported yet
local function assert_gql_query_ast(func_name, ast)
assert(#ast.definitions == 1,
func_name .. ': expected an one query')
Expand All @@ -425,6 +504,8 @@ local function assert_gql_query_ast(func_name, ast)
type(operation_name))
end

--- The function just makes some reasonable assertions on input
--- and then call graphql-lua execute.
local function gql_execute(qstate, variables)
assert(qstate.state)
local state = qstate.state
Expand All @@ -442,6 +523,12 @@ local function gql_execute(qstate, variables)
operation_name)
end

--- The function parses a query string, validate the resulting query
--- against the GraphQL schema and provides an object with the function to
--- execute the query with specific variables values
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIR, ldoc expects to having period at end of short description. By the way, I personally prefer to split description (or description paragraphs) and each of params with empty lines (only with ---).

--- @tparam table state current state of graphql, including
--- schemas, collections and accessor
--- @tparam string query raw query string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proposed to omit the word 'raw'.

local function gql_compile(state, query)
assert(type(state) == 'table' and type(query) == 'string',
'use :validate(...) instead of .validate(...)')
Expand All @@ -458,6 +545,7 @@ local function gql_compile(state, query)
ast = ast,
operation_name = operation_name,
}

local gql_query = setmetatable(qstate, {
__index = {
execute = gql_execute,
Expand Down
Empty file modified graphql/utils.lua
100644 → 100755
Empty file.
9 changes: 9 additions & 0 deletions test/local/avro_array.result
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
RESULT
---
user_collection:
- user_id: user_id_1
favorite_food:
- meat
- potato
...

Loading