Strange thing while change ngx.arg[1] in body_filter_by_lua*

[waterdeng]

oprenresty version:1.15.8

I want to change file stream while downloading, but I encountered a strange thing
that once I touch ngx.arg[1] in body_filter_by_lua*, client will receive some little response first,
then wait untill nginx process process the whole file.

Client will receive the whole file, but client receives nothing for a quite long time.
Nginx worker process will occupy memory about the same size as the file.

So what's the point?

I test with the following lua code for simplify, same thing happened too.
File has size bigger than 1GB.

location / {
    header_filter_by_lua_block { 
        ngx.header.content_length = nil 
    }
    body_filter_by_lua_block {
        ngx.arg[1]=ngx.arg[1]
    }
}

I know when set ngx.header.content_length to nil,
http response will use chunked Transfer-Encoding,
and client will receive the file content via chunks.
I capture the http protocol with wireshark and
check the chunk response, client receive part of first chunk
and then wait untill the whole file porcessed done.

[zhuizhuhaomeng]

please check if it still exists in openresty 1.19.3

[waterdeng]

please check if it still exists in openresty 1.19.3

nothing changed.

[waterdeng]

If I jsut set ngx.header.content_length to nil in header_filter_by_lua_block and has no body_filter_by_lua_block,
chunk size of chunked Transfer-Encoding http response will be 512KB, the same as output buffer size in nginx conf.

Once touching ngx.arg[1] in body_filter_by_lua_block, chunk size will be 512K+1Bytes, quite weird.

[waterdeng]

Everthing works well when nginx output_buffers is small enough.
I set output_buffers to 16k and download of 2GB file is fine.
Bigger files's download is not tested for now.

[zhuizhuhaomeng]

body_filter_by_lua_block {
    ngx.arg[1]=ngx.arg[1]
}

this config will copy buffer from nginx buffer chain to a lua string, and then the assignment will copy the string buffer back to nginx. so there will so many different lua strings which will consume much of the memory.

there is not need to set ngx.arg[1]=ngx.arg[1] in body_filter_by_lua_block.

[waterdeng]

body_filter_by_lua_block {
    ngx.arg[1]=ngx.arg[1]
}

this config will copy buffer from nginx buffer chain to a lua string, and then the assignment will copy the string buffer back to nginx. so there will so many different lua strings which will consume much of the memory.

there is not need to set ngx.arg[1]=ngx.arg[1] in body_filter_by_lua_block.

Sorry for my expression. 'ngx.arg[1]=ngx.arg[1]' in body_filter_by_lua_block is just for test, it is much more complicated in fact.
But the problem exists both.

[zhuizhuhaomeng]

there is still something I can not explain exactly.
will post a message here if I figure it out.

[waterdeng]

I enable nginx debug log and find epoll_wait blocked about 150s.
I can't figure it out how it works before I know more code details.

[zhuizhuhaomeng]

#1909
@waterdeng here is the PR to fix this issue. It is still under review.

[waterdeng]

I tested with PR #1909 code, problem disappeared.

@zhuizhuhaomeng Thanks.