ngx_http_lua_module.so: undefined symbol: pcre_dfa_exec when dynamic module compiled

[centminmod]

tried latest compile with Nginx 1.9.12 and 0.10.2 and get the following error only when lua nginx is compiled as dynamic module, if non-dynamic it compiles fine

nginx -t
nginx: [emerg] dlopen() "/usr/local/nginx/modules/ngx_http_lua_module.so" failed (/usr/local/nginx/modules/ngx_http_lua_module.so: undefined symbol: pcre_dfa_exec) in /usr/local/nginx/conf/dynamic-modules.conf:8
nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed

/usr/local/nginx/conf/dynamic-modules.conf contents

load_module "modules/ngx_http_brotli_filter_module.so";
load_module "modules/ngx_http_brotli_static_module.so";
load_module "modules/ngx_http_image_filter_module.so";
load_module "modules/ngx_http_xslt_filter_module.so";
load_module "modules/ngx_http_headers_more_filter_module.so";
load_module "modules/ngx_http_set_misc_module.so";
load_module "modules/ngx_http_echo_module.so";
load_module "modules/ngx_http_lua_module.so";
load_module "modules/ngx_pagespeed.so";
load_module "modules/ngx_http_geoip_module.so";
load_module "modules/ngx_stream_module.so";

./configure --with-ld-opt="-ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib" --with-cc-opt="-m64 -mtune=native -mfpmath=sse -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2" --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-http_stub_status_module --with-http_secure_link_module --with-openssl-opt="enable-tlsext" --add-module=../nginx-module-vts --with-libatomic --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_gzip_static_module --add-dynamic-module=../ngx_brotli --add-dynamic-module=../ngx_pagespeed-release-1.10.33.6-beta --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_realip_module --add-module=../ngx-fancyindex-0.3.6 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0rc1 --add-dynamic-module=../set-misc-nginx-module-0.30 --add-dynamic-module=../echo-nginx-module-master --add-module=../redis2-nginx-module-0.12 --add-module=../ngx_http_redis-0.3.7 --add-dynamic-module=../lua-nginx-module-0.10.2 --add-module=../nginx_upstream_check_module-0.3.0 --add-module=../openresty-memc-nginx-module-4f6f78f --add-module=../srcache-nginx-module-0.30 --add-dynamic-module=../headers-more-nginx-module-master --with-http_xslt_module=dynamic --with-pcre=../pcre-8.38 --with-pcre-jit --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.0.2g
checking for OS
 + Linux 2.6.32-042stab112.15 x86_64
checking for C compiler ... found
 + using GNU C compiler
 + gcc version: 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC) 

ls -lah /usr/local/nginx/modules
total 23M
drwxr-xr-x  2 root root 4.0K Mar 20 15:53 .
drwxr-xr-x 10 root root 4.0K Mar 18 10:33 ..
-rwxr-xr-x  1 root root 210K Mar 20 15:53 ngx_http_brotli_filter_module.so
-rwxr-xr-x  1 root root 109K Mar 20 15:53 ngx_http_brotli_static_module.so
-rwxr-xr-x  1 root root 671K Mar 20 15:53 ngx_http_echo_module.so
-rwxr-xr-x  1 root root 126K Mar 20 15:53 ngx_http_geoip_module.so
-rwxr-xr-x  1 root root 286K Mar 20 15:53 ngx_http_headers_more_filter_module.so
-rwxr-xr-x  1 root root 146K Mar 20 15:53 ngx_http_image_filter_module.so
-rwxr-xr-x  1 root root 3.3M Mar 20 15:53 ngx_http_lua_module.so
-rwxr-xr-x  1 root root 770K Mar 20 15:53 ngx_http_set_misc_module.so
-rwxr-xr-x  1 root root 157K Mar 20 15:53 ngx_http_xslt_filter_module.so
-rwxr-xr-x  1 root root  17M Mar 20 15:53 ngx_pagespeed.so
-rwxr-xr-x  1 root root 574K Mar 20 15:53 ngx_stream_module.so

[agentzh]

@centminmod I guess you need to dynamically link against to PCRE in your nginx build because other dynamic modules like ngx_http_lua_module.so may depend on PCRE.

[centminmod]

thanks @agentzh looks like for now probably easiest to just compile as non-dynamic module

[portablesoft]

Try to complile lua-nginx-module as dynamic module under Nginx 1.11.4 and still get the following error:

Starting nginx: nginx: [emerg] dlopen() "/usr/local/nginx/modules/ngx_http_lua_module.so" failed (/usr/local/nginx/modules/ngx_http_lua_module.so: undefined symbol: pcre_dfa_exec)

and with the following configure:

./configure --prefix=/usr/local/nginx --user=www --group=www --with-http_v2_module --with-http_ssl_module --with-http_gzip_static_module --with-http_realip_module --with-openssl=../openssl/ --with-pcre=../pcre/ --with-pcre-jit --add-dynamic-module=../ngx_devel_kit/ --add-dynamic-module=../srcache-nginx-module/ --add-dynamic-module=../set-misc-nginx-module/ --add-dynamic-module=../lua-nginx-module/ --with-cc-opt="-I /opt/pcrejit/include" --with-ld-opt="-L /opt/pcrejit/lib -Wl,-rpath,/opt/pcrejit/lib"

Then, how to config the pcre for the dynamic lua-nginx-module correctly? Thanks.

[agentzh]

Try adding the --with-ld-opt="-E" to your ./configure command line.

[centminmod]

I managed to get lua nginx module to compile with dynamic module just by adding to --with-ld-opt just -lpcre as i have custom compiled pcre at /usr/local/lib

nginx -V
nginx version: nginx/1.11.4
built by gcc 4.9.2 20150212 (Red Hat 4.9.2-6) (GCC)
built with LibreSSL 2.4.3
TLS SNI support enabled
configure arguments: --with-ld-opt='-lrt -ljemalloc -lpcre -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -march=native -g -O3 -fstack-protector-strong --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-http_stub_status_module --with-http_secure_link_module --add-module=../nginx-module-vts --with-libatomic --with-http_gzip_static_module --add-dynamic-module=../ngx_brotli --add-dynamic-module=../ngx_pagespeed-release-1.11.33.4-beta --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-module=../set-misc-nginx-module-0.31 --add-module=../echo-nginx-module-0.60 --add-module=../redis2-nginx-module-0.13 --add-module=../ngx_http_redis-0.3.7 --add-dynamic-module=../lua-nginx-module-0.10.6 --add-module=../memc-nginx-module-0.17 --add-module=../srcache-nginx-module-0.31 --add-module=../headers-more-nginx-module-0.31 --with-pcre=../pcre-8.39 --with-pcre-jit --with-http_ssl_module --with-http_v2_module --with-openssl=../libressl-2.4.3

seems okay

nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

dynamic modules include file /usr/local/nginx/conf/dynamic-modules.conf

load_module "modules/ngx_http_brotli_filter_module.so";
load_module "modules/ngx_http_brotli_static_module.so";
load_module "modules/ngx_http_image_filter_module.so";
load_module "modules/ngx_http_lua_module.so";
load_module "modules/ngx_http_fancyindex_module.so";
load_module "modules/ngx_pagespeed.so";
load_module "modules/ngx_stream_module.so";

dynamic module directory at /usr/local/nginx/modules

total 44M
drwxr-xr-x.  2 root root 4.0K Oct  7 17:57 .
drwxr-xr-x. 10 root root 4.0K Sep 23 12:36 ..
-rwxr-xr-x   1 root root 119K Oct  7 17:57 ngx_http_brotli_filter_module.so
-rwxr-xr-x   1 root root 119K Oct  7 17:50 ngx_http_brotli_filter_module.so.old
-rwxr-xr-x   1 root root 106K Oct  7 17:57 ngx_http_brotli_static_module.so
-rwxr-xr-x   1 root root 106K Oct  7 17:50 ngx_http_brotli_static_module.so.old
-rwxr-xr-x   1 root root 150K Oct  7 17:57 ngx_http_fancyindex_module.so
-rwxr-xr-x   1 root root 150K Oct  7 17:50 ngx_http_fancyindex_module.so.old
-rwxr-xr-x   1 root root 149K Oct  7 17:57 ngx_http_image_filter_module.so
-rwxr-xr-x   1 root root 149K Oct  7 17:50 ngx_http_image_filter_module.so.old
-rwxr-xr-x   1 root root 3.5M Oct  7 17:57 ngx_http_lua_module.so
-rwxr-xr-x   1 root root 3.5M Oct  7 17:50 ngx_http_lua_module.so.old
-rwxr-xr-x   1 root root  17M Oct  7 17:57 ngx_pagespeed.so
-rwxr-xr-x   1 root root  17M Oct  7 17:50 ngx_pagespeed.so.old
-rwxr-xr-x   1 root root 1.1M Oct  7 17:57 ngx_stream_module.so
-rwxr-xr-x   1 root root 1.1M Oct  7 17:50 ngx_stream_module.so.old

[wangwei1237]

add --with-ld-opt='-lpcre -Wl' to the ./configure command.

nginx version: nginx/1.10.2
built by clang 8.0.0 (clang-800.0.38)
built with OpenSSL 1.0.2j 26 Sep 2016
TLS SNI support enabled
configure arguments:
--with-ld-opt='-lpcre -Wl'
--prefix=/Users/wangwei/local/nginx
--with-pcre=/Users/wangwei/Documents/Resources/CodeSource/pcre-8.39
--with-zlib=/Users/wangwei/Documents/Resources/CodeSource/zlib-1.2.8
--with-http_ssl_module
--with-openssl=/Users/wangwei/Documents/Resources/CodeSource/openssl-1.0.2j
--with-http_v2_module
--add-dynamic-module=/Users/wangwei/Documents/Resources/CodeSource/lua-nginx-module

and the nginx seems ok.
nginx: the configuration file /Users/wangwei/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /Users/wangwei/local/nginx/conf/nginx.conf test is successful

[benbro]

Is this fixed because the README still recommends using a dynamic module

[agentzh]

@benbro The document does not recommend building this module as a dynamic module. It simply states that this is possible and supported. If you statically link PCRE into your nginx, then you have to specify the ./configure option --with-ld-opt="-Wl,-E" to export those PCRE symbols for dynamically loaded nginx C modules to see them.

[agentzh]

@wangwei1237 Seems like you are linking two different PCRE libraries into the same nginx executable, one is statically linked via --with-pcre=PATH, and the other is dynamically linked via --with-ld-opt='-lpcre -Wl'. And you are asking for troubles with two PCREs in the same image.

[agentzh]

The recommended way is to always dynamically link against PCRE in your nginx instead of using the --with-pcre=PATH option to link statically against PCRE if any of your dynamically loaded nginx C modules do use PCRE, not just this ngx_http_lua_module BTW.

On the other hand, mixing statically linking and dynamic linking often requires explicit symbol exporting using linker options like -Wl,-E.

[agentzh]

To conclude, this is not really an issue in this module.

[playaround88]

@agentzh sorry, I am new, as u mentioned, how to dynamically link against PCRE in nginx，instead using the --with-pcre=PATH option?

[agentzh]

@playaround88 See how we do that in our packaging scripts here:

https://github.com/openresty/openresty-packaging

Or better, just use our prebuilt Linux packages through our yum/apt repositories:

https://openresty.org/en/linux-packages.html

[playaround88]

@agentzh thanks, but first links is 404.
I kown how to do! just install pcre to linux standard dynamic lib dir.

./configure --prefix=/usr/local/pcre-8.4 --libdir=/usr/local/lib/pcre --includedir=/usr/local/include/pcre
make
make install

and nginx lua work well!
Thank you anyway.

[agentzh]

@playaround88 Sorry, there was a typo there. Just updated my comment above.