change upstream ip

[itpp16]

As tried in a pull a way to change ip of a upstream server, this is the code:

static int
ngx_http_lua_upstream_set_peer_ip(lua_State * L)
{
ngx_str_t host;
ngx_http_upstream_server_t *server;
ngx_http_upstream_srv_conf_t *us;
ngx_http_upstream_rr_peer_t *peer;
ngx_http_upstream_resolved_t *rserver;

if (lua_gettop(L) != 4) {
    return luaL_error(L, "exactly 4 arguments expected");
}

host.data = (u_char *) luaL_checklstring(L, 1, &host.len);

us = ngx_http_lua_upstream_find_upstream(L, &host);
if (us == NULL) {
    lua_pushnil(L);
    lua_pushliteral(L, "upstream not found");
    return 2;
}

server = us->servers->elts;
rserver = us->servers->elts;

server[1].addrs->name.data = (u_char *) lua_tostring(L, 4);
rserver[1].host.data = (u_char *) lua_tostring(L, 4);

peer = ngx_http_lua_upstream_lookup_peer(L);

peer->name.data = (u_char *) lua_tostring(L, 4);
peer->server.data = (u_char *) lua_tostring(L, 4);

lua_pushliteral(L, "name");
lua_pushlstring(L, (char *) peer->name.data, peer->name.len);

return 1;

}

Config:
upstream myLoadBalancerS {
server 192.168.168.2:80 weight=1 fail_timeout=5;
server 192.168.168.26:80 weight=1 fail_timeout=5;
least_conn;
}

Lua code:
location = /luaupstreamip {
default_type text/plain;
content_by_lua '
local upstream = require "ngx.upstream"
ok, err = upstream.set_peer_ip("myLoadBalancerS", false, 1, "192.168.168.17:80")
ngx.print("changed ",ok," ",err,"\n")
';
}

Then following the basic examples to check on the current state;

curl http://127.0.0.1:80/luaupstreams
upstream myLoadBalancerS:
addr = 192.168.168.2:80, weight = 1, fail_timeout = 5, max_fails = 1
addr = 192.168.168.26:80, weight = 1, fail_timeout = 5, max_fails = 1

curl http://127.0.0.1:80/luaupstreamss
[{"current_weight":0,"effective_weight":1,"fail_timeout":5,"fails":0,"id":0,"max
_fails":1,"name":"192.168.168.2:80","server":"192.168.168.2:80","weight":1},{"cu
rrent_weight":0,"effective_weight":1,"fail_timeout":5,"fails":0,"id":1,"max_fail
s":1,"name":"192.168.168.26:80","server":"192.168.168.26:80","weight":1}]

Ok, so far everything looks ok, then I run my new code;

curl http://127.0.0.1:80/luaupstreamip

And check to see what happened;

curl http://127.0.0.1:80/luaupstreams
upstream myLoadBalancerS:
addr = 192.168.168.2:80, weight = 1, fail_timeout = 5, max_fails = 1
addr = 192.168.168.17:80, weight = 1, fail_timeout = 5, max_fails = 1

curl http://127.0.0.1:80/luaupstreamss
[{"current_weight":0,"effective_weight":1,"fail_timeout":5,"fails":0,"id":0,"max
_fails":1,"name":"192.168.168.2:80","server":"192.168.168.2:80","weight":1},{"cu
rrent_weight":0,"effective_weight":1,"fail_timeout":5,"fails":0,"id":1,"max_fail
s":1,"name":"192.168.168.17:80","server":"192.168.168.17:80","weight":1}]

Ok, IP changed, however if I run curl on the location block which does a proxy_pass to myLoadBalancerS, I still see it connecting to 192.168.168.2:80 and 192.168.168.26:80 while the second server (1) should be .17

Obviously I am missing something, it seems the runtime values of the servers do not live in:
ngx_http_upstream_server_t or ngx_http_upstream_srv_conf_t or ngx_http_upstream_rr_peer_t

Any ideas or sample code I can experiment with ?

[agentzh]

@itpp16 Thank you for looking into this. But I need to find some spare time to look into the technical details for you. Thanks for your patience.

[itpp16]

Jumping the gun a bit, maybe others want to experiment or hunt the issue, I've made a devtest version fully functional (apart from this change ip issue) at http://nginx-win.ecsds.eu/devtest/EBLB_upstream_dev1.zip

[jkeys089]

This would be a great feature to have. In the case when your upstream server IP's could change (e.g. AWS elastic loadbalancer) this feature along with https://github.com/openresty/lua-resty-dns would be extremely valuable. I believe there is only a commercial option to solve this now (i.e. nginx+) and I'd much rather have an open solution.

[itpp16]

All done, see https://groups.google.com/forum/#!topic/openresty-en/wt_9m7GvROg (see dev3 archive for source and GUI) and let us know if it works for you, I have pushed it into a production system today as a gunny pig :) all we need now is for agentzh to approve this version, or shoot it down with better coding :)

[ameir]

This is definitely a feature we'd appreciate on a project I'm working on. Tengine has solved this with http://tengine.taobao.org/document/http_upstream_dynamic.html which seems like a good alternative as well.

[itpp16]

Well it's all there including IWCP, doing it with DNS is nice but like I said before:
DNS can be too slow, not under your control, not allowing changes, remote DNS can go offline,
not fast enough when redirecting attacks, etc...

[agentzh]

Now it is recommended to use the balancer_by_lua directive of ngx_lua for this.