Performance issues

[KimSchneider]

Hi,
about this one, I am not really sure if it is an node.js or node-http-proxy issue, but I bet you can help me anyways :)

One of our node-http-proxy servers runs on node 0.4.10 and an quite old node-http-proxy version (before node 0.6.*).
That server manages concurrently 1500+ http long polling requests and about 100 casual http requests per second. No problems, but CPU usage is kind of high.

We wanted to update to node 0.8.* and the current node-http-proxy version to boost performance, but our tests failed.
The performance drops down to 100-400 concurrent long polling requests and about 10 http requests per second. So the proxy does not seem to be able to handle more than 400 requests at one time (just rough numbers because the casual requests and the long polling requests fight each other in numbers ;))

We did set this

var httpProxy = require('http-proxy');
httpProxy.setMaxSockets(5000);

and the ulimit is set to 65535.

Any hint why the performance decreased so drastically?

Thanks, Kim

[mmalecki]

Hey Kim,

this is very interesting. Our proxies currently run node v0.8.14 and node-http-proxy v0.8.4 and we're seeing no performance problems. We confirmed that upgrade from node v0.6 increased proxy performance.

What's the exact node-http-proxy and node version you were using? Can you try profiling it? What operating system is that? Did you do any changes to backend service during the upgrade (as in, did you upgrade node on backend too)?

I'd be very happy to help with fixing it.

[KimSchneider]

Hey Maciej,

thanks for your reply!

We are having this issue since the 0.6.* release of node and the matching node-http-proxy release. I can not profile this error right now, because I don’t have a test environment that is big enough. I have to test this on the live system, so I would have to schedule a maintenance for this.

The servers behind the old proxy are exactly the same as behind the new proxy. We have a firewall that forwards all request to the proxy, we just changed the firewall to forward to the new proxy. So the only component that was updated in that test was the proxy.
The backend servers are all running on node 0.8.* since a couple of weeks without any problems.
All servers are running Ubuntu 12.04.1 LTS (GNU/Linux 3.2.0-32-generic x86_64)
The proxy is running node v0.8.14 and node-http-proxy 0.8.4

By the way, all incoming requests to the proxy are SSL!

Thanks for your help!

Mit freundlichen Grüßen

Herr Kim Schneider
Fachinformatiker
Fachrichtung Anwendungsentwicklung

Krämer IT Solutions GmbH
Tel.: 0 68 81 / 9 36 29 - 76
Fax: 0 68 81 / 9 36 29 - 99

From: Maciej Małecki [mailto:[email protected]]
Sent: Mittwoch, 14. November 2012 23:52
To: nodejitsu/node-http-proxy
Cc: Kim Schneider - Krämer IT Solutions
Subject: Re: [node-http-proxy] Performance issues (#334)

Hey Kim,

this is very interesting. Our proxies currently run node v0.8.14 and node-http-proxy v0.8.4 and we're seeing no performance problems. We confirmed that upgrade from node v0.6 increased proxy performance.

What's the exact node-http-proxy and node version you were using? Can you try profiling it? What operating system is that? Did you do any changes to backend service during the upgrade (as in, did you upgrade node on backend too)?

I'd be very happy to help with fixing it.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/334#issuecomment-10389586.

[abarre]

hi @KimSchneider,

Finally, did you fix the performance problem when the requests are in SSL with node 0.8?

What was the cause of the problem?

[KimSchneider]

No, we didn't. We will move to nginx and replace the node-http-proxy.

Our backend servers that are running the current node version without SSL have no problems. So this is related to nodes' SSL or node-http-proxy. But nginx seems to be the better solution anyway.

[indexzero]

Meh. If you're into that sort of masochism go for it. This is an issue related to SSL performance in node that @indutny is working hard on.

In the meantime you can just put stud in front of node-http-proxy. But I'm pretty sure @KimSchneider is not interested in our advice as he has not provided any demostrable data for us to help debug the problem with besides node versions.

[KimSchneider]

I can not give you any sample data, because I got none. The only data I've got is real time data that our customers produce and I am not allowed to post that.

Just try to open about 2000 long polling connections and try to blast 100 request/second through the proxy. Post data can be any JSON of about 200 chars, results are always { success : true }.

I'll take a look at stud, thanks.

[abarre]

@indexzero, do you have the reference to the issue related to the SSL performance in node?

[Rush]

My setup: node_modules/http-proxy/bin/node-http-proxy --config config-original.json --port 8081

> cat config-original.json
{  "router": { "test.pl": "127.0.0.1:8082" } }

Benchmark command: ab2 -c 100 -n 10000 http://test.pl:8081/de629fb8-ff7f-4920-ab29-6a0f2f4176bf
Command comes from package apache2-utils

All node versions installed via nvn, for example: nvm install 0.6

Summary (3 runs each):

• Node 0.6.21 - Requests per second: 737.89 [#/sec] (mean) 775.86 [#/sec] (mean) 799.24 [#/sec] (mean)
• Node 0.8.23 - Requests per second: 623.13 [#/sec] (mean) 583.87 [#/sec] (mean) 780.87 [#/sec] (mean)
• Node 0.10.18 - Requests per second: 535.42 [#/sec] (mean) 633.88 [#/sec] (mean) 455.18 [#/sec] (mean)
• Node 0.11.7 - Requests per second: 683.65 [#/sec] (mean) 611.34 [#/sec] (mean) 434.85 [#/sec] (mean)
• No proxy - Requests per second: 1269.88 [#/sec] (mean) 1397.67 [#/sec] (mean) 1148.50 [#/sec] (mean)

In general what I observe: Node 0.6 seems to be consistently faster Very rarely 0.8 seems to handle over 700 req/s. Node 0.10 on the other is quite consistent to never achieve > 700. I think something about 0.6 is better for node-http-proxy. Any idea how to speed things up?

[jcrugzz]

Closing due to old age.