Add support for caching

[kul]

Support for caching would be an awesome addition. Something along the lines of nginx

location ~* .(jpg|png|gif|jpeg|css|js|mp3|wav|swf|mov|doc|pdf|xls|ppt|docx|pptx|xlsx)$ {
proxy_buffering on;
proxy_cache_valid 200 120m;
expires 864000;
}

[Marak]

I think that this functionality would be best suited in another library that uses http-proxy as a dependency.

[vvo]

This should be easy to do as a middleware or using a connect middleware like https://github.com/tdebarochez/connect-cache

?

[kul]

Awesome suggestion @vvo

Thanks! i will try this tomorrow.

[kul]

i have given up on this,

var fs = require('fs'),
httpproxy = require('http-proxy'),
connect_cache = require('connect-cache');

var rules = { rules:
    [
        { regex: /.*.js/,ttl:60000  }
    ]
    ,loopback:'localhost:8080'
};

httpproxy.createServer(
    connect_cache(rules),
    8080,'localhost',options).listen(8181);

The request gets cached and everything works fine but once i do ctrl f5 a few time the connect-cache blows out. i have to say its a sad cache.

[vvo]

blows out ?

ctrl+f5 will send out 'cache-control:no-cache' headers which means 'refresh the cache' so there's must be something in the connect cache code to handle this.

Im sure it will work, debug, read the connect cache code.

good luck

[kul]

i think i got it..https://github.com/kul/connect-cache/commit/7b4b7a50536b03de7a11b6aa0c2c78eb3f42f24a

[deitch]

@kul how did you get this to work? I can see how using the connect-cache middleware allowed the cache to handle it if it is already stored. But if it is not (cache miss), then it just passes through to the proxy, so the results of the proxy do not get cached. How did you ever populate the cache?

[kul]

oh i am sorry i deleted the fork by mistake but the original repo had merged the pull request tdebarochez/connect-cache@05bfdc6.
So the idea is to treat it as cache miss which is just send to the server and cached for subsequent requests.

[deitch]

I get that it is a cache miss; what I didn't get is how you got the data in the cache in the first place.

Once you have a cache miss, and you pass it on to http-proxy, how did you intercept the response to add it to the cache?

[kul]

I am sorry if i am missing something here, did you try reading the node http proxy's and connect-cache's home page ?
https://github.com/nodejitsu/node-http-proxy#middleware
https://github.com/tdebarochez/connect-cache#how-it-works
Basically node-http-proxy provides middleware support which are like plugins in node.js world. Caching is taken care by connect-cache from that point onwards.

I may be a little rusty here, long time since i used node-http-proxy. Apologies.
PS: nginx release websocket support.

👾

[philjackson]

@deitch Did you happen to work out how to intercept a response?

[deitch]

Yeah, I used http-proxy, but I wrapped the response. In other words, I have middleware like this:

req.use(myMiddleware);

where myMiddleware is like:

httpProxy = require('http-proxy'), proxy = new httpProxy.HttpProxy(config);
module.exports = function(req,res,next) {
  if (isInCache) {
    // send response from cache
  } else {
    wrapResponse(res);
    proxy.proxyRequest(req,res,httpProxy.buffer(req));
  }
}

wrapResponse wraps every single send* and other relevant calls, and caches everything with a particular key

Conceptually simple, in practice was a beast, but works nicely.

[philjackson]

Thanks, @deitch.