Stefan Caunter

working example vcl for a system with multiple iis7 backends
if the webapp[s] run slowly at times you will need several backends to improve client’s chances of hitting a fast server first.
we always bail out to a long wait for an answer so we don’t show 503s.
the patient and impatient backends are the same webservers, we just poll impatiently and take out slow ones
confirm restarts are working by checking in varnishlog. make sure you have hosts file entries so the hostname entry resolves for the polling check.
client should either get a cache hit if we have the object, or a fast back end on requests that we don’t send directly to the patient director, and if it’s a specific url where we need to wait for a database or some query, we are happy to wait, by using return(lookup), so the object enters the cache for next time.
I have logic for asp sessions and facebook logic too (next time)

# patient backends, no polling, long timeout
backend search1 {
.host = “192.168.0.1”;
.port = “80”;
.first_byte_timeout = 60s;
.between_bytes_timeout = 10s;
}
backend search2 {
.host = “192.168.0.2”;
.port = “80”;
.first_byte_timeout = 90s;
.between_bytes_timeout = 90s;
}
backend my1 {
.host = “web1.example.com”;
.port = “80”;
.first_byte_timeout = 100ms;
.between_bytes_timeout = 500ms;
.probe = {
.url = “/checking”;
.timeout = 100 ms;
.interval = 5s;
.window = 5;
.threshold = 3;
}
}
backend my2 {
.host = “web2.example.com”;
.port = “80”;
.first_byte_timeout = 100ms;
.between_bytes_timeout = 500ms;
.probe = {
.url = “/checking”;
.timeout = 100 ms;
.interval = 5s;
.window = 5;
.threshold = 3;
}
}
# patient director, patience is a property of the backend definition
director patient random {
{ .backend = search1; .weight = 3; }
{ .backend = search2; .weight = 3; }
}
# impatient director, impatience is a property of the backend polling
# you need several backends for best effect, we only show two for example
director impatient random {
{ .backend = my1; .weight = 3; }
{ .backend = my2; .weight = 3; }
}

sub vcl_recv {
# omitting commonly available settings
if (req.restarts == 0) {
set req.backend = impatient;
} else if (req.restarts == 1) {
set req.backend = patient;
} else if (req.restarts == 2) {
set req.backend = patient;
} else if (req.restarts == 3) {
set req.backend = patient;
} else if (req.restarts == 4) {
set req.backend = search1;
}
if (req.url ~ “/slowthing”) {
set req.backend = patient;
return(lookup);
}
if (req.url ~ “/slowthing2”) {
set req.backend = patient;
return(lookup);
}
if (req.url ~ “/control-update”) {
set req.backend = impatient;
return(pipe);
}

}
sub vcl_fetch {
# keep trying if iis does not answer, use vcl_recv restart logic
if (beresp.status != 200 && beresp.status != 403 && beresp.status != 404 && beresp.status != 301 && beresp.status != 302) {
restart;
}
# set things here on objects we send back to clients
if (req.url ~ “^/$” && req.http.host ~ “www.example.com”){
set beresp.ttl = 300s;
set beresp.http.cache-control = “max-age = 300”;
set beresp.http.X-MyCacheReason = “MyHomePage”;
set beresp.grace = 1d;
return(deliver);
}
return(deliver);
}
sub vcl_error {
# avoid the dreaded 503, increase restart number as necessary
if (obj.status == 503 && req.restarts < 5) {
restart;
}
}

Dearth of knowledge here, thought I’d share my experiences.
Getting a busy, complex ASP.NET app to show snappy, fast page response to users is challenging, especially if potential bottlenecks cause intermittent slowness. Varnish is ideal for this, but requires careful adjustment and tuning for best effect. In particular, avoiding 503 errors caused by IIS hangs is most important. Delivering a page eventually is preferable to a quick timeout and a user confronted with the varnish 503 page. With determination, we can eliminate that possibility and speed things up nicely for IIS administrators.
First, varnish must health check the IIS backends. You need a version of varnish that will do this. Second, varnish must retry if the backend does not answer in a timely manner. Same need, post 2.04 version necessary. Third, varnish must wait in certain cases if we have to wait for the backend. Small bit of magic to do this.
I guess the overriding issue here is that ASP.NET apps, or any app for that matter will have problems, and our responsibility as sysadmins is to make them run as well as possible. Externally, we want to be able to send clients to a fast, healthy server, and we want to serve cached content to the extent possible within a demanding freshness use case.
Back end webservers should be checked with requests that time out quickly, and need to be set to quickly take a slow box out of the pool. Check a simple ASP page, not a flat file, and keep the timing reasonable for your app and scale needs. If you know your app lags at times (for whatever reason), you should be able to tune this check so that a server with a lot of requests in its w3wp.exe application pool queue will get marked sick until it clears its problem. Add these backends into your “impatient” director. Serve most requests with this director.
You cannot avoid a slow server altogether this way. IIS might be heading to a slow time but not be marked sick, and it will get requests that will sit there. We have to avoid the dreaded 503 error at all costs, and with a fast timeout you will get them unless you restart in vcl_error. Instead of giving up, varnish tries again. You’ll get a healthy server, and (almost) never show 503 to the client, especially once the varnish cache heats up.
My technique to deal with “have to wait” scenarios (login, POST, search) is to create a “patient” pool. Same servers, but long timeout on the first_bytes_check. Send these type of requests to this director. Instead of a 503, we patiently wait for a response. Result: fast response on most pages, and correct response when patience is a normal part of user expectations.

If varnishd is now started with a -S secret-file argument, user connections are authenticated.

This solves the wide openness of the management access point to varnish. Usual workaround is to simply ssh to a shell on a restricted machine, but if you are running varnish on a system where there is shell access, any user can telnet to the management port and, well, manage varnish (by design).

This solution prevents that situation for hosting providers, and should allow wider deployments on multi-use non-dedicated servers.

http://varnish.projects.linpro.no/wiki/CLI