(Squid) proxy and p2

If you happen to run a (local) proxy underneath you continuous build server, you might be a little bit surprised to learn that caching does show its full potential. E.g. if you look at your squid access log files they will look similar to:

214 188.40.71.70 TCP_REFRESH_UNMODIFIED/200 44988 GET http://download.eclipse.org/releases/kepler/201305310900/plugins/org.eclipse.equinox.common_3.6.200.v20130402-1505.jar.pack.gz - DIRECT/198.41.30.199 application/x-gzip 1370018697.496
222 188.40.71.70 TCP_REFRESH_UNMODIFIED/200 50531 GET http://download.eclipse.org/releases/kepler/201305310900/plugins/org.eclipse.equinox.preferences_3.5.100.v20130422-1538.jar.pack.gz - DIRECT/198.41.30.199 application/x-gzip 1370018697.513

This essentially says that you don't hit the local cache, but want the content reloaded. What you normally want to see though, are cache hits manifesting in TCP_MEM_HIT statements.

0 188.40.71.70 TCP_MEM_HIT/200 77323 GET http://download.eclipse.org/releases/kepler/201305310900/plugins/org.objectweb.asm_3.3.1.v201105211655.jar.pack.gz - NONE/- application/x-gzip 1370018314.277      
2 188.40.71.70 TCP_MEM_HIT/200 473229 GET http://download.eclipse.org/releases/kepler/201305310900/plugins/org.objectweb.asm.source_3.3.1.v201105211655.jar.pack.gz - NONE/- application/x-gzip 1370018314.406

The underlying problem/fix has not been fully diagnosed yet, but configuring squid to ignore reload requests for certain file types fixes the symptoms (an OSGi bundle carries its version number in its name, thus caching this type of files should not result in reading stale content). Just add the following refresh_pattern to the squid proxy that is supposed to speed up your builds.

/etc/squid/squid.conf
refresh_pattern -i _[0-9v.-]+\.jar(\.pack\.gz)?$ 480 20% 4320 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-reload

Below you see the cache stats for squid and all is green. :)

squid_requests-day


Want to comment? Send me an email to blog-comments-2018 at lemmster d.t de and I'll paste it here (I won't publish your address). Why don't you use an external comment service like disqus, you ask? Well, I like to keep this site under my control, comments included. You can use markdown to format your comment.