If you happen to run a (local) proxy underneath you continuous build server, you might be a little bit surprised to learn that caching does show its full potential. E.g. if you look at your squid access log files they will look similar to:
214 188.40.71.70 TCP_REFRESH_UNMODIFIED/200 44988 GET http://download.eclipse.org/releases/kepler/201305310900/plugins/org.eclipse.equinox.common_3.6.200.v20130402-1505.jar.pack.gz - DIRECT/198.41.30.199 application/x-gzip 1370018697.496
222 188.40.71.70 TCP_REFRESH_UNMODIFIED/200 50531 GET http://download.eclipse.org/releases/kepler/201305310900/plugins/org.eclipse.equinox.preferences_3.5.100.v20130422-1538.jar.pack.gz - DIRECT/198.41.30.199 application/x-gzip 1370018697.513
This essentially says that you don't hit the local cache, but want the content reloaded. What you normally want to see though, are cache hits manifesting in TCP_MEM_HIT statements.
0 188.40.71.70 TCP_MEM_HIT/200 77323 GET http://download.eclipse.org/releases/kepler/201305310900/plugins/org.objectweb.asm_3.3.1.v201105211655.jar.pack.gz - NONE/- application/x-gzip 1370018314.277
2 188.40.71.70 TCP_MEM_HIT/200 473229 GET http://download.eclipse.org/releases/kepler/201305310900/plugins/org.objectweb.asm.source_3.3.1.v201105211655.jar.pack.gz - NONE/- application/x-gzip 1370018314.406
The underlying problem/fix has not been fully diagnosed yet, but configuring squid to ignore reload requests for certain file types fixes the symptoms (an OSGi bundle carries its version number in its name, thus caching this type of files should not result in reading stale content). Just add the following refresh_pattern to the squid proxy that is supposed to speed up your builds.
/etc/squid/squid.conf
refresh_pattern -i _[0-9v.-]+\.jar(\.pack\.gz)?$ 480 20% 4320 ignore-no-cache ignore-no-store override-expire override-lastmod ignore-reload
Below you see the cache stats for squid and all is green. :)
