ApacheCon is a week of open source goodness straight from the
source of the Apache Software Foundation, featuring 15 intense
training classes over two days followed by three days of more
than 50 sessions by the creators of open source software such
as the Apache webserver, Tomcat, Lucene, Wicket and more.
Have a Question?
Questions about the program? Interested in
becoming a sponsor?
Please contact the event planners if you have
any questions. We look forward to seeing you in Amsterdam!
Scaling the download infrastructure with your success
When an organization offers files for download, and grows public, infrastructure might need to serve more requests than the single organization can technically handle. One option to scale is using paid services of a content delivery network (CDN). This is often not affordable, especially for Open Source projects. Another way is building up a mirror infrastructore. But choosing an appropriate mirror is often left to the user, and mirrors might be out of date, incomplete, or unreliable. This talk shows how to build a poor man's CDN, using plain mirror servers and a redirecting Apache HTTP server. On the example of download.openSUSE.org, it shows how mirrored content can be transparently integrated into the web service, and how requests are redirected to geographically close mirrors using IP geolocation. The demonstrated infrastructure comprises means to scan mirrors and keeping track of present files in a MySQL database, active monitoring of mirrors, and mod_zrkadlo, a redirecting Apache module built upon the DBD framework. Along the way, the talk discusses different approaches in geographical redirection of bulk data, load sharing with weighted randomization, high availibility and scaling issues, mirror stickiness, and client side robustness. Also discussed are problems that have been encountered in real life, solutions and open questions.