Introduction to Proxy Servers
Do you have a growing family at home slowly eating away at your bandwidth? Maybe you're a web surfing fanatic looking for a little more speed? If you answered yes to either, a caching proxy is for you. This simple addition to your home network can provide you with additional bandwidth by reducing common internet bandwidth usage. Normally these types of proxies are found in the commercial world, but they're just as useful at home. Below is an image of a traditional multi-computer home network.
Traditional Home Network
So what is a caching proxy server? The concept is pretty simple: when a request is made to a website, that content is then saved locally on the local caching proxy server. When another request for the same data is made by any machine on your network, that data is retrieved from your local proxy rather than the internet. The content can be anything from regular website content to a file you downloaded. For those with multiple computers in a single household, the bandwidth savings really adds up with patches and multi computer driver updates. The change to the network configuration is really quite small:
Home Network with Proxy Server
At this point many are likely asking how much this costs. If you read my previous article, you would know the answer right away: "It's free and it's on Linux". I suppose I need to preface that last comment with the qualification that you need some old "junky but functional" hardware lying around. There are many different Linux solutions we can deploy to achieve this goal. For this article I have chosen a solution of Arch Linux, Shorewall, and Squid.
We selected Arch Linux because it is a rolling release and has the latest and greatest packages. If you are not familiar with the phrase "rolling release", in Linux it indicated a distribution that keeps you up-to-date with the latest software updates via the package manager. You will never have to re-install or upgrade your server from one release version to the next with this style of distribution. The great part about a rolling release on a proxy/firewall setup is that once it's set up and working correctly, you will not have to go back and completely overhaul the server when a newer distribution update comes out.
Along with the different types of OS and application solutions, there are also multiple ways to set up a caching proxy. My preferred setup is a transparent caching proxy. A transparent proxy does not require you to make any additional changes to the client computers on your network. You utilize the proxy server as your home gateway, allowing the proxy server to automatically forward the ports to Squid. The second way to utilize Squid would be to set up your client machines to utilize the proxy server via the proxy settings in your browser. Although this may be the easiest way to set up a proxy server, it requires you to make changes for any machine that attaches to your network. The table below shows what I selected for my transparent caching proxy server.
| Test Proxy System | |
| Component | Description |
| Processor |
Intel Pentium 4 3.06GHz (3.06GHz, 130nm, 512K cache, Single-core + Hyper-Threading, 70W) |
| Memory | 2x256MB PC800 RDRAM |
| Motherboard | Asus P4T |
| Hard Drives | 120GB Western Digital SATA |
| Video Card | ATI Radeon 7000 |
| Operating Systems | Arch Linux (32-bit) |
| Network Cards |
Onboard Intel Gigabit PCI 100Mbit 3Com 3c905C-TX |
I could have selected older equipment, but this is what I had laying around the house. As seen in the table, one of the hardware requirements for a transparent proxy is to have two network cards or a dual port network card. We recommend against using wireless for either of the connections to the proxy server, and a Gigabit Ethernet connection from the proxy to the rest of the network is ideal. (The connection to your broadband link can be 100Mbit without imposing any bottleneck.) Another quick suggestion: If you download a fair amount of files, it may be a wise idea to utilize at least a 120GB HDD. The idea is that the more space you have, the longer you can keep your files stored on your proxy server. With storage being so cheap, you could easily add a 500GB or larger drive for under $100.
Now that we have our hardware and a good idea what we want to set up, it's time to get installing. I'll try to keep this portion simple and to the point, although if you have questions later feel free to post a comment.
Facebook
Twitter
RSS
96 Comments
View All Comments
SquattingDog - Tuesday, May 11, 2010 - link
This is a great article, and comes in a very timely fashion, as I am looking to set something like this up in our flat. We have a 20GB monthly cap, and need to distribute the per-GB costs out to each person based on their usage and possibly limit their usage if they exceed 5GB for example. Is this possible with a Linux Proxy or QoS tool? If so, what should I be looking at to do this - and are there any which are quick and easy (in relative terms) to set up? (I am a Linux noob atm)Second question has to do with latency for games. One of the people in our flat plays games like Bad Company 2 online a lot of the time. That, basic browing and MSN are the only things he uses the internet for. What is the added delay with a transparent proxy in place for gaming? I know you mentioned Steam updates not working with proxy caching during the article, Jarred, but what about the gaming itself? Is there a measurable/noticeable latency increase? An increase in the order of 2 - 5ms is acceptable, and we can always get interleaving turned off on our line to mitigate this.
ChrisRice - Tuesday, May 11, 2010 - link
With the setup mentioned in the article you will have no adverse effects to your gaming. I will look up your proxy quota question, I believe there are a bunch of solutions available.SquattingDog - Tuesday, May 11, 2010 - link
Thanks Chris, that would be outstanding!JarredWalton - Wednesday, May 12, 2010 - link
Yeah, I tested gaming and didn't notice any problems with the proxy. Steam works fine BTW, but it doesn't go out through the proxy so the updates aren't cached. (I tried sending the Steam update ports through the proxy but then Steam wouldn't connect... looking around online, numerous folks are saying Valve doesn't allow use of Steam through a proxy.) Bad Company 2 also works fine, as do quite a few other titles I've played.Squid can do a lot of things not discussed in this article, but how well it does them and how easy they are to configure is probably something for a follow-up. As something of a Linux router newbie myself, I'm not quite sure how you go about restricting access and putting download caps on the various clients, but the squid.conf file suggest all of that is possible.
I'll leave the rest to Chris. :-)
SquattingDog - Wednesday, May 12, 2010 - link
Thanks for coming back to me on this Jarred, great news for myself and my other flatmates then - I wouldn't be popular if suddenly everyone's ping went up 30 - 50ms ;)mariush - Wednesday, May 12, 2010 - link
Get a managed switch and use MRTG (http://en.wikipedia.org/wiki/Multi_Router_Traffic_... or Cacti or other solutions to log how much traffic each port does.With a proxy, you'd have to create a username and password for each member in your house or log traffic on the server based on MAC address or IP which is a bit more complicated than simply polling the switch with such software and logging the bytes transferred.
See here some managed switches http://www.newegg.com/Product/ProductList.aspx?Sub...
Though there may be cheaper unmanaged switches which have SNMP feature, the thing you need for logging traffic.
SquattingDog - Wednesday, May 12, 2010 - link
That is a good solution too, however I have existing hardware lying around that I could put to use for the Linux box, and that would require me shelling out for both a new switch and a new wireless router (everyone but me connects via wireless, and it's an all-in-on Netgear DG834G ADSL Modem/Router) - and our pricing here is not as good as yours over there, unfortunately :(SquattingDog - Wednesday, May 12, 2010 - link
This DG834G is v5, so I'm SOL for enabling built-in SNMP, as it doesn't support that. The Proxy server set up would work well for us also, as I frequently have to download Windows updates on various machines which come and go, and having them locally cached will reduce internet usage substantially. :)mindless1 - Tuesday, May 11, 2010 - link
Some of you talked about cost or especially power savings. Has it occurred to you that you can run a proxy on the windows box you probably already leave running most if not all the time and expect a trivial increase in power consumption from doing so?Sure, you'll need have the amount of memory you want to devote added over the amount your system would otherwise need, but in this day and age of multi-gigabyte endowed systems it isn't much to devote 1/4th your memory to the job... if you really need that much which many people won't.
JarredWalton - Wednesday, May 12, 2010 - link
I looked around at various options, but for the free stuff it appears that you'd need to manually configure each browser to go through the Windows proxy (i.e. instead of having a transparent proxy). Anyway, my Windows machines are all even more power hungry than my test proxy, so I don't leave them running at night. But I believe squid is even available for Windows platforms:http://wiki.squid-cache.org/SquidFaq/BinaryPackage...