Introduction to Proxy Servers
Do you have a growing family at home slowly eating away at your bandwidth? Maybe you're a web surfing fanatic looking for a little more speed? If you answered yes to either, a caching proxy is for you. This simple addition to your home network can provide you with additional bandwidth by reducing common internet bandwidth usage. Normally these types of proxies are found in the commercial world, but they're just as useful at home. Below is an image of a traditional multi-computer home network.
Traditional Home Network
So what is a caching proxy server? The concept is pretty simple: when a request is made to a website, that content is then saved locally on the local caching proxy server. When another request for the same data is made by any machine on your network, that data is retrieved from your local proxy rather than the internet. The content can be anything from regular website content to a file you downloaded. For those with multiple computers in a single household, the bandwidth savings really adds up with patches and multi computer driver updates. The change to the network configuration is really quite small:
Home Network with Proxy Server
At this point many are likely asking how much this costs. If you read my previous article, you would know the answer right away: "It's free and it's on Linux". I suppose I need to preface that last comment with the qualification that you need some old "junky but functional" hardware lying around. There are many different Linux solutions we can deploy to achieve this goal. For this article I have chosen a solution of Arch Linux, Shorewall, and Squid.
We selected Arch Linux because it is a rolling release and has the latest and greatest packages. If you are not familiar with the phrase "rolling release", in Linux it indicated a distribution that keeps you up-to-date with the latest software updates via the package manager. You will never have to re-install or upgrade your server from one release version to the next with this style of distribution. The great part about a rolling release on a proxy/firewall setup is that once it's set up and working correctly, you will not have to go back and completely overhaul the server when a newer distribution update comes out.
Along with the different types of OS and application solutions, there are also multiple ways to set up a caching proxy. My preferred setup is a transparent caching proxy. A transparent proxy does not require you to make any additional changes to the client computers on your network. You utilize the proxy server as your home gateway, allowing the proxy server to automatically forward the ports to Squid. The second way to utilize Squid would be to set up your client machines to utilize the proxy server via the proxy settings in your browser. Although this may be the easiest way to set up a proxy server, it requires you to make changes for any machine that attaches to your network. The table below shows what I selected for my transparent caching proxy server.
| Test Proxy System | |
| Component | Description |
| Processor |
Intel Pentium 4 3.06GHz (3.06GHz, 130nm, 512K cache, Single-core + Hyper-Threading, 70W) |
| Memory | 2x256MB PC800 RDRAM |
| Motherboard | Asus P4T |
| Hard Drives | 120GB Western Digital SATA |
| Video Card | ATI Radeon 7000 |
| Operating Systems | Arch Linux (32-bit) |
| Network Cards |
Onboard Intel Gigabit PCI 100Mbit 3Com 3c905C-TX |
I could have selected older equipment, but this is what I had laying around the house. As seen in the table, one of the hardware requirements for a transparent proxy is to have two network cards or a dual port network card. We recommend against using wireless for either of the connections to the proxy server, and a Gigabit Ethernet connection from the proxy to the rest of the network is ideal. (The connection to your broadband link can be 100Mbit without imposing any bottleneck.) Another quick suggestion: If you download a fair amount of files, it may be a wise idea to utilize at least a 120GB HDD. The idea is that the more space you have, the longer you can keep your files stored on your proxy server. With storage being so cheap, you could easily add a 500GB or larger drive for under $100.
Now that we have our hardware and a good idea what we want to set up, it's time to get installing. I'll try to keep this portion simple and to the point, although if you have questions later feel free to post a comment.
Facebook
Twitter
RSS
96 Comments
View All Comments
mindless1 - Tuesday, May 11, 2010 - link
What's the "ethernet wall outlet" supposed to be? If you mean plug your switch straight into a modem of some sort, you're better off having the extra layer of security afforded by the NAT feature on a router. That is especially true with Windows based PCs, nevermind that for many people use of a router also gives them wifi capability.Dravic - Tuesday, May 11, 2010 - link
As a previous poster mention you should look at Smoothwall express 3.0. My current incarnation is running on an old duron kt 266 platform with 1gb of ram and that is complete overkill for something like this. I would also look at the benefits vs the performance loss.With a few as 4 or 5 pc's you are probably getting reduced web browsing performance for the benefit of reduced broadband usage on a small amount of large files. Is it worth having a slower internet 99% of the time in order to increase download time of for 1% of the time? Remember for every image you hit you now have to query this proxy to see if the image is stored locally and possibly if an update copy of the image exist at the original source. Configuration of the proxy will be key. How much do you store , and how long do you cache items before expiration can have a massive effect on regular browsing.
Proxies are really meant for networks with significant number of users hitting the same content repeatedly. Caching the web objects of the most frequently viewed website of 100 people provides real savings in bandwidth and increased browsing speed. For a small group of people the bandwidth saving are usually mild, but now you have increased browsing times across the board.
I think you would better served using a qos solution (also in smoothwall 3.0) over a a squid proxy. On my fios 20/5 mb line qos overhead eats ~ 1mb of total capacity.
In my home network (6 pc's and a few Internet appliances) neither qos or a proxy were beneficial with fios(i know not everyone has 20/5 internet, but this held true even when fios was 10/2). When i was on dial-up-upl the proxy was great for hitting multimedia heavy sights like ESPN.
Either way i do recommend anyone with the know how build there own firewall appliance if they can stand the energy cost. The consumer grade firewall/gateways really are poor and while getting better really don't offer the range of services something like smoothwall (m0n0wall, ipcop, pfense .. etc) does.
Other then my philosophical difference on the benefits, good article. A followup with the most widely used pre built solutions with some kind of browsing benchmarking would be a nice follow up.
dezza - Tuesday, May 11, 2010 - link
I totally agree.Actually the thing that brought me to this site was because a friend once told me that I would not benefit anything from having a "family"-proxy .. And I would think that these comments support that conclusion.
I would say if you're about to do this to it 100% and QoS and DHCP, etc. there is no point in having a server consuming 300-400W running JUST for a proxy that maybe even slows down browsing in the end and brings more maintenance to your home network.
ChrisRice - Tuesday, May 11, 2010 - link
For the two above posts I need to get some data/graphs together to add to the article. Much of what is being said above is simply not true. I'll try to work on this over the next day or two.bob4432 - Tuesday, May 11, 2010 - link
i am not running what this article is about software wise, but my home server is a skt939 3000, 1GB ram, 60GB main drive, 500GB image holding hdd and a 120GB misc hdd running an old pci gpu and i think 3-4 80mm fans on a antec earthwatts 380W psu. my simple network setup is a asus wl520g (i think that is the model number) w/ tomato 1.27 in addition to a 8port GbE switch. the reason for explaining all this, is that combined it all pulls 60W from my ups which was verified by a kill-o-watt.imaheadcase - Tuesday, May 11, 2010 - link
I remember using WinProxy way back in the day (early 90s) for dialup. It worked EXCELLENT. But why "save" bandwidth with something like this when you have broadband?Its not going to save much at all.
ChrisRice - Tuesday, May 11, 2010 - link
As the article refers to "Family Proxy" you could easily run out of bandwidth with broadband. For example if you have a few bandwidth hog room mates or have a wife and kids the savings are very much there. This is also the most simple setup of a proxy which could be expanded on to work with ftp and other ports. I wanted to keep it pretty simple but maybe there is interest in a more advanced setup?micksh - Tuesday, May 11, 2010 - link
How does proxy affect browsing experience? I assume there will be additional latency. Did you compare web page loading times with and without proxy?And how much (in seconds or minutes) does it help when you download large file second time? Does it make things faster if other PCs are doing something else on web?
I actually tried similar setup hoping to make web browsing faster. I had Safesquid on Ubuntu on relatively fast Core 2 Duo PC using 6 Mbs AT&T DSL. It didn't help. Most web servers give content using "post" method so pages could not be cached. I enabled pre-fetching but I guess I could not configure priorities correctly. Pre-fetching made current page to load slower. Without prefetching things still seemed a bit slower because of the latency that additional box gives.
Since I moved to 18 Mbs U-Verse and things are good without proxy.
spazmedia - Tuesday, May 11, 2010 - link
Its nice to see an article on anandtech about Linux. Once you get the hang of it, most Linux distro are FAR simpler to configure then Windows as the config does not change much from distro to distro and from version to version. Also as others have pointed out smoothwall is quite easy to configure. Another useful tool for configuring all aspects of a linux box remotely through https is webmin (http://www.webmin.com/) I've tried it with Fedora and Debian/Ubuntu and it probably is a bit more functionnal with Debian. For ease of use nothing beats Suse though (from Novel)spazmedia - Tuesday, May 11, 2010 - link
BTW for those looking for power savings, its a bit more expensive but a gread idea for this application to use an Atom or low power celeron processor...Or the best is an old laptop (probably need to buy an extra PCMCIA NIC though). Plus you get battery backup if its not too old and battery not worn out. Having said that setting up Linux on most laptops is not trivial given custom hardware most manufacturers implement.