📚 LinuxDocs
Topics:
All Pages8021X HOWTOACP ModemACPI HOWTOADSL Bandwidth Man..ATA RAID HOWTOATM Linux HOWTOAX25 HOWTOAccessibility Dev ..Accessibility HOWTOAdv Bash Scr HOWTOAdv Routing HOWTOAntares RAID sparc..Apache Compile HOWTOApache WebDAV LDAP..Assembly HOWTOAstronomy HOWTOAthlon Powersaving..Authentication Gat..Autodir HOWTOAviation HOWTOAvr Microcontrolle..BRIDGE STP HOWTOBTTVBackspaceDeleteBandwidth Limiting..Bangla HOWTOBash Prompt HOWTOBattery PoweredBelarusian HOWTOBelgian HOWTOBeowulf HOWTOBocaBogoMipsBootdisk HOWTOBridgeC++ dlopenC C++Beautifier HO..C editing with VIM..CDROM HOWTOCDServer HOWTOCable ModemCaudium HOWTOClone HOWTOCompaq Remote Insi..Compaq T1500 HOWTOConexant+Rockwell ..Cryptoloop HOWTODB2 HOWTODHCPDSL HOWTODVD Playback HOWTODebian Binary Pack..Debian JigdoDebian and Windows..Disk Encryption HO..Disk on Chip HOWTODocBook Demystific..DocBook InstallDocBook OpenJade S..Ecology HOWTOEmacspeak HOWTOEncourage Women Li..Encrypted Root Fil..Euro Char SupportEvent HOWTOFedora Multimedia ..Finnish HOWTOFirewall PiercingFlash Memory HOWTOFont HOWTOFramebuffer HOWTOGCC HOWTOGIS GRASSGlibc Install HOWTOHOWTO HOWTOHOWTO INDEXHP HOWTOHandspring VisorHard Disk UpgradeHardware HOWTOHighQuality Apps H..Home Electrical Co..IBM7248 HOWTOIO Perf HOWTOIP AliasIP Masquerade HOWTOIRCImplement Sys Call..Indic Fonts HOWTOInfrared HOWTOIngresII HOWTOInstall StrategiesInstallation HOWTOInstallfest HOWTOIntkeybItalian HOWTOJabber Server Farm..JavaStation HOWTOKerberos Infrastru..Kernel HOWTOKerneldKodak Digitalcam H..LDAP HOWTOLDP Reviewer HOWTOLILO crash rescue ..LVM HOWTOLeased LineLegoLinksys Blue Box R..Linux+Win95Linux+Win9x+Grub H..Linux+Windows HOWTOLinux Complete Bac..Linux Crash HOWTOLinux Gamers HOWTOLinux Modem SharingLinux Promise RAID..Linux i386 Boot Co..LinuxGL QuakeWorld..Lotus DominoR5MILO HOWTOMMBase Inst HOWTOMP3 CD BurningMail User HOWTOMajordomo MajorCoo..Man PageMasquerading Simpl..Medicine HOWTOMindTerm SSH HOWTOMobile IPv6 HOWTOMock MainframeModule HOWTOModulesMotorola Surfboard..Mozilla OptimizationMulti Distro DevNCURSES Programmin..NFS HOWTONFS Root Client mi..NIS HOWTONetMeeting HOWTONetwork boot HOWTONvidia OpenGL Conf..OLSR IPv6 HOWTOOnline Troubleshoo..Oracle 9i Fedora 3..PA RISC Linux Boot..PCTel MicroModem C..PHP Nuke HOWTOPPP HOWTOPagerPalmOS HOWTOPartitionPartition Mass Sto..Partition Mass Sto..Partition RescuePine ExchangePortSlavePost Installation ..Postfix Cyrus Web ..Pre Installation C..Print2WinPrinting HOWTOProcess AccountingProgram Library HO..Proxy ARP SubnetQmail ClamAV HOWTOQmail VMailMgr Cou..Querying libiptc H..RPM HOWTOReading List HOWTORedHat CD HOWTOReliance HOWTORemote BridgingRemote Serial Cons..SCSI 2.4 HOWTOSCSI Generic HOWTOSLIP PPP EmulatorSRM HOWTOSSL Certificates H..Scanner HOWTOScientific Computi..Scripting GUI TclTkSecure CVS PserverSecure Programs HO..Security HOWTOSecurity Quickstar..Security Quickstar..Serial Laplink HOWTOSerial Programming..Slovak HOWTOSmall MemorySmart Card HOWTOSoftware Proj Mgmt..Software Release P..Sound HOWTOSpam Filtering for..Speech Recognition..SquashFS HOWTOSybase ASA HOWTOSybase ASE HOWTOSybase PHP ApacheTCP Keepalive HOWTOTamil Linux HOWTOTimePrecision HOWTOTimeSys Linux Inst..Token RingTraffic Control HO..Traffic Control tc..UPS HOWTOUnix Hardware Buye..Unix and Internet ..UpgradeUsenet News HOWTOUser Authenticatio..VB6 to TclVMS to Linux HOWTOVPN HOWTOValgrind HOWTOVideoLAN HOWTOVim HOWTOVirtual WebWebcam HOWTOWikiText HOWTOWindows Newsreader..Wireless Link sys ..Wireless Sync HOWTOXDM XtermXDMCP HOWTOXFree Local multi ..XFree86 HOWTOXFree86 R200XFree86 Second MouseXFree86 Video Timi..XML RPC HOWTOXWindow Overview H..XWindow User HOWTOXinerama HOWTOXterminalsHtml singleI810 HOWTOLibdc1394 HOWTOOpenMosix HOWTOPhhttpd HOWTOPpp sshText

15.5. Transparent web-caching using netfilter, iproute2, ipchains and squid

This section was sent in by reader Ram Narula from Internet for Education (Thailand). .:: s789club78.hashnode.space ::.

The regular technique in accomplishing this in Linux is probably with use of ipchains AFTER making sure that the "outgoing" port 80(web) traffic gets routed through the server running squid.

There are 3 common methods to make sure "outgoing" port 80 traffic gets routed to the server running squid and 4th one is being introduced here. .:: telegra.ph ::.

Making the gateway router do it.

If you can tell your gateway router to match packets that has outgoing destination port of 80 to be sent to the IP address of squid server. .:: www.bandsworksconcerts.info ::.

BUT

This would put additional load on the router and some commercial routers might not even support this.

Using a Layer 4 switch.

Layer 4 switches can handle this without any problem.

BUT

The cost for this equipment is usually very high. Typical layer 4 switch would normally cost more than a typical router+good linux server.

Using cache server as network's gateway.

You can force ALL traffic through cache server.

BUT

This is quite risky because Squid does utilize lots of CPU power which might result in slower over-all network performance or the server itself might crash and no one on the network will be able to access the Internet if that occurs.

Linux+NetFilter router.

By using NetFilter another technique can be implemented which is using NetFilter for "mark"ing the packets with destination port 80 and using iproute2 to route the "mark"ed packets to the Squid server. .:: www.empyrethegame.com ::. [Code: Couchbase]

|----------------|
| Implementation |
|----------------|

 Addresses used
 10.0.0.1 naret (NetFilter server)
 10.0.0.2 silom (Squid server)
 10.0.0.3 donmuang (Router connected to the Internet)
 10.0.0.4 kaosarn (other server on network)
 10.0.0.5 RAS
 10.0.0.0/24 main network
 10.0.0.0/19 total network

|---------------|
|Network diagram|
|---------------|

Internet
|
donmuang
|
------------hub/switch----------
|        |             |       |
naret   silom        kaosarn  RAS etc.
First, make all traffic pass through naret by making sure it is the default gateway except for silom. Silom's default gateway has to be donmuang (10.0.0.3) or this would create web traffic loop.

(all servers on my network had 10.0.0.1 as the default gateway which was the former IP address of donmuang router so what I did was changed the IP address of donmuang to 10.0.0.3 and gave naret ip address of 10.0.0.1)

Silom
-----
-setup squid and ipchains 

Setup Squid server on silom, make sure it does support transparent caching/proxying, the default port is usually 3128, so all traffic for port 80 has to be redirected to port 3128 locally. This can be done by using ipchains with the following: [Reddit: Bitcoin Discussion]

silom# ipchains -N allow1
silom# ipchains -A allow1 -p TCP -s 10.0.0.0/19 -d 0/0 80 -j REDIRECT 3128
silom# ipchains -I input -j allow1

Or, in netfilter lingo:

silom# iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

(note: you might have other entries as well)

For more information on setting Squid server please refer to Squid FAQ page on http://squid.nlanr.net).

Make sure ip forwarding is enabled on this server and the default gateway for this server is donmuang router (NOT naret).

Naret
-----
-setup iptables and iproute2
-disable icmp REDIRECT messages (if needed)

  1. "Mark" packets of destination port 80 with value 2

     
    naret# iptables -A PREROUTING -i eth0 -t mangle -p tcp --dport 80 \
     -j MARK --set-mark 2
  2. Setup iproute2 so it will route packets with "mark" 2 to silom .:: podcasts.apple.com ::.

    naret# echo 202 www.out >> /etc/iproute2/rt_tables
    naret# ip rule add fwmark 2 table www.out
    naret# ip route add default via 10.0.0.2 dev eth0 table www.out
    naret# ip route flush cache
    

    If donmuang and naret is on the same subnet then naret should not send out icmp REDIRECT messages. In this case it is, so icmp REDIRECTs has to be disabled by:

    naret# echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
    naret# echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
    naret# echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects
    

The setup is complete, check the configuration

On naret:

naret# iptables -t mangle -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         
MARK       tcp  --  anywhere             anywhere           tcp dpt:www MARK set 0x2 

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

naret# ip rule ls
0:      from all lookup local 
32765:  from all fwmark        2 lookup www.out 
32766:  from all lookup main 
32767:  from all lookup default 

naret# ip route list table www.out
default via 203.114.224.8 dev eth0 

naret# ip route   
10.0.0.1 dev eth0  scope link 
10.0.0.0/24 dev eth0  proto kernel  scope link  src 10.0.0.1
127.0.0.0/8 dev lo  scope link 
default via 10.0.0.3 dev eth0 

(make sure silom belongs to one of the above lines, in this case
it's the line with 10.0.0.0/24)

|------|
|-DONE-|
|------|

15.5.1. Traffic flow diagram after implementation

|-----------------------------------------|
|Traffic flow diagram after implementation|
|-----------------------------------------|

INTERNET
/\
||
\/
-----------------donmuang router---------------------
/\                                      /\         ||
||                                      ||         ||
||                                      \/         ||
naret                                  silom       ||
*destination port 80 traffic=========>(cache)      ||
/\                                      ||         ||
||                                      \/         \/
\\===================================kaosarn, RAS, etc.

Note that the network is asymmetric as there is one extra hop on general outgoing path.

Here is run down for packet traversing the network from kaosarn
to and from the Internet.

For web/http traffic:
kaosarn http request->naret->silom->donmuang->internet
http replies from Internet->donmuang->silom->kaosarn

For non-web/http requests(eg. telnet):
kaosarn outgoing data->naret->donmuang->internet
incoming data from Internet->donmuang->kaosarn

Share or Research:

Share on FB Post to X LinkedIn 🤖 Ask AI about this