LVS-NAT - Review request

Brent Cook busterb at mail.utexas.edu
Fri Aug 31 17:56:06 BST 2001


Hi everyone,

  I am happy to say that LVS-NAT works pretty well in my test servers. The
howtos are very thorough, though I have a couple of questions.  I'm
including my setup script as an attachment. Maybe you can point out
anything bone-headed I have done.

First, ssh connections get reset if there is no activity after 5 minutes
or so. At first, I thought persistence was what I wanted, so I set it to
1800 on ssh.  However, that's silly as I now know, since persistence
refers to giving a client the same real server within the persistence
time. Or so I'm lead to believe. Any ideas? I'm now trying to remove
persistence to see if that works. I don't know why that would affect
it though. Direct ssh doesn't disconnect in any case.

Second, anyone want to recommend the best MTA for a LVS system? I'd like
mail to be delivered into users' home directories that are shared over
NFS. So far, qmail looks the best, though I'm just now trying to learn
its eccentricities. Postfix and sendmail don't seem to like delivering
over NFS very much.

Third, just wanted to let everyone know that LVS works with the latest AC
kernels, though you have to comment out EXPORT_SYMBOL(buffermem_pages) in
kernel/ksyms.c, since it's defined elsewhere in AC's version. No, the ssh
timeout isn't tied to this kernel, since I tried it with stock Linus
versions first. AC just had some patches that my redirector/routers liked.

Thanks,
  Brent
-------------- next part --------------
#!/bin/sh
#
# rc.firewall - Texas Union Unix Cluster (ozma.union.utexas.edu)
#
# Brent Cook <busterb at mail.utexas.edu>
#  based on rc.firewall by Oskar Andreasson <blueflux at koffein.net>
# (c) of BoingWorld.com

###########
#
# your LAN's IP range and localhost IP. /24 means to only use the first 24
# bits of the 32 bit IP adress. the same as netmask 255.255.255.0
#
# STATIC_IP is used by me to allow myself to do anything to myself, might
# be a security risc but sometimes I want this. If you don't have a static
# IP, I suggest not using this option at all for now but it's still
# enabled per default and will add some really nifty security bugs for all
# those who skips reading the documentation=)

LAN_IP_RANGE="192.168.1.0/24"
LAN_IP="192.168.1.1"
REALSERVER_IP1="192.168.1.11"
REALSERVER_IP2="192.168.1.12"
LAN_BCAST_ADRESS="192.168.1.255/32"

LOCALHOST_IP="127.0.0.1"
STATIC_IP="146.6.96.9"
INET_IFACE="eth0"
LAN_IFACE="eth1"
IPTABLES="/usr/sbin/iptables --verbose"
IPVSADM="/sbin/ipvsadm"

ENABLE_FIREWALL="yes"

#########
# Load all required IPTables modules
#

# Adds some iptables targets like LOG, REJECT and MASQUARADE.
#
/sbin/modprobe ipt_LOG
#/sbin/modprobe ipt_REJECT
/sbin/modprobe ipt_MASQUERADE

#
# Remove previous routing tables
#
$IPTABLES --flush

#
# Support for owner matching
#
/sbin/modprobe ipt_owner

#
# Support for connection tracking of FTP
#
/sbin/modprobe ip_conntrack_ftp


#CRITICAL:  Enable IP forwarding since it is disabled by default.
#
echo "1" > /proc/sys/net/ipv4/ip_forward


# Enable simple IP FORWARDing and Masquerading
#

$IPTABLES -t nat -A POSTROUTING -o $INET_IFACE -j MASQUERADE
$IPTABLES -A FORWARD -i $LAN_IFACE -j ACCEPT
$IPTABLES -A FORWARD -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A FORWARD -m limit --limit 3/minute --limit-burst 3 -j LOG --log-level DEBUG --log-prefix "IPT FORWARD packet died: "

# check to enable firewall
if [ "$ENABLE_FIREWALL" = "yes" ]; then

#
# set default policies for the INPUT, FORWARD and OUTPUT chains
#

$IPTABLES -P INPUT DROP
$IPTABLES -P OUTPUT DROP
$IPTABLES -P FORWARD DROP

#
# Create separate chains for ICMP, TCP and UDP to traverse
#

$IPTABLES -N icmp_packets
$IPTABLES -N tcp_packets
$IPTABLES -N udpincoming_packets

#
# the allowed chain for TCP connections
#
# This chain will be utilised if someone tries to connect to an allowed
# port from the internet. If they are opening the connection, or if it's
# already established we ACCEPT the packages. This is where the state matching 
# is performed also, we allow NEW, ESTABLISHED and RELATED packets. I think
# this is redundant to do, but it shouldn't hurt, and is nice as an example.

$IPTABLES -N allowed
$IPTABLES -A allowed -p TCP --syn -j ACCEPT
$IPTABLES -A allowed -p TCP -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A allowed -p TCP -j DROP

#
# ICMP rules
#

$IPTABLES -A icmp_packets -p ICMP -s 0/0 --icmp-type 0 -j ACCEPT
$IPTABLES -A icmp_packets -p ICMP -s 0/0 --icmp-type 3 -j ACCEPT
$IPTABLES -A icmp_packets -p ICMP -s 0/0 --icmp-type 5 -j ACCEPT
$IPTABLES -A icmp_packets -p ICMP -s 0/0 --icmp-type 11 -j ACCEPT

#
# TCP rules
# port 26 is actually ssh to the routers
#

$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport ftp-data -j allowed
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport ftp -j allowed
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport smtp -j allowed
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport pop2 -j allowed
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport pop3 -j allowed
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport ssh -j allowed
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport ssh-router -j allowed
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport http -j allowed	
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport https -j allowed
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport ident -j allowed
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport submission -j allowed

#
# UDP ports
#

$IPTABLES -A udpincoming_packets -p UDP -s 0/0 --source-port 53 -j ACCEPT
$IPTABLES -A udpincoming_packets -p UDP -s 0/0 --source-port 123 -j ACCEPT
$IPTABLES -A udpincoming_packets -p UDP -s 0/0 --source-port 2074 -j ACCEPT
$IPTABLES -A udpincoming_packets -p UDP -s 0/0 --source-port 4000 -j ACCEPT

#
# PREROUTING chain.
#
# Do some checks for obviously spoofed IP's 
#

$IPTABLES -t nat -A PREROUTING -i $INET_IFACE -s 192.168.0.0/16 -j DROP
$IPTABLES -t nat -A PREROUTING -i $INET_IFACE -s 10.0.0.0/8 -j DROP
$IPTABLES -t nat -A PREROUTING -i $INET_IFACE -s 172.16.0.0/12 -j DROP

#
# Mark packets in http/https, smtp/submission and ftp groups
#

$IPTABLES -t mangle -A PREROUTING -i $INET_IFACE -p tcp -s 0.0.0.0/0 -d $STATIC_IP --dport ftp -j MARK --set-mark 1
$IPTABLES -t mangle -A PREROUTING -i $INET_IFACE -p tcp -s 0.0.0.0/0 -d $STATIC_IP --dport ftp-data -j MARK --set-mark 1
$IPTABLES -t mangle -A PREROUTING -i $INET_IFACE -p tcp -s 0.0.0.0/0 -d $STATIC_IP --dport http -j MARK --set-mark 2
$IPTABLES -t mangle -A PREROUTING -i $INET_IFACE -p tcp -s 0.0.0.0/0 -d $STATIC_IP --dport https -j MARK --set-mark 2
$IPTABLES -t mangle -A PREROUTING -i $INET_IFACE -p tcp -s 0.0.0.0/0 -d $STATIC_IP --dport smtp -j MARK --set-mark 3
$IPTABLES -t mangle -A PREROUTING -i $INET_IFACE -p tcp -s 0.0.0.0/0 -d $STATIC_IP --dport submission -j MARK --set-mark 3


#
# INPUT chain
#
# establish the basic INPUT chain and filter the packets onto the correct
# chains.
#

$IPTABLES -A INPUT -p ICMP -i $INET_IFACE -j icmp_packets
$IPTABLES -A INPUT -p TCP -i $INET_IFACE -j tcp_packets
$IPTABLES -A INPUT -p UDP -i $INET_IFACE -j udpincoming_packets

$IPTABLES -A INPUT -p ALL -i $LAN_IFACE -d $LAN_BCAST_ADRESS -j ACCEPT
$IPTABLES -A INPUT -p ALL -d $LOCALHOST_IP -j ACCEPT
$IPTABLES -A INPUT -p ALL -d $LAN_IP -j ACCEPT
$IPTABLES -A INPUT -p ALL -d $STATIC_IP -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A INPUT -m limit --limit 3/minute --limit-burst 3 -j LOG --log-level DEBUG --log-prefix "IPT INPUT packet died: "

#
# OUTPUT chain
#
# establish the basic OUTPUT chain and filter them onto the correct chain
#

$IPTABLES -A OUTPUT -p ALL -s $LOCALHOST_IP -j ACCEPT
$IPTABLES -A OUTPUT -p ALL -s $LAN_IP -j ACCEPT
$IPTABLES -A OUTPUT -p ALL -s $STATIC_IP -j ACCEPT
$IPTABLES -A OUTPUT -m limit --limit 3/minute --limit-burst 3 -j LOG --log-level DEBUG --log-prefix "IPT OUTPUT packet died: "

fi

echo "Starting IPVS"
#
# clear old IPVS tables
#
$IPVSADM --clear

#
# ipvs forwarding for FTP service, weighted least connections
#
echo "Adding FTP service..."
$IPVSADM -A -f 1 -s wlc -p 600
$IPVSADM -a -f 1 -r $REALSERVER_IP1:0 -m -w 1
#$IPVSADM -a -f 1 $REALSERVER_IP2:0 -m -w 1

#
# ipvs forwarding for HTTP service, round-robin
#
echo "Adding HTTP/HTTPD service..."
$IPVSADM -A -f 2 -s rr -p 600
$IPVSADM -a -f 2 -r $REALSERVER_IP1:0 -m
$IPVSADM -a -f 2 $REALSERVER_IP2:0 -m

#
# ipvs forwarding for SMTP service, round-robin
#
echo "Adding SMTP service..."
$IPVSADM -A -f 3 -s rr -p 15
$IPVSADM -a -f 3 -r $REALSERVER_IP1:0 -m
$IPVSADM -a -f 3 $REALSERVER_IP2:0 -m

#
# ipvs forwarding for SSH service, weighted least connections
#
echo "Adding SSH service..."
$IPVSADM -A -t $STATIC_IP:ssh -s wlc 
$IPVSADM -a -t $STATIC_IP:ssh -r $REALSERVER_IP1:ssh -m -w 1
$IPVSADM -a -t $STATIC_IP:ssh -r $REALSERVER_IP2:ssh -m -w 1


More information about the lvs-users mailing list