how to support transparent cache cluster in ipvs?

[Horms]

Hi,

this patch seems pretty nice to me, and it seems that it should work
quite well. Have you tested it? If so, could you provide a signed-off-by
line, as described in section 5 of http://linux.yyz.us/patch-format.html
so that I can submit it to netdev for inclusion in the kernel.

I have reformated the patch a bit, it is below. Feel free
to rework the comment if you like.

-- 
Horms
  H: http://www.vergenet.net/~horms/
  W: http://www.valinux.co.jp/en/

[IPVS] transparent proxying

Patch from home_king <home_king at 163.com> to allow a web cluseter using
transparent proxying. It works by simply grabing packets that have the
fwmark set and have not already been processed by ipvs (ip_vs_out) and
throwing them into ip_vs_in.

See: http://archive.linuxvirtualserver.org/html/lvs-users/2006-11/msg00261.html

Normally LVS packets are processed by ip_vs_in fron on the INPUT chain,
and packets that are processed in this way never show up on the FORWARD
chain, so they won't hit this rule.

This patch seems like a good precursor to moving LVS permanantly to
the FORWARD chain. As I'm struggling to think how it could break things.

Reformated to use tabs for indentation (instead of 4 spaces)
Reformated to be < 80 columns wide

Cc: Jinhua Luo <home_king at 163.com>
Signed-off-by: Simon Horman <horms at verge.net.au>

diff --git a/net/ipv4/ipvs/ip_vs_core.c b/net/ipv4/ipvs/ip_vs_core.c
index 1445bb4..5038386 100644
--- a/net/ipv4/ipvs/ip_vs_core.c
+++ b/net/ipv4/ipvs/ip_vs_core.c
@@ -23,7 +23,9 @@
  * Changes:
  *	Paul `Rusty' Russell		properly handle non-linear skbs
  *	Harald Welte			don't use nfcache
- *
+ *	Jinhua Luo                      redirect packets with fwmark on 
+ *					NF_IP_FORWARD chain to ip_vs_in(), 
+ *					mainly for transparent cache cluster
  */
 
 #include <linux/module.h>
@@ -1070,6 +1072,17 @@ ip_vs_forward_icmp(unsigned int hooknum,
 	return ip_vs_in_icmp(pskb, &r, hooknum);
 }
 
+static unsigned int
+ip_vs_forward_with_fwmark(unsigned int hooknum, struct sk_buff **pskb,
+			  const struct net_device *in,
+			  const struct net_device *out,
+			  int (*okfn)(struct sk_buff *))
+{
+	if ((*pskb)->ipvs_property || ! (*pskb)->nfmark)
+		return NF_ACCEPT;
+
+	return ip_vs_in(hooknum, pskb, in, out, okfn);
+}
 
 /* After packet filtering, forward packet through VS/DR, VS/TUN,
    or VS/NAT(change destination), so that filtering rules can be
@@ -1160,9 +1173,17 @@ static int __init ip_vs_init(void)
 		goto cleanup_postroutingops;
 	}
 
+	ret = nf_register_hook(&ip_vs_forward_with_fwmark_ops);
+	if (ret < 0) {
+		IP_VS_ERR("can't register forward_with_fwmark hook.\n");
+		goto cleanup_forwardicmpops;
+	}
+
 	IP_VS_INFO("ipvs loaded.\n");
 	return ret;
 
+  cleanup_forwardicmpops:
+	nf_unregister_hook(&ip_vs_forward_icmp_ops);
   cleanup_postroutingops:
 	nf_unregister_hook(&ip_vs_post_routing_ops);
   cleanup_outops:
@@ -1182,6 +1203,7 @@ static int __init ip_vs_init(void)
 
 static void __exit ip_vs_cleanup(void)
 {
+	nf_unregister_hook(&ip_vs_forward_with_fwmark_ops);
 	nf_unregister_hook(&ip_vs_forward_icmp_ops);
 	nf_unregister_hook(&ip_vs_post_routing_ops);
 	nf_unregister_hook(&ip_vs_out_ops);


Here, I redirect the packets with fwmark to ip_vs_in() on the FORWARD 
chain, and ip_vs_in() will handle the packets which are marked by 
iptables to indicate transparent cache virtual service, but ignore other 
packets (let them continue to flow on the FORWARD chain).

Now you can use ipvs to deploy TP at ease:
@ ipvs director
# sysctl -w net.ipv4.ip_forward=1
# iptables -t mangle -A FORWARD -p tcp -s <internal network> --dport 80 
-j MARK --set-mark 1
# ipvsadm -A -f 1 -s lblcr
# ipvsadm -a -f 1 -r RS1
# ipvsadm -a -f 1 -r RS2

@ RS
# iptables -t mangle -A PREROUTING -p tcp -s <internal network> --dport 
80 -j REDIRECT --to-ports 3128
# cat >> /etc/squid/squid.conf << EOF
  httpd_accel_host virtual
  httpd_accel_port 80
  httpd_accel_with_proxy on
  httpd_accel_uses_host_header on
# /etc/init.d/squid start


--------------000409050907010700040505
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users at LinuxVirtualServer.org
Send requests to lvs-users-request at LinuxVirtualServer.org
or go to http://www.in-addr.de/mailman/listinfo/lvs-users

--------------000409050907010700040505--