The Elliquiy LAMP Stack: IPTables Configuration

Started by Vekseid, March 27, 2009, 07:03:41 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Vekseid

The Elliquiy LAMP Stack

1: Introduction and Overview
2: General Configuration
3: General Security
4: IPTables configuration
5: Postfix configuration
6: ntp configuration
7: Apache compilation and configuration
8: MySQL compilation and configuration
9: PHP compilation and configuration
10: Conclusion and future plans




IPTables

IPTables is a slick, stateful firewall. I publish a script here, but there are some observations I would like to point out, before you run off and try to tighten things further.

Connlimit

This seems like a beautiful idea, but you need to be careful with setting it too tightly. Though usually a problem on the user's end, connections that the kernel times out do not always seem to be seen as closed by the firewall. If it is actually feasible for you to get in touch with your members when this occurs, you can address this accordingly. Otherwise, you may want to refresh the firewall during off-peak hours.

In addition, AOL's proxies can make quite a large number of connections. I have begun to address this, at least in part, by instructing members to skip AOL's browser and signin - or simply leaving AOL altogether. I don't convince all of them, unfortunately, but it does help. I set the connection limit generously initially, and have a tighter connection limit prepared in the event of a small-scale DDOS.

TCP

While I name a part of my script 'TCP Motherfucker do you speak it', it is important to realize that a lot of times the answer is an unequivocal 'no'. This may be IPTables' fault, it may be the client's, but either way you do not want to be blocking legitimate traffic even if it is not in its purest possible form. I explain this in greater detail, along with why, in my script.




Hashlimit

This wonderful, beautiful little item needs a lot more documentation love. Currently, the documentation and tutorials that exist are misleading at best - the only insight was actually found by combing through the kernel source. Even worse, many examples might suggest limit when hashlimit is what is really desired.

--hashlimit-htable-gcinterval is always set to three seconds in my script, except for ping, but you may wish to slow this down if you find your system cpu usage getting high under heavy loads. Your load is going to be based loosely off of htable-size divided by gcinterval. The default is 1000 (one second).

--hashlimit-htable-expire is an easy one to calculate - just pick the time when the hashlimit-upto/above will fill up your bucket, maybe add a bit extra, or less if you don't care about occasional overages. The default is 10000 (ten seconds).

--hashlimit-htable-max to quote the current kernel source: /* FIXME: do something. question is what.. */ It currently fires a kernel warning if the hash table is allowed to grow beyond this size. It defaults to 8 times the htable-size, and has a floor of htable-size if set to a low value. There's no actual limit, however. I'm not convinced that the hashlimit algorithm is all it could be >_> I set this to htable-size since if somehow eight thousand connections are open I rather want to be warned about it.

--hashlimit-htable-size is the size of the hash index. The given  tuple is taken from mode, and a hash of htable-size is computed for it. If two tuples end up with the same hash, they get placed in a chain which is iteratively searched. DO NOT SET THIS LOW EXPECTING IT WILL THROTTLE A DDOS. In theory, that is what htable-max is for. But only in theory. It has a bit of a messy default: num_physpages * page_size / 16384, with a cap of 8192 and a minimum of 16. Pretty much anyone with 256mb+ of RAM is going to reach the cap.
   
In addition to the basic memory for the table itself, you will allocate pointer_bytes * 2 * htable-size in bytes for the table. More memory gets allocated in other functions and I haven't fully gone through the source, but for an amd64 machine that means 16 bytes per bucket. Since this is the size of the full index and I have RAM to spare, I set all of the tables to 8k. The main thing to worry about here is the garbage collection interval, since this means it's checking eight thousand -linked lists- every interval.
   
--hashlimit-srcmask is probably best set to /29 for IPv4 in most situations where srcip is used for the tuple. Not only does this help reduce collision rates, /29's are often the same family or local organization. It works fine to treat them in this manner. The default for this and dstmask is 32 for IPv4 and 128 for IPv6.
   
--hashlimit-mode is a fairly straightforward setting. I would suggest using separate hashes for different ports and destination ips for most scenarios.
   
--hashlimit-burst defaults to five. This determines the maximum size of the bucket which gets filled by upto/above.




IPTables Script

The logging in this script could certainly use some degree of fine-tuning. Regardless, it works fairly well for Elliquiy's configuration. Don't just throw people who trigger your flood limits into a recent list for instant banning - you probably want to allow one grace hit, as people will occasionally reach it. Set a temporary list instead, matching from a specific period (a few hours, say), people who trigger the flood again then go into your banlist for a day or whatever. Combine with mod-evasive or the nginx equivalent along with intelligent scripting and you should have a fairly effective defense against small attacks.

Note: You do need to configure this script. See the text for details.  While testing, you may wish to run a cleaner script as a cron job while doing so:


#!/bin/sh
/sbin/iptables -F
/sbin/iptables -X
/sbin/iptables -P INPUT ACCEPT
/sbin/iptables -P FORWARD ACCEPT
/sbin/iptables -P OUTPUT ACCEPT


Every fifteen minutes or however long you are willing to wait.


#!/bin/sh
#######################################################################
# This is a simple firewall script I developed to protect Elliquiy. It
# is not designed to defend against even a small DDOS, but should
# provide a fairly decent starting point if the need ever came for
# that. Note that this is for a small single-server setup, a VPS or
# larger installation could make better use of different parameters.
#
# This script operates on several principles:
# 1) First, all known good data is whitelisted. This includes the local
#    interface, my chosen IP sets, and UDP and ICMP replies to packets
#    that we have sent out.
# 2) Second, all known bad data is dropped, along with all 'unknown'
#    sorts of data. This includes non-unicast packets, udp packets -
#    since I am not running BIND, and anything trying something goofy,
#    like spoofing my own addresses.
# 3) Third, TCP connections are themselves limited based on their host
#    block. A better limit would probably be 16 connections per /29,
#    but modern browsers are establishing more TCP connections now,
#    and AOL proxies do not play nice, at all.
# 4) Fourth, we treat standard tcp connection packets (ACK and no SYN,
#    FIN, or RST) as automatically valid and accept them normally.
# 5) Fifth, valid new connection packets are flood-limited and sent to
#    a managing chain, has some fun with port scans and limits ssh
#    spamming (assuming they ever actually manage to find the port).
# 6) Valid control tcp packets are sent to a flood limiter, which logs
#    overages. I don't auto-ban here as this is possible to trigger
#    legitimately, but instead I use this to monitor potential trouble
#    and tune my limits accordingly.
# 7) Finally, we accept rate limited incoming pings.
#
# 4-6 might be called "T-C-P Motherfucker Do You Speak It!?" - we do
# not blacklist known bad TCP packets. Rather, we whitelist good ones.
# This gives us better control as we can manage a packet based on what
# we expect it to be doing and how often we expect to see it.
#######################################################################
# This script was developed by Vekseid at http://elliquiy.com
# - vekseid@elliquiy.com
#
# It is released under the simplified BSD license:
#######################################################################
# Copyright (c) 2009, vekseid@elliquiy.com
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
#   notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
#   notice, this list of conditions and the following disclaimer in the
#   documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
# OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
# AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY
# WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#######################################################################
# This script would not have been possible without Oskar Andreasson's
# IPTables Tutorial, found at:
# http://iptables-tutorial.frozentux.net/iptables-tutorial.html
# I additionally made use of a good amount of the information in Jan
# Engelhardt's "Detecting and deceiving network scans", found here:
# http://jengelh.medozas.de/documents/Chaostables.pdf
#######################################################################

#######################################################################
# Define variables to make for easy tuning.
# IPT - Location of the iptables binary
# SELFIPS    - The server's allocated IP addresses (Elliquiy's, here)
# WHITELIST  - My own personal IP.
# WHITERANGE - A trusted subnet. Alternately, trust the interface
#              instead.
# BLACKLIST  - I had a single person attempt to DOS me so I set this.
#              Having not yet dealt with a genuine DDOS, I only have
#              a vague idea of what one would look like in terms of
#              what gets triggered. Thus, this script does not yet have
#              an automated protection routine.
# SSHPORT    - The port I have SSH set to.
# OPENPORTS  - Chosen ports to open. Many will want to add POP3.
# ALLOWPING  - Whether or not to allow public pings. I often have to
#              diagnose problems for my members so sure, why not : )
# CONNLIMIT  - The connlimit match declaration - it declares how many
#              tcp connections may exist for a given IP block.
#              set this too low and you are going to shut out a lot of
#              AOL users.
# USELOG     - Whether to use the basic log. Nothing in these logs are
#              fully reliable so I want to make them easy to disable.
# LOGLEVEL   - Logging line for basic logging.
# LOGLEVEL2  - Established TCP connections give more reliable data.
#              These logs are not able to be disabled in the
#              configuration - in the event of an attack I would use
#              them to pick off offending IP blocks.
#######################################################################
export IPT=/sbin/iptables
export SELFIPS="1.0.0.1-1.0.0.2"
export WHITELIST="1.0.0.0"
export WHITERANGE="192.168.0.0-192.168.255.255"
export BLACKLIST="0.0.0.0"
export SSHPORT=12345
export OPENPORTS="25,80,443"
export ALLOWPING=1
export CONNLIMIT="-m connlimit --connlimit-above 256 --connlimit-mask 29"
export USELOG=1
export LOGLEVEL="--log-level debug --log-ip-options"
export LOGLEVEL2="--log-level info --log-ip-options --log-tcp-sequence --log-tcp-options"
#######################################################################

#######################################################################
# Hashlimits have their own set of declarations.
# HASHLOG    - The hashlimit declaration for basic logging. It's rather
#              heavily limited in order to keep us from flooding.
# HASHLOG2   - Here we record people actually being suspicious.
# HASHLOG3   - 2 is for SYN/RST/FIN floods, 3 is for mass connections
# HASHSSH    - Not needed with this current setup, it exists more to
#              limit getting my auth log slammed than anything.
# HASHPING   - Limit pings from an IP to two per second.
# HASHSYN    - I split this off from the main flood hashlimit as
#              limiting SYN connections in this manner opens up another
#              DDOS window and I want to make that as small as possible
# HASHFLOOD  - This puts a cap on how fast we are willing to tolerate
#              SYN, RST and FIN packets from a client. This stops some
#              less intelligent DOS attempts.
#
# You will wish to tune the SYN and FLOOD rate limits based on your
# content. A lean CMS design may use fewer, while intense AJAX may
# require a higher limit. Obviously, do not set these elements blindly.
#######################################################################
export HASHPING="-m hashlimit --hashlimit-upto 2/second --hashlimit-burst 3 --hashlimit-mode srcip --hashlimit-srcmask 29 --hashlimit-name icmp --hashlimit-htable-size 8192 --hashlimit-htable-max 8192 --hashlimit-htable-gcinterval 1000 --hashlimit-htable-expire 2000"
export HASHLOG="-m hashlimit --hashlimit-upto 1/minute --hashlimit-burst 1 --hashlimit-mode srcip --hashlimit-srcmask 29 --hashlimit-name log --hashlimit-htable-size 8192 --hashlimit-htable-max 8192 --hashlimit-htable-gcinterval 3000 --hashlimit-htable-expire 60000"
export HASHLOG2="-m hashlimit --hashlimit-upto 1/minute --hashlimit-burst 1 --hashlimit-mode srcip --hashlimit-srcmask 29 --hashlimit-name log2 --hashlimit-htable-size 8192 --hashlimit-htable-max 8192 --hashlimit-htable-gcinterval 3000 --hashlimit-htable-expire 60000"
export HASHLOG3="-m hashlimit --hashlimit-upto 1/minute --hashlimit-burst 1 --hashlimit-mode srcip --hashlimit-srcmask 29 --hashlimit-name log3 --hashlimit-htable-size 8192 --hashlimit-htable-max 8192 --hashlimit-htable-gcinterval 3000 --hashlimit-htable-expire 60000"
export HASHSSH="-m hashlimit --hashlimit-upto 3/minute --hashlimit-burst 3 --hashlimit-mode srcip --hashlimit-srcmask 29 --hashlimit-name ssh --hashlimit-htable-size 8192 --hashlimit-htable-max 8192 --hashlimit-htable-gcinterval 3000 --hashlimit-htable-expire 60000"
export HASHSYN="-m hashlimit --hashlimit-upto 6/second --hashlimit-burst 360 --hashlimit-mode srcip --hashlimit-srcmask 29 --hashlimit-name syn --hashlimit-htable-size 8192 --hashlimit-htable-max 8192 --hashlimit-htable-gcinterval 3000 --hashlimit-htable-expire 60000"
export HASHFLOOD="-m hashlimit --hashlimit-upto 6/second --hashlimit-burst 360 --hashlimit-mode srcip --hashlimit-srcmask 29 --hashlimit-name flood --hashlimit-htable-size 8192 --hashlimit-htable-max 8192 --hashlimit-htable-gcinterval 3000 --hashlimit-htable-expire 60000"
#######################################################################

#######################################################################
# Here we set some variables that are not 'user modified'.
#######################################################################
export DROPTARGET=DROP
if [ $USELOG -eq 1 ] ; then
  export DROPTARGET=DRP
fi
#######################################################################

#######################################################################
# Flush current rules and reset policies.
#######################################################################
$IPT -F
$IPT -X
$IPT -P INPUT DROP
$IPT -P FORWARD DROP
$IPT -P OUTPUT ACCEPT
#######################################################################

if [ $USELOG -eq 1 ] ; then
  #####################################################################
  # Log dropped packets. Only make the chain if logging is on.
  #####################################################################
  $IPT -N DRP
  $IPT -A DRP $HASHLOG -j LOG $LOGLEVEL --log-prefix "Dropped Packet: "
  $IPT -A DRP -j DROP
  #####################################################################
fi

#######################################################################
# Accept from localhost, established/related udp and icmp connections,
# and our chosen whitelist. Drop invalid packets and non-unicast
# packets/sources outright, as well as killing funny business.
# Obviously, if you are running a DNS server, you will want to permit
# at least a limited amount of udp traffic.
#######################################################################
$IPT -A INPUT -i lo -j ACCEPT
$IPT -A INPUT -s $WHITELIST -j ACCEPT
$IPT -A INPUT -m iprange --src-range $WHITERANGE -j ACCEPT
$IPT -A INPUT -s $BLACKLIST -j $DROPTARGET
$IPT -A INPUT -p udp -m state --state RELATED,ESTABLISHED -j ACCEPT
$IPT -A INPUT -p icmp -m state --state RELATED,ESTABLISHED -j ACCEPT
$IPT -A INPUT -m state --state INVALID -j $DROPTARGET
$IPT -A INPUT -m pkttype --pkt-type ! unicast -j $DROPTARGET
$IPT -A INPUT -m addrtype ! --src-type UNICAST -j $DROPTARGET
$IPT -A INPUT -m iprange --src-range $SELFIPS -j $DROPTARGET
$IPT -A INPUT -p udp -m state --state NEW -j $DROPTARGET
#######################################################################

#######################################################################
# TCP Traffic rules for established and related connections. This chain
# exists specifically for established flood detection - people who show
# up in this log are not spoofing addresses, so I am more confident
# about logging them and assign them their own space accordingly.
#######################################################################
$IPT -N TCPER
$IPT -A TCPER -p tcp $HASHFLOOD -j ACCEPT
$IPT -A TCPER -p tcp $HASHLOG2 -j LOG $LOGLEVEL2 --log-prefix "TCP Flood: "
$IPT -A TCPER -p tcp -j DROP
#######################################################################

#######################################################################
# TCP Traffic rules for new packets. Since Lenny does not have the
# DELUDE or CHAOS targets we have to make due with the next best thing
# to be suitably annoying to port scanners. The hashlimit for SYN
# floods was placed before entering this chain, so we don't duplicate
# that here.
#######################################################################
$IPT -N TCPN
$IPT -A TCPN -p tcp --dport $SSHPORT $HASHSSH -j ACCEPT
if [ $USELOG -eq 1 ] ; then
  $IPT -A TCPN -p tcp --dport $SSHPORT $HASHLOG -j LOG $LOGLEVEL --log-prefix "SSH Flood: "
  $IPT -A TCPN -p tcp --dport $SSHPORT -j DROP
fi
$IPT -A TCPN -p tcp -m multiport --destination-port $OPENPORTS -j ACCEPT
if [ $USELOG -eq 1 ] ; then
  $IPT -A TCPN -p tcp $HASHLOG -j LOG $LOGLEVEL --log-prefix "TCP New Invalid Port: "
fi
$IPT -A TCPN -p tcp -m statistic --mode random --probability 0.03125 -j REJECT --reject-with host-unreach
$IPT -A TCPN -p tcp -j DROP
#######################################################################

#######################################################################
# TCP Connection limiting.
#######################################################################
$IPT -A INPUT -p tcp $CONNLIMIT $HASHLOG3 -j LOG $LOGLEVEL2 --log-prefix "TCP Connlimit Reached: "
$IPT -A INPUT -p tcp $CONNLIMIT -j REJECT --reject-with tcp-reset
#######################################################################

#######################################################################
# TCP flag whitelist. We only accept valid flag combinations. IE:
# "T-C-P Mother-fucker do-you-speak-it!?"
# The answer can very well be 'no', however, so we do want to be a bit
# forgiving in this regard. SYN/ACK and ACK packets may show up as NEW
# for legitimate traffic in many cases. It is also not uncommon to see
# PSH out of nowhere, however at least for me this is rare enough to be
# tolerable.
#######################################################################
$IPT -A INPUT -p tcp --tcp-flags SYN,ACK,FIN,RST ACK -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPT -A INPUT -p tcp --tcp-flags SYN,FIN,RST,PSH,URG SYN -m state --state NEW $HASHSYN -j TCPN
$IPT -A INPUT -p tcp --tcp-flags ACK,FIN,RST,PSH,URG ACK -m state --state NEW $HASHSYN -j TCPN
$IPT -A INPUT -p tcp --tcp-flags SYN,FIN,RST,PSH,URG SYN -m state --state ESTABLISHED,RELATED -j TCPER
$IPT -A INPUT -p tcp --tcp-flags SYN,FIN,RST,PSH,URG RST -m state --state ESTABLISHED,RELATED -j TCPER
$IPT -A INPUT -p tcp --tcp-flags SYN,FIN,RST,URG FIN -m state --state ESTABLISHED,RELATED -j TCPER
if [ $USELOG -eq 1 ] ; then
  $IPT -A INPUT -p tcp $HASHLOG -m state --state NEW -j LOG $LOGLEVEL --log-prefix "TCP Bad SYN: "
  $IPT -A INPUT -p tcp $HASHLOG -m state --state NEW -j DROP
  $IPT -A INPUT -p tcp $HASHLOG -j LOG $LOGLEVEL --log-prefix "TCP Bad Flags: "
fi
#######################################################################

#######################################################################
# Overall ICMP Rules
#######################################################################
if [ $ALLOWPING -eq 1 ] ; then
  #####################################################################
  # ICMP Incoming Chain. We only allow pings, and fragments are
  # discarded early.
  #####################################################################
  $IPT -N ICMP
  if [ $USELOG -eq 1 ] ; then
    $IPT -A ICMP -p icmp --fragment $HASHLOG -j LOG $LOGLEVEL --log-prefix "ICMP Fragment: "
  fi
  $IPT -A ICMP -p icmp --fragment -j DROP
  $IPT -A ICMP -p icmp --icmp-type echo-request -j ACCEPT
  if [ $USELOG -eq 1 ] ; then
    $IPT -A ICMP -p icmp $HASHLOG -j LOG $LOGLEVEL --log-prefix "ICMP Bad: "
    $IPT -A ICMP -p icmp -j DROP
  fi
  #####################################################################

  #####################################################################
  # Allow icmp packets at 2/second and log flood attempts.
  #####################################################################
  $IPT -A INPUT -p icmp -m state --state NEW $HASHPING -j ICMP
  if [ $USELOG -eq 1 ] ; then
    $IPT -A INPUT -p icmp -m state --state NEW $HASHLOG -j LOG $LOGLEVEL --log-prefix "PING Flood: "
  fi
  #####################################################################
fi
#######################################################################

#######################################################################
# This script was developed by Vekseid at http://elliquiy.com
# - vekseid@elliquiy.com
#######################################################################


After the issue with connlimit I don't bother with iptables-save or iptables-restore. Being able to actually reset the firewall, rules intact, is quite valuable and occasionally leads to happy members : )

Note that the proper way to start this is not in init.d but rather in /etc/network/interfaces using pre-up:


iface eth0 inet static
    pre-up /root/firewall.sh


For example. My personal philosophy would not to use hostnames, ever, but you'll be penalized even harder in this case.