Chapter 17. IPSec configuration

Revision History
Revision 0.22002-04-14CC
initial revision

Table of Contents

Step 1: load ipsec or ipsec509 package
Step 2: generate certificates with openssl
Step 3: boot Bering and move certificates into place
Step 4: configure ipsec.conf
Step 5: configure ipsec.secrets
Step 6: configure Shorewall
Step 7: configure Windows 2000 client


This document assumes that you have a Bering Firewall with an internal interface on eth1 and an external interface on eth0, and that you want to accept IPSec connections from Windows 2000 machines ("roadwarrior" clients or gateways for subnets) on the external interface, then treat those external clients or subnets as members of your internal network.

Also, there is a sizeable portion of this document that covers the configuration of the Windows 2000 IP Security Policy Utility. Please do not let this part slow you down if you are not interested in interoperating with Windows 2000 clients. It is extremely long, and I only wrote it down because most of what I found on the internet about it was pure "click here - click there" stuff and didn't really explain what was going on or the ramifications of "clicking there." I spent a lot of time trying to figure out the dark mysteries of their user interface, so hopefully, no one else will have to wear out their mouse finger trying to do so.

There are more complex configurations than this, which you should be able to understand better after reading.

Comments on ths section should be addressed to its maintainer: Chad Carr .

Step 1: load ipsec or ipsec509 package

Copy the ipsec.lrp or ipsec509.lrp package to the floppy. Also, you must copy the mawk.lrp package since it is needed by the ipsec scripts. You do not need the ifconfig.lrp package

You may need some space to store the packages and the ipsec module and generally a single floppy won't be enough. Check the Bering user's guide section about "Booting Bering from different boot-media" for tips.

Modify the syslinux.cfg file to load the new packages. It might look like this:

display syslinux.dpy
timeout 0
default linux initrd=initrd.lrp init=/linuxrc rw root=/dev/ram0 boot=/dev/fd0u1680:msdos
        PKGPATH=/dev/fd0u1680 LRP=root,etc,local,modules,iptables,pump,keyboard,shorwall,ulogd,dnscache,weblet,ipsec,mawk


The last two lines ("default linux ... ipsec,mawk") must be typed as a single one in syslinux.cfg

Copy the ipsec.o module from the modules package which matches your kernel. Don't even try to do this with mismatching modules, kernel or ipsec utilities!

Install this module using the method described in the main Bering documentation.

Step 2: generate certificates with openssl

Certificates usually need to be generated from the host machine since the router usually doesn't have enough randomness to generate them easily. I use the Debian package, and I assume there is a RedHat package. Here is a link to a document decribing how to compile it from source.

If you do not desire to use certificates (you only wish to use preshared keys) you may skip to Step 4

Make a new certificate authority

# mkdir -p demoCA/private; mkdir -p demoCA/newcerts;
# touch demoCA/index.txt; echo 01 >> demoCA/serial; chmod -R 700 demoCA
# openssl req -x509 -days 3650 -newkey rsa:2048 -keyout demoCA/private/cakey.pem -out demoCA/cacert.pem
# openssl ca -gencrl -out crl.pem

Make your ipsec server certificate

# openssl req -newkey rsa:2048 -keyout serverKey.pem -out serverReq.pem
# openssl ca -policy policy_anything -in serverReq.pem -days 1825 -out serverCert.pem -notext
# openssl x509 -in serverCert.pem -outform DER -out x509cert.der
# fswcert -k serverKey.pem > ipsec.secrets

Make your client certificates

# openssl req -newkey rsa:2048 -keyout clientKey.pem -out clientReq.pem
# openssl ca -policy policy_anything -in clientReq.pem -days 1825 -out clientCert.pem -notext
# openssl pkcs12 -export -inkey clientKey.pem -in clientCert.pem -certfile demoCA/cacert.pem -out clientCert.p12

Put all of this onto your Bering floppy or compact flash card, unmount it and boot it

Step 3: boot Bering and move certificates into place

Put cacert.pem onto your Bering box in the /etc/ipsec.d/cacerts directory (you will have to create this with mkdir). Put crl.pem into the /etc/ipsec.d/crls directory (make this one, too). Put x509cert.der into /etc. Get the info in ipsec.secrets into your /etc/ipsec.secrets file like so:

# echo ipsec.secrets >> /etc/ipsec.secrets

Step 4: configure ipsec.conf

An ipsec.conf file, you'll find, is a very personal thing. A very vanilla setup using preshared keys would look like the following:

config setup

conn %default
	left=<router ip address>
	leftsubnet=<internal subnet>

conn w2k-road-warriors

There is really no substitute for reading the man page, however.

With certificates, the same setup would look like this:

config setup

conn %default
	left=<router ip address>
	leftsubnet=<internal subnet>
	leftid="C=US, ST=CA, O=ipsecgw, CN=me, Email=you@yourdomain"

conn w2k-road-warriors

A couple of things to watch out for. 1) Do not put apostrophes or single quotes in any of your distinguished name fields! It causes blindness and other very bad things to happen. 2) Make sure that the date on your router is between the notBefore and notAfter dates on all your certificates!

Step 5: configure ipsec.secrets

If you wish to use preshared keys, your ipsec.secrets should look like the following:

%any <router ip address>: PSK "<your preshared key>"

if you are dealing with roadwarriors with dynamic ip addresses. If you know the ip address of the endpoint and you do not wish to share the same key amongst multiple roadwarriors, you have the option of specifying the ip address instead of "%any." If you don't want to share keys, and you don't kow the ip addresses of your clients, certificates are your only real option.

Your ipsec gateway's certificate can either have its private key extracted (using fswcert as in Step 2) and put in the ipsec.secrets file, or it can be stored in the /etc/ipsec.d/private directory (in either der or pem format) and be referenced in ipsec.secrets by filename with an optional passphrase.

If you choose to extract the key and keep the whole thing in ipsec.secrets directly, your ipsec.secrets file will look like this:

: RSA {
        Modulus:         0xB664D963F28A...
        PublicExponent:  0x010001
        PrivateExponent: 0x518CA9BE0C55...
        Prime1:          0xED48CBD214FC...
        Prime2:          0xC4C7B7244774...
        Exponent1:       0x314D4BD435BA...
        Exponent2:       0x6237A8E2B3C3...
        Coefficient:     0xCEA15F52310E...

Except the long strings of gibberish will be much longer. The : RSA must start at the left margin, but every other line must be indented (spaces or tabs will do). The file MUST have no more than 700 permissions and be owned by root to be secure.

Otherwise, put the private key (serverKey.pem from Step 2) in /etc/ipsec.d/private, secure it with an optional passphrase (recommended) and reference it in the ipsec.secrets file like so:

: RSA serverKey.pem "<optional passphrase>"

Step 6: configure Shorewall

You need to add a new zone to shorewall to handle hosts that connect through ipsec, and also add a tunnel definition to allow the udp port 500 traffic for Internet Key Exchange (IKE) and protocols 50 and 51 (ESP and AH) that are used for the IPSec payloads.


You must not turn on route filtering for any interfaces involved in ipsec. The "Bering recommended" way to turn this off is to use the /etc/network/options file and change the "spoofprotect" parameter to "no"

Add the gw zone to the /etc/shorewall/zones file

gw	ipsec0

Then use an entry like this in the /etc/shorewall/tunnels file

ipsec	net	gw

Use the ip address of the ipsec endpoint if you have it, because that will be more secure.

Step 7: configure Windows 2000 client

Configuring Windows to do this same thing is much harder. I would say that until you have done it properly once, it borders on black magic. Even if you have done it properly once, if the configuration is even slightly different and you didn't take the time to really understand it the first time, you are in for another rough ride. The way your mouse finger feels after clicking your way through the dialogs for this configuration is just another symbol of how most complicated things are easier and more user friendly in Linux.

It helps to have a custom "management console" when you're dealing with ipsec. You can put this on the desktop or someplace else convenient, and save your mouse finger from exhaustion clicking through menus to find things. Use the following steps:

Start - Run - mmc Console - Add/Remove Snap-in Add - Certificates Add - Computer Account - Local Computer - Finish Add - IP Security Policy Management Add - Local Computer - Finish - Close - OK Console - Save - <wherever you want to put it>

You can just double click on the icon this creates to open the custom console from now on.

In order to configure Windows 2000, there are several basic entities that you must understand. It is easy to get lost in all the clicky-clicky:

	IP Security Rules                - the highest level of granularity.
	                                   IP Security Rules are composed of:
		an IP Filter List        - which packets match the rule?  An
		                           IP Filter list is composed of:
			Filters          - traditional ip address, subnet mask,
					   protocol or port filtering, like
		a Filter Action          - what do we want to do with those
					   packets? Encrypt? Sign?  A Filter
					   Action is composed of:
			Security Methods - Different negotiable combinations of
					   signing and encrypting.  FreeS/WAN
					   works in ESP mode with 3DES
					   encryption and MD5 signing.  This is
					   a custom setting in Windows.
		Authentication Methods   - how do we authenticate the players?
					   Windows can do Kerberos, x.509
					   certificates from a CA (that can be
					   you!) or preshared keys
		a Tunnel Setting         - is this a tunnel?  what is the
		                           endpoint IP Address?
		a Connection Type        - does this IP Security Rule apply to
					   all network connections, or just lan
					   or dialup connections?

Also, for Windows 2000, you must have the Service Pack 2. It will not do the required 3DES encryption without it. You can get it from:

Pretty hairy. For what I wanted to do (tunnel mode ESP with MD5 signing and preshared key authentication) I had to set up two rules, one for inbound traffic specifying the Windows client IP address as the endpoint of the tunnel, and one for outbound traffic specifying the router as the endpoint of the tunnel. I did not want to have to know the IP address of the client, since I want to use DHCP to deliver these addresses, but I haven't worked a way around it yet. Maybe if some Windows people are reading this they can drop a line...

Configure the Windows 2000 client

	a) run the custom mmc console you just made
	b) click on ipsec security policies in left pane
	c) action - create IP security policy
	d) next, choose name (Win2k to FreeS/WAN), uncheck default response
	   rule, check edit properties, finish
	e) add IP security rule to grab outbound traffic and tunnel it to
	   FreeS/WAN using 3DES and MD5
	f) next, enter tunnel endpoint (<router IP Address>), lan connection,
	   preshared key(<your preshared key>)
	g) add both ip filter lists for inbound and outbound traffic, then you
	   can just click on inbound traffic when you're defining that security
	h) add ip filter list, name: "outbound traffic", add filter
	i) next, src: my ip address, dest: any ip address, any proto, finish
		note: My setup is made to tunnel ALL ip traffic through my
		router.  If you are just tunneling traffic to one subnet, you
		should specify that here with the network address and subnet
	j) add another filter list, name "inbound traffic", add filter
	k) next, src: any ip address, dest: my ip address, any proto, finish, close
		note: see note above
	l) select the "outbound traffic" filter list, next
	m) add filter action to encrypt and authenticate with freeswan (3DES and MD5)
	n) next, name "freeswan compatible", negotiate, do not communicate
	   non-ipsec, custom (ESP, MD5, 3DES), edit properties, finish
	o) uncheck allow unsecured but always respond..., check perfect forward
	   security, OK
	p) select the "freeswan compatible" filter action
	q) uncheck edit properties, finish

	The next one is easier, because you have already defined the filter
	lists and filter action during the previous wizards, so you can just
	select them to apply them to the inbound traffic IP Security Rule.
	I'll go through this quickly, but just hit next or okay if I've skipped
	a step.

	r) add another IP security rule
	s) next, tunnel endpoint (<client IP Address>) lan connection,
	   preshared key (<your preshared key>), inbound traffic, freeswan
	   compatible, finish
	t) general tab, advanced, check master key perfect forward security,
	   close, close

You should be done. Right click the "Win2k to FreeS/WAN" IP Security Policy and click assign in the context menu. Bring up a DOS window. Ping your router. If everything is correct, you will get "Negotiating IP Security." as the response to the first four pings, then should be pinging clear after that.

If not, double click on the "Win2k to FreeS/WAN" IP Security Policy to reenter configuration dialogs. You will see the two IP Security Rules you just created. Double click on one of them to check the configuration.

You will see five tabs at the top of the dialog corresponding to the items described at the beginning of this section. First check the "outbound traffic" filter list, then the "inbound traffic" filter list. Double click on them to enter the configuration dialog, then double click the Filter. Do they have the right source and destination addresses? Remember, this will match packets similar to the way ipchains rules do, so if the rule doesn't match properly, then you will not "forward" the packet to the Filter Action, and it will not get encrypted properly.

If the Filter Lists are both okay, move on to the Filter Action. We have the same Filter Action for both IP Security Rules, so we just have to make sure that it says to negotiate security with ESP, 3DES and MD5. We should also ensure again that "Accept unsecured communication..." and "Allow unsecured communication..." are unchecked, 'cause those are not going to accomplish what we are trying to get, and that "Session key Perfect Forward Secrecy" is checked.

Check the Authentication Method to make sure that the shared key is exactly the same as the ipsec.conf file and that there is not a carriage return at the end.

Check the tunnel setting and make sure that the tunnel endpoint is the router ip address for the "outbound traffic" Filter List and the ip address of the Windows 2000 Client for the "inbound traffic" list.

The Connection Type should be LAN only. We don't want to inadvertently try to encrypt our dialup sessions, do we?

If any of these things was wrong, you will have to restart the IPSEC Policy Agent service by clicking Start - Control Panel - Services, right clicking on the service and clicking Restart.