Chapter 9. Compiling the Linux Kernel for LRP

Table of Contents

Kernel Modules
Specifying the Root Volume
Compiling Linux 2.0
Compiling Linux 2.4
Bridge Patch
Important Notes About Compiling a New Kernel

Compiling the Linux kernel does not require much in the way of extra steps, and is not hard at all for someone who is experienced in compiling the Linux kernel. This description will not conver compiling the Linux kernel, but rather what needs to be done to compile the kernel for LRP.

First come the patches. Two are required:

Then comes any additional patches desired. Materhorn and LRP have taken different approaches to kernel patches; Materhorn and Eiger appear to have many more patches available whereas LRP has kept the patches to a minimum.

Oxygen has taken yet another approach. The current Oxygen distribution (March 2001) is using Linux 2.2.18 with the OpenWall [17] patches to make the system more secure; Linux Intrusion Detection System (LIDS) [18] is another possible set of patches that are able to work with OpenWall to provide very solid kernel-based security. Another common patch is the VPN+Masq [19] patch.

Patching a kernel is much simpler than it might appear at first glance. A typical patch installation will be done within the Linux kernel source tree, at the top directory (typically /usr/src/linux), and this command will be executed: patch -p1 < ../mypatch.diff

At times, the patch may be called a *.patch (not entirely correct) instead of a *.diff — the name comes from the program used to create the patch (diff). The -p1 option specifies that the first directory name should be removed from the patch. This is normally desirable; occasionally (almost never) it will be necessary to use -p0.

It can shorten the time to configure a kernel if a currently existing kernel configuration is downloaded and used for the initial settings. Once the kernel configuration has been saved, this downloaded file is no longer needed except for backup.

Once patches have been applied, remember to check for options in the kernel configuration. For the LRP patches, select the following options in the Block Devices section:

<*> RAM disk support
(4096) Default RAM disk size
[*] Initial RAM disk (initrd) support
[*] Initial RAM disk archive (untar) support
[*] Initial RAM disk minix auto fs support
[ ] Initial RAM disk ext2 auto fs support (broken!)

There will probably be options for other patches as well; select the appropriate options as desired. As an example, consider Openwall — a new section in the main menu labeled Security appears:

[*] Non-executable user stack area
[*] Autodetect and emulate GCC trampolines
[*] Restricted links in /tmp
[*] Restricted FIFOs in /tmp
[ ] Restricted /proc
[*] Special handling of fd 0, 1, and 2
[*] Enforce RLIMIT_NPROC on execve(2)
[ ] Destroy shared memory segments not in use

Kernel Modules

Don't forget to configure the modules, such as networking and so forth. If this kernel is to be a distribution kernel of some kind, compile practically everything (as modules). Otherwise, selecting only that which is necessary for system operation will shorten compile time of the modules. Also, not compiling support for something will save space over compiling support with everything as modules.

A good size kernel will be about 480-490k; a "big" kernel is over 500-520k. The kernel is the second biggest thing on the disk; every byte counts — so it is important to try and keep it as small as practicable.

Be sure to not strip the modules; this will cause segmentation faults in insmod. It is critical the symbols are left in; take the modules unchanged from the kernel compilation.

Specifying the Root Volume

In LRP, the root volume is normally specified by the ROOT parameter in syslinux.cfg. However, this is not necessary as the rdev utility can be used on the newly created kernel to set an appropriate default inside the kernel itself. The kernel parameter ROOT overrides this value.

Likewise, the /linuxrc script could write to the file /proc/sys/kernel/real-root-dev the value of the real root device. For /dev/ram1, this would be 0x0101. The major and minor values can be gotten from a ls done on the file in /dev or by reading linux/Documentation/devices.txt in the Linux source code tree.

Compiling Linux 2.0

This version of the Linux kernel had some source code problems that were not uncovered until later versions of GNU gcc. According to the GNU gcc team, the only version of gcc which should be used to compile Linux 2.0 is gcc 2.7.2. Newer versions will generate errors about certain assembly language code and registers being clobbered.

There is a patch to fix this from Florian La Roche at SuSE: http://www.suse.de/~florian/kernel+egcs.html

Compiling Linux 2.4

This version of the Linux kernel has had some substantial growth; getting it down to under 600k takes some dedication and serious work. If designing a LEAF distribution for release, virtually nothing can be compiled into the core as it takes up just too much space. Modularize everything.

If compiling Linux 2.4 for a specific application, do the following:

  • Remove even modular support for anything you don't need. Even modular support takes up memory in the Linux core kernel.

  • Remove modules from packages on the boot disk if the support is contained within the new kernel core.

  • Remember to update kernel modules if needed.

Modules under Linux 2.4 no longer show up in linux/modules; to make this happen you need to "install" the modules to a directory of your choice under the linux tree. For example: make modules_install MODULE_INSTALL_DIR=/usr/src/linux/modules

Linux 2.4 also introduces the devfs (device filesystem) to Linux. This is not currently supported by LEAF systems as of January 2002. This may be supported in the future, but for now you must leave it off since no current systems are designed to use it.

It would also appear that the rtl8139.o driver has disappeared from Linux 2.4; there is now a 8139cp.o module and an 8139too.o driver module. Use these instead.

Patching the Linux 2.4 kernel is problematical; specifically, the initrd_archive and linuxrc_always patches are only available for kernels up to 2.4.5. The nature of the kernel apparently precludes updating these patches, and in any case the initrd RAMdisk boot loader is likely to change in the future. Thus, using any kernels beyond 2.4.5 requires that LRP patches are not used and requires a LEAF distribution that can handle an unpatched kernel. Other kernel patches such as the GR Security patch can still be used as long as they support the kernel version being used.

Bridge Patch

The bridge patch and bridge+firewall patches from Lennert Buytenhek will work fine with the current kernels. However, the new bridge utils will not compile against glibc 2.0, and thus require an upgrade to glibc 2.1. The older bridgex utilities from Matthew Grant may or may not work; this is untested.

Important Notes About Compiling a New Kernel

When compiling a kernel, it is recommended that you do not compile in the /usr/src/linux tree. Linus described it in a post July 27, 2000 this way:

From: Linus Torvalds (torvalds@transmeta.com) Date: Thu Jul 27 2000 - 02:39:51 EST

In article <Pine.LNX.4.02.10007270811460.32194- 101000@prins.externet.hu>, Boszormenyi Zoltan <zboszor@externet.hu> wrote:

/usr/include/asm is a symlink to /usr/src/linux/include/asm, as in the original distribution but usr/src/linux is a 2.4.0-testX tree. With a 2.2.X source tree, it does not produce any warning.

I've asked glibc maintainers to stop the symlink insanity for the last few years now, but it doesn't seem to happen.

Basically, that symlink should not be a symlink. It's a symlink for historical reasons, none of them very good any more (and haven't been for a long time), and it's a disaster unless you want to be a C library developer. Which not very many people want to be.

The fact is, that the header files should match the library you link against, not the kernel you run on.

Think about it a bit.. Imagine that the kernel introduces a new “struct X”, and maintains binary backwards compatibility by having an old system call in the old place that gets passed a pointer to “struct old_X”. It's all compatible, because binaries compiled for the old kernel will still continue to run - they'll use the same old interfaces they are still used to, and they obviously do not know about the new ones.

Now, if you start mixing a new kernel header file with an old binary “glibc”, you get into trouble. The new kernel header file will use the new “struct X”, because it will assume that anybody compiling against it is after the new-and-improved interfaces that the new kernel provides.

But then you link that program (with the new “struct X”) to the binary library object archives that were compiled with the old header files, that use the old “struct old_X” (which used to be X), and that use the old system call entry-points that have the compatibility stuff to take “struct old_X”.

Boom! Do you see the disconnect?

In short, the only people who should update their /usr/include/linux tree are the people who actually make library releases and compile their own glibc, because if they want to take advantaged of new kernel features they need those new definitions. That way there is never any conflict between the library and the headers, and you never get warnings like the above..

Mr. Ulrich Drepper (one of the glibc/gcc guys) gave me a standard “don't use kernel headers directly” answer. But neither gcc.c, neither the above small program use kernel headers. I suppose he referred to /usr/include/linux/* as (I think) he did not understand me.

No. He really meant that you should not use the kernel headers: you should use the headers that glibc came with. It is probably a redhat bug that those headers were a symbolic link.

I would suggest that people who compile new kernels should:

  • NOT do so in /usr/src. Leave whatever kernel (probably only the header files) that the distribution came with there, but don't touch it.

  • compile the kernel in their own home directory, as their very own selves. No need to be root to compile the kernel. You need to be root to install the kernel, but that's different.

  • not have a single symbolic link in sight (except the one that the kernel build itself sets up, namely the “linux/include/asm” symlink that is only used for the internal kernel compile itself)

And yes, this is what I do. My /usr/src/linux still has the old 2.2.13 header files, even though I haven't run a 2.2.13 kernel in a loong time. But those headers were what glibc was compiled against, so those headers are what matches the library object files.

And this is actually what has been the suggested environment for at least the last five years. I don't know why the symlink business keeps on living on, like a bad zombie. Pretty much every distribution still has that broken symlink, and people still remember that the linux sources should go into “/usr/src/linux” even though that hasn't been true in a loong time.

Is there some documentation file that I've not updated and that people are slavishly following outdated information in? I don't read the documentation myself, so I'd never notice ;)

Linus

To summarize, the recommendation then, is to compile Linux kernels in a directory other than /usr/src/linux. This way the kernel headers that were designed for your system remain intact even as you compile new kernels.