Chapter 4. Webconf Architecture How To

Revision History
Revision 1.12006-01-28kp
???

Table of Contents

Introduction
Installation
Building Your Own Webconf Plugin
Packaging Your Own Webconf Plugin
Recommendations

Introduction

This chapter explains the architecture and development guidelines for a web configuration interface. The framework as a whole is called "webconf." The concepts behind Webconf started with the work of Cameron Banta and David Krause, who wrote a template-based configuration engine using mawk. Travis Colbert extended the templates beyond what most thought was possible and proved by example that something more flexible was needed.

What is webconf?

"Webconf" is a framework, or concept. webconf.lrp is an implementation of that framework that exposes the lrcfg.* shell scripts through a web-browser.

The weblet web application has been around since August 2000. The focus of weblet is to provide read-only status information on a LEAF router. Through the use of plugins, other developers can extend weblet, and these extensions appear in a "plugins" area.

The aim of webconf (the concept) is to provide read/write access to a LEAF router using a web interface, to do so with a consistent look-and-feel, and to integrate 3rd party plugins in a way that is transparent to the end-user.

The webconf framework consists of a number of shell scripts and a c-language program that provide an interface for configuring services through a web browser. The framework can determine which packages (lrps) are installed, can present a menu of packages to be configured, can handle restarting services, and process user input.

The framework requires a web server that can handle POST requests and the CGI 1.1 interface, so mini_httpd, thttpd, or a similar web server is required. The sh-httpd server does not currently support POST requests.

The webconf framework was written because other interfaces like webmin require Perl or PHP, which are typically too large to fit in an embedded environment. The goal is to provide the most flexibility in configuring a LEAF router, with the least amount of weight.

Installation

This section describes how to install the webconf.lrp package. This package contains a fully-functional set of menus that provide the monitoring functions of weblet plus web pages that configure and back up installed packages. In addition, webconf-style .lrps are available for some of the common weblet plugin .lrps.

The reference design and webconf.lrp are built for the current Bering-uClibc release. In addition to the shell scripts, the webconf.lrp uses a dynamically linked /usr/bin/haserl; if you want to try webconf on a platform other than uClibc, you will need to recompile the haserl interpreter for your platform. Haserl source code is available at http://haserl.sourceforge.net

Getting webconf installed and running

Follow these steps to get started:

install mini_httpd (mhttpd.lrp)

Reconfigure mini_httpd; it should process any file named .cgi as a cgi. Additionally, if you wish to run weblet in addition to webconf, change the default port of mini_httpd. Here's an example of the /etc/mini_httpd.conf:

  	port=8080
  	cgipat=**.cgi
  	user=sh-httpd
  	logfile=/var/log/mini_httpd.log
  	pidfile=/var/run/mini_httpd.pid
  	max_age=0
	

Change mini_httpd to serve docs from /var/webconf/www. The WWWDIR variable defined in /etc/init.d/mini_httpd determines the document root. Change this from /var/sh-www to /var/webconf/www.

install webconf.lrp

Start mini_httpd: /etc/init.d/mini_httpd start

Building Your Own Webconf Plugin

At least two files are needed to provide new functionality:

A file in /etc/webconf that provides meta-information for the web interface. This file is named <package>.webconf.

The .cgi(s) that actually perform the configuration.

.webconf files

The /etc/webconf/<package>.webconf file describes how the various cgi scripts in the package plug into the webconf menu system. Here is an example of a .webconf file:

	displayname:General
	all:Log Files:/logfiles.cgi
	all:General Health:/general-info.cgi
	all:Active Connections:/connection-info.cgi
	#
	displayname:Save To disk
	all:Backup Packages:/lrcfg.back.cgi
	expert:Destinations:/lrcfg.pkgdisks.cgi
	expert:Edit leaf.cfg:/leafcfg.cgi
	

The first item in the file should be a tag for the "displayname", or the title of the menu category. It is possible to have more than one main category - In the example above, there are two: "General" and "Save to Disk".

After each main category are the menu options within that category. Each line consists of three fields:

1. A tag identifying which "menu-set" this menu option belongs to. In the reference design, the choices are "basic", "expert", or "all", but any number of menu-sets can be defined. The menu options are only displayed if the user has chosen that menu-set. For example, "expert" options are only visible when the user chooses the "expert" menu set. The "all" menu-set tag means that the menu option is displayed in all menu sets.

2. The text shown for the menu option

3. The script to run. The script must end in .cgi, and the path is relative to the document root (/var/webconf/www.)

The example above displays a menu that looks like this:

	General
	   Active Connections
	   General Heath
	   Log Files
	Save To Disk
	   Backup Packages
	   Destinations
	   Edit leaf.cfg
 	

The menu options are sorted, so Active Connections comes first, even though it is defined last in the "General" section. The sort order can be controlled as described later on.

The /var/webconf/lib/menubuilder.sh script takes all /etc/webconf/*.webconf files and sorts them; so if there are two .webconf files that have definitions for a displayname of "General", options from both .webconf files will be combined under one "General" heading.

All options (displayname and menu options) are sorted. In order to control the sorting order, numbers can be prepended to the name. The number will control the sort order, but will not be displayed.

Here is an example of three webconfs using the numeric sorting order, and how they are displayed:

 	/etc/webconf/network.webconf
  	displayname:20Networking
  	all:10Interfaces:interfaces.cgi
  	all:20Traffic Shaping:tc.cgi
 
 	/etc/webconf/shorwall.webconf
  	displayname:30Firewall
  	all:20Advanced:shorewall.adv.cgi
  	all:10Basic Setup:shorewall.cgi
 
 	/etc/webconf/ezipupd.webconf
  	displayname:20Networking
  	all:50Dynamic DNS:ezipupd.cgi
	
 	Networking
   	   Interfaces
   	   Traffic Shaping
   	   Dynamic DNS
 	Firewall
   	   Basic Setup
   	   Advanced
	

Warning: It is possible for one developer to have 20Networking and another developer to specify 30Networking. The net result is two menu headings with the name Networking.

The Configuration Scripts (.cgi)

The .webconf files serve as pointers to the .cgi scripts located in /var/webconf/www. The .cgi files must have execute permissions and be able to be run by the web server.

Typically the .cgi scripts are written in haserl (http://haserl.sourceforge.net) script. This cgi interpreter automatically populates shell variables based on the POST or GET data received from the client.

Haserl Basics

Haserl was written for LEAF routers because PHP is a fast way to write server-side interpreted code, but is too big for a diskette based Linux distro. After several attempts to write a web configuration framework with sh, awk and c programs, haserl was written to provide the bare minimum to fuse HTML with a shell interpreter. This section is not a tutorial on haserl, but provides some basic information to help explain the code in the existing cgi's.

Haserl automatically parses POST and GET requests and exposes any form elements or client-provided data as shell environment variables. For instance, http://some.host.com/index.cgi?myvar=3&yourvar=3 will automatically cause any shell script that you write to have two variables defined: FORM_myvar (set to 3), and FORM_yourvar (set to 2).

Anything inside <? ... ?> tags is shell script. To verify the above really does define FORM_myvar, you could write HTML code like this:

	<p>Your variable "FORM_myvar" is <? echo -n $FORM_myvar ?></p>
	

A limitation of early versions of haserl was each code block was a separate sub-shell. This is no longer the case; there is one sub-shell that is started at the beginning of the script.

Haserl needs to use file descriptor 5 for communication with the sub-shell. You should not close or write to fd 5. (i.e. you should never do something like this in your shell code: echo "hi mom" >&5 It will cause your cgi to fail)

Haserl is installed suid root. When installed this way it sets its uid/gid to the owner of the .cgi script. So if you need a script to do things "root" does, make the file owned by root:root. Other cgi scripts that don't need root access can be owned by by nobody:nogroup or sh-httpd:nogroup.

If you want the client to be able to upload a file, you must include -u on the command line, like this:

#!/usr/bin/haserl -u

Webconf Helper Functions

A number of shell script libraries are available to ease writing .cgi scripts. These libraries also ensure a consistent look-and-feel. The scripts mentioned here are in the /var/webconf/lib directory.

preamble.sh and footer.sh

Two helper scripts are provided to write the leading and trailing HTML necessary for each page. preamble.shdisplays the HTTP and HTML header information. preamble.sh also calls the menubuilder.sh script to display the menu. footer.sh writes the information necessary to close the HTML page.

Here is an example of the minimum necessary to generate a .cgi that looks like all other .cgi's in the webconf.lrp:

 	#!/usr/bin/haserl
 	<? title="Sample Script" /var/webconf/lib/preamble.sh ?>
 
 	<h1>Sample</h1>
 	<p>This is a sample .cgi script.  It doesn't do anything.</p>
 
 	<? /var/webconf/lib/footer.sh ?>
 	

The footer.sh can print your own contact information at the bottom of the footer. To do so, simply add your message as a parameter to footer.sh. Enclose your comment in quotes; footer.sh will list only up to the first unprotected space.

Here's how you can include your own contact information:

	<?	 /var/webconf/lib/footer.sh "This plugin maintained by \
           <a href=\"mailto:me@somewhere.com\">J. R. Hacker</a>" ?>
	

The custom message can contain html (such as color highligting) but please note that this text is inside a table; div tags (such as <h1> should not be used.

menubuilder.sh

The menubuilder.sh script is responsible for building the menu on the left of the page. It is automatically called by preamble.sh. The "menu-set" (expert, basic, etc.) is passed as the first parameter.

svcstat.sh

The svcstat.sh script provides a common way to present the status of a daemon, and buttons to change the daemon's state. This script requires 3 or 4 parameters:

1.The command - either "", "start", "stop", or "restart". This is the command button the client selected. (This corresponds to running /etc/init.d/<service> command ). Even if no button was pushed, a null parameter "" is required.

2.The service - the name of the script in /etc/init.d that is run.

3.The process name - This is the name of the process that is checked to see if the service (daemon) is currently running. For instance, the /etc/init.d/ script is named "webserver", but the process name is "mhttpd". In other cases, there is no process id to check. For instance, /etc/init.d/networking does not have a process id. In this case, the process name must be "-" (dash), and parameter 4 is the command to run to check for a running process.

4.Optional command If the previous parameter is a "-", then this is the command to run to check if the process is running. The command should print output if the service is running, and print nothing if it is not. For instance, to check if networking is running, you could use: '/sbin/ip addr sho eth0 | grep inet'.

Here are typical examples of svcstat.sh use:

 	<? /var/webconf/lib/svcstat.sh "$cmd" squid squid ?>
 	<? /var/webconf/lib/svcstat.sh "$cmd" networking - '/sbin/ip addr sho eth0 | \
	 	grep inet' ?>
	

validator.sh

The validator.sh file contains common functions that can be used to validate web forms. It also contains two functions (report_validation_fail and report_validation_ok) that provide a consistent wat of reporting whether a form contains validated data or not.

The file is usually sourced inside a haserl code block when it is used:

	<? . /var/webconf/lib/validator.sh
	echo "$myinputtextbox" | dostounix >/tmp/$SESSIONID.textbox
	?>
 	

widgets.sh

widgets.sh contains "eye-candy" functions - such as the LED meter in the general-health web page. This function library is also usually sourced inside a code block when needed.

Packaging Your Own Webconf Plugin

Once you have your webconf working, you will want to package it for future use (and sharing with others!). The webconf.lrp is configured to exclude the /etc/webconf and /var/webconf directories, so your changes will not be backed up in any existing package. You must create a package to be able to load the webconf plugin later. There are two ways to do this:

Package as a LWP

If your webconf plugin consists only of /etc/webconf/*.webconf and /var/webconf/www/* files, the recommended packaging is as a Leaf Webconf Plugin. This is because the plugin does not have configuration data that changes and all of the files are in directories that are excluded from other backups. The .lwp package is a tar.gz file with the plugin files. No "lrp" related meta-information is needed.

Name your .lwp the same as the package name. The webconf init script normally looks for .lwp files for all the installed .lrp packages.

Here's how you can create a LWP for dropbear:

	cd /
	tar zcvf ~/dropbear.lwp etc/webconf/dropbear.webconf var/webconf/www/dropbear.cgi
	

You can then copy the dropbear.lwp to your boot media.

Since your .lwp file will share the same name as the man .lrp package, please include your contact information somewhere in the .cgi. For instance, include your contact infromation in the footer.sh tagline. (See footer.sh above.) This is a courtesy so that the package maintainer doesn't get questions based on your plugin.

Package as a LRP

If your webconf plugin consists of files that exist outside of the /var/webconf/* directory tree, you must create a standard .lrp file. This is to ensure that the files from the plugin are not absorbed into another .lrp when a backup is done.

An example of a plugin that must be a .lrp is lrpstat. Lrpstat installs a program in /usr/sbin; this would be absorbed by root.lrp if it is not listed in another /var/lib/lrpkg/*.list file.

When creating a .lrp style plugin, you must add the package to the leaf.cfg file. The Webconf init script will not automatically load .lrps.

Recommendations

Many of these recommendations are intended to allow individual cgi scripts to communicate with each other. By having a common language, it is possible to have one web-page (.cgi) query another for information. This makes it possible to build web based "wizards" for setting up a LEAF router.

It is recommended that the form variable "cmd" be used for the function a .cgi should perform. For instance, when using svcstat.sh "cmd" can be "", "Stop", "Start", or "Restart". Other useful values for "cmd" can be "Save" and "Cancel"

Since parsing the conf files is unique to each package, it may be helpful if each webconf cgi has a standard way of reports its current settings. Something like this:

 if [ "$cmd" = "List" ];
    then cat <<-EOF <!-- #BEGIN WEBCONF SETTINGS LIST $( cat
    /tmp/$SESSIONID.vars ) #END WEBCONF SETTINGS LIST --> </pre> fi
    

To get the current settings, use a construct like this:

 cmd=List /var/webconf/www/dropbear.cgi | \ sed -n
    "/^#BEGIN WEBCONF SETTINGS/,/^#END WEBCONF SETTINGS/p" > /tmp/tempfile
    . /tmp/tempfile 

This is helpful when one webconf needs to know the values in another; such as the networking.cgi needing to know from shorewall which interface is in the "loc" zone.