Written by Bjorn Ekwall <bj0rn@blox.se> in the spring of 1995
(updated in January 1996)

How to modularize
=================

There are some guidelines for modularizing; some of them are
easy to implement but there are a couple of tricky parts as well.

The things to watch out for is that the module-to-be should
really *be* a module. i.e. an entity that supplies a well
defined *separate* functionality.

It should also have a "clean" interface to the rest of the
kernel, i.e. it should not demand that a whole bunch of
new symbols should be added to "linux/kernel/ksyms.c".


The "easy" parts:

	1. Make sure that the first include is this line:
		#include <linux/module.h>
	   Only <linux/config.h> or <linux/autoconf.h> should
	   be included before these lines (see "Note 2" for execptions).
	   This will ensure that the "symbol versions" will
	   work correctly, as well as supplying the necessary
	   module definitions that you will need later on.

	 Note 1:
	   For newer kernels, <linux/version.h> is included
	   by <linux/module.h>, so you don't have to.
	   For older kernels, you will also have to include <linux/version.h>.
	   If you get complaints about "kernel_version" being multiple
	   defined, just remove the "#include <linux_version.h>", or,
	   alternatively, add "#define __NO_VERSION__" before you include
	   <linux/module.h>, or add "-D__NO_VERSION__" to the Makefile CFLAGS.

	 Note 2:
	   If you compile a module that is _not_ included in the
	   kernel tree, and you want to use versioned module symbols, you
	   should also add the following to your CFLAGS in the module Makefile:
	     "-DMODVERSIONS -include /usr/include/linux/modversions.h"
	   Of course you will also have to have "-DMODULE" in CFLAGS as well.

	   Some older versions of gcc had trouble with "-include"; in that
	   case you will have to add "#include <linux/modversions.h>" to
	   the module instead.
	   In this case _only_: add this _before_ <linux/module.h>.


	2. Identify the spots where the module functionality
	   is requested and released.  Usually you will find
	   these spots in "open_*()" and "close_*()".
	   Every time the module supplies a "resource", add:

	   	MOD_INC_USE_COUNT;

	   and every time it releases the resource, add:

	   	MOD_DEC_USE_COUNT;

	   Without these safeguards you will risk the sanity
	   of the module as well as of the kernel.
	   Check the logic so that the INC's and DEC's are
	   balanced, since the "hidden" count must be zero
	   for the module to be unloaded.


	3. Lastly, add the "standard module definitions",
	   using this skeleton:

		#ifdef MODULE

	...
	/* driver specific module definitions */
	...

		int
		init_module(void)
		{
	...
	/* driver specific inititialization, usually "register_*()" */
	...
			if (fail)
				return -EIO;
	...

			return 0;
		}

		void
		cleanup_module(void)
		{
	...
	/* driver specific cleanups, ususally "unregister_*()" */
	...
		}
		#endif /* MODULE */

	
	   Note that due to historical reasons, some modules might still
	   include something like this in their "cleanup_module":

		if (MOD_IN_USE)
			printk("xxx: device busy, remove delayed\n");
		else
		/* driver specific cleanups, ususally "unregister_*()" */
	
	   With recent kernels this is taken care of automatically,
	   so you won't need it.
	   If you feel adventurous, you can clean up your old modules
	   by removing this historic check, but then of course you
	   have also said goodbye to Linux 1.0.* ...


	4. Update the Makefile so that the module will be compiled
	   with the correct flags and so that it will be automatically
	   linked to the correct place in the directory tree.
	   You can take a look at "linux/drivers/net/Makefile" for
	   an example.  If you are lucky, you just add a line or two,
	   where the magic line is:

	   	M_OBJS += my_module.o

	   or (if you use "register_symtab"):

	        MX_OBJS += my_module.o
	   
	   If your Makefile is not already prepared for modules,
	   you will have to modify (or add) the relevant targets.
	   As before: "the previous source is your friend" :-)


The "hard" parts:

	Now you have a potential module that you can try out
	by doing "make modules" in the kernel root directory.

	The compilation will usually go without problems,
	but just wait...

	The command "insmod my_module.o" will more often than
	not protest and give you a list of undefined symbols.

	Whatever you do, *do not* start to add symbols to
	"linux/kernel/ksyms.c" !!!

	Often these "errors" come from the fact that the module
	doesn't have a "clean" interface to the kernel.
	Kernel resident drivers will get their symbols resolved
	from the complete set of kernel symbols, but the module
	only has the exported symbols available.
	Hidden interdependencies between parts of the kernel
	will now be seen as these missing symbols.

	The solution is to clean up the interface, instead
	of just making the dependencies permanent by adding
	a symbol to "linux/kernel/ksyms.c".

	Two examples:

	- In kernels before 1.1.85 there was a function in
	  slhc.o, "ip_csum", that prevented that module to
	  be cleanly loaded. Instead of just adding this
	  symbol to ksyms.c, I noticed that "ip_csum" was
	  *only* used by slhc.o, and that is was defined as
	  an inline function in "linux/net/inet/ip.c".
	  The "correct" solution was to move this "function"
	  from ip.c to ip.h, since this header was included
	  anyway by slhc.o!
	  Problem solved!
	
	- In the 1.1.88 kernel, the nfs.o module complained
	  about "socki_lookup" being undefined.
	  Would you believe that this function was used
	  externally *only* by nfs.o, and that it has later
	  been cleaned up in "linux/net/socket.c" so that the
	  function call can be replaced by a simple assignment!?!

--- linux-1.1.88/fs/nfs/sock.c	Mon Jan 23 09:38:29 1995
+++ linux/fs/nfs/sock.c	Sun Feb  5 01:27:48 1995
@@ -37,9 +37,6 @@
  * ***FIXME*** should probably put this in nfs_fs.h */
 #define NFS_SLACK_SPACE 1024
 
-
-extern struct socket *socki_lookup(struct inode *inode);
-
 #define _S(nr) (1<<((nr)-1))
 
 /*
@@ -81,7 +78,7 @@
 	file = server->file;
 	inode = file->f_inode;
 	select = file->f_op->select;
-	sock = socki_lookup(inode);
+	sock = &inode->u.socket_i;
 	if (!sock) {
 		printk("nfs_rpc_call: socki_lookup failed\n");
 		return -EBADF;


	  Problem solved!

	  This should be your *first* step before you even *start*
	  thinking of adding symbols to "linux/kernel/ksyms.c"...

	  My own "snooper/sniffer" for these cases is just to do
	  "nm -gu my_module.o" and compare with "ksyms -a".
	  After having removed the "module interface":
	  	_Using_Versions
		_init_module
		_cleanup_module
		_kernel_version
		_mod_use_count
	  the remaining undefined symbols can be searched for
	  in all sources in the kernel tree.
	  I'm lazy, so I have prepared two files, like this:
	  	find . -name '*.c' -print > Sources-c
		find . -name '*.h' -print > Sorces-h
	  For every undefined symbol, do:
	  	grep symbol_name `cat Sources-c`
	  	grep symbol_name `cat Sources-h`
	 With this information it is time to look through the
	 sources and to check if there *really* is a need to
	 export the symbol, or if a small rewrite will fix it...

	 This is tedious work, but you only have to do it once :-)
	 For every symbol :-(
	 I promise you that you will learn a lot more of the
	 kernel internals than you ever thought you needed!

	 Anyway, the goal is to create clean interfaces.
	 This will reduce the "bloat" in ksyms.c, but it will
	 also, as a side-effect, clean up the kernel!


	 There is yet another cleanup method that you should
	 contemplate.
	 Since 1.1.23 there has been support for stacked modules,
	 where a module can supply symbols for use by later
	 loaded modules.
	 This is accomplished by allowing insmod to extract the
	 global symbols in a module and to add an extra symbol
	 table to the kernel.

	 For modules that consist of only one source file, it
	 is possible to selectively export symbols by using
	 the "static" keyword on all *local* symbols.

	 For other modules, that are pre-built by "ld -r",
	 this is not possible, which means that the set of
	 exported symbols might become *huge* :-(

	 If the module isn't intended to be a "base" module
	 in a module stack, you can always tell insmod not
	 to export the symbols by doing "insmod -x module.o".

	 But there is another way!

	 You can use the kernel function: "register_symtab()"
	 and select the symbols in your source instead!

	 Actually, one can also explicitly *delete* all symbols
	 from a module by including "register_symtab(0)" in the
	 function "init_module".

	 In "linux/drivers/net", take a look at "Makefile" and "ppp.c"
	 for an example of the use of "register_symtab()".
	 IMHO this is the right way to do it...

	 Bjorn
