Using a Perl make file dependency generator and Bourne shell script for feeding uncompressed data into a command pipeline

Dale Mensch contributes a tool handy for programming project management. It determines the header files that specified target-object files depend on for use in make-description files. This approach automates the error-prone task formerly done manually.

We present an improved version of a tool originally published in O'Reilly and Associates, Inc. Unix Power Tools, a book that presents tips, scripts, and techniques like those published in this column. For this month we present an enhanced version of the zloop program, which runs a specified command on a set of compressed files without changing the disk copy of the compressed file.

It Depends

Dear Dr. Thomas:

Usually, the most error-prone part of creating and maintaining a make-description file, or ``make'' file, is determining the dependencies of the object-file targets on header files. My mkdepend Perl program [Listing 1A] automates this process to help you create error-free make files. My program differs from other dependency generators I've seen in that it can use GNU gmake to automatically generate the list of the define (-D) and include (-I) compiler directives.

The standard make command will list the directories used to search for header files whose names aren't specified as fully qualified path names-that is, begin with a slash (/). These directories are listed as the arguments to the -I command-line flags that are output by make -n. The -n option tells make to print the commands on its standard output but not actually execute them. Additionally, GNU gmake will output the list of define directives. The latter can be used for building different target programs under different environments, such as System V versus BSD.

The mkdepend program runs the C preprocessor for each specified source file using any include or define directives obtained from running make. Preprocessor output names all header-file dependencies for the target object file that corresponds to the source file. Finally, mkdepend reformats this information for inclusion in a make file.

The mkdepend utility assumes that all the source files and the accompanying make file reside in the same directory. You first need to create a prototype make file that specifies how to build the final target program(s). [Listing 1B shows a simple example.] The file should be named makefile or Makefile, names that make will recognize automatically.

Change to the build directory and invoke mkdepend with the source file arguments. The source file names should correspond to the object files for which you wish to determine the header- file dependencies. For instance, to create our completed make file [Listing 1C], specify both module1.c and module2.c (or simply module?.c for this case). Append the mkdepend output to the prototype make file to create the final version.

Configuration Notes. The first section of mkdepend allows flexible configuration changes: standard or GNU gmake (line 9), suffix used for the source files (line 12) and object files (line 15), list of standard directories containing header files that won't change during program development (line 18), list of standard header-file directories for preprocessors that can't locate them (line 21), and the command that runs the C preprocessor that includes the header files but filters out program comments (line 24). Separate multiple directories by vertical bars on lines 18 and 21 because the variables will be interpolated in regular expressions. As written, mkdepend is configured for a conventional Unix-based C development environment.

Tester's Comments: GNU C compilers recognize the -M option to list all the dependencies. Furthermore, the GNU C compiler also recognizes -MM to only report dependencies listed in source files as #include "file.h", not #include <file.h>. Recent X Window Systems distribute makedepend, which also generates dependencies. Unixware's C compiler recognizes an -H option to list included files on its standard error. --Endre Bálint Nagy

The Perl script runs flawlessly under SCO Unix with any combination of include directories specified in the make file or on the command line. I didn't detect any problems when using my Microsoft C compiler, AT&T C compiler, or SCO's optimizing 486/Pentium compiler. Note that it's important that your backup system save the modification time of files so it can restore it for any files restored from your backup. Otherwise, tools like make and SCCS won't work correctly. If you upgrade your operating system you should recompile all your sources and test thoroughly. --Kees Hendrikse

When using mkdepend with GNU gmake under AIX 3.2 on an IBM RS/6000 I get a warning message that the first file specified can't be located, yet the dependencies are printed correctly.--Steve Wright

Unix Power Tool of the Month

File compression is a common approach to reducing disk space consumption. Most Unix systems provide a utility to compress files. System V furnishes pack, while BSD-derived systems--and also System V Release 4--provide compress, which reduces file size even more than pack. However, because of patent troubles with compress, many users are employing the GNU gzip program available from the Free Software Foundation. It is even more effective than compress at compressing data. Furthermore, gzip can uncompress files compressed with either compress or pack.

Of course, all these compression utilities can uncompress compressed files so you can operate on them with conventional Unix utilities. One problem arises if you have a lot of compressed files that you want to access separately. You could uncompress each file in place, but such a process would be slow because of all the disk I/O activity and could even fill up your file system partition. The compress distribution provides zcat and pack provides pcat, both of which write the uncompressed contents to the standard output, which then can be piped into one or more filter utilities.

The zloop utility described in O'Reilly & Associates' Unix Power Tools runs zcat on specified files and then pipes the decompressed contents into a specified pipeline to process groups of compressed files quickly. Jerry Peek was kind enough to provide an enhanced version that also supports GNU gzip, and Kees Hendrikse improved the implementation to support other compression utilities providing the result shown in Listing 2A. Note that GNU zcat will uncompress files compressed with pack, compress, or gzip.

The remaining parts of Listing 2 show some examples. In Part B we print each of three compressed files in the current directory. Here, when pr | lp -d ast is run on the uncompressed version of each file selected, ch01.Z, ch02.Z, and ch03.Z. The header printed by zloop shows each command line as it runs. If there were errors, this heading will let you determine the offending command line. You can suppress the header by redirecting standard error to /dev/null as shown in Listing 2C.

Listing 2D shows an example where grep is used to locate a pattern in the specified files. Not only the command line, but also the pattern must be quoted because it contains white space. Here, grep doesn't know the file name because it reads from its standard input, so the zloop heading makes it easy to see which text came from which file. Note that grep returns a non-zero exit status when its search fails; zloop reports that fact and shows the exact command line it ran.

Acknowledgments

I wish to thank the following readers for their help with testing this month's contributions: Gábor Zahemszky, CoDe Ltd., Budapest, Hungary (ISC 3.2.2 and Coherent 4.2); Kees Hendrikse, Echelon Consultancy, Enschede, The Netherlands (current SCO Unix and Xenix versions); Endre Bálint Nagy, Walton Networking Ltd., Budapest, Hungary (Unixware Application Server 1.0); and Steve Wright, Computer Science Dept., University of South Carolina, Columbia, S.C. (AIX 3.2).