Shell Scripts and Related Files

These scripts are subject to endless repair, tweaking, and refinement. But this will give you the idea, at least. They're not well documented, so to understand them you'll just have to wade through my comments.

From time to time I run a script that scans HTML files for spelling and grammar inconsistencies (UK spelling instead of US, commas after quote marks, etc.), and assorted common typos. The basic script is typosniff, which takes a list of grep-style search patterns from the config file typosniff.patterns.

A few environment variables need to be set first in order for the remaining scripts to work. To do this, I put the following lines in my .cshrc (or .login) file:

setenv ATI ~/Sites/ati				# the website's home dir
setenv HTML_DIR $ATI				# where the html files live
setenv ZIP_DIR $ATI/zip				# where the zip files live
setenv DATE_FORMAT_ATI_FRIENDLY 	"%A %b %e %Y"
setenv DATE_FORMAT_ATI_REV_DATE 	"%A %Y-%m-%d"	
setenv DATE_FORMAT_ATI_VERSION 		"%Y-%m-%d-%H"
setenv DATE_FORMAT_RSS 				"%a, %d %B %Y %H:%M:%S %Z"

The main scripts are generally invoked in the following order:

1. buildsite
2. makebulk
3. upload_to_server

1. buildsite (a tcsh shell script; invoked by me, the webmaster)

Gathers together a list of all the makefiles named "Buildfile" that are located in directories throughout the site, and executes each and every one of them.

Buildfile (a makefile; invoked by buildsite)

Every directory on the site contains a makefile named "Buildfile" that contains instructions on what to do with the files in that directory. This usually means instructions on how to apply time-stamps to the files. (It used to include instructions on assembling a zip archive of the files.) This example is from the ptf ("path to freedom") directory.

stamp_rev-date (a tcsh shell script; invoked by each Buildfile)

Searches a given file for any time-stamp tags and inserts time-stamps there. It in turn calls the general-purpose stamp-insertion script stamp.

css-progress (a tcsh shell script; optionally invoked by webmaster)

Generates a progress report of how the site-wide conversion to CSS is coming along. (Best to invoke this before calling buildsite, to let buildsite give it a time-stamp.

sync-scripts (tcsh shell script; invoked by buildsite)

Copy shell scripts into the tech/scripts directory.

2. makebulk (a tcsh shell script; invoked by the webmaster)

Assembles a zip archive of the entire website. There's a lot going on here, including: removing chunks of text that had been tagged for exclusion from the bulk file (mainly text that won't work properly if the user is offline); preparing a master file list; tweaking the root directory of the bulk file so that the website is located in a directory named "html"; and so on.

BULKMAKE (a makefile invoked makebulk)

BULKMAKE is invoked twice by makebulk: once to create the bulk.zip archive of the website, and once to run bulkfix.

bulkfix (a tcsh shell script; invoked by BULKMAKE)

Removes previously tagged chunks of text from the HTML files in the file bulk.zip.

stamp_bulk (a tcsh script; used by bulkfix)

Stamps a bulk version number and bulk version date wherever it's needed.

bulklist (a tcsh shell script; invoked by makebulk)

Prepares two HTML files: one containing a list of all the files on the site, the other a list of the 25 most-recently modified files.

_newbulkroot (a tcsh shell script; invoked by makebulk)

Unzips bulk.zip into a temporary directory named html, then zips everything back up into the file atibulk.zip, along with a little redirect file named start.html. A casual user who unzips atibulk.zip will see only two items: the html folder and start.html. All the files are neatly tucked away out of sight in that html folder.

insert_chunk (a tcsh shell script; invoked by makebulk).

insert_chunk updates the RSS feed file with the latest info about the bulk file.

3. upload_to_server (a tcsh shell script; invoked by the webmaster)

This assembles a list of files that have been modified since the last upload, and then uploads them to the web host. (N.B. In order to get this script working you'll have to insert your own ftp addresses where indicated.)

Other odds and ends:

backup-htaccess (a tcsh script; invoked by the webmaster)

Retrieves all the .htaccess directives from the remote server and formats them into a single HTML file. I run this from time to time to make sure I have local backup copies of all those directives.