2  INTRODUCTION  MWEAVE §1

1* . Introduction. This program converts a file to a TEX file. It was written by D. E. Knuth in October, 1981; a somewhat similar SAIL program had been developed in March, 1979, although the earlier program used a top-down parsing method that is quite different from the present scheme.

The code uses a few features of the local Pascal compiler that may need to be changed in other installations:

1) Case statements have a default.

2) Input-output routines may need to be adapted for use with a particular character set and/or for printing messages on the user’s terminal.

These features are also present in the Pascal version of TEX, where they are used in a similar (but more complex) way. System-dependent portions of can be identified by looking at the entries for ‘system dependencies’ in the index below.

The “banner line” defined here should be changed whenever is modified. For the Modula-2 versions of , we add a second decimal point and number to the main version

number, so we can differentiate between changes to the standard and changes local to . define banner ≡ ´ ␣ ␣ ␣ ␣ ␣ ␣ ´

2. The program begins with a fairly normal header, made up of pieces that will mostly be filled in later. The input comes from files web file and change file, and the TEX output goes to file tex file.

If it is necessary to abort the job because of a fatal error, the program calls the ‘jump out ’ procedure, which goes to the label end of WEAVE .

define end of WEAVE = 9999 { go here to wrap it up } ⟨ Compiler directives 4 ⟩ program WEAVE (web file, change file, tex file);

label end of WEAVE ; { go here to finish } const ⟨ Constants in the outer block 8 ⟩ type ⟨ Types in the outer block 11 ⟩ var ⟨ Globals in the outer block 9 ⟩

⟨ Error handling procedures 30 ⟩ procedure initialize;

var ⟨ Local variables for initialization 16 ⟩ begin ⟨ Set initial values 10 ⟩ end;

3. Some of this code is optional for use when debugging only; such material is enclosed between the delimiters debug and gubed. Other parts, delimited by stat and tats, are optionally included if statistics about ’s memory usage are desired.

define debug ≡ { change this to ‘debug ≡ ’ when debugging } define gubed ≡ { change this to ‘gubed ≡ ’ when debugging } format debug ≡ begin format gubed ≡ end define stat ≡ { change this to ‘stat ≡ ’ when gathering usage statistics } define tats ≡ { change this to ‘tats ≡ ’ when gathering usage statistics } format stat ≡ begin format tats ≡ end

§4  MWEAVE  INTRODUCTION 3

4. The Pascal compiler used to develop this system has “compiler directives” that can appear in comments whose first character is a dollar sign. In production versions of these directives tell the compiler that it is safe to avoid range checks and to leave out the extra code it inserts for the Pascal debugger’s benefit, although interrupts will occur if there is arithmetic overflow. ⟨ Compiler directives 4 ⟩ ≡

$C−, A+, D− { no range check, catch arithmetic overflow, no debug overhead } debug $C+, D+ gubed { but turn everything on when debugging }

This code is used in section 2.

5. Labels are given symbolic names by the following definitions. We insert the label ‘exit :’ just before the ‘end’ of a procedure in which we have used the ‘return’ statement defined below; the label ‘restart ’ is occasionally used at the very beginning of a procedure; and the label ‘reswitch’ is occasionally used just prior to a case statement in which some cases change the conditions and we wish to branch to the newly applicable case. Loops that are set up with the loop construction defined below are commonly exited by going to ‘done’ or to ‘found ’ or to ‘not found ’, and they are sometimes repeated by going to ‘continue’.

define exit = 10 { go here to leave a procedure } define restart = 20 { go here to start a procedure again } define reswitch = 21 { go here to start a case statement again } define continue = 22 { go here to resume a loop } define done = 30 { go here to exit a loop } define found = 31 { go here when you’ve found it } define not found = 32 { go here when you’ve found something else }

6. Here are some macros for common programming idioms.

define incr (#) ≡ # ← # + 1 { increase a variable by unity } define decr (#) ≡ # ← # − 1 { decrease a variable by unity } define loop ≡ while true do { repeat over and over until a goto happens } define do nothing ≡ { empty statement } define return ≡ goto exit { terminate a procedure call } format return ≡ nil format loop ≡ xclause

7. We assume that case statements may include a default case that applies if no matching label is found. Thus, we shall use constructions like

case x of

1: ⟨ code for x = 1 ⟩; 3: ⟨ code for x = 3 ⟩; othercases ⟨ code for x ≠ 1 and x ≠ 3 ⟩ endcases

since most Pascal compilers have plugged this hole in the language by incorporating some sort of default mechanism. For example, the compiler used to develop and TEX allows ‘others :’ as a default label, and other Pascals allow syntaxes like ‘else’ or ‘otherwise’ or ‘otherwise:’, etc. The definitions of othercases and endcases should be changed to agree with local conventions. (Of course, if no default mechanism is available, the case statements of this program must be extended by listing all remaining cases.)

define othercases ≡ others : { default for cases not listed explicitly } define endcases ≡ end { follows the default case in an extended case statement } format othercases ≡ else format endcases ≡ end 8. The following parameters are set big enough to handle TEX, so they should be sufficient for most applications of . ⟨ Constants in the outer block 8 ⟩ ≡

max bytes = 45000; { 1/ww times the number of bytes in identifiers, index entries, and module names;

must be less than 65536 } max names = 5000; { number of identifiers, index entries, and module names; must be less than 10240 } max modules = 2000; { greater than the total number of modules } hash size = 353; { should be prime } buf size = 100; { maximum length of input line } longest name = 400; { module names shouldn’t be longer than this } long buf size = 500; { buf size + longest name } line length = 80; { lines of TEX output have at most this many characters, should be less than 256 } max refs = 20000; { number of cross references; must be less than 65536 } max toks = 20000; { number of symbols in Pascal texts being parsed; must be less than 65536 } max texts = 2000; { number of phrases in Pascal texts being parsed; must be less than 10240 } max scraps = 1000; { number of tokens in Pascal texts being parsed } stack size = 200; { number of simultaneous output levels }

This code is used in section 2.

9. A global variable called history will contain one of four values at the end of every run: spotless means that no unusual messages were printed; harmless message means that a message of possible interest was printed but no serious errors were detected; error message means that at least one error was found; fatal message means that the program terminated abnormally. The value of history does not influence the behavior of the program; it is simply computed for the convenience of systems that might want to use such information.

define spotless = 0 { history value for normal jobs } define harmless message = 1 { history value when non-serious info was printed } define error message = 2 { history value when an error was noted } define fatal message = 3 { history value when we had to stop prematurely } define mark harmless ≡ if history = spotless then history ← harmless message define mark error ≡ history ← error message define mark fatal ≡ history ← fatal message

⟨ Globals in the outer block 9 ⟩ ≡ history : spotless . . fatal message; { how bad was this run? }

See also sections 13, 20, 23, 25, 27, 29, 37, 39, 45, 48, 53, 55, 63*, 65, 71, 73, 93, 108, 114, 118, 121, 129, 144, 177, 202, 219,

229*, 234, 240, 242, 244, 246, and 258. This code is used in section 2.

10. ⟨ Set initial values 10 ⟩ ≡ history ← spotless ;

See also sections 14, 17, 18, 21, 26, 41, 43, 49, 54, 57, 94, 102, 124*, 126, 145, 203, 245, 248, and 259. This code is used in section 2.

§11  MWEAVE  THE CHARACTER SET 5

11. The character set. One of the main goals in the design of has been to make it readily portable between a wide variety of computers. Yet by its very nature must use a greater variety of characters than most computer programs deal with, and character encoding is one of the areas in which existing machines differ most widely from each other.

To resolve this problem, all input to and is converted to an internal seven-bit code that is essentially standard ASCII, the “American Standard Code for Information Interchange.” The conversion is done immediately when each character is read in. Conversely, characters are converted from ASCII to the user’s external representation just before they are output.

Such an internal code is relevant to users of only because it is the code used for preprocessed constants like . If you are writing a program in that makes use of such one-character constants, you should convert your input to ASCII form, like and do. Otherwise ’s internal coding scheme does not affect you.

Here is a table of the standard visible ASCII codes:

0  1  2  3  4  5  6  7

          ´ 040  ␣       !       "       #       $       %       &       '
          ´ 050  (       )       *       +       ,       -       .       /
          ´ 060  0       1       2       3       4       5       6       7
          ´ 070  8       9       :       ;       <       =       >       ?
          ´ 100  @       A       B       C       D       E       F       G
          ´ 110  H       I       J       K       L       M       N       O
          ´ 120  P       Q       R       S       T       U       V       W
          ´ 130  X       Y       Z       [       \       ]       ^_
          ´ 140  `       a       b       c       d       e       f       g
          ´ 150  h       i       j       k       l       m       n       o
          ´ 160  p       q       r       s       t       u       v       w
          ´ 170  x       y       z       {       |       }       ~

(Actually, of course, code ´ 040 is an invisible blank space.) Code ´ 136 was once an upward arrow ( ), and code ´ 137 was once a left arrow ( ), in olden times when the first draft of ASCII code was prepared; but works with today’s standard ASCII in which those codes represent circumflex and underline as shown. ⟨ Types in the outer block 11 ⟩ ≡

ASCII code = 0 . . 127; { seven-bit numbers, a subrange of the integers }

See also sections 12, 36, 38, 47, 52, and 201. This code is used in section 2.

6  THE CHARACTER SET  MWEAVE §12

12. The original Pascal compiler was designed in the late 60s, when six-bit character sets were common, so it did not make provision for lowercase letters. Nowadays, of course, we need to deal with both capital and small letters in a convenient way, so assumes that it is being used with a Pascal whose character set contains at least the characters of standard ASCII as listed above. Some Pascal compilers use the original name char for the data type associated with the characters in text files, while other Pascals consider char to be a 64-element subrange of a larger data type that has some other name.

In order to accommodate this difference, we shall use the name text char to stand for the data type of the characters in the input and output files. We shall also assume that text char consists of the elements chr (first text char ) through chr (last text char ), inclusive. The following definitions should be adjusted if necessary.

define text char ≡ char { the data type of characters in text files }

define first text char = 0 { ordinal number of the smallest element of text char }

define last text char = 127 { ordinal number of the largest element of text char } ⟨ Types in the outer block 11 ⟩ +≡

text file = packed file of text char ;

13. The  and  processors convert between ASCII code and the user’s external character set

by means of arrays xord and xchr that are analogous to Pascal’s ord and chr functions. ⟨ Globals in the outer block 9 ⟩ +≡ xord : array [text char ] of ASCII code; { specifies conversion of input characters } xchr : array [ASCII code] of text char ; { specifies conversion of output characters } 14. If we assume that every system using is able to read and write the visible characters of standard ASCII (although not necessarily using the ASCII codes to represent them), the following assignment statements initialize most of the xchr array properly, without needing any system-dependent changes. For example, the statement ´ ´ ´ that appears in the present file might be encoded in, say, EBCDIC code on the external medium on which it resides, but will convert from this external code to ASCII and back again. Therefore the assignment statement ´ ´ will appear in the corresponding Pascal file, and Pascal will compile this statement so that xchr [65] receives the character in the external (char ) code. Note that it would be quite incorrect to say ´ , because is a constant of type integer , not char , and because we have = 65 regardless of the external character set. ⟨ Set initial values 10 ⟩ +≡

xchr [´ 40 ] ← ´ ␣´ ; xchr [´ 41 ] ← ´ ´ ; xchr [´ 42 ] ← ´ ´ ; xchr [´ 43 ] ← ´ #´ ; xchr [´ 44 ] ← ´ $´ ; xchr [´ 45 ] ← ´ %´ ; xchr [´ 46 ] ← ´ ´ ; xchr [´ 47 ] ← ´ ´ ´ ´ ; xchr [´ 50 ] ← ´ ´ ; xchr [´ 51 ] ← ´ ´ ; xchr [´ 52 ] ← ´ ´ ; xchr [´ 53 ] ← ´ ´ ; xchr [´ 54 ] ← ´ ´ ; xchr [´ 55 ] ← ´ ´ ; xchr [´ 56 ] ← ´ ´ ; xchr [´ 57 ] ← ´ ´ ; xchr [´ 60 ] ← ´ ´ ; xchr [´ 61 ] ← ´ ´ ; xchr [´ 62 ] ← ´ ´ ; xchr [´ 63 ] ← ´ ´ ; xchr [´ 64 ] ← ´ ´ ; xchr [´ 65 ] ← ´ ´ ; xchr [´ 66 ] ← ´ ´ ; xchr [´ 67 ] ← ´ ´ ; xchr [´ 70 ] ← ´ ´ ; xchr [´ 71 ] ← ´ ´ ; xchr [´ 72 ] ← ´ ´ ; xchr [´ 73 ] ← ´ ´ ; xchr [´ 74 ] ← ´ ´ ; xchr [´ 75 ] ← ´ ´ ; xchr [´ 76 ] ← ´ ´ ; xchr [´ 77 ] ← ´ ´ ; xchr [´ 100 ] ← ´ ´ ; xchr [´ 101 ] ← ´ ´ ; xchr [´ 102 ] ← ´ ´ ; xchr [´ 103 ] ← ´ ´ ; xchr [´ 104 ] ← ´ ´ ; xchr [´ 105 ] ← ´ ´ ; xchr [´ 106 ] ← ´ ´ ; xchr [´ 107 ] ← ´ ´ ; xchr [´ 110 ] ← ´ ´ ; xchr [´ 111 ] ← ´ ´ ; xchr [´ 112 ] ← ´ ´ ; xchr [´ 113 ] ← ´ ´ ; xchr [´ 114 ] ← ´ ´ ; xchr [´ 115 ] ← ´ ´ ; xchr [´ 116 ] ← ´ ´ ; xchr [´ 117 ] ← ´ ´ ; xchr [´ 120 ] ← ´ ´ ; xchr [´ 121 ] ← ´ ´ ; xchr [´ 122 ] ← ´ ´ ; xchr [´ 123 ] ← ´ ´ ; xchr [´ 124 ] ← ´ ´ ; xchr [´ 125 ] ← ´ ´ ; xchr [´ 126 ] ← ´ ´ ; xchr [´ 127 ] ← ´ ´ ; xchr [´ 130 ] ← ´ ´ ; xchr [´ 131 ] ← ´ ´ ; xchr [´ 132 ] ← ´ ´ ; xchr [´ 133 ] ← ´ ´ ; xchr [´ 134 ] ← ´ ´ ; xchr [´ 135 ] ← ´ ´ ; xchr [´ 136 ] ← ´ ´ ; xchr [´ 137 ] ← ´ ´ ; xchr [´ 140 ] ← ´ ` ´ ; xchr [´ 141 ] ← ´ ´ ; xchr [´ 142 ] ← ´ ´ ; xchr [´ 143 ] ← ´ ´ ; xchr [´ 144 ] ← ´ ´ ; xchr [´ 145 ] ← ´ ´ ; xchr [´ 146 ] ← ´ ´ ; xchr [´ 147 ] ← ´ ´ ; xchr [´ 150 ] ← ´ ´ ; xchr [´ 151 ] ← ´ ´ ; xchr [´ 152 ] ← ´ ´ ; xchr [´ 153 ] ← ´ ´ ; xchr [´ 154 ] ← ´ ´ ; xchr [´ 155 ] ← ´ ´ ; xchr [´ 156 ] ← ´ ´ ; xchr [´ 157 ] ← ´ ´ ; xchr [´ 160 ] ← ´ ´ ; xchr [´ 161 ] ← ´ ´ ; xchr [´ 162 ] ← ´ ´ ; xchr [´ 163 ] ← ´ ´ ; xchr [´ 164 ] ← ´ ´ ; xchr [´ 165 ] ← ´ ´ ; xchr [´ 166 ] ← ´ ´ ; xchr [´ 167 ] ← ´ ´ ; xchr [´ 170 ] ← ´ ´ ; xchr [´ 171 ] ← ´ ´ ; xchr [´ 172 ] ← ´ ´ ; xchr [´ 173 ] ← ´ ´ ; xchr [´ 174 ] ← ´ ´ ; xchr [´ 175 ] ← ´ ´ ; xchr [´ 176 ] ← ´ ´ ; xchr [0] ← ´ ␣´ ; xchr [´ 177 ] ← ´ ␣´ ; { these ASCII codes are not used }

15. Some of the ASCII codes below ´ 40 have been given symbolic names in  and  because

they are used with a special meaning.

define and sign = ´ 4 { equivalent to ‘ ’ } define not sign = ´ 5 { equivalent to ‘ ’ } define set element sign = ´ 6 { equivalent to ‘ ’ } define tab mark = ´ 11 { ASCII code used as tab-skip } define line feed = ´ 12 { ASCII code thrown away at end of line } define form feed = ´ 14 { ASCII code used at end of page } define carriage return = ´ 15 { ASCII code used at end of line } define left arrow = ´ 30 { equivalent to ‘ ’ } define not equal = ´ 32 { equivalent to ‘ ’ } define less or equal = ´ 34 { equivalent to ‘ ’ } define greater or equal = ´ 35 { equivalent to ‘ ’ } define equivalence sign = ´ 36 { equivalent to ‘ ’ } define or sign = ´ 37 { equivalent to ‘ ’ } 16. When we initialize the xord array and the remaining parts of xchr , it will be convenient to make use of an index variable, i. ⟨ Local variables for initialization 16 ⟩ ≡ i: 0 . . last text char ;

See also sections 40, 56, and 247. This code is used in section 2.

17. Here now is the system-dependent part of the character set. If is being implemented on a gardenvariety Pascal for which only standard ASCII codes will appear in the input and output files, you don’t need to make any changes here. But at MIT, for example, the code in this module should be changed to

for i ← 1 to ´ 37 do xchr [i] ← chr (i);

’s character set is essentially identical to MIT’s, even with respect to characters less than ´ 40 .

Changes to the present module will make more friendly on computers that have an extended character set, so that one can type things like instead of . If you have an extended set of characters that are easily incorporated into text files, you can assign codes arbitrarily here, giving an xchr equivalent to whatever characters the users of are allowed to have in their input files, provided that unsuitable characters do not correspond to special codes like carriage return that are listed above.

(The present file does not contain any of the non-ASCII characters, because it is intended to be used with all implementations of . It was originally created on a Stanford system that has a convenient extended character set, then “sanitized” by applying another program that transliterated all of the non-standard characters into standard equivalents.) ⟨ Set initial values 10 ⟩ +≡

for i ← 1 to ´ 37 do xchr [i] ← ´ ␣´ ;

18. The following system-independent code makes the xord array contain a suitable inverse to the information in xchr . ⟨ Set initial values 10 ⟩ +≡

for i ← first text char to last text char do xord [chr (i)] ← ´ 40 ;

for i ← 1 to ´ 176 do xord [xchr [i]] ← i;

§19  MWEAVE  INPUT AND OUTPUT 9

19. Input and output. The input conventions of this program are intended to be very much like those of TEX (except, of course, that they are much simpler, because much less needs to be done). Furthermore they are identical to those of . Therefore people who need to make modifications to all three systems should be able to do so without too many headaches.

We use the standard Pascal input/output procedures in several places that TEX cannot, since does not have to deal with files that are named dynamically by the user, and since there is no input from the terminal.

20. Terminal output is done by writing on file term out , which is assumed to consist of characters of type text char :

define print (#) ≡ write(term out , #) { ‘print ’ means write on the terminal }

define print ln(#) ≡ write ln(term out , #) { ‘print ’ and then start new line }

define new line ≡ write ln(term out ) { start new line }

define print nl (#) ≡ { print information starting on a new line } begin new line; print (#); end

⟨ Globals in the outer block 9 ⟩ +≡ term out : text file; { the terminal as an output file }

21. Different systems have different ways of specifying that the output on a certain file will appear on the user’s terminal. Here is one way to do this on the Pascal system that was used in ’s initial development: ⟨ Set initial values 10 ⟩ +≡

rewrite(term out , ´ ´ ); { send term out output to the terminal }

22. The update terminal procedure is called when we want to make sure that everything we have output to the terminal so far has actually left the computer’s internal buffers and been sent.

define update terminal ≡ break (term out ) { empty the terminal output buffer }

23. The main input comes from web file; this input may be overridden by changes in change file. (If change file is empty, there are no changes.) ⟨ Globals in the outer block 9 ⟩ +≡ web file: text file; { primary input } change file: text file; { updates }

24. The following code opens the input files. Since these files were listed in the program header, we assume that the Pascal runtime system has already checked that suitable file names have been given; therefore no additional error checking needs to be done. We will see below that reads through the entire input twice. procedure open input ; { prepare to read web file and change file }

begin reset (web file); reset (change file);

end;

25. The main output goes to tex file. ⟨ Globals in the outer block 9 ⟩ +≡ tex file: text file;

10  INPUT AND OUTPUT  MWEAVE §26

26. The following code opens tex file. Since this file was listed in the program header, we assume that the Pascal runtime system has checked that a suitable external file name has been given. ⟨ Set initial values 10 ⟩ +≡

rewrite(tex file);

27. Input goes into an array called buffer . ⟨ Globals in the outer block 9 ⟩ +≡ buffer : array [0 . . long buf size] of ASCII code;

28. The input ln procedure brings the next line of input from the specified file into the buffer array and returns the value true, unless the file has already been entirely read, in which case it returns false. The conventions of TEX are followed; i.e., ASCII code numbers representing the next line of the file are input into buffer [0], buffer [1], …, buffer [limit − 1]; trailing blanks are ignored; and the global variable limit is set to the length of the line. The value of limit must be strictly less than buf size.

We assume that none of the ASCII code values of buffer [j] for 0 ≤ j < limit is equal to 0, ´ 177 , line feed , form feed , or carriage return. Since buf size is strictly less than long buf size, some of ’s routines use the fact that it is safe to refer to buffer [limit + 2] without overstepping the bounds of the array. function input ln(var f : text file): boolean; { inputs a line or returns false }

var final limit : 0 . . buf size; { limit without trailing blanks }

begin limit ← 0; final limit ← 0;

if eof (f) then input ln ← false

else begin while ¬eoln(f) do begin buffer [limit ] ← xord [f↑]; get (f); incr (limit ); if buffer [limit − 1] ≠ ␣ then final limit ← limit ; if limit = buf size then

begin while ¬eoln(f) do get (f);

decr (limit ); { keep buffer [buf size] empty }

print nl (´ ␣ ␣ ␣ ␣ ´ ); loc ← 0; error ; end; end;

read ln(f); limit ← final limit ; input ln ← true; end;

end;

§29  MWEAVE  REPORTING ERRORS TO THE USER 11

29. Reporting errors to the user. The processor operates in three phases: first it inputs the source file and stores cross-reference data, then it inputs the source once again and produces the TEX output file, and finally it sorts and outputs the index.

The global variables phase one and phase three tell which Phase we are in. ⟨ Globals in the outer block 9 ⟩ +≡ phase one: boolean; { true in Phase I, false in Phases II and III } phase three: boolean; { true in Phase III, false in Phases I and II }

30. If an error is detected while we are debugging, we usually want to look at the contents of memory. A special procedure will be declared later for this purpose. ⟨ Error handling procedures 30 ⟩ ≡

debug procedure debug help; forward ; gubed

See also sections 31 and 33. This code is used in section 2.

31. The command ‘err print (´ ␣ ␣ ´ )’ will report a syntax error to the user, by printing the error message at the beginning of a new line and then giving an indication of where the error was spotted in the source file. Note that no period follows the error message, since the error routine will automatically supply a period.

The actual error indications are provided by a procedure called error . However, error messages are not actually reported during phase one, since errors detected on the first pass will be detected again during the second.

define err print (#) ≡

begin if ¬phase one then begin new line; print (#); error ; end; end

⟨ Error handling procedures 30 ⟩ +≡ procedure error ; { prints ‘ ’ and location of error message }

var k, l: 0 . . long buf size; { indices into buffer }

begin ⟨ Print error location based on input buffer 32 ⟩;

update terminal ; mark error ;

debug debug skipped ← debug cycle; debug help; gubed

end;

12  REPORTING ERRORS TO THE USER  MWEAVE §32

32. The error locations can be indicated by using the global variables loc, line, and changing , which tell respectively the first unlooked-at position in buffer , the current line number, and whether or not the current line is from change file or web file. This routine should be modified on systems whose standard text editor has special line-numbering conventions. ⟨ Print error location based on input buffer 32 ⟩ ≡

begin if changing then print (´ ␣ ␣ ␣´ ) else print (´ ␣ ´ ); print ln(´ ´ , line : 1, ´ ´ ); if loc ≥ limit then l ← limit else l ← loc; for k ← 1 to l do

if buffer [k − 1] = tab mark then print (´ ␣´ )

else print (xchr [buffer [k − 1]]); { print the characters already read } new line; for k ← 1 to l do print (´ ␣´ ); { space out the next line } for k ← l + 1 to limit do print (xchr [buffer [k − 1]]); { print the part not yet read } if buffer [limit ] = then print (xchr [ ]); { end of Pascal text in module names } print (´ ␣´ ); { this space separates the message from future asterisks } end

This code is used in section 31.

33. The jump out procedure just cuts across all active procedure levels and jumps out of the program. This is the only non-local goto statement in . It is used when no recovery from a particular error has been provided.

Some Pascal compilers do not implement non-local goto statements. In such cases the code that appears at label end of WEAVE should be copied into the jump out procedure, followed by a call to a system procedure that terminates the program.

define fatal error (#) ≡ begin new line; print (#); error ; mark fatal ; jump out ; end

⟨ Error handling procedures 30 ⟩ +≡ procedure jump out ;

begin goto end of WEAVE ; end;

34. Sometimes the program’s behavior is far different from what it should be, and prints an error message that is really for the maintenance person, not the user. In such cases the program says confusion(´ ␣ ␣ ␣ ␣ ´ ).

define confusion(#) ≡ fatal error (´ ␣ ␣ ´ ´ ␣ ␣ ´ , #, ´ ´ )

35. An overflow stop occurs if ’s tables aren’t large enough.

define overflow (#) ≡ fatal error (´ ␣ ␣´ , #, ´ ␣ ␣ ´ )

§36  MWEAVE  DATA STRUCTURES 13

36. Data structures. During the first phase of its processing, puts identifier names, index entries, and module names into the large byte mem array, which is packed with seven-bit integers. Allocation is sequential, since names are never deleted.

An auxiliary array byte start is used as a directory for byte mem, and the link , ilk , and xref arrays give further information about names. These auxiliary arrays consist of sixteen-bit items. ⟨ Types in the outer block 11 ⟩ +≡

eight bits = 0 . . 255; { unsigned one-byte quantity }

sixteen bits = 0 . . 65535; { unsigned two-byte quantity }

37. has been designed to avoid the need for indices that are more than sixteen bits wide, so that it can be used on most computers. But there are programs that need more than 65536 bytes; TEX is one of these. To get around this problem, a slight complication has been added to the data structures: byte mem is a two-dimensional array, whose first index is either 0 or 1. (For generality, the first index is actually allowed to run between 0 and ww − 1, where ww is defined to be 2; the program will work for any positive value of ww , and it can be simplified in obvious ways if ww = 1.)

define ww = 2 { we multiply the byte capacity by approximately this amount } ⟨ Globals in the outer block 9 ⟩ +≡ byte mem: packed array [0 . . ww − 1, 0 . . max bytes ] of ASCII code; { characters of names } byte start : array [0 . . max names ] of sixteen bits ; { directory into byte mem } link : array [0 . . max names ] of sixteen bits ; { hash table or tree links } ilk : array [0 . . max names ] of sixteen bits ; { type codes or tree links } xref : array [0 . . max names ] of sixteen bits ; { heads of cross-reference lists }

38. The names of identifiers are found by computing a hash address h and then looking at strings of bytes signified by hash[h], link [hash[h]], link [link [hash[h]]], …, until either finding the desired name or encountering a zero.

A ‘name pointer ’ variable, which signifies a name, is an index into byte start . The actual sequence of characters in the name pointed to by p appears in positions byte start [p] to byte start [p + ww ] − 1, inclusive, in the segment of byte mem whose first index is p mod ww . Thus, when ww = 2 the even-numbered name bytes appear in byte mem[0, ∗] and the odd-numbered ones appear in byte mem[1, ∗]. The pointer 0 is used for undefined module names; we don’t want to use it for the names of identifiers, since 0 stands for a null pointer in a linked list.

We usually have byte start [name ptr + w] = byte ptr [(name ptr + w) mod ww ] for 0 ≤ w < ww , since these are the starting positions for the next ww names to be stored in byte mem.

define length(#) ≡ byte start [# + ww ] − byte start [#] { the length of a name } ⟨ Types in the outer block 11 ⟩ +≡

name pointer = 0 . . max names ; { identifies a name }

39. ⟨ Globals in the outer block 9 ⟩ +≡ name ptr : name pointer ; { first unused position in byte start } byte ptr : array [0 . . ww − 1] of 0 . . max bytes ; { first unused position in byte mem }

40. ⟨ Local variables for initialization 16 ⟩ +≡ wi : 0 . . ww − 1; { to initialize the byte mem indices }

41. ⟨ Set initial values 10 ⟩ +≡

for wi ← 0 to ww − 1 do begin byte start [wi ] ← 0; byte ptr [wi ] ← 0; end;

byte start [ww ] ← 0; { this makes name 0 of length zero }

14  DATA STRUCTURES  MWEAVE §42

name ptr ← 1; 42* . Several types of identifiers are distinguished by their ilk :

normal identifiers are part of the Pascal program and will appear in italic type. roman identifiers are index entries that appear after in the file. wildcard identifiers are index entries that appear after in the file. typewriter identifiers are index entries that appear after in the file. array like, begin like, …, var like identifiers are Pascal reserved words whose ilk explains how they are

to be treated when Pascal code is being formatted. Finally, if c is an ASCII code, an ilk equal to char like + c denotes a reserved word that will be converted

to character c. define normal = 0 { ordinary identifiers have normal ilk } define roman = 1 { normal index entries have roman ilk } define wildcard = 2 { user-formatted index entries have wildcard ilk } define typewriter = 3 { ‘typewriter type’ entries have typewriter ilk } define reserved (#) ≡ (ilk [#] > typewriter ) { tells if a name is a reserved word } define array like = 4 { array, file, set } define begin like = 5 { begin } define case like = 6 { case } define const like = 7 { const, label, type } define div like = 8 { div, mod } define do like = 9 { do, then } define else like = 10 { else } define end like = 11 { end } define for like = 12 { for, if, while, with } define goto like = 13 { implementation, definition, packed } define if like = 14 { not used for — see for like } define in like = 15 { in } define nil like = 16 { nil } define proc like = 17 { procedure, module } define record like = 18 { record } define repeat like = 19 { repeat } define to like = 20 { to } define until like = 21 { until, end } define var like = 22 { var } define loop like = 23 { loop } define elsif like = 24 { elsif } define of like = 25 { of } define char like = 26 { and, or, not, in }

43. The names of modules are stored in byte mem together with the identifier names, but a hash table is not used for them because needs to be able to recognize a module name when given a prefix of that name. A conventional binary seach tree is used to retrieve module names, with fields called llink and rlink in place of link and ilk . The root of this tree is rlink [0].

define llink ≡ link { left link in binary search tree for module names } define rlink ≡ ilk { right link in binary search tree for module names } define root ≡ rlink [0] { the root of the binary search tree for module names }

⟨ Set initial values 10 ⟩ +≡ root ← 0; { the binary search tree starts out with nothing in it } 44. Here is a little procedure that prints the text of a given name on the user’s terminal. procedure print id (p : name pointer ); { print identifier or module name }

var k: 0 . . max bytes ; { index into byte mem }

w: 0 . . ww − 1; { row of byte mem } begin if p ≥ name ptr then print (´ ´ ) else begin w ← p mod ww ;

for k ← byte start [p] to byte start [p + ww ] − 1 do print (xchr [byte mem[w, k]]); end;

end;

45. We keep track of the current module number in module count , which is the total number of modules that have started. Modules which have been altered by a change file entry have their changed module flag turned on during the first phase. ⟨ Globals in the outer block 9 ⟩ +≡ module count : 0 . . max modules ; { the current module number } changed module: packed array [0 . . max modules ] of boolean; { is it changed? } change exists : boolean; { has any module changed? }

46. The other large memory area in keeps the cross-reference data. All uses of the name p are recorded in a linked list beginning at xref [p], which points into the xmem array. Entries in xmem consist of two sixteen-bit items per word, called the num and xlink fields. If x is an index into xmem, reached from name p, the value of num(x) is either a module number where p is used, or it is def flag plus a module number where p is defined; and xlink (x) points to the next such cross reference for p, if any. This list of cross references is in decreasing order by module number. The current number of cross references is xref ptr .

The global variable xref switch is set either to def flag or to zero, depending on whether the next cross reference to an identifier is to be underlined or not in the index. This switch is set to def flag when or or is scanned, and it is cleared to zero when the next identifier or index entry cross reference has been made. Similarly, the global variable mod xref switch is either def flag or zero, depending on whether a module name is being defined or used.

define num(#) ≡ xmem[#].num field

define xlink (#) ≡ xmem[#].xlink field

define def flag = 10240 { must be strictly larger than max modules }

47. ⟨ Types in the outer block 11 ⟩ +≡ xref number = 0 . . max refs ;

48. ⟨ Globals in the outer block 9 ⟩ +≡ xmem: array [xref number ] of packed record

num field : sixteen bits ; { module number plus zero or def flag } xlink field : sixteen bits ; { pointer to the previous cross reference } end;

xref ptr : xref number ; { the largest occupied position in xmem } xref switch, mod xref switch: 0 . . def flag ; { either zero or def flag }

49. ⟨ Set initial values 10 ⟩ +≡

xref ptr ← 0; xref switch ← 0; mod xref switch ← 0; num(0) ← 0; xref [0] ← 0; { cross references to undefined modules } 50. A new cross reference for an identifier is formed by calling new xref , which discards duplicate entries and ignores non-underlined references to one-letter identifiers or Pascal’s reserved words.

define append xref (#) ≡ if xref ptr = max refs then overflow (´ ␣ ´ ) else begin incr (xref ptr ); num(xref ptr ) ← #;

end

procedure new xref (p : name pointer ); label exit ; var q: xref number ; { pointer to previous cross reference }

m, n: sixteen bits ; { new and previous cross-reference value } begin if (reserved (p) ∨ (byte start [p] + 1 = byte start [p + ww ])) ∧ (xref switch = 0) then return; m ← module count + xref switch; xref switch ← 0; q ← xref [p]; if q > 0 then

begin n ← num(q);

if (n = m) ∨ (n = m + def flag ) then return

else if m = n + def flag then begin num(q) ← m; return; end;

end; append xref (m); xlink (xref ptr ) ← q; xref [p] ← xref ptr ;

exit : end;

51. The cross reference lists for module names are slightly different. Suppose that a module name is defined in modules m_{1}, …, m_{k} and used in modules n_{1}, …, n_{l}. Then its list will contain m_{1} + def flag , m_{k} + def flag , …, m_{2} + def flag , n_{l}, …, n_{1}, in this order. After Phase II, however, the order will be m_{1} + def flag , …, m_{k} + def flag , n_{1}, …, n_{l}. procedure new mod xref (p : name pointer );

var q, r: xref number ; { pointers to previous cross references } begin q ← xref [p]; r ← 0; if q > 0 then

begin if mod xref switch = 0 then

while num(q) ≥ def flag do begin r ← q; q ← xlink (q); end

else if num(q) ≥ def flag then begin r ← q; q ← xlink (q); end;

end; append xref (module count + mod xref switch); xlink (xref ptr ) ← q; mod xref switch ← 0; if r = 0 then xref [p] ← xref ptr else xlink (r) ← xref ptr ; end;

52. A third large area of memory is used for sixteen-bit ‘tokens’, which appear in short lists similar to the strings of characters in byte mem. Token lists are used to contain the result of Pascal code translated into TEX form; further details about them will be explained later. A text pointer variable is an index into tok start . ⟨ Types in the outer block 11 ⟩ +≡

text pointer = 0 . . max texts ; { identifies a token list } 53. The first position of tok mem that is unoccupied by replacement text is called tok ptr , and the first unused location of tok start is called text ptr . Thus, we usually have tok start [text ptr ] = tok ptr . ⟨ Globals in the outer block 9 ⟩ +≡

tok mem: packed array [0 . . max toks ] of sixteen bits ; { tokens } tok start : array [text pointer ] of sixteen bits ; { directory into tok mem } text ptr : text pointer ; { first unused position in tok start } tok ptr : 0 . . max toks ; { first unused position in tok mem } stat max tok ptr , max txt ptr : 0 . . max toks ; { largest values occurring } tats

54. ⟨ Set initial values 10 ⟩ +≡ tok ptr ← 1; text ptr ← 1; tok start [0] ← 1; tok start [1] ← 1; stat max tok ptr ← 1; max txt ptr ← 1; tats

18  SEARCHING FOR IDENTIFIERS  MWEAVE §55

55. Searching for identifiers. The hash table described above is updated by the id lookup procedure, which finds a given identifier and returns a pointer to its index in byte start . The identifier is supposed to match character by character and it is also supposed to have a given ilk code; the same name may be present more than once if it is supposed to appear in the index with different typesetting conventions. If the identifier was not already present, it is inserted into the table.

Because of the way ’s scanning mechanism works, it is most convenient to let id lookup search for an identifier that is present in the buffer array. Two other global variables specify its position in the buffer: the first character is buffer [id first ], and the last is buffer [id loc − 1]. ⟨ Globals in the outer block 9 ⟩ +≡ id first : 0 . . long buf size; { where the current identifier begins in the buffer } id loc: 0 . . long buf size; { just after the current identifier in the buffer } hash: array [0 . . hash size] of sixteen bits ; { heads of hash lists }

56. Initially all the hash lists are empty. ⟨ Local variables for initialization 16 ⟩ +≡ h: 0 . . hash size; { index into hash-head array }

57. ⟨ Set initial values 10 ⟩ +≡ for h ← 0 to hash size − 1 do hash[h] ← 0;

58. Here now is the main procedure for finding identifiers (and index entries). The parameter t is set to the desired ilk code. The identifier must either have ilk = t, or we must have t = normal and the identifier must be a reserved word. function id lookup(t : eight bits ): name pointer ; { finds current identifier }

label found ;

var i: 0 . . long buf size; { index into buffer } h: 0 . . hash size; { hash code } k: 0 . . max bytes ; { index into byte mem } w: 0 . . ww − 1; { row of byte mem } l: 0 . . long buf size; { length of the given identifier } p: name pointer ; { where the identifier is being sought }

begin l ← id loc − id first ; { compute the length }

⟨ Compute the hash code h 59 ⟩;

⟨ Compute the name location p 60 ⟩;

if p = name ptr then ⟨ Enter a new name into the table at position p 62 ⟩;

id lookup ← p;

end;

59. A simple hash code is used: If the sequence of ASCII codes is c_{1}c_{2} … c_{m}, its hash value will be

(2^{n−1}c_{1} + 2^{n−2}c_{2} + ⋯ + c_{n}) mod hash size.

⟨ Compute the hash code h 59 ⟩ ≡ h ← buffer [id first ]; i ← id first + 1; while i < id loc do

begin h ← (h + h + buffer [i]) mod hash size; incr (i); end

This code is used in section 58.

§60  MWEAVE  SEARCHING FOR IDENTIFIERS 19

60. If the identifier is new, it will be placed in position p = name ptr , otherwise p will point to its existing location. ⟨ Compute the name location p 60 ⟩ ≡

p ← hash[h]; while p ≠ 0 do

begin if (length(p) = l) ∧ ((ilk [p] = t) ∨ ((t = normal ) ∧ reserved (p))) then

⟨ Compare name p with current identifier, goto found if equal 61 ⟩; p ← link [p]; end;

p ← name ptr ; { the current identifier is new } link [p] ← hash[h]; hash[h] ← p; { insert p at beginning of hash list }

found :

This code is used in section 58.

61. ⟨ Compare name p with current identifier, goto found if equal 61 ⟩ ≡ begin i ← id first ; k ← byte start [p]; w ← p mod ww ; while (i < id loc) ∧ (buffer [i] = byte mem[w, k]) do

begin incr (i); incr (k);

end; if i = id loc then goto found ; { all characters agree } end

This code is used in section 60.

62. When we begin the following segment of the program, p = name ptr . ⟨ Enter a new name into the table at position p 62 ⟩ ≡

begin w ← name ptr mod ww ; if byte ptr [w] + l > max bytes then overflow (´ ␣ ´ ); if name ptr + ww > max names then overflow (´ ´ ); i ← id first ; k ← byte ptr [w]; { get ready to move the identifier into byte mem } while i < id loc do

begin byte mem[w, k] ← buffer [i]; incr (k); incr (i);

end; byte ptr [w] ← k; byte start [name ptr + ww ] ← k; incr (name ptr ); ilk [p] ← t; xref [p] ← 0; end

This code is used in section 58.

20  INITIALIZING THE TABLE OF RESERVED WORDS  MWEAVE §63

63* . Initializing the table of reserved words. We have to get Modula-2’s reserved words into the hash table, and the simplest way to do this is to insert them every time is run. A few macros permit us to do the initialization with a compact program.

define sid14 (#) ≡ buffer [14] ← #; cur name ← id lookup define sid13 (#) ≡ buffer [13] ← #; sid14 define sid12 (#) ≡ buffer [12] ← #; sid13 define sid11 (#) ≡ buffer [11] ← #; sid12 define sid10 (#) ≡ buffer [10] ← #; sid11 define sid9 (#) ≡ buffer [9] ← #; sid10 define sid8 (#) ≡ buffer [8] ← #; sid9 define sid7 (#) ≡ buffer [7] ← #; sid8 define sid6 (#) ≡ buffer [6] ← #; sid7 define sid5 (#) ≡ buffer [5] ← #; sid6 define sid4 (#) ≡ buffer [4] ← #; sid5 define sid3 (#) ≡ buffer [3] ← #; sid4 define sid2 (#) ≡ buffer [2] ← #; sid3 define sid1 (#) ≡ buffer [1] ← #; sid2 define id2 ≡ id first ← 13; sid13 define id3 ≡ id first ← 12; sid12 define id4 ≡ id first ← 11; sid11 define id5 ≡ id first ← 10; sid10 define id6 ≡ id first ← 9; sid9 define id7 ≡ id first ← 8; sid8 define id8 ≡ id first ← 7; sid7 define id9 ≡ id first ← 6; sid6 define id10 ≡ id first ← 5; sid5 define id11 ≡ id first ← 4; sid4 define id12 ≡ id first ← 3; sid3 define id13 ≡ id first ← 2; sid2 define id14 ≡ id first ← 1; sid1

⟨ Globals in the outer block 9 ⟩ +≡ cur name: name pointer ; { points to the identifier just inserted }

§64  MWEAVE  INITIALIZING THE TABLE OF RESERVED WORDS 21

64* . The intended use of the macros above might not be immediately obvious, but the riddle is answered by the following: ⟨ Store all the reserved words 64* ⟩ ≡

id loc ← 15; id3 ( )( )( )(char like + and sign); id5 ( )( )( )( )( )(array like); id5 ( )( )( )( )( )(begin like); id2 ( )( )(to like); id4 ( )( )( )( )(case like); id5 ( )( )( )( )( )(const like); id3 ( )( )( )(div like); id2 ( )( )(do like); id10 ( )( )( )( )( )( )( )( )( )( )(goto like); id4 ( )( )( )( )(else like); id5 ( )( )( )( )( )(elsif like); id3 ( )( )( )(end like); id4 ( )( )( )( )(goto like); id6 ( )( )( )( )( )( )(goto like); id3 ( )( )( )(for like); id4 ( )( )( )( )(goto like); id2 ( )( )(for like); id14 ( )( )( )( )( )( )( )( )( )( )( )( )( )( )(goto like); id6 ( )( )( )( )( )( )(goto like); id2 ( )( )(char like + set element sign); id5 ( )( )( )( )( )(const like); id4 ( )( )( )( )(loop like); id6 ( )( )( )( )( )( )(proc like); id3 ( )( )( )(div like); id3 ( )( )( )(nil like); id3 ( )( )( )(char like + not sign); id2 ( )( )(of like); id2 ( )( )(char like + or sign); id7 ( )( )( )( )( )( )( )(goto like); id9 ( )( )( )( )( )( )( )( )( )(proc like); id9 ( )( )( )( )( )( )( )( )( )(goto like); id6 ( )( )( )( )( )( )(record like); id6 ( )( )( )( )( )( )(repeat like); id6 ( )( )( )( )( )( )(goto like); id3 ( )( )( )(array like); id4 ( )( )( )( )(do like); id2 ( )( )(to like); id4 ( )( )( )( )(const like); id5 ( )( )( )( )( )(until like); id3 ( )( )( )(var like); id5 ( )( )( )( )( )(for like); id4 ( )( )( )( )(for like);

This code is used in section 261.

22  SEARCHING FOR MODULE NAMES  MWEAVE §65

65. Searching for module names. The mod lookup procedure finds the module name mod text [1 . . l] in the search tree, after inserting it if necessary, and returns a pointer to where it was found. ⟨ Globals in the outer block 9 ⟩ +≡ mod text : array [0 . . longest name] of ASCII code; { name being sought for }

66. According to the rules of , no module name should be a proper prefix of another, so a “clean” comparison should occur between any two names. The result of mod lookup is 0 if this prefix condition is violated. An error message is printed when such violations are detected during phase two of .

define less = 0 { the first name is lexicographically less than the second } define equal = 1 { the first name is equal to the second } define greater = 2 { the first name is lexicographically greater than the second } define prefix = 3 { the first name is a proper prefix of the second } define extension = 4 { the first name is a proper extension of the second }

function mod lookup(l : sixteen bits ): name pointer ; { finds module name } label found ; var c: less . . extension; { comparison between two names }

j: 0 . . longest name; { index into mod text }

k: 0 . . max bytes ; { index into byte mem }

w: 0 . . ww − 1; { row of byte mem }

p: name pointer ; { current node of the search tree }

q: name pointer ; { father of node p } begin c ← greater ; q ← 0; p ← root ; while p ≠ 0 do

begin ⟨ Set variable c to the result of comparing the given name to name p 68 ⟩; q ← p; if c = less then p ← llink [q] else if c = greater then p ← rlink [q]

else goto found ; end;

⟨ Enter a new module name into the tree 67 ⟩; found : if c ≠ equal then

begin err print (´ ␣ ␣ ␣ ´ ); p ← 0;

end; mod lookup ← p; end;

67. ⟨ Enter a new module name into the tree 67 ⟩ ≡ w ← name ptr mod ww ; k ← byte ptr [w]; if k + l > max bytes then overflow (´ ␣ ´ ); if name ptr > max names − ww then overflow (´ ´ ); p ← name ptr ; if c = less then llink [q] ← p else rlink [q] ← p; llink [p] ← 0; rlink [p] ← 0; xref [p] ← 0; c ← equal ; for j ← 1 to l do byte mem[w, k + j − 1] ← mod text [j]; byte ptr [w] ← k + l; byte start [name ptr + ww ] ← k + l; incr (name ptr );

This code is used in section 66.

§68  MWEAVE  SEARCHING FOR MODULE NAMES 23

68. ⟨ Set variable c to the result of comparing the given name to name p 68 ⟩ ≡ begin k ← byte start [p]; w ← p mod ww ; c ← equal ; j ← 1; while (k < byte start [p + ww ]) ∧ (j ≤ l) ∧ (mod text [j] = byte mem[w, k]) do

begin incr (k); incr (j); end;

if k = byte start [p + ww ] then if j > l then c ← equal else c ← extension

else if j > l then c ← prefix

else if mod text [j] < byte mem[w, k] then c ← less

else c ← greater ; end

This code is used in sections 66 and 69.

69. The prefix lookup procedure is supposed to find exactly one module name that has mod text [1 . . l] as a prefix. Actually the algorithm silently accepts also the situation that some module name is a prefix of mod text [1 . . l], because the user who painstakingly typed in more than necessary probably doesn’t want to be told about the wasted effort.

Recall that error messages are not printed during phase one. It is possible that the prefix lookup procedure will fail on the first pass, because there is no match, yet the second pass might detect no error if a matching module name has occurred after the offending prefix. In such a case the cross-reference information will be incorrect and will report no error. However, such a mistake will be detected by the processor. function prefix lookup(l : sixteen bits ): name pointer ; { finds name extension }

var c: less . . extension; { comparison between two names } count : 0 . . max names ; { the number of hits } j: 0 . . longest name; { index into mod text } k: 0 . . max bytes ; { index into byte mem } w: 0 . . ww − 1; { row of byte mem } p: name pointer ; { current node of the search tree } q: name pointer ; { another place to resume the search after one branch is done } r: name pointer ; { extension found }

begin q ← 0; p ← root ; count ← 0; r ← 0; { begin search at root of tree } while p ≠ 0 do

begin ⟨ Set variable c to the result of comparing the given name to name p 68 ⟩; if c = less then p ← llink [p] else if c = greater then p ← rlink [p]

else begin r ← p; incr (count ); q ← rlink [p]; p ← llink [p];

end; if p = 0 then

begin p ← q; q ← 0;

end; end;

if count ≠ 1 then if count = 0 then err print (´ ␣ ␣ ␣ ␣ ´ ) else err print (´ ␣ ␣ ´ );

24  LEXICAL SCANNING  MWEAVE §70

prefix lookup ← r; { the result will be 0 if there was no match } end; 70. Lexical scanning. Let us now consider the subroutines that read the source file and break it into meaningful units. There are four such procedures: One simply skips to the next ‘ ␣’ or ‘ ’ that begins a module; another passes over the TEX text at the beginning of a module; the third passes over the TEX text in a Pascal comment; and the last, which is the most interesting, gets the next token of a Pascal text.

71. But first we need to consider the low-level routine get line that takes care of merging change file into web file. The get line procedure also updates the line numbers for error messages. ⟨ Globals in the outer block 9 ⟩ +≡ line: integer ; { the number of the current line in the current file } other line: integer ; { the number of the current line in the input file that is not currently being read } temp line: integer ; { used when interchanging line with other line } limit : 0 . . long buf size; { the last character position occupied in the buffer } loc: 0 . . long buf size; { the next character position to be read from the buffer } input has ended : boolean; { if true, there is no more input } changing : boolean; { if true, the current line is from change file }

72. As we change changing from true to false and back again, we must remember to swap the values of line and other line so that the err print routine will be sure to report the correct line number.

define change changing ≡ changing ← ¬changing ; temp line ← other line; other line ← line; line ← temp line { line ↔ other line }

73. When changing is false, the next line of change file is kept in change buffer [0 . . change limit ], for purposes of comparison with the next line of web file. After the change file has been completely input, we set change limit ← 0, so that no further matches will be made. ⟨ Globals in the outer block 9 ⟩ +≡ change buffer : array [0 . . buf size] of ASCII code; change limit : 0 . . buf size; { the last position occupied in change buffer }

74. Here’s a simple function that checks if the two buffers are different. function lines dont match: boolean;

label exit ;

var k: 0 . . buf size; { index into the buffers }

begin lines dont match ← true;

if change limit ≠ limit then return;

if limit > 0 then

for k ← 0 to limit − 1 do

if change buffer [k] ≠ buffer [k] then return; lines dont match ← false;

§75  MWEAVE  LEXICAL SCANNING 25

exit : end; 75. Procedure prime the change buffer sets change buffer in preparation for the next matching operation. Since blank lines in the change file are not used for matching, we have (change limit = 0) ∧ ¬changing if and only if the change file is exhausted. This procedure is called only when changing is true; hence error messages will be reported correctly. procedure prime the change buffer ;

label continue, done, exit ;

var k: 0 . . buf size; { index into the buffers }

begin change limit ← 0; { this value will be used if the change file ends } ⟨ Skip over comment lines in the change file; return if end of file 76 ⟩; ⟨ Skip to the next nonblank line; return if end of file 77 ⟩; ⟨ Move buffer and limit to change buffer and change limit 78 ⟩;

exit : end;

76. While looking for a line that begins with in the change file, we allow lines that begin with , as long as they don’t begin with or (which would probably indicate that the change file is fouled up). ⟨ Skip over comment lines in the change file; return if end of file 76 ⟩ ≡

loop begin incr (line); if ¬input ln(change file) then return; if limit < 2 then goto continue; if buffer [0] ≠ then goto continue; if (buffer [1] ≥ ) ∧ (buffer [1] ≤ ) then buffer [1] ← buffer [1] + − ; { lowercasify } if buffer [1] = then goto done; if (buffer [1] = ) ∨ (buffer [1] = ) then

begin loc ← 2; err print (´ ␣ ␣ ␣ ␣ ␣ ´ ); end;

continue: end; done:

This code is used in section 75.

77. Here we are looking at lines following the . ⟨ Skip to the next nonblank line; return if end of file 77 ⟩ ≡

repeat incr (line);

if ¬input ln(change file) then begin err print (´ ␣ ␣ ␣ ␣ ␣ ´ ); return; end;

until limit > 0;

This code is used in section 75.

78. ⟨ Move buffer and limit to change buffer and change limit 78 ⟩ ≡ begin change limit ← limit ; if limit > 0 then

for k ← 0 to limit − 1 do change buffer [k] ← buffer [k]; end

This code is used in sections 75 and 79.

79. The following procedure is used to see if the next change entry should go into effect; it is called only when changing is false. The idea is to test whether or not the current contents of buffer matches the current contents of change buffer . If not, there’s nothing more to do; but if so, a change is called for: All of the text down to the is supposed to match. An error message is issued if any discrepancy is found. Then the procedure prepares to read the next line from change file. procedure check change; { switches to change file if the buffers match }

label exit ; var n: integer ; { the number of discrepancies found }

k: 0 . . buf size; { index into the buffers } begin if lines dont match then return; n ← 0; loop begin change changing ; { now it’s true }

incr (line); if ¬input ln(change file) then

begin err print (´ ␣ ␣ ␣ ␣ ␣ ´ ); change limit ← 0; change changing ;

{ false again } return; end;

⟨ If the current line starts with , report any discrepancies and return 80 ⟩; ⟨ Move buffer and limit to change buffer and change limit 78 ⟩; change changing ; { now it’s false } incr (line); if ¬input ln(web file) then

begin err print (´ ␣ ␣ ␣ ␣ ␣ ␣ ´ ); input has ended ← true; return; end;

if lines dont match then incr (n); end;

exit : end;

80. ⟨ If the current line starts with , report any discrepancies and return 80 ⟩ ≡

if limit > 1 then

if buffer [0] = then

begin if (buffer [1] ≥ ) ∧ (buffer [1] ≤ ) then buffer [1] ← buffer [1] + − ;

{ lowercasify } if (buffer [1] = ) ∨ (buffer [1] = ) then

begin loc ← 2; err print (´ ␣ ␣ ␣ ␣ ␣ ´ ); end

else if buffer [1] = then

begin if n > 0 then begin loc ← 2; err print (´ ␣ ␣´ , n : 1, ´ ␣ ␣ ␣ ␣ ␣ ␣ ␣ ´ ); end; return; end;

end

This code is used in section 79.

81. The reset input procedure, which gets  ready to read the user’s  input, is used at the beginning

of phases one and two. procedure reset input ;

begin open input ; line ← 0; other line ← 0; changing ← true; prime the change buffer ; change changing ; limit ← 0; loc ← 1; buffer [0] ← ␣ ; input has ended ← false; end;

82. The get line procedure is called when loc > limit ; it puts the next line of merged input into the buffer and updates the other variables appropriately. A space is placed at the right end of the line. procedure get line; { inputs the next line }

label restart ; begin restart : if changing then changed module[module count ] ← true else ⟨ Read from web file and maybe turn on changing 83 ⟩;

if changing then

begin ⟨ Read from change file and maybe turn off changing 84 ⟩;

if ¬changing then begin changed module[module count ] ← true; goto restart ; end;

end; loc ← 0; buffer [limit ] ← ␣ ; end;

83. ⟨ Read from web file and maybe turn on changing 83 ⟩ ≡ begin incr (line); if ¬input ln(web file) then input has ended ← true else if limit = change limit then

if buffer [0] = change buffer [0] then

if change limit > 0 then check change; end

This code is used in section 82.

84. ⟨ Read from change file and maybe turn off changing 84 ⟩ ≡ begin incr (line); if ¬input ln(change file) then

begin err print (´ ␣ ␣ ␣ ␣ ␣ ´ ); buffer [0] ← ; buffer [1] ← ; limit ← 2; end;

if limit > 1 then { check if the change has ended }

if buffer [0] = then

begin if (buffer [1] ≥ ) ∧ (buffer [1] ≤ ) then buffer [1] ← buffer [1] + − ;

{ lowercasify } if (buffer [1] = ) ∨ (buffer [1] = ) then

begin loc ← 2; err print (´ ␣ ␣ ␣ ␣ ␣ ´ ); end

else if buffer [1] = then begin prime the change buffer ; change changing ; end;

end; end

This code is used in section 82.

85. At the end of the program, we will tell the user if the change file had a line that didn’t match any relevant line in web file. ⟨ Check that all changes have been read 85 ⟩ ≡

if change limit ≠ 0 then { changing is false } begin for loc ← 0 to change limit do buffer [loc] ← change buffer [loc]; limit ← change limit ; changing ← true; line ← other line; loc ← change limit ; err print (´ ␣ ␣ ␣ ␣ ␣ ␣ ´ ); end

This code is used in section 261.

86* . Control codes in , which begin with ‘ ’, are converted into a numeric code designed to simplify ’s logic; for example, larger numbers are given to the control codes that denote more significant milestones, and the code of new module should be the largest of all. Some of these numeric control codes take the place of ASCII control codes that will not otherwise appear in the output of the scanning routines.

define ignore = 0 { control code of no interest to } define verbatim = ´ 2 { extended ASCII alpha will not appear } define force line = ´ 3 { extended ASCII beta will not appear } define vertical bar = 15 { control code for vertical bar character } define begin regular comment = 16 { control code for beginning of comments } define end regular comment = 17 { control code for ending of comments } define begin comment = ´ 11 { ASCII tab mark will not appear } define end comment = ´ 12 { ASCII line feed will not appear } define octal = ´ 14 { ASCII form feed will not appear } define hex = ´ 15 { ASCII carriage return will not appear } define double dot = ´ 40 { ASCII space will not appear except in strings } define no underline = ´ 175 { this code will be intercepted without confusion } define underline = ´ 176 { this code will be intercepted without confusion } define param = ´ 177 { ASCII delete will not appear } define xref roman = ´ 203 { control code for ‘ ’ } define xref wildcard = ´ 204 { control code for ‘ ’ } define xref typewriter = ´ 205 { control code for ‘ ’ } define TeX string = ´ 206 { control code for ‘ ’ } define check sum = ´ 207 { control code for ‘ $’ } define join = ´ 210 { control code for ‘ ’ } define thin space = ´ 211 { control code for ‘ ’ } define math break = ´ 212 { control code for ‘ ’ } define line break = ´ 213 { control code for ‘ ’ } define big line break = ´ 214 { control code for ‘ #’ } define no line break = ´ 215 { control code for ‘ ’ } define pseudo semi = ´ 216 { control code for ‘ ’ } define format = ´ 217 { control code for ‘ ’ } define definition = ´ 220 { control code for ‘ ’ } define begin Pascal = ´ 221 { control code for ‘ ’ } define module name = ´ 222 { control code for ‘ ’ } define new module = ´ 223 { control code for ‘ ␣’ and ‘ ’ } 87. Control codes are converted from ASCII to ’s internal representation by the control code routine. function control code(c : ASCII code): eight bits ; { convert c after }

begin case c of

: control code ← ; { ‘quoted’ at sign } ´ : control code ← octal ; { precedes octal constant } : control code ← hex ; { precedes hexadecimal constant } $ : control code ← check sum; { precedes check sum constant } ␣ , tab mark , : control code ← new module; { beginning of a new module } : control code ← verbatim; : control code ← force line; , : control code ← definition; { macro definition } , : control code ← format ; { format definition } : control code ← begin comment ; { begin-comment delimiter } : control code ← end comment ; { end-comment delimiter } , : control code ← begin Pascal ; { Pascal text in unnamed module } : control code ← join; { concatenate two tokens } : control code ← module name; { beginning of a module name } : begin err print (´ ␣ ␣ ´ ); control code ← ignore;

end; { end of module name should not be discovered in this way } , : control code ← TeX string ; { TEX box within Pascal } : control code ← underline; { set definition flag } : control code ← no underline; { reset definition flag } : control code ← xref roman; { index entry to be typeset normally } : control code ← xref wildcard ; { index entry to be in user format } : control code ← xref typewriter ; { index entry to be in typewriter type } : control code ← thin space; { puts extra space in Pascal format } : control code ← math break ; { allows a break in a formula } : control code ← line break ; { forces end-of-line in Pascal format } # : control code ← big line break ; { forces end-of-line and some space besides } : control code ← no line break ; { cancels end-of-line down to single space } : control code ← pseudo semi ; { acts like a semicolon, but is invisible } ⟨ Special control codes allowed only when debugging 88 ⟩ othercases begin err print (´ ␣ ␣ ␣ ´ ); control code ← ignore;

end endcases; end;

88. If is compiled with debugging commands, one can write , , and to turn tracing fully on, partly on, and off, respectively. ⟨ Special control codes allowed only when debugging 88 ⟩ ≡

debug , , : begin tracing ← c − ; control code ← ignore;

end; gubed

This code is used in section 87.

89. The skip limbo routine is used on the first pass to skip through portions of the input that are not in any modules, i.e., that precede the first module. After this procedure has been called, the value of input has ended will tell whether or not a new module has actually been found. procedure skip limbo; { skip to next module }

label exit ; var c: ASCII code; { character following }

begin loop

if loc > limit then begin get line; if input has ended then return; end

else begin buffer [limit + 1] ← ; while buffer [loc] ≠ do incr (loc); if loc ≤ limit then

begin loc ← loc + 2; c ← buffer [loc − 1];

if (c = ␣ ) ∨ (c = tab mark ) ∨ (c = ) then return; end;

end; exit : end;

90. The skip TeX routine is used on the first pass to skip through the TEX code at the beginning of a module. It returns the next control code or ‘ ’ found in the input. A new module is assumed to exist at the very end of the file. function skip TeX : eight bits ; { skip past pure TEX code }

label done; var c: eight bits ; { control code found }

begin loop

begin if loc > limit then begin get line;

if input has ended then

begin c ← new module; goto done;

end; end;

buffer [limit + 1] ← ; repeat c ← buffer [loc]; incr (loc);

if c = then goto done; until c = ; if loc ≤ limit then

begin c ← control code(buffer [loc]); incr (loc); goto done; end;

end; done: skip TeX ← c;

end; 91. The skip comment routine is used on the first pass to skip through TEX code in Pascal comments. The bal parameter tells how many left braces are assumed to have been scanned when this routine is called, and the procedure returns a corresponding value of bal at the point that scanning has stopped. Scanning stops either at a ‘ ’ that introduces Pascal text, in which case the returned value is positive, or it stops at the end of the comment, in which case the returned value is zero. The scanning also stops in anomalous situations when the comment doesn’t end or when it contains an illegal use of . One should call skip comment (1) when beginning to scan a comment. function skip comment (bal : eight bits ): eight bits ; { skips TEX code in comments }

label done;

var c: ASCII code; { the current character }

begin loop

begin if loc > limit then begin get line;

if input has ended then

begin bal ← 0; goto done;

end; { an error message will occur in phase two } end;

c ← buffer [loc]; incr (loc); if c = then goto done; ⟨ Do special things when c = , , , ; goto done at end 92* ⟩; end;

done: skip comment ← bal ; end;

92* . ⟨ Do special things when c = , , , ; goto done at end 92* ⟩ ≡

if c = then begin c ← buffer [loc]; if (c ≠ ␣ ) ∧ (c ≠ tab mark ) ∧ (c ≠ ) then incr (loc) else begin decr (loc); bal ← 0; goto done;

end { an error message will occur in phase two } end

else if (c = ) ∧ (buffer [loc] ≠ ) then incr (loc)

else if c = begin regular comment then incr (bal )

else if c = end regular comment then begin decr (bal ); if bal = 0 then goto done; end

else if (c = ) ∧ (buffer [loc] = ) then begin incr (bal ); incr (loc); end

else if (c = ) ∧ (buffer [loc] = ) then begin decr (bal ); incr (loc); c ← end regular comment ; if bal = 0 then goto done; end

This code is used in section 91.

32  INPUTTING THE NEXT TOKEN  MWEAVE §93

93. Inputting the next token. As stated above, ’s most interesting lexical scanning routine is the get next function that inputs the next token of Pascal input. However, get next is not especially complicated.

The result of get next is either an ASCII code for some special character, or it is a special code representing a pair of characters (e.g., ‘ ’ or ‘ ’), or it is the numeric value computed by the control code procedure, or it is one of the following special codes:

exponent : The ‘ ’ in a real constant.

identifier : In this case the global variables id first and id loc will have been set to the appropriate values

needed by the id lookup routine. string : In this case the global variables id first and id loc will have been set to the beginning and ending-

plus-one locations in the buffer. The string ends with the first reappearance of its initial delimiter; thus, for example,

                                 ´´´´

will be treated as two consecutive strings, the first being ´ ´ . Furthermore, some of the control codes cause get next to take additional actions:

xref roman, xref wildcard , xref typewriter , TeX string : The values of id first and id loc will be set so that

the string in question appears in buffer [id first . . (id loc − 1)]. module name: In this case the global variable cur module will point to the byte start entry for the module

name that has just been scanned. If get next sees ‘ ’ or ‘ ’, it sets xref switch to def flag or zero and goes on to the next token.

A global variable called scanning hex is set true during the time that the letters through should be treated as if they were digits.

define exponent = ´ 200 { or following a digit }

define string = ´ 201 { Pascal string or precomputed string }

define identifier = ´ 202 { Pascal identifier or reserved word } ⟨ Globals in the outer block 9 ⟩ +≡ cur module: name pointer ; { name of module just scanned } scanning hex : boolean; { are we scanning a hexadecimal constant? }

§95  MWEAVE  INPUTTING THE NEXT TOKEN 33

94. ⟨ Set initial values 10 ⟩ +≡ scanning hex ← false; 95. As one might expect, get next consists mostly of a big switch that branches to the various special cases that can arise.

define up to(#) ≡ # − 24, # − 23, # − 22, # − 21, # − 20, # − 19, # − 18, # − 17, # − 16, # − 15, # − 14, # − 13,

# − 12, # − 11, # − 10, # − 9, # − 8, # − 7, # − 6, # − 5, # − 4, # − 3, # − 2, # − 1, #

function get next : eight bits ; { produces the next input token } label restart , done, found ; var c: eight bits ; { the current character }

d: eight bits ; { the next character }

j, k: 0 . . longest name; { indices into mod text } begin restart : if loc > limit then

begin get line;

if input has ended then

begin c ← new module; goto found ; end;

end; c ← buffer [loc]; incr (loc); if scanning hex then ⟨ Go to found if c is a hexadecimal digit, otherwise set scanning hex ← false 96 ⟩;

case c of

, up to( ), , up to( ): ⟨ Get an identifier 98 ⟩; ´ , : ⟨ Get a string 99 ⟩; : ⟨ Get control code and possible module name 100 ⟩; ⟨ Compress two-symbol combinations like ‘ ’ 97* ⟩ ␣ , tab mark : goto restart ; { ignore spaces and tabs }

othercases do nothing

endcases; found : debug if trouble shooting then debug help; gubed

get next ← c; end;

96. ⟨ Go to found if c is a hexadecimal digit, otherwise set scanning hex ← false 96 ⟩ ≡ if ((c ≥ ) ∧ (c ≤ )) ∨ ((c ≥ ) ∧ (c ≤ )) then goto found else scanning hex ← false

This code is used in section 95.

97* . Note that the following code converts ‘ ’ and ‘ ’ to internal characters which are converted back to ASCII when output. Adjacent vertical bar characters (´ ´ ), which represent the Modula-2 usage of that character rather than that of WEB, are handled the same way.

define compress (#) ≡

begin if loc ≤ limit then begin c ← #; incr (loc); end; end

⟨ Compress two-symbol combinations like ‘ ’ 97* ⟩ ≡ : if buffer [loc] = then compress (double dot )

else if buffer [loc] = then compress ( ); : if buffer [loc] = then compress (left arrow ); : if buffer [loc] = then compress (equivalence sign); : if buffer [loc] = then compress (greater or equal ); : if buffer [loc] = then compress (less or equal )

else if buffer [loc] = then compress (not equal ); : if buffer [loc] = then compress (begin regular comment )

else if buffer [loc] = then compress ( ); : if buffer [loc] = then compress (end regular comment ); : if buffer [loc] = then compress (vertical bar );

This code is used in section 95.

98. ⟨ Get an identifier 98 ⟩ ≡

begin if ((c = ) ∨ (c = )) ∧ (loc > 1) then

if (buffer [loc − 2] ≤ ) ∧ (buffer [loc − 2] ≥ ) then c ← exponent ; if c ≠ exponent then

begin decr (loc); id first ← loc; repeat incr (loc); d ← buffer [loc]; until ((d < ) ∨ ((d > ) ∧ (d < )) ∨ ((d > ) ∧ (d < )) ∨ (d > )) ∧ (d ≠ ); c ← identifier ; id loc ← loc; end;

end

This code is used in section 95.

99. A string that starts and ends with single or double quote marks is scanned by the following piece of the program. ⟨ Get a string 99 ⟩ ≡

begin id first ← loc − 1;

repeat d ← buffer [loc]; incr (loc);

if loc > limit then begin err print (´ ␣ ␣ ␣ ´ ´ ␣ ´ ); loc ← limit ; d ← c; end;

until d = c;

id loc ← loc; c ← string ;

end

This code is used in section 95.

100. After an sign has been scanned, the next character tells us whether there is more work to do. ⟨ Get control code and possible module name 100 ⟩ ≡

begin c ← control code(buffer [loc]); incr (loc); if c = underline then

begin xref switch ← def flag ; goto restart ;

end else if c = no underline then

begin xref switch ← 0; goto restart ; end

else if (c ≤ TeX string ) ∧ (c ≥ xref roman) then ⟨ Scan to the next 106 ⟩

else if c = hex then scanning hex ← true

else if c = module name then ⟨ Scan the module name and make cur module point to it 101 ⟩

else if c = verbatim then ⟨ Scan a verbatim string 107 ⟩;

end

This code is used in section 95.

101. The occurrence of a module name sets xref switch to zero, because the module name might (for example) follow var. ⟨ Scan the module name and make cur module point to it 101 ⟩ ≡

begin ⟨ Put module name into mod text [1 . . k] 103 ⟩; if k > 3 then

begin if (mod text [k] = ) ∧ (mod text [k − 1] = ) ∧ (mod text [k − 2] = ) then

cur module ← prefix lookup(k − 3) else cur module ← mod lookup(k); end

else cur module ← mod lookup(k); xref switch ← 0; end

This code is used in section 100.

102. Module names are placed into the mod text array with consecutive spaces, tabs, and carriage-returns replaced by single spaces. There will be no spaces at the beginning or the end. (We set mod text [0] ← ␣ to facilitate this, since the mod lookup routine uses mod text [1] as the first character of the name.) ⟨ Set initial values 10 ⟩ +≡

mod text [0] ← ␣ ; 103. ⟨ Put module name into mod text [1 . . k] 103 ⟩ ≡

k ← 0; loop begin if loc > limit then

begin get line;

if input has ended then

begin err print (´ ␣ ␣ ␣ ␣ ␣ ´ ); loc ← 1; goto done; end;

end; d ← buffer [loc]; ⟨ If end of name, goto done 104 ⟩; incr (loc); if k < longest name − 1 then incr (k); if (d = ␣ ) ∨ (d = tab mark ) then

begin d ← ␣ ;

if mod text [k − 1] = ␣ then decr (k);

end; mod text [k] ← d; end;

done: ⟨ Check for overlong name 105 ⟩;

if (mod text [k] = ␣ ) ∧ (k > 0) then decr (k)

This code is used in section 101.

104. ⟨ If end of name, goto done 104 ⟩ ≡

if d = then begin d ← buffer [loc + 1]; if d = then

begin loc ← loc + 2; goto done;

end; if (d = ␣ ) ∨ (d = tab mark ) ∨ (d = ) then

begin err print (´ ␣ ␣ ␣ ´ ´ ␣ ´ ); goto done; end;

incr (k); mod text [k] ← ; incr (loc); { now d = buffer [loc] again } end

This code is used in section 103.

105. ⟨ Check for overlong name 105 ⟩ ≡

if k ≥ longest name − 2 then begin print nl (´ ␣ ␣ ␣ ␣ ␣´ ); for j ← 1 to 25 do print (xchr [mod text [j]]); print (´ ´ ); mark harmless ; end

This code is used in section 103.

106. ⟨ Scan to the next 106 ⟩ ≡ begin id first ← loc; buffer [limit + 1] ← ; while buffer [loc] ≠ do incr (loc); id loc ← loc; if loc > limit then

begin err print (´ ␣ ␣ ␣ ´ ´ ␣ ´ ); loc ← limit ;

end else begin loc ← loc + 2;

if buffer [loc − 1] ≠ then err print (´ ␣ ␣ ␣ ␣ ␣ ␣ ␣ ´ ); end;

end

This code is used in section 100.

107. A verbatim Pascal string will be treated like ordinary strings, but with no surrounding delimiters. At the present point in the program we have buffer [loc − 1] = verbatim; we must set id first to the beginning of the string itself, and id loc to its ending-plus-one location in the buffer. We also set loc to the position just after the ending delimiter. ⟨ Scan a verbatim string 107 ⟩ ≡

begin id first ← loc; incr (loc); buffer [limit + 1] ← ; buffer [limit + 2] ← ; while (buffer [loc] ≠ ) ∨ (buffer [loc + 1] ≠ ) do incr (loc); if loc ≥ limit then err print (´ ␣ ␣ ␣ ´ ´ ␣ ´ ); id loc ← loc; loc ← loc + 2; end

This code is used in section 100.

38  PHASE ONE PROCESSING  MWEAVE §108

108. Phase one processing. We now have accumulated enough subroutines to make it possible to carry out ’s first pass over the source file. If everything works right, both phase one and phase two of will assign the same numbers to modules, and these numbers will agree with what does.

The global variable next control often contains the most recent output of get next ; in interesting cases, this will be the control code that ended a module or part of a module. ⟨ Globals in the outer block 9 ⟩ +≡ next control : eight bits ; { control code waiting to be acting upon }

109. The overall processing strategy in phase one has the following straightforward outline. ⟨ Phase I: Read all the user’s text and store the cross references 109 ⟩ ≡

phase one ← true; phase three ← false; reset input ; module count ← 0; skip limbo; change exists ← false; while ¬input has ended do ⟨ Store cross reference data for the current module 110 ⟩; changed module[module count ] ← change exists ; { the index changes if anything does } phase one ← false; { prepare for second phase } ⟨ Print error messages about unused or undefined module names 120 ⟩;

This code is used in section 261.

110. ⟨ Store cross reference data for the current module 110 ⟩ ≡ begin incr (module count ); if module count = max modules then overflow (´ ␣ ´ ); changed module[module count ] ← false; { it will become true if any line changes } if buffer [loc − 1] = then

begin print (´ ´ , module count : 1); update terminal ; { print a progress report } end;

⟨ Store cross references in the TEX part of a module 113 ⟩; ⟨ Store cross references in the definition part of a module 115 ⟩; ⟨ Store cross references in the Pascal part of a module 117 ⟩; if changed module[module count ] then change exists ← true; end

This code is used in section 109.

§111  MWEAVE  PHASE ONE PROCESSING 39

111* . The Pascal xref subroutine stores references to identifiers in Pascal text material beginning with the current value of next control and continuing until next control is ‘ ’ or ‘ ’, or until the next “milestone” is passed (i.e., next control ≥ format ). If next control ≥ format when Pascal xref is called, nothing will happen; but if next control = upon entry, the procedure assumes that this is the ‘ ’ preceding Pascal text that is to be processed.

The program uses the fact that our internal code numbers satisfy the relations xref roman = identifier + roman and xref wildcard = identifier +wildcard and xref typewriter = identifier +typewriter and normal = 0. An implied ‘ ’ is inserted after function, procedure, program, and var. procedure Pascal xref ; { makes cross references for Pascal identifiers }

label exit ;

var p: name pointer ; { a referenced name }

begin while next control < format do

begin if (next control ≥ identifier ) ∧ (next control ≤ xref typewriter ) then begin p ← id lookup(next control − identifier ); new xref (p); if (ilk [p] = proc like) ∨ (ilk [p] = var like) then xref switch ← def flag ; { implied ‘ ’ } end;

next control ← get next ;

if (next control = ) ∨ (next control = begin regular comment ) then return; end;

exit : end;

112* . The outer xref subroutine is like Pascal xref but it begins with next control ≠ and ends with next control ≥ format . Thus, it handles Pascal text with embedded comments. procedure outer xref ; { extension of Pascal xref }

var bal : eight bits ; { brace level in comment }

begin while next control < format do if next control ≠ begin regular comment then Pascal xref else begin bal ← skip comment (1); next control ← ;

while bal > 0 do begin Pascal xref ; if next control = then bal ← skip comment (bal ) else bal ← 0; { an error will be reported in phase two } end; end;

end; 113. In the TEX part of a module, cross reference entries are made only for the identifiers in Pascal texts enclosed in …, or for control texts enclosed in … or … or …. ⟨ Store cross references in the TEX part of a module 113 ⟩ ≡

repeat next control ← skip TeX ;

case next control of

underline: xref switch ← def flag ; no underline: xref switch ← 0; : Pascal xref ; xref roman, xref wildcard , xref typewriter , module name: begin loc ← loc − 2;

next control ← get next ; { scan to }

if next control ≠ module name then new xref (id lookup(next control − identifier )); end;

othercases do nothing

endcases; until next control ≥ format

This code is used in section 110.

114. During the definition and Pascal parts of a module, cross references are made for all identifiers except reserved words; however, the identifiers in a format definition are referenced even if they are reserved. The TEX code in comments is, of course, ignored, except for Pascal portions enclosed in …; the text of a module name is skipped entirely, even if it contains … constructions.

The variables lhs and rhs point to the respective identifiers involved in a format definition. ⟨ Globals in the outer block 9 ⟩ +≡ lhs , rhs : name pointer ; { indices into byte start for format identifiers }

115. When we get to the following code we have next control ≥ format . ⟨ Store cross references in the definition part of a module 115 ⟩ ≡

while next control ≤ definition do { format or definition } begin xref switch ← def flag ; { implied } if next control = definition then next control ← get next else ⟨ Process a format definition 116 ⟩; outer xref ; end

This code is used in section 110.

116. Error messages for improper format definitions will be issued in phase two. Our job in phase one is to define the ilk of a properly formatted identifier, and to fool the new xref routine into thinking that the identifier on the right-hand side of the format definition is not a reserved word. ⟨ Process a format definition 116 ⟩ ≡

begin next control ← get next ; if next control = identifier then

begin lhs ← id lookup(normal ); ilk [lhs ] ← normal ; new xref (lhs ); next control ← get next ; if next control = equivalence sign then

begin next control ← get next ;

if next control = identifier then begin rhs ← id lookup(normal ); ilk [lhs ] ← ilk [rhs ]; ilk [rhs ] ← normal ; new xref (rhs ); ilk [rhs ] ← ilk [lhs ]; next control ← get next ; end;

end; end;

end

This code is used in section 115.

117. Finally, when the TEX and definition parts have been treated, we have next control ≥ begin Pascal . ⟨ Store cross references in the Pascal part of a module 117 ⟩ ≡

if next control ≤ module name then { begin Pascal or module name } begin if next control = begin Pascal then mod xref switch ← 0 else mod xref switch ← def flag ; repeat if next control = module name then new mod xref (cur module);

next control ← get next ; outer xref ; until next control > module name; end

This code is used in section 110.

118. After phase one has looked at everything, we want to check that each module name was both defined and used. The variable cur xref will point to cross references for the current module name of interest. ⟨ Globals in the outer block 9 ⟩ +≡ cur xref : xref number ; { temporary cross reference pointer }

119. The following recursive procedure walks through the tree of module names and prints out anomalies. procedure mod check (p : name pointer ); { print anomalies in subtree p }

begin if p > 0 then begin mod check (llink [p]); cur xref ← xref [p]; if num(cur xref ) < def flag then

begin print nl (´ ␣ ␣ ␣ ´ ); print id (p); print (´ ´ ); mark harmless ; end;

while num(cur xref ) ≥ def flag do cur xref ← xlink (cur xref ); if cur xref = 0 then

begin print nl (´ ␣ ␣ ␣ ´ ); print id (p); print (´ ´ ); mark harmless ; end;

mod check (rlink [p]); end;

end;

120. ⟨ Print error messages about unused or undefined module names 120 ⟩ ≡ mod check (root )

This code is used in section 109.

42  LOW-LEVEL OUTPUT ROUTINES  MWEAVE §121

121. Low-level output routines. The TEX output is supposed to appear in lines at most line length characters long, so we place it into an output buffer. During the output process, out line will hold the current line number of the line about to be output. ⟨ Globals in the outer block 9 ⟩ +≡ out buf : array [0 . . line length] of ASCII code; { assembled characters } out ptr : 0 . . line length; { number of characters in out buf } out line: integer ; { coordinates of next line to be output }

122. The flush buffer routine empties the buffer up to a given breakpoint, and moves any remaining characters to the beginning of the next line. If the per cent parameter is true, a % is appended to the line that is being output; in this case the breakpoint b should be strictly less than line length. If the per cent parameter is false, trailing blanks are suppressed. The characters emptied from the buffer form a new line of output. procedure flush buffer (b : eight bits ; per cent : boolean); { outputs out buf [1 . . b], where b ≤ out ptr }

label done;

var j, k: 0 . . line length;

begin j ← b;

if ¬per cent then { remove trailing blanks }

loop begin if j = 0 then goto done; if out buf [j] ≠ ␣ then goto done; decr (j); end;

done: for k ← 1 to j do write(tex file, xchr [out buf [k]]); if per cent then write(tex file, xchr [ % ]); write ln(tex file); incr (out line); if b < out ptr then

for k ← b + 1 to out ptr do out buf [k − b] ← out buf [k]; out ptr ← out ptr − b; end;

123. When we are copying TEX source material, we retain line breaks that occur in the input, except that an empty line is not output when the TEX source line was nonempty. For example, a line of the TEX file that contains only an index cross-reference entry will not be copied. The finish line routine is called just before get line inputs a new line, and just after a line break token has been emitted during the output of translated Pascal text. procedure finish line; { do this at the end of a line }

label exit ;

var k: 0 . . buf size; { index into buffer }

begin if out ptr > 0 then flush buffer (out ptr , false)

else begin for k ← 0 to limit do

if (buffer [k] ≠ ␣ ) ∧ (buffer [k] ≠ tab mark ) then return; flush buffer (0, false); end;

exit : end;

124* . In particular, the finish line procedure is called near the very beginning of phase two. We initialize the output variables in a slightly tricky way so that the first line of the output file will be ‘ ’. ⟨ Set initial values 10 ⟩ +≡

out ptr ← 1; out line ← 1; out buf [1] ← ; write(tex file, ´ ␣ ´ );

§125  MWEAVE  LOW-LEVEL OUTPUT ROUTINES 43

125. When we wish to append the character c to the output buffer, we write ‘out (c)’; this will cause the buffer to be emptied if it was already full. Similarly, ‘out2 (c_{1})(c_{2})’ appends a pair of characters. A line break will occur at a space or after a single-nonletter TEX control sequence.

define oot (#) ≡ if out ptr = line length then break out ; incr (out ptr ); out buf [out ptr ] ← #;

define oot1 (#) ≡ oot (#) end define oot2 (#) ≡ oot (#) oot1 define oot3 (#) ≡ oot (#) oot2 define oot4 (#) ≡ oot (#) oot3 define oot5 (#) ≡ oot (#) oot4 define out ≡ begin oot1 define out2 ≡ begin oot2 define out3 ≡ begin oot3 define out4 ≡ begin oot4 define out5 ≡ begin oot5

126. The break out routine is called just before the output buffer is about to overflow. To make this routine a little faster, we initialize position 0 of the output buffer to ‘ ’; this character isn’t really output. ⟨ Set initial values 10 ⟩ +≡

out buf [0] ← ;

127. A long line is broken at a blank space or just before a backslash that isn’t preceded by another backslash. In the latter case, a % is output at the break. procedure break out ; { finds a way to break the output line }

label exit ; var k: 0 . . line length; { index into out buf }

c, d: ASCII code; { characters from the buffer } begin k ← out ptr ; loop begin if k = 0 then ⟨ Print warning message, break the line, return 128 ⟩;

d ← out buf [k];

if d = ␣ then begin flush buffer (k, false); return; end;

if (d = ) ∧ (out buf [k − 1] ≠ ) then { in this case k > 1 } begin flush buffer (k − 1, true); return; end;

decr (k);

end; exit : end;

128. We get to this module only in unusual cases that the entire output line consists of a string of backslashes followed by a string of nonblank non-backslashes. In such cases it is almost always safe to break the line by putting a % just before the last character. ⟨ Print warning message, break the line, return 128 ⟩ ≡

begin print nl (´ ␣ ␣ ␣ ␣ ␣ ␣ ␣ ´ , out line : 1); print ln(´ ´ ); for k ← 1 to out ptr − 1 do print (xchr [out buf [k]]); new line; mark harmless ; flush buffer (out ptr − 1, true); return; end

This code is used in section 127.

129. Here is a procedure that outputs a module number in decimal notation. ⟨ Globals in the outer block 9 ⟩ +≡ dig : array [0 . . 4] of 0 . . 9; { digits to output }

130. The number to be converted by out mod is known to be less than def flag , so it cannot have more than five decimal digits. If the module is changed, we output ‘ ’ just after the number. procedure out mod (m : integer ); { output a module number }

var k: 0 . . 5; { index into dig }

a: integer ; { accumulator } begin k ← 0; a ← m; repeat dig [k] ← a mod 10; a ← a div 10; incr (k); until a = 0; repeat decr (k); out (dig [k] + ); until k = 0; if changed module[m] then out2 ( )( ); end;

131. The out name subroutine is used to output an identifier or index entry, enclosing it in braces. procedure out name(p : name pointer ); { outputs a name }

var k: 0 . . max bytes ; { index into byte mem }

w: 0 . . ww − 1; { row of byte mem } begin out ( ); w ← p mod ww ; for k ← byte start [p] to byte start [p + ww ] − 1 do

begin if byte mem[w, k] = then out ( ); out (byte mem[w, k]); end;

§132  MWEAVE  ROUTINES THAT COPY TEX MATERIAL 45

out ( ); end; 132. Routines that copy TEX material. During phase two, we use the subroutines copy limbo, copy TeX , and copy comment in place of the analogous skip limbo, skip TeX , and skip comment that were used in phase one.

The copy limbo routine, for example, takes TEX material that is not part of any module and transcribes it almost verbatim to the output file. No ‘ ’ signs should occur in such material except in ‘ ’ pairs; such pairs are replaced by singletons. procedure copy limbo; { copy TEX code until the next module begins }

label exit ;

var c: ASCII code; { character following sign }

begin loop

if loc > limit then begin finish line; get line; if input has ended then return; end

else begin buffer [limit + 1] ← ; ⟨ Copy up to control code, return if finished 133 ⟩;

end; exit : end;

133. ⟨ Copy up to control code, return if finished 133 ⟩ ≡

while buffer [loc] ≠ do begin out (buffer [loc]); incr (loc); end;

if loc ≤ limit then begin loc ← loc + 2; c ← buffer [loc − 1]; if (c = ␣ ) ∨ (c = tab mark ) ∨ (c = ) then return; if (c ≠ ) ∧ (c ≠ ) then

begin out ( );

if c ≠ then err print (´ ␣ ␣ ␣ ␣ ␣ ␣ ´ ); end; end

This code is used in section 132.

134. The copy TeX routine processes the TEX code at the beginning of a module; for example, the words you are now reading were copied in this way. It returns the next control code or ‘ ’ found in the input. function copy TeX : eight bits ; { copy pure TEX material }

label done;

var c: eight bits ; { control code found }

begin loop

begin if loc > limit then begin finish line; get line;

if input has ended then

begin c ← new module; goto done; end; end;

buffer [limit + 1] ← ; ⟨ Copy up to ‘ ’ or control code, goto done if finished 135 ⟩; end;

done: copy TeX ← c; end;

46  ROUTINES THAT COPY TEX MATERIAL  MWEAVE §135

135. We don’t copy spaces or tab marks into the beginning of a line. This makes the test for empty lines in finish line work. ⟨ Copy up to ‘ ’ or control code, goto done if finished 135 ⟩ ≡

repeat c ← buffer [loc]; incr (loc); if c = then goto done; if c ≠ then

begin out (c);

if (out ptr = 1) ∧ ((c = ␣ ) ∨ (c = tab mark )) then decr (out ptr ); end;

until c = ; if loc ≤ limit then

begin c ← control code(buffer [loc]); incr (loc); goto done; end

This code is used in section 134.

136* . The copy comment uses and returns a brace-balance value, following the conventions of skip comment above. Instead of copying the TEX material into the output buffer, this procedure copies it into the token memory. The abbreviation app tok (t) is used to append token t to the current token list, and it also makes sure that it is possible to append at least one further token without overflow.

define app tok (#) ≡ begin if tok ptr + 2 > max toks then overflow (´ ´ ); tok mem[tok ptr ] ← #; incr (tok ptr ); end

function copy comment (bal : eight bits ): eight bits ; { copies TEX code in comments } label done; var c: ASCII code; { current character being copied }

begin loop

begin if loc > limit then begin get line;

if input has ended then

begin err print (´ ␣ ␣ ␣ ␣ ´ ); loc ← 1; ⟨ Clear bal and goto done 138* ⟩; end;

end; c ← buffer [loc]; incr (loc); if c = then goto done; if (c = ) ∧ (buffer [loc] = ) then

begin incr (loc); c ← end regular comment ; app tok ( ); end

else begin app tok (c);

end; ⟨ Copy special things when c = , , , ; goto done at end 137* ⟩; end;

done: copy comment ← bal ; end; 137* . ⟨ Copy special things when c = , , , ; goto done at end 137* ⟩ ≡

if c = then begin incr (loc); if buffer [loc − 1] ≠ then

begin err print (´ ␣ ␣ ␣ ␣ ␣ ␣ ´ ); loc ← loc − 2; decr (tok ptr ); ⟨ Clear bal and goto done 138* ⟩; end;

end else if (c = ) ∧ (buffer [loc] ≠ ) then

begin app tok (buffer [loc]); incr (loc);

end else if c = begin regular comment then incr (bal )

else if c = end regular comment then begin decr (bal ); if bal = 0 then goto done; end

else if (c = ) ∧ (buffer [loc] = ) then begin incr (bal ); incr (loc); end

This code is used in section 136*.

138* . When the comment has terminated abruptly due to an error, we output enough right braces to keep TEX happy. ⟨ Clear bal and goto done 138* ⟩ ≡

app tok ( ␣ ); { this is done in case the previous character was ‘ ’ } repeat app tok (end regular comment ); decr (bal ); until bal = 0; goto done;

This code is used in sections 136* and 137*.

48  PARSING  MWEAVE §139

139. Parsing. The most intricate part of is its mechanism for converting Pascal-like code into TEX code, and we might as well plunge into this aspect of the program now. A “bottom up” approach is used to parse the Pascal-like material, since must deal with fragmentary constructions whose overall “part of speech” is not known.

At the lowest level, the input is represented as a sequence of entities that we shall call scraps, where each scrap of information consists of two parts, its category and its translation. The category is essentially a syntactic class, and the translation is a token list that represents TEX code. Rules of syntax and semantics tell us how to combine adjacent scraps into larger ones, and if we are lucky an entire Pascal text that starts out as hundreds of small scraps will join together into one gigantic scrap whose translation is the desired TEX code. If we are unlucky, we will be left with several scraps that don’t combine; their translations will simply be output, one by one.

The combination rules are given as context-sensitive productions that are applied from left to right. Suppose that we are currently working on the sequence of scraps s_{1} s_{2} … s_{n}. We try first to find the longest production that applies to an initial substring s_{1} s_{2} …; but if no such productions exist, we find to find the longest production applicable to the next substring s_{2} s_{3} …; and if that fails, we try to match s_{3} s_{4} …, etc.

A production applies if the category codes have a given pattern. For example, one of the productions is

open math semi → open math

and it means that three consecutive scraps whose respective categories are open, math, and semi are converted to two scraps whose categories are open and math. This production also has an associated rule that tells how to combine the translation parts:

\begin{aligned}O_{2} & = O_{1} \\ M_{2} & = M_{1} S opt 5 \end{aligned}

This means that the open scrap has not changed, while the new math scrap has a translation M_{2} composed of the translation M_{1} of the original math scrap followed by the translation S of the semi scrap followed by ‘ ’ followed by ‘opt ’ followed by ‘ ’. (In the TEX file, this will specify an additional thin space after the semicolon, followed by an optional line break with penalty 50.) Translation rules use subscripts to distinguish between translations of scraps whose categories have the same initial letter; these subscripts are assigned from left to right.

also has the production rule semi → terminator

(meaning that a semicolon can terminate a Pascal statement). Since productions are applied from left to right, this rule will be activated only if the semi is not preceded by scraps that match other productions; in particular, a semi that is preceded by ‘open math’ will have disappeared because of the production above, and such semicolons do not act as statement terminators. This incidentally is how is able to treat semicolons in two distinctly different ways, the first of which is intended for semicolons in the parameter list of a procedure declaration.

The translation rule corresponding to semi → terminator is

T = S

but we shall not mention translation rules in the common case that the translation of the new scrap on the right-hand side is simply the concatenation of the disappearing scraps on the left-hand side.

§140  MWEAVE  PARSING 49

140. Here is a list of the category codes that scraps can have.

define simp = 1 { the translation can be used both in horizontal mode and in math mode of TEX } define math = 2 { the translation should be used only in TEX math mode } define intro = 3 { a statement is expected to follow this, after a space and an optional break } define open = 4 { denotes an incomplete parenthesized quantity to be used in math mode } define beginning = 5 { denotes an incomplete compound statement to be used in horizontal mode } define close = 6 { ends a parenthesis or compound statement } define alpha = 7 { denotes the beginning of a clause } define omega = 8 { denotes the ending of a clause and possible comment following } define semi = 9 { denotes a semicolon and possible comment following it } define terminator = 10 { something that ends a statement or declaration } define stmt = 11 { denotes a statement or declaration including its terminator } define cond = 12 { precedes an if clause that might have a matching else } define clause = 13 { precedes a statement after which indentation ends } define colon = 14 { denotes a colon } define exp = 15 { stands for the E in a floating point constant } define proc = 16 { denotes a procedure or program or function heading } define case head = 17 { denotes a case statement or record heading } define record head = 18 { denotes a record heading without indentation } define var head = 19 { denotes a variable declaration heading } define elsie = 20 { else } define casey = 21 { case } define mod scrap = 22 { denotes a module name } debug procedure print cat (c : eight bits ); { symbolic printout of a category }

begin case c of

simp: print (´ ´ ); math: print (´ ´ ); intro: print (´ ´ ); open: print (´ ´ ); beginning : print (´ ´ ); close: print (´ ´ ); alpha: print (´ ´ ); omega: print (´ ´ ); semi : print (´ ´ ); terminator : print (´ ´ ); stmt : print (´ ´ ); cond : print (´ ´ ); clause: print (´ ´ ); colon: print (´ ´ ); exp: print (´ ´ ); proc: print (´ ´ ); case head : print (´ ´ ); record head : print (´ ´ ); var head : print (´ ´ ); elsie: print (´ ´ ); casey : print (´ ´ ); mod scrap: print (´ ´ ); othercases print (´ ´ ) endcases; end; gubed 141. The token lists for translated TEX output contain some special control symbols as well as ordinary characters. These control symbols are interpreted by before they are written to the output file.

break space denotes an optional line break or an en space; force denotes a line break; big force denotes a line break with additional vertical space; opt denotes an optional line break (with the continuation line indented two ems with respect to the normal

starting position)—this code is followed by an integer n, and the break will occur with penalty 10n; backup denotes a backspace of one em; cancel obliterates any break space or force or big force tokens that immediately precede or follow it and

also cancels any backup tokens that follow it; indent causes future lines to be indented one more em; outdent causes future lines to be indented one less em.

All of these tokens are removed from the TEX output that comes from Pascal text between … signs; break space and force and big force become single spaces in this mode. The translation of other Pascal texts results in TEX control sequences , , , , , , corresponding respectively to indent , outdent , opt , backup, break space, force, and big force. However, a sequence of consecutive ‘␣’, break space, force, and/or big force tokens is first replaced by a single token (the maximum of the given ones).

The tokens math rel , math bin, math op will be translated into , , and , respectively. Other control sequences in the TEX output will be ‘ … ’ surrounding identifiers, ‘ … ’ surrounding reserved words, ‘ … ’ surrounding strings, ‘ … force’ surrounding comments, and ‘ n … ’ surrounding module names, where n is the module number.

define math bin = ´ 203 define math rel = ´ 204 define math op = ´ 205 define big cancel = ´ 206 { like cancel , also overrides spaces } define cancel = ´ 207 { overrides backup, break space, force, big force } define indent = cancel + 1 { one more tab ( ) } define outdent = cancel + 2 { one less tab ( ) } define opt = cancel + 3 { optional break in mid-statement ( ) } define backup = cancel + 4 { stick out one unit to the left ( ) } define break space = cancel + 5 { optional break between statements ( ) } define force = cancel + 6 { forced break between statements ( ) } define big force = cancel + 7 { forced break with additional space ( ) } define end translation = big force + 1 { special sentinel token at end of list } 142. The raw input is converted into scraps according to the following table, which gives category codes followed by the translations. Sometimes a single item of input produces more than one scrap. (The symbol ‘ ’ stands for ‘ identifier ’, i.e., the identifier itself treated as a reserved word. In a few cases the category is given as ‘comment ’; this is not an actual category code, it means that the translation will be treated as a comment, as explained below.)

math:
math:
math:
math:
math:
math:
math:
open:
close:
string  simp:  modified string
´ string ´  simp:  ´ modified string ´
string  simp:  modified string
#  math: #
$  math: $
math:
%  math: %
math:
open:
close:
open:
close:
math:
math: opt
math:
simp:
colon:
semi :
identifier simp: identifier
in constant  exp:
digit d  simp: d
other character c math: c
math:
alpha:
beginning : force cancel  intro:
casey :  alpha: force
intro: force backup
math: math bin
omega:
math: math rel
terminator :  elsie: force backup
terminator :  close: force
alpha:
alpha: force
proc: force backup cancel  intro: indent ␣
intro:
cond :  alpha: force
math:

intro: force backup
math: math bin
simp:
math:
omega:
math:
intro:
proc: force backup cancel  intro: indent ␣
proc: force backup cancel  intro: indent ␣
record head :  intro:
beginning : force indent cancel  intro:
alpha:
omega:
math: math rel
intro: force backup
terminator : close: force backup clause:
var head : force backup cancel  intro:
alpha: force
alpha: force
alpha: force  omega:
´ const  simp:  const
const  simp:  const
$  simp:
simp:
math:
stuff simp: stuff
module  mod scrap: n module
#  comment : big force
comment : force
simp: opt
comment : big cancel ␣big cancel
semi :
math:
math:
math:

When a string is output, certain characters are preceded by ‘ ’ signs so that they will print properly.

A comment in the input will be combined with the preceding omega or semi scrap, or with the following terminator scrap, if possible; otherwise it will be inserted as a separate terminator scrap. An additional “comment” is effectively appended at the end of the Pascal text, just before translation begins; this consists of a cancel token in the case of Pascal text in …, otherwise it consists of a force token.

From this table it is evident that will parse a lot of non-Pascal programs. For example, the reserved words ‘ ’ and ‘ ’ are treated in an identical way by from a syntactic standpoint, and semantically they are equivalent except that a forced line break occurs just before ‘for’; Pascal programmers may well be surprised at this similarity. The idea is to keep ’s rules as simple as possible, consistent with doing a reasonable job on syntactically correct Pascal programs. The production rules below have been formulated in the same spirit of “almost anything goes.”

143. Here is a table of all the productions. The reader can best get a feel for how they work by trying them out by hand on small examples; no amount of explanation will be as effective as watching the rules in action. Parsing can also be watched by debugging with ‘ ’.

Production categories [[translations]]  Remarks
1 alpha math colon → alpha math  e.g., case v : boolean of
2 alpha math omega → clause [[C = A ␣$ M $␣ indent O]]  e.g., while x > 0 do
3 alpha omega → clause [[C = A ␣ indent O]]  e.g., file of
4 alpha simp → alpha math  convert to math mode
5 beginning close (terminator or stmt ) → stmt  compound statement ends
6 beginning stmt → beginning [[B_{2} = B_{1} break space S]]  compound statement grows
7 case head casey clause → case head [[C_{4} = C_{1} outdent C_{2} C_{3}]]  variant records
8 case head close terminator → stmt [[S = C_{1} cancel outdent C_{2} T]]  end of case statement
9 case head stmt → case head [[C_{2} = C_{1} force S]]  case statement grows
10 casey clause → case head  beginning of case statement
11 clause stmt → stmt [[S_{2} = C break space S_{1} cancel outdent force]]  end of controlled statement
12 cond clause stmt elsie → clause [[C_{3} = C_{1} C_{2} break space S E ␣ cancel ]] complete conditional
13 cond clause stmt → stmt
[[S_{2} = C_{1} C_{2} break space S_{1} cancel outdent force]]  incomplete conditional
14 elsie → intro  unmatched else
15 exp math simp* → math [[M_{2} = E M_{1} S ]]  signed exponent
16 exp simp* → math [[M = E S ]]  unsigned exponent
17 intro stmt → stmt [[S_{2} = I ␣ opt cancel S_{1}]]  labeled statement, etc.
18 math close → stmt close [[S = $ M $]]  end of field list
19 math colon → intro [[I = force backup $ M $ C]]  compound label
20 math math → math  simple concatenation
21 math simp → math  simple concatenation
22 math stmt → stmt
[[S_{2} = $ M $ indent break space S_{1} cancel outdent force]]  macro or type definition
23 math terminator → stmt [[S = $ M $ T]]  statement involving math
24 mod scrap (terminator or semi ) → stmt [[S = M T force]]  module like a statement
25 mod scrap → simp  module unlike a statement
26 open case head close → math [[M = O $ cancel C_{1} cancel outdent $ C_{2}]] case in field list
27 open close → math [[M = O C]]  empty set [ ]
28 open math case head close → math
[[M_{2} = O M_{1} $ cancel C_{1} cancel outdent $ C_{2}]]  case in field list
29 open math close → math  parenthesized group
30 open math colon → open math  colon in parentheses
31 open math proc intro → open math [[M_{2} = M_{1} math op cancel P ]]  procedure in parentheses
32 open math semi → open math [[M_{2} = M_{1} S opt ]]  semicolon in parentheses
33 open math var head intro → open math [[M_{2} = M_{1} math op cancel V ]] var in parentheses
34 open proc intro → open math [[M = math op cancel P ]]  procedure in parentheses
35 open simp → open math  convert to math mode
36 open stmt close → math [[M = O $ cancel S cancel $ C]]  field list
37 open var head intro → open math [[M = math op cancel V ]]  var in parentheses
38 proc beginning close terminator → stmt [[S = P cancel outdent B C T]] end of procedure declaration
39 proc stmt → proc [[P_{2} = P_{1} break space S]]  procedure declaration grows
40 record head intro casey → casey [[C_{2} = R I ␣ cancel C_{1}]]  record case …
41 record head → case head [[C = indent R cancel ]]  other record structures
42 semi → terminator  semicolon after statement
43 simp close → stmt close  end of field list
44 simp colon → intro [[I = force backup S C]]  simple label
45 simp math → math  simple concatenation

46 simp mod scrap → mod scrap  in emergencies
47 simp simp → simp  simple concatenation
48 simp terminator → stmt  simple statement
49 stmt stmt → stmt [[S_{3} = S_{1} break space S_{2}]]  adjacent statements
50 terminator → stmt  empty statement
51 var head beginning → stmt beginning  end of variable declarations
52 var head math colon → var head intro [[I = $ M $ C]]  variable declaration
53 var head simp colon → var head intro  variable declaration
54 var head stmt → var head [[V_{2} = V_{1} break space S]]  variable declarations grow

Translations are not specified here when they are simple concatenations of the scraps that change. For example, the full translation of ‘open math colon → open math’ is O_{2} = O_{1}, M_{2} = M_{1}C.

The notation ‘simp*’, in the exp-related productions above, stands for a simp scrap that isn’t followed by another simp.

§144  MWEAVE  IMPLEMENTING THE PRODUCTIONS 55

144. Implementing the productions. When Pascal text is to be processed with the grammar above, we put its initial scraps s_{1} … s_{n} into two arrays cat [1 . . n] and trans [1 . . n]. The value of cat [k] is simply a category code from the list above; the value of trans [k] is a text pointer, i.e., an index into tok start . Our production rules have the nice property that the right-hand side is never longer than the left-hand side. Therefore it is convenient to use sequential allocation for the current sequence of scraps. Five pointers are used to manage the parsing:

pp (the parsing pointer) is such that we are trying to match the category codes cat [pp] cat [pp + 1] … to

the left-hand sides of productions. scrap base, lo ptr , hi ptr , and scrap ptr are such that the current sequence of scraps appears in positions

scrap base through lo ptr and hi ptr through scrap ptr , inclusive, in the cat and trans arrays. Scraps located between scrap base and lo ptr have been examined, while those in positions ≥ hi ptr have not yet been looked at by the parsing process.

Initially scrap ptr is set to the position of the final scrap to be parsed, and it doesn’t change its value. The parsing process makes sure that lo ptr ≥ pp + 3, since productions have as many as four terms, by moving scraps from hi ptr to lo ptr . If there are fewer than pp + 3 scraps left, the positions up to pp + 3 are filled with blanks that will not match in any productions. Parsing stops when pp = lo ptr + 1 and hi ptr = scrap ptr + 1.

The trans array elements are declared to be of type 0 . . 10239 instead of type text pointer , because the final sorting phase of uses this array to contain elements of type name pointer . Both of these types are subranges of 0 . . 10239. ⟨ Globals in the outer block 9 ⟩ +≡ cat : array [0 . . max scraps ] of eight bits ; { category codes of scraps } trans : array [0 . . max scraps ] of 0 . . 10239; { translation texts of scraps } pp: 0 . . max scraps ; { current position for reducing productions } scrap base: 0 . . max scraps ; { beginning of the current scrap sequence } scrap ptr : 0 . . max scraps ; { ending of the current scrap sequence } lo ptr : 0 . . max scraps ; { last scrap that has been examined } hi ptr : 0 . . max scraps ; { first scrap that has not been examined }

stat max scr ptr : 0 . . max scraps ; { largest value assumed by scrap ptr }

tats

56  IMPLEMENTING THE PRODUCTIONS  MWEAVE §146

145. ⟨ Set initial values 10 ⟩ +≡ scrap base ← 1; scrap ptr ← 0; stat max scr ptr ← 0; tats 146. Token lists in tok mem are composed of the following kinds of items for TEX output.

• ASCII codes and special codes like force and math rel represent themselves;

• id flag + p represents identifier p ;

• res flag + p represents identifier p ;

• mod flag + p represents module name p;

• tok flag + p represents token list number p;

• inner tok flag + p represents token list number p, to be translated without line-break controls. define id flag = 10240 { signifies an identifier } define res flag = id flag + id flag { signifies a reserved word } define mod flag = res flag + id flag { signifies a module name } define tok flag ≡ mod flag + id flag { signifies a token list } define inner tok flag ≡ tok flag + id flag { signifies a token list in ‘ … ’ } define lbrace ≡ xchr [ ] { this avoids possible Pascal compiler confusion } define rbrace ≡ xchr [ ] { because these braces might occur within comments } debug procedure print text (p : text pointer ); { prints a token list } var j: 0 . . max toks ; { index into tok mem }

r: 0 . . id flag − 1; { remainder of token after the flag has been stripped off } begin if p ≥ text ptr then print (´ ´ ) else for j ← tok start [p] to tok start [p + 1] − 1 do

begin r ← tok mem[j] mod id flag ; case tok mem[j] div id flag of 1: begin print (´ ´ , lbrace); print id (r); print (rbrace);

end; { id flag } 2: begin print (´ ´ , lbrace); print id (r); print (rbrace);

end; { res flag } 3: begin print (´ ´ ); print id (r); print (´ ´ );

end; { mod flag } 4: print (´ ´ , r : 1, ´ ´ ); { tok flag } 5: print (´ ´ , r : 1, ´ ´ ); { inner tok flag } othercases ⟨ Print token r in symbolic form 147 ⟩ endcases; end;

end; gubed 147. ⟨ Print token r in symbolic form 147 ⟩ ≡

case r of

math bin: print (´ ´ , lbrace); math rel : print (´ ´ , lbrace); math op: print (´ ´ , lbrace); big cancel : print (´ ´ ); cancel : print (´ ´ ); indent : print (´ ´ ); outdent : print (´ ´ ); backup: print (´ ´ ); opt : print (´ ´ ); break space: print (´ ´ ); force: print (´ ´ ); big force: print (´ ´ ); end translation: print (´ ´ ); othercases print (xchr [r]) endcases

This code is used in section 146.

148. The production rules listed above are embedded directly into the program, since it is easier to do this than to write an interpretive system that would handle production systems in general. Several macros are defined here so that the program for each production is fairly short.

All of our productions conform to the general notion that some k consecutive scraps starting at some position j are to be replaced by a single scrap of some category c whose translations is composed from the translations of the disappearing scraps. After this production has been applied, the production pointer pp should change by an amount d. Such a production can be represented by the quadruple (j, k, c, d). For example, the production ‘simp math → math’ would be represented by ‘(pp, 2, math, −1)’; in this case the pointer pp should decrease by 1 after the production has been applied, because some productions with math in their second positions might now match, but no productions have math in the third or fourth position of their left-hand sides. Note that the value of d is determined by the whole collection of productions, not by an individual one. Consider the further example ‘var head math colon → var head intro’, which is represented by ‘(pp + 1, 2, intro, +1)’; the +1 here is deduced by looking at the grammar and seeing that no matches could possibly occur at positions ≤ pp after this production has been applied. The determination of d has been done by hand in each case, based on the full set of productions but not on the grammar of Pascal or on the rules for constructing the initial scraps.

We also attach a serial number of each production, so that additional information is available when debugging. For example, the program below contains the statement ‘reduce(pp + 1, 2, intro, +1)(52)’ when it implements the production just mentioned.

Before calling reduce, the program should have appended the tokens of the new translation to the tok mem array. We commonly want to append copies of several existing translations, and macros are defined to simplify these common cases. For example, app2 (pp) will append the translations of two consecutive scraps, trans [pp] and trans [pp + 1], to the current token list. If the entire new translation is formed in this way, we write ‘squash(j, k, c, d)’ instead of ‘reduce(j, k, c, d)’. For example, ‘squash(pp, 2, math, −1)’ is an abbreviation for ‘app2 (pp); reduce(pp, 2, math, −1)’.

The code below is an exact translation of the production rules into Pascal, using such macros, and the reader should have no difficulty understanding the format by comparing the code with the symbolic productions as they were listed earlier.

Caution: The macros app, app1 , app2 , and app3 are sequences of statements that are not enclosed with begin and end, because such delimiters would make the Pascal program much longer. This means that it is necessary to write begin and end explicitly when such a macro is used as a single statement. Several mysterious bugs in the original programming of were caused by a failure to remember this fact. Next time the author will know better.

define production(#) ≡ debug prod (#) gubed;

goto found

define reduce(#) ≡ red (#); production

define production end (#) ≡ debug prod (#) gubed; goto found ; end

define squash(#) ≡

begin sq (#); production end define app(#) ≡ tok mem[tok ptr ] ← #; incr (tok ptr )

{ this is like app tok , but it doesn’t test for overflow }

define app1 (#) ≡ tok mem[tok ptr ] ← tok flag + trans [#]; incr (tok ptr )

define app2 (#) ≡ app1 (#); app1 (# + 1)

define app3 (#) ≡ app2 (#); app1 (# + 2)

149. Let us consider the big case statement for productions now, before looking at its context. We want to design the program so that this case statement works, so we might as well not keep ourselves in suspense about exactly what code needs to be provided with a proper environment.

The code here is more complicated than it need be, since some popular Pascal compilers are unable to deal with procedures that contain a lot of program text. The translate procedure, which incorporates the case statement here, would become too long for those compilers if we did not do something to split the cases into parts. Therefore a separate procedure called five cases has been introduced. This auxiliary procedure contains approximately half of the program text that translate would otherwise have had. There’s also a procedure called alpha cases , which turned out to be necessary because the best two-way split wasn’t good enough. The procedure could be split further in an analogous manner, but the present scheme works on all compilers known to the author. ⟨ Match a production at pp, or increase pp if there is no match 149 ⟩ ≡

if cat [pp] ≤ alpha then

if cat [pp] < alpha then five cases else alpha cases else begin case cat [pp] of

case head : ⟨ Cases for case head 153 ⟩; casey : ⟨ Cases for casey 154 ⟩; clause: ⟨ Cases for clause 155 ⟩; cond : ⟨ Cases for cond 156 ⟩; elsie: ⟨ Cases for elsie 157 ⟩; exp: ⟨ Cases for exp 158 ⟩; mod scrap: ⟨ Cases for mod scrap 161 ⟩; proc: ⟨ Cases for proc 164 ⟩; record head : ⟨ Cases for record head 165 ⟩; semi : ⟨ Cases for semi 166 ⟩; stmt : ⟨ Cases for stmt 168 ⟩; terminator : ⟨ Cases for terminator 169 ⟩; var head : ⟨ Cases for var head 170 ⟩;

othercases do nothing

endcases; incr (pp); { if no match was found, we move to the right }

found : end

This code is used in section 175.

150. Here are the procedures that need to be present for the reason just explained. ⟨ Declaration of subprocedures for translate 150 ⟩ ≡ procedure five cases ; { handles almost half of the syntax }

label found ; begin case cat [pp] of beginning : ⟨ Cases for beginning 152 ⟩; intro: ⟨ Cases for intro 159 ⟩; math: ⟨ Cases for math 160 ⟩; open: ⟨ Cases for open 162 ⟩; simp: ⟨ Cases for simp 167 ⟩;

othercases do nothing

endcases; incr (pp); { if no match was found, we move to the right }

found : end; procedure alpha cases ;

label found ; begin ⟨ Cases for alpha 151 ⟩; incr (pp); { if no match was found, we move to the right }

found : end;

This code is used in section 179.

151. Now comes the code that tries to match each production that starts with a particular type of scrap. Whenever a match is discovered, the squash or reduce macro will cause the appropriate action to be performed, followed by goto found . ⟨ Cases for alpha 151 ⟩ ≡

if cat [pp + 1] = math then begin if cat [pp + 2] = colon then squash(pp + 1, 2, math, 0)(1) else if cat [pp + 2] = omega then

begin app1 (pp); app( ␣ ); app( $ ); app1 (pp + 1); app( $ ); app( ␣ ); app(indent ); app1 (pp + 2); reduce(pp, 3, clause, −2)(2); end;

end else if cat [pp + 1] = omega then

begin app1 (pp); app( ␣ ); app(indent ); app1 (pp + 1); reduce(pp, 2, clause, −2)(3); end

else if cat [pp + 1] = simp then squash(pp + 1, 1, math, 0)(4)

This code is used in section 150.

152. ⟨ Cases for beginning 152 ⟩ ≡

if cat [pp + 1] = close then begin if (cat [pp + 2] = terminator ) ∨ (cat [pp + 2] = stmt ) then squash(pp, 3, stmt , −2)(5); end

else if cat [pp + 1] = stmt then begin app1 (pp); app(break space); app1 (pp + 1); reduce(pp, 2, beginning , −1)(6); end

This code is used in section 150.

153. ⟨ Cases for case head 153 ⟩ ≡

if cat [pp + 1] = casey then

begin if cat [pp + 2] = clause then begin app1 (pp); app(outdent ); app2 (pp + 1); reduce(pp, 3, case head , 0)(7); end;

end else if cat [pp + 1] = close then

begin if cat [pp + 2] = terminator then begin app1 (pp); app(cancel ); app(outdent ); app2 (pp + 1); reduce(pp, 3, stmt , −2)(8); end;

end else if cat [pp + 1] = stmt then

begin app1 (pp); app(force); app1 (pp + 1); reduce(pp, 2, case head , 0)(9); end

This code is used in section 149.

154. ⟨ Cases for casey 154 ⟩ ≡ if cat [pp + 1] = clause then squash(pp, 2, case head , 0)(10)

This code is used in section 149.

155. ⟨ Cases for clause 155 ⟩ ≡

if cat [pp + 1] = stmt then begin app1 (pp); app(break space); app1 (pp + 1); app(cancel ); app(outdent ); app(force); reduce(pp, 2, stmt , −2)(11); end

This code is used in section 149.

156. ⟨ Cases for cond 156 ⟩ ≡

if (cat [pp + 1] = clause) ∧ (cat [pp + 2] = stmt ) then

if cat [pp + 3] = elsie then begin app2 (pp); app(break space); app2 (pp + 2); app( ␣ ); app(cancel ); reduce(pp, 4, clause, −2)(12); end

else begin app2 (pp); app(break space); app1 (pp + 2); app(cancel ); app(outdent ); app(force); reduce(pp, 3, stmt , −2)(13); end

This code is used in section 149.

157. ⟨ Cases for elsie 157 ⟩ ≡ squash(pp, 1, intro, −3)(14)

This code is used in section 149.

158. ⟨ Cases for exp 158 ⟩ ≡

if cat [pp + 1] = math then

begin if cat [pp + 2] = simp then

if cat [pp + 3] ≠ simp then begin app3 (pp); app( ); reduce(pp, 3, math, −1)(15); end;

end else if cat [pp + 1] = simp then

if cat [pp + 2] ≠ simp then begin app2 (pp); app( ); reduce(pp, 2, math, −1)(16); end

This code is used in section 149.

159. ⟨ Cases for intro 159 ⟩ ≡

if cat [pp + 1] = stmt then begin app1 (pp); app( ␣ ); app(opt ); app( ); app(cancel ); app1 (pp + 1); reduce(pp, 2, stmt , −2)(17); end

This code is used in section 150.

160. ⟨ Cases for math 160 ⟩ ≡

if cat [pp + 1] = close then begin app( $ ); app1 (pp); app( $ ); reduce(pp, 1, stmt , −2)(18); end

else if cat [pp + 1] = colon then begin app(force); app(backup); app( $ ); app1 (pp); app( $ ); app1 (pp + 1); reduce(pp, 2, intro, −3)(19); end

else if cat [pp + 1] = math then squash(pp, 2, math, −1)(20)

else if cat [pp + 1] = simp then squash(pp, 2, math, −1)(21)

else if cat [pp + 1] = stmt then begin app( $ ); app1 (pp); app( $ ); app(indent ); app(break space); app1 (pp + 1); app(cancel ); app(outdent ); app(force); reduce(pp, 2, stmt , −2)(22); end

else if cat [pp + 1] = terminator then begin app( $ ); app1 (pp); app( $ ); app1 (pp + 1); reduce(pp, 2, stmt , −2)(23); end

This code is used in section 150.

161. ⟨ Cases for mod scrap 161 ⟩ ≡

if (cat [pp + 1] = terminator ) ∨ (cat [pp + 1] = semi ) then begin app2 (pp); app(force); reduce(pp, 2, stmt , −2)(24); end

else squash(pp, 1, simp, −2)(25)

This code is used in section 149.

162. ⟨ Cases for open 162 ⟩ ≡

if (cat [pp + 1] = case head ) ∧ (cat [pp + 2] = close) then begin app1 (pp); app( $ ); app(cancel ); app1 (pp + 1); app(cancel ); app(outdent ); app( $ ); app1 (pp + 2); reduce(pp, 3, math, −1)(26); end

else if cat [pp + 1] = close then begin app1 (pp); app( ); app( ); app1 (pp + 1); reduce(pp, 2, math, −1)(27); end

else if cat [pp + 1] = math then ⟨ Cases for open math 163 ⟩

else if cat [pp + 1] = proc then

begin if cat [pp + 2] = intro then begin app(math op); app(cancel ); app1 (pp + 1); app( ); reduce(pp + 1, 2, math, 0)(34); end;

end else if cat [pp + 1] = simp then squash(pp + 1, 1, math, 0)(35)

else if (cat [pp + 1] = stmt ) ∧ (cat [pp + 2] = close) then begin app1 (pp); app( $ ); app(cancel ); app1 (pp + 1); app(cancel ); app( $ ); app1 (pp + 2); reduce(pp, 3, math, −1)(36); end

else if cat [pp + 1] = var head then

begin if cat [pp + 2] = intro then begin app(math op); app(cancel ); app1 (pp + 1); app( ); reduce(pp + 1, 2, math, 0)(37); end; end

This code is used in section 150.

163. ⟨ Cases for open math 163 ⟩ ≡

begin if (cat [pp + 2] = case head ) ∧ (cat [pp + 3] = close) then begin app2 (pp); app( $ ); app(cancel ); app1 (pp + 2); app(cancel ); app(outdent ); app( $ ); app1 (pp + 3); reduce(pp, 4, math, −1)(28); end

else if cat [pp + 2] = close then squash(pp, 3, math, −1)(29)

else if cat [pp + 2] = colon then squash(pp + 1, 2, math, 0)(30)

else if cat [pp + 2] = proc then

begin if cat [pp + 3] = intro then begin app1 (pp + 1); app(math op); app(cancel ); app1 (pp + 2); app( ); reduce(pp + 1, 3, math, 0)(31); end;

end else if cat [pp + 2] = semi then

begin app2 (pp + 1); app( ); app( ); app(opt ); app( ); reduce(pp + 1, 2, math, 0)(32); end

else if cat [pp + 2] = var head then

begin if cat [pp + 3] = intro then begin app1 (pp + 1); app(math op); app(cancel ); app1 (pp + 2); app( ); reduce(pp + 1, 3, math, 0)(31); end;

end;

end

This code is used in section 162.

164. ⟨ Cases for proc 164 ⟩ ≡

if cat [pp + 1] = beginning then

begin if (cat [pp + 2] = close) ∧ (cat [pp + 3] = terminator ) then begin app1 (pp); app(cancel ); app(outdent ); app3 (pp + 1); reduce(pp, 4, stmt , −2)(38); end;

end else if cat [pp + 1] = stmt then

begin app1 (pp); app(break space); app1 (pp + 1); reduce(pp, 2, proc, −2)(39); end

This code is used in section 149.

165. ⟨ Cases for record head 165 ⟩ ≡

if (cat [pp + 1] = intro) ∧ (cat [pp + 2] = casey ) then begin app2 (pp); app( ␣ ); app(cancel ); app1 (pp + 2); reduce(pp, 3, casey , −2)(40); end

else begin app(indent ); app1 (pp); app(cancel ); reduce(pp, 1, case head , 0)(41);

end

This code is used in section 149.

166. ⟨ Cases for semi 166 ⟩ ≡ squash(pp, 1, terminator , −3)(42)

This code is used in section 149.

167. ⟨ Cases for simp 167 ⟩ ≡ if cat [pp + 1] = close then squash(pp, 1, stmt , −2)(43) else if cat [pp + 1] = colon then

begin app(force); app(backup); app2 (pp); reduce(pp, 2, intro, −3)(44);

end else if cat [pp + 1] = math then squash(pp, 2, math, −1)(45)

else if cat [pp + 1] = mod scrap then squash(pp, 2, mod scrap, 0)(46) else if cat [pp + 1] = simp then squash(pp, 2, simp, −2)(47)

else if cat [pp + 1] = terminator then squash(pp, 2, stmt , −2)(48)

This code is used in section 150.

168. ⟨ Cases for stmt 168 ⟩ ≡

if cat [pp + 1] = stmt then begin app1 (pp); app(break space); app1 (pp + 1); reduce(pp, 2, stmt , −2)(49); end

This code is used in section 149.

169. ⟨ Cases for terminator 169 ⟩ ≡ squash(pp, 1, stmt , −2)(50)

This code is used in section 149.

170. ⟨ Cases for var head 170 ⟩ ≡ if cat [pp + 1] = beginning then squash(pp, 1, stmt , −2)(51) else if cat [pp + 1] = math then

begin if cat [pp + 2] = colon then begin app( $ ); app1 (pp + 1); app( $ ); app1 (pp + 2); reduce(pp + 1, 2, intro, +1)(52); end; end

else if cat [pp + 1] = simp then begin if cat [pp + 2] = colon then squash(pp + 1, 2, intro, +1)(53); end

else if cat [pp + 1] = stmt then begin app1 (pp); app(break space); app1 (pp + 1); reduce(pp, 2, var head , −2)(54); end

This code is used in section 149.

171. The ‘freeze text ’ macro is used to give official status to a token list. Before saying freeze text , items are appended to the current token list, and we know that the eventual number of this token list will be the current value of text ptr . But no list of that number really exists as yet, because no ending point for the current list has been stored in the tok start array. After saying freeze text , the old current token list becomes legitimate, and its number is the current value of text ptr − 1 since text ptr has been increased. The new current token list is empty and ready to be appended to. Note that freeze text does not check to see that text ptr hasn’t gotten too large, since it is assumed that this test was done beforehand.

define freeze text ≡ incr (text ptr ); tok start [text ptr ] ← tok ptr

172. The ‘reduce’ macro used in our code for productions actually calls on a procedure named ‘red ’, which makes the appropriate changes to the scrap list. procedure red (j : sixteen bits ; k : eight bits ; c : eight bits ; d : integer );

var i: 0 . . max scraps ; { index into scrap memory }

begin cat [j] ← c; trans [j] ← text ptr ; freeze text ;

if k > 1 then

begin for i ← j + k to lo ptr do begin cat [i − k + 1] ← cat [i]; trans [i − k + 1] ← trans [i]; end;

lo ptr ← lo ptr − k + 1;

end; ⟨ Change pp to max(scrap base,pp+d) 173 ⟩; end;

173. ⟨ Change pp to max(scrap base,pp+d) 173 ⟩ ≡ if pp + d ≥ scrap base then pp ← pp + d else pp ← scrap base

This code is used in sections 172 and 174.

174. Similarly, the ‘squash’ macro invokes a procedure called ‘sq ’. This procedure takes advantage of the simplification that occurs when k = 1. procedure sq (j : sixteen bits ; k : eight bits ; c : eight bits ; d : integer );

var i: 0 . . max scraps ; { index into scrap memory }

begin if k = 1 then begin cat [j] ← c; ⟨ Change pp to max(scrap base,pp+d) 173 ⟩; end

else begin for i ← j to j + k − 1 do begin app1 (i); end; red (j, k, c, d); end;

end;

175. Here now is the code that applies productions as long as possible. It requires two local labels (found and done), as well as a local variable (i). ⟨ Reduce the scraps using the productions until no more rules apply 175 ⟩ ≡

loop begin ⟨ Make sure the entries cat [pp . . (pp + 3)] are defined 176 ⟩;

if (tok ptr + 8 > max toks ) ∨ (text ptr + 4 > max texts ) then begin stat if tok ptr > max tok ptr then max tok ptr ← tok ptr ; if text ptr > max txt ptr then max txt ptr ← text ptr ; tats overflow (´ ´ ); end;

if pp > lo ptr then goto done;

⟨ Match a production at pp, or increase pp if there is no match 149 ⟩; end;

done:

This code is used in section 179.

176. If we get to the end of the scrap list, category codes equal to zero are stored, since zero does not match anything in a production. ⟨ Make sure the entries cat [pp . . (pp + 3)] are defined 176 ⟩ ≡

if lo ptr < pp + 3 then

begin repeat if hi ptr ≤ scrap ptr then begin incr (lo ptr ); cat [lo ptr ] ← cat [hi ptr ]; trans [lo ptr ] ← trans [hi ptr ]; incr (hi ptr ); end;

until (hi ptr > scrap ptr ) ∨ (lo ptr = pp + 3); for i ← lo ptr + 1 to pp + 3 do cat [i] ← 0; end

This code is used in section 175.

177. If is being run in debugging mode, the production numbers and current stack categories will be printed out when tracing is set to 2; a sequence of two or more irreducible scraps will be printed out when tracing is set to 1. ⟨ Globals in the outer block 9 ⟩ +≡

debug tracing : 0 . . 2; { can be used to show parsing details } gubed 178. The prod procedure is called in debugging mode just after reduce or squash; its parameter is the number of the production that has just been applied.

debug procedure prod (n : eight bits ); { shows current categories } var k: 1 . . max scraps ; { index into cat } begin if tracing = 2 then

begin print nl (n : 1, ´ ´ );

for k ← scrap base to lo ptr do begin if k = pp then print (´ ´ ) else print (´ ␣´ ); print cat (cat [k]); end;

if hi ptr ≤ scrap ptr then print (´ ´ ); { indicate that more is coming } end;

end; gubed

179. The translate function assumes that scraps have been stored in positions scrap base through scrap ptr of cat and trans . It appends a terminator scrap and begins to apply productions as much as possible. The result is a token list containing the translation of the given sequence of scraps.

After calling translate, we will have text ptr + 3 ≤ max texts and tok ptr + 6 ≤ max toks , so it will be possible to create up to three token lists with up to six tokens without checking for overflow. Before calling translate, we should have text ptr < max texts and scrap ptr < max scraps , since translate might add a new text and a new scrap before it checks for overflow.

⟨ Declaration of subprocedures for translate 150 ⟩ function translate: text pointer ; { converts a sequence of scraps }

label done, found ; var i: 1 . . max scraps ; { index into cat }

j: 0 . . max scraps ; { runs through final scraps }

k: 0 . . long buf size; { index into buffer } begin pp ← scrap base; lo ptr ← pp − 1; hi ptr ← pp; ⟨ If tracing, print an indication of where we are 182 ⟩; ⟨ Reduce the scraps using the productions until no more rules apply 175 ⟩; if (lo ptr = scrap base) ∧ (cat [lo ptr ] ≠ math) then translate ← trans [lo ptr ] else ⟨ Combine the irreducible scraps that remain 180 ⟩; end; 180. If the initial sequence of scraps does not reduce to a single scrap, we concatenate the translations of all remaining scraps, separated by blank spaces, with dollar signs surrounding the translations of math scraps. ⟨ Combine the irreducible scraps that remain 180 ⟩ ≡

begin ⟨ If semi-tracing, show the irreducible scraps 181 ⟩; for j ← scrap base to lo ptr do

begin if j ≠ scrap base then begin app( ␣ ); end;

if cat [j] = math then begin app( $ ); end;

app1 (j); if cat [j] = math then

begin app( $ );

end; if tok ptr + 6 > max toks then overflow (´ ´ ); end;

freeze text ; translate ← text ptr − 1; end

This code is used in section 179.

181. ⟨ If semi-tracing, show the irreducible scraps 181 ⟩ ≡

debug if (lo ptr > scrap base) ∧ (tracing = 1) then begin print nl (´ ␣ ␣ ␣ ␣ ␣´ , module count : 1); print ln(´ ´ ); mark harmless ; for j ← scrap base to lo ptr do

begin print (´ ␣´ ); print cat (cat [j]);

end; end;

gubed

This code is used in section 180.

182. ⟨ If tracing, print an indication of where we are 182 ⟩ ≡

debug if tracing = 2 then begin print nl (´ ␣ ␣ ´ , line : 1, ´ ´ ); mark harmless ; if loc > 50 then

begin print (´ ´ );

for k ← loc − 50 to loc do print (xchr [buffer [k − 1]]); end

else for k ← 1 to loc do print (xchr [buffer [k − 1]]); end

gubed

This code is used in section 179.

§183  MWEAVE  INITIALIZING THE SCRAPS 69

183* . Initializing the scraps. If we are going to use the powerful production mechanism just developed, we must get the scraps set up in the first place, given a Pascal text. A table of the initial scraps corresponding to Pascal tokens appeared above in the section on parsing; our goal now is to implement that table. We shall do this by implementing a subroutine called Pascal parse that is analogous to the Pascal xref routine used during phase one.

Like Pascal xref , the Pascal parse procedure starts with the current value of next control and it uses the operation next control ← get next repeatedly to read Pascal text until encountering the next ‘ ’ or ‘ ’, or until next control ≥ format . The scraps corresponding to what it reads are appended into the cat and trans arrays, and scrap ptr is advanced.

Like prod , this procedure has to split into pieces so that each part is short enough to be handled by Pascal compilers that discriminate against long subroutines. This time there are two split-off routines, called easy cases and sub cases .

After studying Pascal parse, we will look at the sub-procedures app comment , app octal , and app hex that are used in some of its branches.

⟨ Declaration of the app comment procedure 195 ⟩ ⟨ Declaration of the app octal and app hex procedures 196 ⟩ ⟨ Declaration of the easy cases procedure 186* ⟩ ⟨ Declaration of the sub cases procedure 192* ⟩

procedure Pascal parse; { creates scraps from Pascal tokens } label reswitch, exit ; var j: 0 . . long buf size; { index into buffer }

p: name pointer ; { identifier designator } begin while next control < format do

begin ⟨ Append the scrap appropriate to next control 185 ⟩;

next control ← get next ;

if (next control = ) ∨ (next control = begin regular comment ) then return; end;

exit : end;

184. The macros defined here are helpful abbreviations for the operations needed when generating the scraps. A scrap of category c whose translation has three tokens t_{1}, t_{2}, t_{3} is generated by sc3 (t_{1})(t_{2})(t_{3})(c), etc.

define s0 (#) ≡ incr (scrap ptr ); cat [scrap ptr ] ← #; trans [scrap ptr ] ← text ptr ; freeze text ;

end define s1 (#) ≡ app(#); s0 define s2 (#) ≡ app(#); s1 define s3 (#) ≡ app(#); s2 define s4 (#) ≡ app(#); s3 define sc4 ≡ begin s4 define sc3 ≡ begin s3 define sc2 ≡ begin s2 define sc1 ≡ begin s1 define sc0 (#) ≡

begin incr (scrap ptr ); cat [scrap ptr ] ← #; trans [scrap ptr ] ← 0; end

70  INITIALIZING THE SCRAPS  MWEAVE §185

define comment scrap(#) ≡ begin app(#); app comment ; end 185. ⟨ Append the scrap appropriate to next control 185 ⟩ ≡

⟨ Make sure that there is room for at least four more scraps, six more tokens, and four more texts 187 ⟩; reswitch: case next control of

string , verbatim: ⟨ Append a string scrap 189 ⟩; identifier : ⟨ Append an identifier scrap 191* ⟩; TeX string : ⟨ Append a TEX string scrap 190 ⟩;

othercases easy cases

endcases

This code is used in section 183*.

186* . The easy cases each result in straightforward scraps. ⟨ Declaration of the easy cases procedure 186* ⟩ ≡ procedure easy cases ; { a subprocedure of Pascal parse }

begin case next control of

set element sign: sc3 ( )( )( )(math); double dot : sc3 ( )( )( )(math); $ , % , , : sc2 ( )(next control )(math); # : if in def part then

begin pound found ← true; sc2 ( )( # )(math); end

else sc2 ( )( )(math); : sc2 ( )( )(math); : sc2 ( )( )(math); ignore, xref roman, xref wildcard , xref typewriter : do nothing ; , : sc1 (next control )(open); , : sc1 (next control )(close); : sc4 ( )( )( )( )(math); : sc3 ( )(opt )( )(math); , , , , , , , , , , : sc1 (next control )(simp); : sc1 ( )(semi ); : sc2 ( )( )(open); : sc2 ( )( )(close); , vertical bar : sc4 ( )( )( )(force)(semi ); : sc1 ( )(colon); ⟨ Cases involving nonstandard ASCII characters 188* ⟩ exponent : sc3 ( )( )( )(exp); begin comment : sc2 ( )( )(math); end comment : sc2 ( )( )(math); octal : app octal ; hex : app hex ; check sum: sc2 ( )( )(simp); force line: sc2 ( )( )(simp); thin space: sc2 ( )( )(math); math break : sc2 (opt )( )(simp); line break : comment scrap(force); big line break : comment scrap(big force); no line break : begin app(big cancel ); app( ); app( ␣ ); comment scrap(big cancel );

end; pseudo semi : sc0 (semi ); join: sc2 ( )( )(math); othercases sc1 (next control )(math) endcases; end;

This code is used in section 183*.

187. ⟨ Make sure that there is room for at least four more scraps, six more tokens, and four more

texts 187 ⟩ ≡ if (scrap ptr + 4 > max scraps ) ∨ (tok ptr + 6 > max toks ) ∨ (text ptr + 4 > max texts ) then

begin stat if scrap ptr > max scr ptr then max scr ptr ← scrap ptr ; if tok ptr > max tok ptr then max tok ptr ← tok ptr ; if text ptr > max txt ptr then max txt ptr ← text ptr ; tats overflow (´ ´ ); end

This code is used in section 185.

188* . Some nonstandard ASCII characters may have entered by means of standard ones. They are converted to TEX control sequences so that it is possible to keep from stepping beyond standard ASCII. ⟨ Cases involving nonstandard ASCII characters 188* ⟩ ≡ not equal : sc2 ( )( )(math); less or equal : sc2 ( )( )(math); greater or equal : sc2 ( )( )(math); equivalence sign: begin sc2 ( )( )(math);

if ¬pound found then in def part ← false;

end; and sign: sc2 ( )( )(math); or sign: sc2 ( )( )(math); not sign: sc2 ( )( )(math); left arrow : sc2 ( )( )(math);

This code is used in section 186*.

189. The following code must use app tok instead of app in order to protect against overflow. Note that tok ptr + 1 ≤ max toks after app tok has been used, so another app is legitimate before testing again.

Many of the special characters in a string must be prefixed by ‘ ’ so that TEX will print them properly. ⟨ Append a string scrap 189 ⟩ ≡

begin app( );

if next control = verbatim then begin app( ); end

else begin app( );

end; app( ); j ← id first ; while j < id loc do

begin case buffer [j] of

␣ , , # , % , $ , , ´ , ` , , , , , : begin app( );

end; : if buffer [j + 1] = then incr (j)

else err print (´ ␣ ␣ ␣ ␣ ␣ ␣ ␣ ´ );

othercases do nothing

endcases;

app tok (buffer [j]); incr (j);

end; sc1 ( )(simp); end

This code is used in section 185.

190. ⟨ Append a TEX string scrap 190 ⟩ ≡ begin app( ); app( ); app( ); app( ); app( ); app( ); for j ← id first to id loc − 1 do app tok (buffer [j]); sc1 ( )(simp); end

This code is used in section 185.

191* . ⟨ Append an identifier scrap 191* ⟩ ≡ begin p ← id lookup(normal ); case ilk [p] of normal , array like, const like, div like, of like, for like, goto like, nil like, to like: sub cases (p); ⟨ Cases that generate more than one scrap 193* ⟩ othercases begin next control ← ilk [p] − char like; goto reswitch;

end { and, in, not, or } endcases; end

This code is used in section 185.

192* . The sub cases also result in straightforward scraps. ⟨ Declaration of the sub cases procedure 192* ⟩ ≡ procedure sub cases (p : name pointer ); { a subprocedure of Pascal parse }

begin case ilk [p] of normal : sc1 (id flag + p)(simp); { not a reserved word } array like: sc1 (res flag + p)(alpha); { array, file, set } const like: sc3 (force)(backup)(res flag + p)(intro); { const, label, type } div like: sc3 (math bin)(res flag + p)( )(math); { div, mod } for like: sc2 (force)(res flag + p)(alpha); { for, if, while, with } goto like: sc1 (res flag + p)(intro); { packed } nil like: sc1 (res flag + p)(simp); { nil } of like: sc1 (res flag + p)(omega); { of } to like: sc3 (math rel )(res flag + p)( )(math); { to } end; end;

This code is used in section 183*.

193* . ⟨ Cases that generate more than one scrap 193* ⟩ ≡ begin like: begin sc4 (force)(backup)(res flag + p)(force)(beginning ); sc0 (intro);

end; { begin } case like: begin sc0 (casey ); sc2 (force)(res flag + p)(alpha);

end; { case } do like: begin sc2 (res flag + p)(force)(omega); sc0 (beginning ); sc0 (intro);

end; { do, then } elsif like: begin ⟨ Append terminator if not already present 194 ⟩;

sc0 (close); sc0 (terminator ); sc1 (res flag + p)(alpha);

end; else like: begin sc4 (force)(backup)(res flag + p)(force)(terminator );

end; { else } end like: begin ⟨ Append terminator if not already present 194 ⟩;

sc4 (force)(backup)(res flag + p)(break space)(close);

end; { end } loop like: begin sc1 (force)(alpha); sc2 (res flag + p)(force)(omega); sc0 (beginning ); sc0 (intro);

end; { loop } proc like: begin sc4 (force)(backup)(res flag + p)(cancel )(proc); sc3 (indent )( )( ␣ )(intro);

end; { function, procedure, program } record like: begin sc1 (res flag + p)(alpha); sc0 (omega); sc0 (beginning );

end; { record } repeat like: begin sc4 (force)(indent )(res flag + p)(force)(beginning ); sc0 (intro);

end; { repeat } until like: begin ⟨ Append terminator if not already present 194 ⟩;

sc3 (force)(backup)(res flag + p)(close); sc0 (clause);

end; { until } var like: begin sc4 (force)(backup)(res flag + p)(cancel )(var head ); sc0 (intro);

end; { var }

This code is used in section 191*.

194. If a comment or semicolon appears before the reserved words end, else, or until, the semi or terminator scrap that is already present overrides the terminator scrap belonging to this reserved word. ⟨ Append terminator if not already present 194 ⟩ ≡

if (scrap ptr < scrap base) ∨ ((cat [scrap ptr ] ≠ terminator ) ∧ (cat [scrap ptr ] ≠ semi )) then

sc0 (terminator )

This code is used in sections 193*, 193*, and 193*.

195. A comment is incorporated into the previous scrap if that scrap is of type omega or semi or terminator . (These three categories have consecutive category codes.) Otherwise the comment is entered as a separate scrap of type terminator , and it will combine with a terminator scrap that immediately follows it.

The app comment procedure takes care of placing a comment at the end of the current scrap list. When app comment is called, we assume that the current token list is the translation of the comment involved. ⟨ Declaration of the app comment procedure 195 ⟩ ≡ procedure app comment ; { append a comment to the scrap list }

begin freeze text ; if (scrap ptr < scrap base) ∨ (cat [scrap ptr ] < omega) ∨ (cat [scrap ptr ] > terminator ) then

sc0 (terminator ) else begin app1 (scrap ptr ); { cat [scrap ptr ] is omega or semi or terminator }

end; app(text ptr − 1 + tok flag ); trans [scrap ptr ] ← text ptr ; freeze text ; end;

This code is used in section 183*.

196. We are now finished with Pascal parse, except for two relatively trivial subprocedures that convert constants into tokens. ⟨ Declaration of the app octal and app hex procedures 196 ⟩ ≡ procedure app octal ;

begin app( ); app( ); app( ); while (buffer [loc] ≥ ) ∧ (buffer [loc] ≤ ) do

begin app tok (buffer [loc]); incr (loc);

end; sc1 ( )(simp); end;

procedure app hex ; begin app( ); app( ); app( ); while ((buffer [loc] ≥ ) ∧ (buffer [loc] ≤ )) ∨ ((buffer [loc] ≥ ) ∧ (buffer [loc] ≤ )) do

begin app tok (buffer [loc]); incr (loc);

end; sc1 ( )(simp); end;

This code is used in section 183*.

197. When the ‘ ’ that introduces Pascal text is sensed, a call on Pascal translate will return a pointer to the TEX translation of that text. If scraps exist in the cat and trans arrays, they are unaffected by this translation process. function Pascal translate: text pointer ;

var p: text pointer ; { points to the translation }

save base: 0 . . max scraps ; { holds original value of scrap base } begin save base ← scrap base; scrap base ← scrap ptr + 1; Pascal parse; { get the scraps together } if next control ≠ then err print (´ ␣ ␣ ␣ ␣ ␣ ´ ); app tok (cancel ); app comment ; { place a cancel token as a final “comment” } p ← translate; { make the translation } stat if scrap ptr > max scr ptr then max scr ptr ← scrap ptr ; tats scrap ptr ← scrap base − 1; scrap base ← save base; { scrap the scraps } Pascal translate ← p; end; 198* . The outer parse routine is to Pascal parse as outer xref is to Pascal xref : It constructs a sequence of scraps for Pascal text until next control ≥ format . Thus, it takes care of embedded comments. procedure outer parse; { makes scraps from Pascal tokens and comments }

var bal : eight bits ; { brace level in comment }

p, q: text pointer ; { partial comments } begin while next control < format do

if next control ≠ begin regular comment then Pascal parse else begin ⟨ Make sure that there is room for at least seven more tokens, three more texts, and one

more scrap 199 ⟩; app( ); app( ); app( ); bal ← copy comment (1); next control ← ignore; while bal > 0 do

begin p ← text ptr ; freeze text ; q ← Pascal translate; { at this point we have tok ptr + 6 ≤ max toks }

app(tok flag + p); app(inner tok flag + q);

if next control = then begin next control ← ignore; bal ← copy comment (bal ); end

else bal ← 0; { an error has been reported }

end; app(force); app comment ; { the full comment becomes a scrap } end;

end;

199. ⟨ Make sure that there is room for at least seven more tokens, three more texts, and one more

scrap 199 ⟩ ≡ if (tok ptr + 7 > max toks ) ∨ (text ptr + 3 > max texts ) ∨ (scrap ptr ≥ max scraps ) then

begin stat if scrap ptr > max scr ptr then max scr ptr ← scrap ptr ; if tok ptr > max tok ptr then max tok ptr ← tok ptr ; if text ptr > max txt ptr then max txt ptr ← text ptr ; tats overflow (´ ´ ); end

This code is used in section 198*.

§200  MWEAVE  OUTPUT OF TOKENS 77

200. Output of tokens. So far our programs have only built up multi-layered token lists in ’s internal memory; we have to figure out how to get them into the desired final form. The job of converting token lists to characters in the TEX output file is not difficult, although it is an implicitly recursive process. Three main considerations had to be kept in mind when this part of was designed: (a) There are two modes of output, outer mode that translates tokens like force into line-breaking control sequences, and inner mode that ignores them except that blank spaces take the place of line breaks. (b) The cancel instruction applies to adjacent token or tokens that are output, and this cuts across levels of recursion since ‘cancel ’ occurs at the beginning or end of a token list on one level. (c) The TEX output file will be semi-readable if line breaks are inserted after the result of tokens like break space and force. (d) The final line break should be suppressed, and there should be no force token output immediately after ‘ ’.

201. The output process uses a stack to keep track of what is going on at different “levels” as the token lists are being written out. Entries on this stack have three parts:

end field is the tok mem location where the token list of a particular level will end;

tok field is the tok mem location from which the next token on a particular level will be read; mode field is the current mode, either inner or outer .

The current values of these quantities are referred to quite frequently, so they are stored in a separate place instead of in the stack array. We call the current values cur end , cur tok , and cur mode.

The global variable stack ptr tells how many levels of output are currently in progress. The end of output occurs when an end translation token is found, so the stack is never empty except when we first begin the output process.

define inner = 0 { value of mode for Pascal texts within TEX texts }

define outer = 1 { value of mode for Pascal texts in modules } ⟨ Types in the outer block 11 ⟩ +≡

mode = inner . . outer ;

output state = record end field : sixteen bits ; { ending location of token list } tok field : sixteen bits ; { present location within token list } mode field : mode; { interpretation of control tokens } end;

202. define cur end ≡ cur state.end field { current ending location in tok mem } define cur tok ≡ cur state.tok field { location of next output token in tok mem } define cur mode ≡ cur state.mode field { current mode of interpretation } define init stack ≡ stack ptr ← 0; cur mode ← outer { do this to initialize the stack }

⟨ Globals in the outer block 9 ⟩ +≡ cur state: output state; { cur end , cur tok , cur mode } stack : array [1 . . stack size] of output state; { info for non-current levels } stack ptr : 0 . . stack size; { first unused location in the output state stack }

stat max stack ptr : 0 . . stack size; { largest value assumed by stack ptr }

tats

78  OUTPUT OF TOKENS  MWEAVE §204

203. ⟨ Set initial values 10 ⟩ +≡ stat max stack ptr ← 0; tats 204. To insert token-list p into the output, the push level subroutine is called; it saves the old level of output and gets a new one going. The value of cur mode is not changed. procedure push level (p : text pointer ); { suspends the current level }

begin if stack ptr = stack size then overflow (´ ´ ) else begin if stack ptr > 0 then stack [stack ptr ] ← cur state; { save cur end … cur mode }

incr (stack ptr );

stat if stack ptr > max stack ptr then max stack ptr ← stack ptr ; tats cur tok ← tok start [p]; cur end ← tok start [p + 1]; end;

end;

205. Conversely, the pop level routine restores the conditions that were in force when the current level was begun. This subroutine will never be called when stack ptr = 1. It is so simple, we declare it as a macro:

define pop level ≡ begin decr (stack ptr ); cur state ← stack [stack ptr ]; end { do this when cur tok reaches cur end }

206. The get output function returns the next byte of output that is not a reference to a token list. It returns the values identifier or res word or mod name if the next token is to be an identifier (typeset in italics), a reserved word (typeset in boldface) or a module name (typeset by a complex routine that might generate additional levels of output). In these cases cur name points to the identifier or module name in question.

define res word = ´ 201 { returned by get output for reserved words } define mod name = ´ 200 { returned by get output for module names }

function get output : eight bits ; { returns the next token of output } label restart ; var a: sixteen bits ; { current item read from tok mem } begin restart : while cur tok = cur end do pop level ; a ← tok mem[cur tok ]; incr (cur tok ); if a ≥ ´ 400 then

begin cur name ← a mod id flag ;

case a div id flag of

2: a ← res word ; { a = res flag + cur name }

3: a ← mod name; { a = mod flag + cur name }

4: begin push level (cur name); goto restart ;

end; { a = tok flag + cur name } 5: begin push level (cur name); cur mode ← inner ; goto restart ;

end; { a = inner tok flag + cur name } othercases a ← identifier { a = id flag + cur name } endcases; end;

debug if trouble shooting then debug help; gubed get output ← a; end; 207* . The real work associated with token output is done by make output . This procedure appends an end translation token to the current token list, and then it repeatedly calls get output and feeds characters to the output buffer until reaching the end translation sentinel. It is possible for make output to be called recursively, since a module name may include embedded Pascal text; however, the depth of recursion never exceeds one level, since module names cannot be inside of module names.

A procedure called output Pascal does the scanning, translation, and output of Pascal text within ‘ … ’ brackets, and this procedure uses make output to output the current token list. Thus, the recursive call of make output actually occurs when make output calls output Pascal while outputting the name of a module. procedure make output ; forward ; procedure output Pascal ; { outputs the current token list }

var save tok ptr , save text ptr , save next control : sixteen bits ; { values to be restored }

p: text pointer ; { translation of the Pascal text } begin save tok ptr ← tok ptr ; save text ptr ← text ptr ; save next control ← next control ; next control ← ignore; p ← Pascal translate; app(p + inner tok flag ); make output ; { output the list } stat if text ptr > max txt ptr then max txt ptr ← text ptr ; if tok ptr > max tok ptr then max tok ptr ← tok ptr ; tats text ptr ← save text ptr ; tok ptr ← save tok ptr ; { forget the tokens } next control ← save next control ; { restore next control to original state } end; 208. Here is ’s major output handler. procedure make output ; { outputs the equivalents of tokens }

label reswitch, exit , found ; var a: eight bits ; { current output byte }

b: eight bits ; { next output byte } k, k limit : 0 . . max bytes ; { indices into byte mem } w: 0 . . ww − 1; { row of byte mem } j: 0 . . long buf size; { index into buffer } string delimiter : ASCII code; { first and last character of string being copied } save loc, save limit : 0 . . long buf size; { loc and limit to be restored } cur mod name: name pointer ; { name of module being output } save mode: mode; { value of cur mode before a sequence of breaks }

begin app(end translation); { append a sentinel } freeze text ; push level (text ptr − 1); loop begin a ← get output ; reswitch: case a of

end translation: return; identifier , res word : ⟨ Output an identifier 209 ⟩; mod name: ⟨ Output a module name 213 ⟩; math bin, math op, math rel : ⟨ Output a operator 210 ⟩; cancel : begin repeat a ← get output ;

until (a < backup) ∨ (a > big force);

goto reswitch;

end; big cancel : begin repeat a ← get output ;

until ((a < backup) ∧ (a ≠ ␣ )) ∨ (a > big force); goto reswitch; end;

indent , outdent , opt , backup, break space, force, big force: ⟨ Output a control, look ahead in case of line

breaks, possibly goto reswitch 211 ⟩; othercases out (a) { otherwise a is an ASCII character } endcases; end;

exit : end;

209. An identifier of length one does not have to be enclosed in braces, and it looks slightly better if set in a math-italic font instead of a (slightly narrower) text-italic font. Thus we output ‘ ’ but ‘ ’. ⟨ Output an identifier 209 ⟩ ≡

begin out ( ); if a = identifier then

if length(cur name) = 1 then out ( ) else out ( )

else out ( ); { a = res word } if length(cur name) = 1 then out (byte mem[cur name mod ww , byte start [cur name]]) else out name(cur name); end

This code is used in section 208.

210. ⟨ Output a operator 210 ⟩ ≡ begin out5 ( )( )( )( )( ); if a = math bin then out3 ( )( )( ) else if a = math rel then out3 ( )( )( )

else out2 ( )( ); out ( ); end

This code is used in section 208.

211. The current mode does not affect the behavior of ’s output routine except when we are outputting control tokens. ⟨ Output a control, look ahead in case of line breaks, possibly goto reswitch 211 ⟩ ≡

if a < break space then

begin if cur mode = outer then begin out2 ( )(a + −cancel + ); if a = opt then out (get output ) { opt is followed by a digit } end

else if a = opt then b ← get output { ignore digit following opt }

end else ⟨ Look ahead for strongest line break, goto reswitch 212 ⟩

This code is used in section 208.

212. If several of the tokens break space, force, big force occur in a row, possibly mixed with blank spaces (which are ignored), the largest one is used. A line break also occurs in the output file, except at the very end of the translation. The very first line break is suppressed (i.e., a line break that follows ‘ ’). ⟨ Look ahead for strongest line break, goto reswitch 212 ⟩ ≡

begin b ← a; save mode ← cur mode; loop begin a ← get output ;

if (a = cancel ) ∨ (a = big cancel ) then goto reswitch; { cancel overrides everything } if ((a ≠ ␣ ) ∧ (a < break space)) ∨ (a > big force) then

begin if save mode = outer then begin if out ptr > 3 then

if (out buf [out ptr ] = ) ∧ (out buf [out ptr − 1] = ) ∧ (out buf [out ptr − 2] =

) ∧ (out buf [out ptr − 3] = ) then goto reswitch;

out2 ( )(b − cancel + );

if a ≠ end translation then finish line; end

else if (a ≠ end translation) ∧ (cur mode = inner ) then out ( ␣ ); goto reswitch; end;

if a > b then b ← a; { if a = ␣ we have a < b }

end; end

This code is used in section 211.

213. The remaining part of make output is somewhat more complicated. When we output a module name, we may need to enter the parsing and translation routines, since the name may contain Pascal code embedded in … constructions. This Pascal code is placed at the end of the active input buffer and the translation process uses the end of the active tok mem area. ⟨ Output a module name 213 ⟩ ≡

begin out2 ( )( ); cur xref ← xref [cur name]; if num(cur xref ) ≥ def flag then

begin out mod (num(cur xref ) − def flag );

if phase three then

begin cur xref ← xlink (cur xref ); while num(cur xref ) ≥ def flag do

begin out2 ( )( ␣ ); out mod (num(cur xref ) − def flag ); cur xref ← xlink (cur xref ); end; end;

end else out ( ); { output the module number, or zero if it was undefined } out ( ); ⟨ Output the text of the module name 214 ⟩; out2 ( )( ); end

This code is used in section 208.

214. ⟨ Output the text of the module name 214 ⟩ ≡ k ← byte start [cur name]; w ← cur name mod ww ; k limit ← byte start [cur name + ww ]; cur mod name ← cur name; while k < k limit do

begin b ← byte mem[w, k]; incr (k);

if b = then ⟨ Skip next character, give error if not ‘ ’ 215 ⟩;

if b ≠ then out (b)

else begin ⟨ Copy the Pascal text into buffer [(limit + 1) . . j] 216 ⟩; save loc ← loc; save limit ← limit ; loc ← limit + 2; limit ← j + 1; buffer [limit ] ← ; output Pascal ; loc ← save loc; limit ← save limit ; end;

end

This code is used in section 213.

215. ⟨ Skip next character, give error if not ‘ ’ 215 ⟩ ≡

begin if byte mem[w, k] ≠ then begin print nl (´ ␣ ␣ ␣ ␣ ␣ ␣ ´ ); print nl (´ ´ ); print id (cur mod name); print (´ ␣´ ); mark error ; end;

incr (k); end

This code is used in section 214.

216. The Pascal text enclosed in … should not contain ‘ ’ characters, except within strings. We put a ‘ ’ at the front of the buffer, so that an error message that displays the whole buffer will look a little bit sensible. The variable string delimiter is zero outside of strings, otherwise it equals the delimiter that began the string being copied. ⟨ Copy the Pascal text into buffer [(limit + 1) . . j] 216 ⟩ ≡

j ← limit + 1; buffer [j] ← ; string delimiter ← 0;

loop begin if k ≥ k limit then begin print nl (´ ␣ ␣ ␣ ␣ ␣ ␣ ´ ´ ␣ ´ ); print nl (´ ´ ); print id (cur mod name); print (´ ␣´ ); mark error ; goto found ; end;

b ← byte mem[w, k]; incr (k);

if b = then ⟨ Copy a control code into the buffer 217 ⟩ else begin if (b = ) ∨ (b = ´ ) then

if string delimiter = 0 then string delimiter ← b else if string delimiter = b then string delimiter ← 0;

if (b ≠ ) ∨ (string delimiter ≠ 0) then begin if j > long buf size − 3 then overflow (´ ´ ); incr (j); buffer [j] ← b; end

else goto found ;

end; end;

found :

This code is used in section 214.

217. ⟨ Copy a control code into the buffer 217 ⟩ ≡ begin if j > long buf size − 4 then overflow (´ ´ ); buffer [j + 1] ← ; buffer [j + 2] ← byte mem[w, k]; j ← j + 2; incr (k); end

This code is used in section 216.

84  PHASE TWO PROCESSING  MWEAVE §218

218. Phase two processing. We have assembled enough pieces of the puzzle in order to be ready to specify the processing in ’s main pass over the source file. Phase two is analogous to phase one, except that more work is involved because we must actually output the TEX material instead of merely looking at the specifications. ⟨ Phase II: Read all the text again and translate it to TEX form 218 ⟩ ≡

reset input ; print nl (´ ␣ ␣ ␣ ´ ); module count ← 0; copy limbo; finish line; flush buffer (0, false); { insert a blank line, it looks nice } while ¬input has ended do ⟨ Translate the current module 220 ⟩

This code is used in section 261.

219. The output file will contain the control sequence between non-null sections of a module, e.g., between the TEX and definition parts if both are nonempty. This puts a little white space between the parts when they are printed. However, we don’t want to occur between two definitions within a single module. The variables out line or out ptr will change if a section is non-null, so the following macros ‘save position’ and ‘emit space if needed ’ are able to handle the situation:

define save position ≡ save line ← out line; save place ← out ptr

define emit space if needed ≡

if (save line ≠ out line) ∨ (save place ≠ out ptr ) then out2 ( )( )

⟨ Globals in the outer block 9 ⟩ +≡ save line: integer ; { former value of out line } save place: sixteen bits ; { former value of out ptr }

220. ⟨ Translate the current module 220 ⟩ ≡ begin incr (module count ); ⟨ Output the code for the beginning of a new module 221 ⟩; save position; ⟨ Translate the TEX part of the current module 222* ⟩; ⟨ Translate the definition part of the current module 225 ⟩; ⟨ Translate the Pascal part of the current module 230* ⟩; ⟨ Show cross references to this module 233 ⟩; ⟨ Output the code for the end of a module 238 ⟩; end

This code is used in section 218.

221. Modules beginning with the control sequence ‘ ␣’ start in the output with the TEX control sequence ‘ ’, followed by the module number. Similarly, ‘ ’ modules lead to the control sequence ‘ ’. If this is a changed module, we put just before the module number. ⟨ Output the code for the beginning of a new module 221 ⟩ ≡

out ( );

if buffer [loc − 1] ≠ then out ( )

else begin out ( ); print (´ ´ , module count : 1); update terminal ; { print a progress report }

end; out mod (module count ); out2 ( )( ␣ )

This code is used in section 220.

§222  MWEAVE  PHASE TWO PROCESSING 85

222* . In the TEX part of a module, we simply copy the source text, except that index entries are not copied and Pascal text within … is translated. ⟨ Translate the TEX part of the current module 222* ⟩ ≡

in def part ← false; repeat next control ← copy TeX ;

case next control of

: begin next control ← ignore; init stack ; output Pascal ;

end; : out ( ); octal : ⟨ Translate an octal constant appearing in TEX text 223 ⟩; hex : ⟨ Translate a hexadecimal constant appearing in TEX text 224 ⟩; TeX string , xref roman, xref wildcard , xref typewriter , module name: begin loc ← loc − 2;

next control ← get next ; { skip to }

if next control = TeX string then err print (´ ␣ ␣ ␣ ␣ ␣ ␣ ␣ ␣ ´ ); end;

begin comment , end comment , check sum, thin space, math break , line break , big line break ,

no line break , join, pseudo semi : err print (´ ␣ ␣ ´ ´ ␣ ␣ ␣ ␣ ␣ ´ );

othercases do nothing

endcases; until next control ≥ format

This code is used in section 220.

223. ⟨ Translate an octal constant appearing in TEX text 223 ⟩ ≡ begin out3 ( )( )( ); while (buffer [loc] ≥ ) ∧ (buffer [loc] ≤ ) do

begin out (buffer [loc]); incr (loc); end; { since buffer [limit ] = ␣ , this loop will end }

out ( ); end

This code is used in section 222*.

224. ⟨ Translate a hexadecimal constant appearing in TEX text 224 ⟩ ≡ begin out3 ( )( )( ); while ((buffer [loc] ≥ ) ∧ (buffer [loc] ≤ )) ∨ ((buffer [loc] ≥ ) ∧ (buffer [loc] ≤ )) do

begin out (buffer [loc]); incr (loc); end;

out ( ); end

This code is used in section 222*.

225. When we get to the following code we have next control ≥ format , and the token memory is in its initial empty state. ⟨ Translate the definition part of the current module 225 ⟩ ≡

if next control ≤ definition then { definition part non-empty } begin emit space if needed ; save position; end;

while next control ≤ definition do { format or definition } begin init stack ; if next control = definition then ⟨ Start a macro definition 227* ⟩ else ⟨ Start a format definition 228 ⟩; outer parse; finish Pascal ; end

This code is used in section 220.

226. The finish Pascal procedure outputs the translation of the current scraps, preceded by the control sequence ‘ ’ and followed by the control sequence ‘ ’. It also restores the token and scrap memories to their initial empty state.

A force token is appended to the current scraps before translation takes place, so that the translation will normally end with or (the TEX macros for force and big force). This or is replaced by the concluding or by . procedure finish Pascal ; { finishes a definition or a Pascal part }

var p: text pointer ; { translation of the scraps } begin out2 ( )( ); app tok (force); app comment ; p ← translate; app(p + tok flag ); make output ;

{ output the list } if out ptr > 1 then

if out buf [out ptr − 1] = then if out buf [out ptr ] = then out ptr ← out ptr − 2 else if out buf [out ptr ] = then out buf [out ptr ] ← ;

out4 ( )( )( )( ); finish line; stat if text ptr > max txt ptr then max txt ptr ← text ptr ; if tok ptr > max tok ptr then max tok ptr ← tok ptr ; if scrap ptr > max scr ptr then max scr ptr ← scrap ptr ; tats tok ptr ← 1; text ptr ← 1; scrap ptr ← 0; { forget the tokens and the scraps } end;

227* . ⟨ Start a macro definition 227* ⟩ ≡ begin sc2 ( )( )(intro); { this will produce ‘define ’ } in def part ← true; pound found ← false; next control ← get next ; if next control ≠ identifier then err print (´ ␣ ␣ ␣ ´ ) else sc1 (id flag + id lookup(normal ))(math); next control ← get next ; end

This code is used in section 225.

228. ⟨ Start a format definition 228 ⟩ ≡ begin sc2 ( )( )(intro); { this will produce ‘format ’ } next control ← get next ; if next control = identifier then

begin sc1 (id flag + id lookup(normal ))(math); next control ← get next ; if next control = equivalence sign then

begin sc2 ( )( )(math); { output an equivalence sign }

next control ← get next ;

if next control = identifier then begin sc1 (id flag + id lookup(normal ))(math); sc0 (semi ); { insert an invisible semicolon } next control ← get next ; end;

end; end;

if scrap ptr ≠ 5 then err print (´ ␣ ␣ ␣ ´ ); end

This code is used in section 225.

229* . Finally, when the TEX and definition parts have been treated, we have next control ≥ begin Pascal . We will make the global variable this module point to the current module name, if it has a name. ⟨ Globals in the outer block 9 ⟩ +≡ this module: name pointer ; { the current module name, or zero } in def part , pound found : boolean; { flags to determine context of use of pound sign and vertical bar }

230* . ⟨ Translate the Pascal part of the current module 230* ⟩ ≡ this module ← 0; in def part ← false; if next control ≤ module name then

begin emit space if needed ; init stack ; if next control = begin Pascal then next control ← get next else begin this module ← cur module; ⟨ Check that = or ≡ follows this module name, and emit the

scraps to start the module definition 231 ⟩; end;

while next control ≤ module name do begin outer parse; ⟨ Emit the scrap for a module name if present 232 ⟩; end;

finish Pascal ; end

This code is used in section 220.

231. ⟨ Check that = or ≡ follows this module name, and emit the scraps to start the module

definition 231 ⟩ ≡ repeat next control ← get next ; until next control ≠ ; { allow optional ‘ ’ } if (next control ≠ ) ∧ (next control ≠ equivalence sign) then

err print (´ ␣ ␣ ␣ ␣ ␣ ␣ ␣ ␣ ␣ ´ ) else next control ← get next ; if out ptr > 1 then

if (out buf [out ptr ] = ) ∧ (out buf [out ptr − 1] = ) then begin app(backup); { the module name will be flush left } end;

sc1 (mod flag + this module)(mod scrap); cur xref ← xref [this module]; if num(cur xref ) ≠ module count + def flag then

begin sc3 (math rel )( )( )(math); { module name is multiply defined } this module ← 0; { so we won’t give cross-reference info here } end;

sc2 ( )( )(math); { output an equivalence sign } sc1 (force)(semi ); { this forces a line break unless ‘ ’ follows }

This code is used in section 230*.

232. ⟨ Emit the scrap for a module name if present 232 ⟩ ≡

if next control < module name then begin err print (´ ␣ ␣ ´ ´ ␣ ␣ ␣ ␣ ␣ ´ ); next control ← get next ; end

else if next control = module name then begin sc1 (mod flag + cur module)(mod scrap); next control ← get next ; end

This code is used in section 230*.

233. Cross references relating to a named module are given after the module ends. ⟨ Show cross references to this module 233 ⟩ ≡

if this module > 0 then begin ⟨ Rearrange the list pointed to by cur xref 235 ⟩; footnote(def flag ); footnote(0); end

This code is used in section 220.

234. To rearrange the order of the linked list of cross references, we need four more variables that point to cross reference entries. We’ll end up with a list pointed to by cur xref . ⟨ Globals in the outer block 9 ⟩ +≡ next xref , this xref , first xref , mid xref : xref number ; { pointer variables for rearranging a list } 235. We want to rearrange the cross reference list so that all the entries with def flag come first, in ascending order; then come all the other entries, in ascending order. There may be no entries in either one or both of these categories. ⟨ Rearrange the list pointed to by cur xref 235 ⟩ ≡

first xref ← xref [this module]; this xref ← xlink (first xref ); { bypass current module number } if num(this xref ) > def flag then

begin mid xref ← this xref ; cur xref ← 0; { this value doesn’t matter }

repeat next xref ← xlink (this xref ); xlink (this xref ) ← cur xref ; cur xref ← this xref ;

this xref ← next xref ; until num(this xref ) ≤ def flag ; xlink (first xref ) ← cur xref ; end

else mid xref ← 0; { first list null } cur xref ← 0; while this xref ≠ 0 do

begin next xref ← xlink (this xref ); xlink (this xref ) ← cur xref ; cur xref ← this xref ; this xref ← next xref ; end;

if mid xref > 0 then xlink (mid xref ) ← cur xref else xlink (first xref ) ← cur xref ; cur xref ← xlink (first xref )

This code is used in section 233.

236. The footnote procedure gives cross reference information about multiply defined module names (if the flag parameter is def flag ), or about the uses of a module name (if the flag parameter is zero). It assumes that cur xref points to the first cross-reference entry of interest, and it leaves cur xref pointing to the first element not printed. Typical outputs: ‘ ␣ ’; ‘ ␣ ’; ‘ ␣ ’. procedure footnote(flag : sixteen bits ); { outputs module cross-references }

label done, exit ; var q: xref number ; { cross-reference pointer variable } begin if num(cur xref ) ≤ flag then return; finish line; out ( ); if flag = 0 then out ( ) else out ( ); out4 ( ␣ )( )( )( ); out4 ( )( )( )( ); ⟨ Output all the module numbers on the reference list cur xref 237 ⟩; out ( );

exit : end; 237. The following code distinguishes three cases, according as the number of cross references is one, two, or more than two. Variable q points to the first cross reference, and the last link is a zero. ⟨ Output all the module numbers on the reference list cur xref 237 ⟩ ≡

q ← cur xref ;

if num(xlink (q)) > flag then out ( ); { plural }

out ( );

loop begin out mod (num(cur xref ) − flag ); cur xref ← xlink (cur xref );

{ point to the next cross reference to output } if num(cur xref ) ≤ flag then goto done; if (num(xlink (cur xref )) > flag ) ∨ (cur xref ≠ xlink (q)) then out ( ); { not the last of two } out ( ␣ ); if num(xlink (cur xref )) ≤ flag then out4 ( )( )( )( ); { the last } end;

done:

This code is used in section 236.

238. ⟨ Output the code for the end of a module 238 ⟩ ≡ out3 ( )( )( ); finish line; flush buffer (0, false); { insert a blank line, it looks nice }

This code is used in section 220.

§239  MWEAVE  PHASE THREE PROCESSING 91

239. Phase three processing. We are nearly finished! ’s only remaining task is to write out the index, after sorting the identifiers and index entries. ⟨ Phase III: Output the cross-reference index 239 ⟩ ≡

phase three ← true; print nl (´ ␣ ␣ ´ );

if change exists then

begin finish line; ⟨ Tell about changed modules 241 ⟩; end;

finish line; out4 ( )( )( )( ); finish line; ⟨ Do the first pass of sorting 243 ⟩; ⟨ Sort and output the index 250 ⟩; out4 ( )( )( )( ); finish line; ⟨ Output all the module names 257 ⟩; out4 ( )( )( )( ); finish line; print (´ ´ );

This code is used in section 261.

240. Just before the index comes a list of all the changed modules, including the index module itself. ⟨ Globals in the outer block 9 ⟩ +≡ k module: 0 . . max modules ; { runs through the modules }

241. ⟨ Tell about changed modules 241 ⟩ ≡ begin { remember that the index is already marked as changed } k module ← 1; while ¬changed module[k module] do incr (k module); out4 ( )( )( )( ␣ ); out mod (k module); repeat repeat incr (k module) until changed module[k module];

out2 ( )( ␣ ); out mod (k module); until k module = module count ; out ( ); end

This code is used in section 239.

92  PHASE THREE PROCESSING  MWEAVE §243

242. A left-to-right radix sorting method is used, since this makes it easy to adjust the collating sequence and since the running time will be at worst proportional to the total length of all entries in the index. We put the identifiers into 102 different lists based on their first characters. (Uppercase letters are put into the same list as the corresponding lowercase letters, since we want to have ‘t < TeX < to’.) The list for character c begins at location bucket [c] and continues through the blink array. ⟨ Globals in the outer block 9 ⟩ +≡ bucket : array [ASCII code] of name pointer ; next name: name pointer ; { successor of cur name when sorting } c: ASCII code; { index into bucket } h: 0 . . hash size; { index into hash } blink : array [0 . . max names ] of sixteen bits ; { links in the buckets } 243. To begin the sorting, we go through all the hash lists and put each entry having a nonempty crossreference list into the proper bucket. ⟨ Do the first pass of sorting 243 ⟩ ≡

for c ← 0 to 127 do bucket [c] ← 0;

for h ← 0 to hash size − 1 do begin next name ← hash[h]; while next name ≠ 0 do

begin cur name ← next name; next name ← link [cur name]; if xref [cur name] ≠ 0 then

begin c ← byte mem[cur name mod ww , byte start [cur name]]; if (c ≤ ) ∧ (c ≥ ) then c ← c + ´ 40 ; blink [cur name] ← bucket [c]; bucket [c] ← cur name; end; end; end

This code is used in section 239.

244. During the sorting phase we shall use the cat and trans arrays from ’s parsing algorithm and rename them depth and head . They now represent a stack of identifier lists for all the index entries that have not yet been output. The variable sort ptr tells how many such lists are present; the lists are output in reverse order (first sort ptr , then sort ptr − 1, etc.). The jth list starts at head [j], and if the first k characters of all entries on this list are known to be equal we have depth[j] = k.

define depth ≡ cat { reclaims memory that is no longer needed for parsing } define head ≡ trans { ditto } define sort ptr ≡ scrap ptr { ditto } define max sorts ≡ max scraps { ditto }

⟨ Globals in the outer block 9 ⟩ +≡ cur depth: eight bits ; { depth of current buckets } cur byte: 0 . . max bytes ; { index into byte mem } cur bank : 0 . . ww − 1; { row of byte mem } cur val : sixteen bits ; { current cross reference number }

stat max sort ptr : 0 . . max sorts ; tats { largest value of sort ptr }

245. ⟨ Set initial values 10 ⟩ +≡ stat max sort ptr ← 0; tats

246. The desired alphabetic order is specified by the collate array; namely, collate[0] < collate[1] < ⋯ < collate[100]. ⟨ Globals in the outer block 9 ⟩ +≡ collate: array [0 . . 100] of ASCII code; { collation order }

247. ⟨ Local variables for initialization 16 ⟩ +≡ c: ASCII code; { used to initialize collate } 248. We use the order null < ␣ < other characters < < = < ⋯ < = < < ⋯ < . ⟨ Set initial values 10 ⟩ +≡

collate[0] ← 0; collate[1] ← ␣ ; for c ← 1 to ␣ − 1 do collate[c + 1] ← c; for c ← ␣ + 1 to − 1 do collate[c] ← c;

for c ←  + 1 to  − 1 do collate[c − 10] ← c;
for c ←  + 1 to  − 1 do collate[c − 36] ← c;

collate[ − 36] ← + 1; for c ← + 1 to 126 do collate[c − 63] ← c; collate[64] ← ;

for c ←  to  do collate[c −  + 65] ← c;
for c ←  to  do collate[c −  + 91] ← c;

249. Procedure unbucket goes through the buckets and adds nonempty lists to the stack, using the collating sequence specified in the collate array. The parameter to unbucket tells the current depth in the buckets. Any two sequences that agree in their first 255 character positions are regarded as identical.

define infinity = 255 { ∞ (approximately) } procedure unbucket (d : eight bits ); { empties buckets having depth d }

var c: ASCII code; { index into bucket } begin for c ← 100 downto 0 do

if bucket [collate[c]] > 0 then begin if sort ptr > max sorts then overflow (´ ´ ); incr (sort ptr ); stat if sort ptr > max sort ptr then max sort ptr ← sort ptr ; tats if c = 0 then depth[sort ptr ] ← infinity else depth[sort ptr ] ← d; head [sort ptr ] ← bucket [collate[c]]; bucket [collate[c]] ← 0; end;

end;

250. ⟨ Sort and output the index 250 ⟩ ≡ sort ptr ← 0; unbucket (1); while sort ptr > 0 do

begin cur depth ← cat [sort ptr ];

if (blink [head [sort ptr ]] = 0) ∨ (cur depth = infinity ) then

⟨ Output index entries for the list at sort ptr 252 ⟩ else ⟨ Split the list at sort ptr into further lists 251 ⟩; end

This code is used in section 239.

251. ⟨ Split the list at sort ptr into further lists 251 ⟩ ≡ begin next name ← head [sort ptr ]; repeat cur name ← next name; next name ← blink [cur name];

cur byte ← byte start [cur name] + cur depth; cur bank ← cur name mod ww ; if cur byte = byte start [cur name + ww ] then c ← 0 { we hit the end of the name } else begin c ← byte mem[cur bank , cur byte];

if (c ≤ ) ∧ (c ≥ ) then c ← c + ´ 40 ; end;

blink [cur name] ← bucket [c]; bucket [c] ← cur name; until next name = 0; decr (sort ptr ); unbucket (cur depth + 1); end

This code is used in section 250.

252. ⟨ Output index entries for the list at sort ptr 252 ⟩ ≡ begin cur name ← head [sort ptr ]; debug if trouble shooting then debug help; gubed repeat out2 ( )( ); ⟨ Output the name at cur name 253 ⟩;

⟨ Output the cross-references at cur name 254 ⟩;

cur name ← blink [cur name]; until cur name = 0; decr (sort ptr ); end

This code is used in section 250.

253. ⟨ Output the name at cur name 253 ⟩ ≡ case ilk [cur name] of normal : if length(cur name) = 1 then out2 ( )( ) else out2 ( )( ); roman: do nothing ; wildcard : out2 ( )( ); typewriter : out2 ( )( ); othercases out2 ( )( ) endcases; out name(cur name)

This code is used in section 252.

254. Section numbers that are to be underlined are enclosed in ‘ … ’. ⟨ Output the cross-references at cur name 254 ⟩ ≡

⟨ Invert the cross-reference list at cur name, making cur xref the head 255 ⟩; repeat out2 ( )( ␣ ); cur val ← num(cur xref );

if cur val < def flag then out mod (cur val )

else begin out2 ( )( ); out mod (cur val − def flag ); out ( );

end; cur xref ← xlink (cur xref );

until cur xref = 0; out ( ); finish line

This code is used in section 252.

255. List inversion is best thought of as popping elements off one stack and pushing them onto another. In this case cur xref will be the head of the stack that we push things onto. ⟨ Invert the cross-reference list at cur name, making cur xref the head 255 ⟩ ≡

this xref ← xref [cur name]; cur xref ← 0; repeat next xref ← xlink (this xref ); xlink (this xref ) ← cur xref ; cur xref ← this xref ;

this xref ← next xref ; until this xref = 0

This code is used in section 254.

256. The following recursive procedure walks through the tree of module names and prints them. procedure mod print (p : name pointer ); { print all module names in subtree p }

begin if p > 0 then begin mod print (llink [p]); out2 ( )( ); tok ptr ← 1; text ptr ← 1; scrap ptr ← 0; init stack ; app(p + mod flag ); make output ; footnote(0);

{ cur xref was set by make output } finish line; mod print (rlink [p]); end;

end;

257. ⟨ Output all the module names 257 ⟩ ≡ mod print (root )

This code is used in section 239.

96  DEBUGGING  MWEAVE §258

258. Debugging. The Pascal debugger with which was developed allows breakpoints to be set, and variables can be read and changed, but procedures cannot be executed. Therefore a ‘debug help’ procedure has been inserted in the main loops of each phase of the program; when ddt and dd are set to appropriate values, symbolic printouts of various tables will appear.

The idea is to set a breakpoint inside the debug help routine, at the place of ‘breakpoint :’ below. Then when debug help is to be activated, set trouble shooting equal to true. The debug help routine will prompt you for values of ddt and dd , discontinuing this when ddt ≤ 0; thus you type 2n + 1 integers, ending with zero or a negative number. Then control either passes to the breakpoint, allowing you to look at and/or change variables (if you typed zero), or to exit the routine (if you typed a negative value).

Another global variable, debug cycle, can be used to skip silently past calls on debug help. If you set debug cycle > 1, the program stops only every debug cycle times debug help is called; however, any error stop will set debug cycle to zero. ⟨ Globals in the outer block 9 ⟩ +≡

debug trouble shooting : boolean; { is debug help wanted? } ddt : integer ; { operation code for the debug help routine } dd : integer ; { operand in procedures performed by debug help } debug cycle: integer ; { threshold for debug help stopping } debug skipped : integer ; { we have skipped this many debug help calls } term in: text file; { the user’s terminal as an input file }

gubed

259. The debugging routine needs to read from the user’s terminal. ⟨ Set initial values 10 ⟩ +≡

debug trouble shooting ← true; debug cycle ← 1; debug skipped ← 0; tracing ← 0;

trouble shooting ← false; debug cycle ← 99999; { use these when it almost works }

reset (term in, ´ ´ , ´ ´ ); { open term in as the terminal, don’t do a get }

gubed

§260  MWEAVE  DEBUGGING 97

260. define breakpoint = 888 { place where a breakpoint is desirable } debug procedure debug help; { routine to display various things } label breakpoint , exit ; var k: integer ; { index into various arrays } begin incr (debug skipped ); if debug skipped < debug cycle then return; debug skipped ← 0; loop begin write(term out , ´ #´ ); update terminal ; { prompt }

read (term in, ddt ); { read a list of integers } if ddt < 0 then return else if ddt = 0 then

begin goto breakpoint ; { go to every label at least once } breakpoint : ddt ← 0;

end else begin read (term in, dd );

case ddt of

1: print id (dd ); 2: print text (dd ); 3: for k ← 1 to dd do print (xchr [buffer [k]]); 4: for k ← 1 to dd do print (xchr [mod text [k]]); 5: for k ← 1 to out ptr do print (xchr [out buf [k]]); 6: for k ← 1 to dd do

begin print cat (cat [k]); print (´ ␣´ ); end;

othercases print (´ ´ ) endcases; end;

end; exit : end;

gubed

98  THE MAIN PROGRAM  MWEAVE §261

261. The main program. Let’s put it all together now: starts and ends here.

The main procedure has been split into three sub-procedures in order to keep certain Pascal compilers from overflowing their capacity. procedure Phase I ;

begin ⟨ Phase I: Read all the user’s text and store the cross references 109 ⟩; end;

procedure Phase II ; begin ⟨ Phase II: Read all the text again and translate it to TEX form 218 ⟩; end; begin initialize; { beginning of the main program } print ln(banner ); { print a “banner line” } ⟨ Store all the reserved words 64* ⟩; Phase I ; Phase II ; ⟨ Phase III: Output the cross-reference index 239 ⟩; ⟨ Check that all changes have been read 85 ⟩;

end of WEAVE : stat ⟨ Print statistics about memory usage 262 ⟩; tats { here files should be closed if the operating system requires it }

⟨ Print the job history 263 ⟩; end.

262. ⟨ Print statistics about memory usage 262 ⟩ ≡

print nl (´ ␣ ␣ ␣´ , name ptr : 1, ´ ␣ ␣´ , xref ptr : 1,

´ ␣ ␣ ␣´ , byte ptr [0] : 1); for cur bank ← 1 to ww − 1 do print (´ ´ , byte ptr [cur bank ] : 1); print (´ ␣ ´ ); print nl (´ ␣ ␣´ , max scr ptr : 1, ´ ␣ ␣´ , max txt ptr : 1,

´ ␣ ␣´ , max tok ptr : 1, ´ ␣ ␣´ , max stack ptr : 1, ´ ␣ ´ ); print nl (´ ␣ ␣´ , max sort ptr : 1, ´ ␣ ´ )

This code is used in section 261.

263. Some implementations may wish to pass the history value to the operating system so that it can be used to govern whether or not other programs are started. Here we simply report the history to the user. ⟨ Print the job history 263 ⟩ ≡

case history of

spotless : print nl (´ ␣ ␣ ␣ ´ ); harmless message: print nl (´ ␣ ␣ ␣ ␣ ␣ ␣ ´ ); error message: print nl (´ ␣ ␣ ␣ ␣ ␣ ␣ ␣ ␣ ´ ); fatal message: print nl (´ ␣ ␣ ␣ ␣ ␣ ␣ ´ ); end { there are no other cases }

This code is used in section 261.

§264  MWEAVE  SYSTEM-DEPENDENT CHANGES 99

264. System-dependent changes. This module should be replaced, if necessary, by changes to the program that are necessary to make work at a particular installation. It is usually best to design your change file so that all changes to previous modules preserve the module numbering; then everybody’s version will be consistent with the printed program. More extensive changes, which introduce new modules, can be inserted here; then only the index itself will get a new module number.

100  INDEX  MWEAVE §265

265* . Index. If you have read and understood the code for Phase III above, you know what is in this index and how it got here. All modules in which an identifier is used are listed with that identifier, except that reserved words are indexed only when they appear in format definitions, and the appearances of identifiers in module names are not indexed. Underlined entries correspond to where the identifier was declared. Error messages, control sequences put into the output, and a few other things like “recursion” are indexed here too.

The following sections were changed by the change file: 1, 42, 63, 64, 86, 92, 97, 111, 112, 124, 136, 137, 138, 183, 186, 188, 191, 192, 193, 198, 207, 222, 227, 229, 230, 265.

: 186* . : 130. : 162, 163, 186* . : 189, 253. : 252, 256. : 189. : 254. ␣ : 186* , 189, 193* . # : 186* , 189. $ : 186* , 189. % : 186* , 189. : 189, 209, 253. ´ : 189. : 189, 209, 253. : 186* , 189. ` : 189. : 189. : 189. : 189, 193* . : 186* . : 209, 253. : 131, 189. : 236. : 186* . : 186* . : 198* . : 239. : 227* . : 186* . : 228. : 238. : 239. : 188* . : 196, 224. : 188* . : 186* . : 239. : 186* . : 188* . : 188* . : 221. : 221. : 196, 223. : 212, 226.

: 188* . : 188* , 228, 231. : 186* . : 186* . : 236. : 188* . : 188* . : 213. : 212, 219, 226, 231. : 211, 212. : 211, 212. : 211, 212. : 211, 212. : 211, 212. : 211, 212, 226. : 211, 212, 226. : 253. : 88, 177. : 88, 177. a: \underline{130}, \underline{206}, \underline{208}. alpha: \underline{140}, 142, 143, 149, 192* , 193* . alpha cases : 149, \underline{150}. : 69. and sign: \underline{15}, 64* , 188* . app: \underline{148}, 151, 152, 153, 155, 156, 158, 159,

160, 161, 162, 163, 164, 165, 167, 168, 170, 180, 184, 186* , 189, 190, 195, 196, 198* , 207* , 208, 226, 231, 256.

app comment : 183* , 184, \underline{195}, 197, 198* , 226. app hex : 183* , 186* , \underline{196}. app octal : 183* , 186* , \underline{196}. app tok : \underline{136*} , 137* , 138* , 148, 189, 190, 196,

197, 226. append xref : \underline{50}, 51. app1 : \underline{148}, 151, 152, 153, 155, 156, 159, 160, 162,

163, 164, 165, 168, 170, 174, 180, 195. app2 : \underline{148}, 153, 156, 158, 161, 163, 165, 167. app3 : \underline{148}, 158, 164. array like: \underline{42*} , 64* , 191* , 192* . ASCII code: 11, 86* . ASCII code: \underline{11}, 13, 27, 28, 37, 65, 73, 87, 89, 91,

§265  MWEAVE  INDEX 101

121, 127, 132, 136* , 208, 242, 246, 247, 249. b: \underline{122}, \underline{208}. backup: \underline{141}, 142, 143, 147, 160, 167, 192* , 193* ,

208, 231. bal : \underline{91}, 92* , \underline{112*} , \underline{136*} , 137* , 138* , \underline{198*} . banner : \underline{1*} , 261. begin: 3. begin comment : \underline{86*} , 87, 186* , 222* . begin like: \underline{42*} , 64* , 193* . begin Pascal : \underline{86*} , 87, 117, 229* , 230* . begin regular comment : \underline{86*} , 92* , 97* , 111* , 112* ,

137* , 183* , 198* . beginning : \underline{140}, 142, 143, 150, 152, 164, 170, 193* . big cancel : \underline{141}, 142, 147, 186* , 208, 212. big force: \underline{141}, 142, 147, 186* , 208, 212, 226. big line break : \underline{86*} , 87, 186* , 222* . blink : \underline{242}, 243, 250, 251, 252. boolean: 28, 29, 45, 71, 74, 93, 122, 143, 229* , 258. break : 22. break out : 125, 126, \underline{127}. break space: \underline{141}, 143, 147, 152, 155, 156, 160,

164, 168, 170, 193* , 200, 208, 211, 212. breakpoint : 258, \underline{260}. bucket : \underline{242}, 243, 249, 251. buf size: \underline{8}, 28, 73, 74, 75, 79, 123. buffer : \underline{27}, 28, 31, 32, 55, 58, 59, 61, 62, 63* , 74, 76,

78, 79, 80, 81, 82, 83, 84, 85, 89, 90, 91, 92* , 93, 95, 97* , 98, 99, 100, 103, 104, 106, 107, 110, 123, 132, 133, 134, 135, 136* , 137* , 179, 182, 183* , 189, 190, 196, 208, 214, 216, 217, 221, 223, 224, 260.

byte mem: 36, \underline{37}, 38, 39, 40, 43, 44, 52, 58, 61, 62, 66, 67, 68, 69, 131, 208, 209, 214, 215, 216, 217, 243, 244, 251.

byte ptr : 38, \underline{39}, 41, 62, 67, 262. byte start : 36, \underline{37}, 38, 39, 41, 44, 50, 55, 61, 62,

67, 68, 93, 114, 131, 209, 214, 243, 251. c: \underline{66}, \underline{69}, \underline{87}, \underline{89}, \underline{90}, \underline{91}, \underline{95}, \underline{127}, \underline{132}, \underline{134}, \underline{136*} ,

\underline{140}, \underline{172}, \underline{174}, \underline{242}, \underline{247}, \underline{249}. cancel : \underline{141}, 142, 143, 147, 153, 155, 156, 159, 160,

162, 163, 164, 165, 193* , 197, 200, 208, 211, 212. carriage return: \underline{15}, 17, 28. case head : \underline{140}, 143, 149, 153, 154, 162, 163, 165. case like: \underline{42*} , 64* , 193* . casey : \underline{140}, 142, 143, 149, 153, 165, 193* . cat : \underline{144}, 149, 150, 151, 152, 153, 154, 155, 156,

158, 159, 160, 161, 162, 163, 164, 165, 167, 168, 170, 172, 174, 176, 178, 179, 180, 181, 183* , 184, 194, 195, 197, 244, 250, 260.

: 77, 79, 84. : 85. change buffer : \underline{73}, 74, 75, 78, 79, 83, 85. change changing : \underline{72}, 79, 81, 84. change exists : \underline{45}, 109, 110, 239. change file: \underline{2}, \underline{23}, 24, 32, 71, 73, 76, 77, 79, 84. change limit : \underline{73}, 74, 75, 78, 79, 83, 85. changed module: \underline{45}, 82, 109, 110, 130, 241. changing : 32, \underline{71}, 72, 73, 75, 79, 81, 82, 85. char : 12, 14. char like: \underline{42*} , 64* , 191* . check change: \underline{79}, 83. check sum: \underline{86*} , 87, 186* , 222* . chr : 12, 13, 17, 18. clause: \underline{140}, 142, 143, 149, 151, 153, 154, 156, 193* . close: \underline{140}, 142, 143, 152, 153, 160, 162, 163,

164, 167, 186* , 193* . collate: \underline{246}, 247, 248, 249. colon: \underline{140}, 142, 143, 148, 151, 160, 163, 167,

170, 186* . comment : \underline{142}. comment scrap: \underline{184}, 186* . compress : \underline{97*} . cond : \underline{140}, 142, 143, 149. confusion: \underline{34}. const like: \underline{42*} , 64* , 191* , 192* . continue: \underline{5}, 75, 76. : 106. : 106. control code: \underline{87}, 88, 90, 93, 100, 135. copy comment : 132, \underline{136*} , 198* . copy limbo: \underline{132}, 218. copy TeX : 132, \underline{134}, 222* . count : \underline{69}. cur bank : \underline{244}, 251, 262. cur byte: \underline{244}, 251. cur depth: \underline{244}, 250, 251. cur end : 201, \underline{202}, 204, 205, 206. cur mod name: \underline{208}, 214, 215, 216. cur mode: 201, \underline{202}, 204, 206, 208, 211, 212. cur module: \underline{93}, 101, 117, 230* , 232. cur name: \underline{63*} , 206, 209, 213, 214, 242, 243,

251, 252, 253, 255. cur state: \underline{202}, 204, 205. cur tok : 201, \underline{202}, 204, 205, 206. cur val : \underline{244}, 254. cur xref : \underline{118}, 119, 213, 231, 234, 235, 236, 237,

254, 255, 256. d: \underline{95}, \underline{127}, \underline{172}, \underline{174}, \underline{249}. dd : \underline{258}, 260. ddt : \underline{258}, 260. debug: \underline{3}, \underline{4}, \underline{30}, \underline{31}, \underline{88}, \underline{95}, \underline{140}, \underline{146}, \underline{148}, \underline{177},

\underline{178}, \underline{181}, \underline{182}, \underline{206}, \underline{252}, \underline{258}, \underline{259}, \underline{260}. debug cycle: 31, \underline{258}, 259, 260. debug help: \underline{30}, 31, 95, 206, 252, 258, \underline{260}. debug skipped : 31, \underline{258}, 259, 260. decr : \underline{6}, 28, 92* , 98, 103, 122, 127, 130, 135, 137* ,

138* , 205, 251, 252. def flag : \underline{46}, 48, 50, 51, 93, 100, 111* , 113, 115,

117, 119, 130, 213, 231, 233, 235, 236, 254. definition: \underline{86*} , 87, 115, 225. depth: \underline{244}, 249. dig : \underline{129}, 130. div like: \underline{42*} , 64* , 191* , 192* . do like: \underline{42*} , 64* , 193* . do nothing : \underline{6}, 95, 113, 149, 150, 186* , 189,

222* , 253. done: \underline{5}, 75, 76, 90, 91, 92* , 95, 103, 104, 122, 134,

135, 136* , 137* , 138* , 175, 179, 236, 237. : 133. : 189. double dot : \underline{86*} , 97* , 186* . easy cases : 183* , 185, \underline{186*} . eight bits : \underline{36}, 58, 87, 90, 91, 95, 108, 112* , 122,

134, 136* , 140, 144, 172, 174, 178, 198* , 206, 208, 244, 249.

else: 7. else like: \underline{42*} , 64* , 193* . elsie: \underline{140}, 142, 143, 149, 156. elsif like: \underline{42*} , 64* , 193* . emit space if needed : \underline{219}, 225, 230* . end: 3, 7. end comment : \underline{86*} , 87, 186* , 222* . end field : \underline{201}, 202. end like: \underline{42*} , 64* , 193* . end of WEAVE : \underline{2}, 33, 261. end regular comment : \underline{86*} , 92* , 97* , 136* , 137* , 138* . end translation: \underline{141}, 147, 201, 207* , 208, 212. endcases: \underline{7}. eof : 28. eoln: 28. equal : \underline{66}, 67, 68. equivalence sign: \underline{15}, 97* , 116, 188* , 228, 231. err print : \underline{31}, 66, 69, 72, 76, 77, 79, 80, 84, 85,

87, 99, 103, 104, 106, 107, 133, 136* , 137* , 189, 197, 222* , 227* , 228, 231, 232.

error : 28, \underline{31}, 33. error message: \underline{9}, 263. exit : \underline{5}, 6, 50, 74, 75, 79, 89, 111* , 123, 127, 132,

183* , 208, 236, 260. exp: \underline{140}, 142, 143, 149, 186* . exponent : \underline{93}, 98, 186* . extension: \underline{66}, 68, 69. : 87. f: \underline{28}. false: 28, 29, 72, 73, 74, 79, 81, 94, 96, 109, 110,

122, 123, 127, 188* , 218, 222* , 227* , 230* , 238, 259. fatal error : \underline{33}, 34, 35. fatal message: \underline{9}, 263. final limit : \underline{28}.

finish line: \underline{123}, 124* , 132, 134, 135, 212, 218, 226,

236, 238, 239, 254, 256. finish Pascal : 225, \underline{226}, 230* .

first text char : \underline{12}, 18.
first xref : \underline{234}, 235.
five cases : 149, \underline{150}.
flag : \underline{236}, 237.
flush buffer : \underline{122}, 123, 127, 128, 218, 238.
footnote: 233, \underline{236}, 256.
for like: \underline{42*} , 64* , 191* , 192* .

force: \underline{141}, 142, 143, 146, 147, 153, 155, 156, 160, 161, 167, 186* , 192* , 193* , 198* , 200, 208, 212, 226, 231. force line: \underline{86*} , 87, 186* .

form feed : \underline{15}, 28.

format : \underline{86*} , 87, 111* , 112* , 113, 115, 183* , 198* ,

222* , 225. forward : 30, 207* . found : \underline{5}, 58, 60, 61, 66, 95, 96, 148, 149, 150,

151, 175, 179, 208, 216. freeze text : \underline{171}, 172, 180, 184, 195, 198* , 208. get : 28, 259. get line: 71, \underline{82}, 89, 90, 91, 95, 103, 123, 132,

134, 136* . get next : 93, \underline{95}, 108, 111* , 113, 115, 116, 117, 183* ,

222* , 227* , 228, 230* , 231, 232. get output : \underline{206}, 207* , 208, 211, 212. goto like: \underline{42*} , 64* , 191* , 192* . greater : \underline{66}, 68, 69. greater or equal : \underline{15}, 97* , 188* . gubed: \underline{3}. h: \underline{56}, \underline{58}, \underline{242}. harmless message: \underline{9}, 263. hash: 38, \underline{55}, 57, 60, 242, 243. hash size: \underline{8}, 55, 56, 57, 58, 59, 242, 243. head : \underline{244}, 249, 250, 251, 252. hex : \underline{86*} , 87, 100, 186* , 222* . hi ptr : \underline{144}, 176, 178, 179. history : \underline{9}, 10, 263. : 80. i: \underline{16}, \underline{58}, \underline{172}, \underline{174}, \underline{179}. id first : \underline{55}, 58, 59, 61, 62, 63* , 93, 98, 99, 106,

107, 189, 190. id flag : \underline{146}, 192* , 206, 227* , 228. id loc: \underline{55}, 58, 59, 61, 62, 64* , 93, 98, 99, 106,

107, 189, 190. id lookup: 55, \underline{58}, 63* , 93, 111* , 113, 116, 191* ,

227* , 228. identifier : \underline{93}, 98, 111* , 113, 116, 185, 206, 208,

209, 227* , 228. id10 : \underline{63*} , 64* .

id11 : \underline{63*} .

id12 : \underline{63*} . id13 : \underline{63*} . id14 : \underline{63*} , 64* . id2 : \underline{63*} , 64* . id3 : \underline{63*} , 64* . id4 : \underline{63*} , 64* . id5 : \underline{63*} , 64* . id6 : \underline{63*} , 64* . id7 : \underline{63*} , 64* . id8 : \underline{63*} . id9 : \underline{63*} , 64* . if like: \underline{42*} . ignore: \underline{86*} , 87, 88, 186* , 198* , 207* , 222* . ilk : 36, \underline{37}, 42* , 43, 55, 58, 60, 62, 111* , 116,

191* , 192* , 253. : 215. : 137* . : 228. : 227* . in def part : 186* , 188* , 222* , 227* , 229* , 230* . in like: \underline{42*} . : 66. incr : \underline{6}, 28, 50, 59, 61, 62, 67, 68, 69, 76, 77,

79, 83, 84, 89, 90, 91, 92* , 95, 97* , 98, 99, 100, 103, 104, 106, 107, 110, 122, 125, 130, 133, 135, 136* , 137* , 148, 149, 150, 171, 176, 184, 189, 196, 204, 206, 214, 215, 216, 217, 220, 223, 224, 241, 249, 260.

indent : \underline{141}, 142, 143, 147, 151, 160, 165, 193* , 208. infinity : \underline{249}, 250. init stack : \underline{202}, 222* , 225, 230* , 256. initialize: \underline{2}, 261. inner : 200, \underline{201}, 206, 212. inner tok flag : \underline{146}, 198* , 206, 207* . : 136* . : 103. : 28. input has ended : \underline{71}, 79, 81, 83, 89, 90, 91, 95,

103, 109, 132, 134, 136* , 218. input ln: \underline{28}, 76, 77, 79, 83, 84. integer : 14, 71, 79, 121, 130, 172, 174, 219,

258, 260. intro: \underline{140}, 142, 143, 148, 150, 157, 160, 162, 163,

165, 167, 170, 192* , 193* , 227* , 228. j: \underline{66}, \underline{69}, \underline{95}, \underline{122}, \underline{146}, \underline{172}, \underline{174}, \underline{179}, \underline{183*} , \underline{208}. join: \underline{86*} , 87, 186* , 222* . jump out : 2, \underline{33}. k: \underline{31}, \underline{44}, \underline{58}, \underline{66}, \underline{69}, \underline{74}, \underline{75}, \underline{79}, \underline{95}, \underline{122}, \underline{123}, \underline{127},

\underline{130}, \underline{131}, \underline{172}, \underline{174}, \underline{178}, \underline{179}, \underline{208}, \underline{260}. k limit : \underline{208}, 214, 216. k module: \underline{240}, 241. l: \underline{31}, \underline{58}, \underline{66}, \underline{69}.

last text char : \underline{12}, 16, 18.
lbrace: \underline{146}, 147.
left arrow : \underline{15}, 97* , 188* .
length: \underline{38}, 60, 209, 253.
less : \underline{66}, 67, 68, 69.
less or equal : \underline{15}, 97* , 188* .
lhs : \underline{114}, 116.
limit : 28, 32, \underline{71}, 74, 76, 77, 78, 80, 81, 82, 83, 84,

85, 89, 90, 91, 95, 97* , 99, 103, 106, 107, 123, 132, 133, 134, 135, 136* , 208, 214, 216, 223. line: 32, \underline{71}, 72, 76, 77, 79, 81, 83, 84, 85, 182. : 128. line break : \underline{86*} , 87, 186* , 222* . line feed : \underline{15}, 28. line length: \underline{8}, 121, 122, 125, 127. lines dont match: \underline{74}, 79. link : 36, \underline{37}, 38, 43, 60, 243. llink : \underline{43}, 66, 67, 69, 119, 256. lo ptr : \underline{144}, 172, 175, 176, 178, 179, 180, 181. loc: 28, 32, \underline{71}, 76, 80, 81, 82, 84, 85, 89, 90, 91,

92* , 95, 97* , 98, 99, 100, 103, 104, 106, 107, 110, 113, 132, 133, 134, 135, 136* , 137* , 182, 196, 208, 214, 221, 222* , 223, 224.

long buf size: \underline{8}, 27, 28, 31, 55, 58, 71, 179,

183* , 208, 216, 217. longest name: \underline{8}, 65, 66, 69, 95, 103, 105.

loop: \underline{6}.
loop like: \underline{42*} , 64* , 193* .
m: \underline{50}, \underline{130}.
make output : \underline{207*} , \underline{208}, 213, 226, 256.
mark error : \underline{9}, 31, 215, 216.
mark fatal : \underline{9}, 33.
mark harmless : \underline{9}, 105, 119, 128, 181, 182.

math: 139, \underline{140}, 142, 143, 148, 150, 151, 158, 160, 162, 163, 167, 170, 179, 180, 186* , 188* , 192* , 227* , 228, 231.

math bin: \underline{141}, 142, 147, 192* , 208, 210.

math break : \underline{86*} , 87, 186* , 222* .
math op: \underline{141}, 143, 147, 162, 163, 208.

math rel : \underline{141}, 142, 146, 147, 192* , 208, 210, 231.

max bytes : \underline{8}, 37, 39, 44, 58, 62, 66, 67, 69,

131, 208, 244. max modules : \underline{8}, 45, 46, 110, 240.

max names : \underline{8}, 37, 38, 62, 67, 69, 242.
max refs : \underline{8}, 47, 50.

max scr ptr : \underline{144}, 145, 187, 197, 199, 226, 262.

max scraps : \underline{8}, 144, 172, 174, 178, 179, 187,

197, 199, 244. max sort ptr : \underline{244}, 245, 249, 262.

max sorts : \underline{244}, 249.
max stack ptr : \underline{202}, 203, 204, 262.
max texts : \underline{8}, 52, 175, 179, 187, 199.

max tok ptr : \underline{53}, 54, 175, 187, 199, 207* , 226, 262. max toks : \underline{8}, 53, 136* , 146, 175, 179, 180, 187,

189, 198* , 199. max txt ptr : \underline{53}, 54, 175, 187, 199, 207* , 226, 262. mid xref : \underline{234}, 235. : 197. mod check : \underline{119}, 120. mod flag : \underline{146}, 206, 231, 232, 256. mod lookup: 65, \underline{66}, 101, 102. mod name: \underline{206}, 208. mod print : \underline{256}, 257. mod scrap: \underline{140}, 142, 143, 149, 167, 231, 232. mod text : \underline{65}, 66, 67, 68, 69, 95, 101, 102, 103,

104, 105, 260. mod xref switch: 46, \underline{48}, 49, 51, 117. mode: \underline{201}, 208. mode field : \underline{201}, 202. module count : \underline{45}, 50, 51, 82, 109, 110, 181, 218,

220, 221, 231, 241. module name: \underline{86*} , 87, 93, 100, 113, 117, 222* ,

230* , 232. n: \underline{50}, \underline{79}, \underline{178}. : 69. name pointer : \underline{38}, 39, 44, 50, 51, 58, 63* , 66,

69, 93, 111* , 114, 119, 131, 144, 183* , 192* , 208, 229* , 242, 256.

name ptr : 38, \underline{39}, 41, 44, 58, 60, 62, 67, 262. : 119. : 119. new line: \underline{20}, 31, 32, 33, 128. new mod xref : \underline{51}, 117. new module: \underline{86*} , 87, 90, 95, 134. new xref : \underline{50}, 111* , 113, 116. next control : \underline{108}, 111* , 112* , 113, 115, 116, 117,

183* , 185, 186* , 189, 191* , 197, 198* , 207* , 222* , 225, 227* , 228, 229* , 230* , 231, 232.

next name: \underline{242}, 243, 251. next xref : \underline{234}, 235, 255. nil: 6. nil like: \underline{42*} , 64* , 191* , 192* . no line break : \underline{86*} , 87, 186* , 222* . no underline: \underline{86*} , 87, 100, 113. normal : \underline{42*} , 58, 60, 111* , 116, 191* , 192* , 227* ,

228, 253. not equal : \underline{15}, 97* , 188* . not found : \underline{5}. not sign: \underline{15}, 64* , 188* . num: \underline{46}, 49, 50, 51, 119, 213, 231, 235, 236,

237, 254. num field : 46, \underline{48}. octal : \underline{86*} , 87, 186* , 222* . of like: \underline{42*} , 64* , 191* , 192* . omega: \underline{140}, 142, 143, 151, 192* , 193* , 195. oot : \underline{125}. oot1 : \underline{125}. oot2 : \underline{125}. oot3 : \underline{125}. oot4 : \underline{125}. oot5 : \underline{125}. open: 139, \underline{140}, 142, \underline{143}, 150, 186* . open input : \underline{24}, 81. opt : 139, \underline{141}, 142, 143, 147, 159, 163, 186* ,

208, 211. or sign: \underline{15}, 64* , 188* . ord : 13. other line: \underline{71}, 72, 81, 85. othercases: \underline{7}. others : 7. out : \underline{125}, 130, 131, 133, 135, 208, 209, 210,

211, 212, 213, 214, 221, 222* , 223, 224, 236, 237, 241, 254.

out buf : \underline{121}, 122, 124* , 125, 126, 127, 128, 212,

226, 231, 260. out line: \underline{121}, 122, 124* , 128, 219. out mod : \underline{130}, 213, 221, 237, 241, 254. out name: \underline{131}, 209, 253. out ptr : \underline{121}, 122, 123, 124* , 125, 127, 128, 135,

212, 219, 226, 231, 260. outdent : \underline{141}, 143, 147, 153, 155, 156, 160, 162,

163, 164, 208. outer : 200, \underline{201}, 202, 211, 212. outer parse: \underline{198*} , 225, 230* . outer xref : \underline{112*} , 115, 117, 198* . output Pascal : \underline{207*} , 214, 222* . output state: \underline{201}, 202. out2 : \underline{125}, 130, 210, 211, 212, 213, 219, 221, 226,

241, 252, 253, 254, 256. out3 : \underline{125}, 210, 223, 224, 238. out4 : \underline{125}, 226, 236, 237, 239, 241. out5 : \underline{125}, 210. overflow : \underline{35}, 50, 62, 67, 110, 136* , 175, 180, 187,

199, 204, 216, 217, 249. p: \underline{44}, \underline{50}, \underline{51}, \underline{58}, \underline{66}, \underline{69}, \underline{111*} , \underline{119}, \underline{131}, \underline{146}, \underline{183*} ,

\underline{192*} , \underline{197}, \underline{198*} , \underline{204}, \underline{226}. param: \underline{86*} . : 216. Pascal parse: \underline{183*} , 186* , 192* , 196, 197, 198* . Pascal translate: \underline{197}, 198* , 207* . Pascal xref : \underline{111*} , 112* , 113, 183* , 198* . per cent : \underline{122}. Phase I : \underline{261}. Phase II : \underline{261}. phase one: \underline{29}, 31, 109. phase three: \underline{29}, 109, 213, 239. pop level : \underline{205}, 206. pound found : 186* , 188* , 227* , 229* . pp: \underline{144}, 148, 149, 150, 151, 152, 153, 154, 155,

156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 173, 175, 176, 178, 179.

prefix : \underline{66}, 68. prefix lookup: \underline{69}, 101. prime the change buffer : \underline{75}, 81, 84. print : \underline{20}, 31, 32, 33, 44, 105, 110, 119, 128,

140, 146, 147, 178, 181, 182, 215, 216, 221, 239, 260, 262.

print cat : \underline{140}, 178, 181, 260. print id : \underline{44}, 119, 146, 215, 216, 260. print ln: \underline{20}, 32, 128, 181, 261. print nl : \underline{20}, 28, 105, 119, 128, 178, 181, 182,

215, 216, 218, 239, 262, 263. print text : \underline{146}, 260. proc: \underline{140}, 142, \underline{143}, 149, 162, 163, 164, 193* . proc like: \underline{42*} , 64* , 111* , 193* . prod : 148, \underline{178}, 183* . production: \underline{148}. production end : \underline{148}. productions, table of: 143. pseudo semi : \underline{86*} , 87, 186* , 222* . push level : \underline{204}, 206, 208. q: \underline{50}, \underline{51}, \underline{66}, \underline{69}, \underline{198*} , \underline{236}. r: \underline{51}, \underline{69}, \underline{146}. rbrace: \underline{146}. read : 260. read ln: 28. record head : \underline{140}, 142, 143, 149. record like: \underline{42*} , 64* , 193* . recursion: 119, 207* , 256. red : 148, \underline{172}, 174. reduce: \underline{148}, 151, 152, 153, 155, 156, 158, 159, 160,

161, 162, 163, 164, 165, 167, 168, 170, 172, 178. repeat like: \underline{42*} , 64* , 193* . res flag : \underline{146}, 192* , 193* , 206. res word : \underline{206}, 208, 209. reserved : \underline{42*} , 50, 60. reset : 24, 259. reset input : \underline{81}, 109, 218. restart : \underline{5}, 82, 95, 100, 206. reswitch: \underline{5}, 183* , 185, 191* , 208, 212. return: 5, \underline{6}. rewrite: 21, 26. rhs : \underline{114}, 116. rlink : \underline{43}, 66, 67, 69, 119, 256. roman: \underline{42*} , 111* , 253. root : \underline{43}, 66, 69, 120, 257. save base: \underline{197}. save limit : \underline{208}, 214.

save line: \underline{219}.
save loc: \underline{208}, 214.
save mode: \underline{208}, 212.
save next control : \underline{207*} .
save place: \underline{219}.
save position: \underline{219}, 220, 225.
save text ptr : \underline{207*} .
save tok ptr : \underline{207*} .
scanning hex : \underline{93}, 94, 95, 96, 100.
scrap base: \underline{144}, 145, 173, 178, 179, 180, 181,

194, 195, 197. scrap ptr : \underline{144}, 145, 176, 178, 179, 183* , 184, 187, 194, 195, 197, 199, 226, 228, 244, 256.

sc0 : \underline{184}, 186* , 193* , 194, 195, 228.

sc1 : \underline{184}, 186* , 189, 190, 192* , 193* , 196, 227* ,

228, 231, 232. sc2 : \underline{184}, 186* , 188* , 192* , 193* , 227* , 228, 231. sc3 : \underline{184}, 186* , 192* , 193* , 231. sc4 : \underline{184}, 186* , 193* . : 104. : 105. semi : 139, \underline{140}, 142, 143, 149, 161, 163, 186* ,

194, 195, 228, 231. set element sign: \underline{15}, 64* , 186* .

sid1 : \underline{63*} .
sid10 : \underline{63*} .
sid11 : \underline{63*} .
sid12 : \underline{63*} .
sid13 : \underline{63*} .
sid14 : \underline{63*} .
sid2 : \underline{63*} .
sid3 : \underline{63*} .
sid4 : \underline{63*} .
sid5 : \underline{63*} .
sid6 : \underline{63*} .
sid7 : \underline{63*} .
sid8 : \underline{63*} .
sid9 : \underline{63*} .

simp: \underline{140}, 142, 143, 148, 150, 151, 158, 160, 161, 162, 167, 170, 186* , 189, 190, 192* , 196. sixteen bits : \underline{36}, 37, 48, 50, 53, 55, 66, 69, 172, 174, 201, 206, 207* , 219, 236, 242, 244.

skip comment : \underline{91}, 112* , 132, 136* .

skip limbo: \underline{89}, 109, 132. skip TeX : \underline{90}, 113, 132. : 35. sort ptr : \underline{244}, 249, 250, 251, 252. special string characters: 189. split procedures: 149, 183* , 261. spotless : \underline{9}, 10, 263. sq : 148, \underline{174}. squash: \underline{148}, 151, 152, 154, 157, 160, 161, 162,

163, 166, 167, 169, 170, 174, 178. stack : 201, \underline{202}, 204, 205. stack ptr : 201, \underline{202}, 204, 205. stack size: \underline{8}, 202, 204. stat: \underline{3}. stmt : \underline{140}, 143, 149, 152, 153, 155, 156, 159, 160,

161, 162, 164, 167, 168, 169, 170. string : \underline{93}, 99, 185. : 99. string delimiter : \underline{208}, 216. sub cases : 183* , 191* , \underline{192*} . system dependencies: \underline{1*} , 2, 4, 7, 12, 17, 20, 21, 22,

24, 26, 28, 32, 33, 259, 260, 261, 263, 264. s0 : \underline{184}. s1 : \underline{184}. s2 : \underline{184}. s3 : \underline{184}. s4 : \underline{184}. t: \underline{58}. tab mark : \underline{15}, 32, 87, 89, 92* , 95, 103, 104,

123, 133, 135. tats: \underline{3}. temp line: \underline{71}, 72. term in: \underline{258}, 259, 260. term out : \underline{20}, 21, 22, 260. terminator : 139, \underline{140}, 142, 143, 149, 152, 153, 160,

161, 164, 166, 167, 179, 193* , 194, 195. : 222* . tex file: \underline{2}, \underline{25}, 26, 122, 124* . TeX string : \underline{86*} , 87, 93, 100, 185, 222* . text char : \underline{12}, 13, 20. text file: \underline{12}, 20, 23, 25, 28, 258. text pointer : \underline{52}, 53, 144, 146, 179, 197, 198* ,

204, 207* , 226. text ptr : \underline{53}, 54, 146, 171, 172, 175, 179, 180, 184,

187, 195, 198* , 199, 207* , 208, 226, 256. thin space: \underline{86*} , 87, 186* , 222* . : 34. this module: \underline{229*} , 230* , 231, 233, 235. this xref : \underline{234}, 235, 255. to like: \underline{42*} , 64* , 191* , 192* . tok field : \underline{201}, 202. tok flag : \underline{146}, 148, 195, 198* , 206, 226. tok mem: \underline{53}, 136* , \underline{146}, 148, 201, 202, 206, 213. tok ptr : \underline{53}, 54, 136* , 137* , 148, 171, 175, 179, 180,

187, 189, 198* , 199, 207* , 226, 256. tok start : 52, \underline{53}, 54, 144, 146, 171, 204. tracing : 88, \underline{177}, 178, 181, 182, 259. trans : \underline{144}, 148, 172, 176, 179, 183* , 184, 195,

197, 244. translate: 149, \underline{179}, 180, 197, 226. trouble shooting : 95, 206, 252, \underline{258}, 259. true: 6, 28, 29, 71, 72, 74, 79, 81, 82, 83, 85, 93, 100, 109, 110, 122, 127, 128, 186* , 227* , 239, 258, 259. typewriter : \underline{42*} , 111* , 253. unbucket : \underline{249}, 250, 251. underline: \underline{86*} , 87, 100, 113. : 87. until like: \underline{42*} , 64* , 193* . up to: \underline{95}. update terminal : \underline{22}, 31, 110, 221, 260. var head : \underline{140}, 142, 143, 148, 149, 162, 163,

170, 193* . var like: \underline{42*} , 64* , 111* , 193* . verbatim: \underline{86*} , 87, 100, 107, 185, 189. : 107. vertical bar : \underline{86*} , 97* , 186* . w: \underline{44}, \underline{58}, \underline{66}, \underline{69}, \underline{131}, \underline{208}. WEAVE : \underline{2}. : 79. web file: \underline{2}, \underline{23}, 24, 32, 71, 73, 79, 83, 85. : 76, 80, 84. wi : \underline{40}, 41. wildcard : \underline{42*} , 111* , 253. write: 20, 122, 124* , 260. write ln: 20, 122. ww : 8, \underline{37}, 38, 39, 40, 41, 44, 50, 58, 61, 62, 66, 67, 68, 69, 131, 208, 209, 214, 243, 244, 251, 262.

xchr : \underline{13}, 14, 16, 17, 18, 32, 44, 105, 122, 128,

146, 147, 182, 260. xclause: 6. xlink : \underline{46}, 50, 51, 119, 213, 235, 237, 254, 255. xlink field : 46, \underline{48}. xmem: 46, \underline{48}. xord : \underline{13}, 16, 18, 28. xref : 36, \underline{37}, 46, 49, 50, 51, 62, 67, 119, 213,

231, 235, 243, 255. xref number : \underline{47}, 48, 50, 51, 118, 234, 236.

xref ptr : 46, \underline{48}, 49, 50, 51, 262.

xref roman: \underline{86*} , 87, 93, 100, 111* , 113, 186* , 222* . xref switch: 46, \underline{48}, 49, 50, 93, 100, 101, 111* ,

113, 115. xref typewriter : \underline{86*} , 87, 93, 111* , 113, 186* , 222* . xref wildcard : \underline{86*} , 87, 93, 111* , 113, 186* , 222* . : 222* , 232. : 231.

§265  MWEAVE  NAMES OF THE SECTIONS 107

⟨ Append a string scrap 189 ⟩ Used in section 185. ⟨ Append a TEX string scrap 190 ⟩ Used in section 185. ⟨ Append an identifier scrap 191* ⟩ Used in section 185. ⟨ Append the scrap appropriate to next control 185 ⟩ Used in section 183*.

⟨ Append terminator if not already present 194 ⟩ Used in sections 193*, 193*, and 193*. ⟨ Cases for alpha 151 ⟩ Used in section 150. ⟨ Cases for beginning 152 ⟩ Used in section 150. ⟨ Cases for case head 153 ⟩ Used in section 149. ⟨ Cases for casey 154 ⟩ Used in section 149. ⟨ Cases for clause 155 ⟩ Used in section 149. ⟨ Cases for cond 156 ⟩ Used in section 149. ⟨ Cases for elsie 157 ⟩ Used in section 149. ⟨ Cases for exp 158 ⟩ Used in section 149. ⟨ Cases for intro 159 ⟩ Used in section 150. ⟨ Cases for math 160 ⟩ Used in section 150. ⟨ Cases for mod scrap 161 ⟩ Used in section 149. ⟨ Cases for open math 163 ⟩ Used in section 162. ⟨ Cases for open 162 ⟩ Used in section 150. ⟨ Cases for proc 164 ⟩ Used in section 149. ⟨ Cases for record head 165 ⟩ Used in section 149. ⟨ Cases for semi 166 ⟩ Used in section 149. ⟨ Cases for simp 167 ⟩ Used in section 150. ⟨ Cases for stmt 168 ⟩ Used in section 149. ⟨ Cases for terminator 169 ⟩ Used in section 149. ⟨ Cases for var head 170 ⟩ Used in section 149.

⟨ Cases involving nonstandard ASCII characters 188* ⟩ Used in section 186*. ⟨ Cases that generate more than one scrap 193* ⟩ Used in section 191*.

⟨ Change pp to max(scrap base,pp+d) 173 ⟩ Used in sections 172 and 174.

⟨ Check for overlong name 105 ⟩ Used in section 103. ⟨ Check that all changes have been read 85 ⟩ Used in section 261. ⟨ Check that = or ≡ follows this module name, and emit the scraps to start the module definition 231 ⟩

Used in section 230*. ⟨ Clear bal and goto done 138* ⟩ Used in sections 136* and 137*.

⟨ Combine the irreducible scraps that remain 180 ⟩ Used in section 179. ⟨ Compare name p with current identifier, goto found if equal 61 ⟩ Used in section 60. ⟨ Compiler directives 4 ⟩ Used in section 2. ⟨ Compress two-symbol combinations like ‘ ’ 97* ⟩ Used in section 95. ⟨ Compute the hash code h 59 ⟩ Used in section 58. ⟨ Compute the name location p 60 ⟩ Used in section 58. ⟨ Constants in the outer block 8 ⟩ Used in section 2. ⟨ Copy a control code into the buffer 217 ⟩ Used in section 216. ⟨ Copy special things when c = , , , ; goto done at end 137* ⟩ Used in section 136*. ⟨ Copy the Pascal text into buffer [(limit + 1) . . j] 216 ⟩ Used in section 214. ⟨ Copy up to ‘ ’ or control code, goto done if finished 135 ⟩ Used in section 134. ⟨ Copy up to control code, return if finished 133 ⟩ Used in section 132. ⟨ Declaration of subprocedures for translate 150 ⟩ Used in section 179. ⟨ Declaration of the app comment procedure 195 ⟩ Used in section 183*. ⟨ Declaration of the app octal and app hex procedures 196 ⟩ Used in section 183*. ⟨ Declaration of the easy cases procedure 186* ⟩ Used in section 183*. ⟨ Declaration of the sub cases procedure 192* ⟩ Used in section 183*. ⟨ Do special things when c = , , , ; goto done at end 92* ⟩ Used in section 91. ⟨ Do the first pass of sorting 243 ⟩ Used in section 239.

108  NAMES OF THE SECTIONS  MWEAVE §265

⟨ Emit the scrap for a module name if present 232 ⟩ Used in section 230*. ⟨ Enter a new module name into the tree 67 ⟩ Used in section 66. ⟨ Enter a new name into the table at position p 62 ⟩ Used in section 58. ⟨ Error handling procedures 30, 31, 33 ⟩ Used in section 2.

⟨ Get a string 99 ⟩ Used in section 95. ⟨ Get an identifier 98 ⟩ Used in section 95.

⟨ Get control code and possible module name 100 ⟩ Used in section 95.

⟨ Globals in the outer block 9, 13, 20, 23, 25, 27, 29, 37, 39, 45, 48, 53, 55, 63*, 65, 71, 73, 93, 108, 114, 118, 121, 129,

144, 177, 202, 219, 229*, 234, 240, 242, 244, 246, 258 ⟩ Used in section 2.

⟨ Go to found if c is a hexadecimal digit, otherwise set scanning hex ← false 96 ⟩ Used in section 95.

⟨ If end of name, goto done 104 ⟩ Used in section 103.

⟨ If semi-tracing, show the irreducible scraps 181 ⟩ Used in section 180. ⟨ If the current line starts with , report any discrepancies and return 80 ⟩ Used in section 79. ⟨ If tracing, print an indication of where we are 182 ⟩ Used in section 179. ⟨ Invert the cross-reference list at cur name, making cur xref the head 255 ⟩ Used in section 254. ⟨ Local variables for initialization 16, 40, 56, 247 ⟩ Used in section 2. ⟨ Look ahead for strongest line break, goto reswitch 212 ⟩ Used in section 211. ⟨ Make sure that there is room for at least four more scraps, six more tokens, and four more texts 187 ⟩

Used in section 185.

⟨ Make sure that there is room for at least seven more tokens, three more texts, and one more scrap 199 ⟩

Used in section 198*.

⟨ Make sure the entries cat [pp . . (pp + 3)] are defined 176 ⟩ Used in section 175. ⟨ Match a production at pp, or increase pp if there is no match 149 ⟩ Used in section 175.

⟨ Move buffer and limit to change buffer and change limit 78 ⟩ Used in sections 75 and 79.

⟨ Output a control, look ahead in case of line breaks, possibly goto reswitch 211 ⟩ Used in section 208.

⟨ Output a operator 210 ⟩ Used in section 208. ⟨ Output a module name 213 ⟩ Used in section 208.

⟨ Output all the module names 257 ⟩ Used in section 239. ⟨ Output all the module numbers on the reference list cur xref 237 ⟩ Used in section 236.

⟨ Output an identifier 209 ⟩ Used in section 208.

⟨ Output index entries for the list at sort ptr 252 ⟩ Used in section 250. ⟨ Output the code for the beginning of a new module 221 ⟩ Used in section 220. ⟨ Output the code for the end of a module 238 ⟩ Used in section 220. ⟨ Output the cross-references at cur name 254 ⟩ Used in section 252.

⟨ Output the name at cur name 253 ⟩ Used in section 252.

⟨ Output the text of the module name 214 ⟩ Used in section 213. ⟨ Phase I: Read all the user’s text and store the cross references 109 ⟩ Used in section 261. ⟨ Phase II: Read all the text again and translate it to TEX form 218 ⟩ Used in section 261. ⟨ Phase III: Output the cross-reference index 239 ⟩ Used in section 261. ⟨ Print error location based on input buffer 32 ⟩ Used in section 31. ⟨ Print error messages about unused or undefined module names 120 ⟩ Used in section 109. ⟨ Print statistics about memory usage 262 ⟩ Used in section 261.

⟨ Print the job history 263 ⟩ Used in section 261.

⟨ Print token r in symbolic form 147 ⟩ Used in section 146. ⟨ Print warning message, break the line, return 128 ⟩ Used in section 127. ⟨ Process a format definition 116 ⟩ Used in section 115. ⟨ Put module name into mod text [1 . . k] 103 ⟩ Used in section 101. ⟨ Read from change file and maybe turn off changing 84 ⟩ Used in section 82. ⟨ Read from web file and maybe turn on changing 83 ⟩ Used in section 82. ⟨ Rearrange the list pointed to by cur xref 235 ⟩ Used in section 233. ⟨ Reduce the scraps using the productions until no more rules apply 175 ⟩ Used in section 179. ⟨ Scan a verbatim string 107 ⟩ Used in section 100.

⟨ Scan the module name and make cur module point to it 101 ⟩ Used in section 100.

⟨ Scan to the next 106 ⟩ Used in section 100. ⟨ Set initial values 10, 14, 17, 18, 21, 26, 41, 43, 49, 54, 57, 94, 102, 124*, 126, 145, 203, 245, 248, 259 ⟩ Used in section 2.

⟨ Set variable c to the result of comparing the given name to name p 68 ⟩ Used in sections 66 and 69. ⟨ Show cross references to this module 233 ⟩ Used in section 220. ⟨ Skip next character, give error if not ‘ ’ 215 ⟩ Used in section 214. ⟨ Skip over comment lines in the change file; return if end of file 76 ⟩ Used in section 75. ⟨ Skip to the next nonblank line; return if end of file 77 ⟩ Used in section 75. ⟨ Sort and output the index 250 ⟩ Used in section 239. ⟨ Special control codes allowed only when debugging 88 ⟩ Used in section 87. ⟨ Split the list at sort ptr into further lists 251 ⟩ Used in section 250. ⟨ Start a format definition 228 ⟩ Used in section 225. ⟨ Start a macro definition 227* ⟩ Used in section 225. ⟨ Store all the reserved words 64* ⟩ Used in section 261. ⟨ Store cross reference data for the current module 110 ⟩ Used in section 109. ⟨ Store cross references in the definition part of a module 115 ⟩ Used in section 110. ⟨ Store cross references in the Pascal part of a module 117 ⟩ Used in section 110. ⟨ Store cross references in the TEX part of a module 113 ⟩ Used in section 110. ⟨ Tell about changed modules 241 ⟩ Used in section 239. ⟨ Translate a hexadecimal constant appearing in TEX text 224 ⟩ Used in section 222*. ⟨ Translate an octal constant appearing in TEX text 223 ⟩ Used in section 222*. ⟨ Translate the current module 220 ⟩ Used in section 218. ⟨ Translate the definition part of the current module 225 ⟩ Used in section 220. ⟨ Translate the Pascal part of the current module 230* ⟩ Used in section 220. ⟨ Translate the TEX part of the current module 222* ⟩ Used in section 220.

⟨ Types in the outer block 11, 12, 36, 38, 47, 52, 201 ⟩ Used in section 2.

The WEAVE processor

(Version 2.8)

Section Page

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 The character set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 5 Input and output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 9 Reporting errors to the user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 11 Data structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 13 Searching for identifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 18 Initializing the table of reserved words . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 20 Searching for module names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 22 Lexical scanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 24 Inputting the next token . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 32 Phase one processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 38 Low-level output routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 42 Routines that copy TEX material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 45 Parsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 48 Implementing the productions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 55 Initializing the scraps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 69 Output of tokens . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 77 Phase two processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 84 Phase three processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 91 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 96 The main program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 98 System-dependent changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264 99 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 100