[mmh] / man / mh-format.man

.TH MH-FORMAT %manext5% "March 24, 2012" "%nmhversion%"
.\"
.\" %nmhwarning%
.\"
.SH NAME
mh-format \- format file for nmh message system
.SH DESCRIPTION
Several
.B nmh
commands utilize either a
.I format
string or a
.I format
file during their execution.  For example,
.B scan
uses a format string which directs it how to generate the scan listing
for each message;
.B repl
uses a format file which directs it
how to generate the reply to a message, and so on.
.PP
There are a few alternate scan listing formats available
in
.IR nmh/etc/scan.time ,
.IR nmh/etc/scan.size ,
and
.IR nmh/etc/scan.timely .
Look in
.I nmh/etc
for other
.B scan
and
.B repl
format files which may have been written at your site.
.PP
It suffices to have your local
.B nmh
expert actually write new format
commands or modify existing ones.  This manual section explains how to
do that.  Note: familiarity with the C
.B printf
routine is assumed.
.PP
A format string consists of ordinary text, and special multi-character
escape sequences which begin with `%'.  When specifying a format
string, the usual C backslash characters are honored: `\\b', `\\f',
`\\n', `\\r', and `\\t'.  Continuation lines in format files end with
`\\' followed by the newline character.
.\" TALK ABOUT SYNTAX FIRST, THEN SEMANTICS
.SS SYNTAX
Format strings are built around
.IR "escape sequences" .
There are three types of escape sequences: header
.IR components ,
built-in
.IR functions ,
and flow
.IR control .
Comments may be inserted in most places where a function argument is
not expected.  A comment begins with `%;' and ends with a (non-escaped)
newline.
.PP
A
.I component
escape is specified as
.RI `%{ component }',
and
exists for each header found in the message being processed.  For example
.RI `%{ date }'
refers to the \*(lqDate:\*(rq field of the appropriate message.
All component escapes have a string value.  Normally, component values are
compressed by converting any control characters (tab and newline included)
to spaces, then eliding any leading or multiple spaces.  However, commands
may give different interpretations to some component escapes; be sure
to refer to each command's manual entry for complete details.  Some commands
(such as
.B ap
and
.BR mhl )
use a special component
.RI `%{ text }'
to refer to the text being processed; see their respective man pages for
details and examples.
.PP
A
.I function
escape is specified as
.RI `%( function )'.
All functions are built-in, and most have a string or numeric value.
A function escape may have an
.IR argument .
The argument follows the function escape: separating
whitespace is discarded:
.RI `%( function " " argument )'.
.PP
In addition to literal numbers or strings, 
the argument to a function escape can be another function, a component,
or a control escape.  When the argument is a function or a
component, they are listed without a leading `%'.  When control escapes
are used as function arguments, they written as normally, with
a leading `%';
.SS "Control escapes"
.PP
A
.I control
escape is one of: `%<', `%?', `%|', or `%>'. 
These are combined into the conditional execution construct:
.PP
.RS 5
.nf
.RI "%< " condition " " "format-text"
.RI "%? " condition " " "format-text"
    \&...
.RI "%| " "format-text"
%>
.fi
.RE
.PP
Extra white space is shown here only for clarity.  These
constructs may be nested without ambiguity.  They form a general
.B if\-elseif\-else\-endif
block where only one of the
format-texts
is interpreted.  In other 
words, `%<' is like the "if", `%?' is like the "elseif", `%|' is like 
"else", and `%>' is like "endif".
.PP
A `%<' or `%?' control escape causes its condition to be evaluated.  
This condition is a
.I component
or
.IR function .
For integer valued functions or components, the condition is true
if the function return or component value is non-zero, and false if zero.
For string valued functions or components, the condition is true
if the function return or component value is 
a non-empty string, and false for an empty string.
.PP
The `%?' control escape is optional, and may there may be more
than one `%?' control escape in a conditional block.
The `%|' control escape
is also optional, but may be included at most once.
.SS "Function escapes"
Functions expecting an argument generally
require an argument of a particular type.
In addition to the number and string types,
these include:
.PP
.RS 5
.nf
.ta +\w'Argument 'u +\w'An optional component, 'u
.I "Argument    Description     Example Syntax"
literal A literal number        %(\fIfunc\fR 1234)
        or string               %(\fIfunc\fR text string)
comp    Any component           %(\fIfunc\fR\^{\fIin-reply-to\fR\^})
date    A date component        %(\fIfunc\fR\^{\fIdate\fR\^})
addr    An address component    %(\fIfunc\fR\^{\fIfrom\fR\^})
expr    Nothing %(\fIfunc\fR)
        or a subexpression      %(\fIfunc\fR\^(\fIfunc2\fR\^))
        or control escape       %(\fIfunc\fR %<{\fIreply-to\fR\^}%|%{\fIfrom\fR\^}%>)
.fi
.RE
.PP
The types
.I date
and
.I addr
have the same syntax as
.IR comp ,
but require that the header component be a date string, or address
string, respectively.
.PP
Most arguments not of type
.IR expr
are required.
When escapes are nested (via expr arguments), evaluation is done from inner-most to outer-most.
As noted above, for the
expr
argument type, 
functions and components are written without a
leading `%'.
Control escape arguments must use a leading `%', preceded by a space.
.PP
For example,
.PP
.RS 5
.nf
%<(mymbox{from}) To: %{to}%>
.fi
.RE
.PP
writes  the  value of the header component \*(lqFrom:\*(rq to the
internal register named str; then (\fImymbox\fR\^) reads str and
writes its result to the internal register named 
.IR num ; 
then the control escape evaluates 
.IR num .  
If
.IR num
is non-zero, the
string \*(lqTo:\*(rq is printed  followed  by  the  value  of  the
header component \*(lqTo:\*(rq.
.SS Evaluation
The evaluation of format strings is performed
by a small virtual machine.
The machine is capable of evaluating nested expressions
as described above, and in addition
has an integer register
.IR num ,
and a text string register
.IR str .
When a function escape that
accepts an optional argument is processed,
and the argument is not present, the current value of either
.I num
or
.I str
is used as the argument: which register is
used depends on the function, as listed below.
.PP
Component escapes write the value of their message header in
.IR str .
Function escapes write their return value in
.I num
for functions returning integer or boolean values, and in
.I str
for functions returning string values.  (The boolean type is a subset
of integers with usual values 0=false and 1=true.)  Control escapes
return a boolean value, setting
.I num
to 1 if the last explicit condition
evaluated by a `%<' or `%?' control
succeeded, and 0 otherwise.
.PP
All component escapes, and those function escapes which return an
integer or string value, evaluate to their value as well as setting
.I str
or
.IR num .
Outermost escape expressions in
these forms will print
their value, but outermost escapes which return a boolean value
do not result in printed output.
.SS Functions
The function escapes may be roughly grouped into a few categories.
.PP
.RS 5
.nf
.ta \w'Fformataddr 'u +\w'Aboolean 'u +\w'Rboolean 'u
.I "Function    Argument   Result       Description"
msg             integer message number
cur             integer message is current (0 or 1)
unseen          integer message is unseen (0 or 1)
size            integer size of message
strlen          integer length of \fIstr\fR
width           integer output buffer size in bytes
charleft                integer bytes left in output buffer
timenow         integer seconds since the UNIX epoch
me              string  the user's mailbox (username)
myhost          string  the user's local hostname
myname          string  the user's name
localmbox               string  the complete local mailbox
eq      literal boolean \fInum\fR == \fIarg\fR
ne      literal boolean \fInum\fR != \fIarg\fR
gt      literal boolean \fInum\fR > \fIarg\fR
match   literal boolean \fIstr\fR contains \fIarg\fR
amatch  literal boolean \fIstr\fR starts with \fIarg\fR
plus    literal integer \fIarg\fR plus \fInum\fR
minus   literal integer \fIarg\fR minus \fInum\fR
divide  literal integer \fInum\fR divided by \fIarg\fR
modulo  literal integer \fInum\fR modulo \fIarg\fR
num     literal integer Set \fInum\fR to \fIarg\fR.
num             integer Set \fInum\fR to zero.
lit     literal string  Set \fIstr\fR to \fIarg\fR.
lit             string  Clear \fIstr\fR.
getenv  literal string  Set \fIstr\fR to environment value of \fIarg\fR
profile literal string  Set \fIstr\fR to profile component \fIarg\fR 
                        value
.\" dat literal int     return value of dat[arg]
nonzero expr    boolean \fInum\fR is non-zero
zero    expr    boolean \fInum\fR is zero
null    expr    boolean \fIstr\fR is empty
nonnull expr    boolean \fIstr\fR is non-empty
void    expr            Set \fIstr\fR or \fInum\fR
comp    comp    string  Set \fIstr\fR to component text
compval comp    integer Set \fInum\fR to \*(lq\fBatoi\fR(\fIcomp\fR\^)\*(rq
.\" compflag    comp    integer Set \fInum\fR to component flags bits (internal)
.\" decodecomp  comp    string  Set \fIstr\fR to RFC-2047 decoded component text
decode  expr    string  decode \fIstr\fR as RFC-2047 (MIME-encoded) 
                        component
unquote expr    string  remove RFC-2822 quotes from \fIstr\fR
trim    expr            trim trailing white-space from \fIstr\fR
putstr  expr            print \fIstr\fR
putstrf expr            print \fIstr\fR in a fixed width
putnum  expr            print \fInum\fR
putnumf expr            print \fInum\fR in a fixed width
.\" addtoseq literal    add msg to sequence (LBL option)
putlit  expr            print \fIstr\fR without space compression
nodate  string  integer Argument not a date string (0 or 1)
formataddr      expr            append \fIarg\fR to \fIstr\fR as a
                        (comma separated) address list
concataddr      expr            append \fIarg\fR to \fIstr\fR as a
                        (comma separated) address list,
                        including duplicates,
                        see Special Handling
putaddr literal         print \fIstr\fR address list with
                        \fIarg\fR as optional label;
                        get line width from \fInum\fR
.fi
.RE
.PP
The (\fIme\fR\^) function returns the username of the current user.  The
(\fImyhost\fR\^) function returns the
.B localname
entry in
.IR mts.conf ,
or the local hostname if
.B localname
is not configured.  The (\fImyname\fR\^) function will return the value of
the
.B SIGNATURE
environment variable if set, otherwise will return the passwd GECOS field
(truncated at the first comma if it contains one) for
the current user.  The (\fIlocalmbox\fR\^) function will return the complete
form of the local mailbox, suitable for use in a \*(lqFrom\*(rq header.
It will return the
.RI \*(lq Local-Mailbox \*(rq
profile entry if it is set; if it is not, it will be equivalent to:
.PP
.RS 5
.nf
%(myname) <%(me)@%(myhost)>
.fi
.RE
.PP
The following functions require a date component as an argument:
.PP
.RS 5
.nf
.ta \w'Fformataddr 'u +\w'Aboolean 'u +\w'Rboolean 'u
.I "Function    Argument        Return  Description"
sec     date    integer seconds of the minute
min     date    integer minutes of the hour
hour    date    integer hours of the day (0-23)
wday    date    integer day of the week (Sun=0)
day     date    string  day of the week (abbrev.)
weekday date    string  day of the week
sday    date    integer day of the week known?
                        (1=explicit,0=implicit,\-1=unknown)
mday    date    integer day of the month
yday    date    integer day of the year
mon     date    integer month of the year
month   date    string  month of the year (abbrev.)
lmonth  date    string  month of the year
year    date    integer year (may be > 100)
zone    date    integer timezone in hours
tzone   date    string  timezone string
szone   date    integer timezone explicit?
                        (1=explicit,0=implicit,\-1=unknown)
date2local      date            coerce date to local timezone
date2gmt        date            coerce date to GMT
dst     date    integer daylight savings in effect? (0 or 1)
clock   date    integer seconds since the UNIX epoch
rclock  date    integer seconds prior to current time
tws     date    string  official 822 rendering
pretty  date    string  user-friendly rendering
.fi
.RE
.PP
These functions require an address component as an argument.  
The return value of functions noted with `*' is computed from
the first address present in the header component.
.PP
.RS 5
.nf
.ta \w'Fformataddr 'u +\w'Aboolean 'u +\w'Rboolean 'u
.I "Function    Argument        Return  Description"
proper  addr    string  official 822 rendering
friendly        addr    string  user-friendly rendering
addr    addr    string  mbox@host or host!mbox rendering*
pers    addr    string  the personal name*
note    addr    string  commentary text*
mbox    addr    string  the local mailbox*
mymbox  addr    integer List has the user's address? (0 or 1)
host    addr    string  the host domain*
nohost  addr    integer no host was present (0 or 1)*
type    addr    integer host type* (0=local,1=network,
                        \-1=uucp,2=unknown)
path    addr    string  any leading host route*
ingrp   addr    integer address was inside a group (0 or 1)*
gname   addr    string  name of group*
.fi
.RE
.PP
(A clarification on (\fImymbox\fR\^{\fIcomp\fR\^}) is in order.
This function checks each of the addresses in the header component
\*(lq\fIcomp\fR\*(rq against the user's mailbox name and any
.RI \*(lq Alternate-Mailboxes \*(rq.
It returns true if any address matches,
however, it also returns true if the \*(lq\fIcomp\fR\*(rq header is not
present in the message.  If needed, the (\fInull\fR\^) function can be
used to explicitly test for this case.)
.SS Formatting
When a function or component escape is interpreted and the result will
be immediately printed, an optional field width can be specified to
print the field in exactly a given number of characters.  For example, a
numeric escape like %4(\fIsize\fR\^) will print at most 4 digits of the
message size; overflow will be indicated by a `?' in the first position
(like `?234').  A string escape like %4(\fIme\fR\^) will print the first 4
characters and truncate at the end.  Short fields are padded at the right
with the fill character (normally, a blank).  If the field width argument
begins with a leading zero, then the fill character is set to a zero.
.PP
The functions (\fIputnumf\fR\^) and (\fIputstrf\fR\^)
print their result in exactly the number of characters
specified by their leading field width argument.  For example,
%06(\fIputnumf\fR\^(\fIsize\fR\^)) will print the message
size in a field six characters wide filled with leading zeros;
%14(\fIputstrf\^\fR{\fIfrom\^\fR}) will print the \*(lqFrom:\*(rq header
component in fourteen characters with trailing spaces added as needed.
For \fIputstrf\fR, using a negative value for the field width causes
right-justification of the string within the field, with padding on
the left up to the field width.
The functions (\fIputnum\fR\^) and
(\fIputstr\fR\^) are somewhat special: they print their result in the minimum number of characters
required, and ignore any leading field width argument.  The (\fIputlit\fR\^)
function outputs the exact contents of str register without any changes
such as duplicate space removal or control character conversion.
.PP
The available output width is kept in an internal register; any output
past this width will be truncated.
.SS Special Handling
A few functions have different behavior depending on what command they are
being invoked from.
.PP
In
.BR repl
the (\fIformataddr\fR\^) function stores all email addresses encountered into
an internal cache and will use this cache to suppress duplicate addresses.
If you need to create an address list that includes previously-seen
addresses you may use the (\fIconcataddr\fR\^) function, which is identical
to (\fIformataddr\fR\^) in all other respects.  Note that (\fIconcataddr\fR\^)
will NOT add addresses to the duplicate-suppression cache.
.SS Other Hints and Tips
Sometimes to format function writers it is confusing as to why output is
duplicated.  The general rule to remember is simple: If a function or
component escape is used where it starts with a %, then it will generate
text in the output file.  Otherwise, it will not.
.PP
A good example is a simple attempt to generate a To: header based on
the From: and Reply-To: headers:
.PP
.RS 5
.nf
%(formataddr %<{reply-to}%|%{from})%(putaddr To: )
.fi
.RE
.PP
Unfortuantely if the Reply-to: header is NOT present, the output line that is
generated will be something like:
.PP
.RS 5
.nf
My From User <from@example.com>To: My From User <from@example.com>
.fi
.RE
.PP
What went wrong?  When performing the test for the
.B if
clause (%<), the component is not output because it is considered an
argument to the
.B if
statement (hence the rule about the lack of % applies).  But the component
escape in our
.B else
statement (everything after the `%|') is NOT an argument to anything; the
syntax is that it is written with a %, and thus the value of that component
is output.  This also has the side effect of setting the 
.I str
register, which is later picked up by the (\fIformataddr\fR\^) function
and then output by (\fIputaddr\fR\^).  This format string has another bug
as well; there should always be a valid width value in the
.I num
register when (\fIputaddr\fR\^) is called, otherwise bad formatting can take
place.
.PP
The solution is to use the (\fIvoid\fR\^) function; this will prevent the
function or component from outputting any text.  With this in place (and
using (\fIwidth\fR\^) to set the
.I num
register for the width, a better implementation would look like:
.PP
.RS 3
.nf
%(formataddr %<{reply-to}%|%(void{from})%(void(width))%(putaddr To: )
.fi
.RE
.PP
It should be noted here that the side-effects of functions and component
escapes still are in force: as a result each component
test in the 
.B if\-elseif\-else\-endif
clause sets the
.I str
register.
.PP
As an additional note, the (\fIformataddr\fR\^) and (\fIconcataddr\fR\^)
functions have some behavior when it comes to the
.I str
register.  The starting point of the register is saved and is used to
build up entries in the address list.
.PP
You will find the
.B ap
and
.B fmtdump
utilities invaluable in debugging problems with format strings.
.SS Examples
With all this in mind,
here's the default format string for
.BR scan .
It's been divided into several pieces for readability.
The first part is:
.PP
.RS
.nf
%4(msg)%<(cur)+%| %>%<{replied}\-%?{encrypted}E%| %>
.fi
.RE
.PP
which says that the message number should be printed in four digits.
If the message is the current message then a `+' else a space should
be printed; if a \*(lqReplied:\*(rq field is present then a `\-'
else if an \*(lqEncrypted:\*(rq field is present then an `E' otherwise
a space should be printed.  Next:
.PP
.RS
.nf
%02(mon{date})/%02(mday{date})
.fi
.RE
.PP
the month and date are printed in two digits (zero filled) separated by
a slash. Next,
.PP
.RS 5
.nf
%<{date} %|*%>
.fi
.RE
.PP
If a \*(lqDate:\*(rq field was present,
then a space is printed, otherwise a `*'.
Next,
.PP
.RS 5
.nf
%<(mymbox{from})%<{to}To:%14(decode(friendly{to}))%>%>
.fi
.RE
.PP
if the message is from me, and there is a \*(lqTo:\*(rq header,
print \*(lqTo:\*(rq followed by a \*(lquser-friendly\*(rq rendering of the
first address in the \*(lqTo:\*(rq field; any MIME-encoded
characters are decoded into the actual characters.
Continuing,
.PP
.RS 5
.nf
%<(zero)%17(decode(friendly{from}))%>
.fi
.RE
.PP
if either of the above two tests failed,
then the \*(lqFrom:\*(rq address is printed
in a mime-decoded, \*(lquser-friendly\*(rq format.
And finally,
.PP
.RS 5
.nf
%(decode{subject})%<{body}<<%{body}>>%>
.fi
.RE
.PP
the mime-decoded subject and initial body (if any) are printed.
.PP
For a more complicated example, next consider
a possible
.I replcomps
format file.
.PP
.RS 5
.nf
%(lit)%(formataddr %<{reply-to}
.fi
.RE
.PP
This clears
.I str
and formats the \*(lqReply-To:\*(rq header 
if present.  If not present, the else-if clause is executed.
.PP
.RS 5
.nf
%?{from}%?{sender}%?{return-path}%>)\\
.fi
.RE
.PP
This formats the 
\*(lqFrom:\*(rq, \*(lqSender:\*(rq and \*(lqReturn-Path:\*(rq
headers, stopping as soon as one of them is present.  Next:
.PP
.RS 5
.nf
%<(nonnull)%(void(width))%(putaddr To: )\\n%>\\
.fi
.RE
.PP
If the \fIformataddr\fR result is non-null, it is printed as
an address (with line folding if needed) in a field \fIwidth\fR
wide with a leading label of \*(lqTo:\*(rq.
.PP
.RS 5
.nf
%(lit)%(formataddr{to})%(formataddr{cc})%(formataddr(me))\\
.fi
.RE
.PP
.I str
is cleared, and the \*(lqTo:\*(rq and \*(lqCc:\*(rq headers, along with the user's
address (depending on what was specified with
the \*(lq\-cc\*(rq switch to \fIrepl\fR\^) are formatted.
.PP
.RS 5
.nf
%<(nonnull)%(void(width))%(putaddr cc: )\\n%>\\
.fi
.RE
.PP
If the result is non-null, it is printed as above with a
leading label of \*(lqcc:\*(rq.
.PP
.RS 5
.nf
%<{fcc}Fcc: %{fcc}\\n%>\\
.fi
.RE
.PP
If a
.B \-fcc
.I folder
switch was given to
.B repl
(see
.IR repl (1)
for more details about %{\fIfcc\fR\^}),
an \*(lqFcc:\*(rq header is output.
.PP
.RS 5
.nf
%<{subject}Subject: Re: %{subject}\\n%>\\
.fi
.RE
.PP
If a subject component was present,
a suitable reply subject is output.
.PP
.RS 5
.nf
%<{message-id}In-Reply-To: %{message-id}\\n%>\\
%<{message-id}References: %<{references} %{references}%>\\
%{message-id}\\n%>
\-\-\-\-\-\-\-\-
.fi
.RE
.PP
If a message-id component was present, an \*(lqIn-Reply-To:\*(rq header is
output including the message-id, followed by a \*(lqReferences:\*(rq
header with references, if present, and the message-id.
As with all
plain-text, the row of dashes are output as-is.
.PP
This last part is a good example for a little more elaboration.
Here's that part again in pseudo-code:
.PP
.RS 5
.nf
.ta .5i 1i 1.5i 2i
if (comp_exists(message-id))  then
        print (\*(lqIn-reply-to: \*(rq)
        print (message-id.value)
        print (\*(lq\\n\*(rq)
endif
if (comp_exists(message-id)) then
        print (\*(lqReferences: \*(rq)
        if (comp_exists(references)) then
              print(references.value);
        endif
        print (message-id.value)
        print (\*(lq\\n\*(rq)
endif
.fi
.RE
.PP
.\" (Note that this pseudocode begs the question ``why not just
.\" support this syntax?''  MH has been hacked on for a long time...)
.\".PP
One more example: Currently,
.B nmh
supports very
large message numbers, and it is not uncommon for a folder
to have far more than 10000 messages.
.\" (Indeed, the original MH
.\" tutorial document by Rose and Romine is entitled "How to
.\" process 200 messages a day and still get some real work
.\" done."  The authors apparently only planned to get
.\" real work done for about 50 days per folder.)
Nontheless (as noted above)
the various scan format strings are inherited
from older MH versions, and are generally hard-coded to 4
digits of message number before formatting problems
start to occur.  
The nmh format strings can be modified to behave more sensibly with larger
message numbers:
.PP
.RS
.nf
%(void(msg))%<(gt 9999)%(msg)%|%4(msg)%>
.fi
.RE
.PP
The current message number is placed in \fInum\fP.
(Note that
.RI ( msg )
is an int function, not a component.)
The
.RI ( gt )
conditional
is used to test whether the message number
has 5
or more digits.
If so, it is printed at full width, otherwise
at 4 digits.
.SH "SEE ALSO"
.IR scan (1),
.IR repl (1),
.IR ap (8),
.IR dp (8)
.SH CONTEXT
None