1 Request For Comments: draft
14 Post Office Protocol (revised)
15 Extended Service Offerings
18 Wed Dec 18 23:17:52 1985
23 Northrop Research and Technology Center
25 Palos Verdes Peninsula, CA 90274
32 This memo suggests a simple method for workstations to dynamically
33 access mail from a discussion group server, as an extension to an
34 earlier memo which dealt with dynamically accessing mail from a
35 mailbox server using the Post Office Protocol (POP). This RFC
36 specifies a proposed protocol for the ARPA Internet community, and
37 requests discussion and suggestions for improvements. All of the
38 extensions described in this memo to the POP are OPTIONAL.
40 Request For Comments: draft M. Rose
41 POP (revised) Extended Service Offerings NRTC
45 Introduction and Motivation
47 It is assumed that the reader is familiar with the previous memo
48 that discusses the Post Office Protocol (POP) [MRose84]. This memo
49 describes extensions to the POP which enhance the service it offers
50 to clients. This additional service permits a client host to access
51 discussion group mail, which is often kept in a separate spool area,
52 using the general POP facilities.
54 The next section describes the evolution of discussion groups and
55 the technologies currently used to implement them. To summarize:
57 o An exploder is used to map from a single address to
58 a list of addresses which subscribe to the list, and redirects
59 any subsequent error reports associated with the delivery of
60 each message. This has two primary advantages:
61 - Subscribers need know only a single address
62 - Responsible parties get the error reports and not
65 o Typically, each subscription address is not a person's private
66 maildrop, but a system-wide maildrop, which can be accessed
67 by more than one user. This has several advantages:
68 - Only a single copy of each message need traverse the
69 net for a given site (which may contain several local
70 hosts). This conserves bandwidth and cycles.
71 - Only a single copy of each message need reside on each
72 subscribing host. This conserves disk space.
73 - The private maildrop for each user is not cluttered
74 with discussion group mail.
76 Despite this optimization of resources, further economy can be
77 achieved at sites with more than one host. Typically, sites with
78 more than one host either:
80 1. Replicate discussion group mail on each host. This
81 results in literally gigabytes of disk space committed to
82 unnecessarily store redundant information.
84 2. Keep discussion group mail on one host and give all users a
85 login on that host (in addition to any other logins they may
86 have). This is usually a gross inconvenience for users who
87 work on other hosts, or a burden to users who are forced to
90 As discussed in [MRose84], the problem of giving workstations
91 dynamic access to mail from a mailbox server has been explored in
92 great detail (originally there was [RFC918], this prompted the
93 author to write [MRose84], independently of this [RFC918] was
94 upgraded to [RFC937]). A natural solution to the problem outlined
95 above is to keep discussion group mail on a mailbox server at each
96 site and permit different hosts at that site to employ the POP to
97 access discussion group mail. If implemented properly, this
98 avoids the problems of both strategies outlined above.
100 ASIDE: It might be noted that a good distributed filesystem
101 could also solve this problem. Sadly, "good"
102 distributed filesystems, which do not suffer
103 unacceptable response time for interactive use, are
104 few and far between these days!
106 Given this motivation, now let's consider discussion groups, both in
107 general and from the point of view of a user agent. Following this,
108 extensions to the POP defined in [MRose84] are presented. Finally,
109 some additional policy details are discussed along with some initial
112 What's in a Discussion Group
114 Since mailers and user agents first crawled out of the primordial
115 ARPAnet, the value of discussion groups have been appreciated,
116 (though their implementation has not always been well-understood).
118 Described simply, an discussion group is composed of a number of
119 subscribers with a common interest. These subscribers post mail to
120 a single address, known as a distribution address. From this
121 distribution address, a copy of the message is sent to each
122 subscriber. Each group has a moderator, which is the person that
123 administrates the group. The moderator can usually be reached at a
124 special address, known as a request address. Usually, the
125 responsibilities of the moderator are quite simple, since the mail
126 system handles the distribution to subscribers automatically. In
127 some cases, the interest group, instead of being distributed
128 directly to its subscribers, is put into a digest format by the
129 moderator and then sent to the subscribers. Although this requires
130 more work on the part of the moderator, such groups tend to be
133 Unfortunately, there are a few problems with the scheme outlined
134 above. First, if two users on the same host subscribe to the same
135 interest group, two copies of the message get delivered. This is
136 wasteful of both processor and disk resources.
138 Second, some of these groups carry a lot of traffic. Although
139 subscription to an group does indicate interest on the part of a
140 subscriber, it is usually not interesting to get 50 messages or so
141 delivered to the user's private maildrop each day, interspersed with
142 personal mail, that is likely to be of a much more important and
145 Third, if a subscriber on the distribution list for a group becomes
146 "bad" somehow, the originator of the message and not the moderator
147 of the group is notified. It is not uncommon for a large list to
148 have 10 or so bogus addresses present. This results in the
149 originator being flooded with "error messages" from mailers across
150 the ARPA Internet stating that a given address on the list was bad.
151 Needless to say, the originator usually could not care less if the
152 bogus addresses got a copy of the message or not. The originator is
153 merely interested in posting a message to the group at large.
154 Furthermore, the moderator of the group does care if there are bogus
155 addresses on the list, but ironically does not receive notification.
157 There are various approaches which can be used to solve some or all
158 of these problems. Usually these involve placing an exploder agent
159 at the distribution source of the discussion group, which expands
160 the name of the group into the list of subscription addresses for
161 the group. In the process, the exploder will also change the
162 address that receives error notifications to be the request address
163 or other responsible party.
165 A complementary approach, used in order to cut down on resource
166 utilization of all kinds, replaces all the subscribers at a single
167 host (or group of hosts under a single administration) with a single
168 address at that host. This address maps to a file on the host,
169 usually in a spool area, which all users can access. (Advanced
170 implementations can also implement private discussion groups this
171 way, in which a single copy of each message is kept, but is
172 accessible to only a select number of users on the host.)
174 The two approaches can be combined to avoid all of the problems
177 Finally, a third approach can be taken, which can be used to aid
178 user agents processing mail for the discussion group: In order to
179 speed querying of the maildrop which contains the local host's copy
180 of the discussion group, two other items are usually associated with
181 the discussion group, on a local basis. These are the maxima and
182 the last-date. Each time a message is received for the group on the
183 local host, the maxima is increased by at least one. Furthermore,
184 when a new maxima is generated, the current date is determined.
185 This is called the last date. As the message is entered into the
186 local maildrop, it is given the current maxima and last-date. This
187 permits the user agent to quickly determine if new messages are
188 present in the maildrop.
190 NOTE: The maxima may be characterized as a monotonically
191 increasing quanity. Although sucessive values of the
192 maxima need not be consecutive, any maxima assigned
193 is always greater than any previously assigned value.
197 To formalize these notions somewhat, consider the following 7
198 parameters which describe a given discussion group from the
199 perspective of the user agent (the syntax given is from [RFC822]):
201 NAME Meaning: the name of the discussion group
202 Syntax: TOKEN (ALPHA *[ ALPHA / DIGIT / "-" ])
203 (case-insensitive recognition)
204 Example: unix-wizards
206 ALIASES Meaning: alternates names for the group, which
207 are locally meaningful; these are
208 typically used to shorten user typein
209 Syntax: TOKEN (case-insensitive recognition)
212 ADDRESS Meaning: the primary source of the group
214 Example: Unix-Wizards@BRL
216 REQUEST Meaning: the primary moderator of the group
218 Example: Unix-Wizards-Request@BRL
220 FLAGS Meaning: locally meaningful flags associated
221 with the discussion group; this memo
222 leaves interpretation of this parameter
223 to each POP implementation
227 MAXIMA Meaning: the magic cookie associated with the
228 last message locally received for the
229 group; it is the property of the magic
230 cookie that it's value NEVER decreases,
231 and increases by at least one each time
232 a message is locally received
233 Syntax: decimal number
236 LASTDATE Meaning: the date that the last message was
239 Example: Thu, 19 Dec 85 10:26:48 -0800
241 Note that the last two values are locally determined for the
242 maildrop associated with the discussion group and with each message
243 in that maildrop. Note however that the last message in the
244 maildrop have a different MAXIMA and LASTDATE than the discussion
245 group. This often occurs when the maildrop has been archived.
247 Finally, some local systems provide mechanisms for automatically
248 archiving discussion group mail. In some cases, a two-level archive
249 scheme is used: current mail is kept in the standard maildrop,
250 recent mail is kept in an archive maildrop, and older mail is kept
251 off-line. With this scheme, in addition to having a "standard"
252 maildrop for each discussion group, an "archive" maildrop may also
253 be available. This permits a user agent to examine the most recent
254 archive using the same mechanisms as those used on the current mail.
258 The following commands are valid only in the TRANSACTION state of
259 the POP. This implies that the POP server has already opened the
260 user's maildrop (which may be empty). This maildrop is called the
261 "default maildrop". The phrase "closes the current maildrop" has
262 two meanings, depending on whether the current maildrop is the
263 default maildrop or is a maildrop associated with a discussion
266 In the former context, when the current maildrop is closed any
267 messages marked as deleted are removed from the maildrop currently
268 in use. The exclusive-access lock on the maildrop is then released
269 along with any implementation-specific resources (e.g.,
272 In the latter context, a maildrop associated with a discussion group
273 is considered to be read-only to the POP client. In this case, the
274 phrase "closes the current maildrop" merely means that any
275 implementation-specific resources are released. (Hence, the POP
276 command DELE is a no-op.)
278 All the new facilities are introduced via a single POP command,
279 XTND. All positive reponses to the XTND command are multi-line.
281 The most common multi-line response to the commands contains a
282 "discussion group listing" which presents the name of the discussion
283 group along with it's maxima. In order to simplify parsing all POP
284 servers are required to use a certain format for discussion group
289 This memo makes no requirement on what follows the maxima in the
290 listing. Minimal implementations should just end that line of the
291 response with a CRLF pair. More advanced implementations may
292 include other information, as parsed from the message.
294 NOTE: This memo STRONGLY discourages implementations from
295 supplying additional information in the listing.
299 Arguments: the name of a discussion group (optionally)
300 Restrictions: may only be given in the TRANSACTION state.
303 If an argument was given, the POP server closes the current
304 maildrop. The POP server then validates the argument as the
305 name of a discussion group. If this is successful, it opens
306 the maildrop associated with the group, and returns a
307 multi-line response containing the discussion group listing.
308 If the discussion group named is not valid, or the associated
309 archive maildrop is not readable by the user, then an error
310 response is returned.
312 If no argument was given, the POP server issues a multi-line
313 response. After the initial +OK, for each discussion group
314 known, the POP server responds with a line containing the
315 listing for that discussion group. Note that only
316 world-readable discussion groups are included in the multi-line
319 In order to aid user agents, this memo requires an extension to
320 the scan listing when an "XTND BBOARDS" command has been given.
321 Normally, a scan listing, as generated by the LIST, takes the
326 where MSGNO is the number of the message being listed and SIZE
327 is the size of the message in octets. When reading a maildrop
328 accessed via "XTND BBOARDS", the scan listing takes the form
332 where MAXIMA is the maxima that was assigned to the message
333 when it was placed in the BBoard.
344 C: XTND BBOARDS system
350 Arguments: the name of a discussion group (required)
351 Restrictions: may only be given in the TRANSACTION state.
354 The POP server closes the current maildrop. The POP server
355 then validates the argument as the name of a discussion group.
356 If this is successful, it opens the archive maildrop associated
357 with the group, and returns a multi-line response containing
358 the discussion group listing. If the discussion group named is
359 not valid, or the associated archive maildrop is not readable
360 by the user, then an error response is returned.
362 In addition, the scan listing generated by the LIST command is
363 augmented (as described above).
369 C: XTND ARCHIVE system
375 Arguments: the name of a discussion group (required)
376 Restrictions: may only be given in the TRANSACTION state.
379 The POP server validates the argument as the name of a
380 discussion group. If this is unsuccessful, then an error
381 response is returned. Otherwise a multi-line response is
382 returned. The first 14 lines of this response (after the
383 initial +OK) are defined in this memo. Minimal implementations
384 need not include other information (and may omit certain
385 information, outputing a bare CRLF pair). More advanced
386 implementations may include other information.
388 Line Information (refer to "Definition of Terms")
391 2 ALIASES, separated by SP
392 3 system-specific: maildrop
393 4 system-specific: archive maildrop
394 5 system-specific: information
395 6 system-specific: maildrop map
396 7 system-specific: encrypted password
397 8 system-specific: local leaders, separated by SP
400 11 system-specific: incoming feed
401 12 system-specific: outgoing feeds
405 Most of this information is entirely too specific to the UCI
406 Version of the Rand MH Message Handling System[MRose85].
407 Nevertheless, lines 1, 2, 9, 10, 13, and 14 are of general
408 interest, regardless of the implementation.
414 C: XTND X-BBOARDS system
418 S: /usr/bboards/system.mbox
419 S: /usr/bboards/archive/system.mbox
420 S: /usr/bboards/.system.cnt
421 S: /usr/bboards/.system.map
425 S: system-request@nrtc
427 S: dist-system@nrtc-gremlin
429 S: Thu, 19 Dec 85 00:08:49 -0800
434 Depending on the particular entity administrating the POP service
435 host, two additional policies might be implemented:
437 1. Private Discussion Groups
439 In the general case, discussion groups are world-readable, any user,
440 once logged in (via a terminal, terminal server, or POP, etc.), is
441 able to read the maildrop for each discussion group known to the POP
442 service host. Nevertheless, it is desirable, usually for privacy
443 reasons, to implement private discussion groups as well.
445 Support of this is consistent with the extensions outlined in this
446 memo. Once the AUTHORIZATION state has successfully concluded, the
447 POP server grants the user access to exactly those discussion groups
448 the POP service host permits the authenticated user to access. As a
449 "security" feature, discussion groups associated with unreadable
450 maildrops should not be listed in a positive response to the XTND
453 2. Anonymous POP Users
455 In order to minimize the authentication problem, a policy
456 permitting "anonymous" access to the world-readable maildrops for
457 discussion groups on the POP server may be implemented.
459 Support of this is consistent with the extensions outlined in this
460 memo. The POP server can be modified to accept a USER command for a
461 well-known pseudonym (i.e., "anonymous") which is valid with any
462 PASS command. As a "security" feature, it is advisable to limit
463 this kind of access to only hosts at the local site, or to hosts
464 named in an access list.
466 Experiences and Conclusions
468 All of the facilities described in this memo and in [MRose84] have
469 been implemented in MH #6.1. Initial experiences have been, on the
470 whole, very positive.
472 After the first implementation, some performance tuning was
473 required. This consisted primarily of caching the datastructures
474 which describe discussion groups in the POP server. A second
475 optimization pertained to the client: the program most commonly
476 used to read BBoards in MH was modified to retrieve messages only
477 when needed. Two schemes are used:
479 o If only the headers (and the first few lines of the body) of
480 the message are required, e.g., for a scan listing, then only
481 these are retrieved. The resulting output is then cached, on
484 o If the entire message is required, then it is retrieved intact,
487 With these optimizations, response time is quite adequate when the
488 POP server and client are connected via a high-speed local area
489 network. In fact, the author uses this mechanism to access certain
490 private discussion groups over the ARPAnet. In this case, response
491 is still good. When a 9.6Kbps modem is inserted in the path,
492 response went from good to almost tolerable (fortunately the author
493 only reads a few discussion groups in this fashion).
495 To conclude: the POP is a good thing, not only for personal mail
496 but for discussion group mail as well.
501 "Post Office Protocol (revised)", University of Delaware.
504 [MRose85] M.T. Rose, J.L. Romine.
505 "The Rand MH Message Handling System: User's Manual",
506 University of California, Irvine. (November, 1985)
508 [RFC822] D.H. Crocker.
509 "Standard for the Format of ARPA Internet Text
510 Messages", University of Delaware. (August, 1982)
512 [RFC918] J.K. Reynolds.
513 "Post Office Protocol", USC/Information Sciences Institute.
516 [RFC937] M. Butler, J.B. Postel, et. al.
517 "Post Office Protocol - Version 2", USC/Information Sciences