1
2
3
4
5
6
7 Network Working Group K. Moore
8 Request for Comments: 1522 University of Tennessee
9 Obsoletes: 1342 September 1993
10 Category: Standards Track
11
12
13 MIME (Multipurpose Internet Mail Extensions) Part Two:
14 Message Header Extensions for Non-ASCII Text
15
16 Status of this Memo
17
18 This RFC specifies an Internet standards track protocol for the
19 Internet community, and requests discussion and suggestions for
20 improvements. Please refer to the current edition of the "Internet
21 Official Protocol Standards" for the standardization state and status
22 of this protocol. Distribution of this memo is unlimited.
23
24 Abstract
25
26 This memo describes an extension to the message format defined in RFC
27 1521 [1], to allow the representation of character sets other than
28 ASCII in RFC 822 (STD 11) message headers. The extensions described
29 were designed to be highly compatible with existing Internet mail
30 handling software, and to be easily implemented in mail readers that
31 support RFC 1521.
32
33 1. Introduction
34
35 RFC 1521 describes a mechanism for denoting textual body parts which
36 are coded in various character sets, as well as methods for encoding
37 such body parts as sequences of printable ASCII characters. This
38 memo describes similar techniques to allow the encoding of non-ASCII
39 text in various portions of a RFC 822 [2] message header, in a manner
40 which is unlikely to confuse existing message handling software.
41
42 Like the encoding techniques described in RFC 1521, the techniques
43 outlined here were designed to allow the use of non-ASCII characters
44 in message headers in a way which is unlikely to be disturbed by the
45 quirks of existing Internet mail handling programs. In particular,
46 some mail relaying programs are known to (a) delete some message
47 header fields while retaining others, (b) rearrange the order of
48 addresses in To or Cc fields, (c) rearrange the (vertical) order of
49 header fields, and/or (d) "wrap" message headers at different places
50 than those in the original message. In addition, some mail reading
51 programs are known to have difficulty correctly parsing message
52 headers which, while legal according to RFC 822, make use of
53 backslash-quoting to "hide" special characters such as "<", ",", or
54 ":", or which exploit other infrequently-used features of that
55
56
57
58 Moore [Page 1]
59
60 RFC 1522 MIME Part Two September 1993
61
62
63 specification.
64
65 While it is unfortunate that these programs do not correctly
66 interpret RFC 822 headers, to "break" these programs would cause
67 severe operational problems for the Internet mail system. The
68 extensions described in this memo therefore do not rely on little-
69 used features of RFC 822.
70
71 Instead, certain sequences of "ordinary" printable ASCII characters
72 (known as "encoded-words") are reserved for use as encoded data. The
73 syntax of encoded-words is such that they are unlikely to
74 "accidentally" appear as normal text in message headers.
75 Furthermore, the characters used in encoded-words are restricted to
76 those which do not have special meanings in the context in which the
77 encoded-word appears.
78
79 Generally, an "encoded-word" is a sequence of printable ASCII
80 characters that begins with "=?", ends with "?=", and has two "?"s in
81 between. It specifies a character set and an encoding method, and
82 also includes the original text encoded as graphic ASCII characters,
83 according to the rules for that encoding method.
84
85 A mail composer that implements this specification will provide a
86 means of inputting non-ASCII text in header fields, but will
87 translate these fields (or appropriate portions of these fields) into
88 encoded-words before inserting them into the message header.
89
90 A mail reader that implements this specification will recognize
91 encoded-words when they appear in certain portions of the message
92 header. Instead of displaying the encoded-word "as is", it will
93 reverse the encoding and display the original text in the designated
94 character set.
95
96 NOTES
97
98 This memo relies heavily on notation and terms defined STD 11, RFC
99 822 and RFC 1521. In particular, the syntax for the ABNF used in
100 this memo is defined in STD 11, RFC 822, as well as many of the
101 terms used in the grammar for the header extensions defined here.
102 Successful implementation of this protocol extension requires
103 careful attention to the details of both STD 11, RFC 822 and RFC
104 1521.
105
106 When the term "ASCII" appears in this memo, it refers to the "7-
107 Bit American Standard Code for Information Interchange", ANSI
108 X3.4-1986. The MIME charset name for this character set is "US-
109 ASCII". When not specifically referring to the MIME charset name,
110 this document uses the term "ASCII", both for brevity and for
111
112
113
114 Moore [Page 2]
115
116 RFC 1522 MIME Part Two September 1993
117
118
119 consistency with STD 11, RFC 822. However, implementors are
120 warned that the character set name must be spelled "US-ASCII" in
121 MIME message and body part headers.
122
123 2. Syntax of encoded-words
124
125 An "encoded-word" is defined by the following ABNF grammar. The
126 notation of RFC 822 is used, with the exception that white space
127 characters MAY NOT appear between components of an encoded-word.
128
129 encoded-word = "=?" charset "?" encoding "?" encoded-text "?="
130
131 charset = token ; see section 3
132
133 encoding = token ; see section 4
134
135 token = 1*<Any CHAR except SPACE, CTLs, and especials>
136
137 especials = "(" / ")" / "<" / ">" / "@" / "," / ";" / ":" / "
138 <"> / "/" / "[" / "]" / "?" / "." / "="
139
140 encoded-text = 1*<Any printable ASCII character other
141 than "?" or SPACE>
142 ; (but see "Use of encoded-words in message
143 ; headers", section 5)
144
145 Both "encoding" and "charset" names are case-independent. Thus the
146 charset name "ISO-8859-1" is equivalent to "iso-8859-1", and the
147 encoding named "Q" may be spelled either "Q" or "q".
148
149 An encoded-word may not be more than 75 characters long, including
150 charset, encoding, encoded-text, and delimiters. If it is desirable
151 to encode more text than will fit in an encoded-word of 75
152 characters, multiple encoded-words (separated by CRLF SPACE) may be
153 used.
154
155 While there is no limit to the length of a multiple-line header
156 field, each line of a header field that contains one or more
157 encoded-words is limited to 76 characters.
158
159 The length restrictions are included not only to ease
160 interoperability through internetwork mail gateways, but also to
161 impose a limit on the amount of lookahead a header parser must employ
162 (while looking for a final ?= delimiter) before it can decide whether
163 a token is an encoded-word or something else.
164
165 The characters which may appear in encoded-text are further
166 restricted by the rules in section 5.
167
168
169
170 Moore [Page 3]
171
172 RFC 1522 MIME Part Two September 1993
173
174
175 3. Character sets
176
177 The "charset" portion of an encoded-word specifies the character set
178 associated with the unencoded text. A charset can be any of the
179 character set names allowed in an RFC 1521 "charset" parameter of a
180 "text/plain" body part, or any character set name registered with
181 IANA for use with the MIME text/plain content-type [3]. (See section
182 7.1.1 of RFC 1521 for a list of charsets defined in that document).
183
184 Some character sets use code-switching techniques to switch between
185 "ASCII mode" and other modes. If unencoded text in an encoded-word
186 contains control codes to switch out of ASCII mode, it must also
187 contain additional control codes such that ASCII mode is again
188 selected at the end of the encoded-word. (This rule applies
189 separately to each encoded-word, including adjacent encoded-words
190 within a single header field.)
191
192 When there is a possibility of using more than one character set to
193 represent the text in an encoded-word, and in the absence of private
194 agreements between sender and recipients of a message, it is
195 recommended that members of the ISO-8859-* series be used in
196 preference to other character sets.
197
198 4. Encodings
199
200 Initially, the legal values for "encoding" are "Q" and "B". These
201 encodings are described below. The "Q" encoding is recommended for
202 use when most of the characters to be encoded are in the ASCII
203 character set; otherwise, the "B" encoding should be used.
204 Nevertheless, a mail reader which claims to recognize encoded-words
205 MUST be able to accept either encoding for any character set which it
206 supports.
207
208 Only a subset of the printable ASCII characters may be used in
209 encoded-text. Space and tab characters are not allowed, so that the
210 beginning and end of an encoded-word are obvious. The "?" character
211 is used within an encoded-word to separate the various portions of
212 the encoded-word from one another, and thus cannot appear in the
213 encoded-text portion. Other characters are also illegal in certain
214 contexts. For example, an encoded-word in a "phrase" preceding an
215 address in a From header field may not contain any of the "specials"
216 defined in RFC 822. Finally, certain other characters are disallowed
217 in some contexts, to ensure reliability for messages that pass
218 through internetwork mail gateways.
219
220 The "B" encoding automatically meets these requirements. The "Q"
221 encoding allows a wide range of printable characters to be used in
222 non-critical locations in the message header (e.g., Subject), with
223
224
225
226 Moore [Page 4]
227
228 RFC 1522 MIME Part Two September 1993
229
230
231 fewer characters available for use in other locations.
232
233 4.1. The "B" encoding
234
235 The "B" encoding is identical to the "BASE64" encoding defined by RFC
236 1521.
237
238 4.2. The "Q" encoding
239
240 The "Q" encoding is similar to the "Quoted-Printable" content-
241 transfer-encoding defined in RFC 1521. It is designed to allow text
242 containing mostly ASCII characters to be decipherable on an ASCII
243 terminal without decoding.
244
245 (1) Any 8-bit value may be represented by a "=" followed by two
246 hexadecimal digits. For example, if the character set in use
247 were ISO-8859-1, the "=" character would thus be encoded as
248 "=3D", and a SPACE by "=20". (Upper case should be used for
249 hexadecimal digits "A" through "F".)
250
251 (2) The 8-bit hexadecimal value 20 (e.g., ISO-8859-1 SPACE) may be
252 represented as "_" (underscore, ASCII 95.). (This character may
253 not pass through some internetwork mail gateways, but its use
254 will greatly enhance readability of "Q" encoded data with mail
255 readers that do not support this encoding.) Note that the "_"
256 always represents hexadecimal 20, even if the SPACE character
257 occupies a different code position in the character set in use.
258
259 (3) 8-bit values which correspond to printable ASCII characters other
260 than "=", "?", "_" (underscore), and SPACE may be represented as
261 those characters. (But see section 5 for restrictions.)
262
263 5. Use of encoded-words in message headers
264
265 An encoded-word may appear in a message header or body part header
266 according to the following rules:
267
268 (1) An encoded-word may replace a "text" token (as defined by RFC
269 822) in any Subject or Comments header field, any extension
270 message header field, or any RFC 1521 body part field for which
271 the field body is defined as "*text". An encoded-word may also
272 appear in any user-defined ("X-") message or body part header
273 field.
274
275 Ordinary ASCII text and encoded-words may appear together in the
276 same header field. However, an encoded-word that appears in a
277 header field defined as "*text" MUST be separated from any
278 adjacent encoded-word or "text" by linear-white-space.
279
280
281
282 Moore [Page 5]
283
284 RFC 1522 MIME Part Two September 1993
285
286
287 (2) An encoded-word may appear within a comment delimited by "(" and
288 ")", i.e., wherever a "ctext" is allowed. More precisely, the
289 RFC 822 ABNF definition for "comment" is amended as follows:
290
291 comment = "(" *(ctext / quoted-pair / comment / encoded-word) ")"
292
293 A "Q"-encoded encoded-word which appears in a comment MUST NOT
294 contain the characters "(", ")" or " encoded-word that appears in
295 a "comment" MUST be separated from any adjacent encoded-word or
296 "ctext" by linear-white-space.
297
298 (3) As a replacement for a "word" entity within a "phrase", for
299 example, one that precedes an address in a From, To, or Cc
300 header. The ABNF definition for phrase from RFC 822 thus
301 becomes:
302
303 phrase = 1*(encoded-word / word)
304
305 In this case the set of characters that may be used in a "Q"-
306 encoded encoded-word is restricted to: <upper and lower case
307 ASCII letters, decimal digits, "!", "*", "+", "-", "/", "=", and
308 "_" (underscore, ASCII 95.)>. An encoded-word that appears
309 within a "phrase" MUST be separated from any adjacent "word",
310 "text" or "special" by linear-white-space.
311
312 These are the ONLY locations where an encoded-word may appear. In
313 particular, an encoded-word MUST NOT appear in any portion of an
314 "addr-spec". In addition, an encoded-word MUST NOT be used in a
315 Received header field.
316
317 Each encoded-word MUST encode an integral number of octets. The
318 encoded-text in each encoded-word must be well-formed according to
319 the encoding specified; the encoded-text may not be continued in the
320 next encoded-word. (For example, "=?charset?Q?=?= =?charset?Q?AB?="
321 would be illegal, because the two hex digits "AB" must follow the "="
322 in the same encoded-word.)
323
324 Each encoded-word MUST represent an integral number of characters. A
325 multi-octet character may not be split across adjacent encoded-words.
326
327 Only printable and white space character data should be encoded using
328 this scheme. However, since these encoding schemes allow the
329 encoding of arbitrary octet values, mail readers that implement this
330 decoding should also ensure that display of the decoded data on the
331 recipient's terminal will not cause unwanted side-effects.
332
333 Use of these methods to encode non-textual data (e.g., pictures or
334 sounds) is not defined by this memo. Use of encoded-words to
335
336
337
338 Moore [Page 6]
339
340 RFC 1522 MIME Part Two September 1993
341
342
343 represent strings of purely ASCII characters is allowed, but
344 discouraged. In rare cases it may be necessary to encode ordinary
345 text that looks like an encoded-word.
346
347 6. Support of encoded-words by mail readers
348
349 6.1. Recognition of encoded-words in message headers
350
351 A mail reader must parse the message and body part headers according
352 to the rules in RFC 822 to correctly recognize encoded-words.
353
354 Encoded-words are to be recognized as follows:
355
356 (1) Any message or body part header field defined as "*text", or any
357 user-defined header field, should be parsed as follows: Beginning
358 at the start of the field-body and immediately following each
359 occurrence of linear-white-space, each sequence of up to 75
360 printable characters (not containing any linear-white-space)
361 should be examined to see if it is an encoded-word according to
362 the syntax rules in section 2. Any other sequence of printable
363 characters should be treated as ordinary ASCII text.
364
365 (2) Any header field not defined as "*text" should be parsed
366 according to the syntax rules for that header field. However,
367 any "word" that appears within a "phrase" should be treated as an
368 encoded-word if it meets the syntax rules in section 2.
369 Otherwise it should be treated as an ordinary "word".
370
371 (3) Within a "comment", any sequence of up to 75 printable characters
372 (not containing linear-white-space), that meets the syntax rules
373 in section 2, should be treated as an encoded-word. Otherwise it
374 should be treated as normal comment text.
375
376 6.2. Display of encoded-words
377
378 Any encoded-words so recognized are decoded, and if possible, the
379 resulting unencoded text is displayed in the original character set.
380
381 When displaying a particular header field that contains multiple
382 encoded-words, any linear-white-space that separates a pair of
383 adjacent encoded-words is ignored. (This is to allow the use of
384 multiple encoded-words to represent long strings of unencoded text,
385 without having to separate encoded-words where spaces occur in the
386 unencoded text.)
387
388 In the event other encodings are defined in the future, and the mail
389 reader does not support the encoding used, it may either (a) display
390 the encoded-word as ordinary text, or (b) substitute an appropriate
391
392
393
394 Moore [Page 7]
395
396 RFC 1522 MIME Part Two September 1993
397
398
399 message indicating that the text could not be decoded.
400
401 If the mail reader does not support the character set used, it may
402 (a) display the encoded-word as ordinary text (i.e., as it appears in
403 the header), (b) make a "best effort" to display using such
404 characters as are available, or (c) substitute an appropriate message
405 indicating that the decoded text could not be displayed.
406
407 If the character set being used employs code-switching techniques,
408 display of the encoded text implicitly begins in "ASCII mode". In
409 addition, the mail reader must ensure that the output device is once
410 again in "ASCII mode" after the encoded-word is displayed.
411
412 6.3. Mail reader handling of incorrectly formed encoded-words
413
414 It is possible that an encoded-word that is legal according to the
415 syntax defined in section 2, is incorrectly formed according to the
416 rules for the encoding being used. For example:
417
418 (1) An encoded-word which contains characters which are not legal for
419 a particular encoding (for example, a '-' in the "B" encoding),
420 is incorrectly formed.
421
422 (2) Any encoded-word which encodes a non-integral number of
423 characters or octets is incorrectly formed.
424
425 A mail reader need not attempt to display the text associated with an
426 encoded-word that is incorrectly formed. However, a mail reader MUST
427 NOT prevent the display or handling of a message because an encoded-
428 word is incorrectly formed.
429
430 7. Conformance
431
432 A mail composing program claiming compliance with this specification
433 MUST ensure that any string of non-white-space printable ASCII
434 characters within a "*text" or "*ctext" that begins with "=?" and
435 ends with "?=" be a valid encoded-word. ("begins" means: at the
436 start of the field-body or immediately following linear-white-space;
437 "ends" means: at the end of the field-body or immediately preceding
438 linear-white-space.) In addition, any "word" within a "phrase" that
439 begins with "=?" and ends with "?=" must be a valid encoded-word.
440
441 A mail reading program claiming compliance with this specification
442 must be able to distinguish encoded-words from "text", "ctext", or
443 "word"s, according to the rules in section 6, anytime they appear in
444 appropriate places in message headers. It must support both the "B"
445 and "Q" encodings for any character set which it supports. The
446 program must be able to display the unencoded text if the character
447
448
449
450 Moore [Page 8]
451
452 RFC 1522 MIME Part Two September 1993
453
454
455 set is "US-ASCII". For the ISO-8859-* character sets, the mail
456 reading program must at least be able to display the characters which
457 are also in the ASCII set.
458
459 8. Examples
460
461 From: =?US-ASCII?Q?Keith_Moore?= <moore@cs.utk.edu>
462 To: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <keld@dkuug.dk>
463 CC: =?ISO-8859-1?Q?Andr=E9_?= Pirard <PIRARD@vm1.ulg.ac.be>
464 Subject: =?ISO-8859-1?B?SWYgeW91IGNhbiByZWFkIHRoaXMgeW8=?=
465 =?ISO-8859-2?B?dSB1bmRlcnN0YW5kIHRoZSBleGFtcGxlLg==?=
466
467 From: =?ISO-8859-1?Q?Olle_J=E4rnefors?= <ojarnef@admin.kth.se>
468 To: ietf-822@dimacs.rutgers.edu, ojarnef@admin.kth.se
469 Subject: Time for ISO 10646?
470
471 To: Dave Crocker <dcrocker@mordor.stanford.edu>
472 Cc: ietf-822@dimacs.rutgers.edu, paf@comsol.se
473 From: =?ISO-8859-1?Q?Patrik_F=E4ltstr=F6m?= <paf@nada.kth.se>
474 Subject: Re: RFC-HDR care and feeding
475
476 From: Nathaniel Borenstein <nsb@thumper.bellcore.com>
477 (=?iso-8859-8?b?7eXs+SDv4SDp7Oj08A==?=)
478 To: Greg Vaudreuil <gvaudre@NRI.Reston.VA.US>, Ned Freed
479 <ned@innosoft.com>, Keith Moore <moore@cs.utk.edu>
480 Subject: Test of new header generator
481 MIME-Version: 1.0
482 Content-type: text/plain; charset=ISO-8859-1
483
484 9. References
485
486 [1] Borenstein N., and N. Freed, "MIME (Multipurpose Internet Mail
487 Extensions) Part One: Mechanisms for Specifying and Describing
488 the Format of Internet Message Bodies", RFC 1521, Bellcore,
489 Innosoft, September 1993.
490
491 [2] Crocker, D., "Standard for the Format of ARPA Internet Text
492 Messages", STD 11, RFC 822, UDEL, August 1982.
493
494 [3] Reynolds, J., and J. Postel, "Assigned Numbers", STD 2,
495 RFC 1340, USC/Information Sciences Institute, July 1992.
496
497
498
499
500
501
502
503
504
505
506 Moore [Page 9]
507
508 RFC 1522 MIME Part Two September 1993
509
510
511 10. Security Considerations
512
513 Security issues are not discussed in this memo.
514
515 11. Author's Address
516
517 Keith Moore
518 University of Tennessee
519 107 Ayres Hall
520 Knoxville TN 37996-1301
521
522 EMail: moore@cs.utk.edu
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562 Moore [Page 10]
563 
