Technical Documentation: jstedfast/MimeKit
โน๏ธ Provenance: Multi-Tier Ingestion ยท CodeWiki Reference
Overview (`README.md`)
MimeKit
| Package | Latest Release | Latest Build |
|---|---|---|
| MimeKit | ||
| MimeKit.Core | ||
| MimeKit.Cryptography | ||
| MailKit |
| Platform | Build Status | Code Coverage | Static Analysis |
|---|---|---|---|
| Linux/Mac | |||
| Windows |
Obsolete Packages:
| Package | Latest Release |
|---|---|
| MimeKitLite | |
| MailKitLite |
What is MimeKit?
MimeKit is a C# library which may be used for the creation and parsing of messages using the Multipurpose
Internet Mail Extension (MIME), as defined by numerous IETF specifications.
Upgrading from MimeKit 4.x? MimeKit 5.0 contains a number of breaking changes. See the
Porting Guide for step-by-step migration instructions
and the Release Notes for the complete list of changes.
Contents
- Donate
- History
- Performance
- License Information
- Installing via NuGet
- Getting the Source Code
- Building
- Using MimeKit
- Parsing Messages
- Getting the Body of a Message
- Converting HTML to Plain Text
- Converting Enriched Text to HTML
- Traversing a MimeMessage
- Quick and Dirty Enumeration of Message Body Parts
- Getting the Decoded Content of a MIME Part
- Extracting TNEF (winmail.dat) Attachments
- Checking Messages for MIME Compliance Problems
- Creating a Simple Message
- Creating a Message with Attachments
- Creating a Message Using a BodyBuilder (not Arnold Schwarzenegger)
- Initializing MimeKit's Cryptography Support
- Preparing to use MimeKit's S/MIME support
- Preparing to use MimeKit's PGP/MIME support
- Encrypting Messages with S/MIME
- Decrypting S/MIME Messages
- Encrypting Messages with PGP/MIME
- Decrypting PGP/MIME Messages
- Digitally Signing Messages with S/MIME or PGP/MIME
- Verifying S/MIME and PGP/MIME Digital Signatures
- Signing Messages with DKIM
- Verifying DKIM Signatures
- Signing Messages with ARC
- Verifying ARC Signatures
- Verifying DMARC
- Contributing
- Reporting Bugs
- Documentation
- .NET Foundation
Donate
MimeKit is a personal open source project that I have put thousands of hours into perfecting with the
goal of making it the very best MIME parser framework for .NET. I need your help to achieve this.
Donating helps pay for things such as web hosting, domain registration and licenses for developer tools
such as a performance profiler, memory profiler, a static code analysis tool, and more. It also helps
motivate me to continue working on the project.
History
As a developer and user of email clients, I had come to realize that the vast majority of email client
(and server) software had less-than-satisfactory MIME implementations. More often than not these email clients
created broken MIME messages and/or would incorrectly try to parse a MIME message thus subtracting from the full
benefits that MIME was meant to provide. MimeKit is meant to address this issue by following the MIME specification
as closely as possible while also providing programmers with an extremely easy to use high-level API.
This led me, at first, to implement another MIME parser library called GMime
which is implemented in C and later added a C# binding called GMime-Sharp.
Now that I typically find myself working in C# rather than lower level languages like C, I decided to
begin writing a new parser in C# which would not depend on GMime. This would also allow me to have more
flexibility in that I'd be able to use Generics and create a more .NET-compliant API.
Performance
Performance has been a core design goal of MimeKit from the very beginning. MimeKit's parser works directly on
raw byte buffers rather than first converting the input into strings or splitting it into lines, which keeps
both CPU time and memory allocations to a minimum. Some of the techniques it relies on:
- A low-level, allocation-conscious streaming reader (
MimeReader) that the higher-levelMimeParserbuilds on. - Pooled buffers and lazily-decoded headers so that only the data you actually access is ever decoded.
- Hardware-accelerated (SIMD) scanning and base64/uuencode encoding and decoding on x64 and ARM64 platforms
when supported by the runtime. - Source stream storage (
MimeParserwithstorage: MimeContentStorage.Source) so that MIME part content can reference
the original stream instead of being copied into memory.
The result is a parser that can comfortably process very large messages and multi-gigabyte mbox files while
remaining spec-compliant and resilient against malformed or malicious input.
Long before .NET Core made SIMD available to C# developers, MimeKit was already fast thanks to a number of
parser design choices and optimizations that are described in more detail in the following blog posts:
- Optimization Tips & Tricks used by MimeKit: Part 1
- Optimization Tips & Tricks used by MimeKit: Part 2
If you are curious about the details, the Benchmarks
directory contains a BenchmarkDotNet suite covering the parser, encoders,
decoders, filters and more, which you can run on your own hardware.
License Information
MIT License
Copyright (C) 2012-2026 .NET Foundation and Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.Installing via NuGet
The easiest way to install MimeKit is via NuGet:
dotnet add package MimeKitOr, in Visual Studio's Package Manager Console:
Install-Package MimeKitMimeKit is split into the following packages:
- MimeKit.Core - MIME parsing and message construction, along with
everything else that does not require cryptography (address parsing, encoders and decoders, HTML, RTF and text
converters, TNEF, etc.). - MimeKit.Cryptography - S/MIME, PGP/MIME, DKIM, ARC and DMARC
support. It depends on MimeKit.Core and BouncyCastle. - MimeKit - a meta-package that references both MimeKit.Core and
MimeKit.Cryptography.
If you don't need any of the cryptography features, you can reference MimeKit.Core instead:
dotnet add package MimeKit.CoreIf you do use the cryptography features, see Initializing MimeKit's Cryptography Support.
Getting the Source Code
To clone MimeKit's source code:
git clone https://github.com/jstedfast/MimeKit.gitTo update an existing clone later:
git pullBuilding
In the top-level MimeKit directory, you'll find the following solution file:
- MimeKit.sln - includes the MimeKit.Core and MimeKit.Cryptography projects (targeting .NET Framework 4.6.2/4.7/4.8,
.NETStandard 2.0/2.1, .NET 8.0 and .NET 10.0) as well as the unit tests.
Once you've opened the MimeKit solution file in Visual Studio,
you can choose the Debug or Release build configuration and then build.
Building MimeKit requires the .NET 10.0 SDK (or a version of Visual Studio that
includes it). The unit tests target .NET 8.0, so you will also need the .NET 8.0 runtime in order to run them.
You can also build MimeKit from the command line:
dotnet build MimeKit.sln -c ReleaseNote: The .NET Framework 4.8 target is only built on Windows.
Regenerating the HTML entity decoder
The standalone tools\HtmlEntityDecoderGenerator project generatesMimeKit\Text\HtmlEntityDecoder.g.cs from the checked-inUnitTests\TestData\html\HtmlEntities.json snapshot, which is also used by the decoder tests.
From the repository root, run:
dotnet run --project tools\HtmlEntityDecoderGeneratorThe tool requires the .NET 10.0 SDK and does not download entity data. It preserves the snapshot's
entity order and writes UTF-8 without a BOM, with CRLF line endings. Optional arguments-- <entities.json> [<output.cs>] override the input and output paths. After changing the snapshot,
regenerate the decoder and run the UnitTests.Text.HtmlEntityDecoderTests fixture.
Using MimeKit
Parsing Messages
One of the more common operations that MimeKit is meant for is parsing email messages from arbitrary streams.
There are two ways of accomplishing this task.
The first way is to use one of the Load methods
on MimeMessage:
// Load a MimeMessage from a stream
var message = MimeMessage.Load (stream);Or you can load a message from a file path:
// Load a MimeMessage from a file path
var message = MimeMessage.Load ("message.eml");The second way is to use the MimeParser class. For the most
part, using the MimeParser directly is not necessary unless you wish to parse a Unix mbox file stream. However, this is
how you would do it:
// Load a MimeMessage from a stream
var parser = new MimeParser (stream, MimeFormat.Entity);
var message = parser.ParseMessage ();For Unix mbox file streams, you would use the parser like this:
// Load every message from a Unix mbox
var parser = new MimeParser (stream, MimeFormat.Mbox);
while (!parser.IsEndOfStream) {
var message = parser.ParseMessage ();
// do something with the message
}Getting the Body of a Message
A common misunderstanding about email is that there is a well-defined message body and then a list
of attachments. This is not really the case. The reality is that MIME is a tree structure of content,
much like a file system.
Luckily, MIME does define a set of general rules for how mail clients should interpret this tree
structure of MIME parts. The Content-Disposition header is meant to provide hints to the receiving
client as to which parts are meant to be displayed as part of the message body and which are meant
to be interpreted as attachments.
The Content-Disposition header will generally have one of two values: inline or attachment.
The meaning of these values should be fairly obvious. If the value is attachment, then the content
of said MIME part is meant to be presented as a file attachment separate from the core message.
However, if the value is inline, then the content of that MIME part is meant to be displayed inline
within the mail client's rendering of the core message body. If the Content-Disposition header does
not exist, then it should be treated as if the value were inline.
Technically, every part that lacks a Content-Disposition header or that is marked as inline, then,
is part of the core message body.
There's a bit more to it than that, though.
Modern MIME messages will often contain a multipart/alternative MIME container which will generally contain
a text/plain and text/html version of the text that the sender wrote. The text/html version is typically
formatted much closer to what the sender saw in his or her WYSIWYG editor than the text/plain version.
The reason for sending the message text in both formats is that not all mail clients are capable of displaying
HTML.
The receiving client should only display one of the alternative views contained within the multipart/alternative
container. Since alternative views are listed in order of least faithful to most faithful with what the sender
saw in his or her WYSIWYG editor, the receiving client should walk over the list of alternative views starting
at the end and working backwards until it finds a part that it is capable of displaying.
Example:
multipart/alternative
text/plain
text/htmlAs seen in the example above, the text/html part is listed last because it is the most faithful to
what the sender saw in his or her WYSIWYG editor when writing the message.
To make matters even more complicated, sometimes modern mail clients will use a multipart/related
MIME container instead of a simple text/html part in order to embed images and other content
within the HTML.
Example:
multipart/alternative
text/plain
multipart/related
text/html
image/jpeg
video/mp4
image/pngIn the example above, one of the alternative views is a multipart/related container which contains
an HTML version of the message body that references the sibling video and images.
Now that you have a rough idea of how a message is structured and how to interpret various MIME entities,
the next step is learning how to traverse the MIME tree using MimeKit.
Note: For your convenience, MimeKit's MimeMessage class has two properties that can help you get thetext/plain or text/html version of the message body. These are TextBody and HtmlBody,
respectively.
Keep in mind, however, that at least with the HtmlBody property, it may be that the HTML part is
a child of a multipart/related, allowing it to refer to images and other types of media that
are also contained within that multipart/related entity. This property is really only a convenience
property and is not a really good substitute for traversing the MIME structure yourself so that you
may properly interpret related content.
Converting HTML to Plain Text
Use MimeKit.Text.HtmlToText to turn an HTML document or fragment into readable plain text:
var converter = new MimeKit.Text.HtmlToText {
OutputLinkUrls = true,
OutputImageAltText = true
};
string text = converter.Convert ("<h1>Hello</h1><p>Visit <a href=\"https://example.com\">Example</a>.</p>");The result is:
# Hello
Visit Example (https://example.com).The converter preserves paragraph breaks, nested lists, blockquotes and preformatted whitespace.
It accepts strings, streams or a TextReader/TextWriter pair. Output is not word-wrapped;
line endings follow the destination writer's NewLine property. Headers and footers are literal
text, and caller-owned readers, writers and streams remain open.
This is a structural text conversion, not browser rendering or Markdown serialization. CSS and
inline emphasis styling are not reproduced; tables are flattened without borders or aligned columns.
Script, style, title, template and hidden-element content is omitted, and external resources are
never fetched. Link URLs and image alt text are enabled by default and can be disabled independently.
For untrusted input, MaxElementDepth defaults to 4096, MaxInputLength to 16,777,216 UTF-16
characters and MaxOutputLength to 67,108,864 UTF-16 characters. The input limit includes markup
and suppressed content; the output limit includes generated indentation, URLs and line endings.
The literal header and footer are excluded from these length limits. Exceeding a limit throwsFormatException; a streaming destination may already contain partial output.
Converting Enriched Text to HTML
Use MimeKit.Text.EnrichedToHtml to convert RFC 1896 text/enriched content:
var converter = new MimeKit.Text.EnrichedToHtml {
OutputHtmlFragment = true
};
string html = converter.Convert ("<bold>Hello</bold> <color><param>red</param>world</color>!");The converter supports emphasis, font families, colors, relative font sizes, alignment,
paragraph indentation, excerpts, language tags and nofill. It applies enriched-text newline
folding and escapes literal text, including << for a literal less-than sign. Unknown commands
are ignored, their body text is retained, and parameter content is never displayed. This does
not support the older text/richtext format.
Strings, streams and TextReader/TextWriter pairs are supported. By default the result is
wrapped in <html>; set OutputHtmlFragment to omit that wrapper. Caller-owned readers,
writers and streams remain open. Text headers and footers are converted with TextToHtml;
headers and footers explicitly marked as HTML are trusted markup, not sanitized input.
For untrusted input, MaxElementDepth and MaxParameterLength default to 4096, MaxInputLength
to 16,777,216 UTF-16 characters and MaxOutputLength to 67,108,864 UTF-16 characters. The input
limit includes commands and suppressed parameters, but excludes configured headers and footers.
The output limit includes all markup, escaping, headers and footers. Exceeding a limit throwsFormatException; a streaming destination may already contain partial output.
Traversing a MimeMessage
The MimeMessage.Body is the top-level MIME entity of the message. Generally, it will either be aTextPart or a Multipart.
As an example, if you wanted to rip out all of the attachments of a message, your code might look
something like this:
var attachments = new List<MimePart> ();
var multiparts = new List<Multipart> ();
var iter = new MimeIterator (message);
// collect our list of attachments and their parent multiparts
while (iter.MoveNext ()) {
var multipart = iter.Parent as Multipart;
var part = iter.Current as MimePart;
if (multipart != null && part != null && part.IsAttachment) {
// keep track of each attachment's parent multipart
multiparts.Add (multipart);
attachments.Add (part);
}
}
// now remove each attachment from its parent multipart...
for (int i = 0; i < attachments.Count; i++)
multiparts[i].Remove (attachments[i]);Quick and Dirty Enumeration of Message Body Parts
If you would rather skip the proper way of traversing a MIME tree, another option that MimeKit provides
is a simple enumerator over the message's body parts in a flat (depth-first) list.
You can access this flat list via the BodyParts property, like so:
foreach (var part in message.BodyParts) {
// do something
}Another helper property on the MimeMessage class is the Attachments property which works
much the same way as the BodyParts property except that it will only contain MIME parts
which have a Content-Disposition header value that is set to attachment.
Getting the Decoded Content of a MIME Part
At some point, you're going to want to extract the decoded content of a MimePart (such as an image) and
save it to disk or feed it to a UI control to display it.
Once you've found the MimePart object that you'd like to extract the content of, here's how you can
save the decoded content to a file:
// This will get the name of the file as specified by the sending mail client.
// Note: this value *may* be null, so you'll want to handle that case in your code.
var fileName = part.FileName;
using (var stream = File.Create (fileName)) {
part.Content.DecodeTo (stream);
}You can also get access to the original raw content by "opening" the Content. This might be useful
if you want to pass the content off to a UI control that can do its own loading from a stream.
using (var stream = part.Content.Open ()) {
// At this point, you can now read from the stream as if it were the original,
// raw content. Assuming you have an image UI control that could load from a
// stream, you could do something like this:
imageControl.Load (stream);
}There are a number of useful filters that can be applied to a FilteredStream, so if you find this type of
interface appealing, I suggest taking a look at the available filters in the MimeKit.IO.Filters namespace
or even write your own! The possibilities are limited only by your imagination.
Extracting TNEF (winmail.dat) Attachments
Outlook and Exchange sometimes send messages with an application/ms-tnef attachment, usually namedwinmail.dat. It holds the message's real bodies and attachments, packed in Microsoft's Transport Neutral
Encapsulation Format, which most mail clients can't read. MimeKit parses these attachments as TnefParts.
The easiest way to handle a TnefPart is to convert it to MIME:
foreach (var tnefPart in message.BodyParts.OfType<TnefPart> ()) {
using (var tnef = tnefPart.LoadTnefMessage ())
using (var result = tnef.ConvertToMime ()) {
// result.Message is a regular MimeMessage with the bodies and attachments from the TNEF part.
// It doesn't depend on the TnefMessage, but it is disposed along with the result.
foreach (var attachment in result.Message.Attachments.OfType<MimePart> ()) {
// The file name comes from the sender, so never use it as a path without sanitizing it.
var fileName = Path.GetFileName (attachment.FileName ?? "attachment.dat");
using (var stream = File.Create (fileName))
attachment.Content.DecodeTo (stream);
}
// result.Losses lists anything that couldn't be represented in MIME.
}
}If you don't need MIME, TnefMessage exposes the Subject, Recipients, TextBody, HtmlBody, RtfBody,Attachments and raw MAPI Properties directly. TnefMessage.Load (stream) loads a standalone winmail.dat
file. TNEF data is untrusted input, so TnefOptions limits nesting depth, attachment count and data sizes.
If you're porting TNEF code from MimeKit 4.x, see the
FAQ and the TNEF section of the Porting Guide.
Checking Messages for MIME Compliance Problems
MimeKit's parser is very forgiving: it will happily parse messages that violate the MIME specifications in
all sorts of ways. Sometimes, though, you'll want to know about those violations, whether you are debugging
a mail client, rejecting malformed messages, or looking for messages that different mail components might
interpret differently (a common technique used to smuggle content past spam and malware filters).
To find out, set the ComplianceLogger property of the MimeParser (or MimeReader) to an
IMimeComplianceLogger implementation.
The parser will then report a MimeComplianceIssue
for each problem that it finds, including the type of violation, its severity, its categories (interoperability,
data loss and/or security) and where in the stream it was found:
class ConsoleComplianceLogger : IMimeComplianceLogger
{
public void Log (in MimeComplianceIssue issue)
{
Console.WriteLine ("{0}:{1}: {2}: {3} [{4}]", issue.LineNumber, issue.ColumnNumber,
issue.Severity, issue.Description, issue.Violation);
}
}
// ...
using (var stream = File.OpenRead ("message.eml")) {
var parser = new MimeParser (stream) {
ComplianceLogger = new ConsoleComplianceLogger (),
ComplianceOptions = new MimeComplianceOptions {
// Since the issues can be produced by untrusted input, limit how many times each one gets reported.
MaxIssuesPerViolation = 10
}
};
var message = parser.ParseMessage ();
}If you only need the compliance report and not the MimeMessage, use a MimeReader instead; it scans the message
without building the object tree. For a complete example, see the
MimeAnalyzer sample, which reports the issues
in a compiler-style file:line:column: severity: message format.
HtmlTokenizer/HtmlToHtml, RtfToText/RtfToHtml and TnefReader have similar ComplianceLogger properties of their own.
Creating a Simple Message
Creating MIME messages using MimeKit is really trivial.
var message = new MimeMessage ();
message.From.Add (new MailboxAddress ("Joey", "[email protected]"));
message.To.Add (new MailboxAddress ("Alice", "[email protected]"));
message.Subject = "How you doin?";
message.Body = new TextPart ("plain") {
Text = @"Hey Alice,
What are you up to this weekend? Monica is throwing one of her parties on
Saturday and I was hoping you could make it.
Will you be my +1?
-- Joey
"
};A TextPart is a leaf-node MIME part with a text media-type. The first argument to the TextPart constructor
specifies the media-subtype, in this case, "plain". Another media subtype you are probably familiar with
is the "html" subtype. Some other examples include "enriched", "rtf", and "csv".
The Text property is the easiest way to both get and set the string content of the MIME part.
Creating a Message with Attachments
Attachments are just like any other MimePart, the only difference is that they typically have
a Content-Disposition header with a value of "attachment" instead of "inline" or noContent-Disposition header at all.
Typically, when a mail client adds attachments to a message, it will create a multipart/mixed
part and add the text body part and all of the file attachments to the multipart/mixed.
Here's how you can do that with MimeKit:
var message = new MimeMessage ();
message.From.Add (new MailboxAddress ("Joey", "[email protected]"));
message.To.Add (new MailboxAddress ("Alice", "[email protected]"));
message.Subject = "How you doin?";
// create our message text, just like before (except don't set it as the message.Body)
var body = new TextPart ("plain") {
Text = @"Hey Alice,
What are you up to this weekend? Monica is throwing one of her parties on
Saturday and I was hoping you could make it.
Will you be my +1?
-- Joey
"
};
// create an image attachment for the file located at path
var attachment = new MimePart ("image", "gif") {
Content = new MimeContent (File.OpenRead (path), ContentEncoding.Unspecified),
ContentDisposition = new ContentDisposition (ContentDisposition.Attachment),
ContentTransferEncoding = ContentEncoding.Base64,
FileName = Path.GetFileName (path)
};
// now create the multipart/mixed container to hold the message text and the
// image attachment
var multipart = new Multipart ("mixed");
multipart.Add (body);
multipart.Add (attachment);
// now set the multipart/mixed as the message body
message.Body = multipart;Of course, that is just a simple example. A lot of modern mail clients such as Outlook or Thunderbird will
send out both a text/html and a text/plain version of the message text. To do this, you'd create aTextPart for the text/plain part and another TextPart for the text/html part and then add them to amultipart/alternative like so:
var attachment = CreateAttachment ();
var plain = CreateTextPlainPart ();
var html = CreateTextHtmlPart ();
// Note: it is important that the text/html part is added second, because it is the
// most expressive version and (probably) the most faithful to the sender's WYSIWYG
// editor.
var alternative = new Multipart ("alternative");
alternative.Add (plain);
alternative.Add (html);
// now create the multipart/mixed container to hold the multipart/alternative
// and the image attachment
var multipart = new Multipart ("mixed");
multipart.Add (alternative);
multipart.Add (attachment);
// now set the multipart/mixed as the message body
message.Body = multipart;Creating a Message Using a BodyBuilder (not Arnold Schwarzenegger)
If you are used to System.Net.Mail's API for creating messages, you will probably find using a BodyBuilder
much more friendly than manually creating the tree of MIME parts. Here's how you could create a message body
using a BodyBuilder:
var message = new MimeMessage ();
message.From.Add (new MailboxAddress ("Joey", "[email protected]"));
message.To.Add (new MailboxAddress ("Alice", "[email protected]"));
message.Subject = "How you doin?";
var builder = new BodyBuilder ();
// Set the plain-text version of the message text
builder.TextBody = @"Hey Alice,
What are you up to this weekend? Monica is throwing one of her parties on
Saturday and I was hoping you could make it.
Will you be my +1?
-- Joey
";
// generate a Content-Id for the image we'll be referencing
var contentId = MimeUtils.GenerateMessageId ();
// Set the html version of the message text
builder.HtmlBody = string.Format (@"<p>Hey Alice,<br>
<p>What are you up to this weekend? Monica is throwing one of her parties on
Saturday and I was hoping you could make it.<br>
<p>Will you be my +1?<br>
<p>-- Joey<br>
<center><img src=""cid:{0}"" alt=""selfie.jpg""></center>", contentId);
// Since selfie.jpg is referenced from the html text, we'll need to add it
// to builder.LinkedResources and then set the Content-Id header value
builder.LinkedResources.Add (@"C:\Users\Joey\Documents\Selfies\selfie.jpg");
builder.LinkedResources[0].ContentId = contentId;
// We may also want to attach a calendar event for Monica's party...
builder.Attachments.Add (@"C:\Users\Joey\Documents\party.ics");
// Now we just need to set the message body and we're done
message.Body = builder.ToMessageBody ();Initializing MimeKit's Cryptography Support
As of MimeKit 5.0, MimeKit's S/MIME, PGP/MIME, DKIM, ARC and DMARC support lives in the MimeKit.Cryptography
assembly. In order for the parser to create cryptographic MIME entities such as MultipartSigned,MultipartEncrypted and ApplicationPkcs7Mime (instead of a plain Multipart or MimePart), the
cryptography module needs to be initialized before any messages are parsed. The best place to do this
is during your application's startup:
using MimeKit.Cryptography;
CryptographyModule.Initialize ();Note: The cryptography module is also initialized automatically the first time the CryptographyContext
class is used (for example, when calling CryptographyContext.Register ()), and it is safe to callCryptographyModule.Initialize () more than once.
Preparing to use MimeKit's S/MIME support
Before you can begin using MimeKit's S/MIME support, you will need to decide which
S/MIME implementation to use.
If your application runs exclusively on Windows, you'll probably want to use theWindowsSecureMimeContext backend which is based on the Windows Certificate
Store and uses Microsoft's built-in S/MIME implementation.
On the other hand, if your application or service runs on other platforms such as Linux,
macOS, iOS, or Android or is expected to run cross-platform, then you will need to use
one of the BouncyCastle-based S/MIME context classes:
- The
TemporarySecureMimeContextbackend uses an in-memory certificate store and will
require your application to load all of the certificates it expects to use into the
context before it can be used. - The
DefaultSecureMimeContextbackend is designed to use a SQL database for certificate
storage such as System.Data.SQLite.
If you opt for using the DefaultSecureMimeContext backend, you'll need to implement
your own DefaultSecureMimeContext subclass. Luckily, it's very simple to do.
Assuming you've chosen System.Data.SQLite, here's how you'd implement your ownDefaultSecureMimeContext class:
using System.Data.SQLite;
using MimeKit.Cryptography;
namespace MyAppNamespace {
class MySecureMimeContext : DefaultSecureMimeContext
{
public MySecureMimeContext () : base (OpenDatabase ("C:\\wherever\\certdb.sqlite"))
{
}
static IX509CertificateDatabase OpenDatabase (string fileName)
{
var builder = new SQLiteConnectionStringBuilder ();
builder.DateTimeFormat = SQLiteDateFormats.Ticks;
builder.DataSource = fileName;
if (!File.Exists (fileName))
SQLiteConnection.CreateFile (fileName);
var sqlite = new SQLiteConnection (builder.ConnectionString);
sqlite.Open ();
return new SqliteCertificateDatabase (sqlite, "password");
}
}
}Once you've implemented or chosen a suitable SecureMimeContext, you'll need to register it with MimeKit:
CryptographyContext.Register (typeof (MySecureMimeContext));Now you are ready to encrypt, decrypt, sign and verify S/MIME messages!
Note: If you choose to use the WindowsSecureMimeContext or TemporarySecureMimeContext backend,
you should register that class instead.
Preparing to use MimeKit's PGP/MIME support
Like with S/MIME support, you also need to register your own OpenPgpContext. Unlike S/MIME, however,
you don't need to choose a database if you subclass GnuPGContext because it uses GnuPG's PGP keyrings
to load and store public and private keys. If you choose to subclass GnuPGContext, the only thing you
you need to do is implement a password callback method:
using MimeKit.Cryptography;
namespace MyAppNamespace {
class MyGnuPGContext : GnuPGContext
{
public MyGnuPgContext () : base ()
{
}
protected override string GetPasswordForKey (PgpSecretKey key)
{
// prompt the user (or a secure password cache) for the password for the specified secret key.
return "password";
}
}
}Once again, to register your OpenPgpContext, you can use the following code snippet:
CryptographyContext.Register (typeof (MyGnuPGContext));Now you are ready to encrypt, decrypt, sign and verify PGP/MIME messages!
Encrypting Messages with S/MIME
S/MIME uses an application/pkcs7-mime MIME part to encapsulate encrypted content (as well as other things).
var joey = new MailboxAddress ("Joey", "[email protected]");
var alice = new MailboxAddress ("Alice", "[email protected]");
var message = new MimeMessage ();
message.From.Add (joey);
message.To.Add (alice);
message.Subject = "How you doin?";
// create our message body (perhaps a multipart/mixed with the message text and some
// image attachments, for example)
var body = CreateMessageBody ();
// now to encrypt our message body using our custom S/MIME cryptography context
using (var ctx = new MySecureMimeContext ()) {
// Note: this assumes that "Alice" has an S/MIME certificate with an X.509
// Subject Email identifier that matches her email address. If she doesn't,
// try using a SecureMailboxAddress which allows you to specify the
// fingerprint of her certificate to use for lookups.
message.Body = ApplicationPkcs7Mime.Encrypt (ctx, message.To.Mailboxes, body);
}Decrypting S/MIME Messages
As mentioned earlier, S/MIME uses an application/pkcs7-mime part with an "smime-type" parameter with a value of
"enveloped-data" to encapsulate the encrypted content.
The first thing you must do is find the ApplicationPkcs7Mime part (see the section on traversing MIME parts).
if (entity is ApplicationPkcs7Mime) {
var pkcs7 = (ApplicationPkcs7Mime) entity;
if (pkcs7.SecureMimeType == SecureMimeType.EnvelopedData)
return pkcs7.Decrypt ();
}Encrypting Messages with PGP/MIME
Unlike S/MIME, PGP/MIME uses multipart/encrypted to encapsulate its encrypted data.
var joey = new MailboxAddress ("Joey", "[email protected]");
var alice = new MailboxAddress ("Alice", "[email protected]");
var message = new MimeMessage ();
message.From.Add (joey);
message.To.Add (alice);
message.Subject = "How you doin?";
// create our message body (perhaps a multipart/mixed with the message text and some
// image attachments, for example)
var body = CreateMessageBody ();
// now to encrypt our message body using our custom PGP/MIME cryptography context
using (var ctx = new MyGnuPGContext ()) {
// Note: this assumes that "Alice" has a public PGP key that matches her email
// address. If she doesn't, try using a SecureMailboxAddress which allows you
// to specify the fingerprint of her public PGP key to use for lookups.
message.Body = MultipartEncrypted.Encrypt (ctx, message.To.Mailboxes, body);
}Decrypting PGP/MIME Messages
As mentioned earlier, PGP/MIME uses a multipart/encrypted part to encapsulate the encrypted content.
A multipart/encrypted contains exactly 2 parts: the first MimeEntity is the version information while the
second MimeEntity is the actual encrypted content and will typically be an application/octet-stream.
The first thing you must do is find the MultipartEncrypted part (see the section on traversing MIME parts).
if (entity is MultipartEncrypted) {
var encrypted = (MultipartEncrypted) entity;
return encrypted.Decrypt ();
}Digitally Signing Messages with S/MIME or PGP/MIME
Both S/MIME and PGP/MIME use a multipart/signed to contain the signed content and the detached signature data.
Here's how you might digitally sign a message using S/MIME:
var joey = new MailboxAddress ("Joey", "[email protected]");
var alice = new MailboxAddress ("Alice", "[email protected]");
var message = new MimeMessage ();
message.From.Add (joey);
message.To.Add (alice);
message.Subject = "How you doin?";
// create our message body (perhaps a multipart/mixed with the message text and some
// image attachments, for example)
var body = CreateMessageBody ();
// now to digitally sign our message body using our custom S/MIME cryptography context
using (var ctx = new MySecureMimeContext ()) {
// Note: this assumes that "Joey" has an S/MIME signing certificate and private key
// with an X.509 Subject Email identifier that matches Joey's email address.
message.Body = MultipartSigned.Create (ctx, joey, DigestAlgorithm.Sha1, body);
}For S/MIME, if you have a way for the user to configure which S/MIME certificate to use
as their signing certificate, you could also do something more like this:
// now to digitally sign our message body using our custom S/MIME cryptography context
using (var ctx = new MySecureMimeContext ()) {
var certificate = GetJoeysX509Certificate ();
var signer = new CmsSigner (certificate);
signer.DigestAlgorithm = DigestAlgorithm.Sha1;
message.Body = MultipartSigned.Create (ctx, signer, body);
}If you'd prefer to use PGP instead of S/MIME, things work almost exactly the same except that you
would use an OpenPGP cryptography context. For example, you might use a subclass of theGnuPGContext that comes with MimeKit if you want to re-use the user's GnuPG keyrings (you can't
use GnuPGContext directly because it has no way of prompting the user for their passphrase).
For the sake of this example, let's pretend that you've written a minimal subclass ofMimeKit.Cryptography.GnuPGContext that only overrides the GetPassword() method and
that this subclass is called MyGnuPGContext.
// now to digitally sign our message body using our custom OpenPGP cryptography context
using (var ctx = new MyGnuPGContext ()) {
// Note: this assumes that "Joey" has a PGP key that matches his email address.
message.Body = MultipartSigned.Create (ctx, joey, DigestAlgorithm.Sha1, body);
}Just like S/MIME, however, you can also do your own PGP key lookups instead of
relying on email addresses to match up with the user's private key.
// now to digitally sign our message body using our custom OpenPGP cryptography context
using (var ctx = new MyGnuPGContext ()) {
var key = GetJoeysPrivatePgpKey ();
message.Body = MultipartSigned.Create (ctx, key, DigestAlgorithm.Sha1, body);
}Verifying S/MIME and PGP/MIME Digital Signatures
As mentioned earlier, both S/MIME and PGP/MIME typically use a multipart/signed part to contain the
signed content and the detached signature data.
A multipart/signed contains exactly 2 parts: the first MimeEntity is the signed content while the secondMimeEntity is the detached signature and, by default, will either be an ApplicationPgpSignature part or
an ApplicationPkcs7Signature part (depending on whether the sending client signed using OpenPGP or S/MIME).
Because the multipart/signed part may have been signed by multiple signers, it is important to
verify each of the digital signatures (one for each signer) that are returned by theMultipartSigned.Verify() method:
if (entity is MultipartSigned) {
var signed = (MultipartSigned) entity;
foreach (var signature in signed.Verify ()) {
try {
bool valid = signature.Verify ();
// If valid is true, then it signifies that the signed content has not been
// modified since this particular signer signed the content.
//
// However, if it is false, then it indicates that the signed content has
// been modified.
} catch (DigitalSignatureVerifyException) {
// There was an error verifying the signature.
}
}
}It should be noted, however, that while most S/MIME clients will use the preferred multipart/signed
approach, it is possible that you may encounter an application/pkcs7-mime part with an "smime-type"
parameter set to "signed-data". Luckily, MimeKit can handle this format as well:
if (entity is ApplicationPkcs7Mime) {
var pkcs7 = (ApplicationPkcs7Mime) entity;
if (pkcs7.SecureMimeType == SecureMimeType.SignedData) {
// extract the original content and get a list of signatures
MimeEntity extracted;
// Note: if you are rendering the message, you'll want to render the
// extracted mime part rather than the application/pkcs7-mime part.
foreach (var signature in pkcs7.Verify (out extracted)) {
try {
bool valid = signature.Verify ();
// If valid is true, then it signifies that the signed content has not
// been modified since this particular signer signed the content.
//
// However, if it is false, then it indicates that the signed content
// has been modified.
} catch (DigitalSignatureVerifyException) {
// There was an error verifying the signature.
}
}
}
}Signing Messages with DKIM
In addition to OpenPGP and S/MIME, MimeKit also supports DKIM signatures. To sign a message using DKIM,
you'll first need a private key. In the following example, assume that the private key is saved in a
file called privatekey.pem:
var headers = new HeaderId[] { HeaderId.From, HeaderId.Subject, HeaderId.Date };
var signer = new DkimSigner ("privatekey.pem", "example.com", "brisbane", DkimSignatureAlgorithm.RsaSha256) {
HeaderCanonicalizationAlgorithm = DkimCanonicalizationAlgorithm.Simple,
BodyCanonicalizationAlgorithm = DkimCanonicalizationAlgorithm.Simple,
AgentOrUserIdentifier = "@eng.example.com",
QueryMethod = "dns/txt",
};
// Prepare the message body to be sent over a 7bit transport (such as older versions of SMTP).
// Note: If the SMTP server you will be sending the message over supports the 8BITMIME extension,
// then you can use `EncodingConstraint.EightBit` instead.
message.Prepare (EncodingConstraint.SevenBit);
signer.Sign (message, headers);As you can see, it's fairly straight forward.
Verifying DKIM Signatures
Verifying DKIM signatures is slightly more involved than creating them because you'll need to write a custom
implementation of the IDnsResolver interface. Typically, this custom class will use your chosen DNS library
to look up the DNS TXT records requested by MimeKit during verification of DKIM signature headers. MimeKit
takes care of parsing the DKIM public key records, so all your resolver needs to do is return the raw TXT
record strings along with whether the domain exists or the lookup failed.
Once you've implemented a custom IDnsResolver, verifying signatures is fairly trivial. Most of the work
needed will be in the IDnsResolver implementation. As an example of how to implement this interface,
here is one possible implementation using the DnsClient library:
/* Detailed source-code truncated for AI context efficiency. */Signing Messages with ARC
Signing with ARC is similar to DKIM but quite a bit more involved. In order to sign with
ARC, you must first validate that the existing message is authentictic and produce
an ARC-Authentication-Results header containing the methods that you used to
authenticate the message as well as their results.
The abstract ArcSigner
class provided by MimeKit will need to be subclassed before it can be used. An example subclass
that provides 2 different implementations for generating the ARC-Authentication-Results header
can be seen below:
/* Detailed source-code truncated for AI context efficiency. */Once you have a custom ArcSigner class, the actual logic for signing is almost identical to DKIM.
Note: As with the DKIM signing example above, assume that the private key is saved in a
file called privatekey.pem:
var headers = new HeaderId[] { HeaderId.From, HeaderId.Subject, HeaderId.Date };
var signer = new MyArcSigner ("privatekey.pem", "example.com", "brisbane", DkimSignatureAlgorithm.RsaSha256) {
HeaderCanonicalizationAlgorithm = DkimCanonicalizationAlgorithm.Relaxed,
BodyCanonicalizationAlgorithm = DkimCanonicalizationAlgorithm.Relaxed,
AgentOrUserIdentifier = "@eng.example.com"
};
// Prepare the message body to be sent over a 7bit transport (such as older versions of SMTP).
// Note: If the SMTP server you will be sending the message over supports the 8BITMIME extension,
// then you can use `EncodingConstraint.EightBit` instead.
message.Prepare (EncodingConstraint.SevenBit);
signer.Sign (message, headers); // or SignAsyncVerifying ARC Signatures
Just like with verifying DKIM signatures, you will need to implement the IDnsResolver
interface. To see an example of how to implement this interface, see the DKIM signature verification
example above.
The ArcVerifier works exactly the same as the DkimVerifier except that it is not necessary
to provide a Header argument to the Verify or VerifyAsync method.
var verifier = new ArcVerifier (new DnsResolver ());
var results = await verifier.VerifyAsync (message);
// The Chain results are the only real important results.
Console.WriteLine ("ARC results: {0}", results.Chain);
// As required by RFC 8617, any validation error (including DNS failures) results in a Fail.
// The ChainErrors and the Reason/Exception properties of each header result provide diagnostics.
if ((results.ChainErrors & ArcValidationErrors.DnsTemporaryFailure) != 0)
Console.WriteLine ("The ARC chain could not be validated due to a temporary DNS failure; try again later.");
if (results.MessageSignature?.Reason != null)
Console.WriteLine ("ARC-Message-Signature: {0}", results.MessageSignature.Reason);Verifying DMARC
The DmarcVerifier determines whether a message's author domain (the domain in the From header)
was authenticated by an aligned DKIM signature or SPF check, and which policy the domain owner has
requested for messages that fail. It uses the same IDnsResolver interface as the DkimVerifier
to discover the DMARC policy and determine Organizational Domains, so no Public Suffix List is needed.
MimeKit does not implement SPF, so you will need to supply the result of your own SPF check of theMAIL FROM (or HELO) domain. If you have not already verified the DKIM signatures, theDmarcVerifier will do it for you using its DkimVerifier property.
var resolver = new DnsResolver (); // from the DKIM example above
var verifier = new DmarcVerifier (resolver);
// The result of an SPF check performed by your SMTP server or a third-party SPF library.
var spf = new SpfCheckResult (SpfStatus.Pass, "bounces.example.com");
var result = await verifier.VerifyAsync (message, spf);
Console.WriteLine ("DMARC result for {0}: {1}", result.AuthorDomain, result.Status);
switch (result.Status) {
case DmarcStatus.Pass:
// An aligned DKIM signature (result.AlignedDkimResult) or SPF result passed.
break;
case DmarcStatus.Fail:
// result.Policy is the policy requested by the domain owner (already adjusted for the
// sp=, np= and t= tags): None, Quarantine or Reject. The final decision is up to you.
Console.WriteLine ("Requested policy: {0}", result.Policy);
break;
case DmarcStatus.TempError:
// A DNS failure or a DKIM/SPF temporary error prevented a definitive result; try again later.
break;
case DmarcStatus.PermError:
// The message has no From header, multiple From headers, or no usable author domain.
break;
case DmarcStatus.None:
// The author domain does not publish a DMARC policy.
break;
}
// result.Errors provides additional diagnostics, and the result can be added to an
// Authentication-Results header using result.ToAuthenticationMethodResult ().If you have already verified the DKIM signatures, you can pass those results to the DmarcVerifier
to avoid verifying them twice:
var dkimResults = await verifier.DkimVerifier.VerifyAsync (message);
var result = await verifier.VerifyAsync (message, dkimResults, spf);Contributing
The first thing you'll need to do is fork MimeKit to your own GitHub repository. For instructions on how to
clone it, see the section titled Getting the Source Code.
If you use Visual Studio for Mac or MonoDevelop,
all of the solution files are configured with the coding style used by MimeKit. If you use Visual Studio on Windows or
some other editor, please try to maintain the existing coding style as best as you can.
Once you've got some changes that you'd like to submit upstream to the official MimeKit repository,
send me a Pull Request and I will try to review your changes in a timely manner.
If you'd like to contribute but don't have any particular features in mind to work on, check out the issue
tracker and look for something that might pique your interest!
Reporting Bugs
Have a bug or a feature request? Please open a new
bug report
or
feature request.
Before opening a new issue, please search through any existing issues
to avoid submitting duplicates. It may also be worth checking the
FAQ for common questions that other developers
have had.
If you are getting an exception from somewhere within MimeKit, don't just provide the Exception.Message
string. Please include the Exception.StackTrace as well. The Message, by itself, is often useless.
Documentation
API documentation can be found at https://www.mimekit.net/docs.
A copy of the XML-formatted API reference documentation is also included in the NuGet package.
.NET Foundation
MimeKit is a .NET Foundation project.
This project has adopted the code of conduct defined by the Contributor Covenant to clarify expected behavior in our community. For more information, see the .NET Foundation Code of Conduct.
Overview (`samples/MessageReader/README.md`)
MessageReader
This Windows Forms sample displays an email using MimeKit and
Ganss.Xss.HtmlSanitizer. The sanitizer is a
dependency of this sample only; it is not a dependency of MimeKit.
Sanitizing email
HtmlPreviewVisitor selects the message body and sanitizes text/html parts
using HtmlSanitizer.SanitizeDocument, rather than a hand-written tag filter.SanitizeDocument processes the whole document, including styles in the head,
while related MIME resources are still in scope.
Plain and flowed text are converted with TextToHtml and FlowedToHtml,
respectively. Their generated HTML is used directly, without an additional
sanitization pass. The resulting body, whether sanitized HTML or converted text,
is passed to WebBrowser.DocumentText.
The constructor defines a deliberately limited, read-only policy for HTML message
bodies:
- Keep common formatting elements, tables, inline styles, CSS classes, and
sanitized stylesheet rules, including@media. - Keep a limited set of CSS properties for colors, fonts, spacing, and layout.
Drop resource-loading properties, including background images, cursors,
filters, and generated content. Drop CSS custom properties,@import, and@font-face. Some messages will consequently look different. - Remove scripts, event handlers, forms, embedded documents, SVG/MathML, metadata,
and external stylesheet links. - Remove link destinations and other navigation attributes. Links remain as text;
they are not opened in the preview or in an external browser. - Block remote, relative, local-file, and sender-supplied
data:image URLs unless
the reference resolves to a permitted image MIME part within the message.
An absoluteContent-Locationcan resolve to a MIME part; it is never fetched.
FilterUrl rejects URLs by default. For an image element, it may replace the URL
with a newly generated data: URI for a PNG, JPEG, GIF, or BMP part found in the
current multipart/related scope. It uses cid: or Content-Location resolution
without downloading anything. Missing images and other MIME types lose their
source attribute, while alternative text and dimensions are retained.
The event runs even when the original URI scheme was rejected. The sample uses
this to resolve cid: references without globally allowing data: or cid:.
It never cancels a sanitizer removal event to restore arbitrary input. MIME
resolution happens during sanitization, before the visitor leaves the related
part; there is no subsequent rewriting of the sanitized HTML.
Rendering boundary
MessageViewWindow disables navigation, new windows, browser shortcuts, the
browser context menu, and drag-and-drop navigation for all message bodies,
including converted text. Sanitizer failures propagate; the sample does not fall
back to displaying the original HTML.
The existing WinForms WebBrowser control uses the legacy Internet Explorer
engine. This sample demonstrates the integration, not a general-purpose browser
sandbox or a guarantee of pixel-perfect rendering. Data-URI image support and
CSS rendering depend on the control's document mode. For a production viewer, use
a maintained browser engine, configure its isolation, script/native-bridge,
navigation, and resource-loading controls, and impose application-appropriate
message/image size limits. Keep HtmlSanitizer and its dependencies updated.
This policy renders a separate document. Do not assume that its styles are
suitable for inserting directly into the DOM of a webmail application's own UI;
use an appropriately isolated rendering context there.
Build and run
Use a Visual Studio Developer PowerShell with the .NET desktop build tools and
the .NET Framework 4.8 targeting pack:
msbuild .\samples\MessageReader\MessageReader.sln -restore -p:Configuration=Debug
.\samples\MessageReader\MessageReader\bin\Debug\MessageReader.exeThe sample references the MimeKit project in this checkout. On startup it displays
the bundled xamarin3.eml message.
The sample's NUnit project checks the email policy, inline-resource resolution,
text/HTML body selection, and loading message previews into the Windows browser
control. It does not add sanitizer dependencies to MimeKit's core test suite:
msbuild .\samples\MessageReader\MessageReader.Tests\MessageReader.Tests.csproj -restore -p:Configuration=Debug
dotnet test .\samples\MessageReader\MessageReader.Tests\MessageReader.Tests.csproj --no-build --no-restoreOverview (`samples/MimeAnalyzer/README.md`)
MimeAnalyzer
A console sample that showcases MimeKit's MIME compliance validation API by reporting
violations as compiler-style diagnostics: a quoted snapshot of the offending line with a
caret pointing at the exact column, much like the warnings and errors produced by clang or gcc.
Building
cd samples/MimeAnalyzer
dotnet buildThe project currently uses a ProjectReference to the in-tree MimeKit project because the
compliance API has not shipped yet. Once MimeKit 5.0 is released, this can be swapped for a
regular PackageReference.
Usage
MimeAnalyzer [options] <file.eml> [<file.eml> ...]Each argument is an individual message file, parsed with MimeFormat.Default.
| Option | Description |
|---|---|
-c, --context |
How the message is being used, which affects how severely a few violations are rated. Default: transport. |
-s, --severity <minor|major|critical> |
Only report issues at or above this severity. Default: minor. |
--category <list> |
Only report issues in these comma-separated categories: interoperability, dataloss, security. |
-r, --remarks |
Print the detailed explanation of each issue. |
-W, --werror |
Report every issue as an error. |
--max-issues <n> |
Stop collecting after n issues. Default: 1000. |
--max-per-violation <n> |
Ask the parser to report each violation at most n times. Default: 0, meaning no limit. |
--no-caret |
Do not quote the offending source line. |
--no-color |
Disable colored output. |
-q, --quiet |
Only print the per-file summary. |
-h, --help |
Print the help text. |
Color is also disabled automatically when the NO_COLOR environment variable is set or when
output is redirected.
Exit codes
| Code | Meaning |
|---|---|
| 0 | No issue was reported as an error. |
| 1 | An issue was reported as an error, or a file could not be read. |
| 2 | The command line could not be parsed. |
Example output
$ MimeAnalyzer addr.eml
addr.eml:2:9: warning: An address had an opening angle bracket without a closing one, or a closing bracket without an opening one. [UnbalancedAngleBracketsInAddress]
2 | To: Bad <[email protected], [email protected]>
| ^
addr.eml:2:47: warning: An address had an opening angle bracket without a closing one, or a closing bracket without an opening one. [UnbalancedAngleBracketsInAddress]
2 | To: Bad <[email protected], [email protected]>
| ^
addr.eml: 2 warningsAdding -r appends the rationale behind each rule, and the categories it falls under:
$ MimeAnalyzer -r raw-koi8r-header.eml
raw-koi8r-header.eml:3:10: note: A MIME part or message header contained 8-bit bytes where only 7-bit bytes were expected. [Unexpected8BitBytesInHeader]
3 | Subject: \xEF\xD4\xCB\xD5\xC4\xC1 \xCF\xCE \xD0\xCF\xD1\xD7\xC9\xCC\xD3\xD1?
| ^
| = categories: Interoperability, DataLoss
| = Older Internet Message Format specifications require that headers are strictly US-ASCII
| = while the newer Internationalized Email Headers specification allows for UTF-8. Header
| = values that are not US-ASCII should be encoded using the encoding mechanism described in
| = the MIME specification and/or should be valid UTF-8 as allowed in the Internationalized
| = Email Headers specification.
raw-koi8r-header.eml: 1 noteThe repository's UnitTests/TestData/compliance/ directory contains a corpus of deliberately
non-compliant messages that make a good demo. Any number of files may be passed at once:
MimeAnalyzer ../../UnitTests/TestData/compliance/incomplete-header.eml \
../../UnitTests/TestData/compliance/raw-koi8r-header.emlWhen more than one file is given, each is analyzed and summarized independently.
The bundled sample message
Each file in that corpus isolates a single violation. noncompliant.eml, next to this README,
is the opposite: one message that packs in as many different violations as a single message can
plausibly hold โ roughly sixty issues spanning every category and all three severities.
MimeAnalyzer noncompliant.emlIt is a multipart/mixed whose headers contain a dozen malformed addresses, repeated Date andSubject fields, a header field name containing a space, raw koi8-r bytes, a bare linefeed, an
oversized line, and a preamble that is not 7-bit clean. Its child parts then cover the body-side
rules: repeated Content-Type/Content-Transfer-Encoding headers, 8-bit and null bytes in the
body, broken base64, quoted-printable and uuencoded content, an illegal transfer encoding on amessage/rfc822 part and on a nested multipart, an unparsable Content-Type, a missing and an
unusable boundary parameter, a header that is never terminated, and a multipart that is never
closed.
Because it deliberately contains NUL bytes, 8-bit bytes in several charsets, a bare linefeed and
a line over 1000 characters long, it is not a file that survives hand-editing. It is generated
instead by gen-sample.ps1:
pwsh ./gen-sample.ps1Edit that script rather than the .eml when you want to add or change a case.
How it works
Compliance validation is exposed on MimeReader, the low-level scanner that MimeParser is
built on top of. Because MimeReader only scans the message rather than constructing aMimeMessage object graph, it is a cheap way to lint a message:
var collector = new ComplianceCollector (maxIssues);
using (var stream = File.OpenRead (fileName)) {
var reader = new MimeReader (stream) {
ComplianceLogger = collector,
ComplianceOptions = new MimeComplianceOptions {
Context = context,
MaxIssuesPerViolation = maxPerViolation
}
};
reader.ReadMessage ();
}ComplianceCollector implements IMimeComplianceLogger, whose single Log (in MimeComplianceIssue)
method is called for each violation found. By default MimeKit places no limit on how many issues it
will report, and the number is bounded only by the size of the message โ a message built for the
purpose can produce one issue every few bytes. There are two defenses, and the sample uses both:
MimeComplianceOptions.MaxIssuesPerViolation(--max-per-violation) bounds the report at the
source. The budget is per violation rather than a single total, so that a flood of one cheap
violation cannot push a more interesting one out of the report. When a budget runs out, a singleTooManyComplianceIssuesissue is reported so the report is never silently incomplete.- The collector's own cap (
--max-issues), because a logger cannot assume that every caller
configures the first one.
ComplianceCollector does not print TooManyComplianceIssues as a diagnostic: it has no source
construct to point at, and counting it as a warning would inflate the summary โ and, under--werror, the exit code โ for a message that may be fine apart from being verbosely wrong in one
place. It is reported as the issue limit reached note instead.
Each MimeComplianceIssue carries a Severity, a set of Categories, a one-based LineNumber,
and a one-based ColumnNumber (or 0 when the column is unknown, in which case the sample
suppresses the caret rather than misleadingly pointing at column 1).
Severity is mapped onto the familiar diagnostic levels:
MimeComplianceSeverity |
Diagnostic | Color |
|---|---|---|
Critical |
error |
red |
Major |
warning |
yellow |
Minor |
note |
cyan |
Narrowing an approximate position
Not every violation can be pinned to a single byte while parsing. Some describe an element as a
whole โ an unparsable Content-Type, a repeated header โ and others could only be narrowed
further by re-scanning input the parser has already moved past. Making every parse pay for that
would be a poor trade when most callers never look at the positions.
So each issue also carries a PositionKind saying what its position actually refers to:
MimeCompliancePositionKind |
Meaning |
|---|---|
Exact |
the position is the offending byte |
LineStart |
the violation is somewhere on this line |
ElementStart |
the violation is somewhere in this header or body part |
This lets the cost be shifted to the tool, which pays it only for the issues it actually prints.SourceText.TryLocateOffendingByte does exactly that: for the 8-bit and null-byte violations it
scans forward from the reported position for the first byte to blame, then maps that back onto a
line and column.
The difference is easiest to see on a folded header, where the parser reports the start of the
header but the bad bytes are on a continuation line:
8bit-folded-header.eml:4:2: note: A MIME part or message header contained 8-bit bytes where only 7-bit bytes were expected. [Unexpected8BitBytesInHeader]
4 | \xEF\xD4\xCB\xD5\xC4\xC1 folded
| ^Without narrowing, that diagnostic would point at column 1 of line 3 โ at the Subject: field
name, a line above the actual problem.
Note that for the 8-bit violations the sample looks for the first non-ASCII byte rather than
the byte where UTF-8 validation fails. The two are not the same: in koi8-r text such asEF D4 CB, 0xEF opens what looks like a three-byte UTF-8 sequence, so where validation gives
up depends on how the following bytes happen to combine and the caret can land mid-run. The first
non-ASCII byte is where the charset mistake actually begins.
Rendering the source line
The message is held in memory as raw bytes and is never decoded into a string. This matters
for two reasons: non-compliant messages routinely contain byte sequences that are not valid in
any charset, so decoding would destroy the very evidence being reported; and decoding would
shift the column numbers away from the byte offsets the parser reported.
Instead, the offending line is rendered byte by byte: tabs expand to the next 8-column tab stop,
printable ASCII passes through unchanged, and every other byte is escaped as \xNN. Since an
escape occupies four display columns for a single byte, the caret's position has to be computed
during rendering rather than taken directly from the reported column.
Very long lines โ the compliance corpus includes one over 1000 characters โ are elided to a
window centered on the caret, sized from the terminal width.
License
Released under the MIT license, the same as MimeKit itself.
Overview (`UnitTests/TestData/rtf/README.md`)
RTF test corpus
This directory holds the RTF documents used by UnitTests/Text/RtfCorpusTests.cs to exerciseRtfToText, RtfToHtml and the internal RtfInterpreter (sync/async parity).
The documents are grouped into one subdirectory per origin. Each subdirectory has a README.md
giving the upstream project, the exact commit the files were taken from, their original paths, and
the license they are distributed under (a copy of which sits alongside them).
| Directory | Origin | License |
|---|---|---|
apache-tika |
apache/tika | Apache-2.0 |
mimekit |
Written for MimeKit | MIT (same as MimeKit) |
rtf-parse |
mlewand/rtf-parse | MIT |
rtf.js |
tbluemel/rtf.js | MIT |
rtf2html |
wp-xyz/rtf2html | MIT |
rtfparserkit |
joniles/rtfparserkit | Apache-2.0 |
RtfPipe |
erdomke/RtfPipe | MIT |
striprtf |
joshy/striprtf | BSD-3-Clause |
How the documents were selected
Roughly 200 RTF files from permissively licensed RTF projects were scanned for the RTF features
they exercise (control words, destinations, code pages, \fcharset values, \uN edge cases,\fromhtml encapsulation, malformed structure, etc). A greedy set cover then picked the smallest
set of files that still covers every feature found, keeping the corpus small (~37 KB).
Documents that contain personal information (names, medical or legal content, obituaries, author
metadata) or third-party copyrighted prose were excluded even when they would have added coverage.
The features they would have covered are exercised by the synthetic documents inmimekit instead.
Expected output
The .rtf files are byte-for-byte copies of the upstream files and are marked binary in.gitattributes so that line-ending normalization never alters them.
The <name>.txt and <name>.html files next to each document are the expected RtfToText andRtfToHtml output. They are generated by MimeKit, not taken from upstream. If a test finds that an
expected-output file is missing, it writes the current output and reports the test as
Inconclusive. To regenerate after an intentional behavior change, delete the affected files, re-run
the tests, and review the diff.
Overview (`UnitTests/TestData/rtf/RtfPipe/README.md`)
RtfPipe test documents
- Upstream: https://github.com/erdomke/RtfPipe
- Commit:
9851ab97d693037073d567d74346184ae2f3d4b3 - Original directory:
RtfPipe.Tests/Files/ - License: MIT, Copyright (c) 2018 Eric Domke. See
LICENSE.txt, copied unmodified from the upstream repository.
The .rtf files are unmodified copies. The .txt and .html files are expected output generated
by MimeKit.
| File | Exercises |
|---|---|
Test01.rtf |
\strike, \ul, \cb, \fcharset2 (symbol), \uc0, the \upr/\ud Unicode destination pair |
Issue35.rtf |
Bookmarks (\bkmkstart/\bkmkend), \revtbl, hidden text (\v), \striked, \uld, \uldb |
tabs.rtf |
\ansicpg10000 (Mac Roman), an out-of-range \fcharset256 |
RtfParserTest_fail_4.rtf |
An empty document |
RtfParserTest_fail_3.rtf |
A document with unclosed groups |
Overview (`UnitTests/TestData/rtf/apache-tika/README.md`)
Apache Tika test documents
- Upstream: https://github.com/apache/tika
- Commit:
5486c9a2b75bb7354cfa2b66dcfd2a90fc6bb5ff - Original directory:
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/test/resources/test-documents/ - License: Apache License 2.0. See
LICENSE.txtandNOTICE.txt, both copied unmodified from the upstream repository.
The .rtf files are unmodified copies. The .txt and .html files are expected output generated
by MimeKit.
| File | Exercises |
|---|---|
testRTFTIKA_1713.rtf |
\fromhtml1 encapsulated HTML (\htmlrtf, \*\htmltag), \ansicpg65001, escaped \{ \}, and an extra } that closes the root group early, followed by more content |
testRTFUnicodeUCNControlWordCharacterDoubling.rtf |
\uc2 with \'hh fallback bytes, Shift-JIS \fcharset128 |
testRTFHyperlinkAndStyles.rtf |
HYPERLINK fields, \fcharset1, escaped \\ |
testRTFInvalidUnicode.rtf |
Surrogate pairs given as \uN, and unpaired surrogates |
testRTFWindowsCodepage1250.rtf |
\ansicpg1250 (Central European) |
Overview (`UnitTests/TestData/rtf/mimekit/README.md`)
MimeKit synthetic test documents
These documents were written for MimeKit and are distributed under the same MIT license as
MimeKit itself. They cover features found only in upstream corpus files that were excluded because
they contain personal information or third-party copyrighted prose. They also cover a few edge cases
no upstream file exercised. All text in them is neutral filler.
The .txt and .html files are expected output generated by MimeKit.
| File | Exercises |
|---|---|
nested-tables.rtf |
Replaces RtfPipe's Issue34.rtf. Nested tables (\intbl, \cell/\row, \nestcell/\nestrow, \*\nesttableprops both inside and after the content group, \nonesttables fallback text), \ansicpg1251 with \fcharset204 Cyrillic, non-breaking space \~, \qc. |
skipped-destinations.rtf |
Destinations that must not render: \*\themedata, \*\colorschememapping, \*\xmlnstbl, \*\datastore, footnote separators (\ftnsep, \ftnsepc, \aftnsep, \aftnsepc), \headerl/\headerr/\headerf, \footerl/\footerr/\footerf, annotations (\*\atnid, \*\atnauthor, \*\annotation), unknown \* destinations. Also an \object whose \*\objdata is skipped while its \result renders, and 12 levels of group nesting. |
charsets-and-sections.rtf |
\fcharset136 (Big5) with adjacent high-byte \'hh escapes, \cpg10000, a malformed {*\rsidtbl (missing backslash, so * is literal text), \footnote, \super, \page, \sect. |
Overview (`UnitTests/TestData/rtf/rtf-parse/README.md`)
rtf-parse test documents
- Upstream: https://github.com/mlewand/rtf-parse
- Commit:
66d997465122468ae0660d2d6cd05f03e2a7cc3a - Original directory:
test/_fixtures/ - License: MIT, Copyright (c) 2017 Marek Lewandowski. See
LICENSE.txt, copied unmodified from the upstream repository.
The .rtf files are unmodified copies. The .txt and .html files are expected output generated
by MimeKit.
| File | Exercises |
|---|---|
extractedNonshppict.rtf |
A fragment with no {\rtf1 header, containing a \nonshppict destination |
Overview (`UnitTests/TestData/rtf/rtf.js/README.md`)
rtf.js test documents
- Upstream: https://github.com/tbluemel/rtf.js
- Commit:
85fddf55b2f262bfd450769120c18c9ccef1f21c - Original directory:
samples/.common/data/rtf/ - License: MIT, Copyright (c) 2015 Thomas Bluemel. See
LICENSE.txt, copied unmodified from the upstream repository.
The .rtf files are unmodified copies. The .txt and .html files are expected output generated
by MimeKit.
| File | Exercises |
|---|---|
simplepngbin.rtf |
A \pict containing raw binary data via \binN (raw 8-bit bytes and C0 control characters that must not be read as RTF syntax) |
Overview (`UnitTests/TestData/rtf/rtf2html/README.md`)
rtf2html test documents
- Upstream: https://github.com/wp-xyz/rtf2html
- Commit:
2083240e74e7e032ee0aba9ce9b5b41b906c7d87 - Original directory:
test_files/generated_by_RichMemo/ - License: MIT, Copyright (c) 2021 Werner Pamler. See
LICENSE.txt, copied unmodified from the upstream repository.
The .rtf files are unmodified copies. The .txt and .html files are expected output generated
by MimeKit.
| File | Exercises |
|---|---|
Basic_Test.rtf |
\highlight, \sub, \nosupersub, \ulnone |
Overview (`UnitTests/TestData/rtf/rtfparserkit/README.md`)
RTF Parser Kit test documents
- Upstream: https://github.com/joniles/rtfparserkit
- Commit:
26c80c8700eef4c127a7a12ec4af35f190993b21 - Original directory:
src/test/resources/com/rtfparserkit/parser/standard/data/, excepttestSpecialChars.rtf, which comes fromsrc/test/resources/com/rtfparserkit/parser/raw/data/ - License: Apache License 2.0. See
LICENSE.txt, copied unmodified from the upstreamlicence.txt. The upstream project has noNOTICEfile.
The .rtf files are unmodified copies. The .txt and .html files are expected output generated
by MimeKit.
| File | Exercises |
|---|---|
testGitHubIssue6.rtf |
Color table, \cf/\chcbpat, HYPERLINK fields, \info, \header, style sheet and list tables, \ql/\qr/\qj, \line, \tab, \uN |
testSpecialChars.rtf |
Typographic symbol control words (\bullet, \emdash, \endash, \emspace, \enspace, \qmspace, quotes), \uc1, raw newlines |
test874Encoding.rtf |
\ansicpg874 (Thai), and many \fcharset values (161, 162, 163, 177, 178, 186, 222, 238) |
test10001Encoding.rtf |
\mac and \ansicpg10001 (Mac Japanese), \fcharset77/\fcharset78 |
testKoreanEncoding.rtf |
\ansicpg949, \fcharset129, \mmathPr |
test437Encoding.rtf |
\ansicpg437, \footer |
testJapaneseUtf8Encoding.rtf |
\cpg65001 |
testTurkishEncoding.rtf |
\ansicpg1254 |
testMultiByteHex.rtf |
\ansicpg932 with double-byte characters split across \'hh escapes |
test950Encoding.rtf |
\ansicpg950 (Big5), including a lone lead byte |
test10007Encoding.rtf |
\ansicpg10007 (Mac Cyrillic) |
testNegativeUnicode.rtf |
Negative \uN values (16-bit two's complement) |
Overview (`UnitTests/TestData/rtf/striprtf/README.md`)
striprtf test documents
- Upstream: https://github.com/joshy/striprtf
- Commit:
4f856ae78d1911fd395601891a729da0a6996ba1 - Original directory:
tests/rtf/ - License: BSD 3-Clause, Copyright (c) 2018 Joshy Cyriac. See
LICENSE.txt, copied unmodified from the upstream repository.
The .rtf files are unmodified copies. The .txt and .html files are expected output generated
by MimeKit.
| File | Exercises |
|---|---|
TX_RTF32_18.0.541.501.rtf |
\background and \shpinst shape destinations, control space (\ ) |
specialchars.rtf |
Optional hyphen \- and non-breaking hyphen \_ |
issue_28.rtf |
\ansicpg936 (Simplified Chinese), \fcharset134 |
unicode.rtf |
\uN values above 32767, \pgdsctbl, \userprops |
issue_20.rtf |
\ansicpg0 (an invalid code page) |
--- METRICS ---
- Files Extracted: 12
- Estimated Token Budget: ~20585 tokens
- Canonical Reference: https://codewiki.google/github.com/jstedfast/MimeKit