Work around for importing Mime messages into Exchange Online using the Microsoft Graph

One of the more vexing and grating issues when converting an EWS application to the Microsoft Graph to import messages (or other MIME content) into a mailbox is that, while you can import a message using MIME, it will always be created as a draft message. Another issue is that MIME is only supported on the SearchFolder /Messages endpoint rather then the explicit folder /MailFolders/{id}/Messages endpoint.

Mime Content was well supported in EWS which meant you could use it to solve a few issues firstly the import issue https://learn.microsoft.com/en-us/exchange/client-developer/exchange-web-services/how-to-import-items-by-using-ews-in-exchange . In EWS you can set the pidtagmessageflag extended property when you create the Message from Mime, but in the Graph you can’t do this in the same create operation and then once a message is created the pidtagmessageflag extended property value can’t be changed and your stuck with the message in draft. You also had the ability to import it into any folder you wished and it also allowed you to solve the forward as an email issue https://gsexdev.blogspot.com/2013/07/forwarding-message-as-attachment-using.html.

MIME and MAPI and how they play together in Exchange

Before we talk about the workaround lets talk a little bit more about Mime and Exchange and why things get a bit weird sometimes. MIME, or Multipurpose Internet Mail Extensions, is an open standard defined in the Internet Engineering Task Force (IETF) RFC 2045 through RFC 2049. It was created to extend the basic functionality of email by allowing the transfer of non-ASCII characters, multimedia content (like images, audio, and video), and attachments . MAPI is a proprietary messaging architecture created by Microsoft in the 80-90’s (before the final rfc’s for MIME) it was first used in MSMail and then later in Exchange . When Exchange stores an Email it is stored in the Extensible Storage Engine (ESE) as a collection of Mapi properties. When you send a message inside your tenant those Mapi properties get sent serialized in TNEF (transport neutral encapsulation format) but when it goes outside of your tenant or onPrem Exchange org that Message then gets converted to Mime in the Transport pipeline (and sent over the SMTP protocol). Also if you where to use an IMAP or Pop client to connect to a Mailbox and read a message then the Exchange Mail Store (or the client access part of it) would do an on the fly conversion from Mapi to Mime. What is offered in Graph and EWS is this ability for Exchange to do the on-the-fly MIME to MAPI conversion (and via versa if you where exporting) as part of using the Protocol . In EWS this happen when you use the MimeContent property and Graph when you use $value segment.

Outlook also has its own MAPI to Mime conversion (and via versa) https://learn.microsoft.com/en-us/office/client-developer/outlook/mapi/iconvertersessioniunknown so if you where using this Mapi provider you could do the conversion completely client side.

So What Went Wrong in the Graph(IMO)?

The strength of EWS is that it actually got implemented and used in real world mail clients like OWA and the Mac Entourage client (which meant it was forged in the fire) which have now all gone (or are going) to use the proprietary Microsoft's native sync technology that succeeded ActiveSync. The way in which MIME support was implemented in the Graph is where it went wrong. I think this is a case of just product focusing on adding a feature to tick a box rather then looking at how it was being used and meeting the developers at the coalface which is what you are forced to do when you have to create a functional mail client that uses an API eg OWA etc.

How Can This Be Fixed in the service (IMO)?

Meet the people and requirements at the coal face eg if they had just followed the MimeContent pattern used in EWS that would have alleviated both the with importing and forwarding messages migration challenges from EWS. Additionally, a conversion library that facilitates direct conversion from MIME to FTS (Fast Transfer Stream) would also be a valuable solution. The one thing that AI won’t do from a product prospective is go out in the real world and look at what’s happening and is probably what is going to separate good vs bad product in the future.

Workaround at the Client/Developer side.

The new Import/Export API in the Microsoft Graph provides a workable, albeit more complex, workaround for the import issue. Here's how it works:

1. Create the Message: Use the standard "create message from MIME" method, which will create a draft message from the MIME content using the /Message endpoint.

2. Export the Message: Export the draft message you created from Mime as an FTS stream.

3. Modify the FTS Stream: Search the exported FTS stream for the PidTagMessageFlags property (using a binary pattern match) and change its value to make the message appear as a sent message rather than a draft.

4. Reimport the Stream: Import the modified FTS stream back into the desired target folder.

5. Delete the Draft message you created

---

Important Considerations

Typically, you should avoid modifying an FTS stream directly, as most properties in the stream (and the stream itself) include length definitions. Any mismatch can lead to that stream then becoming corrupted or at least unusable. In the Structure in the FTS Stream the length is typically a 4-byte integer (32-bit) that precedes the actual string data in the stream. This length value is used to delimit the size of the string so for example If you replace a 5-character string with a 7-character string and fail to update the length property, the parser will misread the stream, leading to errors. Because the PidTagMessageFlags property is a fixed length PT_LONG property the length of the value is always the same. In this case its safe to modify as long as you write back a valid long value or in this case a valid long that represents the bitmask of the Message flags.

Sample Code

I have built some sample code in both C# and PowerShell to demonstrate the workaround using the respective Graph SDK’s for each versions. The Beta of .net Graph SDK now supports the Import Export API while the PowerShell Graph SDK support isn’t available yet so there is some custom code there.

I’ve put the C# version https://github.com/gscales/Powershell-Scripts/blob/master/Graph101/GraphSDK/GraphImportMimeWorkAround.cs and the PowerShell version https://github.com/gscales/Powershell-Scripts/blob/master/Graph101/GraphSDK/EmlImportWorkAround.ps1. The use the Powershell version it expects a Graph Connection with the following scopes eg

$scopes = @("Mail.ReadWrite", "MailboxItem.Read", "MailboxItem.ImportExport")

Connect-MgGraph -Scopes $scopes -TenantId yourdomain.com

Invoke-ImportEmlFile -UserId gscales@yourdomain.com -EmlPath C:\Users\gscal\Downloads\message3.eml

What this workaround doesn’t solve

The issue with forwarding messages in the Microsoft Graph remains unresolved. However, you can work around this by exporting the message you want to forward as MIME, constructing the forward message as MIME, attaching the existing message, and then sending it as MIME.

To achieve this, consider using a library like MimeKit to build the message, add the necessary attachments, and handle the MIME construction process.