You've Got Mail! - The PFF File Format

My recent experimentation and blog post on the analysis of the Microsoft Extensible Storage Engine (ESE) database used by Microsoft  Windows Desktop Search (WDS) prompted me to begin looking at other ways Microsoft utilizes the ESE file format. Microsoft Outlook also utilizes the ESE in the form of the Personal Folder File (PFF) format. This includes the Personal Storage Table (PST) and Outlook Storage Table (OST) files which are commonly known as Outlook Data Files. The former (PST) is used in a non-enterprise setting when configuring outlook with email services such as pop/smtp and the later is created in enterprises with Outlook cached mode is enabled. Other forms of PFF include the Personal Address Book (PAB).

Joachim Metz has also done a fair amount of research on the PFF file structure as part of his libpff project. During the time of his research, the PFF file format was largely unknown. In 2010, however, Microsoft published the open specification on the PFF format and made it available as part of the MSDN Library.

The first four bytes of the file header contains the file signature of "!BDN " (0x2142444e). The 9th and 10th byte contain the content type which is 'SM' for PST (0x534D) and 'SO' (0x534F) for OST.

 
Metz's libpff pffexport utility will parse either file type. Once parsed, pffexport exports the following information on messages;

• Internet Email Headers
• Outlook Headers
• Conversation Index
• Recipients
• Message Body 
• Attachments 

Prior to Outlook 2007 there were three forms of file encryption available for PFF files; none, compressible, and high encryption. Metz documents the following about the two later options;

...actually more of a way to obfuscate the information in the PFF than real means to ensure confidentiality....

Microsoft's Open Specification document on the PST file structure also confirm Metz's findings on PFF encryption prior to Outlook 2007. They now recommended the use of Encrypted File System (EFS) or BitLocker Encryption to secure these files. Consequently, versions of Outlook after 2007 use compressible encryption and high encryption is no longer available.

Additionally, Microsoft Outlook allows users to set a password on their PST files. This password however, is a weak 32-but Cyclic Redundancy Check (CRC32) and consequently, is subject to collisions. This has been know for quite some time and Microsoft has documented this;

The PST Password, which is stored as a property value in the Message store, is a superficial mechanism that requires the client implementation to enforce the stored password. Because the password itself is not used as a key to the encoding and decoding cipher algorithms, it does not provide any security benefit to preventing the PST data to be read by unauthorized parties.

Metz clarifies this a bit more in his research. Applications, such as Microsoft Outlook, are conforming to the password protection but in reality, none of the data is actually protected by the password. Consequently, the libpff pffexport utility can export all items stored in the PFF file without supplying the password.

The libpff utility was able to parse the email headers and content on both the PST and OST files during my testing.

This certainly could be useful to forensics practitioners. The aforementioned, lack of security of these files however got me thinking more about the use of products such as Outlook Anywhere (RPC over HTTP) in the corporate world. Outlook Anywhere allows users to access corporate email on their personal computers using Microsoft Outlook. Consequently, corporate email would be stored in the local PFF file on the user's home system. Unless Whole Disk Encryption or other means were being used to secure the file system, then the potential risk to the intellectual property of corporation could be significant.

Happy Hunting!