Understanding Exchange Databases Disk Consumption

Exchange databases need disk space, once they run out of disk space they can not continue to function. My aim in this article is not to describe methods of preventing or solving the disk space problem(even though I will provide a tip or two)- my aim is to describe and explain the way that Exchange databases uses disk space.

As a direct evolutionary phase that stems from my first article the subject I would like to address in my second article is again disk space consumption but this time by the Exchange databases.

What will you gain from this you ask?

The answer is simple - a better understanding of how databases consume disk space which will enable you to manage the disk space you control in a more efficient manner.

Uncovering the databases

Exchange databases are based on the JET engine, actually a variant of the JET engine called Extensible Storage Engine (ESE). ESE is a multi-user database that supports full Data Manipulation Language (DML) and Data Definition Language(DDL). ESE is optimized for fast retrieval of data since this is the main function that the database performs.

Examining the versions of the ESE engines used by the different versions of Exchange server will uncover the fact that there is a difference between Exchange 5.5 and Exchange 2000/2003.

Exchange 5.5 uses a version of the ESE called ESE97 while Exchange 2000/2003 uses a version called ESE98. Now this may seem a subtle difference but it's not.

To understand the differences just take a look at the database file types in the different versions…

EDB files

The rich-text database (EDB) is the database type we are used to seeing in pre-2000 Exchange servers. The EDB database may also be called the MAPI database since it hosts all information submitted by the MAPI client(Outlook). The data itself is stored in a proprietary format called Microsoft Database Encapsulated Format (MDBEF).

It is worthwhile to mention that all messages (whether they are MAPI messages, SMTP or HTTP messages) have some of their properties saved in the rich-text database.

STM files

The native content database (STM) or the streaming database holds messages that were submitted by non-MAPI clients (post Exchange 5.5). The file is called a streaming file since data is added to it sequentially in its native format. The data itself inside the STM file is not encoded or encrypted in any way so if a store is dismounted the file can be viewed using a text editor.

Dealing with the files

The process of understanding how disk space is consumed by Exchange 5.5 databases is more straightforward since there is only one file type that consumes disk space. Exchange 2000/2003 on the other hand is a bit more complex since it uses two types of files for each "store". 

Nevertheless, when looking at disk space consumption the two files in use by Exchange 2000/2003 (EDB & STM) should be regarded as one entity- the store.

Dissecting disk space usage

To understand disk space consumption by Exchange databases follow figure 1 and the description for each section.

Fig 1: Description of disk space

1. Hard Drive/Disk Space – The information is saved on –you guessed it - a hard drive!!! Any unused space on the hard drive can be used for the databases/store.
2. Database/Store – The database is the placeholder for the information. The database files grow by 2MB increments. To prevent excessive fragmentation of the database on the hard drive you can change the increments to 16MB. This change can be done by using ADSIEDIT and accessing a specific storage group through the Configuration partition of the Active Directory. On the storage groups properties edit the msExchESEParamDbExtensionSize parameter and set it to 4096.
3. Data- This part of the database is the actual data that users will access and update.
4. Recoverable Items - Restoring a specific item on an Exchange server can be quite a pain (even though using a Recovery Storage Group makes it a lot easier). To help system administrators deal with this problem each store can retain items for a specific time (specified by the administrator). Items that are past their retention time (and under specific circumstances backed-up) will be tombstoned and removed from the database/store.
5. Recyclable Space - Space that is free and can be reused by new messages. Keep in mind that this space may be fragmented and spread inside the database. In figure 1 the space is at the end of the database since the database has undergone online maintenance.

Show me the numbers

Checking the space used by the database itself is quite straightforward - use any tool such as the Windows Explorer to check the database file size and/or the free space on the disk.

To be on top of things Exchange administrators should always know the size of the different parts of their Exchange databases.

Measuring Recyclable Space

Measuring the recyclable space can be done by either searching for event 1221 in the Application log or by using ESEUTIL to provide the information.

Event 1221 provides a conservative estimate of recyclable space in the database. If the database will be defragmented (offline) then you will gain at least the space specified in the event.

Currently there is some controversy around the question of event 1221 including both the EDB and the STM file for Exchange 2000/2003. It seems that this event only includes the EDB file.

Fig 2: Event 1221 on Exchange 5.5

To receive a more accurate estimate of the white space the ESEUTIL tool can be used. Invoking the ‘ESEUTIL.EXE /MS [database.edb] command on an unmounted store will provide you with a more accurate estimate on the recyclable space found in the store. The output of the command is similar to the following one:

The first part of the output called “SLV SPACE DUMP” provides information about the STM file. By multiplying the value of “Free:” with 4096 bytes you will receive the size of the recyclable space in the STM file.

The second part of the output called “SPACE DUMP” provides information about the EDB file. By multiplying the number at the bottom of the “Available” column with 4096 bytes you will receive the size of the recyclable space in the EDB file.

Measuring the size and quantity of recoverable items

When you are pressed for disk space you will use any and all items at your disposal to gain disk space -this may include jettisoning the Recoverable Items. To measure the size of the recoverable items you can use Performance Monitor. The specific counter you use is called: “Total Size of Recoverable Items”.

Fig 3: Monitoring the size of recoverable items.

If you need to jettison the items you can lower the retention time on the store and configure the online maintenance process to occur - this is not recommended unless you are really pressed for space.

Contemplating

When you understand a system you become one with the system-------- really!!! :)

On a more serious note a better understanding of the system will help you be a better system administrator so I hope that I helped you with this journey…

 

• Author: Erik Rozman