In-Depth

Outlook Administration: Avoiding E-Mail Archiving Pitfalls in Microsoft Outlook 2007 & 2010

E-mail archiving has changed in the newer versions of Outlook. Here's how you can stave off some confusion and avoid help-desk calls.

• By Derek Schauland
• 10/15/2011

Sometime this year, my organization is planning to make the jump from Office 2003 to Office 2010. I'm not really sure what to expect. As a longtime Office 2003 user, Office 2010 seems a bit foreign to me in a lot of ways -- but mainly in Outlook. Some of the functions in Outlook 2010 behave differently than they did in previous versions of office.

Sure there's the Ribbon and the mouse-friendly interface, but those aren't the changes I'm talking about here. The thing I've come across as behaving very differently is message archiving in Outlook 2010. My goal here is to point out the differences -- sometimes minor, but still important -- between the way archiving worked in Office 2003 and the way it works in Office 2007 and 2010.

The Normal Process of Archiving E-Mails
When mail comes into Outlook, it has a date stamp on it that allows you to see when you received a message. So, when your boss comes running down the hall asking you about the message he sent to you on whichever date, you can peek into your inbox and see, "Hey, he did send me an e-mail on Friday at 2 p.m." It also provides a really good way for you to organize, sort and find messages. However, for consolidation reasons, many organizations -- my own included -- impose limits on mailbox size. In this scenario, the user is allowed to keep, say, 500GB of storage in his mailbox for what I would consider "live" e-mail. Once this limit is reached, incoming messages will start bouncing and being returned to sender, which in many cases is not a good thing.

To combat this limit, Microsoft has provided a way for users (or the system administrators who support them) to archive messages and other items in a mailbox so that they're moved to a PST file or something like it stored on the network. From there, the messages can be backed up to tape or disk or whatever the medium du jour happens to be. Because I'm trying to do right by my users, I like to put the ownership of the mailbox on them and make them partially responsible for archiving their messages and keeping their mailboxes clean. In most cases, this works pretty well.

Users can create archive scenarios that allow them to have messages older than so many days or weeks or months moved to their offline archives or out of their mailbox archives. When this happens, these messages no longer take up space in their actual mailboxes.

It's All About the Date
In Outlook 2003, archiving happens by the received-on date or sent date. This makes a lot of sense, because if I receive an e-mail on June 1, 2011, and my archive settings are configured to archive mail older than a month and run every three days, when July 2 hits this e-mail should get archived. It seems that this is the logical way of thinking about how messages should be moved, and in Outlook 2003 (and older versions) this is how it works. But in Outlook 2007 and later, messages get archived by their modified date. Just like other files, messages have a modified date, which is not exposed in the interface in Outlook. Users won't be aware of it right away, but they can expose the date by selecting a column header called "modified date."

Field Chooser
In most cases, the modified date and received date or the sent date will be the same if you process your mail out of your inbox and don't move it around after the fact. Archiving will function just as you would logically expect it to. However, some people like to use folders to manage their mail and will drag messages into folders for safekeeping as they've read or processed them. They'll do this without using rules or anything else of the sort. Once a message is dragged to a new folder, the modified date changes, just as it would with any other file. If you move it or touch it or open it or do anything to it, it changes the modified date. When this happens, that message may not land on the same archiving schedule as you think it will.

I'm not trying to point out a way to use Outlook differently just yet; I simply want to make you aware that archiving may behave in a new manner than what you'd expect.

When working with messages in Outlook 2007 and 2010, take the time to consider the modified date and how people might manage their mail if you (or those you work with) are folder users. This might be something that could cause some help-desk calls in your organization.

[Click on image for larger view.]

Figure 1. The field chooser dialog box.

Why Does Outlook Behave This Way?
Any mail in your Outlook mailbox is just like any file on your hard disk or a network share that you access regularly. The files that are hot or accessed frequently may be handled differently than files that are cold or accessed once a year. Newer methods of backup can take this into account, keeping the newer files more readily available than the older ones. Outlook is doing the same thing by basing archiving on messages that are touched very often rather than on just the date on which they were received. Mail that you might need to read or reference because you use it a lot is less likely to be archived and moved out of your way.

It seems rather odd that this was not mentioned in any documentation I've seen. I'm OK with not being able to change the archive within Outlook to use different dates from a user perspective, but I wish this information would be presented when Outlook is configured. I didn't discover it until I was having trouble with a mail archive and couldn't figure out why no messages were being moved. Perhaps including an option to opt in or opt out of this new method of archiving would've been a good start, just to make everyone aware of it.

However, once people get used to this new feature -- as is the case with the Ribbon in Office -- I think they'll like it a little better. At first it was sort of confusing and took considerable time to troubleshoot, but now that I'm aware of this change to archiving I don't think it's quite so bad. It could be useful in allowing users to determine which messages are frequently used and which messages are ready for archiving.

One thing I got out of finding all of this out the hard way is that it's always good to reevaluate your archiving strategy, even if only for your own mailbox. This method of archiving will make me think hard about how often I want my mail archived, and it will likely be a much more frequent occurrence for me even though I don't tend to move mail between folders very often. In the event that I do move any messages around, keeping the archiving frequent should produce similar results to those of older versions of Outlook.

Changing User Behavior (or Administrator Thinking)
Because the method of archiving can't be changed, one thing that can be done is to train users and administrators on how Outlook works. This should include more frequent archiving discussion and possibly the use of an archive folder set as a working folder rather than simply archiving directly out of the inbox (or mailbox).

When I began considering using an archive as working storage for my e-mail, I found the idea completely foreign. I think of archives as stored material, much of which may never be heard from again, so using this as primary mail was quite strange. Because the e-mail needs to be moved to the archive almost as soon as it arrives in order for the working-storage archival method to work, rules, third party add-ons or manual drag-and-drop organization might work best in this scenario.

[Click on image for larger view.]

Figure 2. The modified date in Outlook 2007 and later.

As mail arrives, it gets shuffled off to the archive as soon as possible, perhaps by a rule that has been created to move mail to an archive folder. This way, there's rarely mail left in the inbox, and all replying, forwarding and general use happens from the archive folders. Using a rule or manual method circumvents the need to run auto-archiving. Because of this, there's no worry about issues surrounding the modified date.

Another possibility would be completely manual. As messages are processed, they could be moved to an open archive folder with a simple drag and drop. This method works similarly to Outlook's archive method but would put control completely in the hands of the user. For those who keep folders inside their inboxes or mailboxes, this might be a decent alternative. This way, the messages still get moved and the user still gets folder organization. In this case, auto archiving is likely to be disabled for all folders. I might leave sent items alone and let auto archive crank those to PST files every week or so, as those don't change much once sent.

The third alternative way to handle archiving is mostly beyond the scope of this article, as third-party add-ons do exist to help manage mail. The one I've seen, ClearContext from ClearContext Corp., makes use of its own rules engine and management tasks to help manage mail into and out of an inbox. In looking at the rules method and the previously described manual method, this is much like a combination of the two. Looking into the particulars of an application like ClearContext is another article altogether, but for now I'll say it's certainly worth checking out.

Remember: Move Mail Carefully
Just remember to be mindful when moving e-mail around in your mailbox in Office 2007 or 2010. Mail that doesn't move should be minimally affected, but messages dragged into or out of folders for organizational purposes may not behave as you need them to when the prompt for auto archiving next appears.

The use of the modified date in archiving does make message handling a bit more consistent with how other files are handled -- but making this known to the user, possibly in the "What's New" documentation, might have been a good idea. I suspect there are many users who don't move their mail around before archiving, but there are likely to be just as many who do. Those users might start calling the help desk if e-mail is not getting moved as expected, if at all. The stated size limit on PST and OST files in Outlook 2010 is 20GB. It's likely that this will be large enough for all the mail your users store; however, creating the habit of cleaning out archive folders on a yearly basis couldn't hurt, just to ensure that the 20GB ceiling stays high enough to prevent failing and corrupt archive files. The 2GB limit in Outlook 2003 has caused quite a few issues with corrupt files (certainly enough for me), and the increased size allowed for archives -- with some better management habits -- might be just the thing to get archiving exactly where it needs to be.

I'm sure that the issues with archiving are many, and as data increases in size and attachments grow, the limits and methods used to safely back up and store e-mail will change drastically. But for now, I think Microsoft is doing a pretty good job, even if Outlook users need to be aware of how archiving works and even if it takes some effort on their part to help things along.