In-Depth

Outlook Administration: Avoiding E-Mail Archiving Pitfalls in Microsoft Outlook 2007 & 2010

E-mail archiving has changed in the newer versions of Outlook. Here's how you can stave off some confusion and avoid help-desk calls.

Sometime this year, my organization is planning to make the jump from Office 2003 to Office 2010. I'm not really sure what to expect. As a longtime Office 2003 user, Office 2010 seems a bit foreign to me in a lot of ways -- but mainly in Outlook. Some of the functions in Outlook 2010 behave differently than they did in previous versions of office.

Sure there's the Ribbon and the mouse-friendly interface, but those aren't the changes I'm talking about here. The thing I've come across as behaving very differently is message archiving in Outlook 2010. My goal here is to point out the differences -- sometimes minor, but still important -- between the way archiving worked in Office 2003 and the way it works in Office 2007 and 2010.

The Normal Process of Archiving E-Mails
When mail comes into Outlook, it has a date stamp on it that allows you to see when you received a message. So, when your boss comes running down the hall asking you about the message he sent to you on whichever date, you can peek into your inbox and see, "Hey, he did send me an e-mail on Friday at 2 p.m." It also provides a really good way for you to organize, sort and find messages. However, for consolidation reasons, many organizations -- my own included -- impose limits on mailbox size. In this scenario, the user is allowed to keep, say, 500GB of storage in his mailbox for what I would consider "live" e-mail. Once this limit is reached, incoming messages will start bouncing and being returned to sender, which in many cases is not a good thing.

To combat this limit, Microsoft has provided a way for users (or the system administrators who support them) to archive messages and other items in a mailbox so that they're moved to a PST file or something like it stored on the network. From there, the messages can be backed up to tape or disk or whatever the medium du jour happens to be. Because I'm trying to do right by my users, I like to put the ownership of the mailbox on them and make them partially responsible for archiving their messages and keeping their mailboxes clean. In most cases, this works pretty well.

Users can create archive scenarios that allow them to have messages older than so many days or weeks or months moved to their offline archives or out of their mailbox archives. When this happens, these messages no longer take up space in their actual mailboxes.

It's All About the Date
In Outlook 2003, archiving happens by the received-on date or sent date. This makes a lot of sense, because if I receive an e-mail on June 1, 2011, and my archive settings are configured to archive mail older than a month and run every three days, when July 2 hits this e-mail should get archived. It seems that this is the logical way of thinking about how messages should be moved, and in Outlook 2003 (and older versions) this is how it works. But in Outlook 2007 and later, messages get archived by their modified date. Just like other files, messages have a modified date, which is not exposed in the interface in Outlook. Users won't be aware of it right away, but they can expose the date by selecting a column header called "modified date."

Field Chooser
In most cases, the modified date and received date or the sent date will be the same if you process your mail out of your inbox and don't move it around after the fact. Archiving will function just as you would logically expect it to. However, some people like to use folders to manage their mail and will drag messages into folders for safekeeping as they've read or processed them. They'll do this without using rules or anything else of the sort. Once a message is dragged to a new folder, the modified date changes, just as it would with any other file. If you move it or touch it or open it or do anything to it, it changes the modified date. When this happens, that message may not land on the same archiving schedule as you think it will.

I'm not trying to point out a way to use Outlook differently just yet; I simply want to make you aware that archiving may behave in a new manner than what you'd expect.

When working with messages in Outlook 2007 and 2010, take the time to consider the modified date and how people might manage their mail if you (or those you work with) are folder users. This might be something that could cause some help-desk calls in your organization.


[Click on image for larger view.]
Figure 1. The field chooser dialog box.

Why Does Outlook Behave This Way?
Any mail in your Outlook mailbox is just like any file on your hard disk or a network share that you access regularly. The files that are hot or accessed frequently may be handled differently than files that are cold or accessed once a year. Newer methods of backup can take this into account, keeping the newer files more readily available than the older ones. Outlook is doing the same thing by basing archiving on messages that are touched very often rather than on just the date on which they were received. Mail that you might need to read or reference because you use it a lot is less likely to be archived and moved out of your way.

It seems rather odd that this was not mentioned in any documentation I've seen. I'm OK with not being able to change the archive within Outlook to use different dates from a user perspective, but I wish this information would be presented when Outlook is configured. I didn't discover it until I was having trouble with a mail archive and couldn't figure out why no messages were being moved. Perhaps including an option to opt in or opt out of this new method of archiving would've been a good start, just to make everyone aware of it.

However, once people get used to this new feature -- as is the case with the Ribbon in Office -- I think they'll like it a little better. At first it was sort of confusing and took considerable time to troubleshoot, but now that I'm aware of this change to archiving I don't think it's quite so bad. It could be useful in allowing users to determine which messages are frequently used and which messages are ready for archiving.

One thing I got out of finding all of this out the hard way is that it's always good to reevaluate your archiving strategy, even if only for your own mailbox. This method of archiving will make me think hard about how often I want my mail archived, and it will likely be a much more frequent occurrence for me even though I don't tend to move mail between folders very often. In the event that I do move any messages around, keeping the archiving frequent should produce similar results to those of older versions of Outlook.

Changing User Behavior (or Administrator Thinking)
Because the method of archiving can't be changed, one thing that can be done is to train users and administrators on how Outlook works. This should include more frequent archiving discussion and possibly the use of an archive folder set as a working folder rather than simply archiving directly out of the inbox (or mailbox).

When I began considering using an archive as working storage for my e-mail, I found the idea completely foreign. I think of archives as stored material, much of which may never be heard from again, so using this as primary mail was quite strange. Because the e-mail needs to be moved to the archive almost as soon as it arrives in order for the working-storage archival method to work, rules, third party add-ons or manual drag-and-drop organization might work best in this scenario.


[Click on image for larger view.]
Figure 2. The modified date in Outlook 2007 and later.

As mail arrives, it gets shuffled off to the archive as soon as possible, perhaps by a rule that has been created to move mail to an archive folder. This way, there's rarely mail left in the inbox, and all replying, forwarding and general use happens from the archive folders. Using a rule or manual method circumvents the need to run auto-archiving. Because of this, there's no worry about issues surrounding the modified date.

Another possibility would be completely manual. As messages are processed, they could be moved to an open archive folder with a simple drag and drop. This method works similarly to Outlook's archive method but would put control completely in the hands of the user. For those who keep folders inside their inboxes or mailboxes, this might be a decent alternative. This way, the messages still get moved and the user still gets folder organization. In this case, auto archiving is likely to be disabled for all folders. I might leave sent items alone and let auto archive crank those to PST files every week or so, as those don't change much once sent.

The third alternative way to handle archiving is mostly beyond the scope of this article, as third-party add-ons do exist to help manage mail. The one I've seen, ClearContext from ClearContext Corp., makes use of its own rules engine and management tasks to help manage mail into and out of an inbox. In looking at the rules method and the previously described manual method, this is much like a combination of the two. Looking into the particulars of an application like ClearContext is another article altogether, but for now I'll say it's certainly worth checking out.

Remember: Move Mail Carefully
Just remember to be mindful when moving e-mail around in your mailbox in Office 2007 or 2010. Mail that doesn't move should be minimally affected, but messages dragged into or out of folders for organizational purposes may not behave as you need them to when the prompt for auto archiving next appears.

The use of the modified date in archiving does make message handling a bit more consistent with how other files are handled -- but making this known to the user, possibly in the "What's New" documentation, might have been a good idea. I suspect there are many users who don't move their mail around before archiving, but there are likely to be just as many who do. Those users might start calling the help desk if e-mail is not getting moved as expected, if at all. The stated size limit on PST and OST files in Outlook 2010 is 20GB. It's likely that this will be large enough for all the mail your users store; however, creating the habit of cleaning out archive folders on a yearly basis couldn't hurt, just to ensure that the 20GB ceiling stays high enough to prevent failing and corrupt archive files. The 2GB limit in Outlook 2003 has caused quite a few issues with corrupt files (certainly enough for me), and the increased size allowed for archives -- with some better management habits -- might be just the thing to get archiving exactly where it needs to be.

I'm sure that the issues with archiving are many, and as data increases in size and attachments grow, the limits and methods used to safely back up and store e-mail will change drastically. But for now, I think Microsoft is doing a pretty good job, even if Outlook users need to be aware of how archiving works and even if it takes some effort on their part to help things along.

comments powered by Disqus

Reader Comments:

Thu, Mar 7, 2013 bill usa

microsoft consistently blames the size of mailboxes as the culprit on slow performance. even though the limit of pst files in ol has grown to 20 or even 50 gb, they still blame the size if it's over 2 gb. the old limit. whats';s even worse however, is that their archiving system doestnt work if the ol client is connected to an exchange server.works fine for pop3 clients but not exchange.they are losing the business marktet. have one customer who wants to revert back to 2003 exchange because the 2010 exchange is so slow. congrats microsoft, you want to be like mac and not have servers or bussiness support , you are almost there

Wed, Sep 12, 2012 marshal

Dear SLG, You can easily find and view from Archive folder, Go to menu > folder list > archive folder In Microsoft Outlook, Auto archive is available by default and runs automatically at scheduled intervals.

Fri, Aug 10, 2012 SLG

I am not happy with the fact that my work email account is the only place I can retrieve my archived emailsfrom. I get emails from work when I am at home and some ask me for info that I have in archived emails and from home there is no way to open the archived emails up.

Fri, Jan 13, 2012

Outlook has not really changed much my first use of Outlook 97. That's right Outlook 1997, later I upgraded to Outlook 98, and then to 2000 which offered VBA and I loved it. However, for that past 10+ years what has been happening at Microsoft!!! Is anyone working? I know the licensing department is because they are collecting my EA renewal license every year. But the product remained the same for most part. I've been using Outlook 2010, and dont see much value being added other than the Message Grouping, that is something I had for a while using 3rd party tools. So in effect, I dont see any improvements. You guys are loosing your edge, soon the pacman Google is going to swallow Microsoft.

Fri, Oct 28, 2011 Kent

Microsoft does NOT support PST's on network shares despite their prevalent use there. http://support.microsoft.com/kb/297019 PST's in my experience are an unreliable method of document retention.

Fri, Oct 28, 2011 Coleen

The article says "500 GB" of e-mail data per user...I'm assuming that's a typo (MB).

Fri, Oct 28, 2011 Bill

Wishful thinking and bad information... Outlook 2003 archives on Modified date also... Whish we could get Outlook 2010 to archive on received but no go. Check your facts junior...

Fri, Oct 28, 2011 Todd

You might want to double check your statement that Outlook 2003 archives on received on date - maybe provide the MS article that told you that. It too, Outlook 2003, also archives on modified date the same as Outlook 2007 and 2010. You just didn't notice the modified date issue until you migrated to 2010 and changed all your email dates.

Fri, Oct 28, 2011 BB Canada

I partially agree with ED... PST can be a nightmare to manage. On the other side, I used to create PST files for over 10 years and keep them archived by Year, so I can easily grab back. Especially if you change company on a regular basis, you may want to keep a trail of important e-mails over the long run, despite not working anymore for this company. You never know when spectres are coming back to haunt you :-). Thanks for the great article, I always wondered why some message just never got archived ! I also realized that some messages are not archived (despite the proper modified date) when they still have reminder flag set on them... seems logical.

Thu, Oct 27, 2011 Ed Atlanta

First, get the organization to stop implementing such an archaic document retention policy. Keeping onyt 500MB of data is like telling users that they only get 2 feet of filing space. It has nothing to do with the real document retention requirements of the organization. Next, get rid of PSTs. They are very labor intensive. If you lose a disk, if you migrate machines, if you want ot use two machines, they just become painful to correctly administer. If a message is worth keeping, it's worth keeping on the server! If yout organization sues or gets sued, you are likely to have to search the organization for message pertaining to a subject. That's not just the server, it's every dang workstation in the organization! So, first, builf the Exchange server so that it has enough storage to fulfill your requirements. With 2TB disks, that's a pretty easy task to accomplish. Create a real document retention policy, not just a "to keep my mailbox server from getting full" policy. Look at the new Exchange 2010 SP1 policies, they actually start doing document retention the way that it is supposed to be done. And most importantly get rid of every PST that exists. PST=BAD!

Add Your Comment Now:

Your Name:(optional)
Your Email:(optional)
Your Location:(optional)
Comment:
Please type the letters/numbers you see above

Redmond Tech Watch

Sign up for our newsletter.

I agree to this site's Privacy Policy.