Never Again

Tale of the Vanishing E-Mails

IT folks are left scratching their heads as messages disappear into e-mail limbo.

It began as a normal day for our small (we have three staff members) IT shop, with the usual helpdesk calls from various users. One such call was from a user who was not receiving e-mails from a certain AOL sender. What puzzled this user was that she would get some e-mails from this sender as long as they were not sent as part of a group mailing or did not have any attachment in them.

Our helpdesk went through the cursory troubleshooting techniques of checking for a firewall block and looking in the quarantine and junk e-mail folders, but nothing was found. We told her we would keep the troubleshooting ticket open.

A Growing Problem
The following week the CEO of our company complained that he was not receiving mail from a certain sender when that sender e-mailed him something containing HTML contents in the body of the e-mail. This was followed by a complaint from another user group not being able to receive e-mails containing attachments. Again, we looked at the usual culprits that would block the e-mail, but came up empty-handed. It was beginning to get rather annoying.

We called the senders' IT department to seek its help in determining this mystery. They checked their logs to make sure the e-mails in question had indeed been sent. They also confirmed that they had not received any non-delivery report (NDR) in their servers.

We were using Postini as our filtering service, so we called them to see if their server was blocking such mails. We sent them the Internet headers of other e-mails received from the same sources so they could diagnose them. They sent us details of logs of the "missing" e-mails that showed that Postini's server had forwarded the e-mails to our Exchange server.

Vanishing Act
So now the question was: "Where did all these e-mails go?" We could trace the path of these e-mails all the way up to our server and yet they weren't getting distributed to the appropriate recipients. We even opened port 25 on our PIX firewall that was initially set to allow mail on our Exchange Server from Postini only.

This action produced a disastrous result. Now our server was open to the whole wide world and we were deluged with spam.

Finally, we looked at the Exchange System Manager on the Exchange Server and activated the Message Tracking Center. It showed the message from AOL being received, so we went into the Message History to look for details. It showed the message being submitted to Categorizer but there was no indication as to what happened to the message after that.

What's Your Worst IT Nightmare?
Write up your story in 300-600 words and e-mail it to Editor Ed Scannell at [email protected]. Use "Never Again" as the subject line and be sure to include your contact information for story verification.

Seeing Double
Normally this would have shown the message to be queued for local delivery, but that wasn't the case. It was then that we came upon a stunning realization: Our mail was not only being checked by Postini, but we were also running Trend Micro's ScanMail for Microsoft Exchange where, by default, attachment blocking under the virus-scan function had been enabled.

So while Postini was allowing the mail with attachments to go through after filtering it, Trend Micro was stopping it from going any further. This is why there was no trace of those e-mails even when Postini had forwarded them to our Exchange Server. We took the check mark off the box that enabled attachment blocking, put the restriction on port 25 back on the firewall and, Presto! Everything was back to normal.

About the Author

Syed Asif is the IS director for Queens Centers for Progress in Jamaica, N.Y.


comments powered by Disqus