Logging SMTP protocol activity

Like many of my posts, this one also starts with "By default,... ". Like many folks, I'm always curious about the defaults used for almost everything in Exchange/SMTP (and /IIS/Windows/AD et al), and at times intrigued by the choices made.

One of my pet peeves has been about how installing Exchange changes the maximum number of recipients on the SMTP virtual server from the 100 set by SMTP svc sans Exchange to a mind-boggling 64,000 recipients after our favorite mail server modifies the SMTP stack, but there's plenty of fodder there for yet another blog post - a quite lengthy one if I have the time. :)

This one's about logging SMTP protocol activity, which isn't alive by default. In other words SMTP logging is disabled by default. Unfortunately, many sysadmins only find out about this *after* they start troubleshooting an SMTP-related issue.

You can enable SMTP logging (also referred to as "protocol logging" or "SMTP protocol logging" by some) from SMTP virtual server properties | General tab.

You're faced with a choice of formats - W3C Extended Log File Format, NCSA Common Log File Format, Microsoft IIS Log Format and ODBC Logging. The latter lets you log to a database which is what some organizations do with such logs.

Only the W3C Extended Log File Format is - well - extended! (The rest only alllow logging of certain fields).

You can select the fields the W3C format will log. By default it logs only 4 fields:
1) Time (no date)
2) c-ip (Client IP Address)
3) cs-method (Method): These are the SMTP commands issued by the client like HELO, MAIL, RCPT, DATA, et al.
4) cs-uri-stem (Uri Stem): which doesn't log anything (inserts a dash), afaik
5) sc-status (Protocol Status): SMTP response codes like 220, 250, 500, etc.

The above fields provide very basic and at times completely inadequate information for troubleshooting, particularly in higher volume environments.



The following fields should be logged (in most cases):
1) Date
2) Time
3) c-ip (Client IP Address)
4) cs-username (User Name) - the fqdn provided by sending server on HELO
5) s-sitename - SMTP virtual server instance name (e.g. SMTPSVC2)
6) s-computername - (Server Name) - name of server, useful if aggregating logs from multiple servers into a monitoring/logging database
7) s-ip - Server IP Address - useful if server has multiple IP addresses
8) cs-method (Method) - SMTP command e.g. HELO, MAIL, RCPT, DATA, QUIT
9) cs-uri-query (URI Query) - command parameter (like +from:foo@foo.com -
gets the from, to addresses, sending server fqdn given at HELO, and Message ID. Nothing between DATA and end of data sequence is logged)
10) sc-status (Protocol Status) - SMTP protocol response like 220, 250, 500,
et al

Optional:
11) sc-bytes (Bytes Sent): ***if you really need this info - handy at times
for troubleshooting***
12) cs-bytes (Bytes Received): ***if you really need this info - handy at
times for troubleshooting***
13) time-taken (Time Taken): ****if you really want this info***

Not sure:
1) sc-win32-status (Win32 Status): a numerical value, I've never found the use for this.
2) s-port (Server Port) - always logs as 0 afaik, even if SMTP vs is listening
on a port other than 25

Rolling over SMTP logs:
Rolling over logs on a daily basis seems like a good choice. Smaller or low-volume environments can roll over weekly, monthly, or when the file reaches a certain size. The unlimited file size option means the log will never roll over - not recommended.

The option to use local time for file naming and rollover exists - the timestamps in the logs will always use UTC (if you've gone through the whole Daylight Savings Time issue earlier this year you probably understand very clearly why this makes sense :).

Location of SMTP logs: By default, SMTP logs are located in %systemroot%\system32\LogFiles, in folders created for each SMTP virtual server instance - e.g. SmtpSvc1 for the default/first SMTP virtual server instance.

Starting logging is the first step, and brings with it some additional decisions to make and procedures to follow. It's a good idea to ask yourself why you're logging the information, how long does it remain valuable for that purpose, when will you remove old log files (and if any compliance policies in your organization require you to archive logs for a particular period of time).