Exchange Server 2007: Managing And Filtering Anti-Spam Agent Logs

Exchange 2007 includes a number of anti-spam agents to filter spam. The anti-spam agents log their actions in (anti-spam) agent logs. The default agent log locations:

• Exchange 2010: \Exchange Server\V14\TransportRoles\Logs\AgentLog
• Exchange 2007: \Exchange Server\TransportRoles\Logs\AgentLog

Agent Log Configuration

You can’t change the agent log location. Here are the available config options:

1. Enable/Disable agent log: On transport servers with anti-spam agents installed, agent logging is enabled by default. You can disable it by adding the AgentLogEnabled key, and setting it to FALSE.
2. Max file size: By default, the maximum file size of each agent log is 10 Mb (10485760 bytes). You can modify it by adding AgentLogMaxFileSize key and providing the desired value (in bytes). Parsing very large log files can be slower— the default max file size works for most deployments.
3. Max directory size:: By default, the size of the directory is capped at 250 Mb (262144000 bytes). You can modify it by adding the AgentLogMaxDirectorySize key and providing the desired value (in bytes). In a high volume environment, you must consider the volume of log files generated in a day, and determine the space required based on how many days’ worth of log files you want to retain for troubleshooting or to meet your organization’s retention requirements.
4. Agent log age: The age of a log is capped at 30 days. You can modify it by adding the AgentLogMaxAge key and providing the desired value. Logs are flushed when they’re older than the AgentLogMaxAge, or if the directory size reaches the AgentLogMaxDirectorySize (250 Mb by default), whichever happens first. If it’s important to maintain agent logs for a certain number of days, you must consider the volume of logs generated daily, and raise the AgentLogMaxDirectorySize if required.

Exchange 2007 RTM has only one configuration option for the agent log – that to enable or disable it. Exchange Server 2007 SP1 includes the additional configuration parameters listed above to control the max directory size, file size and age of agent logs added.

The agent log configuration parameters can be controlled by editing the EdgeTransport.exe.config file, located in \Exchange Server\Bin folder on Edge and Hub Transport servers.

Note, the key names in EdgeTransport.exe.config are case-sensitive.

Disable agent logging

To disable agent logging, insert the following key under <appsettings></appsettings> in the config file:

<add key=”AgentLogEnabled” value=”FALSE” />

Modify agent log configuration

In the following example, we modify max directory size to 500 Mb, file size to 20 Mb, and age to 60 days, by creating new keys in EdgeTransport.exe.config:

<add key=”AgentLogMaxDirectorySize” value=”524288000″ />
<add key=”AgentLogMaxFileSize” value=”20971520″ />
<add key=”AgentLogMaxAge” value=”60.00:00:00″ />

Parsing the agent log

You can parse the agent log using the Get-AgentLog command from the shell.

Agent log format and fields

Agent log file names are named AGENTLOGyyyymmdd-nnnn.log. It’s easy to decipher yyyy stands for the year, mm is the month, and dd is the date. The nnnn is a serial number, starting at 0001, rolled over when more than one log is generated in a day. The date is UTC.

Each agent log starts with the following header fields:

#Software: Microsoft Exchange Server
#Version: 8.0.0.0
#Log-type: Agent Log
#Date: 2010-08-07T00:00:01.500Z
#Fields: Timestamp,SessionId,LocalEndpoint,RemoteEndpoint,EnteredOrgFromIP,
MessageId,P1FromAddress,P2FromAddresses,Recipient,NumRecipients,
Agent,Event,Action,SmtpResponse,Reason,ReasonData,Diagnostics

Here’s what an entry in the agent log looks like, as output by the Get-AgentLog cmdlet. Note, not all fields are displayed by default when you use the cmdlet:

Timestamp : 4/16/2007 12:39:49 AM
SessionId : 08C948C83FB951AC
IPAddress : 72.46.133.113
MessageId :
P1FromAddress : [email protected]
P2FromAddresses : {}
Recipients : {[email protected]}
Agent : Connection Filtering Agent
Event : OnRcptCommand
Action : RejectCommand
SmtpResponse : 550 5.7.1 Recipient not authorized, your IP has been found on a block list
Reason : BlockListProvider
ReasonData : Spamhaus SBL-XBL
Diagnostics :

As seen in the above output, the logs provide adequate information for reporting on anti-spam activity, as well as for troubleshooting anti-spam issues like messages not being received/wrongly filtered out.

Getting to know the agent logs will make troubleshooting such issues much easier.

Searching agent logs

By default, the Get-AgentLog command returns all the entries in the agent logs. This can take a long time and results in all entries scrolling by quickly in your shell window. The only filtering options available with the Get-AgentLog cmdlet are start and end dates.

Filter by date and time

Agent logs can be constrained to a particular date and time – the recommended way to perform most agent log searches, unless you want to immerse yourself in 30 days (or 250 Mb) of anti-spam goodness! This is done using the -StartDate and -EndDate parameters, as shown in this example:

Get-AgentLog -StartDate “4/16/2007” -EndDate “4/17/2007”

You can also constrain it further by adding time of the day:

Get-AgentLog -StartDate “4/17/2007 8:00 AM” -EndDate “4/17/2007 2:00 PM”

Search messages by recipients and senders

Although the Get-AgentLog command only takes these 3 parameters – location, StartDate, and EndDate, you can further filter the logs using most of its logged fields by using pipelining.

To filter the log to show messages to a particular recipient:

Get-AgentLog -StartDate “4/16/2007” -EndDate “4/17/2007” | where {$_.recipients -like “[email protected]”}

To search for messages from a particular sender:

Get-AgentLog -StartDate “4/16/2007” -EndDate “4/17/2007” | where {$_.P1FromAddress -like “[email protected]” -or $_.P2FromAddresses -like “[email protected]”}

Search messages from a sender domain

To search for messages from a particular domain:

Get-AgentLog -StartDate “4/16/2007” -EndDate “4/17/2007” | where {$_.P1FromAddress -like “*somedomain.com” -or $_.P2FromAddress -like “*somedomain.com”}

Search messages filtered by a specified anti-spam agent

To filter by the anti-spam agent that acted on a message, e.g. Connection Filtering Agent:

Get-AgentLog -StartDate “4/15/2007” -EndDate “4/17/2007” | where {$_.Agent -eq “Connection Filtering Agent”}

Similarly, you can filter by other agents that write to the agent logs: 1) Content Filter Agent 2) SenderID agent 3) Sender Filter agent 4) Recipient Filter agent and 5) Edge Rules agent.

Searching by IP address

To filter agent logs by the sending host’s IP address, use the following command:

Get-AgentLog -StartDate “4/15/2007” -EndDate “4/17/2007” | where {$_.IPAddress -eq “72.46.133.113”}

Search messages blocked by RBLs

The reason field in each log entry specifies the reason supplied by the anti-spam agent that takes the action. For instance, as seen in the agent log entry shown earlier in this article, the agent that acted on the message is the Connection Filtering Agent, the reason is BlockListProvider (i.e. “RBL” or “Real-time Block List”, known as IP Block Lists in Exchange Server 2007). The ReasonData field gives you the name of the IP Block List Provider, as configured in Exchange. In the above agent log entry, it is “Spamhaus SBL-XBL”. To constrain the search for messages blocked by IP Block List Providers:

Get-AgentLog -StartDate “4/15/2007” -EndDate “4/17/2007” | where {$_.Reason -eq “BlockListProvider”}

To get a list of all IP addresses blocked by IP Block List Providers:

Get-AgentLog -StartDate “12/21/2007” | where {$_.Reason -eq “BlockListProvider”} | ft Timestamp,IPAddress,ReasonData

You can also look for messages blocked by a particular IP Block List Povider:

Get-AgentLog -StartDate “4/15/2007” -EndDate “4/17/2007” | where {$_.ReasonData -eq “Spamhaus SBL-XBL”}

Search by SCL thresholds

For messages scanned by the Content Filter Agent, the Reason field contains details like SCLAtOrAboveDeleteThreshold, SCLAtOrAboveRejectThreshold, etc. The ReasonData field contains the SCL value assigned to the message. To get a list of messages above the SCLDeleteThreshold, use the following command:

Get-AgentLog -StartDate “4/15/2007” -EndDate “4/17/2007” | where {$_.reason -eq “SCLAtOrAboveDeleteThreshold”}

As shown in the above examples, you can use the Get-AgentLog command and pipe the data to filter it based on the fields logged. You can get more details about agent logs – including the fields logged, from the Managing Agent Logging section in Exchange Server 2007 documentation.

2/9/2008: Added sidebar about additional configuration parameters available in SP1.

Tags
Administration   Anti-Spam   Exchange 2007   Exchange 2010   Security   SMTP