Errata: Edge Subscription and Synchronization white paper

The white paper on Edge Subscription and Synchronization has the following error:

Under Recipient Information:

Distribution groups are not replicated to ADAM.

Distribution Groups are in fact replicated to ADAM using EdgeSync. In Exchange Server 2007 SP1, Distribution Group membership (the member attribute) is also replicated.

On Windows Server 2008, ADAM is replaced by Active Directory Lightweight Directory Services (AD LDS).

Note that new Distribution Groups created in Exchange Server 2007 are set to receive mail only from authenticated senders by default— preventing them from receiving internet mail. Any Exchange recipients set to receive mail only from authenticated senders are not replicated by EdgeSync.

Related Posts:
- Configuring firewalls and name resolution for Edge Transport servers
- New Distribution Groups do not receive internet email by default

Assigning SCL to messages scanned by 3rd-party antispam filters

Here's a problem I had a hard time resolving on Exchange Server 2003. Exchange Server 2007's Transport Rules resolve this within minutes.

Pretend you're taking a Microsoft exam:
Scenario: "You are the Exchange administrator for your organization... ".

Exchange has the Content Filter Agent (CFA), and the Edge Transport Server role designed to be a non-domain-joined mail gateway, located in perimeter networks. However, the Unix/Linux/Security folks in your organization don't trust Exchange to do the filtering (or act as the mail gateway). They insist on using open source anti-spam software, such as SpamAssasin (yes Eric, I remember CRM114 - the Controllable Regex Mutilator... and all the cool stuff you can command it to do. Just to set the record straight, I was suitably impressed back then, and continue to be so till this day.. :) on the non-Exchange SMTP gateways. After tweaking it for a number of weeks, they are able to make it work the way they want it to, or are close to it. It's implementation time!

Their solution is to insert an X-header in messages that looks like this:

X-Spam-Status:yes

That's it. Their job ends there.

As the Exchange team/administrator, your job is to ensure messages with that header end up in users' Junk Mail folder.

Exchange Server 2003 did not provide any way, out-of-the-box, to be able to inspect message headers and stamp SCL. To add some contextual fun - back then, our "solution" was a doc that showed the end-users how to create an Outlook rule to move such messages to Junk Mail. Only if Exchange/Windows admins could code... I can hear you think.

 Creating a Transport Rule: Exchange Server 2007's Transport Rules functionality allows you to accomplish this easily. Here's how:
1. Fire up EMC | Organization Config | Hub Transport | Transport Rules tab
2. Click on New Transport Rule in the Action pane
3. Give the new rule a name, add a comment if you wish
4. In the Conditions page, select the condition when a message header contains specific words
5. In the Step 2 edit box, click on the message header link

6. Type X-Spam-Status | click OK
7. In the edit box, click on the specific words link
8. Type yes | click OK | click Next
9. In the Actions page, select the action set the spam confidence level to value
10. In the rule description, click on the 0 link and add a value that's above your SCLJunkThreshold | click Next
11. On the Exceptions page, click Next if you do not want any exceptions to this rule
12. Click New | click Finish to close the wizard

Or you can use the following commands:

$condition = Get-TransportRulePredicate HeaderContains
$condition.MessageHeader = "X-Spam-Status"
$condition.words = @("yes")
$action = Get-TransportRuleAction SetSCL
$action.SCLValue = 5
new-TransportRule "Stamp SCL" -condition @($condition) -action @($action)

 Disabling the Content Filter agent: Since you have a 3rd-party filtering solution running on your non-Exchange SMTP host(s), you can disable the Content Filter Agent. Messages exceeding SCLJunkThreshold will still be moved to Junk Mail folder.

Disable-TransportAgent "Content Filter Agent"

Alternatively, you can leave the CFA enabled, but disable the Delete, Reject and Quarantine actions.

Set-ContentFilterConfig -SCLDeleteEnabled $false -SCLRejectEnabled $false -SCLQuarantineEnabled $false

Send a test message with the X-header X-Spam-Status:yes. The message has the SCL value set by the Transport Rule. If it is above the SCLJunkThreshold, it should be delivered to the Junk Mail folder.

Why this doesn't work with Delete, Reject or Quarantine thresholds enabled?

The Content Filter Agent fires on the EndOfData SMTP event. On Hub Transport servers, the Transport Rules Agent fires on the OnRoutedMessage event. If the CFA is left enabled, along with any of the above actions, messages may get deleted, rejected or quarantined. The Transport Rules Agent never gets to see deleted and rejected messages at all.

What about quarantined messages?

It's understandable if the message is already deleted or rejected - there's nothing left for the Transport Rule agent to act on! However, why doesn't the agent act on quarantined messages? With Pipeline Tracing turned on, I spent some precious cycles trying to unravel the mystery, with little success. As it turned out (thanks to helpful Exchange team folks for pointing out... ), I was overlooking the obvious - Quarantined messages are wrapped in an NDR "envelope". If you've ever logged into the quarantine mailbox and looked at quarantined messages, you know what these look like. (Related post: "HOW TO: Expose original senders and recipients of quarantined messages") The Transport Rules Agent does not see the original message headers, including our X-Spam-Status header. As a result, the rule never fires.

Life on the Edge

If using an Edge Transport server in the mix, things change. The Edge Rule agent fires on the EndOfData event, unlike Hub Transport servers. When two transport agents fire on the same SMTP event, you can set the priorities of each accordingly so one fires before the other. This is described in "How to Make the SCL Value Available to Edge Transport Rules" in Exchange Server 2007 documentation.

However, it's a lot simpler to just disable the Content Filter Agent. This meets the requirements of this scenario, where anti-spam filtering is done by 3rd-party anti-spam filters and Exchange isn't required to do any filtering at all.

Keeping tabs on Antispam filters: A few handy scripts in Exchange Server 2007

One of the more useful improvements in Exchange Server 2007 is the abundance of logging for different features and components (read previous post "Exchange Server 2007: How many logs hath thee?"). In particular, the antispam agent logs fill an important gap in monitoring, reporting and troubleshooting message flow as it relates to antispam agents (read previous post "Exchange Server 2007: Managing And Filtering Anti-Spam Agent Logs"). As a messaging/Exchange administrator, you want to be able to pin point what the antispam agents have been up to, and determine if particular messsages, or sending hosts, domains or email addresses have been blocked by any of the antispam agents. The antispam agent logs and the get-agentlog command allow you to do this quickly and efficiently.

In addition to the the get-agentlog command, Exchange Server 2007 also ships with a number of canned scripts that help you keep tabs on what the agents are doing. These scripts are found in the \Exchange Server\Scripts directory, where \Exchange Server is the path of the Exchange Server 2007 installation. Note, no documentation or support is available for these scripts - they are meant to be examples you can use to write your own scripts. Let's take a peek in the directory and see what we find.

Commandline parameters used by antispam scripts

Most of the following scripts take the same (optional) parameters.
-top n: Where n is the number of results to display. If not specified, the script defaults to (top) 10.
-StartDate: Start date/time
-EndDate: End date/time
-location: path of agent log files. If no path is specified, the agent works against the default agent log file location.

1. Get-AntispamFilteringReport.ps1: Takes one of the following values as a mandatory parameter: 1) connections 2) commands 3) messagesrejected 4) messagesdeleted 5) messagesquarantined, and displays statistics for each agent.
2. Get-AntispamSCLHistogram.ps1: Provides a breakdown of number of messages stamped with each SCL value.
3. Get-AntispamTopBlockedSenderDomains.ps1: Lists top ten sender domains from which mail was blocked.
4. Get-AntispamTopBlockedSenderIPs.ps1: Lists the top ten IP addresses blocked by antispam agents, and number of messages blocked from each.
5. Get-AntispamTopBlockedSenders.ps1: Lists the top ten blocked senders (SMTP email addresses) and number of messages blocked from each. The script can report on P1 (i.e. address in message envelope in the MAIL header) or P2 addresses (from headers in message body like FROM). Specify the option as a commandline argument (optional parameters -top n, -StartDate, and EndDate used in this example):

.\Get-AntispamTopBlockedSenders.ps1 P1 -top 20 -StartDate "12/1/2007" -EndDate "12/10/2007"

Replace the P1 in the above command with P2 to report on P2 senders.
6. Get-AntispamTopRBLProviders.ps1: Lists the top ten RBLs (aka "IP Block List Providers") and messages blocked by each (read previous post: "Exchange Server 2007: How are RBLs performing?").
7. Get-AntispamTopRecipients.ps1: Lists the top ten recipient addresses that receive spam. The addresses may or may not exist in your Organization - the reporting is based on actions taken by antispam agents on incoming messages. Recipient Filtering with Recipient Validation (dropping messages for recipients that do not exist in AD/GAL) is a great way to drop a large number of messages.

The following scripts are not used for reporting:
8. Reset-AntispamUpdates.ps1: Uninstalls any antispam updates and reinstalls the original "out-of-box" antispam data.
9. Install-AntispamAgents.ps1: By default, antispam agents are not installed on the Hub Transport server role (read previous post: "HOW TO: Install anti-spam agents on Hub Transport server"). This script installs them if you need to.
10. Uninstall-AntispamAgents.ps1: Uninstalls antispam agents.

Disabled mailboxes: Can they really receive email?

Some truths you live with for a lifetime, like Outlook users cannot send email using an alternate email address (with Outlook in MAPI mode - read previous post: "HOW TO: Send as alternate email address"). Others change as Microsoft Exchange evolves, either through new versions of Exchange server, or service packs and hotfixes.

Disabled mailboxes cannot receive email. Or rather, could not receive email. This has been true all this while, and hasn't changed in Exchange 2000, Exchange Server 2003, including SP1 and SP2.

The reason is described in Microsoft KBA 319047 - "You receive a non-delivery report when you send a message to a disabled account".

In addition to not being able to receive email, administrators face a few other operational issues when managing disabled mailboxes:

- A common scenario: An employee leaves. You disable the account, as part of a standard operating procedure followed in most organizations. You assign his/her manager/co-worker/replacement permissions to access the mailbox. If the mailbox is disabled, they can't access it!
- A disabled mailbox needs to be be enabled first before it can be moved (KBA 278966: "You cannot move or log on to an Exchange resource mailbox")
- The Application Event Log is flooded with annoying Event ID 9548s, informing you that the disabled account does not have a msExchMasterAccountSID attribute populated - something most Exchange administrators have probably gnashed their teeth at a few times a day.

Workarounds exist to populate the msExchMasterAccountSID attribute with the well-known SELF SID (KBA 322890 "How to associate an external account with an existing Exchange 2000 mailbox"), but not something you want to do on a regular interval after every account, or a bunch of them, are disabled.

The hotfix mentioned in KBA 903158: "A hotfix is available to modify the way that Exchange Server 2003 handles a disabled Active Directory user account that is associated with an Exchange Server 2003 mailbox" changes that behavior. It makes the Store act as if a disabled account with a null/empty msExchMasterAccountSID attribute actually has the SELF SID.

All the above actions (and more) complete successfully for disabled accounts. Yes, disabled accounts receive email if you have this hotfix applied - or any subsequent hotfix that updates Store.exe to version 6.5.7234.3 or later.

But what if I don't want disabled accounts to receive email? To prevent disabled accounts from receiving any email, setup Delivery Restrictions (in ADUC | user -> properties | Exchange General tab) to:


Figure 1: Setting Delivery Restrictions on a recipient to prevent receiving mail

1. Receive mail from authenticated users only: With Recipient Filtering enabled, this will drop internet mail at the gateway or the first Exchange server that receives inbound internet mail
2.Receive mail only from: a particular Distribution Group (use a Distribution Group with no members).

220 Welcome to a California mail server: Fighting spam with SMTP Banners

If you are interested in messaging and fighting spam, you probably watch the legal response to spam with some interest. Given the nature of email and the art of remaining anonymous or otherwise untraceable that spammers seem to have mastered, anti-spam laws were written off as largely ineffective, or even ridiculous. (The FTC begs to differ, in its report to Congress on the CAN-SPAM Act's effectiveness - PDF available here). Legislation and (its) enforcement can never be the sole approach to a problem mostly attributed to technology.

The much-talked about first conviction under the federal CAN-SPAM Act was handed out in California - to Nicholas Tombros in Los Angeles in 2004. Since then, quite a few spammers have had their date with the law, with the accompanying media brouhaha.

California anti-spam laws: California also has its own antispam laws - enacted as amendments to the Business and Professions Code. The rather lame one - Section 17538.4, allows California Attorney General to sue spammers, with a list of IFs that make it more like a "License To Spam". Section 17538.45 is the one of interest, aimed at protecting email service providers from spammers. Email service providers can file civil suits against spammers and claim greater of the following amounts:
- the actual monetary loss suffered by reason of that violation
- liquidated damages of $50 per message initiated or delivered, up to a maximum of $25,000 per day.

Any organization that provides the ability to send or receive email to registered users through equipment located in California (and is an "intermediary" in sending/receiving email) is an email service provider for the purpose of this law.

Interesting, as technology-related legislative action always is.

If your mail servers are located in California (and even if they are not... ), it makes sense to change the SMTP banner (SMTP 220 response) to give notice to a sender that your system should not be used to send unsolicited electronic mail advertisements.

The California Business and Professions Code Section 17538.45 states:

(3) (A) In any action brought pursuant to paragraph (1), the electronic mail service provider shall be required to establish as an element of its cause of action that prior to the alleged violation, the defendant had actual notice of both of the following:

(i) The electronic mail service provider's policy on unsolicited electronic mail advertising.

(ii) The fact that the defendant's unsolicited electronic mail advertisements would use or cause to be used the electronic mail service provider's equipment located in this state.

(B) In this regard, the Legislature finds that with rapid advances in Internet technology, and electronic mail technology in particular, Internet service providers are already experimenting with embedding policy statements directly into the software running on the computers used to provide electronic mail services in a manner that displays the policy statements every time an electronic mail delivery is requested. While the state of the technology does not support this finding at present, the Legislature believes that, in a given case at some future date, a showing that notice was supplied via electronic means between the sending and receiving computers could be held to constitute actual notice to the sender for purposes of this paragraph.

Let's leave the legal interpretation of laws to those who practice law. Regardless of whether your organization intends to take legal action against spammers or not, one of the important principles of information security is having policies in place and communicating them clearly. Not only does it demonstrate your organization's intent, it can be a potential deterrent for folks who may otherwise claim ignorance of such policies.

(While we're on the subject of SMTP banners, for historical context, also take a look at Paul Hobbes & John Levine's draft from 1998 titled "Anti-UBE and Anti-UCE Keywords in SMTP Banners". Much later, a SMTP extension was also proposed to include such information - RFC 3865 - A No Soliciting Simple Mail Transfer Protocol (SMTP) Service Extension.)

Changing the SMTP banner: Assuming the policy is in place, the SMTP banner can be changed easily to communicate it, as hosted email security and compliance services provider Postini (acquired by Google not too long ago) does.

220 Postini ESMTP 11 y6_12_2c0 ready. California Business and Professions Code Section 17538.45 forbids use of this system for unsolicited electronic mail advertisements.

On Exchange Server 2003/2000 or if using Windows Server's SMTP service component on SMTP gateways, the banner text can be changed by setting the IIS Metabase value ConnectResponse, using any Metabase editor or the adsutil.vbs script that ships with IIS (in the Inetpub\AdminScripts directory by default).

cscript adsutil.vbs set smtpsvc/1/ConnectResponse "220 California Business and Professions Code Section 17538.45 forbids use of this system for unsolicited electronic mail advertisements."

Note, the numeric value 1 after smtpsvc/ in the above command refers to the SMTP Virtual Server ID. Each SMTP VS has one, the default/first one starting at 1. To get a list of SMTP Virtual Servers using adsutil.vbs:
adsutil.vbs enum /p smtpsvc

Exchange Server 2007 makes changing the banner a relatively simple task. Note, the banner should start with the SMTP response code 220. Fire up the shell:

Set-ReceiveConnector "Connector Name" -banner "220 (Optional: Server.domain.com). California Business and Professions Code Section 17538.45 forbids use of this system for unsolicited electronic mail advertisements."

That wasn't too difficult, was it? If that simple banner change deters a a single spammer from victimizing your users, and in the process buys you the legal ammunition (that may some day help your organization take legal action, should it decide to.... ), is it worth it?

Connection Filtering, RBLs and SMTP logs

Exchange Server 2003's Connection Filtering feature allows you to block connections from IP addresses explicitly added to the Global Deny List, or drop messages from IP addresses listed on a RBL (Real-Time Blackhole List / Real-Time Block List).

Note: The term "RBL" is commonly used to describe DNS Black Lists (DNSBLs), but it's a trademark of Mail Abuse Prevention System (MAPS), one of the first RBLs. It was acquired by Trend Micro in 2005. Exchange Server 2007 uses the term IP Block List Provider.

When connections from IP addresses listed on the Global Deny List are dropped, you don't find any traces of those dropped connections in SMTP logs [for more details about SMTP logging in Exchange Server 2003, read previous post "Logging SMTP protocol activity"].This is understandable - the IP addresses are explicitly added to the deny list by an administrator, and a SMTP session is never initiated. It's not as important to log these (though many would prefer to capture information about connections so blocked). If in doubt about which hosts are getting blocked, all one needs to do is look at this list.

When it comes to DNSBLs, the behavior is different. Connections are not dropped. Messages are dropped on RCPT TO: command. This allows you to receive mail for addresses on the Exception List. RFC 2821 requires senders to be able to contact the postmaster of a domain. You can also add some of the generic addresses from your domain to the Exception List if you don't want email to these addresses blocked.

Here's a demo of the SMTP session. As seen in the session, the sending host gets a 550 5.7.1 error code because the IP address is listed on a DNSBL, and the recipient (in the RCPT TO: command) is not listed on the Exception List.

Here's what the SMTP log shows (some irrelevant fields removed):

2007-07-25 19:04:54 172.31.1.10 0 HELO - - 250 0 49 4 0 SMTP - - - -
2007-07-25 19:04:59 172.31.1.10 0 MAIL - +from:foo@mydomain.com 250 0 41 26 50 SMTP - - - -
2007-07-25 19:05:11 172.31.1.10 0 RCPT - +to:jadams@exchangelabs.net 550 0 0 31 5017 SMTP - - - -
2007-07-25 19:05:15 172.31.1.10 0 QUIT - - 240 22943 72 4 0 SMTP - - - -

SMTP logs the response code 550, but not the enhanced status code 5.7.1. Having said that, this enhanced status code doesn't by itself allow you to narrow it down to DNSBLs.

Exchange Server 2007's Connection Filtering agent behaves differently - connections from IP addresses explicitly listed in the IP Block List (equivalent of the Global Deny List in Exchange Server 2003) are not blocked. This change in behavior allows such connections to be logged. Detailed information is available from the agent logs [read previous post "Exchange Server 2007: Managing And Filtering Anti-Spam Agent Logs"] about why a message was accepted/rejected by anti-spam agents.

One example of the kind of reporting/troubleshooting capabilities this provides: you can search the agent logs for messages blocked by Connection Filtering agent based on DNSBL lookups:

Get-AgentLog -StartDate "7/27/2007" | where {$_.Reason -eq "BlockListProvider"}

The Get-AgentLog output:

Timestamp : 7/27/2007 6:42:08 AM
SessionId : 08C99DA727C8EF31
IPAddress : 68.45.233.59
MessageId :
P1FromAddress : dbuuxwxjlm@bobach.com
P2FromAddresses : {}
Recipients : {bf5b7b59@mydomain.com}
Agent : Connection Filtering Agent
Event : OnRcptCommand
Action : RejectCommand
SmtpResponse : 550 5.7.1 Recipient not authorized, your IP has been found on
a block list
Reason : BlockListProvider
ReasonData : Spamhaus SBL-XBL
Diagnostics :

SCRIPT: Turning on Filter Junk Email

OWA users who never logon using Outlook do not have their Junk Mail filtering option turned on by default. Exchange MVP Glen Scales has a script here that allows you to turn this on programmatically for Exchange Server 2007 users.

Why Get-TransportAgent doesn't agree with the Exchange console

You disable a particular anti-spam agent - let's say the Content Filtering Agent - from the Exchange console.



Next, you use the Get-TransportAgent command to get the status of transport agents - and surprisingly the Content Filter Agent shows up as Enabled!



Why doesn't Get-TransportAgent agree with the Exchange console about the status of an agent?

Fellow MCT and Exchange geek David Elfassy blogged about it recently.

The Anti-Spam tab is located under the Organization Configuration | Hub Transport node in the console. Disabling an agent from the console prevents the agent on all transport servers in the Organization from taking any action. It's an Organization-wide or global setting.

The Get-TransportAgent command is transport server-specific. It shows the status of agents on a particular server. With the Org-wide configuration set to disabled, an agent enabled on a particular transport server behaves like a soldier instructed to fire but not kill. (May not be the best analogy - it doesn't exactly scare spam, and it doesn't do much else).

To draw a parallel to Exchange Server 2003, you can enable settings on a particular filter - e.g. Recipient Filtering, in Global Settings. However, you still have the flexibility of being able to enable/disable it on a per-SMTP virtual server basis. This is somewhat similar.

To get the global/Org-wide status of a particular filter, you need to use the corresponding command for that filter.
- For Content Filter Agent: Get-ContentFilterConfig
- For Recipient Filter Agent: Get-RecipientFilterConfig
- For Sender Filter Agent: Get-SenderFilterConfig
- For SenderID Agent: Get-SenderIDConfig
- For Protocol Analysis Agent: Get-SenderReputationConfig

If you've carefully looked at the above list, you're probably wondering why there's no mention of Connection Filtering Agent in it. The Connection Filtering Agent handles multiple tasks which are listed individually in the console.

To check the global setting for configuration of each, use the corresponding config command:
- For IP Block List: Get-IPBlockListConfig
- For IP Block List Providers: Get-IPBlockListProvidersConfig
- For IP Allow List: Get-IPAllowListConfig
- For IP Allow List Providers: Get-IPAllowListProvidersConfig

To "disable" an agent in Organization Configuration (equivalent of what you do in the console) using the shell, use the corresponding Set command, e.g.:

Set-ContentFilterConfig -Enabled $false

Next, let's take a look at how you can disable an agent on a transport server. Generally, if the Get-Blah command returns a property like Enabled: True, you can use Set-Blah -Enabled $false to disable it (You probably know where we're going with this by now... :). Not so for Get-TransportAgent. Its "Set" counter-part - Set-TransportAgent allows you to modify only one property -Priority. You can change the order in which these agents fire by changing their Priority.

You use the Disable-TransportAgent command to disable an agent on a transport server:

Disable-TransportAgent "Connection Filtering Agent"

No points for guessing the command used to enable a transport agent - Enable-TransportAgent

Header Firewall: Why spammers can't insert fake SCL (and other Exchange Organization) X headers

After almost a year and a half of working with Exchange Server 2007, I still stumble into features I haven't tested before. When it happens, it blows holes into my theories about how well I know E12. :)

It started when investigating why a certain message - likely not from an authenticated source, would have a SCL of -1 (seen in message headers as X-MS-Exchange-Organization-SCL: -1).

One suggestion that got me thinking: it may have been a fake header, perhaps inserted by a spammer. I tested this a few times in different combinations (by inserting SCL and PCL headers before, after and in between other X-headers, like the ones below:


Note: Exchange uses the PCL and SCL headers seen above, but the QCL header in the above screenshot is fabricated/fake.

When the above messages came in, all the headers with X-MS-Exchange-Organization* were removed by Exchange. Nice work, Microsoft, I said to myself, and asked for an explanation.



Someone from the team immediately pointed me to Header Firewall - a supercool feature (not so much in what it does but the fact that it exists). It removes protected Exchange Organization and Forest headers from messages that enter or exit the Exchange Organization. This is the reason you're not likely to see fake SCL, PCL, SenderID, and other such headers in messages received by Exchange Server 2007 Edge or Hub Transport servers.

Preserving Organization and Forest X headers: If you want these to be preserved for inbound mail from particular sources, it is recommended to do so using a separate Receive Connector, just as you would create one if you needed to allow unauthenticated relaying [read previous post "Exchange Server 2007: How To Allow Relaying"], and assign MS-Exch-Accept-Headers-Organization and MS-Exch-Accept-Headers-Forest permissions. Similarly, to send Organization and Forest headers to particular external mail systems, it is recommended to create a separate Send Connector for those address spaces, and assign the MS-Exch-Send-Headers-Organization and MS-Exch-Send-Headers-Forest permissions.

Getting back to the original question - when do you see a SCL of -1? Either when the sender is authenticated, or the sending host is on the IP Allow List.

Read more about the Header Firewall feature in the Exchange Server 2007 documentation. A list of Exchange Organization and Forest x-headers can also be found there.

Exchange Server 2007 Enterprise CALs: Let Microsoft do the filtering

Microsoft introduced the concept of Standard and Enterprise Client Access Licenses (CALs) for Exchange Server 2007. In previous versions, only the server part of Exchange came in Standard or Enterprise flavors, the latter allowing you the use of enterprise features like more number of Storage Groups and Stores, no limits on Store sizes, clustering, etc.

With a lot more features and functionality in Exchange Server 2007, the Client Access Licenses come in both flavors as well.

If you're planning to buy Enterprise CALs for your Exchange Server 2007 deployment, these come with an interesting benefit. Yes, Microsoft's ForeFront for Exchange is part of the deal, and though it's a good product (its predecessor - Sybari Antigen for Exchange, was for long considered the best of breed anti-virus products for Exchange), that's not the interesting benefit.

That distinction goes to Microsoft Exchange Hosted Filtering, which is part of the Enterprise CAL offering. Yes, this is Microsoft's software product + services play - you can outsource your anti-spam and anti-virus filtering headaches to Microsoft. You may not have to deploy any Edge Transport servers (depending on your organization's security policies, of course.. ).

If you're wondering what the new Standard and Enterprise CALs offer, this page has it all. (Well, almost... - the nuances of Exchange Server 2007 licensing are lost to many, so always get a quote and details from your software vendor or Microsoft rep.).

Things to note:
- the Enterprise CAL is an add-on to the Standard CAL.
- Depending on Exchange Server features used, not all users need an Enterprise CAL.
- Standard Journaling, which offers the same granularity as Exchange Server 2003 - that of journaling at the Store level, does not require an Enterprise CAL
- Premium Journaling: Per user or per distribution list journaling - you need an Enterprise CAL for only the users that'll be subject to/use premium journaling.
- Simply because you require or have Entperprise CALs for users doesn't necessarily mean you require Enterprise Edition licenses for Exchange Server. The Enterprise Edition is required only if you want to scale up beyond 5 Storage Groups/Mailbox Stores, or if you want to use clustering using Exchange Server 2007's Single Copy Cluster (SCC) or Cluster Continuous Replication (CCR).
- As Paul Robichaux notes in WinITPro's "Exchange and Outlook UPDATE" newsletter, there has been a licensing change in CAL requirements: Managed Default Folders like Inbox, Deleted Items, Junk Mail, etc. no longer require an Enterprise CAL, restoring the equivalent of Exchange Server 2003/2000 Mailbox Manager policies (i.e. Recipient Policies with Mailbox Manager settings) without an additional licensing cost. Managed Custom Folders feature, which allows you to create and manage custom folders in user mailboxes, requires an Enterprise CAL.

Note: Mailbox and Public Folder Stores in Exchange Server 2007 Std. Edition do not have size restrictions like the 16 Gb. limit in previous versions. The limit was raised to a max of 75 Gb. in Exchange Server 2003 SP2, and removed completely from Stores in the Std Edition of Exchange Server 2007.

Moving items to Junk Mail folder doesn't do much

Scenario: Junk mail makes it to your Inbox, without getting filtered by any of Exchange's anti-spam filters.
Action: User moves item to Junk Mail folder.
Result: This accomplishes little more than having the piece of UCE moved out of the Inbox. Contrary to what many may think and like to believe, moving a message manually to the Junk Mail folder doesn't really have Outlook or Exchange do anything at all with that item. No, the sender or sending domain or host do not get blacklisted or added to Microsoft Outlook's Blocked Senders List. Yes, you're not likely to see a similar message from same sender, domain or host get treated any differently based on that action.

To make Microsoft Outlook filter messages from the same sender, the sender needs to be added to the Blocked Senders list manually (by right-clicking message | Junk E-mail | Add Sender to Blocked Senders List).



Does the Blocked Senders List make it to the Exchange Hub or Edge server using the Safelist Aggregation feature? No. Safelist Aggregation is only used for aggregating the safe list, as the name of this feature correctly suggests. The Blocked Senders List is only used by Microsoft Outlook, without any server-side action being taken on messages from senders on this list.

Related Posts:
- Exchange 2007 Content Filter: How to move messages to users' Junk Mail folder
- TechEd 2007: Notes from the BOF Session on Fighting Spam With Exchange Server 2007
- Exchange 2007 Content Filter: The Whitelist Is Here!
- HOW TO: Expose original senders and recipients of quarantined messages
- Connection Filtering, RBLs and SMTP logs

TechEd 2007: Notes from the BOF Session on Fighting Spam With Exchange Server 2007

Tuesday night was a Birds of a Feather (BOF) night, with plenty of BOF sessions and chalk talks scheduled. My session on Fighting Spam With Exchange Server 2007 (not a very creative title, I know... but it got the message across :) was scheduled for 7:45 PM. I was positive I'll probably be the only person in the room - 7:45 PM isn't exactly the time I'd want to attend a TechEd session on anything, with the exception of a few good speakers.

On a side note, keeping my expectations low also meant I could claim the attendance was more than I expected even if only one person showed up. Nevertheless, the room filled up as we got closer to the start time.

It was an interesting discussion on anti-spam in Exchange Server 2007, as well as Exchange 2003.

Some takeaways:

• Exchange's anti-spam features consist of more than just Intelligent Message Filter (IMF) - there's 1) Connection Filter: allows using IP Allow Lists, IP Block Lists, IP Block List Providers (RBLs) and IP Allow List Providers 2) Sender Filter: block sending domain or SMTP address 3) Recipient Filter: block internet messages to list of valid recipients, perform Recipient Validation to block messages for non-existent recipients 4) Sender ID Filter: looks up SPF records in purported sending domain's DNS zone, protects from address spoofing and phishing 5) Content Filter (aka "IMF"): Blocks messages based on content
• 3 Gateway Thresholds and Actions: Exchange Server 2007's Content Filter Agent supports 3 gateway thresholds and actions - 1) Delete 2) Reject and 3) Quarantine
• Quarantine functionality: Though the quarantine functionality in Exchange Server 2007 is a step forward compared to Exchange Server 2003's archiving feature, it's not as full-featured as that offered by many third-party products. End-users have no control over the quarantine - an administrator has to inspect quarantined messages and make the decision to deliver or delete. However, it's important to understand that a) it's not compulsory to use the quarantine and b) The end-user mechanism for accessing suspicious messages is the Junk Mail folder c) Quarantining is one more layer available to admins - between the gateway actions of Rejecting or Deleting mail and the Store action of delivering to Junk Mail
• Exchange Server 2007's Connection Filter agent does not immediately drop connections from IP addresses explicitly listed on the IP Block List. Exchange Server 2003 does this for IPs listed in its Global Deny List. This change in behaviour allows such "filtered" connections to be logged in the agent log. Some customers like to see these logged, whereas many want connections to be dropped immediately.
• Many customers prefer anti-spam appliances like Barracuda and IronPort in the perimeter, and consider these very effective, and generally lower cost compared to an Exchange 2007 Edge Transport server.
• No studies or test results are available yet to demonstrate how the Edge Transport server performs compared to such appliances. It would be great to have a feature and performance comparison of these.
• Many customers are interested in Greylisting - a technique that registers each connection's "triplet" of a) Sending host's IP address b) Sender's address c) Recipient's address and looks up a database. If not found, the connection is immediately dropped and the triplet added to the database. The technique is based on the idea that valid RFC-compliant SMTP hosts will attempt to resend the message after a set interval. The (receiving) SMTP server will find the triplet in the database upon the second attempt, and consequently accept the connection.
• Interested in learning more about Greylisting? Evan Harris has a whitepaper titled "The Next Step in Spam Control: Greylisting"
• Exchange Server 2007 SDK has the code for a Greylisting agent. This needs to be compiled into a DLL using Visual Studio 2005, and installed using the accompanying installation script. Since this is just a programming sample provided in the SDK, it makes sense to test this adequately in a test environment and not expect any production support
• Freeware Greylisting transport sinks/add-ins for Exchange Server 2003 are available on the web, like this one from Chris on Grynx.com
• Over a period of time, the effectiveness of techniques like Greylisting may decrease, but as of now it still eliminates 80-90% of spam.
• SMTP Tarpitting is a technique that inserts a delay in SMTP responses to SMTP clients that display suspicious protocol behavior. More details about tarpitting in a previous post titled "Enabled by default: SMTP Tarpit in Exchange Server 2007"
• Safelist Aggregation: This allows Exchange to aggregate senders marked as safe by (Outlook/OWA) users. If an EdgeSync subscription exists (in topologies with an Edge server), the safe list is replicated to the Edge as part of EdgeSync. Exchange's use of safe lists can reduce false positives significantly. It's not required to have an Edge server - Hub Transport servers with anti-spam agents installed can also use this feature.
  

TechEd 2007: Let's talk about fighting spam with Exchange Server 2007

If you're going to be at Microsoft TechEd 2007 in Orlando next week, and are interested in sharing your experiences in fighting spam using Exchange Server's anti-spam features, here's a Birds-of-a-feather (BOF) session just for you.

What: BOF session "Fighting Spam With Exchange Server 2007"
When: Tuesday, June 5th @ 7:45 PM
Where: Microsoft TechEd, Orlando, FL
What's a BOF session? : Birds-of-a-feather sessions are not presentations or lectures. These are meant to be interactive discussions, sans the PowerPoint slides and demos you typically see in breakout sessions.
Why so late?: Tuesday night is an all-BOF session night, with no other conference activities. Microsoft will be serving food and drinks.

Abstract:
“Exchange Server 2007 anti-spam features are a significant improvement over previous versions, and come with interesting deployment and topology questions. Are these effective in the fight against spam? This BOF session will be a round table for Exchange administrators to talk about their experiences with Exchange Server 2007 anti-spam features, including issues such as Edge servers, use of IP Block List Providers, and new features and techniques such as Sender Reputation, and Safelist Aggregation.”
Audience: IT Professional

Grimes Gripes About RBLs

InfoWorld columnist Roger Grimes gripes about Real-Time Block Lists (aka Real-Time Blackhole Lists, DNSBLs or DNS-based Block Lists. The Exchange Server 2007 term for RBLs is "IP Block List Providers").

The feelings towards RBLs are understandable - you won't be a fan of RBLs if your IP addresses are the ones getting listed, and your users breathing down your neck about bounced messages. It's important to monitor RBLs to determine if your IP addresses are listed in any. (Incidentally, Zenprise for Exchange does this automatically several times a day - configurable by the administrator - and alerts you if any of your IP addresses are listed on RBLs. Your external IP addresses are detected automatically based on messages sent to Zenprise's Zmail hosted service).

Once listed, some RBL providers make the process of getting IP addresses delisted a long and painful one. At times, and in spite of your best efforts and assurances by the provider that the IP address(es) will be delisted, they continue to live on in such lists. On the other end of the spectrum are RBL providers that make the delisting process as simple as entering the listed IP address in a form on their web site, without any checks in place. I would shy away from using the latter.

Regardless, if you are on the other side, as a victim of spam rather than a victim of RBLs, good/reputed RBL providers like Spamhaus (which has its detractors as well - read previous post: "Spamhaus, the E360Insight lawsuit and the future of RBLs"), can help in dramatically reducing spam at the mail gateway - by as much as 90+ percent. This further reduces the stress on mechanisms like content filters (like Intelligent Message Filter or IMF in Exchange Server 2003, and the Content Filter agent in Exchange Server 2007).

Whether you use RBLs to block inbound internet mail from black-listed addresses or not, it makes sense to monitor whether your IP addresses are listed or not on a regular basis.

Read Grimes' column titled "Why I hate RBLs" on InfoWorld.com.

HOW TO: Expose original senders and recipients of quarantined messages

The Content Filter Agent in Exchange Server 2007 allows you to quarantine messages above the SCLQuarantineThreshold. Messages so quarantined are delivered to a quarantine mailbox, specified in the Content Filter configuration. [Read previous post for more info, "Exchange 2007 Content Filter: How to move messages to users' Junk Mail folder"]

Messages arriving in the quarantine mailbox look like NDRs. They appear to be sent from the postmaster address of the default domain. As such, you can't easily see the original sender and recipient's addresses, as shown in the screenshot below:



"How to Recover Quarantined Messages from the Spam Quarantine Mailbox" in the Exchange Server 2007 documentation shows how to use a custom Outlook form to expose the original sender - the doc has code that can be copied to a text editor like Notepad and saved as a custom form (QTNE.CFG), which can be used in Outlook to expose the relevant MAPI property as a field that can be added to the view.

This exposes the original sender, but doesn't expose the original recipient of the message. To expose the recipient, you need to modify the form from the above link and add the relevant MAPI properties. You can download the QTNE.CFG file I modified, and save it in the correct Forms folder for your locale.

For US English, the path is: \Program Files\Microsoft Office\Office12\Forms\1033. You can find out the locale ID (LCID) for your locale from KB 221435: List of supported locale identifiers in Word..

What was added in my Modified Quarantine Extension Form:
Under Properties

Property02=OriginalRecipient

After [Property.OriginalSender] section:

[Property.OriginalRecipient]
Type=30
NmidInteger=0x0E04
DisplayName=Original Recipient

Follow the procedure to add the form to Outlook,:

1. Tools | Options | Other tab | Advanced Options | Custom Forms | Manage Forms | Install.
2. Navigate to the path where you saved the QTNE.CFG file, and select the file.



3. Click OK a few times till all the dialog boxes are closed. The form is now installed.

Now let's make those fields show up!

1. In the Outlook messages pane, right-click the column heading on the top | select Custom.
2. In the Custom View dialog box, click Fields | from the "Select available fields from:" drop-down (which says "Frequently-used fields" by default), select Forms [screenshot].
3. Select the Modified Quarantine Extension Form | click Add
4. In the Show Fields dialog box, select Original Sender and Original Recipient | click Add.



5. Additionally, you can remove unnecessary fields and order the fields the way you want [screenshot of fields I have selected].

Once you're done, the Original Sender and Original Recipient fields are displayed as shown in the screenshot below:



When the message list pane is expanded in Outlook, this is what it looks like.



Exposing these fields makes managing the quarantine mailbox easier.

Exchange Server 2007: Protect users from spam from your own domain

One of the common complaints from users and many messaging folks is spam received from senders that appear to be from your own domain. SMTP mail is exchanged with anonymous internet hosts without any authentication. Headers are effortlessly spoofed. Rather than use an unregistered or invalid domain in the From: field, many spammers find it attractive to use an address from your domain - even an invalid one, or worse - a perfectly valid one. (It is advisable to not have resolution of anonymous senders enabled. Read previous post "A Late New Year's Resolution: Do Not Resolve Anonymous Senders")

Messages that appear to be sent from your domain probably have a better shot at making it through some spam filters, and certainly have a higher chance of being opened by recipients in your domain, once they do make it to the Inbox.

One way of tackling this menace is by using SenderID filter to drop messages that return a FAIL result (from a SPF record lookup). It's advisable to publish SPF records for your domain(s), if you haven't already done so. Explicitly designate SMTP hosts authorized to send for your domain(s). However, the impact of SenderID and whether you should consider dropping email based on it is an open debate, and the topic of another post.

In addition to the above, Exchange Server 2007 allows you to configure ReceiveConnectors to drop messages where the FROM address is from your own domain. It's a good idea to do so if you don't expect any legitimate messages sent without authentication (with your domain in the From: field). To configure a ReceiveConnector to reject such messages:

Remove-ADPermsission "ReceiveConnector-Name" –user “NT AUTHORITY\Anonymous Logon" –extendedrights “ms-Exch-SMTP-Accept-Authoritative-Domain-Sender”

Consider doing this only on the ReceiveConnector that receives inbound internet mail. There may be a need for certain - generally internal - senders like copiers/scanners, and other non-Exchange mail hosts to be able to send with an address from your domain in the FROM field, without authentication. As discussed in previous post titled "Exchange Server 2007: How To Allow Relaying", it is advisable to create an additional ReceiveConnector for such trusted/internal hosts.

RFC 2821, HELO again: Validating the HELO/EHLO domain

RFCs 2821 and 1869 specify the format of HELO/EHLO commands issued by a SMTP client to initiate a session.

RFC 2821 on HELO/EHLO command:

4.1.1.1 Extended HELLO (EHLO) or HELLO (HELO)
These commands are used to identify the SMTP client to the SMTP server. The argument field contains the fully-qualified domain name of the SMTP client if one is available. In situations in which the SMTP client system does not have a meaningful domain name (e.g., when its address is dynamically allocated and no reverse mapping record is available), the client SHOULD send an address literal (see section 4.1.3), optionally followed by information that will help to identify the client system. y The SMTP server identifies itself to the SMTP client in the connection greeting reply and in the response to this command.

(paragraph snipped here)

In any event, a client MUST issue HELO or EHLO before starting a mail transaction.

These commands, and a "250 OK" reply to one of them, confirm that both the SMTP client and the SMTP server are in the initial state, that is, there is no transaction in progress and all state tables and buffers are cleared.

Syntax:

ehlo = "EHLO" SP Domain CRLF
helo = "HELO" SP Domain CRLF

The syntax, as the last 2 lines show - depending on whether you're using EHLO or HELO, should be:
HELO foo.somedomain.com
or
EHLO foo.somedomain.com
(followed by a CR LF)

The IIS/Exchange SMTP stack validates this syntax by default. SMTP clients that do not adhere to this syntax - by appending a period or a space at the end of the domain or address literal (IP address) get a 501 5.5.4 Invalid Address error. Exchange Server 2007, which has a brand new SMTP stack of its own, also continues this domain validation, with a slight modification in the verbiage: 501.5.5.4 Invalid domain name.

However, some SMTP clients do format the EHLO/HELO commands with a trailing period/dot or a space. Though not RFC-compliant, many mail systems allow it, including webmail services like Google's Gmail and Microsoft's Hotmail/Live Mail (I wasn't able to test this with Yahoo Mail today...- Bharat).

You can stop IIS/Exchange SMTP servers from checking if the domain in EHLO/HELO is well formatted by changing the value of SmtpDomainValidationFlags metabase property in IIS to a non-zero value. The property does not exist by default, but you can create it using a tool like the Metabase Explorer (in IIS 6 Resource Kit Tools) - it's a DWORD, with an ID of 36992. You can get step-by-step instructions on how to do this in KBA 291828: "501 5.5.4 Invalid Address" error message from a sending UNIX server.

A post in Microsoft's Exchange newsgroups enquired if this was going to open up a security hole in SMTP. Other than allowing domains or address literals not formatted correctly according to SMTP standards, which many SMTP clients and webmail systems allow and probably use - as stated above, I can't imagine this change creating any serious security issues. Having said that, the particular post was in regard to not being able to receive email from Google Alerts. I continue to receive email from Google Alerts without making any changes to the default behavior of SMTP in IIS/Exchange.

However, the IIS/Exchange SMTP stack does domain validation for a reason. One possibility to consider when evaluating whether to turn it off - it's likely that it could allow sessions from spammers that indulge in such behavior.

Exchange Server 2007: Managing And Filtering Anti-Spam Agent Logs

The anti-spam agents in Exchange Server 2007 log their actions in agent logs. By default, agent logs reside in \Exchange Server\TransportRoles\Logs\AgentLog. Each agent log is 10 Mb in size. The size of the directory is capped at 250 Mb, the age of a log to 30 days. (Logs are flushed when they "age" up to 30 days or if the directory size reaches 250 Mb - whichever happens first).

Additional agent log configuration parameters in SP1

Exchange Server 2007 RTM: In the RTM version, there's only one configuration option for the agent log - that to enable or disable it.

Exchange Server 2007 SP1: Additional parameters to control the max directory size, file size and age of agent logs added.

The config options available:
1) Enable/Disable agent log: Boolean value - TRUE/FALSE
2) Max directory size: In bytes. If not specified, the default is 250 Mb or 262144000 bytes.
3) Max file size: If not specified, the default is 10 Mb or 10485760 bytes.
4) Max age: If not specified, the default is 30 days.

The agent log configuration parameters can be controlled by editing the EdgeTransport.exe.config file, located in \Exchange Server\Bin folder on Edge and Hub Transport servers.

To disable agent logging, insert the following key under <appsettings></appsettings> in the config file:

<add key="AgentLogEnabled" value="FALSE" />

In the following example, we modify max directory size to 500 Mb, file size to 20 Mb, and age to 60 days, by creating new keys in EdgeTransport.exe.config:

<add key="AgentLogMaxDirectorySize" value="524288000" />
<add key="A