Friday, 9 September 2016

Think Load Balancing Before You Ditch Your Exchange DAG Administrative Access Point

With Exchange 2013 SP1, Microsoft introduced an alternative to the traditional failover cluster on which DAGs are based: no virtual IP, no Cluster Name Object, no worries. For
details see https://technet.microsoft.com/en-AU/library/dd979799(v=exchg.150).aspx.

Backup is the most quoted reason why you may still want to have an administrative access point. Backup software needs an always-on resource to connect to in order to back up
Exchange. And that is, conveniently, the cluster administrative access point. However backup software vendors are catching up with the times and they don't really need a
cluster resource anymore.

So why would you still want to have one? Well, think of inbound SMTP load balancing, and, implicitly, High Availability (HA).

Have you ever tried to load-balance SMTP? If so then you've probably encountered some or all of the following:
  • In the Exchange transport logs, all SMTP traffic appears to be coming from a single IP address - that of the load balancer. How do you tell the "real" source without looking at the e-mail headers?
  • You have a number of internal devices that need to send unauthenticated e-mail to any recipient, including external ones. Think open relay with restricted access. But if all traffic seems to come from the load balancer, then the only way to allow your multifunction device to scan a document to an external mailbox is to authorise the load balancer as approved source of unauthenticated SMTP traffic. Or you need to send e-mail alerts to an external party as par of your support agreement. Suddenly you have a security problem and a career challenge (you'll cause your organization's servers to be blacklisted in no time and you'll need to explain that to your manager).
  • Enable source routing on the load balancer. Bingo, your Exchange server sees the real source. Open relay issue solved. But hey, that will result in asymmetrical routing. Not ideal and usually difficult to diagnose if issues arise. So by solving one problem you introduced another one...

Most of these, and some other aspects, are documented in Paul Cunningham's blog at http://exchangeserverpro.com/issues-with-load-balancing-smtp-traffic/.

The alternative: hang on to your cluster administrative point. The benefits:
  • You can send SMTP traffic from internal sources straight to the DAG administrative access point, bypassing the load balancer. No open relay issues, no transport log analysis issues.
  • No asymmetrical routing issues.
  • Multi-role servers are becoming the norm. In fact, with Exchange 2016, it is the only option. Therefore as long as you have at least one online DAG member, you'll also have implicitly a Hub Transport server too to accept SMTP traffic which will be reachable via the - guess what - thine good olde DAG administrative access point. Therefore service is maintained even if individual DAG members fail.
The solution has one obvious drawback: all inbound SMTP traffic will be processed without any load balancing, by a single Exchange server, the one which holds the name and IP address of the DAG
administrative access point.

As at the time of this post I am yet to see an Exchange infrastructure whith such a heavy inbound SMTP traffic that load balancing (not to be confused with HA) would be a necessity. Therefore the only drawback that I can spot is so insignificant that it can be safely discounted in most environments.

Having said that, I learned to "never say never". If you know of any circumstances where this workaround would not work then please drop me a comment.

Cheers,
Zoltan

Tuesday, 21 June 2016

Delete That Stubborn Exchange 2016 Database

Quick solution for the impatient: move/remove/disable User, Archive, Arbitration, PublicFolder and AuditLog (yes, AuditLog - new in Ex2016) mailboxes.


So you have your shiny new Exchange 2016 server, but you don't like the name and/or location of the database created for you by default. You create a new mailbox database, give it a nice name and place it on its shiny new drive.

Time to remove the default database. Well, you could technically rename it and move it elsewhere and just keep using it, but that would render the rest of the article irrelevant.

You move all user mailboxes, then what you do is:

Remove-MailboxDatabase "Mailbox Database random_number"

Bummer! That didn't work. Forgot about the various other mailbox types...



It's a new database, so no user mailboxes are on it. You check it, just in case. Then you read the eye-hurting screenful of red text and you work your way through the list of mailbox types you'll need to delete:
  • User mailboxes (we covered it above).
  • Mailbox plans - oh, you can't even run the Get-MailboxPlan command 'coz it's a cloud command and your server is on-prem. That was a tad misleading...
  • Archives
  • Public folders
  • Arbitration mailboxes
You try again and say Bummer! for the second time today. All looks clear:


Time to scratch head. Aha, forgot about the monitoring mailboxes (well, they don't need to be removed in order to delete the database, but you say "well, just in case...")...:

...so you disable or remove the monitoring mailboxes:


Try again... Bummer, bummer, bummer! What else??? You read the Get-Mailbox online reference. You also read the output of Get-Help Get-Mailbox -Full. Nothing.

Then you hit the Web, spend the next half day searching, only to arrive here. Says the article:



Huh?... You saw the online reference and the Get-Help result, and they both say that it is not your business as an administrator to mess with the AuditLog parameter. In fact you couldn't even tell what it is for as Microsoft chose not to disclose it:



Get-Help is no different:









You follow the instructions, only to find that you do have an AuditLog mailbox. In fact Exchange creates one in the default database when it is installed - so the above article too is misleading in that it isn't straight forward about how you ended up having one.








You move/remove/disable the mailbox, try to delete the database and - BINGO!








Grab a well-deserved beer and start growing back some of the hair you pulled out.

It isn't the first time that (mis)documentation of the Remove-MailboxDatabase command caused some head-scratching. Tony Redmond blogged about it here. The Exchange team also blogged about it here, suggesting to use the -Verbose switch to figure out who is still in the database. But guess what: the -Verbose switch didn't work for me on my Exchange 2016 CU1 server:














Well, here we are a few years after Exchange 2013 has been released, and "the relevant team" (as the Microsoft blog puts it), despite its "awareness", chose to carry across the same inaccuracies in the documentation, adding some more inaccuracies to it in the process.

Tuesday, 14 June 2016

Get or Set Mailbox Database Maximum Size

Here are two scripts to get or set the maximum mailbox database size.
Download: MaxDatabaseSize.zip

The scripts are based on this TechNet article. Snippets were pinched from here and there, and therefore due acknowledgement applies accordingly.

The scripts work with DAG-replicated, as well as single-copy mailbox databases, and were tested on Exchange 2010 and 2013. It should also work on Exchange 2016, however I haven't tested it.

The scripts do NOT work with Exchange 2007 as they rely on remote PowerShell which is unavailable in PowerShell v1. Remote PowerShell requirements are detailed here. Practical application examples of PowerShell remoting are here.

The scripts don't accept piped input.

Both scripts should be invoked in an elevated Exchange Management Shell console:


Getting the Maximum Database Size Limit

Syntax:

Get-MaximumMailboxDatabaseSize.ps1 -Database <DatabaseName>

Default size returned if no limit has been configured - maximum default size is determined by your Exchange server version and edition:


After setting the maximum size to 500Gb:
 

Listing the maximum size of every mailbox database in the environment:

Get-MailboxDatabase | ForEach {.\Get-MaximumMailboxDatabaseSize.ps1 $_.Name}

Output on my 2-server test system where each node has a local mailbox database, as well as a DAG-replicated database on both nodes in the DAG:



Setting the Maximum Database Size Limit

Use the Set-MaximumMailboxDatabaseSize.ps1 script to set the maximum size of a mailbox database. The script finds all servers that host a copy of the database and updates the Registry on each server.

Syntax:

Set-MaximumMailboxDatabaseSize.ps1 -Database <DatabaseName> -Size <New_Maximum_Database_Size_Limit_In_Gb>

Set-MaximumMailboxDatabaseSize.ps1 -Database <DatabaseName> -Default

The following command configures the maximum size of DB1 to 500Gb:

Set-MaximumMailboxDatabaseSize.ps1 -Database DB1 -Size 500
 
 

Effectively the script sets the Database Size Limit in GB Registry value as per the above TechNet article.


In order to revert the size limit to default, simply delete the Registry entry from all servers and DAG members where a copy of the database exists:

Set-MaximumMailboxDatabaseSize.ps1 -Database DB1 -Default



Licence Agreement

The scripts are licenced under the NMF (Not My Fault) agreement.
Edit, change and use the scripts at your own risk.
Give credit where due.

Download: MaxDatabaseSize.zip

Enjoy :-)

Tuesday, 3 May 2016

List All SMTP Addresses for a Domain

Hi there,

Here is a quick script that lists all SMTP addresses for a given domain on an Exchange 2007/2010/2013 (and probably 2016 too) server.

The script dumps a raw, unsorted list of SMTP addresses to a text file that can be directly consumed by, say, Symantec.cloud to populate its recipient validation list (this was the purpose I produced it for).

Download it from https://gallery.technet.microsoft.com/Find-all-SMTP-addresses-9dc01450.

The NMF (Not My Fault) licence applies. Test it first and use it at your own risk.

Enjoy :-)

Sunday, 13 March 2016

Find hosts that send SMTP traffic to an Exchange server

Recently I had to identify all SMTP sources that were still sending mail via legacy servers, with a view to reconfigure them and decommission the old infrastructure.

Wrote a script that did just that for me. Download it from https://gallery.technet.microsoft.com/Find-SMTP-sources-from-48ccb9ef.

An obvious prerequisite is to have Receive connector logging enabled for some time to allow traffic to be captured.

Enjoy :-)

Tuesday, 24 November 2015

Error Sending Email to Subdomains from O365

Suppose you have an on-premise Exchange and SharePoint server. You configure SharePoint to allow adding content by e-mail (see here). Your main domain is yourdomain.com and the SharePoint domain is sp.yourdomain.com. A sample SharePoint address may look like documents@sp.yourdomain.com.

You set up a dedicated Send connector and scope it for the sp.yourdomain.com domain, using the SharePoint server as smart host.

It works just fine.

Then you enable Hybrid Exchange, move a couple of mailboxes into the cloud, and you switch your MX record(s) to Exchange Online Protection.

In this scenario users with mailboxes in the cloud may fail to submit documents to SharePoint via e-mail and they'll get the following error:

LED=550 5.4.101 Proxy session setup failed on Frontend with '554 5.4.4 SMTPSEND.DNS.Non-ExistentDomain; nonexistent domain'


Here is what fixed it for me, and hopefully it will fix it for you too:

1. In the O365 Exchange Admin Center, under Mail flow / Accepted domains:
  • Change your parent domain (yourdomain.com) from Authoritative to Internal Relay and enable Accept mail for all subdomains (see here - a tad old, due for an update).
2. Under Mail Flow / Connectors:
  • Edit the connector which routes mail from O365 to your organization and click Next until the Edit Connector page is displayed.
  • Add the *.yourdomain.com domain to the list.
  • Click next and work your way through the wizard. Don't change anything else. Save the settings. In the process you'll need to provide a valid e-mail address to validate the connection. Provide one in the yourdomain.com domain.
3. On the on-premise Exchange server configure the following:
  • Create a new accepted domain of type "External Relay" for sp.yourdomain.com:

  • There should already be a dedicated Send connector. If not, create a new dedicated Send connector for the sp.yourdomain.com address space:

  • On the Send connector, ensure that the SharePoint server is listed as smart host. Add it if it isn't.


E-mail will now start flowing as expected.

Your environment  might be slightly different, but the principle still applies.


Wednesday, 29 July 2015

SIDHistory and Exchange Resource Forests


The Environment
I have been recently involved in a business consolidation project where two businesses joined forces and formed a new entity. Inevitably, it had a strong migration component.

My client stipulated that assuming the new identity in the early stages is critical and it has to be reflected in communications. Therefore e-mail was the first component to be migrated. The plan was to let users log on and access resources such as file, print, application etc. as normal, with their LEGACY\oldlogon account, while sending/receiving e-mail should happen under their NEW\newlogon identity.

To achieve this first milestone, the new AD forest has been created and Exchange 2013 CU8 has been installed and configured.
The timeline was unclear as to when will users start to log on with their NEW\newlogon accounts, and in relation to that, whether users will start logging on to the new forest *before* or *after* other resources will have been migrated.
For this reason user accounts have been migrated along with the SIDHistory attribute in a “disabled” state, and their mailboxes moved cross-forest as Linked Mailboxes. Effectively an Account/Resource forest topology has been built. For more on Account/Resource forests see https://technet.microsoft.com/en-us/library/aa998031(v=exchg.150).aspx.

The Issue
All worked well, except delegation. Delegation was flaky at best, users regularly losing access, and admins unsuccessfully trying to re-establish permissions. While initially Get-MailboxPermission displayed the correct LEGACY\oldlogon accounts, they’ve been replaced, automagically, with NEW\newlogon as time passed. Inconsistency was the only constant.

I searched the Internet far and wide, asked on professional forums, however my efforts yielded no useful results.

I ended up opening a case with Microsoft.

The Workaround
For each SID in the shared mailbox’s ACL Exchange tries to find its corresponding AD account, and it searches its own forest first. It finds the SID of the LEGACY\oldlogon account in the SIDHistory attribute of the migrated NEW\newaccount, and it says “BINGO! This is the account!” Therefore it stops searching and does NOT follow the chain of trusted forests to identify the real owner of the SID. Therefore it incorrectly assigns permissions to NEW\newlogon in the resource forest instead of LEGACY\oldlogon in the account forest.

If the SID on the shared mailbox’s ACL isn’t found in the Exchange server’s home forest, then, and only then, Exchange will move on to find the account in one of the trusted forests.
To date the only way to get delegated access to work again is to clear the sIDHistory attribute of the NEW\newlogon account. There are a number of ways to clear the SIDHistory. My preferred tool is Ashley McGlone’s PowerShell module.

WARNING: Do NOT clear the SIDHistory unless the effects are fully understood AND if the effects will not affect the business in a negative way! Otherwise consider a career change.
While clearing the SIDHistory solves the immediate issue, it introduces new challenges later on when the merger/migration is refocused onto accounts and resources. Further steps must be carefully planned and coordinated, user accounts potentially re-migrated to re-populate the sIDHistory attribute, otherwise users risk to lose access (and the admin his/her job).

In my opinion the logic by which Exchange finds delegated accounts is flawed, and therefore I see the removal of SIDHistory not as a “fix” but rather as a quick and dirty “workaround”, lacking strategic view of the bigger picture. Hope Microsoft will fix it one day.

Happy migration!