Ouch, another way-too-long blog title. Anyways. I’m doing sort of a summer job for my previous employer while waiting for my next project to start. Being a hosted service provider, they’re looking for ways to expose key performance metrics to their customers. For the hosted Exchange service, we looked at using the built-in reports for Exchange 2010 in System Center Operations Manager 2012SP1, which they’re using to monitor the Exchange environment. However, we found that either the reports don’t deliver any metrics the regular customer will understand, or the reports simply do not work (this seems to be a known issue with the Exchange 2010 MP for Ops Manager).
After some healthy discussion, we decidede to keep it simple: Measure the average speed of mail flow from the Exchange service and out to the Internet, and back.
The old Exchange 2003 management pack for Ops Manager actually included functionality along these lines, but the newer versions don’t.
So, here’s what we did: We deployed an Azure VM and installed the hmail server (which is freeware) on it. hMail is a simple-to-use mail server with an excellent com api which makes it really easy to access using PowerShell. The “plumbing” itself was done in PowerShell, and implemented using regular scheduled tasks on the Azure VM. Simply put, the scripts to the following:
1. Retrieve a list of test mailboxes (these reside on the on-prem Exchange servers)
2. For each testmailbox, log on to the mailbox (through autodiscover) using EWS and send a mail to the hmail server in azure. Each mail is stamped with a guid.
3. After mail is sent, upload the details (mailbox, time sent, logon time, guid) to a database using REST calls to the MVC Web API site
4. List all emails in the hmail inbox and map each mail to the correct “test” already in the MVC Web API database (the guid is used for mapping).
5. Update the “test” with received time
A custom SCOM management pack using PowerShell-based rules and monitors are used to discover the test mailboxes and retrieve stats for each test mailbox. Thanks to my “weather forecasting” management pack we already had the basic skeleton of a PowerShell-driven management pack in place. A quick search+replace on object names and then some editing in the scom 2007R2 authoring console was all it took (except for creating new PowerShell scripts for discovery, of course).
I won’t paste any code here since this was a paid task and the MP and code is basically the property of my employer, but you get the idea. However, I’ll post a few gotcha’s:
1. The MVC4 Web API implementation of its rest interface throws some troubles around date handling. Might be because we throw nb-no locale’d dates at it, while the azure web site runs on a server with en-us locale, but it turned out that formating datetimes in PowerShell using
get-date -format s
was the way to go.
Also, when pushing data using invoke-restmethod from PowerShell we also saw some errors until we added
as a parameter (I would think PowerShell would be smart enough to do that on it’s own, but that’s just me.
-Using autodiscover with EWS takes a long time! This Exchange service uses redirects, which easily takes up to 30 seconds. This slowed the scripts down considerably, so we built in support for both autodiscover and “hard-coded” ews address into the ews module functions.
It also paid off to separate the code into several modules. We built an “ews” module to handle all communication with the exchange web service, and another “hmail” for talking to hmail through COM. The rest was put in a separate “mailflowtest” module which referenced the two first ones.
Finally, we added some monitoring in ops manager to check that the Azure web site is up and running at all times, and that the tests returned from the webservice are always fresh. From conception to finished in about 10 days of work using PowerShell, System Center, Visual Studio and a little bit of cloud computing magic. Fun!