It's been a really busy week since I posted my first post on Exchange Web Services (EWS). I have learned a lot in that short period of time that I want to share with you. Whether you are an OpenEdge, Java or .NET developer, I think this post is going to have some information for all of you.
In my first post, I told you about the background story – I need to enable an OpenEdge CRM application to create, modify and delete calendar and task items in Microsoft Exchange. I also need Exchange to let me know any time a calendar or task item is changed so that I can update the OpenEdge database accordingly. Simple use cases.
When I left off last week, my next step was to get Exchange subscriptions working, and, boy, what a trip that has been.
Types of Subscriptions
There are two types of subscription models available with Exchange:
- Pull Subscriptions – You set up a subscription with Exchange and it gives you an indicator (called a Watermark) that keeps track of what events you have already received. You pole the server at an interval and it gives you a response with all events that have happened since that last Watermark.
- Push Subscriptions – You set up a subscription with Exchange and tell it to call you back on another WebService every time something happens. The WebService has to acknowledge receipt of the message in order to keep the subscription alive. The message you get contains a list of items that have been affected since the last time a message was acknowledged.
I'm not interested in polling the server so I have chosen to go with a Push Subscription. Much of the information that follows applies to both subscriptions, but the Push Subscription differs in that you do not poll the server for the changes.
To set up a Push Subscription, you send a SOAP message to EWS, authenticating against the Active Directory as you would with any other message to EWS. The message contains a list of folders that you want to know about, the list of event types that you want to be notified of, the URL that you should be called back on, and a StatusFrequency node that tells it how often to poll the URL – more about this later.
You can subscribe to virtually any update on virtually any folder in a mailbox, including the calendar, tasks, contacts, and mail folders.
The diagram to the left shows the architecture that I finally ended up with. It is a little more complicated than it would be if OpenEdge were not in the picture, but it serves to explain the process, nonetheless. The numbers in parenthesis below refer to the numbers in the diagram.
OpenEdge initiates the subscription by sending a message (1.1) to a Java WebService that is deployed on a Glassfish Application Server. The WebService forwards that subscription request (1.2) on to the IIS server that is running on the Windows 2008 Server that hosts the Microsoft Exchange Client Access Server (CAS). IIS passes the message onto the EWS API which registers the subscription with the CAS. CAS responds with a message that contains the Subscription ID and the Watermark which is tied to the last event that was sent.
As I mentioned, the message that is sent to the EWS API (1.2) contains the URL for the service that is to receive all event notifications. It also contains the StatusFrequency node.
I set up a servlet on my Glassfish Application Server to receive and respond to the notifications. The following is a sample subscription request message:
<?xml version="1.0" encoding="utf-8"?> <soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"> <soap:Body> <Subscribe xmlns="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"> <PushSubscriptionRequest> <t:FolderIds> <t:DistinguishedFolderId Id="calendar"/> </t:FolderIds> <t:EventTypes> <t:EventType>CreatedEvent</t:EventType> <t:EventType>ModifiedEvent</t:EventType> <t:EventType>DeletedEvent</t:EventType> </t:EventTypes> <t:StatusFrequency>3</t:StatusFrequency> <t:URL>http://server.example.com/ewsCalTest/NotificationService</t:URL> </PushSubscriptionRequest> </Subscribe> </soap:Body> </soap:Envelope>
The StatusFrequency Node tells EWS to send a message every x minutes to verify that the subscription is still active. This serves three purposes:
- It allows Exchange to figure out if the server that is listening is still around and cancel any dead subscriptions if necessary;
- It allows the server that is being called to determine that it is still getting messages from the Exchange Server; and
- It allows the server that is being called to unsubscribe from any future messages.
Exchange needs to carefully manage resources, so canceling any unnecessary subscriptions is an important way to do that. To maintain a subscription, the server that is receiving the notifications has to respond with a SOAP message that includes a SubscriptionStatus node with the value "OK".
The expected response is as follows:
<?xml version="1.0" encoding="utf-8"?> <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope"> <soap:Body> <SendNotificationResult xmlns="http://schemas.microsoft.com/exchange/services/2006/messages"> <SubscriptionStatus>OK</SubscriptionStatus> </SendNotificationResult> </soap:Body> </soap:Envelope>
If a subscriber does not respond appropriately, EWS goes into retry mode. In other words, if EWS does not get a SOAP message formatted exactly the way it expects, it assumes the server has not received the message. It waits for 30 seconds and tries again. If it still receives no response, it waits for a minute and tries again. EWS will continue retrying, doubling the wait time, until the wait time exceeds the StatusFrequency. So in my case, when I first set this up, I set the StatusFrequency to 3 minutes. It tried to send the notification and waited 30, then 60, then 120 seconds and canceled the subscription because, if it had gone to 240 seconds, it would have exceeded the 3 minutes set as the StatusFrequency.
Less obvious, though, is the reason behind purposes 2 and 3. Remember that Exchange is only going to send messages when something changes. Well, some users have very quiet mail boxes. They may be out of the office, or it may simply be after hours where nothing is going on. If Exchange does not let the other server know that it is still sending messages, how is that server to know that the subscription is still valid?
The other thing that is important is that there is no way to send a message to Exchange to tell it to cancel a Push Subscription. The only way to cancel the subscription is by responding to a status message with the SubscriptionStatus set to "Unsubscribe".
Now that you understand how to subscribe, the next step is to start receiving messages from EWS. We'll pick up where we left off in the diagram above.
EWS starts off by sending a notification (2.1) to the server. The notification could be one of two things:
- A notification of events that have taken place on the server; or
- A status message.
If this is a status message, all you need to do is respond with the OK response (2.2) I discussed above. In fact, you need to respond with the OK message if you want to carry on receiving notifications no matter what, but responding with an OK has an additional effect if you receive a notification of events.
First let's understand a notification of events. An event notification message contains a list of events that have taken place against the items that you are monitoring. It does not contain the details of what changed – just the fact that something did. If you want to know what changed, you have to call the GetItem method of the EWS to get the item and look at its changes. That's what we do in step 2.3.
Each notification includes a PreviousWatermark for the whole message and a Watermark for each event. By keeping track of the watermarks, it is possible to restart the subscription from the last event that was processed. When you acknowledge the event notification with an OK, Exchange moves the subscription to the next watermark automatically.
Once I have processed the event notification and received the event, I am making a Dynamic OpenClient call (2.4) to the OpenEdge AppServer to notify the AppServer of the change. The ABL code on the AppServer can then do what it needs to do to respond to the event.
Once I got this working, I decided to test it out to see what the performance looked like. Now bear in mind, all of the machines shown in the diagram are different virtual machines on one Dell Poweredge T710 running VMWare 4.0 ESXi. The T710 has 2×4 core processors and 16GB of memory. It has 4 network cards connected to a 1GB switch. The Windows Server is on a separate network card to the other three machines, which are all Centos 64-bit boxes.
The round trip for establishing the subscription takes an average of 82ms.
I then ran a test harness that generates 100,000 calendar items across 80 mailboxes. It took a little over 2 hours for the test harness to complete. I tracked performance for each of the following pieces:
- The OpenEdge client takes an average of 100 milliseconds to make the call through the Java WebService to EWS to create the appointment.
- From the time I receive the acknowledgement for the appointment creation to the time that the notification has been received (2.1) and acknowledged (2.2) and the details of the notification have been received (2.3) takes an average of 150 milliseconds.
- The call across the Dynamic OpenClient to the AppServer, which is a call to a procedure that has no temp-tables or XML documents, averages 80 milliseconds.
All told, this means that from the time the request for the appointment is initiated to the time the notification of its update is received by the AppServer takes an average of 330 milliseconds – less than half a second.
Now clearly, my environment has very little traffic and I have a lot of horsepower backing it, but it if this is properly tuned and the additional processing that is to be added is not excessive, a one second response time is well within the realms of possibility.
Problems and Challenges
In getting this to work, I ran into a couple of nasty obstacles.
First, Microsoft's API is really not a good example of a WebServices API. Neither Java nor OpenEdge will parse the WSDLs without manual editing and even if you manage to generate the code you need from the WSDL, it really doesn't work very well. I have been forced to manually code the interaction with the EWS API and that is a lot of work. Having said that, it performs very well and I get to write the XML exactly the way it needs to be for EWS to accept it.
That brings me to the second thing about this API. Microsoft's error reporting is shockingly bad for commercial software. Compared to the kind of error reporting I get from Tomcat, Glassfish, ActiveMQ, and several other Open Source solutions, it's incredible that commercial software can be this badly engineered. It is so hard to figure out what is wrong when something goes bad that you will spend hours on things like malformed or unrecognized XML. EWS simply returns a 500 http error code (Internal Server Error) with absolutely no indication of what caused it, and there is nothing in any of the logs to help you.
Update: Since I wrote this article, I have found (and Kris C. has also pointed this out in the comments) that making a call to the getErrorStream() method on the connection object will provide more detailed information on what failed, and you get an entire SOAP message back that describes details of the 500 error. I therefore eat my words about error reporting.
I still think that there is room for improvement because it is very hard to diagnose exactly which node is out of order if you submit a request where the XML nodes are not in the exact order specified by the EWS API, but it's nowhere near as bad as what my original post probably led you to believe.
There is more information in Part 3 of the series and there is even sample code that demonstrates exception retrieval.
The third thing that really cost me a lot of time is that you cannot call a regular Java or OpenEdge WebService from EWS because it won't look at the WSDL. I was therefore forced to manually code the Java servlet that accepts an http post.
These problems effectively sidelined OpenEdge as a direct endpoint for EWS notifications. The architecture that I am building will definitely rely on all communication with EWS going via Java services.
Where to next?
Now that I have the prototype code working, my next step is to fix up some of the security issues I need to resolve. NTLMv2 authentication remains a real problem for Java in communicating with EWS, and I am looking for a solution around that. I also need to deal with some of the certificate related issues that I have.
I have also done some work on the Exchange Impersonation stuff, so my next blog post will probably talk about this.
Finally, I'm planning a walkthrough of some example code that actually demonstrates the functionality, like I did with the Dynamic OpenClient code late last year.