Exchange Web Services – Subscriptions and Notifications

It's been a really busy week since I posted my first post on Exchange Web Services (EWS). I have learned a lot in that short period of time that I want to share with you. Whether you are an OpenEdge, Java or .NET developer, I think this post is going to have some information for all of you.

In my first post, I told you about the background story – I need to enable an OpenEdge CRM application to create, modify and delete calendar and task items in Microsoft Exchange. I also need Exchange to let me know any time a calendar or task item is changed so that I can update the OpenEdge database accordingly. Simple use cases.

When I left off last week, my next step was to get Exchange subscriptions working, and, boy, what a trip that has been.

Types of Subscriptions

There are two types of subscription models available with Exchange:

  1. Pull Subscriptions – You set up a subscription with Exchange and it gives you an indicator (called a Watermark) that keeps track of what events you have already received. You pole the server at an interval and it gives you a response with all events that have happened since that last Watermark.
  2. Push Subscriptions – You set up a subscription with Exchange and tell it to call you back on another WebService every time something happens. The WebService has to acknowledge receipt of the message in order to keep the subscription alive. The message you get contains a list of items that have been affected since the last time a message was acknowledged.

I'm not interested in polling the server so I have chosen to go with a Push Subscription. Much of the information that follows applies to both subscriptions, but the Push Subscription differs in that you do not poll the server for the changes. 

Setting up a Push Subscription

To set up a Push Subscription, you send a SOAP message to EWS, authenticating against the Active Directory as you would with any other message to EWS. The message contains a list of folders that you want to know about, the list of event types that you want to be notified of, the URL that you should be called back on, and a StatusFrequency node that tells it how often to poll the URL – more about this later. 

You can subscribe to virtually any update on virtually any folder in a mailbox, including the calendar, tasks, contacts, and mail folders.

Exchange Web Service Push Subscription

The diagram to the left shows the architecture that I finally ended up with. It is a little more complicated than it would be if OpenEdge were not in the picture, but it serves to explain the process, nonetheless. The numbers in parenthesis below refer to the numbers in the diagram.

OpenEdge initiates the subscription by sending a message (1.1) to a Java WebService that is deployed on a Glassfish Application Server. The WebService forwards that subscription request (1.2) on to the IIS server that is running on the Windows 2008 Server that hosts the Microsoft Exchange Client Access Server (CAS). IIS passes the message onto the EWS API which registers the subscription with the CAS. CAS responds with a message that contains the Subscription ID and the Watermark which is tied to the last event that was sent.

As I mentioned, the message that is sent to the EWS API (1.2) contains the URL for the service that is to receive all event notifications. It also contains the StatusFrequency node.

I set up a servlet on my Glassfish Application Server to receive and respond to the notifications. The following is a sample subscription request message:

<?xml version="1.0" encoding="utf-8"?>
    <soap:Envelope xmlns:xsi=""
            <Subscribe xmlns="" 
                        <t:DistinguishedFolderId Id="calendar"/>

The StatusFrequency Node

The StatusFrequency Node tells EWS to send a message every x minutes to verify that the subscription is still active. This serves three purposes:

  1. It allows Exchange to figure out if the server that is listening is still around and cancel any dead subscriptions if necessary;
  2. It allows the server that is being called to determine that it is still getting messages from the Exchange Server; and
  3. It allows the server that is being called to unsubscribe from any future messages.

Exchange needs to carefully manage resources, so canceling any unnecessary subscriptions is an important way to do that. To maintain a subscription, the server that is receiving the notifications has to respond with a SOAP message that includes a SubscriptionStatus node with the value "OK". 

The expected response is as follows:

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="">
    <SendNotificationResult xmlns="">

If a subscriber does not respond appropriately, EWS goes into retry mode. In other words, if EWS does not get a SOAP message formatted exactly the way it expects, it assumes the server has not received the message. It waits for 30 seconds and tries again. If it still receives no response, it waits for a minute and tries again. EWS will continue retrying, doubling the wait time, until the wait time exceeds the StatusFrequency. So in my case, when I first set this up, I set the StatusFrequency to 3 minutes. It tried to send the notification and waited 30, then 60, then 120 seconds and canceled the subscription because, if it had gone to 240 seconds, it would have exceeded the 3 minutes set as the StatusFrequency. 

Less obvious, though, is the reason behind purposes 2 and 3. Remember that Exchange is only going to send messages when something changes. Well, some users have very quiet mail boxes. They may be out of the office, or it may simply be after hours where nothing is going on. If Exchange does not let the other server know that it is still sending messages, how is that server to know that the subscription is still valid?

The other thing that is important is that there is no way to send a message to Exchange to tell it to cancel a Push Subscription. The only way to cancel the subscription is by responding to a status message with the SubscriptionStatus set to "Unsubscribe".

Receiving Notifications

Now that you understand how to subscribe, the next step is to start receiving messages from EWS. We'll pick up where we left off in the diagram above.

EWS starts off by sending a notification (2.1) to the server. The notification could be one of two things:

  1. A notification of events that have taken place on the server; or
  2. A status message.

If this is a status message, all you need to do is respond with the OK response (2.2) I discussed above. In fact, you need to respond with the OK message if you want to carry on receiving notifications no matter what, but responding with an OK has an additional effect if you receive a notification of events.

First let's understand a notification of events. An event notification message contains a list of events that have taken place against the items that you are monitoring. It does not contain the details of what changed – just the fact that something did. If you want to know what changed, you have to call the GetItem method of the EWS to get the item and look at its changes. That's what we do in step 2.3.

Each notification includes a PreviousWatermark for the whole message and a Watermark for each event. By keeping track of the watermarks, it is possible to restart the subscription from the last event that was processed. When you acknowledge the event notification with an OK, Exchange moves the subscription to the next watermark automatically.

Once I have processed the event notification and received the event, I am making a Dynamic OpenClient call (2.4) to the OpenEdge AppServer to notify the AppServer of the change. The ABL code on the AppServer can then do what it needs to do to respond to the event.


Once I got this working, I decided to test it out to see what the performance looked like. Now bear in mind, all of the machines shown in the diagram are different virtual machines on one Dell Poweredge T710 running VMWare 4.0 ESXi. The T710 has 2×4 core processors and 16GB of memory. It has 4 network cards connected to a 1GB switch. The Windows Server is on a separate network card to the other three machines, which are all Centos 64-bit boxes. 

The round trip for establishing the subscription takes an average of 82ms.

I then ran a test harness that generates 100,000 calendar items across 80 mailboxes. It took a little over 2 hours for the test harness to complete. I tracked performance for each of the following pieces:

  • The OpenEdge client takes an average of 100 milliseconds to make the call through the Java WebService to EWS to create the appointment.
  • From the time I receive the acknowledgement for the appointment creation to the time that the notification has been received (2.1) and acknowledged (2.2) and the details of the notification have been received (2.3) takes an average of 150 milliseconds.
  • The call across the Dynamic OpenClient to the AppServer, which is a call to a procedure that has no temp-tables or XML documents, averages 80 milliseconds.

All told, this means that from the time the request for the appointment is initiated to the time the notification of its update is received by the AppServer takes an average of 330 milliseconds – less than half a second.

Now clearly, my environment has very little traffic and I have a lot of horsepower backing it, but it if this is properly tuned and the additional processing that is to be added is not excessive, a one second response time is well within the realms of possibility.

Problems and Challenges

In getting this to work, I ran into a couple of nasty obstacles. 

First, Microsoft's API is really not a good example of a WebServices API. Neither Java nor OpenEdge will parse the WSDLs without manual editing and even if you manage to generate the code you need from the WSDL, it really doesn't work very well. I have been forced to manually code the interaction with the EWS API and that is a lot of work. Having said that, it performs very well and I get to write the XML exactly the way it needs to be for EWS to accept it.

That brings me to the second thing about this API. Microsoft's error reporting is shockingly bad for commercial software. Compared to the kind of error reporting I get from Tomcat, Glassfish, ActiveMQ, and several other Open Source solutions, it's incredible that commercial software can be this badly engineered. It is so hard to figure out what is wrong when something goes bad that you will spend hours on things like malformed or unrecognized XML. EWS simply returns a 500 http error code (Internal Server Error) with absolutely no indication of what caused it, and there is nothing in any of the logs to help you.

Update: Since I wrote this article, I have found (and Kris C. has also pointed this out in the comments) that making a call to the getErrorStream() method on the connection object will provide more detailed information on what failed, and you get an entire SOAP message back that describes details of the 500 error. I therefore eat my words about error reporting.

I still think that there is room for improvement because it is very hard to diagnose exactly which node is out of order if you submit a request where the XML nodes are not in the exact order specified by the EWS API, but it's nowhere near as bad as what my original post probably led you to believe.

There is more information in Part 3 of the series and there is even sample code that demonstrates exception retrieval.

The third thing that really cost me a lot of time is that you cannot call a regular Java or OpenEdge WebService from EWS because it won't look at the WSDL. I was therefore forced to manually code the Java servlet that accepts an http post.

These problems effectively sidelined OpenEdge as a direct endpoint for EWS notifications. The architecture that I am building will definitely rely on all communication with EWS going via Java services.

Where to next?

Now that I have the prototype code working, my next step is to fix up some of the security issues I need to resolve. NTLMv2 authentication remains a real problem for Java in communicating with EWS, and I am looking for a solution around that. I also need to deal with some of the certificate related issues that I have.

I have also done some work on the Exchange Impersonation stuff, so my next blog post will probably talk about this.

Finally, I'm planning a walkthrough of some example code that actually demonstrates the functionality, like I did with the Dynamic OpenClient code late last year

11 thoughts on “Exchange Web Services – Subscriptions and Notifications”

  1. Hi Bruce,
    This is great stuff.
    You mention that you were unable to use Progress as the end-point for receiving the posts from exchange. Did you consider simply using WebSpeed? We do this now for one of the web services we had to provide that required us to provide an industry standard WSDL. It seems like it would be easy enough to get the post in WebSpeed and parse the XML as needed.


    Hi Tom,

    No. I did not consider using WebSpeed, and I'm not going to either. There's another piece to this that I have not discussed yet that I really want to prove before I unequivocally state it.

    Most of the code in the Java side is plumbing that OpenEdge should not have to deal with. When OpenEdge registers its subscription in 1.1, it will tell the Java WebService exactly what it wants to hear about – "all calendar items in this mailbox with these attendees and this property". Although EWS will filter, it won't give me that level of granularity. So the subscription that is passed in 1.2 will be  a subscription to all events in the calendar for a mailbox. 

    In my testing, it seems that there are going to be a lot of messages that come back from Exchange, and I am probably only interested in about 1/3rd of them. That means that if Progress were to serve as the end point, 2/3rds of the messages that it receives would be wasted calls.

    As we both know, OpenEdge is going to take longer to process each of those messages than Java would and as the AppServer is single-threaded, I will be tying up a process unnecessarily. I would rather only feed OpenEdge exactly what I want it to process, so ultimately the code that is currently in the Java HTTP Servlet will make use of XPath queries and a rule engine to decide what messages calls to actually make to OpenEdge.

    Make sense?


  3. A couple of other things about the OpenEdge vs Java as the end point thing that I did not mention:

    As I noted in the article, it takes 150ms on average to receive the notification from EWS, respond to EWS with the acknowledgement, call EWS to get the item information, and parse that item object so that it is ready for the call to Progress. That's reasonable performance and that's for a message that is an event notification, not a status message.

    Status messages are handled in, on average, 50ms from the time that Exchange initiates the call to the time it received the response.

    I am not confident that I could achieve the same level of performance with Progress because I would have to go through the broker (whether WebSpeed or AppServer) on each request to get the agent process, and then the agent would need to parse the XML file, build the response, and return it. 

    This truly is a case of "horses for courses". Java is a better racehorse for this course than OpenEdge is, and I think it is better able to scale and handle the load.

  4. Thanks for the great set of articles.
    I'd be interested to hear if you have had any progress getting your solution to work with NTLMv2.

    1. I’m glad you found the articles useful. Let me have any feedback you have that could be used to improve them.

      There are solutions available to working with the NTLMv2 authentication, but they all require purchasing commercial software that addresses the issue, and none of them are Microsoft-sanctioned solutions. I have settled on digest-based authentication as the implementation that I have set up in any case requires client-side certificates that assert the credentials of the client, so digest authentication is more than adequate.

  5. Actually exchange server returns the brief error message of what is possibly wrong about the xml, other than 500 internal server error.

    so if you do connection.getErrorStream() and read the result, it contains a xml including the error message.

    1. Thanks for the note, Kris. Really good point.

      The code that supports Part 3 of the article series actually does this. At the time I wrote this article, I was still finding my way around the error handling and missed the fact that you get a response back in the error stream if you get a 500 error code on the HTTP post.

      Having said that, even with the error messages coming back in the stream, it is still very difficult to figure out that, for example, the nodes in the XML document are in the wrong order.

Come on... Argue with me about this...