I covered a lot of material about the push subscription mechanism in an earlier post, and if you haven't read it, I highly recommend you go back and revisit it as this article builds on that one. It also builds on part 1, part 2, and part 3 of this series, so you should probably have worked through those, too. 

Goals and Background

By now, you know that the reason that I have embarked on this project is to integrate Microsoft Exchange with a CRM application that is in Progress OpenEdge. The first part of the exercise was to figure out how to create calendar items based on data entered into the CRM system, and how to automate that for a large number of sales people without having to log in to each mailbox.

Now that we know how to do that, the next step is handling the situation where the sales person makes a change to a calendar item from Outlook or a mobile device. The item change needs to make it back to the CRM application, and for that, the Exchange WebServices Push Subscription API is going to be essential.

When a change is made to an Exchange Server calendar item, the Push Subscription API can send a message to an HTTP target of your choice. As long as the HTTP target knows how to parse the SOAP message that is sent by Exchange, it can react to the events that it are sent.

In this example, we are going to concentrate on 3 events that occur against objects in the default calendar folder for a mailbox; the Created-, Modified-, and DeletedEvents.

The Subscription Process

By now, the diagram to the left is all too familiar, but it clarifies the process really well, so I am going to walk through it again.

The first step (1.1) is to initiate a call to the Java EWS WebService to register a subscription. The EWS WebService then makes a call to Exchange (1.2) to tell it to call back to the Java HTTP Servlet. 

When something happens to an item for which we have registered a subscription, Exchange will call the HTTP Servlet (2.1) with a notification message.

Now the order of what happens next is something that you will want to change for production code, because it should happen in the order illustrated in the diagram, but I have decided to take the simple road with the attached code and process everything in a synchronous way. What follows is the order in which I am doing the processing in the sample code.

After parsing the message notification message that was received (2.1), the code determines if there are any items that need to be processed. A notification message only contains the ItemID of the object that has changed. It does not contain the item itself. This means that we need to call the Exchange Server back to get the item's information (2.3).

Once we have the item's information, we can react to it, and in the example code, this includes calling OpenEdge (2.4) with information about the appointment that has changed. It is really important to note that only notifications that actually affect an item are passed on to OpenEdge. This significantly reduces the processing that OpenEdge has to perform.

Finally, when we are done with the processing, we respond to Exchange with an acknowledgement message (2.2) so that it will continue to send us messages.

If you want more information about the details of this process, refer to my detailed description in the blog posting entitled Exchange Web Services – Subscriptions and Notifications.

It is important to remember that Exchange will poll the Java HTTP Servlet at regular intervals to check that it still wants to receive notifications. The frequency is user-defined as explained in the aforementioned article. These StatusEvents do not require any processing, other than an acknowledgement of receipt back to the Exchange Server (2.2). So the Java HTTP Servlet also performs the function of keeping the subscription alive.

I have also provided a mechanism for unsubscribing from the notifications. This mechanism works by keeping track of all the subscriptions that are currently active, and then returning an Unsubscribe response (2.2) to the Exchange Server after a request to unsubscribe has been received in 1.1.

The Code

Now that you understand how the process works, let's take a look at the code.

First, I need to point out that the download contains 2 zip files. If you are a Java developer with no interest in OpenEdge, you should use the file called EWSAPI_JavaOnly.zip. The difference between this code and the EWSAPI_OpenEdge.zip file is that the latter actually makes AppServer calls to an OpenEdge AppServer, whereas the former simply displays the results of processing the messages in the log file.

The EWSAPI_OpenEdge.zip files can also be installed per the regular instructions, but there are a few additional steps that you have to perform. These are documented in a file called OpenEdge_Setup.pdf that is contained in the downloaded zip file. Essentially, you will need to create a database using a definition that is provided, set up an OpenEdge AppServer that has access to the database, and deploy some OpenEdge ABL code to the AppServer so that it can receive the calls. You will also need to copy your OpenClient runtime libraries to the Glassfish installation so that it can make the OpenClient calls. As I said, all of this is covered in OpenEdge_Setup.pdf.

Like the previous articles, you can download the code and set up your workspace as described in part 2.

What's New?

The EWSAPI project that is included in both the EWSAPI_JavaOnly.zip and the EWSAPI_OpenEdge.zip files contains the following code changes in the com.tsg.ews root package (filenames in italics are new files):

• EWSListener.java – This code, which is new, contains the Java code for the Java HTTP Servlet that listens for notifications from Exchange.
• EWSImpersonationCalendar.java – This code, which is the code for the Impersonation-based calendar WebService, has been modified to support two new operations – Subscribe and Unsubscribe.

In addition, the com.tsg.ews.calendar package contains the following code changes:

• Appointment.java – A minor modification has been made to the retrieve method to support retrieval with a ChangeKey. I also changed the code to use namespaces.

Finally, the com.tsg.ews.service package contains a lot of code changes (again, filenames in italics are new files):

• EWSResponse.java – This code, which deserializes responses from the Exchange Web Service, has been altered to support namespaces in the deserialization of the XML. It also now supports a SubscriptionID and a Watermark property.
• ExchangeService.java – There are a couple of minor changes to expose some methods outside of the class.

An enumeration and a new class have been added to enhance some of the XML parsing that I have been doing: 

• EWSNamespaces.java – This is an enumeration of the namespaces used in Exchange WebServices messages.
• NamespaceContextImpl.java – This class extends NamespaceContext and is used in namespace resolution of the XPath queries. It makes use of EWSNamespaces to perform this resolution. 

The following three classes have been added to support the registration and long-term monitoring of subscriptions:

• SubscriptionManager.java – This class manages all subscriptions. 
• SubcriptionList.java – This class is a collection of subscriptions and is used for serialization and deserialization of all subscriptions.
• Subscription.java – This class is used to register a subscription and serialize it and it keeps track of the watermarks for the subscription.

The remaining new classes and enumerations have been added to parse notifications from Exchange Server:

• Notification.java – This class is the parent class for each notification. It can be instantiated from an Exchange Web Service SOAP Notification message and it will deserialize the message into its internal objects. The notification, itself, has properties for the SubscriptionId, PreviousWatermark, and a collection of SubscriptionEvents. It also has a flag that indicates whether it is just a status event.
• SubscriptionEvent.java – A Notification may include one or more SubscriptionEvent objects. These events will be one of the EventTypes (see the next bullet point) for which the subscription has been registered (CreatedEvent, ModifiedEvent, DeletedEvent, or StatusEvent). After the event has been parsed, it contains properties for EventType, SubscriptionID, Watermark (each event will have a different watermark), TimeStamp, AffectedItem (the item that actually changed), and ParentFolder (the folder that contains the item that changed). If the AffectedItem is a calendar item, the calendar item will be retrieved (if possible) and stored in the Appointment property.
• EventType.java – This is an enumeration of the supported event types.
• IDHolder.java – When we receive an event, it comes with two elements; the first is either a FolderId or an ItemId, and the second is a ParentFolderId. These nodes contain two attributes; an ID and a ChangeKey. The IDHolder has three properties, two of which map to the ID and the ChangeKey. The third is an IDType, described in the next bullet.
• IDType.java – This enumeration contains items for each possible type of ID that can be contained in the IDHolder.

The OpenEdge pieces

If you are using the OpenEdge version of the code (EWSAPI_OpenEdge.zip), the EWSAPI project contains the following additional packages:

• com.tsg.common.collections
• com.tsg.common.interfaces
• com.tsg.dynamicopenclient
• com.tsg.dynamicopenclient.data
• com.tsg.dynamicopenclient.valueholders
• com.tsg.exceptions
• com.tsg.ews.openedge

All but the last package are modified copies of the code that accompanies the OpenEdge Dynamic OpenClient Java Example article that I wrote back in November 2009.

The last package contains one class, OpenEdgeCallFactory, that is responsible for making the OpenClient call to the AppServer.

The EWSAPI project for OpenEdge also contains an additional folder, oepieces, that contains the ABL procedure (ewscall.p) that it expects to call and a DF file (test.df) that contains the database definitions for the database table that ewscall.p expects to write to.

I'm not planning to cover the OpenEdge part of this in any detail later in the article. All ewscall.p does is accept a set of parameters and write their values directly to a table in an OpenEdge database. The ewscall.p procedure needs to be deployed to an OpenEdge AppServer. The AppServer needs to be running in STATE-FREE mode against a database that contains the ews_update table that is defined in the test.df file.

Additional Setup

Once you have set up the code and have it running, you need to update the WebService definition for the CalendarService in soapUI. The screenshot at left should help you do this. Right-click on the EWSImpersonationCalendarPortBinding tree node, and select Update Definition from the context menu.

Two additional nodes will be added – Subscribe and Unsubscribe. As you will note from the screenshot, I have created two copies of the RegisterServer request and 8 copies of the Subscribe request.

I have two Exchange servers – One running Exchange 2010 and one running Exchange 2007 SP3. Each Exchange Server has 4 mailboxes that I am using for testing, hence 2 RegisterServer requests and 8 Subscribe requests.

One other thing I would recommend is if you have an MSDN subscription and you can install Outlook 2010, do so. It will make your life a whole lot easier for testing. In Outlook 2010, you can add more than one Exchange mailbox to your list of mailboxes which means that you can work on any of the test mailboxes from just one Windows login. This makes testing much, much easier.

Establishing a Subscription

As I have described, there are two parts to the Subscription API – the Subscription and the Notification. This code walk-through is therefore going to be divided into two main sections covering each of those parts of the API. This section covers setting up the subscription, and the subscription starts at the CalendarService WebService which is in the EWSImpersonationCalendar class in com.tsg.ews.

EWSImpersonationCalendar

This code was included in part 3 of this series, but we have added two additional operations in Lines 76 through 98; Subscribe and Unsubscribe:

@WebMethod(operationName = "Subscribe")
@WebResult(name = "Response")
public EWSResponse subscribe(@WebParam(name = "ServerID") String serverID,
				 @WebParam(name = "UserAccount") String account,
				 @WebParam(name = "CallbackURL") String url,
				 @WebParam(name = "OpenEdgeURL") String oeUrl) {
	Subscription sub = new Subscription(serverID);
	sub.setAccount(account);
	sub.setListenerURL(url);
	sub.setOpenedgeURL(oeUrl);
	EWSResponse resp = sub.subscribe();
	if (!resp.hasError()) {
		SubscriptionManager.getInstance().register(sub);
	}
	return resp;
}

@WebMethod(operationName = "Unsubscribe")
@WebResult(name = "Response")
public EWSResponse unsubscribe(@WebParam(name = "SubscriptionID") String subscriptionID) {
	SubscriptionManager.getInstance().unsubscribe(subscriptionID);
	return new EWSResponse("NoError", 0, "Success");
}

These two operations rely heavily on the concept of the SubscriptionManager which keeps track of all active subscriptions and ensures that they are persisted to a local XML file. This means that when a notification is received later, we have the ability to relate that notification back to the subscription that initiated it, even if we shutdown and restart the Glassfish server. We will definitely talk more about this later once you have a chance to understand the concepts that affect it.

Looking at the Subscribe operation/method in lines 76 through 91, the operation expects to receive a ServerID as an input parameter. This ServerID is the ServerID we retrieved from a previous RegisterServer call. The UserAccount is used for Exchange Impersonation and to specify the mailbox for which the subscription is to be registered. The CallbackURL is the URL of the Java HTTP Servlet that will handle the notifications, and the OpenEdgeURL is the URL that should be used to call the OpenEdge AppServer. 

Lines 82 through 85 establish a Subscription object and in line 86, the subscribe method is called. We'll look at the subscribe method in a little more detail in a moment, but for now just accept that it is responsible for actually making the call to the Exchange WebService and obtaining a SubscriptionID. In production code, the Subscription object would be instantiated by the SubscriptionManager, but it makes for more readable example code to do it the way I have done it here.

Lines 87 and 88 check the error status on the EWSResponse object and register the subscription with the SubscriptionManager if the subscription was successful. Whether the subscription was successful or not, the EWSResponse object is returned from the WebService, and if it was successful, it contains the SubscriptionID and Watermark for the subscription.

The Unsubscribe operation in lines 93 through 98 pass the SubscriptionID to the unsubscribe method on the SubscriptionManager.

There is a special case that the Unsubscribe method will support: If an "*" is passed as the SubscriptionID, all existing subscriptions will be canceled.

The Subscription Object

I said we would take a closer look at the subscribe method, and we'll do that in just a moment. First, though, open the Subscription.java class that is in the com.tsg.ews.service package. For the most part, this object is a set of properties, including the ServiceID, ListenerURL, OpenEdgeURL, Account, SubscriptionID and Watermark. The class has JAXB serialization annotations associated with it so that it can be serialized to and from XML.

There is also a canceled flag. The canceled flag is set to false until the cancel method in line 130 is called.

The subscribe method is in lines 97 through 128. The SOAP message that needs to be sent is built between lines 98 through 118:

public EWSResponse subscribe() {
	StringBuffer request = new StringBuffer();
        request.append("<Subscribe \n");
        request.append("               xmlns=\"http://schemas.microsoft.com/exchange/services/2006/messages\"\n");
        request.append("               xmlns:t=\"http://schemas.microsoft.com/exchange/services/2006/types\">\n");
        request.append("    <PushSubscriptionRequest>\n");
        request.append("\n");
        request.append("        <t:FolderIds>\n");
        request.append("            <t:DistinguishedFolderId Id=\"calendar\"/>");
        request.append("        </t:FolderIds>\n");
        request.append("        <t:EventTypes>\n");
        request.append("            <t:EventType>CreatedEvent</t:EventType>\n");
        request.append("            <t:EventType>ModifiedEvent</t:EventType>\n");
        request.append("            <t:EventType>DeletedEvent</t:EventType>\n");
        request.append("        </t:EventTypes>\n");
        request.append("        <t:StatusFrequency>3</t:StatusFrequency>\n");
        request.append("        <t:URL>");
        request.append(this.getListenerURL());
        request.append("</t:URL>\n");
        request.append("    </PushSubscriptionRequest>\n");
        request.append("</Subscribe>\n");
        SOAPRequest message = new SOAPRequest(request.toString(), this.getAccount());

The Subscription message has been discussed in some detail in my previous post on Subscriptions and Notifications, and this message is largely the same message as that one, with the exception that the user can specify the URL, and the instantiation of the SOAPRequest object in line 118 (the last line) results in the use of Exchange Impersonation to make the request.

The DistinguishedFolderId node that is added in line 105 will result in notifications being generated based on any changes to the folder(s) that is(are) listed – in this case, just the calendar folder. The following folders are supported by name:

• calendar
• contacts
• deleteditems
• drafts
• inbox
• journal
• notes
• outbox
• sentitems
• tasks
• msgfolderroot
• root
• junkemail
• searchfolders
• voicemail

Any folder can be monitored, but if it is not one of these folders, you need to supply the FolderID for the folder.

The EventTypes node in lines 107 through 111 defines the list of events for which notifications should be received. This list could include any of the following values:

• CopiedEvent
• CreatedEvent
• DeletedEvent
• ModifiedEvent
• MovedEvent
• NewMailEvent
• FreeBusyChangedEvent (Exchange 2010 only)

The StatusFrequency node is described in a lot more detail in my previous post. In this case, I am asking for Exchange to send a status event to the Java HTTP Servlet every 3 minutes. This gives me the opportunity to cancel the subscription every 3 minutes. I have mixed feelings about status frequencies in terms of how long they should be, and I'll talk more about this later in this article.

Once the message is built, we call makeRequest on the ExchangeService object with the message in lines 119 through 128:

        EWSResponse response;
        try {
		response = getService().makeRequest(message);
		subscriptionID = response.getSubscriptionId();
		watermark = response.getWatermark();
	} catch (Exception e) {
		response = new EWSResponse(e);
	}
	return response;
}

If the call is successful, the EWSResponse object will contain a unique SubscriptionID and a Watermark. It is important to understand that the SubscriptionID is unique, even if you have already registered a subscription for this mailbox. In other words, it is possible to have more than one subscription active for a mailbox.

The SubscriptionManager Object

The SubscriptionManager object is responsible for tracking all active subscriptions. In essence, it is a persistence mechanism for maintaining the state of a subscription over a long period of time. The SubscriptionManager takes care of persisting the subscription details to an XML file and providing a central point through which all changes to a subscription can take place. Any time any object needs information about a subscription, it makes a call to the SubscriptionManager to obtain information about the subscription.

Any changes made to a subscription are persisted to the XML file, so that if the Glassfish server is restarted, and the subscription is not lost, the SubscriptionManager will be able to maintain the context of any subscriptions that are currently active.

Why do we care about this context anyway? For a number of reasons:

1. We associate the OpenEdge URL with a subscription and there is no way to include this information with the information that we submit to Exchange Server so there is no way to know what we need to do when we receive a notification if we don't persist the context of the subscription; 
2. To be able to specifically cancel a subscription, the SubscriptionManager needs to know about it. Remember that the only ways to cancel a subscription are by responding to a notification with an unsubscribe response, or by not responding and waiting for it to time out. Ideally, we need the ability to do this gracefully, and that means responding to a notification with the unsubscribe response. It may be several minutes between the time that an unsubscribe request is received by the Java EWS WebService and the time a notification is received by the Java HTTP Servlet, so all the SubscriptionManager can do is note that the subscription is canceled. When the next notification comes in, the Java HTTP Servlet can then act on the cancellation;
3. In a production environment where there may be numerous mailboxes that are all being monitored concurrently, it would be far more efficient to receive all notifications for one mailbox as a set. The problem is that when a subscription is registered, there may be no way of knowing what other subscriptions will be registered for the same mailbox. Although the implementation that I have included does not do this, the SubscriptionManager is the ideal place to maintain a link between subscriptions for the same mailbox so that the load on the Exchange Server can be minimized and delegated to a separate middleware service; and
4. If the Glassfish server goes down for any reason, context needs to be maintained so that when it is restarted, it can re-establish the subscriptions as they were running before it was terminated. Of course, in a production environment with load balancing in place, it would be unlikely that the Glassfish server farm would not be available, but the notion of more than one Glassfish server immediately means that the subscription context needs to be persisted centrally.

So although the SubscriptionManager that is provided with this example is somewhat primitive, it performs a key role in maintaining continuity and performance for subscriptions. 

The SubscriptionManager code is not particularly interesting so I am not going to spend any time walking through it. It definitely has holes in it, but it works for the examples that we are using here.

Establishing a Subscription

Now that you know how the code works for establishing a subscription, let's go ahead and register a subscription for one of the mailboxes. Assuming you already have the code installed and running in Eclipse and the Glassfish server running, open soapUI and open a request under the Subscribe operation for the CalendarService:

The ServerID element should contain a UUID for a server that was registered using the RegisterServer operation. The UserAccount should contain the e-mail address of the mailbox that you want to monitor.

The CallbackURL is the first new element you need to complete. This is the URL that the Exchange Server should use for all notifications. This URL will be in the form http://<server-name>:8080/EWSAPI/EWSListener where <server-name> is the machine that is running your Glassfish server (your local machine). Note that you cannot use localhost as the server name. Calls to this URL are going to come from your Exchange Server so it needs a DNS-registered name.

The OpenEdgeURL is only of interest to you if you are an OpenEdge developer. If this element contains anything but a string that begins with AppServer:// it will be ignored. If you are using the default asbroker1 AppService, the URL that I have in the screenshot (AppServer://localhost:5162/asbroker1) should work for you, assuming your AppServer is running on the same machine as your Glassfish server.

Once you have filled out the request parameters, submitting the request will get you a response similar to the one in the following screenshot:

The Response node contains 3 attributes. The ErrorState attribute should have a value of "NoError" and should be followed by a SubscriptionID attribute and a Watermark attribute, both with strings of characters that don't mean anything. The SubscriptionID uniquely identifies the subscription and can be used to unsubscribe through the Unsubscribe operation.

If you switch your focus to the Console view in Eclipse for the Glassfish Server Log, you will see messages start popping up occasionally in the view like the one below:

INFO: Notification:
=============
	Account: froggf@intangere.internal.intangere.com
	Subscription Id: LQBpbnRhbmdlcmVleHMuaW50YW5nZXJlLmludGVybmFsLmludGFuZ2VyZS5jb20QAAAAdyIIAinbTUGf+zpPjEyTaw==
	Prev Watermark: AQAAANnz8X+xbA5HtnQ8PXPRJfHGLgAAAAAAAAE=
	More Events: false
	Status: true
	Event: StatusEvent	Watermark: AQAAANnz8X+xbA5HtnQ8PXPRJfHGLgAAAAAAAAE=

INFO: <?xml version="1.0" encoding="utf-8"?><soap:Envelope ... </soap:Envelope>
INFO: Total execution time: 205

These messages will pop up every 3 minutes and indicate that the Glassfish server is receiving messages from the Exchange Server.

If you switch to Outlook and create a calendar item in Outlook, you will see a message something like the one below about 20 seconds after you create it:

INFO: Notification:
=============
	Account: froggf@intangere.internal.intangere.com
	Subscription Id: LQBpbnRhbmdlcmVleHMuaW50YW5nZXJlLmludGVybmFsLmludGFuZ2VyZS5jb20QAAAAdyIIAinbTUGf+zpPjEyTaw==
	Prev Watermark: AQAAANnz8X+xbA5HtnQ8PXPRJfHGLgAAAAAAAAE=
	More Events: false
	Status: false
	Event: ModifiedEvent	Watermark: AQAAANnz8X+xbA5HtnQ8PXPRJfHJLgAAAAAAAAE=
			TimeStamp: Wednesday, July 7, 2010 6:39:42 PM PDT
		Affected Item-- Type: FolderId	ID: ...	ChangeKey: AgAAAA==
	Event: ModifiedEvent	Watermark: AQAAANnz8X+xbA5HtnQ8PXPRJfHKLgAAAAAAAAE=
			TimeStamp: Wednesday, July 7, 2010 6:39:42 PM PDT
		Affected Item-- Type: FolderId	ID: ...	ChangeKey: AgAAAA==
	Event: ModifiedEvent	Watermark: AQAAANnz8X+xbA5HtnQ8PXPRJfHLLgAAAAAAAAE=
			TimeStamp: Wednesday, July 7, 2010 6:39:42 PM PDT
		Affected Item-- Type: FolderId	ID: ...	ChangeKey: AgAAAA==
	Event: CreatedEvent	Watermark: AQAAANnz8X+xbA5HtnQ8PXPRJfHRLgAAAAAAAAE=
			TimeStamp: Wednesday, July 7, 2010 6:39:42 PM PDT
		Affected Item-- Type: ItemId	ID: ...	ChangeKey: DwAAAA==
		Appointment-- Subject: New Appointment for Software Gorilla test
			Location: Who cares where! Just make sure it works!!
			Start: Wednesday, July 7, 2010 7:00:00 PM PDT	End: Wednesday, July 7, 2010 8:00:00 PM PDT

INFO: <?xml version="1.0" encoding="utf-8"?><soap:Envelope ... </soap:Envelope>
INFO: Total execution time: 75

I have edited this slightly so it fits on the screen. This notification informs you that the Exchange Server has called the Java HTTP Servlet which has processed the notification.

If you supplied an OpenEdge URL for the AppServer, and you connect to the database that contains the ews_update table from OpenEdge and execute the following code, you will find entries in the database for each appointment that you created:

for each ews_update:
    display skip(1)
        ews_update.cEmailAddress skip
        ews_update.cSubject skip
        ews_update.cLocation skip
        ews_update.dTimeStamp skip
        ews_update.dStartTime ews_update.dEndTime skip(2)
      with 1 down side-labels width 100.
end.

There are a couple of important things to note about the data in the OpenEdge database. If you are not living in the UTC timezone, you will see a difference between the time that OpenEdge reports and the time that notification reported. That's to be expected. Time in the database is reported as UTC. In my case, I am in the PDT (Pacific Daylight Time) timezone.

Second, it looks like the subject and location have been truncated. Again, no surprise. I set the formats in the database fields to X(50) for the subject and location, although the database will store more than that for them.

Looking back to the notification that we received from Exchange, you'll note that there is a Watermark for each event, and that each Watermark is different (sometimes by only one character). The Watermark provides a way of going back and verifying that you have received all events from Exchange, but it is important to realize that subscribing with an old Watermark can have significant performance impacts on Exchange.

You'll also notice that there is a Total Execution Time for each notification. This is the time taken (in milliseconds) from the time the notification was received, until the time the response was flushed out to the HTTP stream. In other words, it is the total time taken to process the request. What is interesting about this is that it takes as much as 4 times longer to process a StatusEvent than it does to process a real notification. I have determined that this has to do with the time it takes to read the original notification message off the input stream, and I have not yet figured out how to shorten that time.

Now that you have seen the notification mechanism working, let's look at what is going on under the covers. 

Receiving Notifications – EWSListener

As noted, notifications are received on the http://<server-name>:8080/EWSAPI/EWSListener URL. To achieve this, the EWSListener.java class in com.tsg.ews listens on the URL for incoming requests, so go ahead an open EWSListener.java in Eclipse.

All of the work associated with processing a request occurs in the doPost method that runs between lines 52 and line 170. I have included a lot of comments in the code that is in the zip file, so I am going to extract sections of it here and exclude the comments that are in the source. Line numbers, though, correspond with the source in the zip file.

Receiving the Notification (2.1 in the diagram above)

Lines 52 through 63 deal with receiving the notification:

protected void doPost(HttpServletRequest request,
		HttpServletResponse response) throws ServletException, IOException {
	boolean safeDoc = false;
	boolean canceled = false;

	long startTime = GregorianCalendar.getInstance().getTimeInMillis();
	long endTime;

	try {

		String result = ExchangeService.getResult(request.getInputStream());
		Document doc = loadXML(result);
		safeDoc = true;

		Notification nfcn = null;
		try {
			nfcn = new Notification(doc);

The notification XML document is received from Exchange in the request object. The response object will contain our response back to Exchange later.

The safeDoc flag is used to indicate that we have successfully parsed the XML document and that it is indeed a document that we expect. The canceled flag indicates that the response to Exchange should be an unsubscribe message. The startTime and endTime fields are used to calculate the total elapsed time of the call.

In line 63, we make a call to getResult (now a static method) on the ExchangeService class. This method simply reads all the data off an input stream. 

Now that we have a character string, it's time to do something with it, and in line 66, we make a call to the internal loadXML method which parses the XML string into a DOM document.

In line 74, we instantiate a Notification object with the XML document and the Notification object takes care of parsing the the DOM into appropriate objects. 

Parsing the XML

I'm not going to spend any time walking through the code that parses the XML document, but the following XML is from an actual notification for the creation of a new appointment (I've truncated IDs for brevity):

<?xml version="1.0" encoding="utf-8"?>
<soap11:Header />
    <soap11:Body>
        <m:SendNotification xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"
                               xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages">
            <m:ResponseMessages>
                <m:SendNotificationResponseMessage ResponseClass="Success">
                    <m:ResponseCode>NoError</m:ResponseCode>
                        <m:Notification>
                            <t:SubscriptionId>LwBncnzAg=</t:SubscriptionId>
                            <t:PreviousWatermark>AQAAAAE=</t:PreviousWatermark>
                            <t:MoreEvents>false</t:MoreEvents>
                            <t:ModifiedEvent>
                                <t:Watermark>AQAAAAAE=</t:Watermark>
                                <t:TimeStamp>2010-07-05T18:06:29Z</t:TimeStamp>
                                <t:FolderId Id="AQAoAnnJwAAAx4AAAA=" ChangeKey="AgAAAA==" />
                                <t:ParentFolderId Id="AQAoAAAxIAAAA=" ChangeKey="AQAAAA==" />
                            </t:ModifiedEvent>
                            <t:CreatedEvent>
                                <t:Watermark>AQAAAAE=</t:Watermark>
                                <t:TimeStamp>2010-07-05T18:06:29Z</t:TimeStamp>
                                <t:ItemId Id="AAAoAGF/4YLFkwAAAAQJAgAA" ChangeKey="DwAAAA==" />
                                <t:ParentFolderId Id="AQAoAAAx4AAAA=" ChangeKey="AQAAAA==" />
                            </t:CreatedEvent>
                        </m:Notification>
                    </m:SendNotificationResponseMessage>
            </m:ResponseMessages>
        </m:SendNotification>
    </soap11:Body>
</soap11:Envelope>

The Notification element contains a SubscriptionId, PreviousWatermark and MoreEvents element. When the code parses this XML document, it creates a Notification object to contain this information.

The Notification element also contains an Event element for each event for which you are receiving a notification. In this case, the first Event element is a ModifiedEvent and it applies to a Folder (in this case, the calendar folder that has been modified by the addition of a new item). The second event, the CreatedEvent element, contains an ItemId element that identifies the calendar item that was created.

The Notification is an iterable collection of SubscriptionEvent objects, so when the code parses these elements, it creates and stores a SubscriptionEvent object to hold each of these Event elements. Each Event node contains a Watermark that is applicable for the event that took place and a TimeStamp – the time that the event took place. These values are recorded against the SubscriptionEvent object.

An Event element will contain either a FolderId element or an ItemId element which is the object that was affected by the event. This element is parsed and an IDHolder object is created to contain it and it is stored as the AffectedItem property of a SubscriptionEvent object.

There is also a ParentFolderId which is the ID of the folder that contains the object that is affected by the event. When this element is parsed, its data is also stored in an IDHolder object in the ParentFolder property of the SubscriptionEvent object.

In the case of the above notification, one Notification object was created and it contained two SubscriptionEvent objects – one for the ModifiedEvent and one for the CreatedEvent. 

Marrying the Notification and the Subscription

Now that we have parsed the notification, we'll go back to the code in line 77 through 86 and deal with marrying the Notification with a Subscription object:

	Subscription sub = SubscriptionManager.getInstance().getSubscription(nfcn.getSubscriptionId());
	if (sub != null) {
		nfcn.setAccount(sub.getAccount());

		canceled = sub.isCanceled();
		if (canceled)
			SubscriptionManager.getInstance().clearSubscription(sub);

		if (!nfcn.isStatus()) {
			getAppointments(sub.getService(),nfcn);
		}

In line 77 we ask the SubscriptionManager for the Subscription object associated with the SubscriptionId in the Notification object. The next set of code only executes if we found a Subscription object.

In line 80, we set the account (e-mail address) for the notification to the account of the subscription and in line 84, we check to see if the subscription has been canceled by a call to the Unsubscribe operation. If it has, we set the canceled flag and remove the subscription from the SubscriptionManager because this is the last notification we will receive for this subscription.

As I mentioned earlier, Exchange will send notifications on a regular basis (as defined by StatusFrequency). These notifications do not contain any data; they are simply status messages (think of them as pings). When we parse the SOAP message into the Notification object, if the notification is a StatusEvent, we set a flag to indicate that this is only a status message. In line 91 we check to see if this is a status message, and if not, we make a call to getAppointments which is responsible for retrieving appointments from the Exchange Server.

getAppointments (2.3 in the diagram above)

The getAppointments method is in lines 176 through 209:

private void getAppointments(ExchangeService service, Notification nfcn) {

	for (SubscriptionEvent se: nfcn) {

		if (se.getEventType() != EventType.STATUS
				&& se.getEventType() != EventType.DELETED
				&& se.getAffectedItem() != null
				&& se.getAffectedItem().getIdType() == IDType.ITEM) {

			Appointment appt = new Appointment(service);
			appt.setEmailAddress(nfcn.getAccount());

			EWSResponse resp = appt.retrieve(se.getAffectedItem().getId(),
                                                  se.getAffectedItem().getChangeKey());
			if (!resp.hasError()) {
				se.setAppointment(appt);
			}
			else {
				if (resp.getResponseCode().equals("ErrorItemNotFound")) {
					se.setEventType(EventType.DELETED);
				}
				else {
					System.out.println(...);
				}
			}
		}
	}
}

As you will recall, the notification that we receive from Exchange only contains the ID of the item that has changed. It does not provide any information about the item itself. In getAppointments we take care of retrieving those items.

Notification objects are iterable collections of SubscriptionEvents so we loop through all the SubscriptionEvents (line 179) in the Notification collection, ignoring anything that is not a Created- or ModifiedEvent for an ItemId (lines 183 to 186).

We then instantiate an Appointment object (line 189), set the account for it (line 190), and attempt to retrieve it from the Exchange Server (line 193).

If we're successful retrieving the appointment, we set the Appointment property of the SubscriptionEvent from the appointment we retrieved (line196).

If we failed with an ErrorItemNotFound message from Exchange, it means that the item has been deleted so we cannot find any information about it. The event type therefore needs to be changed to a DeletedEvent (lines 200 through 202). This may seem strange, but remember that Exchange will move a deleted item to the Deleted Items folder which means we will receive a ModifiedEvent, rather than a DeletedEvent.

Calling OpenEdge (2.4 in the diagram above)

Returning to the code in doPost, lines 96 through 103 deal with the call to OpenEdge:

	System.out.println(nfcn.print());

	// **OESPECIFIC
	// The following code is inactive in the Java zip file version of the code.
	// If the subscription has an OpenEdge AppServer URL in it, let's call OpenEdge.
	if (sub.getOpenedgeURL().startsWith("AppServer://")) {
		OpenEdgeCallFactory.callOpenEdge(sub, nfcn);
	}

Line 96 just displays the notification information in human readable form – it is responsible for the Notification messages in the server log.

Line 101 through 103 checks if the OpenEdge URL for the subscription starts with the string "AppServer://" and, if so, calls the callOpenEdge method which is in lines 68 through 99 of the OpenEdgeCallFactory class:

public static void callOpenEdge(Subscription sub, Notification nfcn) {
	checkInitialized();
	for (SubscriptionEvent se: nfcn) {
		if (se.getAffectedItem() != null && se.getAffectedItem().getIdType() == IDType.ITEM) {
			Call call = new Call(OECALL);
			call.setParameterValue(PAR_EMAIL, sub.getAccount());
			call.setParameterValue(PAR_SUBSCRIPTION_ID, nfcn.getSubscriptionId());
			call.setParameterValue(PAR_EVENT_TYPE, se.getEventType().toString());
			call.setParameterValue(PAR_TIME_STAMP, se.getTimeStamp());
			call.setParameterValue(PAR_WATERMARK, se.getWatermark());
			call.setParameterValue(PAR_ITEM_ID, se.getAffectedItem().getId());
			call.setParameterValue(PAR_CHANGE_KEY, se.getAffectedItem().getChangeKey());
			Appointment appt = se.getAppointment();
			if (appt != null) {
				call.setParameterValue(PAR_SUBJECT, appt.getDescription());
				call.setParameterValue(PAR_LOCATION, appt.getLocation());
				call.setParameterValue(PAR_NOTES, appt.getNotes());
				call.setParameterValue(PAR_START_TIME, appt.getStartDate());
				call.setParameterValue(PAR_END_TIME, appt.getEndDate());
				call.setParameterValue(PAR_REMINDER_MINUTES, appt.getReminderMinutes());
				call.setParameterValue(PAR_REMINDER_SET, appt.isReminderSet());
			}
			call.executeCall(sub.getOpenedgeURL());
			if (call.isReturnError()) {
				System.out.println(...);
			}
			else if (call.getFailure() != null) {
				System.out.println(...);
			}
		}
	}
}

This method simply instantiates a Call object (line 72), sets the parameters for the call to ewscall.p (lines 73 through 89), and executes the call using the OpenEdge URL (line 90). A successful call to ewscall.p will result in the data being written to the OpenEdge database. If you read my article on the OpenEdge Dynamic OpenClient Java Example, this code will be reasonably familiar.

At this point, all that is left to do is update the subscription and respond to the Exchange Server.

Updating the Subscription

From the information in David Sterling, et al's book, Inside Microsoft Exchange Server 2007 Web Services, I had understood that the Watermark was associated with a subscription, but in working with this a little more, I have come to the conclusion that the Watermark is associated with the Exchange Server as a whole. I had been hoping to use the Watermark to guarantee that I knew that I had received all events from the Exchange Server, but it does not seem to work the way I had understood. I'm still piecing this together at the time of writing, so the code that is included with the example is predicated on my understanding from Sterling, et al.

Lines 107 through 127 deal with updating the subscription:

			if (!sub.getWatermark().equals(nfcn.getPreviousWatermark())) {
				System.out.println(...);
			}

			String lastWm = null;
			for (SubscriptionEvent se : nfcn) {
				lastWm = se.getWatermark();
			}

			if (lastWm != null) {
				sub.setWatermark(lastWm);
			}
		}
		else {
			canceled = true;
			System.out.println(...);
		}
	}

In line 107 through 109, we check to see if the subscription's current watermark matches the previous watermark of the notification we received. If it doesn't (and if my understanding of Sterling, et al is correct, it should), we print a warning message to the log.

In lines 112 through 116, we loop through all the SubscriptionEvent objects and store the last watermark and line 117 through 119, we set the Watermark on the Subscription to the Watermark of the last SubscriptionEvent. 

In lines 121 through 125 we deal with the situation where we were unable to match the notification with a subscription – we set the canceled flag to true so that the subscription will terminate.

Responding to Exchange (2.2 in the diagram above)

In lines 138 through 170 we deal with responding to Exchange:

	if (!safeDoc) {
		response.sendError(HttpServletResponse.SC_BAD_REQUEST);
	}
	else {
		String str;

		if (canceled) {
			str = getResponseXML(UNSUBSCRIBE);
		}
		else {
			str = getResponseXML(OK);
		}

		response.setCharacterEncoding("UTF-8");
		response.setStatus(HttpServletResponse.SC_OK);
		response.setContentType("text/xml; charset=UTF-8");
		response.setContentLength(str.length());

		PrintWriter w = response.getWriter();
		w.print(str);
		w.flush();
	}
	response.flushBuffer();
	endTime = GregorianCalendar.getInstance().getTimeInMillis();
	System.out.println(String.format("Total execution time: %1$s", (Long) (endTime - startTime)));
}

In lines 139 through 141, we handle the situation where we received a bad message from Exchange – we return an HTTP 400 error.

In lines 143 through 151 we check to see if the canceled flag is set. If it is, we build an Unsubscribe response, otherwise we just build an OK acknowledgement response. Lines 154 through 157 set some properties of the response object and lines 160 through 167 write the response to the output stream and flush the buffers. 

Finally, in lines 168 and 169 we determine the amount of time that has elapsed since the notification was received and we display this.

Summary

That's it. Now you have seen how to handle the Exchange Web Services Push Subscription API in Java. Although this may seem like a convoluted piece of code, I have found it to be extraordinarily stable. I have experimented with taking down the Glassfish server and bringing it back up and the notifications keep on coming.

I have left the server running for days with no updates happening on the Exchange Server with no problem at all. I have also exposed it to some pretty serious stress and it has performed with no problems at all on fairly small hardware and software configurations.

Most of the time, notifications, including the call back to Exchange Server and OpenEdge, are being processed in around 100ms. StatusEvents take just over 200ms.

The one thing I am still having trouble with is trying to decide on an appropriate StatusFrequency. I would like to set the StatusFrequency to 10 minutes. That way, it will take 30 minutes for a subscription to expire. The problem is that clean cancellations rely on regular status events, so anything longer than 3 minutes can seem like an eternity. One thing I have definitely concluded is that the StatusFrequency needs to be configurable.

I hope this article has been useful. The next article in this series is going to deal with determining availability, but I am going to be engaged in some pretty serious development work for the next few weeks so it is likely that the next article in this series will only see the light of day toward the end of the Summer. 

Let me have your feedback…

Goals and Background

Part 2 of this series of articles is dedicated to giving you an overview of the process of connecting to Exchange and doing some basic Calendar item work. Through the sample code, you will learn how to:

1. Connect to the Exchange Web Service from Java;
2. Create calendar items on the Exchange Server;
3. Get calendar items from the Exchange Server;
4. Delete calendar items from the Exchange Server; and
5. Connect to the Java service from OpenEdge to perform the same operations.

The Java functionality will be exposed as a Web Service for other platforms to leverage, which is how we will get at it from OpenEdge. The OpenEdge code leverages the new GUI for .NET and object-oriented extensions, so you may find the example interesting if you are an OpenEdge developer who has not done this before.

One of the things that I have learned about integration through bitter, painful experience, is that it is much easier if you remove as many variables as possible, so, for the purposes of this exercise, I am only going to expose a few of the calendar properties. This helps to keep the code simple and this article reasonably short. I think I succeeded at the former, but the latter… not so much.

The code that you will be looking at is code that I wrote as part of my research into the EWS API and, as such, it is prototype code. What this means is that it is ugly. I have not spent a lot of time commenting it and it does not resemble the code that I would deploy to production from an object-model point of view. But that is OK. A prototype is intended to prove a concept.

There is nowhere that integration is harder than when you try and integrate Microsoft's technologies with a non-Microsoft platform. Microsoft seems to make it deliberately difficult. I have therefore deliberately hand-coded the SOAP messages that are used to communicate with Exchange in this example. I started out by trying to use Java frameworks like Axis 2 and JAX-WS for the SOAP piece, but I quickly ran into problems. So I decided that I would remove those variables and prove I could get the code working. Now that I am satisfied that I can do what I need to be able to do, I will go back and decide what I am going to do about integrating one of these frameworks into the mix. 

Another important factor that I should point out about this prototype is that I plan to integrate this whole solution with an Enterprise Service Bus (specifically, Apache ServiceMix). So it is really important to me that this can function as a properly service-oriented solution – hence the focus on web service enablement. Ultimately the service will generate messages that can be captured and interrogated in the service bus using a complex event processing engine.

Setting up the Code

Assuming you followed my instructions in part 1and now have the environment working, you can download the code for this article. The zip file contains two projects:

• EWSAPI – This is an Eclipse Java EE  project that is intended to be deployed to a Glassfish server. It contains the Java code that does the actual communication with Exchange and accepts WebService requests from external sources.
• OEEWS – This is an Eclipse project that is intended to be deployed in an OpenEdge Architect workspace that calls the Java WebService.

Both of these projects can be imported in the same way as I described in part 1, under the heading "Running a quick test to prove it works", although you should be aware that EWSAPI should be imported into a Java EE workspace, whereas OEEWS should be imported into an OpenEdge Architect workspace. I have had problems trying to run OE Architect in the same workspace as a Java EE configuration, so I don't recommend you try it.

After you have imported EWSAPI into your Java EE workspace, go ahead and drag it across to the Glassfish server in the server view and it will be deployed for you, ready to run.

The OEEWS project does not require any special configuration. Just import it into OpenEdge Architect and you will be good to go. Note that this code was written and compiled under 10.2B01 so I don't know how compatible it is with earlier versions.

Turn on line numbers

In the remainder of this series of articles, I will be cross-referencing the code that I have provided in the zip file. When I do so, I will mention specific line numbers that map to the line numbers in the source file. It will help you a lot if you turn the line number feature on for all editors in Eclipse so that you can follow along.

Also note that I occasionally include portions of the source code in the article for easy reference, but when I do, I do not include the line numbers so the line number reference refers to the example code.

To turn line numbers on, follow these steps:

1. From the Window menu, choose "Preferences"
2. Expand the "General->Editors" nodes and select "Text Editors"
3. Make sure the "Show line numbers" check box is checked as in the screenshot at left.

Understanding Code Contents

As mentioned, the code for this first article is divided into a Java EE project and an OpenEdge project. The two projects will grow over this series of articles. Here's a brief description of these two projects and how they relate to the rough architecture that I am working on.

EWSAPI – Exchange Web Services API

The EWSAPI project is a Java Web Service that accepts Web Service requests to a defined API, turns around and passes them on to Microsoft Exchange. 

In the diagram at left, therefore, the EWSAPI project contains the code that is the endpoint for the arrow numbered 1.1, and the source for the arrow numbered 1.2.

An important part of a SOA architecture is a canonical data model that can be used to abstract the integration from the implementation, and for me, this service will later act as the separation point between the business layer and the Microsoft Exchange communication. As I work through this in more detail, it is becoming apparent that what I am currently thinking of as the EWSAPI will ultimately be split into two completely separate components. I think there is a good case to be made for an abstraction API that allows you to plug in other adapters to work for other calendar and e-mail servers.

The source code for this article in the EWSAPI project contains three WebService operations:

• CreateAppointment
• GetAppointment
• DeleteAppointment

Something to be aware of is that the APIs that we are going to be working with can do substantially more than I am going to talk about in this article. For example, an appointment can have an invitee list, recurrence, attachments, and so on. I am more focused on showing you how to create the appointment on the Exchange Server, retrieve the appointment from Exchange, and delete it. I may come back to the more detailed stuff in later posts.

As an aside, I wrote most of this code before I decided on the structure of these articles. My test environment is already set up to support things like Exchange Impersonation and Subscriptions. As a result, some of the code that I am providing you here already supports some of that functionality, but some of it is only half-implemented, so don't go onto those things until we get to those articles because the code may not work properly. 

OEEWS – OpenEdge Exchange Web Services API

The second project is the OEEWS project which contains the OpenEdge ABL code that allows you to call each of these APIs from OpenEdge. I have created a .NET style UI in OpenEdge that allows you to connect to the service, generate 10 appointments automatically and randomly, fetch an appointment from the server, view it, and delete it.

Yes, I know my UI is ugly, but it's a prototype so who cares. If you don't like it, fix it.

The OEEWS component corresponds with the trigger point for the 1.1 arrow in the diagram above, so this article is purely about communicating with Exchange. We are not worried about the subscription API (all the #2 arrows) just yet. 

Another thing to know about the OEEWS code is that I have liberally mixed procedural, object-oriented and .NET UI code in together. My primary audience for the OpenEdge code is people who have never worked with the OO stuff before, so I want them to get comfortable with it and realize that it is not that complicated.

Now that you know what is in the mix, let's get down to the code and understand what is going on. If you are an OpenEdge ABL developer, this first section is going to look a little scary – it's Enterprise Java. You may not recognize the code, but I am pretty sure you will understand it once I have walked through it, so hang on tight. You won't want to skip down to the ABL section because there are concepts that I am going to talk about that you need to understand in the following section.

The EWS API

The Java-based EWSAPI consists of 5 source code files in 3 packages. The files are:

• EWSCalendar.java – Contains the source code file for the web service itself and it lives in the org.tsg.ews package;
• Appointment.java – Contains a class that represents a Calendar item, including properties for the appointment and methods to create, get, and delete appointments. It lives in the org.tsg.ews.calendar package;
• EWSResponse.java – Contains the source code for a class that acts as a container for the response that is returned from Exchange. It lives in the org.tsg.ews.service package;
• ExchangeService.java – Contains a class that abstracts the connection to an Exchange Web Service. It lives in org.tsg.ews.service package; and
• SOAPRequest.java – Contains a class that is responsible for actually calling the Exchange Web Service.

Go ahead and open Eclipse for Java EE, and, assuming you have already imported the EWSAPI project into your workspace, expand the EWSAPI node. Underneath that, expand the "Java Resources: src" node and expand each of the packages. Then go ahead and open EWSCalendar.java so that your UI looks like this:

Now let's dig into this code. 

EWSCalendar

The EWSCalendar class is the main entry point in the the Java code on the server. The class exposes a Web Service that consists of 3 methods:

• createAppointment;
• getAppointment; and
• deleteAppointment.

Each of these methods does pretty much what its name implies. The code in EWSCalendar actually does very little work. It is the facade that exposes the service for use as a Web Service. 

Web Services Annotations

Setting up a Web Service in Java EE is as simple as annotating a class for use as a Web Service. In line 15, there is an attribute that annotates the following class, EWSCalendar, as a Web Service with a name "AppointmentService". 

Lines 18 and 19 contain annotations that specify that createAppointment should be treated as a Web Service operation called "CreateAppointment" and that the node in the SOAP document that is returned from the call should have a node called "Response" that contains the output parameter for the call. In line 20, createAppointment is defined as returning an EWSResponse object, and the remainder of line 20 through line 30 define the parameters for the createAppointment call. Each parameter is annotated with a WebParam annotation that provides the name of the XML node that must be created for its value.

Lines 45 through 51 do the same thing for the GetAppointment operation and lines 58 through 64 do the same thing for DeleteAppointment. That's all that is required to define a Web Service. I don't need to generate a WSDL or anything. Java EE does that all for me automatically based on the annotations for the methods.

Now if you are a Java programmer looking at the parameters for this call, you may be wondering why the parameters are not defined as objects. The simple answer is because it is a lot easier to work with this call in OpenEdge if each parameter is specifically defined, than it is to expect an appointment object, say, as an input parameter for CreateAppointment.

CreateAppointment

Going back to to line 31, the code for CreateAppointment looks like this:

ExchangeService srvc = new ExchangeService(hostName, domain, username, passwd);
Appointment appt = new Appointment(srvc);
appt.setDescription(description);
appt.setLocation(location);
appt.setStartDate(startDate);
appt.setEndDate(endDate);
appt.setNotes(notes);
appt.setReminder(reminder);
appt.setReminderMinutes(minutes);
EWSResponse resp = appt.create();
return resp;

In line 32, I create an ExchangeService object, using the host name for the Exchange CAS Server that hosts the Exchange Web Service API, the domain name of the user for whom an appointment is being created, and the user name and password for that user. We'll dive into what this code actually does in a few minutes, but for the moment let's finish off the createAppointment call.

Line 33 instantiates an Appointment object using the ExchangeService object created in the line before and lines 33 through 39 set a number of properties of that appointment. Description is actually the subject of the appointment and lines 38 and 39 set the reminder for the appointment. 

Line 40 calls the create method on the appointment and this is the code that does all the work. The create method returns an EWSResponse object, which is annotated for JAXB XML serialization to produce the XML results that we need.

The Appointment Object and JAXB XML Serialization

Now I'd like you to open Appointment.java in the org.tsg.ews.calendar package. You can skip down to lines 22 and 23, because there is something here that affects getAppointment later. You'll note in these two lines that there are two more annotations:

@XmlRootElement(name = "Appointment")
@XmlAccessorType(XmlAccessType.PROPERTY)
public class Appointment {

The two @Xml annotations are for JAXB which will automatically serialize the whole Appointment class into a root element called "Appointment." The second annotation tells JAXB to look at all get… and set… methods as properties and turn them into XML. If you scroll down to line 70, you'll see an annotation "@XmlElement(name = "Subject") just before the getDescription method that tells JAXB to serialize this property into an XML element (not an attribute) called "Subject". So the value of this property will be serialized as follows:

<Subject>Go catch some flies - 01</Subject>

Again, XML serialization for classes is built in to Java EE. This piece of things is important for you to understand, because the same mechanism has been applied to EWSResponse so that its data can be serialized into the SOAP message that will be the return value to the CreateAppointment call.

The Create Method

Now scroll down to the create() method in Appointment.java in line 278:

public EWSResponse create() {
	DatatypeFactory dtf = DatatypeFactory.newInstance();
	StringBuffer request = new StringBuffer();
	request.append("<CreateItem \n");
	request.append("               SendMeetingInvitations=\"SendToNone\"\n");
	request.append(String.format("               xmlns=\"%1$s\"\n", SOAPRequest.MESSAGES_URI));
	request.append(String.format("               xmlns:t=\"%1$s\">\n", SOAPRequest.TYPES_URI));
	request.append("    <Items>\n");
	request.append("        <t:CalendarItem>\n");
	request.append("            <t:Subject>");
	request.append(description);
	request.append("</t:Subject>\n");
	if (notes != null) {
		request.append("            <t:Body BodyType=\"Text\">");
		request.append(notes);
		request.append("</t:Body>\n");
	}
	if (reminder) {
		request.append("            <t:ReminderIsSet>true</t:ReminderIsSet>");
		request.append("            <t:ReminderMinutesBeforeStart>");
		request.append(Integer.toString(reminderMinutes));
		request.append("</t:ReminderMinutesBeforeStart>\n");
	}
	if (startDate != null) {
		request.append("            <t:Start>");
		request.append(dtf.newXMLGregorianCalendar(startDate).toXMLFormat());
		request.append("</t:Start>\n");
	}
	if (endDate != null) {
		request.append("            <t:End>");
		request.append(dtf.newXMLGregorianCalendar(endDate).toXMLFormat());
		request.append("</t:End>\n");
	}
	if (location != null) {
		request.append("            <t:Location>");
		request.append(location);
		request.append("</t:Location>\n");
	}
	request.append("        </t:CalendarItem>\n");
	request.append("    </Items>\n");
	request.append("</CreateItem>\n");

Lines 280 through 319 simply build up the body of the SOAP message that needs to be sent to Exchange for it to create the appointment.

One of the things about Exchange Web Services that is critical to understand is that there can be no mistake in this XML. It needs to appear exactly as it is defined in the Exchange SDK, otherwise it will not work. The order of the elements is absolutely critical. I have spent a lot of time trying to figure out why a call was not working, only to find that the the XML was badly formatted or a node was spelled incorrectly. Exchange does not help you here. You simply get a 500 – Internal Server Error message from the Web Server which is about as much use as an udder on a bull.  

In line 282, we start out by defining a CreateItem node that contains an Items node that has one CalendarItem node in it. Namespacing is very important, so lines 284 and 285 set up the appropriate namespaces from constants defined in the SOAPRequest class.

Lines 288 through 290 serialize the subject while lines 291 through 295 handle the Notes or Body of the appointment. Lines 296 through 301 handle the reminder and lines 302 through 311 handle the start and end dates for the appointment. After handling the location in lines 312 through 316, we end off by closing out the XML tags.

The rest of the code in this method looks like this:

SOAPRequest req = new SOAPRequest(request.toString(), emailAddress);
EWSResponse retVal = service.makeRequest(req);
lastResponse = retVal;
id = retVal.getId();
changeKey = retVal.getChangeKey();
return retVal;

Now it's important to realize that all we have created is the part of the SOAP message that tells Exchange what we want it to do – create a calendar item. We haven't wrapped it in a SOAP message yet. Line 321 takes care of doing this by creating a SOAPRequest object based on the request string that we just created. Ignore the fact that we pass in an emailAddress at this point, because that will be used later for the Exchange Impersonation article.

If you want to go and explore the SOAPRequest class, feel free to do so. It simply creates the SOAP envelope and plugs the request information that we passed in into the body of the SOAP message. No rocket science in this.

The next step (line 322) is to call the makeRequest() method on the ExchangeService object that was passed in when we created the appointment.  We'll dig into this call in a minute, but let's just finish out understanding what the rest of this method is doing. 

The makeRequest() method call returns us an EWSResponse object that we can return to the caller. Before we do that, though, we set two very important pieces of information inside this Appointment object.

Items and Item IDs

When any item (e-mail, contact, task, or appointment) is created on an Exchange Server, it is allocated two pieces of information:

1. An Item ID. This is universally unique inside Exchange. No two items will ever have the same Item ID, even across different mailboxes, and this becomes the mechanism that we need to refer to the item later;
2. A ChangeKey. This piece of information is almost like a SHA1 or MD5 checksum that tells what the state was of the item at the point that you created it. This becomes very important later when we update these items.

You'll note that I have switched from referring to the things we are creating as Calendar items to simply calling them Items. That's because in Exchange, everything that can be created inside a folder (something we'll deal with later) is considered an Item. E-mail, Contacts, Tasks and Appointments are all items. This is another important concept to get clear, because if you look back at line 282, you'll note that the node that we created was a CreateItem node and that it contained a CalendarItem node inside an Items node.

Lines 324 and 325 set the values of the Item ID and ChangeKey for this appointment respectively. I also store the EWSResponse object in the Appointment in case it is needed later. I don't have code that uses this right now, though. Line 326 returns the EWSResponse object to the caller and lines 328 through 330 catch any exceptions and turn them into an EWSResponse object that can be serialized.

ExchangeService

The next thing to do is to take a look at the code in the ExchangeService class in the org,tsg.ews.service package. The guts of the communication with Exchange happens in the makeRequest method, which begins at line 57. You'll note that the first thing that makeRequest does is call "setAuthenticator" in line 58. The code in setAuthenticator is critical to our communication with Exchange, so scroll down to line 91 and we will pick up there.

setAuthenticator

The following is the code that is contained in setAuthenticator:

private void setAuthenticator() {
	System.setProperty("http.auth.preference", BASIC_AUTHENTICATION);
    	System.setProperty("http.auth.digest.validateServer", "false");
    	System.setProperty("http.auth.digest.validateProxy", "false");

	Authenticator.setDefault(new Authenticator() {
		@Override
		protected PasswordAuthentication getPasswordAuthentication() {
			StringBuffer user = new StringBuffer(domain);
			user.append("\\");
			user.append(username);
			System.out.println("Authentication...");
	   		System.out.print("Requesting Prompt: ");
			System.out.println(this.getRequestingPrompt());
	    		System.out.print("Requesting Scheme: ");
			System.out.println(this.getRequestingScheme());
	   		System.out.print("Requestor Type: ");
			System.out.println(this.getRequestorType());

			return new PasswordAuthentication(user.toString(), password.toCharArray());
		}

	});
}

This code has gone through a number of iterations and there are a lot of debug messages that I have put in here to try and figure out what I could do about the authentication piece of things. As I mentioned in part 1, this code currently only works with basic authentication switched on. The first System.setProperty call sets the authentication mode to basic authentication. The next two properties only apply with digest authentication and you can ignore them. 

The Authenticator.setDefault() call sets a default authenticator class that takes the user name and password that were supplied earlier, and makes them available for authentication purposes when the Exchange Web Service requests credentials.

The big problem with this code is that it sets these credentials for the entire JVM. This is a bad thing, especially in the Java EE environment, but there are solutions that I have already found to fix this problem that I will blog about later. For now, the purposes of what I am currently trying to achieve, this is good enough, and an important thing to remember is that I am looking at this as server-side code so Exchange Impersonation is a key component of the strategy going forward. In the impersonation model, the problem is not as serious as it is right now.

The reason this is such a serious risk right now is that you need to shut down and restart the Glassfish AppServer between changing the user for whom the appointment is being created. But I figured that as this is the only article that will use this model, we're OK with that limitation.

Making the call

Now we need to go back to line 60 in the makeRequest function:

URL ewsURL = new URL(String.format(EWS_URL, this.hostName));  // Substitute the host name into the URL.);
HttpsURLConnection ewsConn = (HttpsURLConnection) ewsURL.openConnection();
ewsConn.setRequestMethod("POST");
ewsConn.setRequestProperty("Content-type", "text/xml;utf-8");
ewsConn.setDoInput(true);
ewsConn.setDoOutput(true);

Line 60 sets up a URL for the Exchange Server. The String.format call substitutes the hostName provided into the EWS_URL constant so that the Exchange Web Service API is called. The constant is defined in line 25.

Update: Since I wrote this article, I have installed and tested Microsoft Exchange Server 2007 SP3 and Microsoft Exchange Server 2010. There is one minor modification you will need to make to the sourced code that accompanies this article for it to work with these. Line 63 of the source code file "ExchangeService.java" that accompanies this article reads:

ewsConn.setRequestProperty("Content-type", "text/xml;utf-8");

It may need to be changed to read:

ewsConn.setRequestProperty("Content-type", "text/xml;charset=utf-8");

Next we set up an https connection, set the request method to post, and several other properties of the connection. Line 67 contains a debug message that displays the entire XML for the SOAP message just prior to posting the call to the Exchange Web Server and lines 69 through 72 actually post the message to the Exchange Server:

PrintWriter pout = new PrintWriter(new OutputStreamWriter(ewsConn.getOutputStream(), "UTF-8"), true);
pout.print(message.getMessage());
pout.flush();
pout.close();

Receiving the response

All calls to Exchange are followed by a response:

EWSResponse resp;
if (ewsConn.getResponseCode() == HttpsURLConnection.HTTP_OK) {
        BufferedReader bin = new BufferedReader(new InputStreamReader(ewsConn.getInputStream()));

        StringBuilder result = new StringBuilder();
        String line;
        while( (line = bin.readLine()) != null )
        	result.append(line);
        System.out.println(result.toString());
        resp = new EWSResponse(result.toString());

In line 75, we check to see that the response we got from the Web Service was an OK – 200. If it is, line 76 instantiates a BufferReader to read the URL connection's input stream. Lines 78 through 81 read the contents of that stream into a string which we display in line 82 so we can see the response we got from Exchange.

Line 83 then takes that response and instantiates an EWSResponse object with the string we got back, and the EWSResponse object parses the SOAP message that we received from Exchange and stores the response in object. Among the things that is returned is a ChangeKey, an ID and the error status of the call. I'm not going to spend much time talking about EWSResponse here, other than to say that it parses the XML using the standard DOM in the JVM.

The remainder of the makeRequest() method deals with what happens if we don't get an OK http response:

} else {
	System.out.println (String.format("Error code: %1$s - %2$s", ewsConn.getResponseCode(), ewsConn.getResponseMessage()));
	resp = new EWSResponse("HTTPError", ewsConn.getResponseCode(), ewsConn.getResponseMessage());
}
return resp;

Line 85 provides the details of the response as a debug message and line 86 creates an EWSResponse object that provides details of the error. The most likely scenarios for this response are 401 errors (HTTP authentication failed) or 500 errors (the XML was not properly understood by Exchange and it reports an "Internal Server Error"). The 500 errors are a pain because there is no information available on why you got them. You have to guess, and it is this that made me hand-code the XML that is being sent to the Exchange Server.

Now that we have an EWSResponse object, it can be returned to the caller, serialized and returned from the WebService to whatever the source was.

Testing it out with soapUI

The next step is to try and test this. Make sure you have your Glassfish server running and that the EWSAPI code is deployed to the server. You can check this by switching to the Servers view in Eclipse for Java EE and looking at the status of the GlassFish V3 Java EE 6 server. Start the server if necessary and drag the whole EWSAPI node to the GlassFish server node and it will be deployed for you.

The next step is to start soapUI and create a new project. Right-click on the Projects node and create a new project. You should see the following dialog:

The URL that need to type in the "Initial WSDL/WADL" field is:

http://localhost:8080/EWSAPI/AppointmentService?wsdl

Once you have entered that URL, make sure the check boxes are as they appear in the screenshot above, and choose the OK button. SoapUI will generate a project for AppointmentService and it will have three operations under it. Go ahead and expand the tree so that all the nodes are visible. The three operations are CreateAppointment, GetAppointment, and DeleteAppointment. Each of them has a "Request 1" node under it. I have renamed those nodes to "CreateApptRequest", etc, so that when a request is open, I can differentiate it from the others.

Before we execute a call, open Outlook for the account that you are testing against so you can see what happens in Outlook when the appointment is created.

Double-click on the CreateAppointment request in soapUI and open the XML editor will open with the XML Soap message in the left-hand pane. Remember, there is no validation, so make sure you type all the right values between the tags in the XML document.

The first parameter for the call is the Exchange Server. You need to type the fully qualified domain name of the Exchange Web Services host that you are trying to call. Next is the domain. This is the domain that is used for the Windows login. Username and password are fairly self-explanatory, as are the Description, Location and Notes elements. 

SetReminder should be either "true" or "false". If "true", a reminder will be set for the appointment that will go off a certain number of minutes before the appointment. ReminderMinutes is an integer that specifies how many minutes lead-time the reminder should have.

StartDateTime and EndDateTime are exactly what they say they are, but they are ISO dates that include a time zone. The date is in yyyy-mm-dd order and is followed by a T to indicate that the rest of the field is all time. The time is a 24 hour time format, including a 3 digit decimal for the number of milliseconds. So 5:00pm is 17:00:00.000. The last section of the time is an offset from Universal Standard Time(UTC). I'm on the West Coast of the US, so in the Northern Hemisphere Summer, we are 7 hours behind UTC. Thus my timezone information is -07:00. If I were in South Africa, I'd be two hours ahead of UTC which would be +02:00.

Once you have all these fields filled out, you can execute the call. In the right-hand pane of the request, you should get a SOAP message back that tells you what happened. If all went well, there will be a single response node that contains three attributes – ChangeKey, ItemID and ErrorStatus. The ErrorStatus should be "NoError" and the other two attributes will have very long strings in them that mean nothing, but I've already explained what these are. 

If you get this, your appointment has been created in Exchange and if you have Outlook open in the background, a second or two later the appointment will show up in the calendar as you can see in the screenshot above.

If you are having trouble getting any kind of response back, go to Eclipse for Java EE, and switch to the Glassfish Server log in the Console view as follows:

You will need to select the down-arrow next to the console button in the toolbar in the Console view. Select the log file from the two console sources, and it will show the messages that were output as the code was running.

Now that we have successfully created the appointment, you need to keep the result of the call around, because you are going to need it in a few minutes when we run the GetAppointment call.

For the moment, though, let's switch back to the Eclipse for Java EE and look at the code for GetAppointment and DeleteAppointment.

GetAppointment

If we switch back to the EWSCalendar class, lines 45 through 56 contain the code for retrieving an appointment from Exchange:

@WebMethod(operationName = "GetAppointment")
@WebResult(name = "Appointment")
public Appointment getAppointment(@WebParam(name = "ExchangeServer") String hostName,
				@WebParam(name = "Domain") String domain,
				@WebParam(name = "User") String username,
				@WebParam(name = "Password") String passwd,
				@WebParam(name = "AppointmentID") String id) {
	ExchangeService srvc = new ExchangeService(hostName, domain, username, passwd);
	Appointment appt = new Appointment(srvc);
	appt.retrieve(id);
	return appt;
}

As I mentioned earlier, the first couple of lines are annotations needed for exposing this call as a Web Service called GetAppointment. Note that this method returns an object of type Appointment, the class we learned about earlier on. All this call does is instantiate an ExchangeService object, instantiate an Appointment object, retrieve it, and return it. The Java EE WebServices framework takes care of serializing the Appointment object to XML.

The retrieve() method takes an Item ID as a parameter so that it retrieves the calendar item by means of that Item ID.

Appointment.retrieve()

The retrieve() method in line 157 of the Appointment class is responsible for crafting the XML message that gets sent to the Exchange Server:

public EWSResponse retrieve(String itemID) {
	try {
		StringBuffer request = new StringBuffer();
	        request.append("<GetItem \n");
	        request.append(String.format("               xmlns=\"%1$s\"\n", SOAPRequest.MESSAGES_URI));
	        request.append(String.format("               xmlns:t=\"%1$s\">\n", SOAPRequest.TYPES_URI));
	        request.append("    <ItemShape>\n");
	        request.append("          <t:BaseShape>Default</t:BaseShape>\n");
	        request.append("          <t:AdditionalProperties>\n");
	        request.append("              <t:FieldURI FieldURI=\"item:Body\"/>\n");
	        request.append("              <t:FieldURI FieldURI=\"item:ReminderIsSet\"/>\n");
	        request.append("              <t:FieldURI FieldURI=\"item:ReminderMinutesBeforeStart\"/>\n");
	        request.append("          </t:AdditionalProperties>\n");
	        request.append("    </ItemShape>\n");
	        request.append("    <ItemIds>\n");
	        request.append(String.format("        <t:ItemId Id=\"%1$s\"/>\n",itemID));
	        request.append("    </ItemIds>\n");
	        request.append("</GetItem>\n");
		SOAPRequest req = new SOAPRequest(request.toString(), emailAddress);
		EWSResponse retVal = service.makeRequest(req);
		if ((!retVal.hasError()) && retVal.getElement() != null) {
			parseItem(retVal.getElement());
		}
		lastResponse = retVal;
		return retVal;
	}
	catch (Exception e) {
		return new EWSResponse(e);
	}
}

The first thing to note about this code is that there is nothing in it that indicates that the object that we are retrieving is a Calender Item. As far as Exchange is concerned, we are retrieving an item, and it does not care what type of item it is. So the call we make is a GetItem call.

Lines 163 through 170 define an ItemShape object. The ItemShape tells Exchange what we are interested in retrieving about this item. The BaseShape node tells it to retrieve the most basic properties of the item, but we need additional data like the Body (or notes), the Reminder flag and the number of minutes in advance that the reminder is required. So the AdditionalProperties node asks Exchange to retrieve these properties, too.

Lines 171 through 173 tell Exchange what item IDs to return – in our case there is always only one.

In line 175 we create the SOAP envelope for the call and in line 176 we call makeRequest to retrieve the data from Exchange. Assuming all went well with the call, we set the values of all the fields in the Appointment by calling parseItem() in line 178. I'm not going to spend a lot of time explaining the XML parsing. Just take it from me that the code parses the XML node for the appointment and gets the right values out.

Of course, if the call failed, we need the client to know that, so we store the EWSResponse object in the appointment. The advantage of that is that the EWSResponse object is XML-serializable so it will be serialized with the Appointment object. Another nice thing about JAXB is that it will ignore null elements (unless you tell it not to). What this means is that if the Appointment object's properties are null, they will not be written to the XML response.

That's pretty much it. Now let's see what happens when we run it.

Testing GetAppointment with soapUI

Before you switch back to soapUI, go to the appointment that we created earlier, and make some changes to it in Outlook. Add some stuff in the notes and change the date and time or something. Now switch back to soapUI and highlight and copy (Ctrl+C) the Item ID that is in the Response node in the right side pane of the CreateApptRequest window we opened earlier. 

Now open the GetAppointment request and set all the values for the credentials as you did earlier:

The Appointment ID node needs to contain the Item ID that you copied from the CreateApptRequest above. Once you have pasted that in and executed the call, you should get a response that contains all the information about the appointment that we requested. Note that the changes that you made are retrieved with the call. I should mention that sometimes you have to wait a few seconds for Exchange to commit your changes and notify the Client Access Server before your changes become available for retrieval by Exchange Web Services.

DeleteAppointment

Last, but not least is DeleteAppointment. The code for this is actually very simple. If we go back to Eclipse for Java EE and look at EWSCalendar again, lines 58 through 67 contain the code that does the work:

@WebMethod(operationName = "DeleteAppointment")
@WebResult(name = "Response")
public EWSResponse deleteAppointment(@WebParam(name = "ExchangeServer") String hostName,
				@WebParam(name = "Domain") String domain,
				@WebParam(name = "User") String username,
				@WebParam(name = "Password") String passwd,
				@WebParam(name = "AppointmentID") String id) {
	ExchangeService srvc = new ExchangeService(hostName, domain, username, passwd);
	EWSResponse resp = Appointment.delete(srvc, id, null);
	return resp;
}

By now, hopefully you understand everything down to line 66. In line 66, we call Appointment.delete() and pass in the service and an Item ID. When the call is complete, we simply return the EWSResponse object that should give us error status.

Appointment.delete()

The guts of the work happens in the Appointment object, so switch to that editor and go to line 26:

public static EWSResponse delete(ExchangeService srvc, String itemID, String emailAddress) {
	try {
		StringBuffer request = new StringBuffer();
	        request.append("<DeleteItem \n");
	        request.append(String.format("               xmlns=\"%1$s\"\n", SOAPRequest.MESSAGES_URI));
	        request.append(String.format("               xmlns:t=\"%1$s\"\n", SOAPRequest.TYPES_URI));
	        request.append("           DeleteType=\"MoveToDeletedItems\"\n");
	        request.append("           SendMeetingCancellations=\"SendToNone\">\n");
	        request.append("    <ItemIds>\n");
	        request.append(String.format("        <t:ItemId Id=\"%1$s\"/>\n",itemID));
	        request.append("    </ItemIds>\n");
	        request.append("</DeleteItem>\n");
		SOAPRequest req = new SOAPRequest(request.toString(), emailAddress);
		EWSResponse retVal = srvc.makeRequest(req);
		return retVal;
	}
	catch (Exception e) {
		return new EWSResponse(e);
	}
}

Again, hopefully this code looks very similar to the GetItem call we did earlier. The DeleteItem node has two interesting attributes in lines 32 and 33. You'll see the DeleteType attribute which tells Exchange to move the deleted appointment to the Deleted Items folder. You can specify exactly what to do with the item, including a hard delete – permanent deletion.

In line 33, the SendMeetingCancellations attribute tells Exchange what to do about notifying people about the cancellation of the appointment.

Line 34 through 36 contains the list of Item IDs to be deleted (in our case, just one). Lines 38 through 40 create the SOAP envelope, make the call to Exchange and obtain and return the EWSResponse object. That's all there is to it.

Testing DeleteAppointment with soapUI

The last step in the Java tests is to see if the DeleteAppointment API works. Make sure you have the Item ID for the appointment you created earlier, and open the DeleteAppointment request. As with the GetAppointment API, paste the Item ID into the Appointment ID node:

If all goes well, the response will be contained in a Response node and the ErrorStatus will be NoError. If you switch to Outlook, the appointment has been removed and if you switch to the Deleted Items folder, you will see the Calendar Item in that folder.

Java Summary

That's all there is to the Java side of things for this article. You have now learned how to use Java to create, get and delete calendar items using the Exchange Web Services API. The next article will focus on Exchange Impersonation.

If you're a Java developer, and you have no interest in the OpenEdge component, thank you for your attention. I hope this was useful to you.

If you are an OpenEdge developer, don't go anywhere – we're going to see how to call this Web Service from inside OpenEdge. 

The OE EWS Components

If you have been following the code up until now, you have a running instance of Glassfish and it's ready for you to test the OpenEdge code that calls these three Web Service operations. Go ahead and open OpenEdge Architect and make sure you have imported the OEEWS project into OpenEdge Architect.

Once you have imported it, expand the src folder all the way down until your Resources are expanded as in the screenshot below:

Just so there is no confusion, my screenshots were taken of the code as I have it checked into my Subversion repository, so you can ignore the date, time and user name next to each source code element. That is information that Subversion adds.

The first code that we are going to look at is in the CreateAppointments.p procedure as this code creates a batch of 10 appointments, so go ahead and open this procedure in OE Architect.

CreateAppointments.p

There really is no rocket science about this code. For the most part, it is ordinary OpenEdge ABL/4GL code. Lines 14 and 15 of the source file do contain two statements that are specific to object-oriented ABL:

USING Progress.Lang.*.
USING com.tsg.ews.*.

These two lines tell the compiler that my code references objects in the Progress.Lang package and in the com.tsg.ews package. Don't fret about this just yet.

Line 20 references a temp-table definition that is in an include file while lines 21 through 24 define 3 input parameters and an output parameter table for the temp-table defined in the include file.

The rest of the code down to line 39 is pretty standard 4GL, but in line 39, I define a variable called "appt" that is of type com.tsg.ews.Appointment. If you cast your eyes across to the Resources view, you will see a file called Appointment.cls under the src/com/tsg/ews folder. Appointment.cls contains an OpenEdge class definition, and we will look at this code a little later. The Using statement in line 15 told Progress to look in the com/tsg/ews folder for any classes that I refer to.

The interesting code starts in line 43 (I have removed comments below for brevity):

REPEAT iCount = 1 TO 10:

    appt = new Appointment().
    appt:Subject = pcDescription + " - " + STRING(iCount, "99").
    appt:Location = pcLocation.
    appt:Notes = pcNotes.
    appt:StartDateTime = figureOutStartDate(dWorkDate).
    appt:EndDateTime = DATETIME-TZ(DATE(appt:StartDateTime), MTIME(appt:StartDateTime) + iDuration, TIMEZONE ).
    dWorkDate = appt:EndDateTime.

    appt:SaveAppointment().        

    CREATE ttAppointment.
    IF appt:ItemID <> ? AND
       appt:ItemID <> "" THEN
       cString = appt:ItemID.
    ELSE
       cString = STRING(iCount).  

    ASSIGN
        ttAppointment.description = appt:Subject
        ttAppointment.startdate = DATETIME(DATE(appt:StartDateTime), MTIME(appt:StartDateTime)).
        ttAppointment.enddate = DATETIME(DATE(appt:EndDateTime), MTIME(appt:EndDateTime)).
        ttAppointment.itemid = cString.
    .

END.

This repeat loop occurs 10 times. In line 46, I create a new instance of an Appointment object and I assign the reference to the appt variable. What this means is that I have an empty Appointment object. In lines 21- 23 I defined 3 input parameters for the appointment description, location and notes. In 47, 48 and 49, I set the Appointment's corresponding properties from these input parameters.

In line 50, I set the start date and time for the appointment to the return value of the figureOutStartDate function. I'm not going to walk through the code for the function here, but I'll tell you that it goes and calculates a date on a work day (Monday through Friday) between 9 and 5, skipping an hour for lunch at 12:00pm. It also sets a variable that is global to this procedure called iDuration. This contains the length of the appointment in milliseconds. In line 51, I set the end date and time to the start date and time plus the duration.

The upshot is that I will get 10 appointments that follow each other of random duration and with random gaps between them starting from now, until all 10 appointments are scheduled.

Line 57 calls SaveAppointment and this method on the Appointment object is responsible for calling the Web Service. We'll get to this code in a minute or two. When the appointment is successfully saved, we set the ItemID property of the Appointment object to the return value from Exchange.

Lines 60 through 72 create a temp-table record for this appointment and store the subject, start date, end date and item id. This is just regular 4GL code. 

Appointment.cls

Next, we'll take a look at the code in Appointment.cls. Go ahead an open it in the editor.

Appointment.cls is an OpenEdge ABL class that uses OpenEdge Object-Orientation. Lines 16 through 20 declare the class and define some variables. Lines 22 through 24 define the default constructor for the Appointment object. This is the code that is called when an object is instantiated as it was in line 46 of CreateAppointment.p. Line 26 through 30 declares a constructor that we will use later for GetAppointment. 

Lines 32 through 68 declare several properties for the Appointment object that we used to set the appointment values in lines 47 through 51 of CreateAppointments.p above.

Skip on down to line 87, and we will take a look at the code that actually saves the appointment:

METHOD PUBLIC LOGICAL SaveAppointment():
    DEFINE VARIABLE srvc AS EWSService NO-UNDO.
    DEFINE VARIABLE hHandle AS HANDLE NO-UNDO.
    DEFINE VARIABLE cRetVal AS LONGCHAR NO-UNDO.
    DEFINE VARIABLE cNotes AS LONGCHAR NO-UNDO.
    cNotes = "<![CDATA[" + THIS-OBJECT:Notes + "]]>".
    srvc = EWSService:GetService().
    hHandle = srvc:CalenderAPI.
    RUN CreateAppointment IN hHandle
       (INPUT srvc:ExchangeServer,
        INPUT srvc:Domain,
        INPUT srvc:UserName,
        INPUT srvc:Password,
        INPUT THIS-OBJECT:Subject,
        INPUT THIS-OBJECT:Location,
        INPUT THIS-OBJECT:StartDateTime,
        INPUT THIS-OBJECT:EndDateTime,
        INPUT cNotes,
        INPUT THIS-OBJECT:ShouldSetReminder,
        INPUT THIS-OBJECT:ReminderMinutes,
        OUTPUT cRetVal).
    parseResponse(cRetVal).
    RETURN TRUE.
END.

The first few lines (88-91) define variables that we will use later. In line 92, we wrap the Notes field in a CDATA node so that we can handle any data that can be put in it. It could contain HTML, XML or any other kind of data. 

Line 93 calls the GetService method on the EWSService object. EWSService is another class that I created and all it does is wrap the processing around the connection information for the Java and Exchange Web Services. It also takes care of establishing the connection to the Java Web Service. In line 94, we get the handle to that Web Service connection and in lines 96 through 107, we make the call to the Web Service and receive the response in the cRetVal variable.

Line 108 takes care of parsing the response so that the ItemID is stored into the appointment object.

That is all there is to it. Now let's take a look at how we call this code. 

AppointmentViewer.cls

Go to the Resources view and right-click on the AppointmentViewer.cls file. From the context menu, select "Open with…->OpenEdge Visual Designer." Your screen should look like this:

The AppointmentViewer class is a .NET style UI with OpenEdge code behind it. Double-click on the "Generate Appointments" button and you will be taken to a section of code in this form that calls the CreateAppointments.p procedure we looked at earlier. The code is in line 81 through 92 of the editor that is now open:

METHOD PRIVATE VOID btnGenerate_Click( INPUT sender AS System.Object, INPUT e AS System.EventArgs ):
    CLOSE QUERY qAppointment.
    RUN CreateAppointments.p (INPUT txtSubject:Text, INPUT txtLocation:Text, INPUT txtNotes:Text, OUTPUT TABLE ttAppointment).
    OPEN QUERY qAppointment
        FOR EACH ttAppointment NO-LOCK
            BY ttAppointment.startdate.
    THIS-OBJECT:bindingSource1:Handle = QUERY qAppointment:HANDLE.
    bindingSource1:RefreshAll().
    dataGridView1:Refresh().
    RETURN.
END METHOD.

This code closes the qAppointment quert, which is a query based on the ttAppointment temp-table defined in the ttappointment.i include file. The code then calls CreateAppointments.p which returns the ttAppointment table. The input parameters for this call are derived from the Subject, Location and Notes text boxes that are at the top of the Appointments tab in the AppointmentViewer window. Once the call is complete, we reopen the query, refresh the binding source and data grid so that the data is displayed on the screen.

Running the Appointment Viewer

Now it's time to run the Appointment Viewer. From the OpenEdge Architect Run menu, choose Run->Run as…->OpenEdge Application. If you are asked to save the file, you can go ahead and do so. When you run the code, you should see the screen at left.

Enter the name of your Java Server and make sure you include the port number, which, by default, will be 8080. For these initial runs, you can make the call using localhost for the name of the server.

The other prompts in the top half of the screen are the same as those we referred to in the calls that we made from soapUI above.

Before you generate the appointments, you should make sure that Outlook is open in the background so you can see the effect of the call.

Once you enter the password, the Generate Appointments button is enabled. Go ahead and enter a Subject, Location and some notes and then choose the Generate Appointments button.

If things are working properly, you will see a slight pause and the grid at the bottom of the screen will populate with a list of appointments that were automatically generated for you by the CreateAppointments.p procedure. 

Your populated screen should look something like the following screen. It is possible that you will see a series of numbers in the Item ID column. If you do, there was a problem creating the appointments on the server. If the appointments were created properly, you will see the ItemID populated with very long strings of characters as below:

In Outlook, you should now see a set of 10 appointments that were created that correspond with the dates and times in the data grid in the Appointment Viewer. Select a row by clicking on the gray area to the left of the data grid. When you have selected a row, the entire row will be highlighted an the Get Appointment button will become active. Choose the Get Appointment button and your screen should look like this:

You will note that the data in the AppointmentView dialog that pops up includes data that we do not have in the temp-table. That's because we fetched the appointment from the Exchange Server, not from the local cache. If you compare the appointment with the appointment properties for the corresponding appointment in Outlook, you will see that they match, so go ahead and edit the contents of the appointment in Outlook – something like this:

Now save and close the appointment in Outlook and give yourself a good 10 to 15 seconds so that Exchange has a chance to let the Client Access Server know about the change, then close the AppointmentView dialog by choosing the OK button, and open it again by choosing the Get Appointment button. The changes that you made in Exchange will now show up in the Appointment View dialog:

Finally, choose the Delete button in the AppointmentView dialog, and the appointment will be removed from the calendar:

If you go and look at the Deleted Items folder in Outlook, you will find the deleted appointment there.

Get Appointment and Delete Appointment

Close down the AppoinmentViewer and go back to OpenEdge Architect. If you go to line 100 of the code for the AppointmentViewer.cls class, you will find the code that fetches the appointment from the server. The code determines which line was highlighted in the data grid and obtains the ItemID from the data grid. It then uses that ItemID to create a new appointment object and the Appointment class calls GetAppointment in the constructor. GetAppointment, which is in line 70 of Appointment.cls simply calls the Java Web Service to get the information.

When it is done, the AppointmentView dialog is displayed with the contents of the Appointment object.

If you right-click on the AppointmentView (not AppointmentViewer) class in the Resources view and you choose "Open With…->OpenEdge Visual Designer", you will get the AppointmentView dialog opened in OpenEdge Architect:

If you now double-click the Delete button, you will see the code that calls DeleteAppointment on the Appointment class. In the Appointment class, this code simply calls the DeleteAppointment operation on the Java Web Service.

Summary

That's it. The code that does all of this is fairly simple and straight-forward, once you understand what it is doing. Most of the complexity associated with this code is really around the communication between Java and the Exchange Server. The OpenEdge connectivity is reasonably simple because the Java code handles the complex work.

This is just the beginning of a prototype that we will expand on over the next few articles. I hope this has been helpful and interesting. Feel free to dig into the code that is supplied. 

In the next article, we will dive into Exchange Impersonation.

Actually, now is a really good time to have this conversation because with the work I am doing on the Exchange Web Service code, I have just finished revamping my internal infrastructure to support the equipment and software I need to do the job. So this is going to be a two-part article. In this part, I'll tell you about the software development components that I use. In the next part, I'll tell you about the infrastructure components. The problem is that you need to understand some of the details of why, so I'm going to start with a little background.

Background

There are a few basic facts about me that effect everything else:

1. I am not rich. I have to rely on the most cost-efficient solutions I can. That means that I prefer open source/free software where it makes sense.
2. I am not cheap. I recognize that building software that is commercial grade requires a serious commitment to quality. That means a serious commitment to the development process, which, in turn, requires things like solid designs, automated builds and testing, and extremely good discipline.
3. I am the worst perfectionist. A gentleman I used to work for back in South Africa, Jan Fernhout, used to tell me that I would way over-engineer things because I wanted a perfect design. That is true. I have to be very disciplined about remembering that I need to deliver something.
4. I hate mundane work. If at all possible, someone else is going to do the work for me, and I there is no way that I am going to manually do something that can be completely automated. If you make me do it, I will bleat like a sheep until you fix the problem – just ask some of my co-workers.

There are also a few facts about the development work that I do that affect everything, too:

1. I believe that Windows is for the desktop or as a departmental product and Unix is what the enterprise needs. No one has proven me wrong on this. For every example of a large Windows product installation, I can point you at hundreds of more successful, more cost-efficient, more stable, less administered Unix installations that just run.
2. I do a lot of work with both Unix and Windows.
3. Java is the most platform-independent language for back-end work.
4. .NET is the richest GUI experience you can have outside the browser on the Windows platform.
5. OpenEdge is a fact of my life. I have used it more than any other development platform. I have made a good living off it, and I love the product, warts (many of them) and all. OpenEdge is kinda like an old pair of shoes that fits your feet a certain way and you're always most comfortable wearing them, no matter how ugly they are.

Machines and Operating Systems

My desktop machine – and primary development workstation – is a Windows 7-based HP Pavilion Entertainment 64-bit laptop with 6GB of RAM and 450GB of disk space. It runs nothing but Windows, and I have a ton of software on it.

My primary server machine is a  Dell PowerEdge T710 with 2 4-Core Intel Xeon processors, 16GB of RAM and 2 500GB drives. It runs VMWare ESXi (see screenshot at right) with 20 different virtual machines on it, some constantly running and some not. I'll tell you more about this in the next post on infrastructure. There are 6 VMs, though, that are important to this discussion:

• A Windows Server 2008R2 primary domain controller (pdc);
• A Windows Server 2008R2 build server (winbuild);
• A Windows 7 32-bit build machine (clientbuild);
• A CentOS-64 development server that hosts a whole lot of stuff (develop);
• A CentOS-64 build server (build64); and
• A CentOS-32 build server (build32) .

I then have 2 other machines that are both older machines that I use for various things. I'll talk about these in my infrastructure post, too.

Components of a Development Environment

As I said, I'm the worst perfectionist. So development starts with an idea, and ends when the code has been packaged, deployed, and installed on the end-user's machine, and they have completed user acceptance testing. In other words, it not only spans the entire Enterprise Architecture stack, but it also spans the Software Development Life-Cycle (SDLC). 

Now as much as I said I am not rich, there are two very expensive tool-sets that I have to buy:

1. Visual Studio Premium with MSDN. This one sticks in my throat every time I pay for it. The only reason I need the Premium subscription is for the Windows Server and Microsoft Exchange Server licenses. The more than $5500 that this product costs is very hard to justify, and the $2300 a year charge for renewing the subscription is as bad as paying tax to the IRS. I am a firm believer that development tools should be free. If you want me to develop products for your platform, give me the tools I need to do it.
2. PSDN Professional. Although I also believe that Progress should provide its tools for free, I am less resentful about it, probably because I used to work for the company. I still think the $3,000 that we pay for PSDN is excessive, but there you have it. My guess is there would be more people developing Progress-based applications if the development tools were given away free.

I need the MSDN license for the operating system and server licenses that I am using for Exchange Server. For my non-Microsoft-based development, I rely heavily on the Java and LAMP (Linux-Apache-MySQL-PHP) stack. Because I have some experience with PHP, I try as far as possible to use Open Source tools that are based on PHP rather than Perl. I don't know Perl and I don't particularly want to spend the time on it.

So starting from concept, here are the tools that I use to get my stuff done:

•  Sparx System's Enterprise Architect (EA) for all my modeling (not just diagrams, models). The product cost me about $300 for the original license, which included the MDG technology for Eclipse integration, and I use it as a design tool from the conception of an idea all the way through delivery. It contains the business process models, the requirements, and all the UML diagrams and test cases. I am not a fan of model-to-code driven development. I think the fact that we have been looking at the equivalent of CASE tools for the last 20 years shows that there will always be those cases where the code cannot be automatically generated for you. Two of EA's strongest suites are the traceability it provides from requirements to test case, and the incredibly powerful document generation facility. I literally generate all my documents from Enterprise Architect.
• Collabnet Subversion is used as my Source Code Control System. Everything goes into Subversion, including my models, source code, document artifacts, and even my research code. Subversion runs on my development server (develop) on CentOS. Subversion has clients for all kinds of platforms, so I have AnkhSVN plugged into Visual Studio to control my .NET code, I have Subclipse plugged into Eclipse for all my Java code, and I use TortoiseSVN at the operating system level to control documents and stuff that are outside of either development environment.
• Mantis is used as my issue tracking system. There is an Eclipse plugin for Mantis so I can get at it directly from my project workspaces in Eclipse.
• Hudson CI is used as my automated build and test environment. It runs on develop, but starts builds on build64, build32, winbuild, and clientbuild. I use both Apache Ant and Apache Maven to build Java code, and I use PCT to build OpenEdge code in Hudson. I use Nant to build .NET code.
• Unit testing is the scope of JUnit and NUnit although I am vehemently opposed to test-driven development. I think it is flawed and creates really lousy software.
• Eclipse is my Integrated Development Environment of choice. I have at least two installations of Eclipse at any one time. One is for non-Enterprise development (stuff that does not require Java EE) and the other is for Enterprise development (Java EE). The Eclipse screenshot at the top of this post shows my working environment for non-JavaEE work. From the screenshot, you will notice that I have OpenEdge running inside the Eclipse environmnent (Database Explorer is at the bottom, and there is an OpenEdge Editor open above it. You can also see an Enterprise Architect model in the right-hand pane, and my Hudson build status is in the left-hand pane. In the Eclipse screenshot at the bottom of this post that shows my working environment for JavaEE work, you will see the Mylyn task list at the bottom on the left, with the Mantis task repository open in the pane on the bottom right and a Mantis task open on the right. All of the tools that I have highlighted are integrated right into my Eclipse IDE in both environments. 
• Visual Studio is used only for development work that doesn't work in Eclipse. So most .NET/Windows-based development is done in Visual Studio. I have both Visual Studio 2008 and 2010 installed. 

All the tools I have discussed so far have plugins for Eclipse and, as the screenshots show, that is the configuration in which I use them.

A couple of other products that I use extensively that are not really integrated into the development environment per se, are:

• Beyond Compare is one of those tools that I just cannot imagine not having. It is a tool for doing advanced 3-way compares of files and it is, without reservation, the best diff tool that I know. It runs on both Windows and Linux, and I use it on both.
• WinZip is another tool that I use so much, I cannot imagine being without it. It is used for creating zip archives, but it can just as easily read and understand Linux tarballs and .jar and .war files.
• Nero is an outstanding tool for burning media, especially CD or DVD burning.

Most of my publishing stuff is done with either WordPress or Drupal. Like most of the other stuff I use that is web-based, these two are both PHP-based solutions. PHP is not a hard language to learn and it is very good at getting pretty sophisticated web apps up and running relatively quickly. WordPress and Drupal both have extension APIs that make it possible to build pretty sophisticated plugins for both in PHP – something I have done reasonably effectively.

Conclusion

Most of the tools I have mentioned above are freeware or open source or really, really cost-effective, and the good thing about them is that they lead you to build much better quality code than would otherwise be the case. If you are not looking at tools like these, I would highly recommend giving them a try.

In my first post, I told you about the background story – I need to enable an OpenEdge CRM application to create, modify and delete calendar and task items in Microsoft Exchange. I also need Exchange to let me know any time a calendar or task item is changed so that I can update the OpenEdge database accordingly. Simple use cases.

When I left off last week, my next step was to get Exchange subscriptions working, and, boy, what a trip that has been.

Types of Subscriptions

There are two types of subscription models available with Exchange:

1. Pull Subscriptions – You set up a subscription with Exchange and it gives you an indicator (called a Watermark) that keeps track of what events you have already received. You pole the server at an interval and it gives you a response with all events that have happened since that last Watermark.
2. Push Subscriptions – You set up a subscription with Exchange and tell it to call you back on another WebService every time something happens. The WebService has to acknowledge receipt of the message in order to keep the subscription alive. The message you get contains a list of items that have been affected since the last time a message was acknowledged.

I'm not interested in polling the server so I have chosen to go with a Push Subscription. Much of the information that follows applies to both subscriptions, but the Push Subscription differs in that you do not poll the server for the changes. 

Setting up a Push Subscription

To set up a Push Subscription, you send a SOAP message to EWS, authenticating against the Active Directory as you would with any other message to EWS. The message contains a list of folders that you want to know about, the list of event types that you want to be notified of, the URL that you should be called back on, and a StatusFrequency node that tells it how often to poll the URL – more about this later. 

You can subscribe to virtually any update on virtually any folder in a mailbox, including the calendar, tasks, contacts, and mail folders.

The diagram to the left shows the architecture that I finally ended up with. It is a little more complicated than it would be if OpenEdge were not in the picture, but it serves to explain the process, nonetheless. The numbers in parenthesis below refer to the numbers in the diagram.

OpenEdge initiates the subscription by sending a message (1.1) to a Java WebService that is deployed on a Glassfish Application Server. The WebService forwards that subscription request (1.2) on to the IIS server that is running on the Windows 2008 Server that hosts the Microsoft Exchange Client Access Server (CAS). IIS passes the message onto the EWS API which registers the subscription with the CAS. CAS responds with a message that contains the Subscription ID and the Watermark which is tied to the last event that was sent.

As I mentioned, the message that is sent to the EWS API (1.2) contains the URL for the service that is to receive all event notifications. It also contains the StatusFrequency node.

I set up a servlet on my Glassfish Application Server to receive and respond to the notifications. The following is a sample subscription request message:

<?xml version="1.0" encoding="utf-8"?>
    <soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xmlns:xsd="http://www.w3.org/2001/XMLSchema"
           xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
           xmlns="http://schemas.microsoft.com/exchange/services/2006/messages"
           xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
        <soap:Body>
            <Subscribe xmlns="http://schemas.microsoft.com/exchange/services/2006/messages"
                    xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
                <PushSubscriptionRequest>
                    <t:FolderIds>
                        <t:DistinguishedFolderId Id="calendar"/>
                    </t:FolderIds>
                    <t:EventTypes>
                         <t:EventType>CreatedEvent</t:EventType>
                         <t:EventType>ModifiedEvent</t:EventType>
                         <t:EventType>DeletedEvent</t:EventType>
                    </t:EventTypes>
                    <t:StatusFrequency>3</t:StatusFrequency>
                    <t:URL>http://server.example.com/ewsCalTest/NotificationService</t:URL>
                </PushSubscriptionRequest>
            </Subscribe>
        </soap:Body>
    </soap:Envelope>

The StatusFrequency Node

The StatusFrequency Node tells EWS to send a message every x minutes to verify that the subscription is still active. This serves three purposes:

1. It allows Exchange to figure out if the server that is listening is still around and cancel any dead subscriptions if necessary;
2. It allows the server that is being called to determine that it is still getting messages from the Exchange Server; and
3. It allows the server that is being called to unsubscribe from any future messages.

Exchange needs to carefully manage resources, so canceling any unnecessary subscriptions is an important way to do that. To maintain a subscription, the server that is receiving the notifications has to respond with a SOAP message that includes a SubscriptionStatus node with the value "OK". 

The expected response is as follows:

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope">
  <soap:Body>
    <SendNotificationResult xmlns="http://schemas.microsoft.com/exchange/services/2006/messages">
      <SubscriptionStatus>OK</SubscriptionStatus>
    </SendNotificationResult>
  </soap:Body>
</soap:Envelope>

If a subscriber does not respond appropriately, EWS goes into retry mode. In other words, if EWS does not get a SOAP message formatted exactly the way it expects, it assumes the server has not received the message. It waits for 30 seconds and tries again. If it still receives no response, it waits for a minute and tries again. EWS will continue retrying, doubling the wait time, until the wait time exceeds the StatusFrequency. So in my case, when I first set this up, I set the StatusFrequency to 3 minutes. It tried to send the notification and waited 30, then 60, then 120 seconds and canceled the subscription because, if it had gone to 240 seconds, it would have exceeded the 3 minutes set as the StatusFrequency. 

Less obvious, though, is the reason behind purposes 2 and 3. Remember that Exchange is only going to send messages when something changes. Well, some users have very quiet mail boxes. They may be out of the office, or it may simply be after hours where nothing is going on. If Exchange does not let the other server know that it is still sending messages, how is that server to know that the subscription is still valid?

The other thing that is important is that there is no way to send a message to Exchange to tell it to cancel a Push Subscription. The only way to cancel the subscription is by responding to a status message with the SubscriptionStatus set to "Unsubscribe".

Receiving Notifications

Now that you understand how to subscribe, the next step is to start receiving messages from EWS. We'll pick up where we left off in the diagram above.

EWS starts off by sending a notification (2.1) to the server. The notification could be one of two things:

1. A notification of events that have taken place on the server; or
2. A status message.

If this is a status message, all you need to do is respond with the OK response (2.2) I discussed above. In fact, you need to respond with the OK message if you want to carry on receiving notifications no matter what, but responding with an OK has an additional effect if you receive a notification of events.

First let's understand a notification of events. An event notification message contains a list of events that have taken place against the items that you are monitoring. It does not contain the details of what changed – just the fact that something did. If you want to know what changed, you have to call the GetItem method of the EWS to get the item and look at its changes. That's what we do in step 2.3.

Each notification includes a PreviousWatermark for the whole message and a Watermark for each event. By keeping track of the watermarks, it is possible to restart the subscription from the last event that was processed. When you acknowledge the event notification with an OK, Exchange moves the subscription to the next watermark automatically.

Once I have processed the event notification and received the event, I am making a Dynamic OpenClient call (2.4) to the OpenEdge AppServer to notify the AppServer of the change. The ABL code on the AppServer can then do what it needs to do to respond to the event.

Performance

Once I got this working, I decided to test it out to see what the performance looked like. Now bear in mind, all of the machines shown in the diagram are different virtual machines on one Dell Poweredge T710 running VMWare 4.0 ESXi. The T710 has 2×4 core processors and 16GB of memory. It has 4 network cards connected to a 1GB switch. The Windows Server is on a separate network card to the other three machines, which are all Centos 64-bit boxes. 

The round trip for establishing the subscription takes an average of 82ms.

I then ran a test harness that generates 100,000 calendar items across 80 mailboxes. It took a little over 2 hours for the test harness to complete. I tracked performance for each of the following pieces:

• The OpenEdge client takes an average of 100 milliseconds to make the call through the Java WebService to EWS to create the appointment.
• From the time I receive the acknowledgement for the appointment creation to the time that the notification has been received (2.1) and acknowledged (2.2) and the details of the notification have been received (2.3) takes an average of 150 milliseconds.
• The call across the Dynamic OpenClient to the AppServer, which is a call to a procedure that has no temp-tables or XML documents, averages 80 milliseconds.

All told, this means that from the time the request for the appointment is initiated to the time the notification of its update is received by the AppServer takes an average of 330 milliseconds – less than half a second.

Now clearly, my environment has very little traffic and I have a lot of horsepower backing it, but it if this is properly tuned and the additional processing that is to be added is not excessive, a one second response time is well within the realms of possibility.

Problems and Challenges

In getting this to work, I ran into a couple of nasty obstacles. 

First, Microsoft's API is really not a good example of a WebServices API. Neither Java nor OpenEdge will parse the WSDLs without manual editing and even if you manage to generate the code you need from the WSDL, it really doesn't work very well. I have been forced to manually code the interaction with the EWS API and that is a lot of work. Having said that, it performs very well and I get to write the XML exactly the way it needs to be for EWS to accept it.

That brings me to the second thing about this API. Microsoft's error reporting is shockingly bad for commercial software. Compared to the kind of error reporting I get from Tomcat, Glassfish, ActiveMQ, and several other Open Source solutions, it's incredible that commercial software can be this badly engineered. It is so hard to figure out what is wrong when something goes bad that you will spend hours on things like malformed or unrecognized XML. EWS simply returns a 500 http error code (Internal Server Error) with absolutely no indication of what caused it, and there is nothing in any of the logs to help you.

The third thing that really cost me a lot of time is that you cannot call a regular Java or OpenEdge WebService from EWS because it won't look at the WSDL. I was therefore forced to manually code the Java servlet that accepts an http post.

These problems effectively sidelined OpenEdge as a direct endpoint for EWS notifications. The architecture that I am building will definitely rely on all communication with EWS going via Java services.

Where to next?

Now that I have the prototype code working, my next step is to fix up some of the security issues I need to resolve. NTLMv2 authentication remains a real problem for Java in communicating with EWS, and I am looking for a solution around that. I also need to deal with some of the certificate related issues that I have.

I have also done some work on the Exchange Impersonation stuff, so my next blog post will probably talk about this.

Finally, I'm planning a walkthrough of some example code that actually demonstrates the functionality, like I did with the Dynamic OpenClient code late last year. 

A couple of months back, a gentleman who has now become a friend and business partner, Jim Ford of Ford Audio-Visual, came to me and asked me if there was any way to get at all the calendar items in his sales organization's calendars with the intention of integrating it with his Progress OpenEdge CRM system. Jim is using Exchange 2007 for his e-mail and calendaring solutions.

I was aware that Microsoft had released a new API for Exchange in Exchange 2007 called Exchange Web Services (EWS), and so I said that I needed to do a little research on the API, but I was pretty sure that it was possible. Sure enough, MSDN has some documentation of the API and Microsoft is touting it as the replacement for all APIs that communicate with Exchange. Web Services – how hard can it be?

The Production Environment

In the production environment, an OpenEdge client running on a Unix server needs to initiate a WebService call to Exchange to create, update and delete appointments for sales people. These appointments will be created in the user interface for the CRM application.

Sales people then need to be able to change and delete these appointments from their Outlook clients or their Blackberry/iPhones. Any changes to the appointments need to be recorded as history updates in the CRM system, so Exchange needs to notify the OpenEdge CRM system of these changes.

So the updates to appointments need to happen from both directions and the communication is going to happen cross-platform (Unix to Windows and vice-versa).

Initial Idea

My initial plan was to create a prototype that did the two-way communication directly from OpenEdge to EWS. I just wanted to prove that it was possible. The first step was to build an OpenEdge WebServices Client that created a calendar item in the Exchange store. Unfortunately, for various reasons, this is much harder than I thought it would be.

First, the EWS WSDL is not a standard WSDL. EWS is installed as part of the Client Access Server (CAS) component of Exchange. In production, you can have multiple Mailbox Servers and the CAS has to discover which mailbox server services a client. As a result, the URL that is used to actually call EWS is different depending on which Mailbox Server hosts the mailbox. The WSDL therefore does not contain a "service" node and the OpenEdge WSDL parser cannot parse the WSDL. I tried working around that by creating a dummy WSDL with a service node inside it.

The second problem was that I ran into all kinds of problems with the complex type structure that exist in the WSDL. The types and messages live in a separate XSD file and the WSDL parser did not like some of the message definitions as they included optional types.

The last straw was that the connection to EWS is an SSL connection that, by default, wants to use NTLM authentication and I battled for hours to get this to work. I would probably have figured the problem out earlier if I had had some more clear indications from OpenEdge of exactly what the problem was, but I finally decided to go and try the connection from Java.

The Bible

Before I started on the Java example, I decided that I should probably test that I could even make this happen in .NET. Even that is no small feat. The examples that come with the Exchange SDK are about as useful as an udder on a bull, and I eventually decided to bite the bullet and look for a book on the subject.

I normally avoid Microsoft Press books because, more often than not, they simply regurgitate the standard Microsoft examples and contribute very little additional value. That is not the case with Inside Microsoft Exchange 2007 Web Services. Authors David Sterling, Benjamin Spain, Michael Mainer, Mark Taylor, and Huw Upshall have done an outstanding job of creating a book that really gets to the heart of the technology. They are all members of the Exchange development team, and it shows. The book provides enough background information that it is essential reading for anyone trying to do this stuff.

Java to EWS

Armed with my new reference, I now set about building a Java client that could create an appointment in Exchange, figuring that I could expose the Java client as a Web Service that OpenEdge could consume.

Now, before you throw your toys out your crib and tell me that that is a very expensive way to do things from a performance point of view, let me just clarify that I would have ended up going down this road eventually, anyway. Ultimately, the solution will be exposed as a service on an ESB, and that ESB is likely to be based on a JMS MOM, like Apache ActiveMQ, or SonicMQ. So this is not a wasted exercise. The reason that I need this on an ESB is that the events that flow in each direction need to be available to other services, too.

Java to EWS was not a lot easier to get working. I still had to work around the WSDL not having a "service" node issue, but Java gives you finer-grained control of the parsing of the WSDL. Moreover, in my initial implementation, I did not use JAX-WS or any other WebServices technologies like Axis. I simply used an HTTP request to the WebService to get through the issues. This made life a whole lot easier.

The SSL problem reared its ugly head again, but this time I quickly learned 2 things:

1. Exchange creates a default certificate for you when you install it. Either you have to install the certificate as a trusted certificate in the certificate store, or you have to set up a certificate authority on your Windows Server and build signed certificates. I went with the former idea and that resolved the first part of the SSL problem.
2. Exchange is installed to support only NTLM authentication, but NTLM authentication is a pain to work with from non-Microsoft technologies. You can change the settings of the EWS virtual directory on the Microsoft IIS Server that runs on the CAS to support basic and digest authentication. For testing, I set this up as basic, but I now have digest working. I will probably go back and revisit NTLM again later, but it is very hard to find any information to help with solving problems, so I may simply require digest authentication.

Once I got past these issues, creating the calendar item was actually very simple.

OpenEdge through the Java WebService

The next step was to take what I had created and expose it as a Java Web Service. I went with Glassfish V3 as the application server just to test this and in about a half hour I had the Web Service up and running and soapUI was creating appointments through the Java Web Service without any problems.

I then went back to OpenEdge and used its WSDL analyzer to analyze the WSDL. It worked really well. I then built a quick client that called the Java Web Service from OpenEdge and it worked first time.

Next Steps

Although this is now a working prototype, it is exactly that – a prototype. There are several pieces that still have to be proven:

1. I need to subscribe to change updates from Exchange.
2. I need to get to updates for all sales person mailboxes. This means I need to use Exchange Impersonation and that has a number of security risks associated with it.
3. I need to make sure these updates get through to OpenEdge efficiently which means I need a test harness that can generate several thousand appointments a minute and have them update the OpenEdge AppServer without breaking it.

Conclusion

Having said that, I am really excited and very optimistic about how this is turning out. I have learned more about Windows Server, Microsoft Exchange, Active Directory, Exchange Web Services, OpenEdge Web Services, and Java Web Services than I had ever imagined possible.

In the next few weeks I will be providing an update on where I finally ended up with this from an architectural point of view.