Actually, now is a really good time to have this conversation because with the work I am doing on the Exchange Web Service code, I have just finished revamping my internal infrastructure to support the equipment and software I need to do the job. So this is going to be a two-part article. In this part, I'll tell you about the software development components that I use. In the next part, I'll tell you about the infrastructure components. The problem is that you need to understand some of the details of why, so I'm going to start with a little background.

Background

There are a few basic facts about me that effect everything else:

1. I am not rich. I have to rely on the most cost-efficient solutions I can. That means that I prefer open source/free software where it makes sense.
2. I am not cheap. I recognize that building software that is commercial grade requires a serious commitment to quality. That means a serious commitment to the development process, which, in turn, requires things like solid designs, automated builds and testing, and extremely good discipline.
3. I am the worst perfectionist. A gentleman I used to work for back in South Africa, Jan Fernhout, used to tell me that I would way over-engineer things because I wanted a perfect design. That is true. I have to be very disciplined about remembering that I need to deliver something.
4. I hate mundane work. If at all possible, someone else is going to do the work for me, and I there is no way that I am going to manually do something that can be completely automated. If you make me do it, I will bleat like a sheep until you fix the problem – just ask some of my co-workers.

There are also a few facts about the development work that I do that affect everything, too:

1. I believe that Windows is for the desktop or as a departmental product and Unix is what the enterprise needs. No one has proven me wrong on this. For every example of a large Windows product installation, I can point you at hundreds of more successful, more cost-efficient, more stable, less administered Unix installations that just run.
2. I do a lot of work with both Unix and Windows.
3. Java is the most platform-independent language for back-end work.
4. .NET is the richest GUI experience you can have outside the browser on the Windows platform.
5. OpenEdge is a fact of my life. I have used it more than any other development platform. I have made a good living off it, and I love the product, warts (many of them) and all. OpenEdge is kinda like an old pair of shoes that fits your feet a certain way and you're always most comfortable wearing them, no matter how ugly they are.

Machines and Operating Systems

My desktop machine – and primary development workstation – is a Windows 7-based HP Pavilion Entertainment 64-bit laptop with 6GB of RAM and 450GB of disk space. It runs nothing but Windows, and I have a ton of software on it.

My primary server machine is a  Dell PowerEdge T710 with 2 4-Core Intel Xeon processors, 16GB of RAM and 2 500GB drives. It runs VMWare ESXi (see screenshot at right) with 20 different virtual machines on it, some constantly running and some not. I'll tell you more about this in the next post on infrastructure. There are 6 VMs, though, that are important to this discussion:

• A Windows Server 2008R2 primary domain controller (pdc);
• A Windows Server 2008R2 build server (winbuild);
• A Windows 7 32-bit build machine (clientbuild);
• A CentOS-64 development server that hosts a whole lot of stuff (develop);
• A CentOS-64 build server (build64); and
• A CentOS-32 build server (build32) .

I then have 2 other machines that are both older machines that I use for various things. I'll talk about these in my infrastructure post, too.

Components of a Development Environment

As I said, I'm the worst perfectionist. So development starts with an idea, and ends when the code has been packaged, deployed, and installed on the end-user's machine, and they have completed user acceptance testing. In other words, it not only spans the entire Enterprise Architecture stack, but it also spans the Software Development Life-Cycle (SDLC). 

Now as much as I said I am not rich, there are two very expensive tool-sets that I have to buy:

1. Visual Studio Premium with MSDN. This one sticks in my throat every time I pay for it. The only reason I need the Premium subscription is for the Windows Server and Microsoft Exchange Server licenses. The more than $5500 that this product costs is very hard to justify, and the $2300 a year charge for renewing the subscription is as bad as paying tax to the IRS. I am a firm believer that development tools should be free. If you want me to develop products for your platform, give me the tools I need to do it.
2. PSDN Professional. Although I also believe that Progress should provide its tools for free, I am less resentful about it, probably because I used to work for the company. I still think the $3,000 that we pay for PSDN is excessive, but there you have it. My guess is there would be more people developing Progress-based applications if the development tools were given away free.

I need the MSDN license for the operating system and server licenses that I am using for Exchange Server. For my non-Microsoft-based development, I rely heavily on the Java and LAMP (Linux-Apache-MySQL-PHP) stack. Because I have some experience with PHP, I try as far as possible to use Open Source tools that are based on PHP rather than Perl. I don't know Perl and I don't particularly want to spend the time on it.

So starting from concept, here are the tools that I use to get my stuff done:

•  Sparx System's Enterprise Architect (EA) for all my modeling (not just diagrams, models). The product cost me about $300 for the original license, which included the MDG technology for Eclipse integration, and I use it as a design tool from the conception of an idea all the way through delivery. It contains the business process models, the requirements, and all the UML diagrams and test cases. I am not a fan of model-to-code driven development. I think the fact that we have been looking at the equivalent of CASE tools for the last 20 years shows that there will always be those cases where the code cannot be automatically generated for you. Two of EA's strongest suites are the traceability it provides from requirements to test case, and the incredibly powerful document generation facility. I literally generate all my documents from Enterprise Architect.
• Collabnet Subversion is used as my Source Code Control System. Everything goes into Subversion, including my models, source code, document artifacts, and even my research code. Subversion runs on my development server (develop) on CentOS. Subversion has clients for all kinds of platforms, so I have AnkhSVN plugged into Visual Studio to control my .NET code, I have Subclipse plugged into Eclipse for all my Java code, and I use TortoiseSVN at the operating system level to control documents and stuff that are outside of either development environment.
• Mantis is used as my issue tracking system. There is an Eclipse plugin for Mantis so I can get at it directly from my project workspaces in Eclipse.
• Hudson CI is used as my automated build and test environment. It runs on develop, but starts builds on build64, build32, winbuild, and clientbuild. I use both Apache Ant and Apache Maven to build Java code, and I use PCT to build OpenEdge code in Hudson. I use Nant to build .NET code.
• Unit testing is the scope of JUnit and NUnit although I am vehemently opposed to test-driven development. I think it is flawed and creates really lousy software.
• Eclipse is my Integrated Development Environment of choice. I have at least two installations of Eclipse at any one time. One is for non-Enterprise development (stuff that does not require Java EE) and the other is for Enterprise development (Java EE). The Eclipse screenshot at the top of this post shows my working environment for non-JavaEE work. From the screenshot, you will notice that I have OpenEdge running inside the Eclipse environmnent (Database Explorer is at the bottom, and there is an OpenEdge Editor open above it. You can also see an Enterprise Architect model in the right-hand pane, and my Hudson build status is in the left-hand pane. In the Eclipse screenshot at the bottom of this post that shows my working environment for JavaEE work, you will see the Mylyn task list at the bottom on the left, with the Mantis task repository open in the pane on the bottom right and a Mantis task open on the right. All of the tools that I have highlighted are integrated right into my Eclipse IDE in both environments. 
• Visual Studio is used only for development work that doesn't work in Eclipse. So most .NET/Windows-based development is done in Visual Studio. I have both Visual Studio 2008 and 2010 installed. 

All the tools I have discussed so far have plugins for Eclipse and, as the screenshots show, that is the configuration in which I use them.

A couple of other products that I use extensively that are not really integrated into the development environment per se, are:

• Beyond Compare is one of those tools that I just cannot imagine not having. It is a tool for doing advanced 3-way compares of files and it is, without reservation, the best diff tool that I know. It runs on both Windows and Linux, and I use it on both.
• WinZip is another tool that I use so much, I cannot imagine being without it. It is used for creating zip archives, but it can just as easily read and understand Linux tarballs and .jar and .war files.
• Nero is an outstanding tool for burning media, especially CD or DVD burning.

Most of my publishing stuff is done with either WordPress or Drupal. Like most of the other stuff I use that is web-based, these two are both PHP-based solutions. PHP is not a hard language to learn and it is very good at getting pretty sophisticated web apps up and running relatively quickly. WordPress and Drupal both have extension APIs that make it possible to build pretty sophisticated plugins for both in PHP – something I have done reasonably effectively.

Conclusion

Most of the tools I have mentioned above are freeware or open source or really, really cost-effective, and the good thing about them is that they lead you to build much better quality code than would otherwise be the case. If you are not looking at tools like these, I would highly recommend giving them a try.

Back in January, 2001, I was involved in a project while I was working at Progress Software Corporation that later became known as Progress Dynamics (I’m going to shorten the name to Dynamics for the rest of this article). The whole idea behind Dynamics is that you design your application components and the definition is stored in a repository – a relational database. At run time, the design of the UI is read from the repository and the UI is rendered based on the definition.

In version 2 of the product, we added functionality to allow the user interface to be rendered in a browser using dynamic HTML and JavaScript. The whole idea was to assemble the application once and render it for any platform and ultimately that could have included .NET, Eclipse RCP, and any other imaginable UI.

How does this have anything to do with content management? Well here’s the thing… content management systems essentially store the definition of the content of a web site in a repository – again, a relational database – and render the web pages from the repository. The subtle difference is that the rendering engine for Dynamics is a set of 4GL programs. For a content management system it is a set of programs (various languages including PHP), cascaded style sheets, HTML, XHTML, DHTML and JavaScript.

One of the things that was really cool about Dynamics was the ability to add custom modules to extend Dynamics for your application’s needs. Most of the content management systems have a pluggable architecture that allows you to do the same thing and you can even customize the rendering engine to your own needs.

As with Dynamics, the focus of content management systems is to get the content writer (application developer in the case of Dynamics) to focus on building content rather than worrying about editing CSS, HTML and XHTML.

The joy of the repository in both cases lies in the functionality it provides for searching and organizing the content or application components.

So now you can see why I like the content management system idea. In the same way as I still believe in the core principles of Dynamics – a rendered UI that is portable across multiple technologies – I also believe that content management systems provide the same flexibility for web sites.

I should point out that content management systems are not new. There are many of them out there and there are some very good, expensive commercial ones out there. If you are a large enough enterprise to warrant the expense of the investment in a commercial, 24x7x365 content management system, you should go ahead and look at these. Some of the top contenders in this area are:

• Interwoven
• Percussion
• SDL Tridion
• Vignette

I’m not in that market.

I am a firm believer in the open source initiative and there are a number of very good open source solutions out there. I had also narrowed my focus to something that ran on Linux with Apache, MySQL and PHP (LAMP). There are a number of solutions in this area and given the timeframe that I had committed to, I needed to go on some recommendations.

I settled early on on two alternatives, and I went with them because my web hosting provider supported them both. The alternatives were Joomla and Drupal. Early on it seemed that Joomla was the best solution. It seemed to be really easy to work with and getting content onto the site was snap. I was up and running in no time so I selected Joomla for my personal web-site.

Of course, requirements change and when I left Earthlink in April, I needed to get our company web-site, Intangere, LLC up and running fairly quickly. At this point I started looking for new modules, themes and options for extending Drupal.

That was when I realized that Joomla may not have been the best choice. I then started looking into the extension mechanism for Joomla, and although it can be extended, there are some problems doing it. Drupal has a much larger selection of new modules and themes than Joomla does and it also provides a very extensive, well-documented extension model so that it is relatively easy to build on.

Both of these options require some understanding of how they work. They are not outstandingly intuitive when you first start working with them, but once you get the hang of them, they do the job really well. Drupal just does it a little better.

Intangere, LLC, is a completely Drupal-based site and we are very satisfied with what you can achieve in a fairly short period of time.

It’s very interesting to me that these content management systems are using the same ideas as we had when we built Dynamics back in 2001. There is clearly something to the idea of rendering from a database and it’s interesting to see how each has solved the problems that we had to deal with in Dynamics in different ways.

What is clear is that content management systems are a critical part of what has been called "Web 2.0" – the next generation of the internet – and the thing that makes them really interesting to me is how they provide the one of the core enabling technologies for social media/networking.

The real value in them will come from leveraging the social media component to engage customers in more mutualistic relationships.

What started out as a project to establish a web presence for my family very quickly turned into a project about researching content management systems. All of this coincided with my departure from Earthlink and I decided that this would be a very worthwhile exercise for a number of other reasons.

I’m going to be posting some information on my Joomla findings in a day or two, but I have been very quiet for a while and I wanted to give you a heads up on why I moved to WordPress first.

One of the things I have found about content management systems is how simple it is to backup and recreate the site and copy data around. The fact that everything lives in a database and the fact that the database is easily accessible from SQL means that you can get at the content and move it between your web hosting provider and your internal servers very easily.

The tool that I was using for The Software Gorilla before this was not easy to work with, there was no extensibility in it, and the data was not accessible. It had to go.

During my investigation of other content management systems, I found that each of them had components that helped with writing a blog, but nothing did the job just the way I wanted it. Although I have standardized on one of the content management systems for three other web-sites, I wanted something that could specifically handle blogs.

I had heard a lot of good about WordPress, especially as far as its extensibility is concerned. I set off on a research expedition travelling to a number of web sites that are WordPress based.

WordPress is a content management system specifically designed around blogs. It is ideal for publishing your ideas, getting feedback, having discussions and managing… well… the content of your blog. It provides great stats and it has numerous plugins that help you to extend it. For example, if you want to have a CAPTCHA to block spam responses, there are plugins for that.

It has extensive sets of themes and looks really good when it is set up properly. I’m still experimenting with my theme and I will probably end up building my own theme eventually.

The thing I like most about WordPress is that allows me to focus on writing the stuff that you are reading rather than worrying about the administration and layout of the blog.

The thing that really made me sit up and take notice is that companies like the New York Times, Yahoo and People magazine are all making use of this piece of software to run their blogs. That probably means I can run mine on it pretty safely.

As far as set up is concerned, it is really easy to get going with it and if you would prefer to, you can have WordPress host your blog for you at WordPress.com.

The switchover took me a little while because I needed to get my content from my old system and that was pretty much a manual process.

So far so good. I’m fairly satisfied with the switch and I hope you like it too. Let me know what you think.

A number of people have pitched Drupal to me as a competitor to Joomla. One person wrote:

“When we surveyed content management systems …, Joomla was one of the ones we looked at, but on balance Drupal won out. I recently converted my … site to Drupal and brought up two other sites … on the same code base, but with very different appearance and different features on each. I am very impressed.”

I have spent some time evaluating both of these content management systems and while I have chosen to go with Joomla myself, Drupal could probably have done the job I wanted to get done. My initial post was in no way meant to imply that Drupal was not equaly as capable. There were a couple of things that swayed my vote to Joomla.

1. By the time I looked into Drupal, I already understood how Joomla worked. The argument that Drupal is easier to use therefore did not carry the water for me that it would for others and I felt like Joomla’s flexibility was probably better than Drupal’s provided I was willing to make the investment;
2. I am not scared of getting my hands dirty with PHP, MySQL, CSS and XHTML. That means that if I find an obstacle with Joomla, I have the flexibility to get into it with these technologies if I need to; and
3. I like what Joomla has done to address and caution against security issues within Joomla itself. Security is a very real consideration for any web-site and I think the Joomla team understand this. They may not have all the answers but they definitely understand the issues. You should at least look at the Joomla security site and their security forum before you implement a site using Joomla.

In fact, it is the latter point that made me realize I needed to take down my family site while I work on making sure that I have dealt with any potential security threats. It will be back up in a couple of weeks. 

In a discussion on the issue, someone I was talking to made the statement that they did not want a content management system that required a database because of the administration issues associated with it. To me, that argument does not make any sense because if you go with static HTML instead, you still have the complexity of managing the content and determining what the broken links are. With a good content management system, that process can be easily automated. So although you may not have the expense of the database maintenance, you do have the expense of the static HTML management. Most of the maintenance stuff around my 4 Joomla installations is all automated anyway and I simply check the e-mail that I get on a nightly basis to verify that the backup verification process worked. Moreover, there is so much functionality that you simply get for free that I would hate to have to write manually.

Finally, a gentleman by the name of Jeff Pilant pointed me at a web site that is absolutely invaluable on the security side of things. The site calls itself “Common Weakness Enumeration – A Community-Developed Dictionary of Software Weakness Types” and it enumerates 25 of the most dangerous mistakes that developers make that can create serious security threats in their code. This is a highly recommended read. The information is also available in PDF form and although a lot of this information is available in a number of the security books that are already out there, this is a very valuable and concise reminder.