As I said in my last post, I do not like to do work that could be automated. A large part of the work that should be automated is the work around the build process. More than anything else, successful software development depends on being able to produce a repeatable build process where the code that is built is thoroughly tested, installed and verified before it is considered stable. To get to the point of understanding how this all works, the hardware and network infrastructure is pretty important. So that's where I am going to start.

Hardware

The diagram at left shows the hardware configuration that I have in place. I have a wireless hardware router that is connected to my DSL line to the internet. Connected to that router are several Linksys hubs (there's only one shown in the diagram because it is the only one that bears upon the discussion), and a Netgear 10/100/1000 switch. Why do I have all of these?

Well I have several machines that are unrelated to my work that are connected to the network. All of our entertainment devices (DIRECTV set-top boxes, Playstation, Wii, and Blue-Ray players), my kids' notebooks and desktop PCs, and my wife's notebook are all connected to the network, some via wires and others wirelessly.

The Dell Inspiron Laptop and its companion Dell Dimension Desktop are two older machines (they're both over 5 years old) that I use for all kinds of testing and experimentation. I'm not afraid to replace and reinstall operating systems and software on these machines all the time. The Inspiron Laptop functions as my Linux server when I am presenting and I need a server away from home.

My HP Pavilion notebook is my workhorse workstation. All my blogging, development work, specs, and photography stuff (yes, I enjoy amateur photography in my copious spare time) all happen on my notebook.

One of the most useful and valuable purchase I ever made was the purchase of a LaCie 2Big Network drive. It is a network file store that hangs off the Netgear switch at 1Gb. This device contains all our backups, and acts as a file server for all the other devices on the network. I have a few 1TB external drives and once a week I plug in an external drive and back up the LaCie drive and store it off-site. All the other equipment backs up on to this device so we can always go back to it to restore.

The Dell PowerEdge T710 is a relatively new addition to the network. As I mentioned in my previous post, it hosts the virtual machines that I need to get most of my work done. It has 4 physical network cards that are all connected to the Netgear switch. 

When I first got the PowerEdge I was worried about how much power it would consume. Surprisingly, it is more cost-efficient than the Dell Dimension Desktop, which is why I was able to free up the Desktop. Until fairly recently, the Desktop performed the role of DNS, DHCP server, NIS server and DMZ. That has now changed. The DMZ has now moved onto the PowerEdge and runs in a separate VM. The DNS, DHCP and NIS server now also run in a separate VM and the firewalls are set up appropriately. My network is also segmented so that the personal network and the work network are completely separate.

Virtualization

I have been looking at virtualization for quite a while. When I worked for my last employer, we had a couple of Sun servers that were running OpenEdge AppServers, databases and other software that I was using to diagnose issues with their OpenEdge to Java configuration. I was surprised at how the system coped with the load that I threw at it, even though it was running off the same processors.

When I started work on the Microsoft Exchange Integration project, I knew I was going to need at least 2 Windows 2008 Servers and a couple of Linux boxes. I spent some time with a colleague of mine and discussed the issue and he suggested looking into VMWare again. I had used VMWare Fusion on my Apple Powerbook at my previous employer, but had not really understood how powerful this product really is.

My colleague told me about VMWare ESXi, which is a free version of VMWare that will run on a single machine and allow you to administer that machine as an isolated server. We spec'ed out what we thought I would need for the work I am doing and I ordered the PowerEdge accordingly. When it arrived, I installed VMWare ESXi on it and it has been set up and running now for almost 2 months with no down time.

What has stunned me is how efficiently this machine runs. At any point in time, I have at least 8 machines (3 of which are Windows Servers) running on this machine. The thing that I completely missed out on is how much time the machine spends doing nothing so that there is a lot of overlapping processor downtime that can be exploited. You can easily over-provision this machine and still have it perform very well. 

I'm going to need to spend some money on another 16GB of memory and another 2 TB of disk space, but that is a very simple upgrade for this box. 

The Big Score with Virtualization

The thing that I had not considered about virtualization that has been the biggest score is the fact that everything around the virtual machine management can be scripted. Not only that, it can be scripted from a Linux virtual machine running on the hardware that is running the virtual machine!

Now the reason this is really cool is because of the automation of the build process, and this is where the whole infrastructure discussion culminates.

As a software engineer, I live by the following rule:

 The software does not work until it is packaged, installed, and run on a(set of) machine(s) that is(are) not your development environment.

In other words, the fact that the code works on my notebook does not mean that it works in production. To prove that it does, I have to check it in, have an automated build kick off and build the code, have the automated unit tests run, deploy the code to the test environment, have it run there, and have the tests all pass. Finally, and this is really important for the Microsoft Exchange Integration project, I need to understand how all of this is likely to perform under load.

Building and Installing

With virtualization, I am able to create snapshots of machines, power them up by means of a script, install new software by means of a script, extract the results, power down the machines, revert back to the snapshot, and start again whenever I want to.

By means of one script, I am able to validate how successfully the build ran, and as it is a cron job that runs off my development server on a nightly basis, I have a good build and test report first thing every morning when I get up.

Stress Testing

Moreover, the test is able to use my quiet time to run any stress tests that need to be run. Assuming the build is successful (95% of tests pass, and none of the critical tests fail), the build is stored to the LaCie drive. At 8:00am every morning, the stress test kicks off because this is when the network is likely to be the quietest and the others in the house will not be affected. The script powers down any machines that are not needed during the stress test, starts the stress test machines, runs the stress tests during which time the processors are pegged at 95%, and keeps a log of how the stress tests ran.

At the end of the test (at 4:00pm) it e-mails a report for the tests and reverts to the snapshots for each of the virtual machines after it powers them down. Before it reverts to the snapshot, it copies the virtual machines across to the LaCie drive so that I can always restore the tests and see what the logs looked like if there is any question.

The cool thing about the test ending at 4:00pm is that I normally get the e-mail for stress test on my Blackberry as I am driving out of the parking garage at work, so I know what I am in for when I get home that evening.

The Value of Laziness

The real value of being lazy is that because I won't do repetitive things manually, I have put together a fairly stable build configuration that allows me to prove the code thoroughly before it ever makes it out the door. There are several important things that this process relies on, though.

I have to be ruthless about building tests for each of the new features that I build. This does not mean I believe in Test Driven Development – I don't. Test Driven Development is about building tests and then coding for the tests. I believe in building the code and then figuring out how to break it, and trust me, I am good at breaking my own code.

I have to make sure I add additional load to stress test additional features where I can. This can be really tedious because I spend a lot of time writing code that generates data.

Finally, I couldn't do this without software like Hudson CI, JUnit, NUnit, Apache Ant, NAnt, Apache Maven and most of all, VMWare.

Windows 7

Windows 7 has now been running on 4 machines here at home since I installed it back in November. Three of the 4 are 32-bit machines. The other is a 64-bit machine.

Two of the 32-bit machines are used by my two teenage kids. My son is studying drafting at college and he uses his machine extensively to run various CAD packages, including AutoCAD. He reports that his machine is much more stable now than it was with Vista, which was what his machine shipped with in March last year.

My daughter is at college, too, but her use of her machine will never really show up any major problems. She uses all the standard stuff like MS Office 2007 and Internet Explorer 8. She used to complain a lot about her machine going to a black screen prior to the Windows 7 upgrade, but I have heard nothing new since.

My wife uses her machine for college studies and accounting-related stuff. We run both Quicken and Quickbooks and it all works fine. We have not seen anything on her machine that causes a problem with one exception.

Slideshow Screen Saver Hangs

We have a couple of LaCie 2-Big network drives that contain shared data and backups. Among the shared data on one of the drives is our library of photographs. My wife uses the built-in screensaver that creates a slideshow of photographs onto the display. We have found that this screensaver will work for a period of time, but when she comes back to the machine to start working again, the machine freezes on the screensaver. We have switched to different screensavers and we don't have the problem with others so we are putting it down to that.

I have seen a similar problem on my 64-bit machine with the screensaver, but the difference is that the interface only becomes unresponsive for a period of about 20 seconds. After that it comes back whereas with the 32-bit machine, it hangs permanently.

Other Applications

On my 64-bit notebook I am running a lot of stuff, including Eclipse v3.3, 3.4 and 3.5, Java 1.5 and 1.6, Java EE, Glassfish, MySQL, Visual Studio 2008, DIRECTV Supercast which allows me to watch NFL games on my notebook, and Netflix's video player. Other than the issue with the screensaver, I have had no problem with this machine. It has been substantially more stable than when it was running Vista and I'd go as far as to say that it is more stable than my work notebook which runs Windows XP SP3.

Homegroups

A couple of people asked me about Homegroups and how effective they were. I set up a homegroup expecting that it would simplify things, but all it really does is ease file sharing. There is not a lot of value to it that I do not get from having a central share on the LaCie 2-big drive, so I have switched off the homegroup functionality.

Google Chrome Browser

I installed Google's Chrome Browser on my machine a while ago, but had not really used it. I had been using Internet Explorer 8 and did not really spend much time evaluating it.

A while ago, I was looking for a browser I could use at work. Corporate restrictions had me using Internet Explorer 6 which is not satisfactory. I had previously used Opera, but it fell short on a number of things that I needed it to do. I decided to re-evaluate Firefox and Chrome.

I ended up going with Google Chrome. Chrome is much faster than any of the others. It loads instantly and the first page comes up immediately. Screen-painting is instantaneous. When I compared this with any of the others, whether Firefox, Internet Explorer or Opera, there is just no comparison. Internet Explorer 8, for example, takes 3 or 4 seconds to start and another second or two to download my home page which is on my local network.  Firefox is even slower. Chrome is literally instant. The load time is imperceptible.

There are some things, though, that don't work with Chrome. Hosting a web meeting using AT&T's web meeting software does not work, but Webex does. Most plugins work as expected, but occasionally I run into pages that don't and I am forced to switch to Internet Explorer to work around those.

Another big thing with Chrome is that even though I use it on all my workstations, I have never had it crash or hang. I cannot say the same for either Internet Explorer or Firefox.

More than 90% of my browsing is now done using Chrome and it just works. What's interesting is that Chrome is new software and it is more stable than Internet Explorer or Firefox. If this is any indication of what is still to come, I think Google is onto something.

Conclusion

I am very satisfied with the improvements in Windows 7. I would be very happy if I could replace XP on my workstation at work with Windows 7. There are some nice features that I have gotten used to. I prefer the Windows 7 task bar. I also like being able to drag windows to the side of the screen and have Windows figure out that I want the windows tiled next to each other. Mostly, though, I am happier about the stability and I have nothing that I have tried to run that does not work on Windows 7, so I am pretty satisfied with it. I feel like it is generally faster than Vista was, but I could not substantiate that with empirical data, so it is really a feeling more than something I can substantiate.

I am extremely satisfied with Chrome and I am now running it on my Linux boxes as well. It's fast, and it works. It has become my browser of choice.

Internet Explorer 9

On Wednesday, Microsoft announced a significant re-investment in Internet Explorer. After just three weeks of development, they are talking about Internet Explorer 9 and its new features. When I read the announcement, I didn't think it said much so I was wondering why it was that Microsoft was in such a hurry to get what I thought was a very half-baked announcement out there.

The crux of the news is that Internet Explorer 9 is going to contain some significant enhancements to the way that it uses hardware and this should make it perform much better. It is also going to be far more standards-compliant, and we're not just talking Microsoft standards, if Steven Sinofsky is to be believed.

My take was that if I could just get it to stop crashing, that would be a good start. All the problems I have had with Internet Explorer have resulted in me installing virtually every browser out there, including Safari, FireFox, Opera, and, more recently, Google Chrome. Actually, IE8 is much better now on Windows 7, but I have been fishing for something else for a while.

Google Chrome Browser 

Yes, for those of you that are not aware of it, Google has its own browser, too, called Google Chrome, and it is available for all Windows platforms. There have been 30 million downloads of the software and I have been using it now for quite a while. It works well on Windows 7, too.

Actually, Chrome is my favorite browser, largely because it works with everything and it is much faster than any of the others. The pages on my DSL line at home are there instantly. I have not found anything that Chrome does not do… very well.

Google Chrome OS

The reason Chrome is so interesting is because of Google's announcement in July that it was entering the Operating System market. Google firmly believes that the future of computing is in the cloud. Their operating system, to be named Chrome OS, is intended to run on netbooks and the Chrome Browser is intended to be the interface that is used to do everything. Applications will be hosted in the Chrome Browser as web apps, and the operating system will be free and open source.

On Thursday, Google released the code for Chrome OS as open source and announced more information about it. It's being based on Linux and will be released to consumers a year from now.  From the post, Google is trying to address 3 issues with Chrome OS:

1. Simplifying user experience by making everything a web app and avoiding the need for users to have to install and manage programs;
2. Improve security by running everything inside the Chrome browser; and
3. Significantly improve performance by improving startup speed to a matter of seconds.

Is Chrome OS Already Tarnished?

Tom Krazit of CNET attended the announcement meeting and posted a detailed account of the meeting. One aspect of his post raised some concerns. According to Krazit, Google is going to specify components for Chrome OS netbooks and existing netbooks will not work. Furthermore, Chrome OS will not run on hard drives; it needs solid state drives. That means that you can forget running Chrome OS on your existing notebook. 

Bottom line: You have to buy a brand new netbook to run Chrome OS. 

In this economy, Google expects us to go out an buy new hardware to run their OS when many of us have gone through upgrade processes recently to support Vista/Windows 7? That makes no sense. My kids, my wife and I would all probably switch to Chrome OS on netbooks if it was supported on existing hardware. We are all at least 3 years from buying new hardware so Chrome OS will have to wait.

Of course… I will get a new netbook with Chrome OS because I am a technology junky, but my point is that I don't believe the market can bear this right now. 

The Timing of Microsoft's Announcement

I started out this post talking about Internet Explorer 9 and how I thought Microsoft's announcement is half-baked. They showed up at their Professional Developers Conference in LA and Steven Sinofsky was demonstrating IE9 without the normal fanfare that is associated with a new product launch.

It was only when I read Tom Krazit's post that it suddenly dawned on me why this was so important. Microsoft is still on its heels with the internet. Microsoft never took the internet seriously when it first started because they never saw it as a viable mechanism for making money. They woke up to that late in the 90s and built Internet Explorer to fight Netscape because they saw Netscape as a potential threat to their dominance on the desktop.

Remember: Microsoft makes money out of two products: 1) Windows and 2) Office. Everything else is chump-change, comparatively speaking. Threaten Windows or Office revenue, and Microsoft has serious worries.

Google is not just threatening Windows with Chrome OS; They are also threatening Office. If their operating system will store everything remotely and all apps are web apps, Microsoft has a serious problem if Chrome takes off.

Google made it clear in July that this announcement would be coming. They also indicated they were communicating with partners about hardware.  Microsoft has to be worried because no one there thought that Google would still be a serious internet company in 2010, but Google is very much there, and growing, albeit much more slowly than earlier. 

The fact that Chrome OS is going to require new hardware is exactly the reason that Microsoft made what is going to prove to be a very, very important move. If IE9 is able to do all that Steven Sinofsky says it will, especially in terms of the standards compliance, Google could be in trouble. At least with IE9 I will be able to use my existing machine. With Chrome OS, I have to buy new hardware. Microsoft may just have been given the Get-Out-Of-Jail-Free card it needs. You can bet they will use this time to get ahead and Google will be left playing catch-up.