What’s in a Checkbox

Just what’s in a checkbox? That is the question I was asking myself after 6+ hours and 5 engineers finally figured out 1 little checkbox needed to be unchecked for an entire phone system to start working.

The Backstory:

I was working on consolidating some call center queues last night to merge 2 of our call groups together. This involved running some Windows updates on the voicemail server.

Well, I started noticing some strange behavior like phones couldn’t download the configuration when rebooted and our call manager software on our workstations were flaking out – badly.

The Discovery:

It went from bad to worse Monday morning when I started by calling our partner’s technical support. Within about an hour we were bumping up to vendor support, and the vendor then bumped up to tier 3 support.

We had about 5 engineers and support personnel working on the issue, and our poor office staff trying to field calls from customers stating they couldn’t leave voicemails and were getting dropped.

The Find:

After looking at everything from just about every angle, the tier 3 support engineer asked why we were getting this Windows login prompt when trying to access a web portal to manage the phone system. At this point, we were at the point of curtly replying, “It’s your phone system, why don’t you tell us…”

Needless to say, I decided to take the reigns and look in Internet Information Services (IIS). For those of you who are not as familiar with Microsoft’s offering, there are a few different levels of security. I noticed 2 of the 3 were checked, which just didn’t look right.

I decided to wing it, because what did I have to lose? I unchecked integrated authentication in favor of anonymous and did an IISreset command. The thought was the phone web-management interface had its own security, so we didn’t need another layer.

The Fix:

Magically, everyone started being able to use their voicemail and call manager software. It was as if someone had opened a door and information just started flowing. I really couldn’t believe it.

Of course the vendor blamed us – or me more specifically for checking the box in the first place because there was just no way any Microsoft patch could’ve done that.

The Lessons:

1. Don’t underestimate the power of the small things in life.
2. If you are going to build your systems on top of other systems, you better know both.
3. Don’t tell me I can’t run Windows updates because then you are underwriting not patching security vulnerabilities. How about keeping up (refer to rule number 2.
4. Next time, try pulling the thumb instead of pointing the finger.

Ken Stewart’s blog, ChangeForge.com, focuses on the collision between the constantly changing worlds of business and technology. Ken is also the Director of Technology at Kearns Business Solutions.