"Why do all Engineers Suck?"

The other day I was browsing digg's "upcomming" and found this, where a project manager for some software project at a "very large internet company [in] Palo Alto" complains that all of the projects she is involved in seem to be buggy, miss deadlines, and miss the launch date. She seems to think reason for this is that "...all Engineers Suck". She then proceeds to wonder: "Why are there any bugs in the first place. Are they engineers or not?". Nowhere in her rant does she seem to consider that she might be part of the problem. Just like any other job, its completely useless to blame the job for being hard.

In my best guess she is fresh out of MBA school, arrogant, and (un)blissfully unaware of the challenges of technical projects, esp. software projects. Clearly the technical part must be the easy part; MBA school was just sooo hard.

In the end she questions whether she should move her career outside of technology. Clearly my opinion is yes.

But regardless of the ridiculousness of this misguided young project manager, the underlying question is important. What should management know about how to manage software projects? Software is hard to do right. Why else would the biggest names in software have had problems meeting release dates and going over budget? The reason its so hard is that it is full of uncertainty. Virtually all non-trivial software projects take a large amount of experience and skill to make any kind of relevant predictions about time & resource requirements. Managing that risk requires being in good communication and on good terms with the engineers who know enough about the project to know where the risk is.

So how do you deal with, or reduce the risk? The famous Joel Spolsky of Joel on Software claims that the way around this is via micromanaging schedules down to the day (weeks are too large). I disagree. I agree when he says "..software developers are building things that they've never built before. If they had, they'd just sell you another copy of the CD-ROM.". But I would use that very sentiment to argue that this is why you cannot plan down to the day.

I do not believe that completely upfront design works for most teams. Its just not possible to conceive of all of the possibilities up front. Except for extremely experienced developers, we learn as we go. The time necessary to own the concepts behind the problems you are working on is unpredictable.

Another problem is the process for making design decisions. When a developer is designing a subsystem in a project, many many design decisions have to be made. So many that sometimes that its hard to decide where to spend time, and where to just do the most obvious thing. Often decisions expected to be inconsequential become problems down the line. It is then necessary to go back and redesign to correct for the unforeseen issues, which can eat up lots of resources, but *must* be done lest the project become a mess of hacks. And thats just the beginning of the challenges developers face.

So I argue this: You have to put the management of the risk in the hands of an engineer. The manager of a project, where possible needs to be hands on. Specifically someone who is actually doing some of the work in the project coding and designing. No one can understand the trade-offs involved in what needs to be done right and what needs to be right-now without being directly involved in the development. Anyone who tries to do this from outside will cause bad design decisions, other reductions in quality, and discontent in the engineers. And, let me tell you, discontented engineers are a *very* bad thing.

So my advice is this: find a respected leader within the team and direct the team through the leader because, for the most part, when it comes to engineering, engineers respect only engineers.