Are We There Yet?

"So when will you be done with this development project?"

I don't know about you, but I hate this question. There simply is no good answer for it. It seems like such a simple question with a simple DateTime valued answer. One of these days I swear I'll answer with, "Oh, I'll be done next Tuesday at 2:34pm." just to see what happens.

And seriously, businesses hate that we have such difficulty answering the question. It seems perfectly reasonable for them to want to know when they can plan to have the new processes that they know they desperately need. Developers demand high salaries and are ostensibly professionals, they should be able to give a professional answer, right?

The Road is Well Paved

The thing is, software development is a lot harder than people expect it to be--and this includes software professionals. Even simple software projects can run afoul of hidden complexities that can destroy well meaning estimates and make everyone unhappy. And no matter how you hedge your answers, people simply don't remember all your caveats, maybes, and what ifs that you use to indicate uncertainty.

The end result is that developers seldom make their ship-by dates and companies become disillusioned and impatient with all software development. That's not helpful for anybody, but it's pretty much the rule anymore.

And the fact of the matter is that the vast majority of developers (and development managers) never learn how to answer the estimate question. They'll move from company to company, repeating the cycle of hope, suspicion, and disappointment over and over again. Which works well enough for the developers in the boom times when the demand for development is so high that mildly talented house plants can get hired as developers.

So a lot of people are making the same mistakes over and over. Businesses can be excused for assuming that this is simply the way things are and feel confident in their distrust of software professionals. They've been there, done that, bought the t-shirt.

Paying the Toll

This environment causes developers who care about these kinds of things a lot of heartburn. Everyone pays for the ongoing cycle of disillusionment. I believe that this is what really prompts posts like the recent ones from Ted Neward talking about professional ethics. And I've been known to throw my own hat into the ring as well.

We get tired of paying for the sins of those who have gone before. And I'm not referring to the messed up legacy code we stumble into, either. Frankly, messed up code is the least of your problems coming into a situation with a client who has been burned by previous developer promises. Companies that have had deadline after deadline missed have a degree of mistrust that is very hard to overcome.

We pay for this distrust in a hundred different ways. The thing is, trust is a paying commodity in business. Working with partners you trust means a whole lot of overhead you can simply skip. An analogy: if I trust a plumber to fix my sink quickly and professionally, I can go get a burger and leave him to it. It's only when I don't have that trust that I have to pay the additional overhead of having someone I do trust watching to make sure he's not napping under the sink.

Want to see a business manager go into a dreamy fantasy? Ask them what it'd be like to be able to trust their software developers (in house or not). The more experience they've had with developers the more intense the fantasy.

The Rubber Meets the Road

We have a couple of areas of friction in businesses that exacerbate this situation. The main disconnect with business managers is that we have borrowed terminology and tools from other disciplines without understanding that our processes are fundamentally different. It's tricky because the temptation to use manufacturing terminology is immense. After all, we are creating a product of sorts. This makes so much sense on an intuitive level that it's hard to realize that the comparison is misleading and potentially dangerous.

I wish we could retrain everyone to make analogies to other business specialties. Scientific research or law come to mind as potentially useful analogies because both are similarly plagued by the impact of unique situations, changing ground rules, and unforeseen complexities. It would be interesting to investigate how managing software development like a patent application or drug research would change how we look at the problems involved. We might have stumbled onto iterative cycles and responding to altered requirements a whole lot sooner, for example.

Paying Attention

The real problem, though, is that most developers (and even most development managers) don't take the time to learn about common friction points. Nor do they take the time to build relations with their business counterparts so that you have some political capital (aka trust) to use when it is needed. It's easy to forget that much of the progress in software development practices are pretty recent in terms of business processes. After all, business managers don't move at the speed of light and changes tend to take time to penetrate those layers.

Which means that a whole lot of industry advances aren't even theory yet in the board room.

And the fact of the matter is that you cannot expect a business manager to understand what makes Agile practices work. Or the reason that strong unit testing saves time over the long run even though it takes more time up front. Learning to communicate at a level that is sufficiently detailed for smart business decisions without getting bogged down into the jargon inherent in any specialty is an invaluable skill, and one best learned earlier than later. That means thoroughly understanding those theories yourself--not just on the surface or in buzzword compliance. It also means learning to communicate that understanding from orbit, 30,000 ft, 5,000 ft, and right on the ground. This is hard to do. It takes practice. It also takes exposure to business manager types. I'm not sure which is harder...

Something to think about, though: not learning this skill leaves you at the mercy of those who do learn it.

My point, though, is that it takes both. You have to learn your profession so thoroughly that you can deconstruct its "best practices" ("design patterns", whatever) and rebuild them from basic principles on the fly. AND you have to learn to communicate that understanding comfortably to people of varying familiarity with software development in a business environment.

That's what it takes to be a true professional. It's easy to let those two skills fall out of balance. Individuals who understand both are invaluable to a company. Also rare. Companies who discover someone capable of both are often surprised at how much smoother things run with that person placed where they can do the most good--a point Jeff Atwood's latest on becoming a better programmer drives home.

So I don't have a formula for quick and accurate estimates. Just a lot of hard work. Still, here's a tip for free: anyone asking for a firm delivery date is inherently assuming BDUF. Once you know that, you know where to start your answer.

29. January 2007 18:19 by Jacob | Comments (4) | Permalink

Two Things I Regret

Have you ever been in an interview and gotten some variation on the question "What do you regret most about your last position?" Everyone hates questions like that. They're a huge risk with little upside for you. You're caught between the Scylla of honesty and the Charybdis of revealing unflattering things about yourself.

Still, such questions can be very valuable if used personally for analysis and improvement. In that light, I'll share with you two things I regret about my stay at XanGo. Since I've ripped on the environment there in the past, it's only fair if I elaborate on things that were under my control at the time--things I could have done better.

Neglecting the User

Tim was the Senior IT Manager (the position that became IT Director once XanGo had grown up a bit). He was the best boss I ever had. His tech skills were top-notch (if somewhat "old school"). In addition, he knew his executives and how to communicate with them on a level they understood. It was a refreshing experience to have someone good at both technology and management (and since he's no longer my boss, you can take that to the bank :)).

After a little break-in time as the new Software Development Manager, Tim and I discussed what we needed to do for the organization. Tim's advice was to establish a pattern of delivering one new "toy" for our Distributors each month. He said that the executive board are very attached to the Distributors and that keeping things fresh and delivering new functionality and tools to them each month would make sure that we had enough of the right kind of visibility. Goodwill in the bank, so to speak.

This sounded like a great idea, and frankly, "toy" was loosely defined enough that it shouldn't have been a hard thing to do. It turned out to be a lot harder than expected, however. In my defense, I'll point out that we were experiencing between 15% and 20% growth per month and that we had done so since the company had started a year and a half before. That growth continued my entire tenure there. Now, if you've never experienced that kind of growth, let me point out some of what that means.

First off, using the Rule of 72 (the coolest numeric rule I know) will tell you that we were doubling every 4 to 5 months (in every significant measure--sales, revenue, Distributors, traffic, shipping, everything).

In case you've never experienced that kind of growth, it feels like ice-skating with a jet engine fired up on your back. Even good architecture will strain with that kind of relentless growth. When this happens, you become hyper-vigilant for signs of strain. This vigilance has sufficient reality to be important to maintain. Unfortunately, it also makes it easy to forget your users.

Developers like to live in a pristine world of logic and procedure. Unfortunately, life, and users, aren't like that. If they were, there'd be less need for developers. Users don't see all the bullets you dodged. They take for granted the fact that a system originally designed for a small start up is now pulling off enterprise-level stunts. They don't see it, so it doesn't exist. It is very easy to get caught up in the technology and forget that often it is the little touches that make your product meaningful. Sometimes the new report you spent an hour hacking together means more than the three weeks of sweating out communication with a new bank transaction processor. And by means more, I mean "is more valuable than".

Not that you can afford to neglect your architecture or needed improvements to sustain the needs of the company and prepare for foreseeable events. If you ignore that little glitch in the payments processing this month, you have really no excuse when it decides to spew chunks spectacularly next month.

What I'm saying here is that you have to balance functionality with perceived value. You have to know your users and their expectations because if you aren't meeting those expectations, no amount of technical expertise or developer-fu is going to help you when things get rough. In the case of XanGo, I could have afforded to ease up on the architecture enough to kick out monthly toys for the users. Yeah, some things would have been a touch rockier, but looking back there was room for a better balance.

Premature Deprecation

When I arrived at XanGo, our original product (a customized vertical market app written in VB6 on MS SQL Server) was serving way beyond its original specifications. We'd made some customizations, many of them penetrating deep into the core of the product. Our primary concern, however, was the Internet application used by our Distributors in managing their sales. We spent a month or two moving it from ASP to ASP.NET and ironing out bugs brought on by the number of concurrent users we had to maintain. We also removed the dependence on a couple of VB6 modules that were spitting out raw HTML (yeah, I know. All I can say is that I didn't design the monster).

Anyway, after that was well enough in hand, we gave a serious look to that VB6 vertical market app. Since VB6 wasn't all that hot at concurrent data access and couldn't handle some of the functionality we were delivering over the new web app, we decided that it should be phased out. Adding to this decision was the fact that we had lost control of the customizations to that app and what we had wouldn't compile in its present state.

Now developers (and for any management-fu I may have acquired, I remain a developer at heart) tend to be optimistic souls, so we figured "no big", we'll be replacing this app anyway. And we set to work. Bad choice. In a high growth environment, the inability to fix bugs now takes on a magnified importance. Replacing an application always takes longer than you expect if only because it's so easy to take the current functionality for granted. Any replacement has to be at least as good as the current application, and should preferably provide significant, visible improvements.

The result of this decision was that we limped along for quite some time before we finally came to the conclusion that we absolutely must have the ability to fix the current app. We paid a lot of political capital for that lack. In the end, it took a top developer out of circulation for a while but once it was done, it was astonishing how much pressure was lifted from Development.

It's the Users, Stupid

No, I did not mean to say "It's the stupid users." When it comes right down to it, software exists to serve users, not the other way around. As developers, it is easy to acquire a casual (or even vehement) dislike of our users. They are never satisfied, they do crazy stuff that makes no sense, and they're always asking for more. It's tempting to think that things would be so much better without them.*

I got into computers because I like making computers do cool stuff. Whatever gets developers into computers, though, it's a good idea to poke your head up periodically and see what your users are doing. Get to know who they are. Find out what they think about what you've provided for them. Losing that focus can cost you. Sometimes dearly.

*I think one of the draws of Open Source is that the developer is the user. It's also the primary drawback. But that's a post for another day.


17. November 2006 20:10 by Jacob | Comments (0) | Permalink

More Validation

It's always nice to have your opinions confirmed by someone you respect. Joel Spolsky's latest series on recruiting developers has a final section that includes essentially the same point I made a couple months ago: that specific technologies aren't as important in hiring developers as general savvy is. He even issues the same caveat that you do need domain experts for leavening. Nice. As is usual with Joel, he includes a thorough analysis of the issue he explores so there's a lot of good rumination there.


8. September 2006 09:59 by Jacob | Comments (0) | Permalink

Experienced Developers

The following applies mainly to in-house business software development. It might or might not apply to ISV or other product development houses. I think that there's room for broad application, but you can hit my list of software blogs if you want some quality sources for more generalized ISV or product development exploration.
Back when I was looking to hire developers a couple years ago, I knew some programmers that I wanted on my team. I had worked with them before and knew what they were capable of. Unfortunately, they didn't have much experience with .NET--our platform at the time. I made the case then (to my boss and HR) that a good developer is a good developer and the language and framework are more or less immaterial for people with a broad set of experience. The syntax and capabilities of the language and environment are teachable and a developer's broad experience helps them pick it up quickly.
I was fortunate enough to bring one of those developers (who is currently working in indie games) onto my team. The experience was great. I was right that he was able to come up to speed and pick up the nuances of the environment very quickly. Further, he was able to solve some issues outside of the framework in ways that saved us a lot of time (using Python for text file manipulation for example).
So does this mean that you can toss out environment-specific skill sets when evaluating who to hire? Not in my opinion. You see at that time we already had a core group that was intimately familiar with the capabilities of .NET. Domain-specific knowledge is crucial in producing software applications that actually do what they are supposed to do in a reasonable timeframe. People who have already digested the learning curve are invaluable in blazing the way for others who haven't yet done so. The reason specialists are crucial is simple--somebody who has already walked the trail is able to point out the pitfalls and shortcuts and help the whole team move quickly. The new guy was able to come up to speed so fast because he had people he could ask when he had questions and who could explain why things worked the way they did.
I came out of this experience with respect for having a diverse programming team. You have to have at least some specialists who know the environment inside and out. These specialists are going to be the limit of what you can accomplish in a lot of ways so they need certain skills--chief among them, the ability to communicate with other developers (because they'll do more of that than they probably want to and because a specialist who can't communicate isn't able to influence the project nearly as well as one who can).
But I personally don't think it is a good idea to have a team or department composed solely of specialists. Any software development that requires more than two people is going to run into problems in a wide array of domains and having a broad skill set to draw on can be very useful. What proportion of specialists to generally kick-ass developers do you need? Well, that's a good question. I welcome ideas in the comments, but I suspect this is one of those things that is highly contextual and thus at the heart of the "art" of software development. I'm thinking that a narrowly defined ISV benefits with a higher ratio of specialists than, say, a multi-level marketing company looking to manage their necessarily custom business needs.
25. July 2006 19:30 by Jacob | Comments (0) | Permalink


<<  February 2017  >>

View posts in large calendar