Kudos on unscrewing yourself, MSDN!

Seen and heard (but mostly seen) – MSDN’s shiny skinny new look.

Compare and contrast the two looks. Here’s a random page about one of the classes you can use to hammer on XML, the way it was before.

And here's the new low-bandwidth edition of the same page.

Oh hey look – I don’t have to dig through panes and curse quietly to myself when that fucking clownishly giant left pane class navigator slides in and out because I’m using Firefox’s incremental search-as-you-type or target where I wants to scroll.

The garnish disappears quickly and I’m left chewing on the meat of the information that I wanted to get at.

Executive summary – dear designers, please consider servicing the needs of your users rather than merely servicing yourself.

Love, me.

Tripping into the valley of fail

C# 2.0 introduced nullable types to the language (apparently late in the dev cycle - more on that soon), something that I could have used way back when.

I know, LtU duders - nobody can prove that we really need null and it's a terrible idea. Or an OK enough idea in the absence of rigorous mathematical proofs but, and don't let nobody in on this, I nearly flunked out of my 9th grade math class (which was really the advanced 10th grade math class; I can't explain it neither) because I could not prove my way out of a paper bag. Calculus eluded me and vector math haunts my nightmares. I'm no math pro and this is a blind spot I'm all too aware of.

But null's really useful, honest.

In our application, we have to deal with dates that the user's supplying. Since this is client-side input, we have to deal with two possible fundamental problems with the dates. Did they forget to enter a (mandatory) date? Did they enter an invalid date?

I mention dates because they're (in .Net) a value type. Like other value types, once you create the variable, they're automatically assigned a value as opposed to reference types, which are null until you instantiate them (the pointer, she points nowhere). And then there's string, which lives in a state of sin in the gray area between value and reference (it's a reference type but when you pass/copy it, you get a copy of the string rather than a pointer to the object). Officially, a string's an "immutable reference type" (thanks, Google!). Tangentially, are there other immutable reference types in the .Net framework? Damned if I can think of one.

So what's any of this have to do with nullable values?

The user forgot to enter a date. OK. What value should we use to represent the fact that that the user entered an invalid date? DateTime.MinValue, maybe? Sounds reasonable.

That's covered, so on to fundamental bloops number two - the user entered an invalid date. Hmm. DateTime.MinValue's already been recycled as a magic number to represent "missing date", so we'll use DateTime.MaxValue. Game, set, ma... oh, wait.

You mean we'll have a need throughout the application to use DateTime.MaxValue to represent things that are open-ended?

Now we've got a problem. Do we want to pick out a second magic number to represent the fact that the user entered an invalid date? Maybe treat an invalid date and a date that hasn't been entered identically? Maybe we want to wrap dates in a struct that contains a DateTime and booleans for invalid/missing date?

Hmm. That code's starting to stink pretty badly. There's got to be a better way.

That's where I was hoping nullable types would come to the rescue. Where previously we had to use magic numbers to represent error states in our values, we can now use a not-so-magic value - null.

Suddenly, we're not looking to wrap things in a struct and perform all sorts of acts that no shower will ever quite rinse off. User forgot to enter a value? Null. Invalid date? DateTime.Minvalue. Take in the win.

Did I mention that nullable types came late in the development cycle? Let's take a look at a little code and see how it works out.

string nullStr = null;

nullStr.ToString();

Trying to execute a method on an uninitialized variable - what do you get? A NullReferenceException. No-brainer.

int? nullInt = null;

nullInt.ToString();

Let's try the same thing on a nullable integer. We should get the same thing, right? Wrong. It returns string.Empty. To get the same result, you'd have to say...

int? nullInt = null;

nullInt.Value.ToString();

What the shit? Nullable types have a .Value property? Pro move, guys. Way to leak the fuck out of that abstraction.

Truth be told, ToString() not horking a NullReferenceException doesn't bother me that much. It's the unexpected coalescing of a null value that gets me. I set out with my golden hammer to create a new li'l method called ToStringOrNull() and hang it off of Nullable<T> that does what I'd have designed ToString() to do in the first place - return a null string if the value's null and call the generic ToString() function otherwise. But I can't attach a constraint to that function because Nullable<T> is a structure, not a class. Fail, fail again.

Polluting the namespace with this feels wrong, so what do I do?

Tell other developers to always use nullableType.Value.ToString() and hope that nobody slips up?

Add bunches of tests to our increasingly tag soup-y MVC app (and hope that nobody forgets to do it)?

Not good times. Small inconsistencies pile up until you're so busy bookkeeping for them that you can pretty easily lose sight of the bigger picture. Either that or you grow your Unix beard out and spend your days using your phallus to point to chapter and verse for the reference specification for your language of choice on Usenet. The latter's not an option for me since I can't grow a beard and the former ain't pretty neither.

I'm hoping that I can sneak in some elegant solution to calm this jittery behavior, but I've got no idea what it'll look like.

I just wanted to give a special shout-out to whoever for the head-scratching behavior. Wait. Is the person behind [DebuggerStepThrough] behind this? By all that is unholy, I will get you for this. These. Whatever.

Unit testing - prefer messages

tl;dr version - if your unit test tool lets you associate (informational) messages with your test assertions, use the fuck out of them.  It's great that you're driving towards 100% code coverage.  How much greater will it be in 2 months when someone (probably you) breaks a test and has a useful indicator of what exactly was being exercised rather than trying to puzzle out a simple assertion? 

I'm sadly new to the unit testing game, so I've been learning the wrong way to do things at an astonishing clip, while mistakenly stumbling over things that work by accident every now and then.

I never quite understood the hubbub over unit testing - why do I want to do extra work that doesn't go towards getting a working product out the door?  Now that I'm writing oodles of unit tests, I understand exactly why I want to write them - they save my ass early and often.

Case in point - the object to XML mapper (this isn't NIHS; I genuinely can't use LINQ to XML because the hierarchy the external service produces is not only unpublished but subject to change) I'm writing.  It's been working, but I noticed that it was... how shall I say... less than performant?

So I set out to start refactoring critical sections of the code.  I started by gut - I started taking FxCop up on its suggestion to use IXPathNavigable and knocked bunches of stuffs out.  Minor improvements.

Then I stopped programming by guesswork and profiled a generic run pattern.  Creating objects (with objects creating other objects), updating the persistent XML store, blah blah blah.  Found the genuinely astonishingly slow parts of my code and broke out the chainsaw to fix them up.

For a change, I had a really, really high level of confidence in all the changes that I was making.  Before unit tests, it was just more guesswork as to what I might be breaking outside of the code that I was touching just by looking at it funny and changing this postfix increment to a prefix increment (OK; that's an exaggeration, but you kinda know what I mean).  Now that I've got unit tests in place, it's a whole different story.  I can try things out and see if anything breaks in real time.  If the coverage is good enough, I've got silly confidence that everything's on the up-and-up.  If it isn't, whatever.  Adding a few more tests isn't moving mountains.

But a strange thing happened along the way - unit tests that I'd written a month or two earlier started breaking.  Even stranger, I had no idea what some of them were doing.  Not many of them, but I was clueless about the provenance of some of the tests that I'd written a couple of months earlier.

That sends up red flags for me - there's still value in having those unit tests, but I can recognize that if I don't have a little more context associated with them, they're going to bit rot really, really fast.  I started by putting comments above the tests explaining what they were doing, but that felt kind of unsatisfying.

Then I cracked open the manual and discovered that you can attach messages with your assertions.  Epic win.

I find that when I write unit tests, I slip into a lightweight QA state of mind - I think less about the cases that should work and more about the edge cases, the awkward states that I can put my code into to get it to break.  It gives me a chance to stand back and re-examine the code from that stance as well as getting a feel for how easy the class is to use, since I have to instantiate objects (and everything else in its dependency graph) before I can start to test it.

The time that I'm thinking about what the class is doing for me and how to use it lends itself naturally to embedding context in the tests.  Not simple messages like "validating that CountryCode gets populated when the object's hydrated from XML" but "validating that nullable enumerations are being populated properly."

Prefer messages in the unit tests you write.  They'll help you make the most out of your unit tests as you write them and they'll help you understand your unit tests when they break down the road.

Mea culpa

When I first started writing .Net code, I was all about implementing IDisposable because I figured that the GC wouldn't be as smart, as fast, as efficient as the stuff I could write. I mean, sure - they optimize for the general case, but who knows better than I do just when to free memory and resources? Not some jackhole Microsoft programmer, amiritepeople?

Since those were the heady days of VC and no clients demanding things change yesterday, I actually spent half a day working with the clumsy spike I'd slapped together and let it fly - it worked well enough under load, so I was happy. Then I ripped out my destructor and let it roll again - I was figuring I'd see the CPU thrashing as .Net's garbage collector did its thing, working on the general, sub-optimal, case it was written for. No egghead knows better than I do how and when this should run!

Except that there was no difference that I could see. If anything, run time was a little faster and memory overhead was a little lower. I mean, probably statistical noise faster and lower (that Excel spreadsheet's been lost to the river of time at this point), but that was a pretty well-defined zenslap moment for me.

I thought about it a little and realized that oh yeah - that garbage collector. I'm not allocating memory either. People way smarter than me have already implemented a garbage collector so I don't have to worry about allocating and freeing memory on the fly. The bold promise of distilling your codebase down to actual business logic rather than bookkeeping allocations and all that.

It's a solved problem, so why am I solving it again, only invariably worse this time? Maybe it was written for a "general case" (whatever the fuck that means because I obviously can't defend it) but it was a pretty good general case.

This all came flooding back to me on Friday. I'm working on a bit of code - an object-XML mapper. This isn't as stupid as it sounds (I hope), honest. It's running, well. Not so good. I mean, it does what I want it to, just way way way slower than I want it to run.

One of the "optimizations" I made was ripping out a lambda expression iterating over a singleton (I know, I know) - I figured that there ain't nothing faster than a hand-rolled for loop with a break condition... right? But I wasn't making any headway with the other two offending methods after re-ordering my if block, so I decided that I might as well, you know, test it out to see how it performed.

I didn't check memory this time around, but damn if it wasn't just as fast as the for loop. Maybe a little faster, even.

Again, the zenslap - the framework's made by people way smarter than I am. I need to count on them to have done their homework and made stuff easy to use and scary fast.

Stop reinventing the wheel. I'd bark at co-workers who tried to roll their own second-rate mechanism for mapping objects to an XML hierarchy we don't control, so why am I confident in my ability to roll my own iteration loop? On some level, doesn't it make sense that smart people who get paid to work on iterators might find a way to wring a little more out of them?

It's not easy discovering that something so simple that you've taken for granted for so long (a for loop!!!) is halfway obsolete, but it's liberating once you get over yourself and embrace it.

So here's to you, whoever implemented .Any() - you did a helluva job. Way better than the jackass who shat out [DebuggerStepThrough].

Software management - The Oregon Trail model

OK, forget reading Peopleware and The Mythical Man-Month. If you want to manage developers, there's only one - no wait! - two things you need to do.

1. Be born in the late 70's/early 80's
2. Play The Oregon Trail on an Apple II

That's it! If you can make it to the end of the trail, you know everything that you need to know about successfully managing software, so go successfully manage the shit out of a project or three!

Wait. You're still here? I should go on? OK.

You have your hunters, guys who strike out in hopes of lucking into a bounty of delicious, delicious meat. Speaking authoritatively as someone who's never hunted or caught anything of note while fishing, I'll say that a good hunter operates off of instinct, innate ("animal") intelligence, skill and luck. Legends tell the tale of the hunter who struck off into the deep of the jungle with nothing but a knife and came back carrying a gorilcopterous (work with me people, I live on the edge of the tundra) or some big tasty animal that lives in the jungle⁰.

Then you've got the gatherers who plant crops and (hopefully) eventually got to harvest the bounty. It takes a deep investment in time and resources to see a field from seed to harvest, and even then there's any number of natural disasters that can beset you along the way. This isn't to say it's easy. On some level, we all understand how plants grow. They eat sunshine, water and carbon dioxide and crap oxygen and chlorophyll, right? On another level, there's a hell of a lot that goes into it - seeds cost money. Machines to till and harvest cost money. Irrigation costs money. You've got to know when to plant. What to plant. When to rotate your crops.

Developers are an impatient bunch, which means that we don't generally have the patience for the small, far-off, rewards and repetitive work that it takes to grow a crop. But we get hungry, so we're going to strike out and hunt, hoping to bag the next seminal idea. OK, it's probably not a seminal idea by any stretch of the imagination, but for a brief moment in time, you really feel like you've crafted something brilliant and have found something more than you could have hoped for, something that's somehow bigger than you could have imagined.

But eventually, the quarry of the once-fertile plains and forests of imagination!!!!!!! will run dry. Your hunters will pick up on the fact that there's no more ideas to be hunted, only tedious fields that demand regular, monotonous maintenance, and they'll migrate elsewhere. This is all well and good, but every now and then wouldn't you like a steak to go with those potatoes?

But again, I get ahead of myself - let's take a step back to see what a dated educational game has to do with any of this.

In The Oregon Trail, you're in charge of seeing a family¹ make the trip on the Oregon Trail from Missouri² to Oregon³.

You notice how they emphasize "try"? It's because of a simple fact that you don't want to admit to yourself - no matter how simple it seems like it should be, that shit is fucking hard to accomplish⁴. To the outsider, it's as simple as getting from point A to point B. That's really all there is to it, right?

Go figure - there's nuances that you hadn't even considered when you started out. Could this mean that you'll eventually discover that there are nuances upon nuances? This makes the decision of something that seems as simple as when to leave a dizzyingly difficult one.

Leave early and you'll be freezing and making slow, painful, progress⁵. Leave too late and you'll face the unenviable, super-difficult, task of over-wintering⁶. Face it - you've tried to make a trip like this before and you've probably still got the bruises and scars from the last one, but you keep telling yourself that this time will be better.

You need supplies to make the trip. Oxen to move your cart⁷, clothing to keep you warm⁸, food⁹, replacement parts¹⁰... and bullets for hunting¹¹. And naturally, you're constrained to a budget that's tighter than you'd like.

Making matters worse (you mean it gets worse?), you're probably making the trip with a bunch of egomaniacal (wo)man-children. I mean, we all strive to be egoless in our tasks, but we all take an undue amount of pride in an elegant solution and invariably take it the wrong way when someone points out the glaring flaws in our implementation. Or maybe I'm the only one, who knows?

But... about those supplies. Some of them are effectively interchangable. If you've got a good hunter (you have a good hunter, right?), you can exchange bullets for food in the wild - that's the biggest bang (pun unintended) for your buck. Then you can trade food for just about anything else you need along the way¹². You can run pretty lean-and-mean¹³ but you need enough food to keep your bellies full.

You're going to want to hunt (because it's fun!), but not every hunt can be a winner.

What can I tell you? Times is tough out there. Even with a gang of elite hunters firing on all cylinders and a pile of food (ideas!) as high as the eye can see, sometimes nature just doesn't smile on you. As in any pursuit, it's entirely seemly that you can do everything right and still fail.

Not only are times tough as hell, but you're fairly constantly reminded of your own failures and the failures of others in a big, somber way.

Is there any wonder why Einstein up there didn't make it? Wasn't cut out for it. You can't teach people how to hunt - they're born with it or they'll never get it, now matter how blue in the face you get trying to explain it to them.

The trip itself? It's possible that your people are entirely happy in Missouri right now. You can crack that whip all they want, but if you don't instill a powerful longing in them to reach the promised land of Oregon, you'll never make it. In fact, you might find yourself unwillingly invited to a Donner Party. And not to ruin the surprise, but everyone at the party will be eating but you.

---

0. you know, your run-of-the-mill whispered about in legends superhacker
1. your development team
2. the start of the project
3. the project's end (you do have a concrete end-point in mind, right?)
4. #9 on Software's Classic Mistakes - Wishful Thinking
5. sometimes you have the toolchain you need to complete your project, other times you're kind of winging it as you go along - bootstrapping too much of your own technology stack only shrinks the chances that you'll ever make it
6. running out of funding
7. computers
8. technology stack - compiler, language, yadda yadda
9. ideas
10. source control and the other niceties of modern development (build server, unit tests)
11. ain't nothing more dire than running out of bullets - there's no metaphor here
12. dumbest way of describing the point of open source ever?
13. agile??!?

Aging codebases, a postscript

Ron Avitzur's comment on my post wondering aloud as to whether code can age gracefully done tickled my brain. I mean, I think about code a lot. I think about software and making software a lot.

But for some reason, I hadn't ever pondered how truly strange software is - if you look at it a certain way, it's never truly broken. It may be old, it may be ungainly, but it will always do something.

Sure, the hardware that it ran on may disappear or cease to function and the compilers may evaporate... but that doesn't mean that the code's broken. It just means that you've got a little work ahead of you porting that software to the latest and greatest hardware and toolchain and bingo bango your software's back from the dead.

Of course, there's always the question of "should we?". Times move on and inevitably reach a tipping point where maybe it makes sense to let that sleeping codebase lie.

Steve McConnell refers to this as a "doghouse problem" - when it comes to building a doghouse, everyone has an opinion on what color to paint it, what the roofing tiles should be, the exact dimensions, yadda yadda because it's a problem so eminently tractable that we can easily wrap our heads around it. Hell, it's within the realm of reason that we could build it ourselves. We have a pretty good idea of what it'll cost to make and how long it'll take.

But ask us to price the cost, materials, resources and time required to build a skyscraper and... good luck on estimating that one. How hard can it really be though? They're both buildings, right?

A couple of real-world buildings brought this into clear focus for me.

First is an old mill building not too far from where I live. For the years I've been here, it's been uninhabited - a hulking, crumbling mess of a building. 100 years ago, I'm sure it was a bustling center of commerce, an economic powerhouse. Now it's gone past creepy and into sad. It's being demolished to make way for... I don't know. Probably another drug store or something.

(zoom on the crumble)

Second is This Old House, or what's left of it. There was a house there once, nothing here now but the bones of the old building rising like so many dinosaur bones from the ground.

(check them bones)

Either one of them could be broken down as much as possible to be built back up to work again, but it's the house whose bones will remain in place while the mill's ground into the dust. Looking at it, the possibility of fixing it piecemeal doesn't even enter into your mind - it's over a hundred years old, frigging brick and huge. The house, on the other hand? Over a hundred years old, but the frame and chimney are still sound - we can rebuild around them, putting the walls and stairs and windows in different this time.

When looking at open source projects, it's not uncommon for one of the mentioned features to be their "clean codebase" - on some level, this is kind of laughable because who but a geek gives a shit what the code looks like as long as it runs well enough and does something useful for you?

On another level, it shows that there's people who have taken pride in their work, that fix broken windows, and have a vested interest in making sure that a year from now, it will have crumbled less than that mill building.

Think about building something that will stand the test of time and it just might. Make it concise and useful enough and who knows? Its framework just might live a decade or so.

I hope to someday build something so good that I won't be able to hear the lamentations of the developers responsible for supporting it because I'm busy watching Wheel of Fortune too loud or whatever it is that incredibly old people do. Did I mention that I'm watching The Wheel in my rocket car? Because ideally, I'm incredibly old, reasonably well-to-do and the future finally delivers on its promises.

Postscript

I biked by the house again earlier this week. Nothing remained.

I still hold out hope for a rocket car-filled future.

Can codebases gracefully age?

Terrible shame. Monstrous shame.

I really need to stop looking at my old code. More than that, I need my old code to work all the time in the face of needs and requirements that didn't exist when I first wrote it. Is that even possible? It isn't when I put it like that.

Probably not. At this point, I'd say "almost certainly not," but it's not like I've seen a whole lot of other projects' codebases (it's not something that I do for fun).

The sad thing is that, not to sound like a monstrously arrogant prick (even though I am), I'm the best developer where I work, by far. When Larry O'Brien made the observation that bad developers are actively counter-productive, it riled some people, but I'm in complete agreement. I'm the best developer at my office (sometimes by more than an order of magnitude) and I'm out-and-out embarrassed by code I wrote six months ago. As in "how the fuck did this shit ever work?" embarrassed. As in "if I ever come across a time machine, I'm going back in time to punch myself in the throat for that mess" embarrassed.

And it only gets worse.

This isn't humility, this is common sense. Microsoft doesn't want me, Google wouldn't fart in my general direction. I'm pretty sure even Fake Steve Jobs would prefer that I not have any Apple products.

I'm certainly no superstar developer (do they really exist in nature?), but even developers way smarter and better than I am have problems with their codebases, especially as they age.

Microsoft's not having such a smooth time of things with Vista - people are hating it so much that they're apparently seriously thinking about doing away with backwards compatibility, regarded by some as the albatross around Vista's neck. But... backwards compatibility is what sells Microsoft products for a whole lot of people. Microsoft's got scary smart dudes working to insure that your old software will keep working, even if it means that old, really questionable stuff, stays in the OS to keep it working.

They must make bunches of money off of it (there's a school of thought that says that the solid support/backwards compatibility fuels the virtuous cycle that keeps large companies from jumping ship from Microsoft), but with the time overrun and backlash about how assy Vista is, they're thinking about sacrificing a cash cow for a restart on the accretion of bugs they have to support in their codebase.

Apple's another company that's got scary smart dudes working for them. Like, even the guys that don't really work for them are scary smart. Another place where I'm sure I'm the janitors are smarter than I am. Even they've got problems with their codebase.

Even there, when it came time for OS X to come out, they broke backwards compatibility with System 9. There was an emulator or something for it (I could swear my PowerBook has it on it) but they made a clean break of it. With point releases of OS X, they brutally deprecate features and functions, leaving developers to clean up their act or not run in the new version.

It's kind of rough, but it's probably also the right way to go. Yes, as a first course, you want to refactor the hell out of your code. But they've accepted the fact that their codebase doesn't age gracefully and when they find really objectionable code, they rip it out as soon as they can. Developers who support Apple apps seem to not mind it all that much (maybe they do?). As for the users... compare and contrast people's reaction to Leopard and Vista.

I just don't think that it's possible for code to age well. Bits and pieces may stand the test of time, but the whole will become a not-so-secret shame. Well, maybe TeX has aged gracefully, but (and no offense intended) nobody who's as smart as Donald Knuth is wasting their time reading this blog.

(click for more bigger brick)

I think that's a totally lovely sight, but if that were software, a bunch of people (the developers supporting it, the people trying to sell it) would be in a lot of pain because that is unfit.

I mean, OK. First it was designed as a brick edifice to hold a building up. Then someone decided that we could slap an ad for Coca-Cola on it. Then a real genius figured "hey, it worked for Coke, why not make it work for Gold Medal", and in the process, they lost sight of what it could reasonably support.

If that were software, I'd be scared to touch it - if I do anything with it, I'm going to lose the Gold Medal ad or the Coca-Cola ad or both or the lovely interplay between the two. I don't want to be responsible for fucking them up.

It only overtly serves one of its intended functions well at this point, when, if it were software, people would be demanding it support all three. At this point, we can all agree that there are simpler ways to support multiple ads (they're called billboards, have you heard of them?) and maybe we should just strip it back to what it can do well (be a brick wall).

In the face of all this, why do I keep developing? In the hope that next time I'll get it right.

So what about you out there - do any of you work on applications that only get better with age?

Taking your lumps like an enterprise

Obie Fernandez posted recently about SAP, enterprise software peddlers par excellance, getting their asses sued by Waste Management, a behemoth of a company, for a clusterfuck of a deployment.

This reminds me why I want to keep working for small companies - I want to know when other people are making absurd promises that they'll then expect me to uphold as soon as possible and if at all possible, I want to be able to correct them or speak directly to the client and say "Sorry, they fucked up when they said that. Not gonna happen in that amount of time, but let's see how many items on that checklist we can realistically get done in the time and cost constraints you've set forth for us." Scrum has embarrassingly juvenile terms, but I think they really nailed it with the pigs and chickens metaphor.

It also reminds me of a talk I had with my COO a few years back, back in the heady days when I thought software could solve all ills and had no interest in moving any further up the food chain than architect. We were talking about some other clusterfuck that was in the news - a huge enterprise software vendor, a huge client, tens of millions of dollars and... nothing to show for it.

The developer in me was all righteous indignation - "How can they get away with that? Isn't that kind of thing embezzlement or something? If that were me, I would sue the bastards."

His response (and I'm paraphrasing) was a thunderously mighty zenslap.

"There's no point in suing them."

"How can there be no point? It'll serve notice to the next vendor in line that they're not to be trifled with."

"No, there's really no point. Even if they win, they'll never recoup their costs. They'll scare away other vendors and they still won't have what need. It's over."

I couldn't argue with that. I can't.

The contracts are dozens of miles long, filled with all manner of Cover Your Ass wording and dense, vague terminology that likely provide a Get Out of Jail free card to both sides. It'll be tied up in the courts for years, but it doesn't change the essential problem - you've already slipped what you thought was your deadline and run over-budget.

Do you spend time, money and focus trying to chase what you already know is a dead-end or do you effectively shrug your shoulders and focus on what will make you money? Were you completely up-front with all your needs? Were they all clearly outlined (and signed in blood in the contract)? Really?

And remember - if you do go with suing them, your chances of getting that software you swore you needed probably just evaporated. That's money that's probably coming out of your implementation budget (which is, again, already exhausted). If you could have done it in-house, you would have (right?) but you couldn't, so it means you need to find a new vendor. Who is going to be very wary of signing on with you because of the lawsuit, which means even more time and cost to get to the point where you can convince them to start building for you (which is going to take even more time and cost more because they know about your lawsuit and need to double-cover their asses). Knowing that it'll be a pyrrhic victory at best, do you still sue?

So I'm surprised that Waste Management is nutting up and suing the holy bejeesus out of SAP. I am, make no mistake about it, cheering for SAP to lose in the most majestic manner possible. I hope other large companies sue other large, shitty, enterprise software vendors.

And I am hoping that when they're eradicated, I'll figure out a way to become the next SAP. Money isn't the be-all and end-all, but I'm not too much of an asshole to say no to obscene amounts of money for nothing more than putting on a suit and having a nice little dog-and-pony show.

The difference between a shoplifter and a very successful enterprise software firm is that you're scared as fuck to prosecute the enterprise vendor when they rob you blind.

Working Hard to Get You Fired

When we first started porting from ASP 3.0 to C#, I'd never heard of test-driven design (why hast thou forsaken me (in 2000), blogosphere?!??!).

When we had a second round of developer hires, the two guys were gung-ho to get us on the Path Of Continuous Integration and get us all test driven and stuff. Were it not for the fact that we already had 200-odd KLoCs of legacy code of varying quality that would need to be refactored to be testable (in retrospect, I'm aware of just how big of a cop-out that is) and deadlines looming, we might have gotten on board the TDD train. But had the blinders on, so that never happened.

I've done some refactoring to bake in poor man's testability (objects that audit themselves and throw exceptions for as many offenses as possible) but the fact remains that building new functionality and fixing old issues often feels like sticking thumbs in the dam - I can feel when something's going to break, I just can't tell what or where.

About a year ago, the QA department hired someone to do automation for them. I was pretty pumped about it the application that we've built is that it's so big (400 KLoCs of C#, some ASP.Net controls, another 75-100KLoCs of T-SQL, bits and pieces of BizTalk) and so data-driven, at this point it would take something just short of an act of god to get it to a point where it's testable. I have little to no desire to be part of that and have never pushed for that sort of intra-object testing. I remain unsold on the promise of TDD, but I'm not so arrogant as to completely discount it; it's more that, like security, it's not something that can be bolted on. Maybe.

Simulated end-user QA automation - that's the testing that I can get on board with. I've been using the super-hella-sweet Watir for a while now and have handed it out to developers, product development, QA, even customer service reps when I want to replicate problems; it's been a godsend. I've built smoke tests that I can run on each deployment build that have saved my bacon more than once. I don't know how much you can tell about how the application's working or not working from inside of an object, but it's pretty easy to tell when things aren't working with it when your scripts break while being run.

The QA automation guy? Nice enough chap. He's personable, seems to know what he's doing and all, but... a year in and I've got no test suite to run. Enterprise-grade tools mean awkward enterprise licenses (we actually run a fucking license server somewhere or other- what the shit is that?) and all the user-friendly quality you'd expect from anything enterprisey. Plus I suspect that the goals were too lofty, that they were intent on building something beautiful and perfect and settling for nothing less. Which is great and all, but daddy wants his application tested already.

We've hit one of our cyclical lull periods at work. This isn't to say that there aren't issues cropping up all the time, patches being sloppily built and hurriedly and incompletely tested (business as usual!), just that the volume of issues is decreasing and management's hemming and hawing about what new features to build out. I've been sitting on my hands, getting micromanaged on what issues need fixing and what can be left to slide, so I took a look in my Watir folder - hey look, 100-odd scripts testing and regressing issues that we've logged for the past year or so. Most of them broken at this point because of genuine application changes, but whatever.

I set out to fulfilling the promise I saw in testing when I first brought it up - that not only were we saving time with each test, but that we were organically building up a test suite, testing the things we needed to. Getting all of my test scripts working was step one. Step two, the uglier step, was taking the user out of it. Before, I was relying on the end user (the QA person it was aimed at) to push a button in an internal GUI here, look at a piece of data there. But with a few Ruby system() calls and poking at the content of spans and tables (time-consuming as that is), that's fallen by the wayside.

The next hurdle was the database side of things - there's some pieces of data that don't make themselves evident in the application to the end user, so how the hell am I supposed to test for those? Oh - you can pass variables (like the auto-generated user's ID) in to SQLCMD.

So a week and change of me ignoring this and that piddly little issue later, and I've got 3 dozen or so scripts built out and checking things. Another couple of weeks or so and I'm going to have an honest-to-god test suite and, if anyone's paying attention, a QA automation pro's going to have some explaining to do when I've come correct with a reasonably not-too-shitty regression suite in what amounts to my free time and he's over a year in and still hasn't produced smoke tests. And if no one's paying attention and nobody gets canned for it, I'm not losing any sleep over it because I'll have scratched one hell of a big itch.

If only my solution was enterprise-grade, we'd be able to use it too.

Breaking the Hype Cycle

Like a good .Net drone (haw haw), I rambled on down to Waltham, Mass. this past Thursday to get introduced to the brave new world of .Net 3.5 and Visual Studio 2008.

I found myself going through various phases of confusion, boredom and then (eventually) excitement. Confusion over David Chappell (sadly, not the sometimes Rick James impressario) and his speech there. Given that he's written books on stuffs, has a consulting firm and talked about opening for Bill Gates at some development conference, I take it that he knows his thing. That's great and all, but... this was a developer's conference or so I thought? The man can give a good speech, but I'm still clueless as to why he felt the need to sell developers on the .Net ecosystem as a viable platform. Preaching to the converted and all that.

Todd Mancini came out next to give a talk on Visual Studio Team System. When I first started reading about it, it felt like it was just another enterprise Rorschach test but seeing concrete examples of what it does (how many use cases have been fulfilled, how many have been tested by QA, how many have unit tests, code churn, yadda yadda) I can see that behind the scenes it's a pretty slick little piece of work... more on that later.

I was remarkably confused by the talk that Allan da Costa Pinto gave. I'd signed up for a couple of sessions on .Net 3.5 and Visual Studio 2008 and for some reason got treated to an hour and a half on Windows Workflow Foundation and the Windows Communication Framework. He gave an OK enough presentation, but goddamned if I could tell you what either of the technologies are. Repeated over and over is that "you can paint your business processes" but what drawing a workflow diagram's achieving me is still beyond me. I get the feeling that there must be a lot in there, but I also have no idea what's in there or why I'd want to use it. Maybe it's targeted at a different user space or maybe it's a solution in search of a problem? I do know that you can start a process and wait up to 30 days to take the next step in it! Yes, it means fuck all to me too.

Chris Bowen, the last speaker of the day, was finally the guy who got me excited to have gone down there. Away with the 10,000 foot view of what's new with the .Net platform (too goddamned much to follow) and into the new language features (anonymous functions! anonymous types! lambda expressions! extension methods! all that good shit that perversely makes perfect sense since I've been embarrassing myself by trying to wrap my head around Haskell and functional programming) and a good, quick, runthrough of LINQ.

When I first heard about LINQ, it sounded kind of retarded to me. Hey look, it's sort of SQL. Except the select clause comes at the end and it's uglier? No thanks! How is this supposed to do anything for me? Why would I want to write not-quite-SQL and then deal with object/relation impedance on top of that? I can see it working for CRUD, but I've got joins that pull together 25 tables (god help me) and I really have no desire to try to get that to work in code.

But curiously, seeing it in action, it all made sense. LINQ-to-SQL is something that actually had my mouth watering. That same terrifying stored procedure I mentioned relies on an underlying user-defined function that broke the other day as well as views as well as... well, you get the picture. I'm mixing and matching business logic with data retrieval logic and I'm not happy about it. I was never really sold on having data retrieval logic in the business classes (I dallied OR mappers and was never really impressed by what they were doing for me) but I'm changing my mind now.

Autogenerated paramaterized SQL queries? Internet says that LINQ-to-SQL can actually outsmart you when it comes to query optimization... not that this is saying much when it comes to my SQL-fu. Chris gave a run-through of the features that went from what they were in clear terms to how they work in clear terms, and it all really clicked. I was pumped to go home and start ripping out that library of 250-odd stored procedures and start rewriting them in LINQ-to-SQL!
Except, oh yeah. We already have 250-odd stored procedures and they generally run pretty well. Writing them isn't hard, maintaining them isn't that bad. Sure, we've got business logic turds floating in our database punchbowl, but I'm too lazy to fish them out when I've got so many freaking bugs in the application to fix. The language features are really nice (extension methods still have my mouth watering) but by the time I was in the elevator down, I could already feel my excitement to Change The Way We Program dying as I got to thinking what Fred Brooks would say - this all is attacking accidental complexity, leaving the essential complexity of what we do untouched.

So it was the talk on Visual Studio Team System that left me the most sort of excited. The idea that the application forces you to work in sane, cohesive process (unit tested code, tracking user stories and the work projects associated with making them happen, lines of code, bugs open with the stories) seems like it'd be forcing you into dealing with the essential complexity of software development. Rather than pretend that you can crush that Gantt chart down and cut QA out of the cycle, you're forced to accept the fact that the constituent components of the project are missing in action and until that's done, the application isn't.

Nice stuff. Now all I have to do is find a company that'd understand the need for it.

Management Conundrums - The Cowboy

I read an article refuting the value of crunch time and got to wondering about it.
I've been the cowboy before, working 100+ hour weeks. Dreaming in code, pulling 24 hour days, a real Team Player. I don't want to ever go back there again. I'd like to prevent people I work with from going there too.
I found crunch time to be a vicious cycle. Rather than sit back and think my way through the problem, I dove in head-first and bumped my head on the bottom. Hard. Repeatedly.
I found myself working long hours, making mistakes along the way, and then working longer hours because I told myself "I'll just fix this later." And then working longer days because I couldn't remember the fix that I had in my head earlier but it all seemed so clear to me and in retrospect, fuck embracing death march values.

To push my street cred through the roof, I'll just quote Bun B who puts it better than I do.

Start with your head, homie, then use your hands / If you try it in reverse, you don't even stand a chance.

I don't know if it's confirmation bias on my part - I'm a pretty lazy guy (note to anyone daydreaming of hiring me: I'm totally joking about that). Am I just buying into what these people and their so-called "studies" are saying because I'm (*cough* not) lazy and want to believe anyone that's telling me that working a square 40 hours a week is really OK and even preferable?

The thing is, not everyone believes it - and this is not just managers that I'm talking about, either. As the comments on the article over on programming.reddit show (and as my anecdotes reinforce), some developers have an almost bloodthirsty need to believe in the power of crunch time. There were a couple of chaps in there extolling the virtues of brutal hours of work, stating that without doing months of crunch mode, their projects wouldn't have finished. I prodded, trying to lead them down a path to questioning whether the projects might have finished in the same amount of time without crunching for it, but it was rejected out-of-hand.

The Mythical Man-Month gave us the should-be-more-famous truism that "you can’t get a baby in one month from 9 women" - some tasks really are going to take a set amount time no matter how many resources you throw at them. The mother can start pushing on day one or wait until month nine to start pushing; the baby's not coming out any faster no matter how many months she's been pushing. I think that believing that you can accelerate nature (I'm talking development projects now not babies; work with me, people) is equal parts foolhardy and arrogant, but I also think that a good developer needs a healthy dose of both of these qualities.

When it's a team gelling and putting in the overtime because they want and need to, you let it go. That's a no-brainer. Of course, you can't go on auto-pilot. You need to figure out when they're sprinting to a dead end, when they've started sprinting too soon and have been sprinting so hard that it's counter-productive and maybe you should kick them the fuck out of the office at 5 PM on Friday so they can see their families and come back ready to do good work on Monday. You need to reel them in at some point. Like any design pattern, "crunch time" has its use. Just take care to not treat it like a golden hammer and you'll be fine.

It's the lone cowboy that I'm concerned about, and I'm concerned about what to do with them. When it's a team working to a goal, they're building together. When it's the lone person banging away at all hours, they're using their back more than their brain and will miss simple, obvious solutions to their problems and get aggravated at everyone else who isn't putting in the same brutal hours they are.

Part of me wants to save them, but then again I'm not sure that anything can be done with or for them. Reason, logic and studies seem to bounce off of them (it did off of me) like bullets off of Superman's skin.

At my current job, we had a cowboy developer. Good developer, smart guy, put in long-ass hours. Went on vacation, came back in with a custom XML business rules engine (like Drools) that used .Net Reflection and all that good stuff to drive it. It's pretty good code, just that in retrospect... we didn't really need it. I never had the heart to tell him that outright and thousands of lines of code nobody wants to refactor out of the application remain today.

When he eventually left, another of our developers quietly became the new "hard worker" to pick up the imaginary slack left behind. I don't know that I'd even have noticed this quiet change if I hadn't read The Lucifer Principle by Howard Bloom (this and The 48 Laws of Power are Dave Solomon picks for bizarroland must-read management books).

In The Lucifer Principle, Bloom relates an anecdote about a researcher's study at a summer camp. The researcher had identified four major archetypes for kids in group behavior - the "alpha male", the "bully", the "joker" and the "nerd" and conducted an experiment with them (nothing on the level of Milgram or the Stanford Prison Experiment).

The scientist assembled a cabin composed entirely of "leaders," boys who had been dominant, "alpha males" in their old groups. Very quickly, the new cluster sorted itself out according to the familiar pattern. One of the leaders took charge. Another became the bully. A third became the group joker. And one of the formerly commanding lads even became the new group's nerd.

- The Lucifer Principle, Howard Bloom, pg. 92

I know I'm doing a terrible job explaining this, but what it comes down to is that I've seen the shift in my co-worker and I don't think it's a conscious one. He may think that he "needs" to work longer hours (I've certainly danced around it trying to get him to feather the brake pedal on it, so to speak) to get things to work, but could he explain why?

Of course, self-interest on my part demands that I don't push them too hard to not kill themselves. I'm not the boss and if he's not doing the long hours, I'm sort of afraid that someone else is going to give me the stink eye because I'm not checking in code after midnight just before I leave on vacation to show that I'm a team player, never mind that the code's invariably broken in some hideous ways and people are going to have to quickly find and fix all the mistakes I've made because I was tired and rushing.

Moreover, like I said, I suspect that there's some subconscious understanding about the archetypes (for developers, read: "design pattern") of a team, and that the "overachiever" is one of them. I know just how loaded the word archetype is which is why I think it's probably an aft metaphor - if our work habits are indeed an unconscious manifestation of our identity, then there's no clean way to divorce one from the other - the cowboy's going to keep drawing for that six-shooter no matter what you do and resent you for telling him to keep it on safety before he blows his foot off. They may not be able to do it any more than you could conquer someone's fear of snakes just by bringing to their attention the fact that they are, in fact, terrified of snakes (I suspect this is an easier one to break though). Moreover, you get rid of one and another will rise to the occasion.

Unless you do what you can to make them and everyone else on the team aware of your distaste for their overwork and unless your boss (and bosses all the way up) are on board with holding people to a square forty, I doubt it will happen.

Sadly, getting everyone to throw fallacious common sense overboard and accept the fact that working smarter and working harder are just about always mutually exclusive and working smarter beats working harder just about every time isn't an easy task. If it was easy and if these work archetypes didn't exist, then everyone wouldn't be positive that today's Dilbert (for pretty much any value of Dilbert) just had to have been sent in by one of their co-workers.

But I'm no manager, so I keep on doing what I can to make things better, forty hours a week.

Marginal Utility - IsDebug for .Net 2.0

Unfortunately for my co-workers (and especially the QA department), I play build engineer because we don't have a dedicated build engineer or even build server. Small company, that's how it goes.

In recent months, I've stepped my game up. Batch files, playboy. I know how retarded that sounds - I was this close to having Cruise Control up and running, but I got fed up with the memory footprint it left on my labtop. It's not that trivial a matter when I can time out database calls by having a debugger up and running the same time as I have SQL Server Management Studio with a couple of windows open. Any way you slice it, that shit is s-a-d.

But in the past, I wasn't quite as lazy as I am now. Rather than spend a couple of hours cobbling together a full build script (dear me from 3 years ago - is it really so hard to Google csc.exe?), I did it all by hand from inside of Visual Studio. Clean out the destination directory, swap from Debug mode to Release mode, compile, zip up, yadda yadda.

But wouldn't you know it? It's so simple and yet I managed to screw it up. Multiple times. Handed QA releases that were compiled in debug mode or that had assemblies left over from debug mode and they got clued into it (because our app works slightly differently in the two modes (BY DESIGN!!!)). How was I to know? DLLs look like DLLs, people.

Eventually, I found a marvelous little utility called IsDebug out on the wild internets. Look at that! You drag them DLLs onto it and it tells you whether they're debug mode or release mode! Total win!

When I finally sat down and knocked out our app's upgrade to .Net 2.0, I was kind of bummed to find out that IsDebug didn't work anymore. I recompiled, nothing. Checked file properties, still not working.

But wait! Internets to the rescue again with the clue-by-four on what new properties I should be examining!

So I got Jeff's peanut butter in Jim's chocolate and here we are - IsDebug for .Net 2.0.

I've got a compiled version if you're lazy or don't have a compiler (I imagine this utility is especially marginal if you don't) and if you distrust stuff I write because I'm on the internet or because you know that I cut and pasted it together, there's the solution with source code for you too.

HOORAY!

What are you good at?

It's been a confluence of things that's left this question tingling in my head, begging me to ask it to just about everyone at work.
Work's been a firefight - stuff is blowing up and I've been asked to put down what I was working on to help look into it. It's technology that I don't know, code that I don't yet have a mental picture of and unknowns on all sides (our app, our hosting environment, their app, their test methodology, yadda yadda).
It's all finger-pointing and scrambling through the mismanagement playbook. Regularly changing priorities. Micromanagement on a level I didn't think was possible. Scrambling to find consultants who can "help" (from the same consulting company that built a large portion of the app, naturally) and get them put on hot standby even though we haven't been able to pin the problem down yet. More people from outside the organization being brought in to help micromanage. Scheduling people to test round-the-clock, generally frazzled nerves all around.
At some point I took a step back and started to wonder how we ended up in this mess in the first place. It sort of dawned on me that up the food chain, someone failed to ask a simple question that could have saved us all a bunch of time.

What are we good at?

This really is as simple as it sounds. If you're heading up a company, you think in terms of "core competencies." (That's a real term, right?)
You don't over-extend yourself. I can grill a mean steak and I can brew a ferocious cup of coffee, but at the end of the day I'm not going to try to write a cookbook because that'd be even more unreadable than this is.
Unfortunately, that's sort of what the situation that we're in feels like. Rather than define and focus on what products we felt we could successfully accomplish in a set period of time, we were told what we needed to get done in that same period of time.
I can see how it's a risky sell - "rather than over-extend my people and producing n working products, I'd like to focus in and produce n-2 pretty high-quality products that I have confidence that they'll be able to build, test and release." You're talking about setting us back 2 products that we could be selling. Are you out of your mind?
As a strict value prop (that's "value proposition" for those of you not in the know, another vaguely business term that I'm sure I fucked up using) the value of quality isn't easy to wrap your head around. McDonald's doesn't make the highest quality hamburger but they make bank, right?
Right - and when they've failed to ask themselves what they're good at, the market's reminded them. What ever happened to the gourmet menu that I remember hearing about 15 years ago? The pizzas? The steaks? The lobster rolls (I've heard they have them in New England but it's not like I've gone looking for them because what the shit so maybe they still have them?)? They weren't good at them. They spent a lot of time and money developing them and they ultimately ate those development costs and pulled them off the shelves.
What works for a corporation doesn't work for a developer.
When I got to thinking about what makes an object work for me, something one of my first computer science teachers told me when looking over my code came back and stuck in my head. Paraphrasing what he told me...

You should be able to describe a function in one sentence. Furthermore, when it's time for you to describe that function, the word "and" should not be in that sentence. There should be no semi-colons, no subjugate phrases, no hyphens. If you find yourself using the word "and" to describe your function, you're not describing a function, you're describing two functions.

So I discovered the joy of decomposing functions. I didn't get down and dirty by declaring every variable as final. I'm by no means writing functional code and I slip all too often and my functions do two things, but I try to refactor them when I'm able to admit to myself that yeah, that really is doing multiple things.
The code behind the application that's sometimes working like a champ, other times shitting all over the floor... not so much. Giant blocks of code. Twisty turny, deeply nested if blocks. Classes dedicated to re-doing functionality that the .Net framework had built in (if only they had asked Google).
I want to give the codebase one last hug before I put it out of its misery - when I look at it and ask it "what are you good at?", it sighs heavily and shakes its head. It doesn't have to say anything - that look it gives me is all I need to know. I've seen it before, and I'll see it again. Finally it says in a weak little voice, "I'm good at doing whatever they wanted me to do today. I think. What was I doing yesterday? I'm not even sure what I'm doing here."
It's not its fault. It's not their fault. Someone should have asked "what are we good at?" and someone should have responded "not this" and that should have been that.
We were given this weekend off (not that wild horses were going to drag me in for another consecutive day of hair-pulling). Management is good at giving us what we shouldn't even have to ask for. Maybe next time they'll learn to ask us what we can give them instead and we can produce one or two quality products rather than a bunch of garbage and save everyone a lot of agony.
Wishful thinking, right?

Sometimes Foresight is 20/20, Too

I keep trying to come up with really great, really kick-ass ideas for programs that I can sit down and knock out. I want to find that itch that I can conceivably scratch.
I obviously haven't had any.
Every now and then, someone I know will come to me with a "wouldn't it be great if..." idea that they've got, but there's two problems with them. OK, one problem. They're invariably not that great because if they were, I'd have jumped all over it because it felt like that great of an idea and I wouldn't be typing to an empty audience, I'd be dictating this in front of a sold-out crowd at Madison Square Garden because I paid for them to watch me sit here and dictate this tripe to my butler's butler after having built whatever that great idea was.
The second, obviously subjugate, problem there is that I haven't had any great software ideas (of my own or lent to me) that got my juices flowing. I haven't had much more than ideas for the sake of ideas.
Until one fateful day a few weeks ago. I'd gotten back from work and was out on a bike ride to clear my head and stretch my legs out. It was a nice enough late afternoon and climbing the hill, I started to think back to all the silly little programs I wrote when I was a young lad dabbling in BASIC and PASCAL. Terrible "RPG"s and the sort. Cut-rate Ultima Is, minus, well. Everything that would make them playable.
It suddenly became obvious - why don't I write a roguelike in .Net? Since then, I've been kicking around the idea and it's only gotten bigger and, well, dumber. As Google will tell you, the very idea of a roguelike in .Net is a dumb one to begin with. I haven't even started so I couldn't tell you why.
Compounding the shaky base I'd apparently be starting with, some truly awful thoughts have gone through my head.

"Hey, I could write a flexible MVC implementation so that it could be presented by either a web interface or a BBS door!"

Yes, another filthy childish love of mine is olde tyme BBSes and door games. I've tried to come up with a good idea for a door game to produce, but no dice there either.

"Hmm. Since it's online, I could make it multiplayer. I remember reading some design documents for Multihack; how will I handle surreal time? If there's two people on the same dungeon at the same time, how do I handle the dungeon-moves-when-you-say-so aspect of it? Can I fix this?"

Multihack was the multiplayer version of Nethack that never quite saw the light of day. If you've never played a roguelike, the game world doesn't advance until you make your move. You can take a second or a week to make that next move. This presents obvious problems when more than one person inhabits the same level and Player A wants to move NOW while Player B wants to take 10 or 20 minutes to decide whether to read a cursed scroll of levitation or enscribe Elbereth or whatever. Oh yeah, I went there.

"How do I persist the levels? Hmm. Oh, I know - this feels like a pretty relational data model that I'm working with. I'll just install SQL Server."

At that point, a bigger, louder voice in my head screamed "SQL Server for a fucking roguelike? What the shit are you thinking?" Thankfully, foresight finally reared its beautiful head.
If I may, a few words in my defense.
Day in, day out, I write database-backed web applications. I take detours to write "thick" WinForms apps here and there and then multithread them based off of need rather than just because I can (honest) so I've got a really solid comfort level when it comes to, well, ASP.Net up front, C# codebehinds and middleware and SQL Server backing it all.
That said, I recognize that it's more than OK to step out of your comfort zone from time to time. It wasn't comfortable when I took a step back and started to see that what I was trying to pass off as object-oriented code was, being generous, marginally better procedural code with some trappings of objects tacked on as an afterthought. I'd gotten comfortable writing that way as a student and hadn't progressed much since (and coworkers who'd been there before me and had more experience were writing the same).
At a certain point that felt logical to me, I stopped with the parameteritis and the basically stateless objects and started writing object-oriented code they way that I thought it should be written, trying to take care to think about what it was doing and how it would be used now (rather than building things in because you never know), but I think that I'm a better programmer and my software's better and easier to maintain because of it.
It wasn't comfortable when I got started and it was slow going at first, but I'm happy with where I've ended up.
Now that I find myself at the top of a mountain again, do I try to convince myself that I've climbed to the peak of the world or do I take a look around and see that there are other mountains that I should climb?
My class diagrams (which look every bit like a socially inept teenager scribbled them down) for my roguelike can sit in a pile on my desk for a while. A database for a roguelike feels so impossibly wrong that I can appreciate the fact that it's time for me to stretch a bit and tighten up my laces because I've got some climbing to do.

Software development - the journalism model

You probably don't know this about me, but I've studied journalism extensively. Back in third grade (for my vast international readership, that's the grade level you're in when you're ~9 years old), I learned journalism well enough that I think I wrote an article or two for my school paper. Maybe I even edited an edition of it - obviously, my journalistic credentials can't be called into question.

We learned that the essence of journalism can be found in the "Six W's" - Who?, What?, When?, Where?, Why? and, uh, hoW?. We also learned that teacher couldn't spell that day.

I humbly submit that these same Six W's should be applied a bit more rigorously to software development and project management, just in case you're trying to figure out if your project will fail or succeed (and if you're not trying to figure out, it will fail).

The Important W's

(in no particular order)

• What (are we doing)?

This one seems so obvious that it should require no explanation, but the flip side of that is that it's so essential that you're dead in the water if you and everyone else on your team can't explain what you're doing. The team should know what you (collectively) are doing in the macro sense and they should have direction in the micro sense.

If you or any of your team can't accurately (and hopefully, clearly and concisely) verbalize just what you're doing, your project's probably well on its way to being necrotic.

If you're a manager, you should be able to define achievable goals in the macro sense and the tasks needed to achieve them in the micro sense. If you haven't and if your people can't either, see you at the unemployment office.

• Why (are we doing it)?

As a lazy software developer, I pose this question to the project management team a lot more than they'd like me to. It's reflex and I think it does everyone a lot of good. I don't want to try to wield phrases I won't understand in a million years like "cost-value proposition" but they're probably applicable here.

In hackneyed colloquialisms I can wrap my pea brain around, whose itch will your software scratch? Will anyone take one look at it and immediately "get it"? Will it make people spontaneously say "this is fucking cool"? These are all really, really good things - if you know that people will think that it's fucking cool, you and your team probably does too, and will be excited and take pride in what they're doing, and the end product will reflect this.

Alternatively, will people take one look at what you've done and shake their heads and gasp in disgust at the turd you've fished out of the shitter and slapped on the table?

• Who (is doing it)?

As in - do you have the people to accomplish your goal on hand already? Or are you expecting that you'll just subcontract this out and everything will work out just fine? Do you already have a team of studs or do you expect scrubs to "step up to the plate" and get things done?

If you don't have faith in your team's ability to accomplish it, listen to that nagging voice in your head and start running for the hills.

Projects can overcome a lack of manpower, but you probably don't have the stomach or the wallet to grunt your way through it. With amorphous direction and unclear goals, even a team of ICFP-certified development gods are going to flounder.

The Unimportant W's

• How are we doing this?

This applies to those "little things" like IDE, language and framework. I don't mean to underplay the importance of all of these; you'll certainly make your work more difficult (OK, orders of magnitude more difficult) if you choose something pretty inherently unsuited for the task (RoR for an embedded system, C for a web application), but I also think that one of the other W's easily steamrolls it - Who. You put your team of studs up to the task and it stands a reasonable chance of succeeding in spite of itself.

• Where am I doing this?

This doesn't speak so much to physical location as to metaphysical location. Oh, did I just blow your mind?

By that, I mean things like version control (where the project lives), and build management. I'd lump bug tracking in here too. I know that Bill de hÓra would disagree and the developer in me wants to agree with him, but then I got to thinking - "hmm... VCS hasn't been around forever and I'm sure some working projects have teams that still pass around spreadsheets with a list of outstanding bugs on them." More importantly, the Who will overcome it - good people know good practices. If we walk into an shop without version control, we'll set it up. It's not hard to find a bug tracker out on the internets, it's a couple days to roll your own and if all else fails, spreadsheets aren't the dumbest way to go about things (OK, they kind of are). All this goes out the window once you meet a certain critical mass of project size, but the Who will obviate it again, as you'll get good people in there that will make you hire build engineers and all that jazz.

• When

Uh, now? I've lived through multiple dot-bombs, so spare me the "the market just wasn't ready for our innovation" speeches. See Why, jackhole.

Boiling it down to Solomon's Law (YES!), "Code cannot overcome reality." If the reality is that you've got an inept team, are lacking direction, are developing something that nobody wants, it doesn't make one goddamned bit of difference that you've got an A-number-one build system, magical bug tracking, distributed version control and an immaculate code base built on the right language and framework for the job.

No matter how beautiful the package, you're still boxing up a turd.

Oral Traditions in Software Development

The requirements document will be the death of me. I don't have a whole lot of faith in the requirements document if only for the fact that I've yet to work with one that concisely and accurately expresses the needs and intent of everyone that it represents.
I can't stress this highly enough: I'm not very smart and I don't have much of an attention span. That's why "concisely" matters so much to me. You can make that requirements document accurate, but in the process you'll probably end up with something resembling the Yellow Pages for Manhattan. It's accurate up until the point where it puts me to sleep (about three paragraphs that resemble legalese in) and sooner or later, people will get hip to the fact that I didn't read it.
OK, I'm taking that a bit too far. I do read it as much as I think I need to in order to get the flavor of what I'm developing. People sweat way too much over getting into the down and dirty of how things are supposed to work, pre-chewing the details. For downstream consumers of this product (QA, client-facing folks and beyond) maybe this is a good thing? For me, I just want the broad strokes and I'll be able to fill in most of the blanks.
To fill in the gaps between the broad strokes, there's back and forth where I talk it out with the person who wrote the requirements document, because they know what we need better than I do. This works out great for everyone as we quickly hash out details and get back to what we need to do.
Well, it's great until months from now, when QA will have a new set of questions that weren't reflected in the original documentation because they didn't sit in the series of little meetings that development had with the requirements crew. And we'll have another sit down and chat and they'll see what the real intent of the functionality is or I'll go back to the drawing board.
Here's the thing - I really like how this works. Little meetings (30 minutes or so) when needed and when the particulars can fit it into their schedule. I don't have to read much of the documentation which is good for so many reasons, but mainly because people can tell me what they need better than they can write about how things should work.
But here's the other thing - I can see how one would look at this and be calculating a bus number of "one." The person that wrote the requirements document leaves the company and all of a sudden, you're back to just that dry document that's a subset of the expertise that they brought to the table. The developer leaves the company and all the discussions that the product developer had with them that they never inserted back into the original document disappear with it. People are left to scratch their heads over what's going on in with the application, not knowing that this is expected behavior.
Or QA or someone else walks into the application with the expectations laid out by the requirements document and finds that the fantasy of the documents doesn't meet the reality of the application.
I don't like documents.
I like talking things out.
I have no idea how to blend these two competing desires and turn what amounts to an oral tradition into a sustainable development model but if I figure it out, I figure I'll be a billionaire and then it'll be time to settle down to my life's ambition, hunting the most dangerous prey of all - MAN. Just fucking with you, I'll be so rich I'll pay someone to hunt MAN for me.