Breaking the Hype Cycle

Like a good .Net drone (haw haw), I rambled on down to Waltham, Mass. this past Thursday to get introduced to the brave new world of .Net 3.5 and Visual Studio 2008.

I found myself going through various phases of confusion, boredom and then (eventually) excitement. Confusion over David Chappell (sadly, not the sometimes Rick James impressario) and his speech there. Given that he's written books on stuffs, has a consulting firm and talked about opening for Bill Gates at some development conference, I take it that he knows his thing. That's great and all, but... this was a developer's conference or so I thought? The man can give a good speech, but I'm still clueless as to why he felt the need to sell developers on the .Net ecosystem as a viable platform. Preaching to the converted and all that.

Todd Mancini came out next to give a talk on Visual Studio Team System. When I first started reading about it, it felt like it was just another enterprise Rorschach test but seeing concrete examples of what it does (how many use cases have been fulfilled, how many have been tested by QA, how many have unit tests, code churn, yadda yadda) I can see that behind the scenes it's a pretty slick little piece of work... more on that later.

I was remarkably confused by the talk that Allan da Costa Pinto gave. I'd signed up for a couple of sessions on .Net 3.5 and Visual Studio 2008 and for some reason got treated to an hour and a half on Windows Workflow Foundation and the Windows Communication Framework. He gave an OK enough presentation, but goddamned if I could tell you what either of the technologies are. Repeated over and over is that "you can paint your business processes" but what drawing a workflow diagram's achieving me is still beyond me. I get the feeling that there must be a lot in there, but I also have no idea what's in there or why I'd want to use it. Maybe it's targeted at a different user space or maybe it's a solution in search of a problem? I do know that you can start a process and wait up to 30 days to take the next step in it! Yes, it means fuck all to me too.

Chris Bowen, the last speaker of the day, was finally the guy who got me excited to have gone down there. Away with the 10,000 foot view of what's new with the .Net platform (too goddamned much to follow) and into the new language features (anonymous functions! anonymous types! lambda expressions! extension methods! all that good shit that perversely makes perfect sense since I've been embarrassing myself by trying to wrap my head around Haskell and functional programming) and a good, quick, runthrough of LINQ.

When I first heard about LINQ, it sounded kind of retarded to me. Hey look, it's sort of SQL. Except the select clause comes at the end and it's uglier? No thanks! How is this supposed to do anything for me? Why would I want to write not-quite-SQL and then deal with object/relation impedance on top of that? I can see it working for CRUD, but I've got joins that pull together 25 tables (god help me) and I really have no desire to try to get that to work in code.

But curiously, seeing it in action, it all made sense. LINQ-to-SQL is something that actually had my mouth watering. That same terrifying stored procedure I mentioned relies on an underlying user-defined function that broke the other day as well as views as well as... well, you get the picture. I'm mixing and matching business logic with data retrieval logic and I'm not happy about it. I was never really sold on having data retrieval logic in the business classes (I dallied OR mappers and was never really impressed by what they were doing for me) but I'm changing my mind now.

Autogenerated paramaterized SQL queries? Internet says that LINQ-to-SQL can actually outsmart you when it comes to query optimization... not that this is saying much when it comes to my SQL-fu. Chris gave a run-through of the features that went from what they were in clear terms to how they work in clear terms, and it all really clicked. I was pumped to go home and start ripping out that library of 250-odd stored procedures and start rewriting them in LINQ-to-SQL!
Except, oh yeah. We already have 250-odd stored procedures and they generally run pretty well. Writing them isn't hard, maintaining them isn't that bad. Sure, we've got business logic turds floating in our database punchbowl, but I'm too lazy to fish them out when I've got so many freaking bugs in the application to fix. The language features are really nice (extension methods still have my mouth watering) but by the time I was in the elevator down, I could already feel my excitement to Change The Way We Program dying as I got to thinking what Fred Brooks would say - this all is attacking accidental complexity, leaving the essential complexity of what we do untouched.

So it was the talk on Visual Studio Team System that left me the most sort of excited. The idea that the application forces you to work in sane, cohesive process (unit tested code, tracking user stories and the work projects associated with making them happen, lines of code, bugs open with the stories) seems like it'd be forcing you into dealing with the essential complexity of software development. Rather than pretend that you can crush that Gantt chart down and cut QA out of the cycle, you're forced to accept the fact that the constituent components of the project are missing in action and until that's done, the application isn't.

Nice stuff. Now all I have to do is find a company that'd understand the need for it.

Agile Development - Can We Stop Designing?

Over on some programming forums I frequent, someone posed the following question that resulted in some heated development (from Agilists and others).
"We have a team that tells us that because they have adopted 'agile development' they no longer need to do any design work before writing code.

To me this seems bizarre, I can't fathom how one could build any reasonably complex application without a modicum of design (hell, even pencil sketches of UIs or something), but they claim it works.

Is it reasonable to develop complex applications with zero design work?"

BUFD (Big Up-Front Design) has gotten a hell of a black eye, but damned if I can think of a better way to wrap my head around a project in a business domain that I'm completely unfamiliar with.
Generating documents is a mixed bag - I think they're great for the process you use to create them (collaborating with everyone involved in the project, picking brains, exposing flaws and needs and hopefully constraints early) but I think they're awful for the fact that the end product is a document that's outdated a week into the development process (give or take; the point where the bulk of your time is developing rather than documenting) and at that point, the different partners have vastly different ideas about what the document means.
Developers think it's a rough roadmap (do I update the documentation with the way things will work in our system? I'll get that later... OK, not really. Have a meeting? That gets old REAL fast), most everyone else (project managers/QA/stakeholders) think it's litany and that's a sure-fire recipe for friction.
Eschewing the design portion of this and flying by the seat of your pants elevates this from an exercise in friction between developers and, well, everyone else to an exercise in sure-fire catastrophe for everything but the most trivial of implementations.
This isn't to say that reasonably complex applications can't be done without BUFD - I'm sure that they can, but with a big fat caveat: the developers have had a reasonable amount of experience with the business/application domain that they're developing for. There you can come away with a working application in the void of BUFD but with yet another gotcha: no one has any solid expectations of what to expect when the developers pull away the veil.
If you're working in an "agile" fashion, I guess this means that you're going to have rapid iterations from developer to project managers/stakeholders at which point they'll yea or nay it and talk about what else needs to be done, which probably means throwing away a fair bit of what you're doing, which means that instead of doing BUFD, you're doing Smaller Bits And Pieces Of Design All Allong (SBAPODAA). On the upside, there's that boost of "look what we're getting done". On the downside, there's that "holy crap I had to throw out another fucking day's work because Joe Blow From Product Management couldn't have told me yesterday that's not how things work?" and on the real big downside, there's the very real possibility that you're going to scuttle the project wholesale because developers are arrogant and think they know how to do everything so much better that you end up with a brilliant application that just happens to be absolutely useless for what the stakeholders need it to do.
I'm pretty sure that pair programming doesn't solve that one.