Catfish in the Memepool

Devolution in the Air

Over the years, one of of the my favorite OOPSLA pastimes has been attempting to discern the Zeitgeist. What memes are moving, as the marketing 'droids might say. This time around, several distinctive themes emerged. I'll discuss the most vivid of them here. I'll talk about the others, Web Services, the Reflection and Metalevel Architecture Revival, and Refactoring Coming of Age, under separate cover.

The first, and most prominent / evident was Software Devolution.

By this I mean an overdue discussion of the death of quality, the embrace of failure, the acceptance of uncertainty, our exhausted capitulation in the face of complexity, the realization that the programme of the close-world formalist is mocked by reality.

Acknowledging devolution means acknowledging that not only are things worse than we've dared to admit, they are getting worse, and will continue to do so, at an ever accelerating pace. We are entering a worse-is-better world of sweatshops and generative robo-code, where the craftsmen must inevitably retreat to their redoubts in boutiques and curio shops.

Acknowledging devolution means abandoning our fetish for correctness, and learning what to do when (not if) something goes wrong.

Devolution was a theme that impinged from so many directions this year that it was impossible to ignore it. Software Devolution is something of an umbrella under which I am lumping a number of disparate trends. Perhaps the notion will seem more clear once I've cited some of the developments that made this theme seem palpable.

The most glaring evidence of this theme could be seen in Martin Rinard's Breakthrough Ideas presentation. Rather than cling to the notion that every error can even be caught, let alone avoided, Rinard deals statistically with consequences and ensuing carnage entailed in just "eating" them. The approach would seem to have more than a little in common with what the genetic programming people are doing. I need to look at his work more closely to see if this is so.

Embracing the reality of programmer fallibility and failure was a theme of Mary Beth Rosson's The End of Users keynote. She observed that something like ninety percent of all spreadsheets have errors, and that we are in something of a state of denial of the potential consequences of this reality.

Agile methodologists have been promoting the primacy of bottom-up, feedback intensive code and test based design as an alternative to the top-down foresight fetish crowd's traditional ideas. Martin Fowler's Finding Good Design keynote went beyond even this, and offered more than a nod in the direction of whether high-minded design ideals were honored primarily in the breach in practice, and questioned whether the very idea of design would retain its relevance.

For a moment, we all dared to ponder the question of whether craft in design is anything more than gilded filigree, soon to be as anachronistic as a hand-crafted circuit board.

My own Breakthrough Ideas triptych (previewed exclusively here a few weeks back) further reinforced the notion that the gap between what we practice and what we preach is large, and is growing larger. In Praise of Cut and Paste sought to "out" an ubiquitous practice among framework designers and, well, everyone else, that has been regarded as something of a dirty little secret by practitioners, while being scorned by high road methodologists and academics alike.

Big Bucket of Glue suggested, among other things, that integration isn't pretty, that it is often quick and dirty, that essential complexity is here to stay, and that some part of the system must bear the structural stigmata associated with wrangling the rest of the system into a coherent whole. It can be seen as a potential integration layer sequel to our hoary spasm of pomposity Big Ball of Mud.

The End-to-End Principle in Programming Language Design suggested that mainstream languages such as Java focus too closely on minutiae like protection and type checking at the micro level while neglecting any support for the overarching end to end of real, honest to God runtime facilities such as validators, to, well, who knows? Features designed badly at the language level must often be reinvented a level up in a dynamic or adaptive object model layer in order to properly accommodate the needs of any end to end domain specific languages that might emerge. One example of where this phenomenon can be seen the architectural eyesore that the services layer is becoming. It is bad, and getting worse by the minute.

Devolution can be seen in the enthusiastically received Scrapheap Challenge workshop, where participants dumpster dive scraps from the web as fast as they can to solve programming problems.

There was something decidedly devolutionary in Grady Booch’s overdue exhortation to cast models aside, and in his candid admission that no one reads code anyway.

The winners of our Sudoko exercise in the Extravagaria III: Hunting Creativity workshop used twenty-first century scrapheap techniques (they Googled the problem), mocking the attempts of the organizers to demonstrate genetic crossover using the other two groups who'd treated the task as relatively solitary chores.

The notion of "Software Devolution" is drawn from the colorful hatchet job James Noble did on my Biography for the Sashimi vs. the Purple Robots panel. If the shoe fits. It is a notion that fits wonderfully under the aegis of the Post-Modern School of Programming founded by Noble and fellow traveler Robert Biddle at the inaugural Onward! a few years back. We are All Devo indeed.

Among the mantras of this movement are paeans to the centrality of The Program as the essential focus of computer science. Code is seen not only as the source of choice for interesting problems to study, but as the remedy of choice for addressing them. We should address problems with code with code, like real computer scientists, and not with closed form analysis, like mere mathematicians.

Implicit in this is the hope that our discipline is finally becoming mature enough to confidently address indigenous subject matter with indigenous tools and ideas, rather than seeing ourselves as the outcast spawn of some tawdry, long forgotten one-night stand involving the mathematics and physics departments.

The primacy of code, and of runtime mechanisms as a condign recognition of the intricacy and fallibility of code was a central theme of my own closing address to the Dynamic Language Symposium. More of that elsewhere, soon, I hope.

Devolution can be seen in the decline in the impulse towards Linguistic Imperialism, the cooling of the language wars, the increasing irrelevance of Utopian Linguistic Monocultures, and the grudging acceptance that ours is a heterogeneous polyglot world, where applications are increasingly expected to be able to cooperate.

A devolutionary cast can be seen as well in the impulse to use generated code in place of code crafted by human beings. Is generation a symptom, a code smell, signaling the inadequacy of our languages and tools, or is it a harbinger of things to come, the wave of the future, a sign that the John Henrys with their TTL drivin’ keyboards will ultimately be vanquished by these generative steam drills.

Indeed, Richard Gabriel is suggesting that the ultrascale volumes of code that our futures will demand can only be supplied by an automated process that relies on increasing greater degrees of abstraction. This prophet of a grim Worse is Better world may find himself once again the reluctant head of yet another vanguard.

Devolution has two somewhat conflicting connotations, both of which, in the right context, in the right light, fit this discussion to a tee. Devolution suggests de-evolution, revanchism, neoteny, and regression, a descent towards something worse than what we have now. Not a good thing. Devolution also connotes decentralization, a surrender to local control, a process of increasing federation, a spinning-off of constituent parts, as seen in the establishment of a Scottish Parliament, or the refactoring of the Soviet Union into Russia and friends. Think of a process of ball-of-mud componentization, or an annealing process that lets local clusters emerge. Not necessarily retrograde, not necessarily a bad thing at all.

Devolution could be seen as well in the conference organization itself, where satellite symposia offered an alternative to the "take my ball and leave" secession of the Aspects and Agile movements. The adjacent RubyConf, and embedded Wiki and Dynamic Languages events brought fresh plankton to what could have otherwise been a stagnating meme pool, and may point the way to a more federated future for long-in-the-tooth conferences like OOPSLA.

Is devolution a natural consequence of growth? Of maturity? Of riding an exponential? Of Variety and Diversity? Fratricde ebbs because there is more room, and a more diverse herd, self-sustaining communities calve from the central glacier, and set out on their own…

Posted by foote at October 22, 2005 08:15 PM