Closer To The Ideal

Darren Hoyt got me thinking about who is online, with his remarks about RSS feeds:

Google Reader’s popularity is surging and the number of people consuming web content via RSS readers has grown overall, but no one’s claiming RSS is a mainstream concept just yet. It’s mostly the geeks and early-adopters who know what it is and what to do with it, thus they don’t need to be sold on whether a blog offers an RSS feed—their feedreader can auto-detect it based on URI anyway… Yet so many theme designers use valuable screen space to display a prominent (and mysterious, to most) orange RSS icon. Non-techy users will either avoid the icon or click it and be taken to a page of XML gobbledygook.

I started to wonder about non-techies and when and if they will ever think it is natural to subscribe to RSS feeds. I wrote this as a comment on Darren’s site:

My mom is in her 70s and over the last 3 years she has become a regular reader of Google News. I set it as the homepage on FireFox and IE on her computer. After I’d set it up, she surprised me by exploring it enough to learn how to customize it. Since she is fluent in French, she told Google News that she wanted news in both English and French. She now gets titles in both languages.

More recently, she’s become a regular reader of DailyKos. For years my brother has sent her links to occasional posts on DailyKos, but only recently has my mom become a regular reader. Assuming she eventually gets interested in other political blogs, I imagine there is an RSS feed reader in her future.

There are young people whose reading habits were shaped during the era of the Web. They’ve grown up getting their news online. And every year they are another year older (eventually they become that demographic that advertisers value most). On the other hand, there are certain paper magazines that have what I’d guess are slightly older audiences. Think of magazines about fishing or boats. The audience for these magazines will, I suspect, be slowly eroded by the generational shift. To me, the conclusion seems to be that eventually whole categories of print-based magazines will be endangered by their online rivals. If this decade saw the maturation of the tools needed to build complex, dynamic websites, the next decade is likely to see large scale shifts in how the public gets its news. RSS feeds seem like an obvious part of that shift.

At the end of the 80s, tech oriented college kids went onling (the internet was available on all campuses, though it wasn’t yet available to the general public). During the 90s, tech-oriented youth went online. Tech-oriented gay teens, in particular, began to find an escape from harrassment online. I’m using the phrase “tech oriented” rather than “affluent” or “college educated” or any other such phrase. danah boyd has written of the difficulties of defining social classes in America. People involved with the tech industry were among the first online, for obvious reasons. During the current decade, the web became an important resource and support for new moms. People with non-majority political viewpoints found the Internet an important place to converse and organize. Professors began to build reputations online (though the majority of professors still don’t have weblogs and the web has not replaced the peer-reviewed process). Law professors started posting essays on their weblogs, a few of which were cited in court cases. A new generation of journalists found they could launch quite successful careers through their weblogs. The news industry was transformed. The gambling industry was also transformed, and to the extent that we can consider the sports industry separately from the news industry, it too was transformed.

Except for the professors and the tech crowd, most of the other groups that went online had in common a sense of isolation. New moms, gay teens, and folks with minority political viewpoints greeted the web as a place where they could finally connect with others. Gamblers too, for different reasons.

Each year that passes sees the web become more central to people’s lives. This decade saw an explosion of online social networks, such as MySpace and Facebook and, for professional contacts, LinkedIn. People nowadays often meet their romantic partners online.

When the economy recovers from the current recession, society’s acceptance of the web will enter a new phase. The early adopters have all adopted the web, and the middle part of society has mostly embraced it too. The late adopters will be increasingly online over the next few years.

Much of the content now online is subsidized by existing offline publications. For instance, the New York Times online is subsidized by the offline version of the paper. My mom is a very great fan of the New York Times. She subscribes to the paper version of it and she loves to read it. Myself and my brothers only read the New York Times online. So, to some extent, older folks like my mom are subsidizing the reading habits of younger people. As the demand for the older, offline product shrinks, the ability of the offline product to subsidize the online product will also shrink. Personally, I think the New York Times will survive for a very long time to come but other publications will disappear. Some very small niche subjects will likely only be able to support online publications. For online publications, the exciting possibility is how much additional revenue will become available to them.

There are many groups that are slow to adopt technology. There are a few groups who feel that technology, by itself, is contrary to their mission or ideology. My experience with iHanuman.com reminded me that there is a great resistance to technology among some who are students of yoga (some yoga instructors insist that sitting in front of a computer is bad for your body and therefore contrary to yoga). My experience with The Second Road has taught me that the recovery community is only slowly moving online. In fact, in many ways the online recovery community resembles where political blogs were about 6 years ago – they all have small audiences, the overall audience is probably growing quickly, especially for younger folks.

I recall a Second Road meeting where someone asked why anyone would go online to seek support. We were discussing young people (under age 25) with a counselor who worked with youth. We wondered, if a 20 year old is at a party and someone pulls out meth, and that 20 year old is trying to stay clean, won’t they simply call someone else in recovery? Don’t all young people have cell phones, and don’t they use them all the time? The counselor corrected us. Most young people have limited cell phone plans. They are constantly having to save their minutes, or they are out of minutes. Also, at a party, it is much more socially acceptable to say “I’ve got to check my email, can I borrow your computer?” than it is to say “I’ve got to call my sponsor, can I borrow a phone?”

Middle class Baby Bomers are now entirely online, and they now organize their social lives around the web almost to the same extent as the younger cohorts do. Most of the big web companies that have so far thrived have been built or lead by Baby Boomers (early adopters, to be sure). Even those Baby Boomers who missed the first wave of the web eventually joined up. I’ve known Baby Boomers who made a lot of money selling things on Ebay or Amazon. I’ve known Baby Boomers who’ve met their current romantic partners on the web.

The groups that haven’t yet moved online are the ones that have so far had some reason to resist computers, or some obstacle keeping them from the Internet. Rural areas, which are only now getting broad band, poor people and older people are major demographics who’ll be moving online during the next few years.

If a technology is truly disruptive, then it will cause the bankruptcy of many large, established firms. Over the last 15 years, since the web did not cause large-scale bankruptcies for large, established firsm, some people began to doubt whether it was truly disruptive. But consider the way that Clayton M. Christensen first defined a disruptive technology:

New performance introduced by a disruptive technology, which typically begins at a lower level of performance, but rapidly improves until it meets the majority of customers’ needs.

It’s possible that the web is only now reaching the point that it becomes truly disruptive. Many people have rejected it up till this point. The way of accessing it has been through a personal computer, which has been expensive and cumbersome. And broadband connections were slow to spread, especially in the US. Perhaps we are only now arriving at the critical moment of ubiquity.

All the remaining groups, all the factions of the population that have so far resisted the Web, or been unable to reach it, are now (over the next 5 years) likely to be brought online by the collapse of offline alternatives. The massive bankruptcy of the old ways of doing things will leave these groups with less and less options. For those groups who wanted to be online but were kept from doing so, more and more options will probably ease their path.

Conclusions: over the next 5 years, one of the biggest opportunities for content sites will be those that aim to directly replace existing paper-based content. Such sites were much hyped and much predicted in the mid 90s, but it turns out those predictions were premature. 13 years later, I believe we’ve arrived at the moment for such predictions to come true. No one person can be aware of all of the possible demographic niches that are now ready for content via the Web. I am aware of a few possibilities, which I’ve mentioned above. I believe now is the right time for a new wave of investment in content sites.

Alex Marshall has up a great essay at Spotlight on the Region. He suggests large construction projects are now taking longer than they used to:

It took four years to complete in 1904 the city’s first subway system, much of it hacked away using pick axes and mules.

It took less than a decade to complete in 1915 the central water line from the Catskill Mountains to New York City, a mammoth 92-mile aqueduct and tunnel system worthy of the Roman Empire.

It took just over a year to complete in 1930 the Empire State Building, the tallest building in the world by far at the time, an intricately detailed structure of tiles and brick.

Today things generally take a lot longer, whether it’s the public or the private sector at work.

The first portion of the 2nd Avenue subway, which will be built using state-of-the–art tunnel-boring machines, will take six years to complete – if all goes well and enough money is found. Running from 96th Street on the Upper East Side to 63rd Street, it will have just three new stations, spaced every ten to 14 blocks.

Compare that to the city’s first subway line, the IRT, which ran from City Hall up to Grand Central, over to Times Square, and then up to 145th street. The IRT had 28 stations as well as local and express lines. And it was all built in just four years.

So just to make this contrast clear, the first phase of the 2nd Avenue subway in terms of track is a tenth the size of the original IRT and has three stations as opposed to 28. Yet it will take six years to complete as opposed to four for the much larger IRT. Why is this? …

Almost every engineer or administrator I ask about this seems to agree that things do take longer today, but explanations vary. The usual suspect is environmental and regulatory review, but I’m not including that in my perspective. I’m just talking about actual construction time.

The modern, affluent standard of living depends on society’s infrastructure, both physical and intellectual: airports, highways, bridges, ports, telecommunication networks, databases, power plants, and the software that makes it all run. 150 years ago it was still common for massive bridge projects to meet with catastrophic failure. The engineers of the 1800s slowly mastered the art of working with iron and then steel. The projects grew in scale, and the skills of the project managers needed to keep apace.

Nowadays we rarely hear of catastrophic bridge failures. This is a particular type of infrastructure that has come to be well understood, both in its engineering and in the project management that oversees its construction. However, we, as a society, are still struggling to find the right way to develop large software projects. This is the newest type of infrastructure, and the correct way to engineer it and manage its development is still a source of controversy.

Robert L. Glass has compiled a book of entertaining and instructive stories regarding the massive software project failures of our time: Software Runaways: Monumental Software Disasters. Anyone with an interest in the management of software projects should read it.

The worst of these project failures, the most expensive and the most ambitious, was the attempt made by the Federal Aviation Authority to modernize the computer system it uses to keep track of what planes are in the air. The effort began in 1981 and ended in complete failure in 1994. The government hired IBM to do the actual work, and over the course of 14 years, IBM burned through $3.7 billion dollars. Nothing was accomplished. The project was finally shut down by Congress. Nothing came out of the project, not a single piece of software, nor even a line of code, was ever used for anything.

Robert N. Britcher, who was involved with the project, offers a full write up of the project. I will try to entice you to buy the book with some excerpts:

The Advanced Automation System began, in concept, in 1981 and ended in 1994, “terminated for convenience” by the government. Billions of dollars were spent on it. It is hard to describe. You can’t learn anything from the name. You know it’s about air traffic control because I told you, or because you read about it in the papers. Maybe part of the problem was the name. It sounds like the system to end all systems.

One engineer I know described the AAS this way. You’re living in a modest house and you see the refrgerator going. The ice sometimes melts, and the door isn’t flush, and the repairman comes out, it seems, once a month. And now you notice it’s bulky and doesn’t save energy, and you’ve seen the new ones at Sears. So it’s time. The first thing you do is look into some land a couple of states over and think about a new house. Then you get I.M. Pei and some of the other great architects and hold a design run-off. This takes awhile, so you have to put up with the fridge, which is now making a buzzing noise that keeps you awake at night. You look at several plans and even build a prototype or two. Time goes on and you finally choose a design. There is a big bash before building starts. Then you build. And build. The celebrating continues; each brick thrills. Then you change your mind. You really wanted a Japanese house with redwood floors and a formal garden. So you start to re-engineer what you have. Move a few bricks and some sod. Finally, you have something that looks pretty good. Then, one night, you go to bed and notice the buzzing in the refrigerator is gone. Something’s wrong. The silence keeps you awake. You’ve spent too much money! You don’t really want to move! And now you find out the kids don’t like the new house. In fact, your daughter says “I hate it”. So you cut your losses. Fifteen years and few billion dollars later, the old refridgerator is still running. Somehow.

At $3.7 billion, the Advanced Automation System was one of the largest civilian computer contracts ever; maybe the largest. It was the largest single contract in IBM’s history. From the moment it was awarded, until near the project’s demise, IBM patted itself on the back. There was something for everyone, beginning with a great ball in Union Station, featuring Chubby Checker and “The Twist”.

At its peak the project employed over 2,000 people. About a million dollars a day. If you thought like the IBM project manager, this was a good deal. Many people were working and money was being made. It was going to last forever. No one considered that it wouldn’t. And everyone was getting ahead. (One of the ironies of conceptual work is that it is easy to believe you are farther along than you are.)

The AAS must have been the most supervised project in history: this atop its enormous size and complexity, and the extreme and constantly changing requirements. One programmer described it this way: “Working on the project was like working on a car inside the garage with the motor running. Eventually, even the crickets hopping around the tires suffocate.”

What I saw on the FAA’s Advanced Automation System would have made Sisyphus weep… Whatever commitment and discipline there was… was worn down by a battery of watchfulness that I can only ascribe to fear of failure. In spite of tens of millions of dollars spent on new computers for AAS, the most important piece of equipment on the project was the overhead projector. There were endless meetings attended by dozens of people – as if we were never quite sure about the whole thing. The people in charge simply lacked the confidence and the finesse of the space team: NASA, contractors, and astronaughts.

It has been noted by everyone from the New York Times to the Vice-President of the United States that the main problem on the Advanced Automation System was “changing requirements”. For those involved in large-scale computer systems, that is nothing new. No one can perfectly surmise the shape and feel of a system years in advance. Even replacing some aspect of a system you know by heart is not immune from thinking twice about it. … [But] the requirements churn (it was called) on the Advanced Automation System was not normal. It was the result of our enchantment with the computer-human interface, the CHI. The new controller workstation, fronted by a 20″ by 20″ color display, because it was capable of seemingly endles variety of presentations, mesmerized the population of AAS like the O.J. Simpson trial mesmerized the nation…

The project was handed over to human factor pundits, who then drove the design. Requirements became synonymous with preferences. Thousands of labor-months were spent designing, discussing, and demonstrating the possibilities: colors, fonts, overlays, reversals, serpentine lists, toggling, zooming, opaque windows, the list is huge. It was something to see. (Virtually all of the marketing brochures – produced prematurely and in large numbers – sparkled with some rendition or other of the new controller console.) It just wasn’t usable…

The cost of what turned out to be a 14-year human factors study did not pay off. Shortly before the project was terminated a controller on the CBS evening news said: “It takes me 12 commands to do what I used to do with one.” I believe he spoke for everyone with common sense.

Rummaging through one of the closets at the far end of the hall on the fifth floor one day, looking for some standards document, I found an envelope left by someone who left the company – as many did after so many years advancing against stone, while the wheels of commerce were accelerating on what everyone referred to as “the outside”. It contained “A Brief History Of The Advanced Automation System”. It was printed by hand and left, perhaps inadvertantly, or perhaps with the hope that some anthropologist might some day discover it and make a pronouncement. In every important way, it is the truth:

“A young man, recently hired, devotes years to a specification written to the bit level for programs that will never be coded. Another, to a specification that will be replaced. Programmers marry one another, then divorce and marry someone in another subsystem. Program designs are written to severe formats, then forgotten. The formats endure. A man decides to become a woman and succeeds before system testing starts. As testing approaches, she begins a second career on local television, hosting a show on witchcraft. An architect chases a new technology, then another, then changes his mind and goes into management. A veteran programmer writes the same program a dozen times, then transfers. The price of money increases eight times. Programmers sleep in the halls. Committees convene for years to discuss keystroking. An ambitious training manager builds an encyclopedia of manuals no one will ever use. Decisions are scheduled weeks in advance. Workers sit in the hallways. Notions of computing begin in the epoch of A, edge toward B, then come down hard on A + B. Human factors experts achieve Olympian status. The Berlin Wall collapses. The map of Europe is redrawn. Everything is counted. Quality becomes mixed with quantity. Morale is reduced to a quotient, then counted. Dozens of men and women argue for thousands of hours: What is a requirement? A generation of workers retire. The very mission changes and only a few notice. Programming theories come and go. Managers cling to expectations, like a child to a blanket. Presentations are polished to create an impression, then curbed to cut costs. Then they are studied. The work spikes and spikes again. Offices are changed a dozen times. Management retires and returns. The contractor is sold. Software is blamed. Executives are promoted. The years rip by with no end in sight. A company president gets an idea: make large small. Turn methods over to each programmer. Dress down. Count on the inscrutability of programming. Promote good news. Turn a leaf away from the sun. Maybe start over.”