Closer To The Ideal

Some fellow posted a commerical notice to the comp.lang.php newsgroup. For a mere $79 per developer, you can get a PHP framework that does all the same stuff as the free, open-source PHP frameworks. And it has all the same problems, too. In an old post on the Category4 weblog I was ranting about the problems that stem from generating HTML from inside of PHP code. When a PHP framework generates HTML and CSS from inside the PHP code, then programmers end up working on problems that designers could solve much faster. Consider this problem, which the PHP Developers Kit offers as an example of all the wonderful things you can do if you use their $79 framework:

The workhorse of PHP Kit is the container class . It is a box containing other objects, like text, links, paragraphs and even other containers. It is under CSS control so you may create a container using several dozen options, like background color, border style, border color, font size and font color.

Note: the containers above displayed correctly in Microsoft IE when displayed inline (left to right). However, Mozilla Firefox version 1.5.0.4 displayed them left to right but with no width or height except for the width of their borders. When displayed as block (one above the other), Mozilla displayed their size correctly. The display property (inline or block) is not suppose to effect an element’s assigned width and height.

So, um, you really should be able to control these containers from your code, but when you run into a problem, such as different browsers interpreting your CSS differently, you (the programmer) are stuck solving it. The designer, who specializes in CSS, can not solve your CSS problem for you, because they don’t know how to edit the PHP code that generates the CSS. Ideally, all CSS issues should be the domain of your designers, not your programmers, but when all the CSS is controlled by PHP code, then solving this problem is shifted from the designers to the programmers. This leads to workflow problems – in too many web design shops the programmers have too much work to do and the designers too little, and frameworks such as this are to blame. This is bad economics – to the extent that your programmers are paid more than your designers you should be trying to shift work from programmers to designers.

Concentrating all of the important design work into the hands of the programmers is also bad for morale – I’ve worked in shops where the designers had no work to do because they were waiting for the programmers to finish a bit of PHP/CSS template code. It is demeaning and embarrassing for the designers to have no work. I’ve seen them try to make busy work for themselves and they always look a little embarrassed and ashamed. A well-run web design shop will strive to avoid this.

Designers are focused on communicating with other humans, programmers are focused on communicating with machines. One’s users and clients suffer when the actual power to design is taken away from designers and given (even accidentally) to programmers. It is a rare programmer whose insight into user behavior matches that of a highly skilled designer. An intuitive sense of the user experience is the most important skill that a good designer brings to their work. If all the real design power is in the hands of the programmers, one’s websites and web software are likely to be less intuitive than what might have otherwise been achieved. By contrast, a framework that maximizes the power of the designers is a framework that maximizes the chances of a great user experience being achieved.

Again, let me repeat (as I said in that previous post), frameworks like this (I mean the one I’m criticizing) are fine for the individual freelancer who does all their own programming and design work. But these frameworks are harmful to web design firms that work on large sites and who, therefore, need programmers and designers to work together. As soon as you have a division of labor, you need a framework that facilitates workflow, rather than blocks it.

The last time I wrote on this issue (back when I still worked at Category4) a fellow named Matt responded:

While I generally agree on your view of no html in your php, I think there are exceptions. It’s really about choosing what for when and where. For example, form elements being generated by code is acceptable to me. Even as a designer, I would think that looking at a page with a few php/template tags would be better than a page with lots of html; it’s still code. Less control, but as long as the code is doing what you want then there is no worry. And you could always either override/extend the classes generating the code or hack it…. I think that my point is that one could continue to mull over what to use and prematurely optimize resources to the point of getting nothing done.

I’ll say again, if an individual is doing both the programming and the designing, then putting HTML into the PHP might speed things up for them, but as soon as the jobs of programmer and designer are split, then anything that robs designers of their power is likely to lead to work-flow problems. And those cost money, time, morale and sometimes management focus. They also limit the ability of the designers to achieve a great user experience. Matt suggests that form code can be generated by PHP code, but if you do that, you will very likely get cookie-cutter forms. Forms are the single most important area for designers to innovate a great user experience, but the ability of designers to innovate is wiped out if the HTML is generated by PHP code.

As to “you could always either override/extend the classes generating the code or hack it”, only a programmer can do this. Designers can’t do this on their own. If the designers need this done, then they need to wait on the programmer to do it. And then, once again, you end up with a work-flow problem that damages the productivity of the firm. Either the firm’s profits will be reduced or the costs must be past along to the client, driving up what they must pay for the project.

As to “one could continue to mull over what to use and prematurely optimize resources to the point of getting nothing done”, I’m sorry I haven’t stressed this, but the opinions I’m expressing have grown up on me slowly over the last 5 years. In 2002 I started work on a PHP/MySql content management system and I made every mistake that I’m now critical of. Back then, I wrote functions to simplify my forms, such as:

textInput(”author”);

All the HTML was in the function. It’s exactly what I’m now critical of. So what I’m expressing in this post isn’t premature optimization. It’s way-overdue optimization. I’m looking back at the mistakes I’ve made over the last 5 years and I’m trying to express what I’ve learned so others might now avoid making the same mistakes I’ve made.

Alex Marshall has up a great essay at Spotlight on the Region. He suggests large construction projects are now taking longer than they used to:

It took four years to complete in 1904 the city’s first subway system, much of it hacked away using pick axes and mules.

It took less than a decade to complete in 1915 the central water line from the Catskill Mountains to New York City, a mammoth 92-mile aqueduct and tunnel system worthy of the Roman Empire.

It took just over a year to complete in 1930 the Empire State Building, the tallest building in the world by far at the time, an intricately detailed structure of tiles and brick.

Today things generally take a lot longer, whether it’s the public or the private sector at work.

The first portion of the 2nd Avenue subway, which will be built using state-of-the–art tunnel-boring machines, will take six years to complete – if all goes well and enough money is found. Running from 96th Street on the Upper East Side to 63rd Street, it will have just three new stations, spaced every ten to 14 blocks.

Compare that to the city’s first subway line, the IRT, which ran from City Hall up to Grand Central, over to Times Square, and then up to 145th street. The IRT had 28 stations as well as local and express lines. And it was all built in just four years.

So just to make this contrast clear, the first phase of the 2nd Avenue subway in terms of track is a tenth the size of the original IRT and has three stations as opposed to 28. Yet it will take six years to complete as opposed to four for the much larger IRT. Why is this? …

Almost every engineer or administrator I ask about this seems to agree that things do take longer today, but explanations vary. The usual suspect is environmental and regulatory review, but I’m not including that in my perspective. I’m just talking about actual construction time.

Earlier tonight I found myself writing the following email (one of several) to a designer I’m working with. I was trying to explain why I favored “design lead development”.

Dear wwwwww,

I think another way to think of your job, for the next 3 weeks, is to figure out what assumptions people are making, and draw those assumptions out with Photoshop mock-ups. What are JJJ, MMM, CCC and I assuming? We’re relying on you to attack those assumptions, as much as possible, and make things visually explicit.

I guess another way to approach this is for you to ask, what do you feel ignorant about? What questions would you like to ask about this project? In an ideal world you would never use words to ask any questions, but instead you’d use a Photoshop mock-up to ask every question that you’re curious about. I’m not sure if that is an attainable ideal, but it’s worth keeping in our heads. Ideally you’d speak to us in Photoshop, not English.

Of course, this is merely my opinion. If you disagree with me, I trust your judgement about how to proceed.

– lawrence krubner

Chris Makarsky (who I used to work with at Category4) says goodbye to MySpace. I appreciated his scathing critique of the hype surrounding this badly designed site:

I registered for my account two years ago to see what all the commotion was about. I resisted for a few years prior to that, but after Rupert Murdoch’s $550 million spending spree and the site’s continued appearance in mainstream media outlets, I finally started to wonder if perhaps my unfounded prejudices against the site were unwarranted, that the site perhaps did provide some sort of benefit to its users.

However, I quickly discovered I wasn’t wrong at all, and after putting minimal effort into branding my user page, I happily left the site, destined to never return.

I was good on that claim until recently. Just a few months ago, my profile listed my last login date as October 22, 2005 — a badge of superiority I wore with pride. But then someone else using my computer logged in on my behalf one night, stripping me of my distinction. (The form was auto-filled by the browser… boo.)

Interesting bit from the New York Times on the changing relationship between musicians and their fans:

Along the way, he discovered a fact that many small-scale recording artists are coming to terms with these days: his fans do not want merely to buy his music. They want to be his friend. And that means they want to interact with him all day long online. They pore over his blog entries, commenting with sympathy and support every time he recounts the difficulty of writing a song. They send e-mail messages, dozens a day, ranging from simple mash notes of the “you rock!” variety to starkly emotional letters, including one by a man who described singing one of Coulton’s love songs to his 6-month-old infant during her heart surgery. Coulton responds to every letter, though as the e-mail volume has grown to as many as 100 messages a day, his replies have grown more and more terse, to the point where he’s now feeling guilty about being rude.

Coulton welcomes his fans’ avid attention; indeed, he relies on his fans in an almost symbiotic way.

What does it mean when you find yourself writing emails like this?

Dear vvvvvvvvv,

I have not talked to you in two years, since I lived in Richmond. I think you were friends with Laura Denyes? Or you were a friend of one of her friends? I m curious if you are still doing web design and Flash work? We (bluewallllc.com) are looking for people. Please let me know if you are looking for work.

take care,

lawrence krubner

The modern, affluent standard of living depends on society’s infrastructure, both physical and intellectual: airports, highways, bridges, ports, telecommunication networks, databases, power plants, and the software that makes it all run. 150 years ago it was still common for massive bridge projects to meet with catastrophic failure. The engineers of the 1800s slowly mastered the art of working with iron and then steel. The projects grew in scale, and the skills of the project managers needed to keep apace.

Nowadays we rarely hear of catastrophic bridge failures. This is a particular type of infrastructure that has come to be well understood, both in its engineering and in the project management that oversees its construction. However, we, as a society, are still struggling to find the right way to develop large software projects. This is the newest type of infrastructure, and the correct way to engineer it and manage its development is still a source of controversy.

Robert L. Glass has compiled a book of entertaining and instructive stories regarding the massive software project failures of our time: Software Runaways: Monumental Software Disasters. Anyone with an interest in the management of software projects should read it.

The worst of these project failures, the most expensive and the most ambitious, was the attempt made by the Federal Aviation Authority to modernize the computer system it uses to keep track of what planes are in the air. The effort began in 1981 and ended in complete failure in 1994. The government hired IBM to do the actual work, and over the course of 14 years, IBM burned through $3.7 billion dollars. Nothing was accomplished. The project was finally shut down by Congress. Nothing came out of the project, not a single piece of software, nor even a line of code, was ever used for anything.

Robert N. Britcher, who was involved with the project, offers a full write up of the project. I will try to entice you to buy the book with some excerpts:

The Advanced Automation System began, in concept, in 1981 and ended in 1994, “terminated for convenience” by the government. Billions of dollars were spent on it. It is hard to describe. You can’t learn anything from the name. You know it’s about air traffic control because I told you, or because you read about it in the papers. Maybe part of the problem was the name. It sounds like the system to end all systems.

One engineer I know described the AAS this way. You’re living in a modest house and you see the refrgerator going. The ice sometimes melts, and the door isn’t flush, and the repairman comes out, it seems, once a month. And now you notice it’s bulky and doesn’t save energy, and you’ve seen the new ones at Sears. So it’s time. The first thing you do is look into some land a couple of states over and think about a new house. Then you get I.M. Pei and some of the other great architects and hold a design run-off. This takes awhile, so you have to put up with the fridge, which is now making a buzzing noise that keeps you awake at night. You look at several plans and even build a prototype or two. Time goes on and you finally choose a design. There is a big bash before building starts. Then you build. And build. The celebrating continues; each brick thrills. Then you change your mind. You really wanted a Japanese house with redwood floors and a formal garden. So you start to re-engineer what you have. Move a few bricks and some sod. Finally, you have something that looks pretty good. Then, one night, you go to bed and notice the buzzing in the refrigerator is gone. Something’s wrong. The silence keeps you awake. You’ve spent too much money! You don’t really want to move! And now you find out the kids don’t like the new house. In fact, your daughter says “I hate it”. So you cut your losses. Fifteen years and few billion dollars later, the old refridgerator is still running. Somehow.

At $3.7 billion, the Advanced Automation System was one of the largest civilian computer contracts ever; maybe the largest. It was the largest single contract in IBM’s history. From the moment it was awarded, until near the project’s demise, IBM patted itself on the back. There was something for everyone, beginning with a great ball in Union Station, featuring Chubby Checker and “The Twist”.

At its peak the project employed over 2,000 people. About a million dollars a day. If you thought like the IBM project manager, this was a good deal. Many people were working and money was being made. It was going to last forever. No one considered that it wouldn’t. And everyone was getting ahead. (One of the ironies of conceptual work is that it is easy to believe you are farther along than you are.)

The AAS must have been the most supervised project in history: this atop its enormous size and complexity, and the extreme and constantly changing requirements. One programmer described it this way: “Working on the project was like working on a car inside the garage with the motor running. Eventually, even the crickets hopping around the tires suffocate.”

What I saw on the FAA’s Advanced Automation System would have made Sisyphus weep… Whatever commitment and discipline there was… was worn down by a battery of watchfulness that I can only ascribe to fear of failure. In spite of tens of millions of dollars spent on new computers for AAS, the most important piece of equipment on the project was the overhead projector. There were endless meetings attended by dozens of people – as if we were never quite sure about the whole thing. The people in charge simply lacked the confidence and the finesse of the space team: NASA, contractors, and astronaughts.

It has been noted by everyone from the New York Times to the Vice-President of the United States that the main problem on the Advanced Automation System was “changing requirements”. For those involved in large-scale computer systems, that is nothing new. No one can perfectly surmise the shape and feel of a system years in advance. Even replacing some aspect of a system you know by heart is not immune from thinking twice about it. … [But] the requirements churn (it was called) on the Advanced Automation System was not normal. It was the result of our enchantment with the computer-human interface, the CHI. The new controller workstation, fronted by a 20″ by 20″ color display, because it was capable of seemingly endles variety of presentations, mesmerized the population of AAS like the O.J. Simpson trial mesmerized the nation…

The project was handed over to human factor pundits, who then drove the design. Requirements became synonymous with preferences. Thousands of labor-months were spent designing, discussing, and demonstrating the possibilities: colors, fonts, overlays, reversals, serpentine lists, toggling, zooming, opaque windows, the list is huge. It was something to see. (Virtually all of the marketing brochures – produced prematurely and in large numbers – sparkled with some rendition or other of the new controller console.) It just wasn’t usable…

The cost of what turned out to be a 14-year human factors study did not pay off. Shortly before the project was terminated a controller on the CBS evening news said: “It takes me 12 commands to do what I used to do with one.” I believe he spoke for everyone with common sense.

Rummaging through one of the closets at the far end of the hall on the fifth floor one day, looking for some standards document, I found an envelope left by someone who left the company – as many did after so many years advancing against stone, while the wheels of commerce were accelerating on what everyone referred to as “the outside”. It contained “A Brief History Of The Advanced Automation System”. It was printed by hand and left, perhaps inadvertantly, or perhaps with the hope that some anthropologist might some day discover it and make a pronouncement. In every important way, it is the truth:

“A young man, recently hired, devotes years to a specification written to the bit level for programs that will never be coded. Another, to a specification that will be replaced. Programmers marry one another, then divorce and marry someone in another subsystem. Program designs are written to severe formats, then forgotten. The formats endure. A man decides to become a woman and succeeds before system testing starts. As testing approaches, she begins a second career on local television, hosting a show on witchcraft. An architect chases a new technology, then another, then changes his mind and goes into management. A veteran programmer writes the same program a dozen times, then transfers. The price of money increases eight times. Programmers sleep in the halls. Committees convene for years to discuss keystroking. An ambitious training manager builds an encyclopedia of manuals no one will ever use. Decisions are scheduled weeks in advance. Workers sit in the hallways. Notions of computing begin in the epoch of A, edge toward B, then come down hard on A + B. Human factors experts achieve Olympian status. The Berlin Wall collapses. The map of Europe is redrawn. Everything is counted. Quality becomes mixed with quantity. Morale is reduced to a quotient, then counted. Dozens of men and women argue for thousands of hours: What is a requirement? A generation of workers retire. The very mission changes and only a few notice. Programming theories come and go. Managers cling to expectations, like a child to a blanket. Presentations are polished to create an impression, then curbed to cut costs. Then they are studied. The work spikes and spikes again. Offices are changed a dozen times. Management retires and returns. The contractor is sold. Software is blamed. Executives are promoted. The years rip by with no end in sight. A company president gets an idea: make large small. Turn methods over to each programmer. Dress down. Count on the inscrutability of programming. Promote good news. Turn a leaf away from the sun. Maybe start over.”

I need to post something to inaugurate this blog, so I will start by reposting some comments I wrote over at Burningbird.

Shelley Powers suggests one of the things that leads to gender disparities in the tech world:

If Tim Bray wants to get a better idea of why there aren’t as many women seemingly interested in Ruby, Rails, or associated technologies such as Ajax, he needs look no further than sites like Ajaxian. In all of the time I’ve been reading the site, it covered women–our contributions, as writers, as techs, in any capacity–a grand total of three times. Which is why I decided to unsubscribe from the site today. It’s not for everyone interesting in Ajax–only the boys interested in Ajax.

Someone in the comments suggested:

I think part of it, at least in the RoR world, is that there’s just so much focus on who the hotshots are, and so so much cronyism.

My thoughts:

I don’t know if it’s useful to look at this issue on the micro level. Questions like “Does the RoR community have enough women?” miss the point. The problem is large scale. The number of women receiving advanced degress in computer science peaked in 1989 and has since been in steady decline. That number is, I think, a useful metric for measuring women’s participation in computer science. Whereas women have made dramatic gains in the legal profession, the medical profession, and in managing large enterprises, they have lost ground in the computer industry.

I’ve yet to hear a great explanation of what’s chased women away from the tech industry. The decline since 1989 is easy to measure, yet most of the explanations that I’ve heard are contradicted by the dramatic gains women have made in the law and medical industries.

Consider a few of the most simplistic explanations:

Women don’t like hard science? Nothing is tougher than organic chemistry, which tens of thousands of women have had to master on their way to becoming doctors or “nurse practitioners” (the highest level of nursing).

Women don’t like long, brutal hours? I’ve a friend who just went through the UVA medical school. During her last year of school her average work week was 90 hours. Sometimes it was longer. 90 was the average.

Women are no good at math? I could repeat my comment, above, about organic chemistry, which involves some tough math, but I’d also like to add that I’ve been doing programming in script languages for 12 years now, and I’ve never needed advanced math. Not all programming involves gyroscope stabalized frictionless vector navigation for NASA space craft.

Women require balanced lives, and are unwilling to spend time away from their families? Relative to law and medicine, the tech industry provides more flex time and more chances to work at home. Also, women do put in long hours in the fields of law, medicine, management and marketing. It’s just tech, in particular, where women have been driven out.

Women are drawn to the more artistic side of things, they don’t like technical stuff? Pure sexism, but even if it was true, it wouldn’t explain why graphic design for the web (the artistic side of web work) has become so male dominated, with stars like Eric Meyer and Dave Shea keynoting every design conference.

It’s clear that women have been chased away from the tech field since 1989. The numbers are clear about that, and match what anecdotal evidence most of us can offer. But why? I’ve yet to have a eureka moment, where someone offers an explanation so clear that I simply jump and go “That’s it! That explains it all!”