Closer To The Ideal

Clay Shirky is writing about algorithmic authority.

Algorithmic authority is the decision to regard as authoritative an unmanaged process of extracting value from diverse, untrustworthy sources, without any human standing beside the result saying “Trust this because you trust me.” This model of authority differs from personal or institutional authority, and has, I think, three critical characteristics.

First, it takes in material from multiple sources, which sources themselves are not universally vetted for their trustworthiness, and it combines those sources in a way that doesn’t rely on any human manager to sign off on the results before they are published. This is how Google’s PageRank algorithm works, it’s how Twitscoop’s zeitgeist measurement works, it’s how Wikipedia’s post hoc peer review works. At this point, its just an information tool.

Second, it produces good results, and as a consequence people come to trust it. At this point, it’s become a valuable information tool, but not yet anything more.

The third characteristic is when people become aware not just of their own trust but of the trust of others: “I use Wikipedia all the time, and other members of my group do as well.” Once everyone in the group has this realization, checking Wikipedia is tantamount to answering the kinds of questions Wikipedia purports to answer, for that group. This is the transition to algorithmic authority.

As the philosopher John Searle describes social facts, they rely on the formulation X counts as Y in C — in this case, Wikipedia comes to count as an acceptable source of answers for a particular group.

There’s a spectrum of authority from “Good enough to settle a bar bet” to “Evidence to include in a dissertation defense”, and most uses of algorithmic authority right now cluster around the inebriated end of that spectrum, but the important thing is that it is a spectrum, that algorithmic authority is on it, and that current forces seem set to push it further up the spectrum to an increasing number and variety of groups that regard these kinds of sources as authoritative.

There are people horrified by this prospect, but the criticism that Wikipedia, say, is not an “authoritative source” is an attempt to end the debate by hiding the fact that authority is a social agreement, not a culturally independent fact. Authority is as a authority does.

Today, I was at the train station in MetroPark, New Jersey. I was catching the commuter train back into New York City. The terminal that displayed the schedule had an error message on the screen. I do not know why these are so common, but I see error messages like this a lot, on public terminals at bus and train stationsm, and airports. The errors always seem to be on Windows operating systems.  Always.

Brad Delong asks “Why does Berkeley think it should be in the business of building its own content management systems anyway?“.

His complaint:

May I say that a content management system that–if you have been off dealing with another crisis in the middle of a task–decides when you come back and try to save your work that you are no longer logged in and dumps you to a login page after which it dumps you not on the page you were working on but on the root page, LOSING YOUR WORK!!!1!!

Such a content management system is HELLSPAWN!! Is WROSE THAN HILTER!1!!!1!…

He is complaining about bspace, which is based on the open-source
Sakai, a content management system written in Java, and focused on the needs of universities.

I think Delong’s post is a good reminder of how infuriating it can be for users when software fails to behave in the ways users expect. I also suspect this is a good example of an issue that users will regard as a bug, but the programmers will see it simply as a potential future feature that they may or may not add (”Should we catch POST info if a user is not logged in?”).

I should add, WordPress has exactly the same problem. Last week I started writing what I thought would be a short post for this blog, but I got carried away by my theme and wrote a long post. Then I went to get some dinner. I left the browser open, with the post unpublished. I came back after dinner and made some more edits, then hit the publish button – and just like that, my work vanished, because while I was out getting dinner, WordPress logged me out (for some reason I’d assumed that the auto-save feature was also refreshing my session info).

One of the nice things about building my own CMS was that I was free to fix the bugs that bothered me most, and this was a big one for me. I added a feature to my CMS that caught any POST info and showed it on screen, even if the person was logged out. This allowed recovery of the post. I worked on my CMS from 2002-2007 and then abandoned it because I could not keep up with projects like WordPress. Nowadays I force myself to use other people’s open source software, because it is economically rational to do so, but I hate some of the choices they make, and some of the features that they fail to implement.

In the comments, Jacob Davies posts this comment, which I thought was very funny and very on point:

Conversation that has happened more times in my career than I care to mention:

Someone else: “How long of a title shall we allow? 32 characters? 64?”

Me: “FOR THE LOVE OF GOD WHY DO WE NEED TO SET A MAXIMUM LENGTH? IS THIS 1952???”

Someone else: “But what if they put in a really long title and fill up the database?”

Me: “THE VERY NEXT FIELD – THE ‘CONTENTS’ FIELD – IS A FREE-TEXT FIELD WITHOUT A LENGTH CONSTRAINT SO IF THEY WANTED TO FILL THE DATABASE THEY COULD DO IT THERE ANYWAY.”

Someone else: “Won’t it waste space if we allow a variable-length string in the title?”

Me: “OH MY GOD YES A TERRIFYING LOSS OF ABOUT 3 BYTES ON A RECORD THAT IS A MINIMUM OF 1024 BYTES LONG AND OFTEN OVER A MEGABYTE, YOU ARE SO RIGHT.”

Someone else: “Yes but every other system has a length constraint for titles.”

Me: “YES AND I SUPPOSE IF EVERYONE ELSE WAS JUMPING OFF A BRIDGE YOU’D DO IT TOO.”

etc

Computer programmers are subject to some kind of strange mental degeneration in which they rate the potential waste of 0.00001% of the capacity of a modern hard disk as more important than the ability to enter titles longer than 32 characters in length.

Robert Hoekman is writing about the death of “Tyler”, a pit bull dog. I find it unusual, and uncomfortable, to see this much anger in a public forum:

We were all so confident that Rachel, his would-be new owner, was the right person for him. We couldn’t have been more wrong.

Less than nine hours after leaving him in her care, she did the exact thing we explicitly told her not to do, the one thing that most blatantly defies common sense: she let him off of his leash. In less than nine hours, she took away everything we did, everything we hoped for, and every chance Tyler would ever have at living the life he deserved.

Tyler was depending on Rachel. We were depending on her. She failed at her commitment to love and support this wonderful, sweet, and loyal dog so miserably and so quickly that it has absolutely stunned each and every one of the amazing people that have volunteered their time and energy to bring Tyler home safely. Even worse, Rachel opted out of continuing to aid in the search, citing her outrageously absurd belief that Tyler would simply go back to doing what nature designed him to do best: be one with the Earth through his nomadic and migratory instincts.

People who never met Tyler, my wife, nor myself came to the area night after night, morning after morning. They hiked through the preserve. They walked neighborhoods. They posted flyers. They talked to everyone they came across. Every last one of them did so much more than Rachel could be bothered to do. Every one of them cared so much more than she ever could.

I wish more than words can say that last Saturday had never happened — that we had decided to keep Tyler ourselves and not brought him to his new owner’s apartment and said goodbye. I wish we had been able to see her naivete and arrogance before it was too late. I wish so many things were just slightly different than they were.

I will never forgive Rachel for her foolish and fatal decision. Tyler will never have the chance.

Even if “Rachel” is a fictional name, I assume that if I made an effort, I could find out this woman’s real name.

I’m curious if it is ever wise to express this much anger in a public setting? I’d be angry as hell if someone’s lack of responsibility lead to the death of an animal that I cared about, and I might even vent about the incident in public, but I don’t think I’d give enough details that other’s could figure out who I am talking about.

FireFox can crash any machine. Not “crash” in the sense of “blue screen of death” but crash as in “uses up all memory so the machine becomes unresponsive”. This is a reliable fact of using FireFox, regardless of whether you are on Ubuntu Linux, Windows XP, or Mac OS X (I can’t speak of Camino, as I don’t use it).

Sometimes I say this to other programmers and they respond “It’s not FireFox that is the problem, it is the plugins that you use – it is FireBug and Session Manager and all the others.” Of course, any programmer who reveals this attitude needs to be re-educated. If you offer a plugin system that is unable to manage the plugins, then maybe you should not offer that plugin system? It suggests a (possibly frightening?) willingness to shirk responsibility if a programmer defends a plugin system that can crash a computer.

I wonder what Brendan Eich is thinking?

One suggestion for others: if you use FireFox, every time a new version of FireFox comes out, FireFox will ask you if you want to upgrade. I used to always say “yes”. Now I realize, if your computer is more than a year old, you should say “no”. Each version of FireFox tends to be heavier and slower than the previous version. My Ubuntu machine is from 2006, and that is part of the reason why FireFox is so slow on it.

On my Windows machine, I just switched over to Google Chrome as my new default browser. I’m giving up on FireFox. On my Ubuntu machine, I am stuck with FireFox for now. I’m not aware of any other serious browsers for Linux.

For email, I would love to give up on Thunderbird, if I could find a substitute. I run Thunderbird on my main desktop machine which runs Ubuntu. Thunderbird has had a persistent bug that has survived several upgrades (of both Thunderbird and Ubuntu). The bug is with the address auto-completion. If I type an address fast, hit “Enter” to accept and start typing again fast, Thunderbird crashes. This can lose a lot of work for me (Where “work” might simply mean “Opened email and left them open because I found some that were important and so answering them will take some time.”). Apparently there is no equivalent of SessionManager for Thunderbird, no way of remembering which emails were open, waiting for a response, when Thunderbird crashes. No, instead, after Thunderbird crashes, I need to re-start it, go back 3 days, and then read through all my email again, looking for the important ones.

At work we had a deadline today, and I worked through the weekend to meet it. I kept getting feedback from various people testing the site. Some of the email I got was thoughtful, and offered intelligent suggestions about what we should do next. By this morning, I had about 20 emails open, waiting for me to have the time and focus to write a reply. Then Thunderbird crashed and they all vanished. I yelled so loud my throat was sore. Now I have to go back to Friday and read through all the email again, to find the ones that I wanted to respond to.

If I could find something better than Thunderbird, that runs on Linux, I’d switch immediately.

 Yes, that is a broken image link on the Microsoft website (I hit refresh a few times and it was still there). Ironically, the page is talking about a new technique for debugging.

Part of what is becoming a continuing series on the subject of broken web pages.

I find this quote very interesting:

Business realities force apologies. To cope with these difficult realities, we need code and, frequently, we need human beings to apologize. It is essential that businesses have both code and people to manage these apologies.

…

We try too hard as an industry. Frequently, we build big and expensive datacenters and deploy big and expensive computers.

In many cases, comparable behavior can be achieved with a lot of crappy machines which cost less than the big expensive one.

That’s from Pat Helland an ex-Amazon architect. His point is that businesses can get way with building imperfect systems that will occassionally fail, so long as the business realizes that sometimes it will have to apolgize. The simpler systems will be much cheaper than a complicated system that will offer a higher rate of reliability.