Closer To The Ideal

Tim Bray thinks now is the moment when a mainstream concurrent programming language will emerge:

The “Java Moment” · On my recent excursion to Japan, I had the chance for a few conversations with Bruce Tate. He advanced a line of thinking that I found compelling: Right now is the time when the concurrent-programming winners will emerge. He sees an analogy to Object-Orientation in the early nineties: Several O-O languages were in play (most notably C++ and Smalltalk), but it hadn’t penetrated the application-development mainstream. Then Java came along, and turned out to have just the right characteristics to push O-O into the middle of the road. ¶

Thus the analogy. Right at the moment, we have a bunch of candidate technologies to fill in the concurrent-programming void; obvious examples include Erlang, Scala, Clojure, and Haskell. While there are common threads, they differ from each other in many essential ways. Between the lot of them, there are a whole lot of different characteristics. The fact is, we don’t know at this moment which laundry-list of features is going to turn some candidate into The Java Of Concurrency.

I like the idea of studying continuously through the course, so that at the end of a course you do not need to study for the final, because you already know it all:

Furthermore I was lucky enough to talk to some of my students about the experience a few months later. The general consensus was that the material really stuck. Furthermore nobody studied for the final. No joke. As one girl said, “I tried studying because I thought I should, but I gave up after a half-hour because I already knew it all.” That is how I think it should be – if you study properly through the course, then you won’t need to study for the final. Because you’ve already learned it. And you’ll have a leg up on the next course because you still remember the material that everyone else has forgotten.

So were there any downsides? Unfortunately there were some big ones. I had set goals around learning. I failed to set any around happiness. Having to pay attention during class was hard on the class. Also it motivated them to work hard. Since everyone worked hard and they thought that I was going to grade them on a curve, there was a lot frustration that they wouldn’t properly be recognized for their work. (In fact I gave half of them A’s in the end.) This frustration showed up the teacher evaluations at the end of the course. :-(

I also know how hard it is to pay attention and not day dream. That is what sunk me throughout my school career – I could not concentrate, I would always daydream instead.

I’ll now be contributing to the Symfony Nerds blog. My first post is up, and it mostly reviews various bits of Symfony news and articles that I found interesting during the last week or so.

Today, I was at the train station in MetroPark, New Jersey. I was catching the commuter train back into New York City. The terminal that displayed the schedule had an error message on the screen. I do not know why these are so common, but I see error messages like this a lot, on public terminals at bus and train stationsm, and airports. The errors always seem to be on Windows operating systems.  Always.

For people new to Symfony, this is a good write-up of the new forms framework, introduced a year ago in Symfony 1.1.

Joel Spolsky has an odd rant where he says things that are fairly close to being true, but he puts a ranting twist on them so as to make them not true:

You see, everybody else is too afraid of looking stupid because they just can’t keep enough facts in their head at once to make multiple inheritance, or templates, or COM, or multithreading, or any of that stuff work. So they sheepishly go along with whatever faddish programming craziness has come down from the architecture astronauts who speak at conferences and write books and articles and are so much smarter than us that they don’t realize that the stuff that they’re promoting is too hard for us.

Here’s what Zawinski says about Netscape: “It was decisions like not using C++ and not using threads that made us ship the product on time.”

Later, he wrote an email client at Netscape, but the team that was responsible for actually displaying the message never shipped their component. “There was just this big blank rectangle in the middle of the window where we could only display plain text. They were being extremely academic about their project. They were trying to approach it from the DOM/DTD side of things. ‘Oh, well, what we need to do is add another abstraction layer here, and have a delegate for this delegate for this delegate. And eventually a character will show up on the screen.’”

…Zawinski didn’t do many unit tests. They “sound great in principle. Given a leisurely development pace, that’s certainly the way to go. But when you’re looking at, ‘We’ve got to go from zero to done in six weeks,’ well, I can’t do that unless I cut something out. And what I’m going to cut out is the stuff that’s not absolutely critical. And unit tests are not critical. If there’s no unit test the customer isn’t going to complain about that.”

What is really amusing (in the tragic way that amnesia is amusing) is this part:

Remember, before you freak out, that Zawinski was at Netscape when they were changing the world. They thought that they only had a few months before someone else came along and ate their lunch. A lot of important code is like that.

Apparently Joel doesn’t remember how awful the software was. Netscape3? Or, for laughs, Netscape 4, not only the worst browser ever made, but probably the worst piece of software that anyone ever took seriously. This is duct-tape programming in the worst sense – that things were broken and badly patched was obvious to all.

Much of what Joel is saying is transformed into useful advice by Uncle Bob:

Joel’s specific complaints can be ignored. The spirit of simplicity, however, should be taken to heart. If you don’t need a “cool” technology, you are probably better off not using it. Keep things simple. Keep things uncomplicated. Don’t pack your system with every new idea you’ve heard.

Not that you shouldn’t learn new ideas, and new languages, and new APIs; you should! But you don’t have to use them in systems as soon as they are out of the box. Let them cure a bit.

This is an especially good point:

As for Joel’s consistent dismissal of unit testing, he’s just wrong about that. Unit testing (done TDD style) does not slow you down, it speeds you up. One day I hope Joel eventually realizes this. Programmers who say they don’t have time to write tests are living in the stone age. They might as well be saying that man wasn’t meant to fly.

I am curious about Mobicules weblog. Some of their posts are self-serving, but they seem to be trying to get some real information out, which is more than what I’ve seen so far from most of the sites of Indian firms that I’ve looked at. How should I read this post on which platform to use for a massive social site that can scale to tens of millions of users?

ELGG: Due to a very flat and normalized database structure, the only way to scale horizontally with ELGG is to duplicate the database using MySQL replication on multiple servers. The negative with this approach is that it ends up duplicating the complete database, with the result that each machine running a copy of the replicated database will have to be very powerful server, rendering the solution expensive. (MySQL clusters is a standard way of doing it)

Drupal: Similar problems as ELGG with Drupal, but database is not as flat and normalized as ELGG. Scaling up in a similar manner as described for ELGG wity MySQL clusters would probably be cheaper with Drupal.

Symfony: Allows us to use a custom database structure. We can design the database and replicate it as we like. Symfony also uses its own query/object caching mechanism, which is efficient. As an example, the Yahoo bookmarks site supports 20 million users on Symfony.

…Our recommendation in a case where you are looking to scale up to 10s of millions of users would be write everything from scratch (and not use a ‘platform’ like ELGG/Drupal), that allows you to customize and tweak anything and everything. Symfony seems to have a good reputation for enabling creation of very large websites, and so, we would recommend using Symfony as a framework.

I actually agree with their conclusions.

3,500 websites got hacked because they had .svn folders on the live site. The really shocking thing is that Apache.org was one of the sites that got hacked:

A Russian security group has posted a detailed blog post(translation here) about how they managed to extract the source code to over 3,300 websites. The group found that some of the largest and best known domains on the web, such as apache.org and php.net, amongst others, are vulnerable to an elementary information leak that exposes the structure and source of website files. A web surfer is able to extract this information by requesting the hidden metadata directories that popular version control tool Subversion creates.

The actual ‘exploit’ itself has been well known for a long time. It is the fault of the server administrator or developer, rather than the fault of a particular application, since the working metadata directories in Subversion are only required for working copies of code. What is surprising is just how prevalent the problem is – and who it affects. Finding version control metadata directories is as simple as looking for ‘.svn’ or ‘.cvs’ folders within web paths, for example: http://www.test.com/.svn/.

For god’s sake, people, use the export command:

The Subversion Export action copies versioned files from a working copy or the repository, to another directory, allowing you to distribute the files without the .svn directory.

We’ve been using Springloops for a long time, which hosts our Subversion repositories. It has an auto-deploy feature that we rely on – every time we commit, it auto-deploys our files to our dev server. I’m fairly sure that Springloops auto-deploy feature is simply a script that hooks to the post_commit event and which then calls ‘export’ on the files you’ve just modified. I’m guessing I could write a similar script in an hour. This is the right way to deploy files from Subversion to your site. The wrong way is to treat you live site as a working copy of your repository and have it run the ‘update’ command every time the post_commit event fires. The problem with this approach is exactly what the article above describes – you can get hacked.

If you decide you absolutely need to have your site be a working copy of your repository,
the Subversion FAQ also demonstrates how to protect your site:

Add this to your httpd.conf:

# Disallow browsing of Subversion working copy administrative dirs.
<DirectoryMatch “^/.*/\.svn/”>
Order deny,allow
Deny from all
</DirectoryMatch>

For the last few years I’ve been pursuing accidentalism as a theory of economic development. This is a vein of insight that I will continue to mine, as I feel I still have much to learn. But I am fascinated to read Nicholas Carr’s criticism of the theory:

Now, I understand what the Accidentalists are getting at: Technology builds on technology, and at any given time in human history only certain technologies are in the realm of the possible and of those only a subset will actually be developed and put to use. Those technologies will in turn influence the means of production and the modes of consumption both directly (through the characteristics of the technologies themselves) and indirectly (through the economic tradeoffs inherent in using the technologies). Every technology, every means of production, every mode of consumption is hence provisional. Something better or at least cheaper or more convenient may come along tomorrow and displace what we depend on today.

All that’s true. But is it really accurate to describe the process as fundamentally accidental? Does the word “accidental” accurately reflect the complexities of technological and economic development? I don’t think it does. In fact, I think it’s difficult to imagine a poorer choice of word. When you describe an event or a thing as an accident, what you are doing is draining it of all human content. You are saying that human intention and will and desire played no part in its occurrence. A volcano is an accident in human history (if not natural history), and if it’s a big enough one it may well influence the course of that history. But the the book, the printing press, the publishing house, the newspaper, and the newspaper company are not volcanoes. Their development was guided not just by blind circumstance but by human intent and desire. They represent, not just in the abstract but in their concrete forms, something that people wanted and that people consciously brought into being, for human purposes.

I don’t think it is right to say that an event or development is drained of its human component when we call it an accident. Doris Lessing has said that it is part of human nature to be, at all times, in rebellion against circumstance (she said this, I believe, in her book Prisons We Chose To Live Inside). When I read that, I think of her horrible childhood in Africa, and then her horrible first marriage, before she was able to escape to England. She felt herself trapped by circumstance, that is, a series of accidents that built up to a particular situation. Nevertheless, in her case, we are talking about a circumstance full of emotion, trial, stress, anger, striving, dreaming. It would be a mistake, I think, to say that her circumstance lacked a human dimension, even though we must acknowledge that much of what she experienced was shaped by accidents – the accidents of birth (where was she born, who were her parents, how much money did they have), the accidents of her education, the accidents of whom she first met romantically.

Having a fate doesn’t make us non-human.

Oddly, the part of his post that I most agree with is a part that Nicholas Carr offers in jest:

If we suspend our disbelief and accept the Accidentalist view that both the media of the past and the means of their production were accidents, then we have to also view the media of today and the means of their production as accidents. If the book is a historical accident, then the web is a historical accident. If the newspaper publisher is a historical accident, then the blogger is a historical accident. To think otherwise – to think that all mankind’s past blundering has brought us suddenly to a perfected state, that the long chain of accidents has been broken in (surprise!) our very own lifetime – is to abandon any pretense of a consistent and rational view of history and leap into the realm of quasi-religious faith. We were lost, and now we’re found!

Without a doubt, all of our current technologies are accidents. All of our current economic processes are accidents. And all of our future technologies and advances will also be accidents. A comparison to Darwin’s theory of evolution would be useful here – in the same way that wolverines, spiders, humans and bacteria are accidents, so to, every existing business is an accident, existing in a transitional state between what it has been and what it will be.

“Accident,” I hardly need point out, is a word with strongly negative connotations.

I can think of counter-examples. Winston Churchill praised certain accidents of history for allowing Europe to remain free of the Khan, during the Middle Ages. Sir Alexander Fleming openly described the discovery of penicillin as an accident. Charles Darwin suggested that all species were accidents, and many of the biologists who have lived in the modern era have been forceful in asserting that the human race is an accident (I am thinking of Stephen Jay Gould in particular).

Nicholas Carr has a particular target that he is going after:

Accidentalism, in other words. provides the perfect backdrop for the liberation mythology promoted by many of the web’s most ardent proponents, which is built on the idea that old technology put us in chains and new technology is breaking those chains.

I don’t have a problem with Carr criticizing those people. No doubt there are some who are promoting a millennialist interpretation of the Internet. But Carr specifically mentions Clay Shirky, who is clearly not a millennialist.

Any theory of the future that requires a distortion of the past should be greeted not with applause but suspicion.

What should we say of a theory that distorts what people are saying right now, in the present?

10 years ago, when I was getting serious about programming, I read Phillip Greenspun’s book about websites and databases. Greenspun does a great job of explaining some of the thinking, and even traditions, that shaped relational databases. I read that book and formed certain opinions about databases, for instance, that a database should not store the same info in multiple places (that is, the database should be normalized). You would be a fool to store a user’s phone number in 3 different places, because when they change their phone number, there is a risk that one database table with update but another won’t, and then you have inconsistent data about the user.

I’ve been fascinated with the articles, more and more over the last 3 years, arguing for a new style of storing data:

Distributed vs. Relational Databases

Traditional relational databases are 30 years old, are well understood and have a huge ecosystem of tools around them. For that reason, it’s a compelling option when building your application. Postgres, MySQL, and Oracle are all relational databases modeling a schema on entities and relations between those entities. That’s a good, powerful programming model with interesting theoretical properties. But companies with large amounts of data have already gone past what you can reasonably fit on a single machine, even on high-end hardware, and it’s provably impossible to keep the traditional relational model, in particular the ACID properties, while scaling across multiple machines. Even if you’re willing to give up availability, scaling reads (via caching and replication) is difficult with relational databases, and scaling writes by partitioning is either very expensive, very painful from an application programming and operations standpoint, or both.

Cassandra is taking the approach that, given that you’re going to have to give up some parts of the relational model to scale, let’s start over and rethink things. Let’s add things like transparent replication and failover, built-in partitioning and load balancing, multiple data center support, and the ability to add capacity without ever disturbing applications running against the database.

I’m working on image gallery to be used on a site built with Symfony. So I’ve been reading up on all the different techniques out there for managing images for such a site.

Sitepoint has a good tutorial on how to use the sfThumbnail plugin:

We’ll use the sfThumbnail class provided by the plugin to create a thumbnail of each uploaded file. The best place to do this is in the setFilePath() method of the Photo class. Add the following to the Photo.php file:

// in lib/model/Photo.php
public function setFilePath($value)
{
parent::setFilePath($value);
$this->generateThumbnail($value);
}
public function generateThumbnail($value)
{
parent::setFilePath($value);
$uploadDir = sfConfig::get(’sf_upload_dir’);
$thumbnail = new sfThumbnail(150, 150);
$thumbnail->loadFile($uploadDir.’/’.$this->getFilePath());
$thumbnail->save($uploadDir.’/thumbnail/’.$this->getFilePath(), ‘image/png’);
}

One question I had to look up was how to restrict which files users are allowed to upload. That thread on the forum was useful.

I’ve been frustrated with the sound on my Ubuntu machine. It is fine when it boots up, but only one app at a time can use the speakers (I understand that there are patches for this. I’ve been too lazy to install them. The point is the issue isn’t resolved by default, as it is on Mac and Windows machines). And often an app will quit but not release its hold of the speakers, so the only way I can get them working again is to reboot the machine.

Therefore, I am pleased to know that sound is getting better on Linux. I had no idea most of this software existed. Examples:

Lash uptake has been good, and now lash talks dbus… letting it mix in nicely with the rest of the linux desktop ecosystem. Lash is a session system for linux audio programs. It lets you open your 12 different linux audio programs(remember audio in linux is like pipes… pipes with audio running through them instead of water… let’s call them wires… but digital… maybe fiber optics… but not using light… ok whatever… why am I explaining it this way?… you’re not five… too many dots. sorry.) and save your settings for later. The alternative is to each time open your 12 programs, set up the wiring between six of them, start messing around and finally… 2 hours later… realise you were supposed to be setting them up in a certain way rather than making stupid minimal beep noises to a house vocals mixed with a recording of a fart noise – filtered down to retro 8bit samples. Without lash, you couldn’t save that brilliant setup and play with it later.

Audacity, the simple(yet advanced) audio editing work horse is moving towards a 2.0 by the end of the year. Audacity has been around for ages, and has been multi platform for ages. The 1.3 series seems to have been going on forever… but they do regular beta releases, and nightly builds. So it’s pretty easy to get fresh versions. Do proper releases matter that much when new releases are pushed out every day? I guess so.

The Daily WTF offers for mockery a bit of code that has very long, self-descriptive variable names:

switch (mIntegrationDirection)
{
case InventoryIntegrationDirection.DoTheSameChangeInShop:
DoTheSameChangeInShop(GetChangedDataFromInventory(mLastSynchDate));
break;

case InventoryIntegrationDirection.DoTheSameChangeInInventory:
DoTheSameChangeInInventory(GetChangedDataFromShop(mLastSynchDate));
break;

case InventoryIntegrationDirection.DoTheSameChangeInShopThenDoTheSameChangeInInventoryForNotChangedDataInInventory:
DoTheSameChangeInShopThenDoTheSameChangeInInventoryForNotChangedDataInInventory(
GetChangedDataFromInventory(mLastSynchDate),
GetChangedDataFromShop(mLastSynchDate));
break;

case InventoryIntegrationDirection.DoTheSameChangeInInventoryThenDoTheSameChangeInShopForNotChangedData:
DoTheSameChangeInInventoryThenDoTheSameChangeInShopForNotChangedData(
GetChangedDataFromShop(mLastSynchDate),
GetChangedDataFromInventory(mLastSynchDate));
break;

default:
break;
}

But I actually like long variable names. Yes, it can be overdone, but I’d rather deal with the above code than deal with code where the variables have names like “i”, “j”, “l” or “y”. I hate code where “nt” stands for “new template object” (Java code is especially bad in this regard, by tradition). I hate code where “cu” is suppose to refer to “current user”. Such short variable names makes code very difficult to figure out, if you are taking over from another programmer.

I was pleased to see that at least some of the readers over at Hacker News felt the same way I did. I appreciated this set of examples of bad and better:

Papa Bear:
  for(i=ii;i < iii;i++){
    for(j=jj;j < jjj;j++){
      for(k=kk;k < kkk;k++){
        doSomething();
      }
    }
  }
Mama Bear:
  for(YearCounter=FirstYearOfCycle;YearCounter < LastYearOfCycle;YearCounter++){
    for(MonthCounter=FirstMonthOfYear;MonthCounter < LastMonthOfYear;MonthCounter++){
      for(DayCounter=FirstDayOfMonth;DayCounter < LastDayOfMonth;DayCounter++){
        doSomething();
      }
    }
  }
Baby Bear:
  for(Year=FromYear;Year < ThruYear;Year++){
    for(Month=FromMonth;Month < ThruMonth;Month++){
      for(Day=FromDay;Day < ThruDay;Day++){
        doSomething();
      }
    }
  }

A moderate comment, seeing both sides of the issue, was this one:

It appears to me that [the code is] synchronizing inventory change records between a store front and the physical warehouse. I could be wrong, but I imagine it’s something pretty similar to that. And, as much as I do shudder at this code, the names did give me a pretty good idea of the intent.

Exactly. Intelligent, clever programmers sometimes get carried away with writing concise, minimalist code. Though cleverness is good in many cases, it shouldn’t obscure the need to have code that is comprehensible to other programmers. One response I sometimes here is “Well, if the new programmer is really good, then they’ll have no trouble understanding that code.” Yes, but it is unrealistic to think that all of the programmers will always be above average. Some concessions to the average are needed.

PPK has an unusually harsh post about the way drag-n-drop is being implemented in HTML5.

dragthat and dragsomethingelse

In theory dragenter and dragleave could be great events, since they fire when you enter or leave an HTML element in the middle of a drag operation. If that element is a valid drop target you could change its styles ondragenter and ondragleave to indicate this fact to the user.

With the spec being based on the Microsoft API, I expected dragenter and dragleave to emulate mouseenter and mouseleave. But they didn’t. Not even in IE. They’re based on mouseover and mouseout and suck every bit as badly. And their names are wrong.

Mouseover and mouseout are terrible because they bubble up all the time and make it very hard to distinguish important events from unimportant ones. If I mouse over (or drag enter) a child element of the one that the event is set on, it also fires. And that’s exactly what we don’t want. The events will fire incessantly the whole time the mouse is above the element, and we have to work hard in order to distinguish useful events from useless ones.

Mouseenter and mouseleave, on the other hand, fire only when you enter and leave the element they’re defined on, and they don’t bubble. That makes them much easier to use.

They are Microsoft extensions, by the way, and excellent ones at that. They work only in IE.

Note that originally mouseenter and mouseleave had exactly the same IE only compatibility pattern as drag and drop. Mouseenter and mouseleave are a good idea, drag and drop … isn’t. Guess what the other browsers chose to implement?

No wonder web development is such a fucking pain with fucking morons in charge of the browsers.

Wait, that could be construed as an insult to morons.

Ah, what the fuck.

dragbullshit

Dragover, now, has nothing to do with mouseover. It’s exactly the same as the drag event, except that you can set it on any element instead of just the document. Or something. Whatever.

Why do we need the dragover event if we already have the drag event?

[ ... fume ... ]

To cancel its default action!

If we didn’t have to cancel its default action the dragbullshit event would have no fucking point!

And we can’t have pointless events in our nice specification, now, can we?

So we give it a default action. A very complicated default action.

That has to be canceled. Absolutely, positively has to be canceled.

[ ... crickets ... ]

Anybody LISTENING to what I say?

[ ... drag ... ]

I deserve a fucking MEDAL for this. Above and beyond the call of fucking DUTY.

The conventional wisdom has been that any exposure to radiation increases your risk of cancer. However, an incident in Taiwan suggests that low levels of radiation may decrease your risk of cancer.

Chronic radiation is defined as the radiation received slowly or in a low-dose-rate from various sources. It is completely different in nature to the acute gamma or neutron radiation generated from the atomic bomb explosions that occurred in Japan at the end of World War II. Tantalizing insights from people living in higher-than-normal background radiation areas in the world and from nuclear energy workers receiving excess radiation over long years have suggested that chronic radiation might paradoxically be beneficial to humans. However, in the absence of an epidemiological study, it has been impossible to conclude whether chronic radiation is harmless or indeed beneficial to human beings. Fortuitously, an incredible Co-60 contamination incident occurred in Taiwan 21 years ago, which provided the data necessary to demonstrate that chronic radiation is beneficial to human beings.

The contamination occurred during the recycling of metal scrap when a Co-60 source was mixed with metal scrap, melted and drawn into steel bars in the mill. Unaware of the contamination, the steel bars were ultimately used in construction of more than 180 buildings in 1982-84. Most buildings were partitioned into about 1,700 apartments for dwelling, and some buildings for other purpose. The first contaminated apartment was discovered in 1992. The residents in the apartments totaled 10,000 individuals who had been exposed to chronic radiation for at least 9 years and as long as 21 years.

…If the linear-no-threshold (LNT) model constructed based upon data from the atomic explosions in Japan is appropriate for evaluating chronic radiation, such excessive doses received by the contaminated apartment residents could induce at least 35 excess leukemia and 35 solid cancer deaths after 21 years. However, actually no increase cancers were observed. On the contrary, the spontaneous cancer deaths of the residents totaled 243 over 21 years based upon the vital statistics provided by the Taiwanese government. The mortality rate from these cancers dropped to only 3% of the general population as shown in the following graph.

I’m assuming that, if this benefit is real, it comes from killing the cancer cells. Doctors use radiation to kill cancer. Perhaps low levels over the long term are similar, or better, than being treated by doctors. When my dad was treated with radiation, I had the impression that doctors were over doing it. Back then, I wondered what would happen if he received a smaller dose.

Once upon a time, psychologists thought that electric shock therapy might help people with mental illness. People were shocked with high voltages. The benefits were few. Then electric shock therapy went out of fashion. Then it came back in a different form. Now electric therapy uses very low doses of electricity, so low that patients barely notice it. And this is, apparently, one of the most effective treatments for depression. Kitty Dukakis has written about this in her book. I wonder if radiation treatment will go a similar route, with doctors realizing that low doses are more beneficial than high doses?

I’ve worked on projects with 8 or 9 programmers and the managers running the project had no idea which programmers were contributing what. I have often been surprised that some managers are willing to fly in the dark, regarding who is doing what. These tend to be managers with weak technical backgrounds, so perhaps they are unable to analyze the work at the appropriate level.

This analysis of who contributes to Linux seems full of information that managers would want to track about any project:

Beyond that, how one generates statistics from a patch stream is an
interesting question. How does one measure the productivity of
programmers? One possibility is to look at the number of changesets
merged. By that metric, this is the list of the most prolific contributors
to 2.6.20:

Developers with the most changesets
Al Viro	241	4.8%
Andrew Morton	92	1.8%
Jiri Slaby	92	1.8%
Adrian Bunk	87	1.7%
Gerrit Renker	79	1.6%
Josef Sipek	79	1.6%
Avi Kivity	68	1.4%
Tejun Heo	67	1.3%
Patrick McHardy	63	1.3%
Ralf Baechle	61	1.2%
Randy Dunlap	59	1.2%
Alan Cox	58	1.2%
Mariusz Kozlowski	57	1.1%
Andrew Victor	53	1.1%
Paul Mundt	52	1.0%
Stefan Richter	49	1.0%
David S. Miller	48	1.0%
Russell King	44	0.9%
Benjamin Herrenschmidt	44	0.9%
Akinobu Mita	43	0.9%

Looking at patch counts rewards developers who put in large numbers of
small patches. Al Viro’s patches include a vast number of code annotations
(to enable better checking with sparse), include file fixups,
etc. Many of the changes are small – many do not affect the resulting
kernel executable at all – but there are a lot of them. Even so, as the
biggest contributor, Al generated less than 5% of the
total changesets added to the kernel. The top 20 contributors, all
together, generated 28% of the total changesets in 2.6.20.

One could make the argument that a better way to look at the problem is by
the number of lines affected by a patch. In this way, a contributor’s
portion of the whole will not depend on whether it has been split into a
long series of small patches or not. On the other hand, simply renaming a
file can make it look like a developer has touched a large amount of code.
Be that as it may, by looking at lines changed (defined
as the greater of the number of lines added or removed by each individual
changeset), one gets a table like this:

Developers with the most changed lines
Jeff Garzik	20712	6.0%
Patrick McHardy	15024	4.3%
Jiri Slaby	13917	4.0%
Avi Kivity	11726	3.4%
Andrew Victor	9710	2.8%
Amit S. Kale	9537	2.7%
Stephen Hemminger	9120	2.6%
Geoff Levand	8396	2.4%
Michael Chan	8307	2.4%
Chris Zankel	8099	2.3%
Mauro Carvalho Chehab	7390	2.1%
Adrian Bunk	6138	1.8%
Yoshinori Sato	5232	1.5%
Al Viro	4981	1.4%
Benjamin Herrenschmidt	4588	1.3%
Thierry MERLE	4549	1.3%
Dan Williams	4516	1.3%
Jonathan Corbet	3924	1.1%
Gerrit Renker	3857	1.1%
Jiri Kosina	3805	1.1%

This summer I was part of a project that, in my opinion, made many mistakes. The entrepreneur and his project manager felt they could build web software even though they had no programmer as part of their core team. They felt the technical work could be handled by a team in India. The 2 of them spent most of their time trying to raise venture money. They spent a lot of time honing their VC pitch – many hours were spent polishing the slides they used in their PowerPoint presentation.

Last time I checked in with them, they project had fallen several months late, they had mostly used up their initial money, and they were having trouble fundraising.

My own theory of the right way to go is “Form an awesome founding team and get your product out the door ASAP. Then, forget everything else, VCs included, and just build.”

So I was really glad to see Meebo CEO Seth Sternberg giving exactly that advice.

These 2 paragraphs, in particular, speak to the mistakes that I just saw on that last project:

At the exact moment you had your idea, ten other people had the exact same idea. There was just something in the environment that made it the right time for folks to think that one up. The race has already begun! Who’s going to execute first? Who’s going to execute best? If you want to waste nine months trying to raise VC money for that idea, great. But six months in, you’re gonna cry when you see someone else put out that same product you’re pitching me right now. Like I said, forget everything else and just get your product out the door. Now.

…First and foremost, find a great founding team. One person is almost never enough. You just can’t do it all. Rather, team up with one or two other people who have skills synergistic – not overlapping – with your own, but with similar goals and passions. I can’t tell you how frequently teams of three business school students tell me they’re going to start the next great consumer Internet company. When I point out that they’re all business people, and wonder who’s going to build the product, they almost always fall back on “we’ll get a couple of undergrads to do it,” or, “we’ll outsource it.” If I hear either one of those, I know the startup’s already dead. Sorry, folks. Harsh, but probably true.

Working with Symfony is the first time I’ve worked with Yaml files. As such, I still have to look up the info when I’m trying to do something like get an array out of a mass of Yaml configurations settings. Searching around, for info on the subject, I found the following, helpful conversation.

ifilipov writes:

Hey guys, my navigation is located in app.yml

all:
  navigation:
    Home: index.php
    About us: about.php
    Events: events.php
    Mailing List: mlist.php
    Contact Us: feedback.php

i’m trying to put all this in array, but it doesn’t work.

$nav = sfConfig::get(’app_navigation’);
print_r($nav);

halfer replies:

I believe to collapse things down to an array, you can also use the dotted syntax, something like this:

all:
   .settings
     navigation:
       Home: index.php
       About us: about.php
       Events: events.php
       Mailing List: mlist.php
       Contact Us: feedback.php

The dotted word can be anything, and is ignored by the YAML parser. Then to access it:

$nav = sfConfig::get(’app_navigation’);
print_r($nav);

Now things get strange.

I thought it would be useful to double-check with the official Yaml spec, to see what the dot really means. But the dot is not in use in any of the examples given in the spec. Instead,

To do a 2 dimensional array, there is this:

Example 2.3. Mapping Scalars to Sequences
(ball clubs in each league)

american:
   – Boston Red Sox
   – Detroit Tigers
   – New York Yankees
national:
   – New York Mets
   – Chicago Cubs
   – Atlanta Braves

Here is something I take to be what in PHP would be considered an associative array inside of non-associative array:

product:
   – sku : BL394D
      quantity : 4
      description : Basketball
      price : 450.00
   – sku : BL4438H
      quantity : 1
      description : Super Hoop
      price : 2392.00

But I find no examples using the dot notation. Which is odd.

(As to the conversation on the forum, partly I link to these articles to give them some rank with Google. I find that Google does a poor job of figuring out what pages are relevant to Symfony searches. I think this is partly because Symfony bloggers don’t link to many tutorials or quick tips, so Google doesn’t have many links on which to form a judgement about what is important.)

I’ve already mentioned one use of public_path(), which is to leave as much HTML as possible in the templates, so that designers have an easy time working with them.

This conversation was related, and caught my interest.

Mo Mughrabi wrote:

Generate absolute URL for upload directory

I was trying to generate an absolute URL inside my form classes using sfContext::getInstance()->getController()->genUrl(); but apparently genUrl only can be used when supplied a route or module/action but i cannot get it to return the absolute path of my uploads folder. Any idea how can i do that?

Eno responded:

We never found a good solution for this (we run across a cluster where we need absolute and cross-domain links). Now we are using the sfDomainRoutePlugin to give us this functionality in url_for() and link_to().

And then Mo Mughrabi said:

I found a way which helped me return the absolute url for any location i
need through the public_path helper.

First i had to include the helper through the getConfigurations

sfContext::getInstance()->getConfiguration()->loadHelpers(array(’Url’));

and then just use the public_path(’uploads/file.jpg’);

Reading this. I thought this was a funny commit note:

added more deprecated stuff

I mean, you can kind of guess the meaning, but the plain meaning suggests they are adding in code that is deprecated and should, therefore, be removed from the code.

Sometimes the append() method in jQuery will move a node, and other times it will clone the node:

jQuery’s .append() behaves different on it’s input depending on if it’s appending to a single node or to multiple nodes. Appending to more than 1 target node:

will not remove the input from the DOM
will clone the input and append to the selection

but, appending to exactly 1 target node does the opposite:

will remove the input from the DOM
will not clone the input and append to the selection

As far as I know, this behavior is true for all functions in the Manipulation area of the docs.

Tom Boutell explains some features that are hidden in his pkToolkitPlugin plugin for Symfony:

* The pkHtml class. Many projects allow users to edit content via a rich text editor. And we all know what happens if the user pastes a Word document in there: the styles of the page wind up hopelessly munged. You can use striptags(), but that doesn’t clean up the CSS, so your page is still a mess.

A common workaround is to use FCK’s “paste as plaintext” mode, which thwarts attempts to paste rich text from another program. That works, after a fashion, but it’s frustrating for users. And none of the workarounds help if the user is actively trying to enter inappropriate HTML in a misguided attempt to re-style their site… then calling you to fix it for the 500th time.

HTML Tidy can do the job, but it has a reputation for being heavy and slow.

Or… you can just use pkHtml::simplify():

There is a woman in England who maintains a blog 100% devoted to listing the weaknesses and failures of WordPress. In this, she provides a much needed service. Lord knows, WordPress has weaknesses. I offer this as a sample of her wisdom and scathing wit:

aw, Scoble got hacked after abandoning wordpress.com for not letting him be quite Special enough to have plugins. One cannot blame Matt for experiencing some schadenfreude, though blaming the host rather than Scoble sounds disappointingly like he’s scared of losing any chance of him and his dollars coming home to VIP-land. I bet he wouldn’t be criticising Rackspace if it were on the wordpress.org affiliate page.

Seriously, though, why is anyone who gives a shit about not having their blog hacked still using wordpress? Do they actually enjoy having to upgrade every couple of weeks? I never thought I’d say this, but there are more important things than pretty themes when you’re choosing a blogtool. I think we can now officially declare that WP is the Windows of blogging. It’s easy, it’s convenient, but the tradeoff is YOU GET WORMS.

This post of hers deserves double points, since she not only attacks WordPress but she also criticizes the second biggest cry baby the Internet has yet produced (only Scoble’s mentor does a better job of being a cry baby: “‘Why has my personality become the issue? They’re using that to try to get me to shut up,’ Winer said in an interview“).

I use WordPress to run this blog. It is one of those platforms that seems to have a lot of potential, though the constant security issues make it a huge hassle over the long term. I think at some point Matt Mullenweg will either have to get much more serious about the infrastructure of the platform, or eventually something like CodeIgniter/ExpressionEngine is going to eat away WordPress’s market share.

I became aware of Javascript libraries in 2005. For a long time I used prototype for everything. I was dazzled by the profusion of libraries: mootools, YUI, ext, and more, but I never had time to experiment with all of them. They all basically did the same things, though they represented different philosophies about how to do things. Then last year it seemed like the community of developers that I followed had suddenly agreed that jQuery was the best library. I’ve been using jQuery for less than a year, but I like it. I think it is strange how, with these libraries, people who never cared about the speed of their Javascript now worry about the speed of their, say for instance, jQuery script. Down in the comments, under this article about the fastest method for turning a string into DOM elements and adding them to the page, I thought this remark had some truth in it:

I find it curious that developers were drawn to tools like jQuery because they abstracted away this sort of thing, yet now that adoption is high people that never did it pure js are looking for speed and moving back in the direction of string concats and straight js.

Just for speed comparison sake have you tried this test using innerHTML and createElement instead of anything jQuery? I don’t think I would want to return to those days, but it would be interesting to know the cost of using jQuery at all. As a side effect of doing that test it would also show the speed diff between those two old rivals, the “correct” DOM approach vs the “hack” string injection. In browsers a few years back the hack always beat the DOM, don’t know if that is still the case today.

My last post was a link to a post by Wez Furlong. Since he gave props to Sara Golemon for the work on PHP streams, a subject I want to know more about, I decided to look her up. Google highlighted a post on her blog about a surprising incident of homophobia, which I’ll link to here. She tried to accompany a friend to her mother’s funeral, but her step-father hired an off-duty police officier to keep the friend out of the funeral, because she was gay. I think it is really disappointing that these kinds of things still happen.

We landed together at CLE on friday night and headed over to her dad’s house (her parents had been long divorced). He greeted us, filled her in on the past few months and her mother’s last days, and we left feeling the weight of the occasion, but ready to grieve. On the way out, her sister called saying, “Some of the family have asked that you don’t show up to the viewing. They say you can come into the funeral home for a few minutes afterward, but not during the general viewing.”

I know, take a minute to absorb that statement. Who are these family members? Vague mumbles always answered this question, but they were certainly not going to tolerate my friend’s presence. We resolved to call around the next day…

It was Saturday and the obituary had come out. Anachronistic newsprint mourned the loss of a sainted mother to my friend’s sister, “special mother” to my friend’s ex-spouse, but no mention of my friend. They had simply erased her, with a nod to her ex as a special kind of insult.

My friend called her sister back, wanting to understand how so much animosity could survive and flourish like that. She wasn’t going to be erased, so she told her sister that the family could just deal with the fact that she would be present at the viewing. It was her mother that’d died, and she intended to say good-bye. A few hours later, news had filtered through to her step-father who called and delivered a matter-of-fact edict. “Here’s your options, and I’m only going to say this once. You can come at 7, after everyone else is done, and take your 15 minutes to say good-bye, or you can not come at all. I’ll have a cop at the door to make sure of that. *click*”

Yes, you heard that right, he was hiring an off-duty police officer to keep her out of her own mother’s services.

And before you ask, yes, her step-father had that legal right, as the funeral home was private property contracted by him. So it was, that the next day, we (My friend, another friend of her’s, and myself), drove to a shopping center near the funeral home and parked where our cars couldn’t be identified and vandalized, and we walked to the funeral home at the official start time. Sure enough, a uniformed police officer stood guard at the entrance.

Seriously, how twisted with fear and loathing do you have to be to keep a person out of their own mother’s funeral?

I have to look this up every time I do it, so I’ll post it here so I can refer back to it easily. The following script is up on Wez Furlong’s website, HTTP POST from PHP, without cURL. He is on the core PHP team. I’m sad to say that he writes:

I don’t think we do a very good job of evangelizing some of the nice things that the PHP streams layer does in the PHP manual, or even in general. At least, every time I search for the code snippet that allows you to do an HTTP POST request, I don’t find it in the manual and resort to reading the source. (You can find it if you search for “HTTP wrapper” in the online documentation, but that’s not really what you think you’re searching for when you’re looking).

Damn, it is a bad sign when folks on the core team are also at a loss. The inconsistency of the PHP language makes it difficult to remember how to do things.

Anyway, here is the script:

function do_post_request($url, $data, $optional_headers = null)
{
$params = array(’http’ => array(
‘method’ => ‘POST’,
‘content’ => $data
));
if ($optional_headers !== null) {
$params['http']['header'] = $optional_headers;
}
$ctx = stream_context_create($params);
$fp = @fopen($url, ‘rb’, false, $ctx);
if (!$fp) {
throw new Exception(”Problem with $url, $php_errormsg”);
}
$response = @stream_get_contents($fp);
if ($response === false) {
throw new Exception(”Problem reading data from $url, $php_errormsg”);
}
return $response;
}

$optional_headers is a string containing additional HTTP headers that you would like to send in your request.

He gives credit to Sara Golemon for much of the work on streams in PHP.

Clients occassionally ask me if I can help with video ideas that they have. I always say no – video is not my specialty. If you need help with video production, consider using Redhouse Productions, run by my friend Meghan Eckman. I’ve seen some of the documentary work she’s done, and it is impressive. And I know she’s has many years of experience with post-production work, and she’s worked with a very wide array of software. She can help with a project at any stage, from the initial conceptualization to the final edits.

Mike Dover links to a study that says only 13% of Wikipedia’s contributors are female. Dover then quotes Phillip Greenspun offering a theory about why:

“A lot more men than women choose to do seemingly irrational things such as become petty criminals, fly homebuilt helicopters, play video games, and keep tropical fish as pets (98 percent of the attendees at the American Child Association convention that I last attended were male). Should we be surprised that it is mostly men who spend 10 years banging their heads against an equation-filled blackboard in hopes of landing a $35,000/year post-doc job? …Young men strive to achieve high status among their peer group. [Yet] men tend to lack perspective and are unable to step back and ask the question “is this peer group worth impressing?”…

So it is with Wikipedia. Why invest your free time wrangling with a politicized Wikipedia bureaucracy of infighting editors and bitter story subjects, all for the honor of creating a free resource for other people and paying out of your own pocket to go to high-level meetings for the Wikimedia elite? If you’re a man, for the honor of being near the “top” of something, no matter how fruitless.

Greenspun is always a mix of insight and blind spots.

There is something appealing about such a pleasant story. In this version of the world, women do not face discrimination, instead, they avoid certain professions because they are wiser than men. That is a clever argument, though probably too clever. There is a whiff of a fairy tale hidden in that happy ending – women are wiser than men and no doubt end up happier than men because they avoid all that bothersome competition over nothing.

However, the premise is suspect. Are men stupid because they engage in petty crime? Does petty crime ever pay? If, for instance, you can make good money selling drugs, then crime might be a rational decision. Flying a home made helicopter may not be wise, unless is brings you publicity you can use. Then it becomes a rational decision.

I’ve read that young men tend to take more physical risks than women. That might be stupid. Or that might be rational. Some risks have high pay-offs. In a larger context, and in the context of career, the more you encourage someone to avoid big risks, the more you may also be encouraging them to avoid big pay offs. Whether or not that is rational depends on a hundred variables – it is never a simple judgement to make.

Despite my reservations, I can imagine some grain of truth in what Greenspun says. In her book You Just Don’t Understand: Women and Men in Conversation Deborah Tannen says that men view themselves as belonging to a hierarchy whereas women view themselves as being embedded in a network. I’ve never been comfortable with that generalization, as it seems to imply that women are innocent regarding social rank. I can not imagine a human being who doesn’t understand social rank. I would be interested, though, if men and women tend measure themselves against hierarchies of different scales. Greenspun’s comment could be read as mocking men who compete for status in hierarchies of very small scales. Certainly I can recall instances where a man is excitedly, urgently explaining to a woman why some small success in some small sub-culture is actually of great importance, and the woman sitting there somewhat bemused, trying to understand why the man sees such importance in so small a triumph.

But this is something that women never do? If we are going to deal in anecdotes, it seems easy enough to find countervailing examples. For every man over-thrilled at his advance to a new rank in a chess tournament, there is a woman over-thrilled at the new pattern her knitting group has decided to work on.

The strategy of Greenspun’s comment is an ancient one. The overall gist is “Men are oppressed by strife and worry, competition and war; women are wiser than all that and remain blissfully happy by focusing on things that have real lasting value.” You can find variation of Greenspun’s comment in every country, in every century. Depending on the century, and the country, things that “have real lasting value” can be defined differently: children, god, honoring one’s ancestors, etc. But the end result of this reasoning, even when it is well-intentioned, is to provide justification for keeping women out of those professions that they might otherwise choose to pursue.

The tech industry has a problem. It is moving against the modern current. Almost all other major professions have opened their doors to women. Women now make up nearly 50% of all new doctors, and almost 40% of all new lawyers. Yet the number of women who receive advanced degrees in computer science peaked in 1989 and has since declined. Disparities of men versus women in any particular online pursuit needs to be closely examined, not cleverly made to seem unimportant.

Ken Robinson suggests that our educational systems destroy the creativity of the students.

When I wrote “How much do websites cost“, I listed some basic errors that many of my clients make, including this:

Assumptions To Avoid

…5.) The idea for my site is unique, therefore completing it quickly is urgent. If we are not the very first site that uses this idea, then we will fail. My idea is so good that others will soon imitate it.

… 12.) I have no experience on the web, but I have uniquely creative ideas, which I think will be enough for me to build a successful online business.

Apparently this kind of reasoning is also rife in Hollywood:

Which brings us to an ugly truth about many aspiring screenwriters: They think that screenwriting doesn’t actually require the ability to write, just the ability to come up with a cool story that would make a cool movie. Screenwriting is widely regarded as the easiest way to break into the movie business, because it doesn’t require any kind of training, skill or equipment. Everybody can write, right? And because they believe that, they don’t regard working screenwriters with any kind of real respect. They will hand you a piece of inept writing without a second thought, because you do not have to be a writer to be a screenwriter.

In the end, execution matters. It doesn’t matter if you are writing or creating a website. Execution matters. Creativity, by itself, doesn’t get you very far. You also need to know how to execute.