Closer To The Ideal

Colin Steele gives Scala a grade of FAIL:

I’m here to report, at the end of Day One, that so far, Scala earns a FAIL.

I know I’ve been poisoned by years of configuration-less applications, of one-line quickie library installation and dependency resolution, and intuitive runtime control. I know, I’m spoiled. Rotten.

Lord have mercy. Scala may end up as the next Big Thing for Java, and I’m happy for all you Java guys out there who won’t have to do so much typing in your class definitions. But me, maybe I’m missing something, but Scala just sent me running back into the arms of Ruby. Or heck, even C. Maybe I’ll learn IO or Go after all.

I have not looked at Scala but I do have the impression that it is aimed exactly, precisely, 180 degrees opposite of the folks who like Ruby. Scala seems to appeal to programmers who LOVED Java, but who just wanted certain things in Java fixed. Some fans of Scala regard is as the 2.0 rebirth of Java. People who dislike Java are going to dislike Scala.

There are languages on the JVM that have clearly borrowed a lot of ideas from Ruby. Groovy would be an obvious example. But Scala is not such a language.

Interesting comparison of Java and Ruby:

But now I’ve conveniently landed on an actual conclusion. And here it is. Remember in that Elements of Programming Style review, I drew special attention to the first rule in the first proper chapter — “Say what you mean, simply and directly”? The more that runs through my mind, the more convinced I am that this deceptively simple-sounding aphorism is the heart of good programming. Seven short words; a whole world of wisdom.

And how can I say what I mean simply and directly if I’m spending all my time allocating temporary arrays and typing public static void main? My code can’t be simple if the functions I’m calling have complex interfaces. My code can’t be direct if it has to faff around making places to put intermediate results. If I am going to abide by the Prime Directive, I need a language that does all the fiddly stuff for me.

Worth knowing about: a jobs board for Ruby On Rails developers.

2 posts on deploying web software with Capistrano.

Craig T Mackenzie writes:

Like most people I use capistrano to deploy my rails applications, at work we host with the excellent railsmachine and they have really helped simplify the process of deploying to their servers using the railsmachine gem, which builds on top of the excellent functionality of capistrano.

The way these tools work by default they only allow you to deploy one instance of your application, which is fine, that’s all they’re are intended to do. But in the client facing world you’re probably going to want to have at least two versions of the same application at different stages of development running on the same server.

A live forward facing version (my-app.com) and a staging version (staging.my-app.com) for client approval / production testing (not code testing, that should stay on your development box) / progressive reviews etc.

So how do we achieve that? The idea is to determine what context you are deploying your application in, and use the fact the capistrano tasks can be chained together to set everything up ready for deploying a revision of your application to a targeted url, independent of other deployments.

Jamis Buck writes:

I’m still really pushing back against adding staging support into
Capistrano itself. You can accomplish what you want without using
environment environment variables by using cap’s -S switch:

cap -S stage=production deploy

Then, your deploy.rb looks like:

# set the default, unless it was set on the CLI
set :stage, “development” unless variables[:stage]

# do the setup, based on the selected stage
case stage
when “production”
set :deploy_to, “…”
role :web, “…”
role :app, “…”
…
when “development”
…
when “demo”
…
else
raise “unsupported staging environment: #{stage}”
end

Colin Steele writes about the challenges that he and his team are facing at Hotelicopter:

We’ve officially run headlong into one of Ruby on Rails’ deficiencies: programming in the large. We’re not interested in computer science-y solutions, only pragmatic ones.

I’m curious what is excluded by the phrase “computer science-y solutions”? I would normally interpret that to mean “we are looking for well tested solutions with wide deployment” but elsewhere he writes:

We’re currently investigating a spectrum of new technologies in the NoSQL realm, including Tokyo Tyrant, MongoDB, Amazon’s SDB, CouchDB, Voldemort, and more. Tis’ a dizzying mix, and things are popping in the space.

So clearly they are looking at some cutting edge technologies. Colin links to an essay about Programming in the large which includes this:

Maintenance and locality are strong arguments in favor of immutability. The less aspects of an object can be changed, the less you have to worry about the execution history. If an object has a two-phase initialization sequence (e.g. this is so in C++ if you need a virtual function during initialization), you have to make sure that the objects are properly initialized; code that gets handed over such an object will have to check that it’s initialized (if only in an assert()). This all vanishes if the language makes sure that no object remains uninitialized, and doesn’t force a two-phase initialization on programmers like C++. (In C++, the “wrong” design decision was that objects mutate from the base type to the subtype when the various constructors are run. It’s this kind of far-reaching consequences (IOW non-locality) that makes language design an art.) If you take immutability to its extremes, nothing can ever be changed. If you wish to change the world, you write a function that returns a list of changes and let the run-time system inspect that list and execute the proper actions. If you have an interactive program, you emit a list that has a function pointer at its end; the function gets fed the next input and is expected to generation another action list.

In the last few years, many programmers have pointed to mutability as one of main problems they face when they build larger systems. Chas Emerick does a good job of highlighting the problem in his post “All my methods take 316 arguments, and I like it that way”

316 arguments to a method (which I don’t think is actually possible in the jvm, but bear with me)? “That’s absurd!”, you’d say. The problem, of course, is that the 3-arg doSomething actually has far more arguments than its signature implies:

The behaviour of every function in a mutable, imperative environment is dependent upon the state of all of the other (variables|attributes|bindings|whatever) in your program at the time the function is invoked.

So, if you have 313 other variables in your program, that 3-arg doSomething is functionally (ha!) operating over 316 arguments.

Would you ever intentionally write a method signature that takes 316 arguments? Would you use any library that contained such a function signature? No? Then why are you using tools that force such craziness upon you?

Chas says “The languages are ready” and he links to some of the major functional languages: Erlang, Clojure, F#, and Fantom. In comments, his readers add in their favorite functional languages: OCaml, Haskell, etc.

One of his readers challenges the idea that functional languages are safer than imperative languages by offering this:

Regarding functions changing state, what about things like this in clojure, Isn’t it like global variables in imperative lang?

(def state (ref #{}))

(defn function that updates state)

(defn another function that updates state)

Chas responds:

You bet. Clojure is not a purely functional programming language, so you can have as much shared state as you want – but the language is going to make you work for those bits of shared state, so you have to “pay” for them. Conversely, imperative languages like Java et al. make you work to achieve immutability, and provide nothing in the way of enabling persistent data structures, etc.

The point is that defaults matter, a lot.

I like the word “default” in this context. In his 2001 book, Effective Java, Joshua Bloch wrote “Favor immutable objects over mutable.” When writing a big system in Java, you work to make your system immutable. In a language like Clojure, the default is just the opposite – you work to make parts of your system mutable.

I have very little experience with functional programming. I am just learning Clojure now (Lisp redone for the JVM). I can not say what benefits its brings. I’m looking forward to learning more in this area. It’ll be interesting to see where functional programming comes to be regarded as a “best practice”. Certainly, it will be interesting to see if CTO’s start using these languages at startups, or whether they will be regarded as “computer science-y solutions”.

I’ve somewhat more experience with the web app frameworks that have emerged since 2004. I’m interested in what Colin wrote here:

I suspect as we muddle along we’ll develop a component-level (service level if you prefer) version of the Law of Demeter, which will drive us to make the right decisions for decoupling. I’m not too worried about that. However, we definitely have issues with reuse. Currently the Ruby on Rails state of the art solution for reuse is the gem. Which, let’s face it, is a pathetic solution.

Some of the frameworks seem to encourage bad habits. I’ve already written of Symfony’s weaknesses in Symfony versus The Law Of Demeter: does Symfony promote bad habits?.

When Ruby on Rails first emerged it was targeting web apps, not web services. Rails has a lot of imitators: Groovy/Grails, PHP/Symfony, etc. These all help create web sites, but not necessarily web services. I suspect a new generation of frameworks will be needed to make this kind of work easier:

The place we’re aiming for is a highly decoupled (and scalable), cohesive set of services, joined through REST APIs and/or fully reused common business models.

In their book Restful Web Services the authors Leonard Richardson and Sam Ruby talk about “the human web” and the “programmable web”. This is from page 2:

The Web you use is full of data: book information, opinions, prices, arrival times, messages, photographs, and miscellaneous junk. It’s full of services: search engines, online stores, weblogs, wikis, calculators, and games. Rather than installing all this data and all these programs on your own computer, you install one program – a web browser – and access the data and services through it.

The programmable web is just the same. The main difference is that instead of arranging its data in attractive HTML pages with banner ads and cute pastel logos, the programmable web usually serves stark, brutal XML documents. The programmable web is not necessarily for human consumption. Its data is intended as input to a software program that does something amazing.

Originally, frameworks like Rails were created to help speed the production of sites for the human web. They have evolved since then, Rails in particular. In fact, Richardson and Ruby use Rails for many of the examples they offer in the book, about how to correctly build a RESTful web service. And yet, the scaffolding systems in these frameworks still tend to automate the production of CRUD web pages, rather than PGPD services. (I do not know the state-of-the-art with Rails, so someone can tell me if I’m wrong about its scaffolding.)

Richardson and Ruby suggest that every module (resource) in a RESTful web service should expose just 4 actions:

POST
GET
PUT
DELETE

These are the HTTP verbs, and they roughly correspond to the standard CRUD actions, except that POST is used for both Create and Update, and PUT is used for uploading files:

Create/Update
Read
Upload
Delete

I suspect we need a new generation of frameworks, or at least new scaffolding systems for the existing frameworks, that automate the setup of PGPD services. That seems like the next obvious step forward.

It is now clear that the intensity of the praise focused on Ruby On Rails in 2005 was partly a revolt against Java. “The Java hyper-enthusiasts have left the building” signaled the end of an era, and the end of a particular type of Java framework – heavy, overly complex, overly strict.

The Java hyper-enthusiasts have left the building, leaving a significant contingent of Java programmers behind, blinking in the bright lights without the constant drumbeat of boosterism. But the majority of programmers, who have been relatively quiet all this time, always knew that Java is a combination of strengths and weaknesses. These folks are not left with any feelings of surprise, but instead they welcome the silence, because it’s easier to think and work. Where did the hyper-enthusiasts go? To Ruby, apparently. This is chronicled in Bruce Tate’s book “Beyond Java,” which should probably be titled “Why Ruby is Better than Java.”

That was December of 2005. At that time it seemed like Java was slowly going to fade away, replaced by lighter languages that encouraged agile practices.

But that is not what happened. Instead, the Java platform evolved:

The idea of “Enterprise Ruby” has become less repellant since Dave Thomas’s infamous keynote at RalsConf 2006. There are a lot of large, lumbering organizations out there that have yet to adopt any of the newer agile language/framework combinations, and Rails has most definitely led the way. I personally believe that in order for Ruby to become more than just a nice language with a great community, it needs to gain adoption in those organizations, and it needs to do it damn quickly. JRuby is by far the best way for that to happen.

There’s another aspect to adoption I think has escaped a lot of Rubyists. In 2006 and 2007, Ruby gained a lot of Java developers who were running away from bloated, over-complicated frameworks and the verbosity and inelegance of Java. When I asked at Ruby conferences in 2005, 2006, and 2007 how many people had done Java development in a former life, almost everyone in the room raised their hands. When I’ve asked the same question in 2008 and 2009, it’s down to less than half the room. Where did they go?

The truth is that the Java platform now has reasonably good answers to Ruby in Groovy, Scala, and Clojure, and reasonably good answers to Rails in Grails and Lift.

I thought the title of this article was funny:

Converting legacy Rails apps to Grails

So now there are “legacy Rails apps”. Sort of makes Rails sounds really ancient. One of the dead languages. You know, Cobalt, Fortran, Rails…

[Update: I've re-written this post to take into account the information that Javier Eguiluz posted in the comments.]

[Update: Jacob Coby points me to Google Trends, which offers the visuals for the numbers that I quote below.]

TIOBE only tracks computer languages, not frameworks. I’m interested in how Symfony compares with the other PHP MVC frameworks. Borrowing the search idea from TIOBE, I just ran these searches:

symfony

codeigniter

cakephp

drupal

The results (how many hits on Google):

Symfony – 6,270,000 (3,900,000 for ‘php’ and ’symfony’ together)

CodeIgniter – 748,000

Cake – 4,540,000

Drupal – 28,500,000

A comparison: Ruby On Rails: 12,600,000

Meanwhile, WordPress blows away everything else: 295,000,000

My guess is that WordPress has retarded the growth of PHP MVC frameworks. The 80/20 rule applies here with some force. WordPress meets the needs of most people who need a website. PHP allows for self-contained software, such as WordPress, which is something the world of Ruby has not seen. Ruby on the web has largely meant Ruby On Rails, which one has to be a programmer to setup and use. Designers, intelligent people who are not programmers, all such people can default to WordPress.

From the point of view of a computer programmer, the code in WordPress is fairly awful. But designers love WordPress, and it needs to be given credit for successfully creating a package that designers feel comfortable with. No designer would know how to set up a Ruby On Rails site, but most web designers know how to set up a WordPress site. And for all its limitations and flaws, it must be acknowledged as the dominant platform written in PHP. Compared to WordPress, all the MVC PHP frameworks are just a footnote.

I’m going to repeat these searches every 3 months, and see how these ranks change over time.

Back in 2005 I read The European Revenge. It was published in 1973. It was written by two journalists from the Economist magazine. They made the daring, controversial argument that at some point in the near future, European car makers such as BMW and Mercedes would be able to make cars that had just as much quality as American cars.

Their daring idea is something for my generation to laugh over. I came of age in an era when it was simply assumed that European cars were naturally better than American cars. Over the course of 30 years, the reputations of the various car makers had reversed positions.

Tonight, I find myself thinking daring, controversial thoughts about Ruby On Rails and PHP.

I recall in late 2005 when I first started hearing about Ruby On Rails. It was an amazing system for building big web sites much faster than anything before had allowed. I watched the video that showed how you could build a weblog system in 15 minutes. The company I worked at then hired a consultant who was also a very great programmer, a fellow who, by coincidence, had been writing about Ruby since 1999, well before most American programmers had heard of the language.

At that time, it seemed like Ruby On Rails was going to sweep all before it. PHP seemed like a dinosaur, headed for extinction. Software written in PHP was notoriously sloppy and insecure, and the programmers who worked with PHP tended to be less experienced than the programmers in almost any other programming community.

I started to study Ruby On Rails on the assumption that all PHP work would eventually disappear, and if I wanted to have a job in 2008, I’d better get with the new, new thing.

But, of course, Ruby On Rails failed to sweep all before it. It had a big impact on programming in general and on web development in particular. But of all the programming communities, Java was probably more profoundly effected than PHP. For Java programmers, Ruby On Rails lead to a re-thinking of fundamental assumptions.

When Ruby On Rails first came out, PHP was not competitive. There were no high productivity frameworks for PHP. But that has changed. PHP now has Cake, CodeIgniter, Symfony, and many other frameworks. These are all full-featured MVC frameworks, with command line scaffolding tools that automate the construction of basic CRUD forms and basic layout templates, and which therefore offer some of the same productivity gains that Ruby On Rails offers.

So much contempt has been poured in the direction of PHP that it seems normal to think it must be inferior to every other programming language. But it is worth remembering that some very good programmers have never liked Ruby:

I classify myself as one of those developers who don’t get Ruby, just like I don’t get Perl. Both are too hard for me. And there seems to be something in my brain that prevents me from learning both. It’s strange I never had the problem with C or C++ or Python or Lisp or awk.

…I just don’t like Ruby. This has more to do with how I think and work than with the true quality of the Ruby language. I’m sure Ruby is a fine programming language offering good OO features. But there is something in it that rubs me the wrong way. I tried to learn it when the first edition of the PickAx book came out. But somehow I can’t get past the “we are so OO, we write 3.abs instead of abs(3)” noise. I’m sorry. It’s not for me. There. I said it.

Most programmers first heard of the Ruby language when they heard about Ruby On Rails, so the language and the framework have always come in a bundle. If we unpack that bundle, do we find that some portion of it is stronger than the other portion? Since some good programmers dislike Ruby, yet the whole programming world has fallen in love with high-productivity frameworks like Rails, isn’t it safe to assume that Rails, the framework, is what made Ruby On Rails so awesome? That the awesomeness is clearly not tied to the language is demonstrated by the vast number of frameworks that have now been implemented in other languages. Most of these framework imitate aspects of Rails. If the awesomeness of Ruby On Rails was due to the Ruby language, then surely more programmers would be learning Ruby, rather than trying to imitate Rails in whatever language they know best. Many of these new frameworks have become successful in their own right, and have kept programmers in one language who might have otherwise converted to Ruby. Certainly, back in 2005, it seemed like there was likely to be a mass conversion of Java programmers to Ruby On Rails. That mass conversion was forestalled by the emergence of frameworks like Grails, written in a language derived from Java.

If the language Ruby doesn’t explain the awesomeness of Ruby On Rails, then that awesomeness can be captured by other languages, including PHP. I’ve been working a lot lately with Symfony, and I am impressed with the tools this framework makes available to me. Although all of the PHP frameworks had a late start, relative to Ruby On Rails, the PHP community now has a much greater diversity of frameworks to choose from, and the competition between the various PHP frameworks is likely to cause all of them to improve at a rapid rate, and also to specialize toward particular niches.

Where does that trend get us in another 3 years? Just as Mercedes went from being an inferior car to being a superior one, it seems to me that we could end up in a situation where the PHP frameworks are actually better than Ruby On Rails for any particular purpose that you can imagine.

I am dissatisfied with the HTML that is auto-generated by the built-in scaffolding system of Symfony.  I am planning to write a plugin that will generate HTML more to my liking. I know that, in the world of Ruby On Rails, there are several plugins that improve upon the built-in scaffolding system of that framework. Since these plugins have a longer history, and are more advanced, than anything in the world of Symfony, I am looking to them for some inspiration.

This is what Hobo says about itself:

It turns out that the hard part is not going fast, but staying flexible. This is where we think Hobo really shines. If you’ve played with “app builders” before, you’ll know about The Wall. The Wall is the point you reach where you have to give up and do it the old way because that one feature you really need just isn’t going to happen. Hobo doesn’t have one.

…Like Rails itself, Hobo encapsulates a number of opinions about web application development. The opinion that is most central to Hobo is this: we have found that there is a great deal of similarity from one web application to the next. Much more similarity in fact, than is supported by today’s frameworks. We would rather implement this stuff once.

Of course this approach is common to all frameworks—everything that Rails provides is there because many or all web applications will need it: database connectivity, session management, working with HTTP, etc. etc. The difference with Hobo is that we are trying to take this idea to a much higher level than is normally expected from a web framework. The ultimate goal is: don’t program your application, just declare it.

This is what ActiveScaffold says about itself:

Most web applications have many more model objects exposed on the backend, or admin side, than they do on the front. Coding interfaces for all those models is redundant and a waste of resources when all you need is CRUD functionality that’s smart enough to handle all your ActiveRecord associations.

Enter the ActiveScaffold plugin:

• An AJAXified table interface for creating, updating, and deleting objects
• Automatic handling of ActiveRecord associations
• Sorting, Search and Pagination
• Graceful JavaScript degradation
• RESTful API support (XML/YAML/JSON) baked in
• Sexy CSS styling and theming support
• More extension points than you can shake a stick at
• Guaranteed to work on Firefox 1+, IE 6+ and Safari 2+
• Released under the MIT License, the same one as Rails itself, so you can use it freely in your commercial applications.

This is what Streamlined says about itself:

Streamlined is a plugin for Ruby on Rails that provides an instant, production-ready UI for your ActiveRecord models. It started as a way to generate Administrative back-ends, but has become more general purpose and flexible over time. Streamlined aims to bring the declarative goodness of ActiveRecord to the view tier. It manages the presentation, creation, and editing of model instances, providing full-featured scaffolds equipped with everything you need to quickly and easily develop a rich user interface for interacting with your data, including:

• Zero-configuration relationship management
• Ajax-powered widgets and transitions
• Live data filtering
• Out-of-the-box sortable lists and pagination
• Exporting to XML, CSV, and JSON
• Declarative and easily-customizable UI development
• Cross-browser support including Firefox, Internet Explorer, and Safari
• Pluggable CSS styling

This gives a sense of the possibilities.

Hobo has an especially interesting goal, to create a framework that is declarative, rather than imperative.

In the short term, my goals for a new scaffolding system are simple:

1.) The HTML should use divs and CSS rather than tables.

2.) Pagination should be included by default.

3.) An administrative dashboard should be created by default.

I know I will want these 3 things from every Symfony project I start, so, for me at least, it makes sense to have  a plugin that automates all this.

Jamis at 37 Signals has up a strange post in which he is critical of modules. I say “strange” because his tone is defensive and he interrupts his post to repeatedly to insist that modules are really great:

The code remains really clean, overall, because we’ve continued to follow the pattern of moving related methods into modules, and just including those modules in the base model.

Got that? Refactoring code out of your main classes and into modules is  a great way to keep your code clean. He emphasizes:

Don’t misunderstand me, though. Modules are awesome. We use them a bunch, and love them.

and then:

Modules are very handy, and as I said before, we still use them extensively.

He seems worried that someone is going to criticize him because of his criticism of modules. Yet his criticism is almost the same as Michele Simionato wrote in last month’s explicitly anti-module post.

Jamis writes:

However, there are two issues with this approach. First, when a class includes lots of modules (as some of ours do), it becomes hard to keep track of where different methods are defined, and even which methods are defined. This can lead to confusion when trying to find the definition of a method, and (worst case) can allow two or more methods with the same name, which is bad. There is no warning if you have methods with the same name in multiple modules, so it isn’t hard to wind up with bugs where you’ve clobbered a method with another. You wind up double-namespacing your methods (e.g., method names in the Avatar module being prefixed with avatar_), to avoid collisions.

…There are more issues with the module approach. What if you have different kinds of avatars (e.g., person versus group, or company)? You can’t subclass modules in Ruby, so you wind up finding other ways to solve it (like including modules in modules, etc.). Also, you can’t reference the avatar as a separate entity; instead, controllers that deal with avatars need to deal with the entity to which the avatar is attached.

And what solution does he offer for these problems? I was thinking he was going to say something really original, because it sounded like he was really wedded to the idea of using modules. But in the end, his solution is to simply not use modules:

Can the avatar code be split out further, somehow? Yes, it can. We’ll do this by making avatars a model of their own and using aggregation (rather than module inclusion, or inheritance) to pull it into Person.

So the solution is simple: don’t use modules, use independent classes and then build your objects through composition. This is probably very good advice, though I was a little surprised by his conclusions, given that he started off sounding like a big fan of modules.

Nikolay Barkhatov feels that a web site should just be an interface to an application:

More I think about web development more I believe there is a clean separation between actual application and web application. Moreover, I believe there is no such thing as web application. Web interface is just an adapter to real application. It’s hard to convince managers to architect application the way where web adapter decoupled from actual application. Managers simply do not see benefits of doing that and do not allocate time for it. Product development ends up with one giant mess called web application where tier separation is a formal thing everybody talks about but nobody can clearly draw the line between them. And finally there is a requirement to quickly come up with remote API for third party developers… Very sad…

It is curious to me that such an idea seems natural, even akin to common sense, to a Java programmer, and yet it has little appeal to non Java-programmers. Why is that? This is a cultural thing. The people drawn to Java are the type of people who want to do everything the theoretically (or academically) correct way.

On some level, Nikolay Barkhatov’s idea sounds very smart. And yet, a lot of other smart people have decided to take another path. David Heinemeier Hansson is a smart person, but he didn’t feel the need to separate the website from the application he was trying to build. Instead, he create the web framework Ruby on Rails, to make it easier to marry code together with a web interface, while keeping things organized. The framework implements an Model View Controller pattern, so that the code and the web interface can work together in an intelligent way. Ruby on Rails was such an obviously good idea that other people began imitating it. Every computer language now has an imitation of Ruby on Rails. For instance, for those using the PHP computer language, there is the Symfony framework.

But having said all that, I admit that I’m fascinated by the idea of separating the application from the web interface. If I wrere to try it, I would write the application in one language, and the web interface in another language. I don’t trust myself to write both in the same language, the temptation would be to cheat, and mix the two together.

If I were to try to separate the application from the website, I might write the application in Java and the web site in PHP. I would use XML to get the data from the Java appliation to the PHP. For instance, suppose I build a site for value investors who want to see whenever a stock falls below a P/E ratio of 10. The application would sit on the server, intake stock information from various sources, look for low P/E ratios, and output the results as XML files. The PHP could read the XML files and construct the website from that.

One advantage of this approach is that the caching is automatic. The PHP only needs to read from static XML files.

How to handle input from users? I suppose that is where the separation of concerns breaks down, and perhaps this is the issue that has made frameworks more popular than tiered approaches to sites. True separation could be maintained if you use normal HTML/PHP forms that are submitted to PHP code that then transforms them into XML files that the Java could then read. But this gets clumsy. I wonder if there are any types of sites where this would be the optimal setup?

Blaine Cook has either been fired or has quit Twitter. Most people assume he’s quiting/fired because of the difficulties that Twitter has had with scaling. Twitter has been a favorite weapon in the hands of Java’s defenders – they use Twitter to bash Ruby On Rails. The media attention given to Cook’s departure reminds me of an incident from a year ago that I wanted to blog about, but didn’t at the time.

The incident last year started when Radical Behavior interviewed Alex Payne, who was one of the developers at Twitter. Alex said:

By various metrics Twitter is the biggest Rails site on the net right now. Running on Rails has forced us to deal with scaling issues – issues that any growing site eventually contends with – far sooner than I think we would on another framework.The common wisdom in the Rails community at this time is that scaling Rails is a matter of cost: just throw more CPUs at it. The problem is that more instances of Rails (running as part of a Mongrel cluster, in our case) means more requests to your database. At this point in time there’s no facility in Rails to talk to more than one database at a time. The solutions to this are caching the hell out of everything and setting up multiple read-only slave databases, neither of which are quick fixes to implement. So it’s not just cost, it’s time, and time is that much more precious when people can['t] reach your site.None of these scaling approaches are as fun and easy as developing for Rails. All the convenience methods and syntactical sugar that makes Rails such a pleasure for coders ends up being absolutely punishing, performance-wise. Once you hit a certain threshold of traffic, either you need to strip out all the costly neat stuff that Rails does for you (RJS, ActiveRecord, ActiveSupport, etc.) or move the slow parts of your application out of Rails, or both.It’s also worth mentioning that there shouldn’t be doubt in anybody’s mind at this point that Ruby itself is slow. It’s great that people are hard at work on faster implementations of the language, but right now, it’s tough. If you’re looking to deploy a big web application and you’re language-agnostic, realize that the same operation in Ruby will take less time in Python. All of us working on Twitter are big Ruby fans, but I think it’s worth being frank that this isn’t one of those relativistic language issues. Ruby is slow.

David Heinemeier Hansson, the inventor of Rails, had an extremely defensive response:

Rails makes the act of developing such a pleasant experience that when you need to follow the same scaling path as every other shared-nothing stack, the contrast can feel stark. And perhaps it’s a natural reaction to feel a need to blame something for that contrast, however natural it is.

[…]

Second, when you work with open source and you discover new requirements not met by the software, it’s your shining opportunity to give something back. Rather than just sit around idle waiting for some vendor to fix your problems, you get the unique chance of being a steward of your own destiny.

[…]

Once the stress of having to deal with that in the moment subsides, I’m convinced that the team will grow beyond the blame game, get their hands dirty as full participants in an open source community, and contribute back their advances to the framework.

Gluttonous had what was, by far, the best reaction to David’s post:

The conversation shifts from where bottlenecks come from when scaling Rails to “you’re saying it’s Rails’ fault, but it’s not”. This is where we start to slide down the slippery slope. It’s easy to argue about blame. David pulls out his “I’m not a vendor, I don’t owe you shit” speech, and says that because of the beauty of open source, Twitter is in charge of their own destiny.

He’s absolutely right. He’s also a big jerk about being right. David seems to imply that Twitter has been sitting on their hands, doing nothing, refusing to solve their own problems, and not contributing back to the community. This is false. It’s also completely beside the point.

Early in the discussion we started to imagine blame where it didn’t exist. Alex never says that Rails should support doing 11k requests a second. He simply said it doesn’t, which is something no one will dispute.

So, we’ve moved into rough territory, but someone goes and does something nice (Woo hoo!). Dr Nic writes a plugin to use multiple databases with Rails. Way cool. Really way cool. Awesome job Nic. This is the part of open source that I love, when people help each other just because it’s an interesting problem and they’re a nice person. Hugs and kittens all around.

DHH tosses down an ”I told you so”. David, what are you doing man? Yes, Dr Nic is awesome, and yes, Ruby is wondrously hackable, and yes, Rails allows plugins easily, BUT why do you keep beating this dead horse? No one said anything about a critical flaw. No one said Rails is inherently flawed and can never be adapted for high traffic sites. Just the opposite is true! Alex and the other guys at Twitter are those who are pushing Rails further than it’s ever gone. These are the pioneers. They’ve gotten to the hard part, and they’re still going, despite the difficulty. Really, does anyone think that sites at that load are running out of the box copies of Rails? It’s just not the case. That’s ok. The fact that Rails can be modified is one of its strengths. There’s no need to be defensive here. That’s the big problem we all need to face.

People often say that DHH has a big ego. That is no crime in itself – in some situations, having a big ego is healthy and even useful. But this behavior – where someone points to a weakness that needs to be improved, and their remark is interpreted as an attack that needs to be counter-attacked – is a major problem. This is the kind of behavior that cripples organizations, leads to pointless in-fighting and turf wars, and saps the energy out of projects.

The solution? People (especially those in leadership positions) should welcome attacks. That way, even when no attack actually exists, as in this case, the original speaker will still get a welcoming response. There are 3 advantages to maintaining a welcoming attitude to attacks:

1.) When an attack really does exist, a welcoming response will often soften the attitude of the person making it.

2.) An open attitude to criticism can not be demonstrated in the face of praise, it can only be demonstrated in the face of criticism. Every product or idea, no matter how good, will have some flaws, and so it is important to be open to negative feedback.

3.) Welcoming attacks saves you from the embarrassment of reacting in a defensive way, when, in fact, no one was ever attacking you.

 Zed Shaw has posted a rant about everything he hates in the Ruby community. The rant is so over-the-top that is impossible not to find it entertaining. It is rare to see anyone with 21 years experience post anything like this in public. Most people in his position would be worried about their careers. He, apparently, is beyond the point of caring.

At times, the rant is almost incoherent:

Alright people, time to get a huge grip on reality’s collar and hold on tight.

Ruby on Rails is not a mother fucking industry!

Jesus fucking christ on a goddamned pike you absolute mother fucking donkey dick sucking morons get a fucking grip!

You are not in an industry. You are a bunch of people barely scraping by in a tiny little sector of a moderate sized piece of the economy. Gaming alone makes you all looks like the pathetic little crumbs I brush out of my toaster when it smells bad.

He makes a few good points though:

Where I work the company is willing to blow huge amounts of money on a consulting firm or hardware, but ends up firing people when times get tight. It’s a universal mass hysteria that paying $100 – $200 per hour for a group of consultants is preferable to simply hiring good employees. At the rates companies pay these consultants they could hire 4 full time employees.

Consultancies used to provide a service by managing the entire project so you didn’t have to do much. Now with Agile and Pair Programming the consulting firms can dupe clients into helping them make the sausage, provide little to no services, yet still charge insane rates. What’s impressive is these consulting firms somehow charge rates that are 5 or 6 times what they pay their employees.

…

Let’s take ThoughtWorks as a classic example of the hysteria. They decided to get into the Ruby on Rails game and went full bore. I was telling people right when Rails came out that doing it for internal projects at big companies would be a huge money maker. Nobody believed me, and now rather than all my smart friends working on cool applications for big money I have ThoughtWorks fucking up my party.

Before you continue this part of the rant ask yourself a question:

How did ThoughtWorks go from 0% Rails business to 60% Rails in just a few short months, but somehow didn’t hire that many top notch Ruby guys? Remember, if 60% of your business is Rails then 60% of your people need Rails training or else you have to hire more people. If they didn’t hire any more people than that means…the people they had were retrained. With two week training courses. Huh? How does that make them experts?

What happens if you do that is you have a group of former C# and Java guys running around writing shitty Ruby code and training on the client’s dime for huge fees.

Some of the post seems to border on libel:

In the two projects I’ve taken from ThoughtWorks I found mountains of horrible, horrible code. They of course try to pull the classic “there’s many ways to do everything in programming” but this time they kind of get caught because Ruby on Rails means stay on the Rails. There is an established best practice way to build web applications with Rails and that’s the entire point of the system. When ThoughtWorks fucked up these projects they did it in such a completely deviated way that it was impossible to defend.

Additionally, the people they placed on these projects were not well trained at all, had no idea about simple Ruby idioms let alone good design, and spent more of their time drinking and having fun than actually getting shit done. At the last project they actually had bottles of Pedialyte in the fridge to help with their hangovers after wild nights partying.

I write this as a note to myself, because I want to go back and look more at these systems. I’ve spent the last year developing my own scaffolding system in PHP, and like most people working on a PHP framework, I’ve been inspired by Ruby On Rails. What programmers want, nowadays, is a system that automates a lot of the grunt work of gettting a new project started. What I didn’t realize is that many people working with Ruby On Rails have given up using the default, built-in scaffolding system and are instead using 3rd party scaffolding systems. There are a wealth of these, and some of them look quite amazing. I need to go back and these out when I have more time:

 Hobo

Streamlined

ActiveScaffold