Closer To The Ideal

When to use the R language

When you have to explore data. At the start of an analytic project, it’s a good idea to create a bunch of graphical visualizations of your data to get a sense of what’s inside it. In terms of its graphical capabilities, R exists in a whole separate dimension from Excel. This was perhaps the most shocking part to me about using R for the first time: I really thought I had a handle on data analysis even though I’d restricted my software to Excel, but boy was I wrong. The visualizations you can create in R are much more sophisticated and much more nuanced. And, philosophically, you can tell that the visualization tools in R were created by people more interested in good thinking about data than about beautiful presentation. (The result, ironically, is a much more beautiful presentation, IMHO.)

Here’s how I’d put the difference to someone who’s familiar with Excel but not yet with R. The graphics creation options that Excel gives you are all based in the graphical user interface. This is what makes Excel relatively easy to use—all your options are laid out before you with nice buttons and fill-in-the-blank boxes. But in order to create a graphical interface that’s easy to use, the creators of Excel had to make a bunch of decisions about what sorts of graphics you are and are not likely to want. With too many choices, the graphical interface becomes cumbersome and frustrating, so to achieve simplicity they had to eliminate options.

And this isn’t a gripe or anything. I can’t say I’d have done a better job designing Excel’s charting graphical interface. I cut my teeth on it.

These limitations become a problem when you want to inspect data visually in a bunch of different ways in order to explore it. R, through a combination of its well-designed base graphics package, the exceptionally well-designed lattice graphics package, and the jaw-droppingly well-designed ggplot2 graphics package, offers a breathtaking array of visualization options that you access through the command line or scripts. It has power that you just can’t get using a graphical interface to generate your charts.

I’ve been going through something similar. I’ve done programming for years and I’ve never needed math skills. But recently, I’ve been thinking of all kinds of data mining projects that I’d like to do, so I’ve been reteaching myself algebra, and next year I hope to learn statistics. So I resonate with this:

A little while ago I started thinking about math. You see, I’ve been writing software for quite a few years now and to be totally honest, I haven’t yet found a need for math in my work. There has been plenty of new stuff I’ve had to learn/master, languages, frameworks, tools, processes, communication skills and library upon library of stuff to do just about anything you can think of; math hasn’t been useful for any of it. Of course this is not surprising, the vast majority of the work I’ve been doing has been CRUD in one form or another, that’s the vast majority of the work most developers do in these interweb times of ours. You do consulting – you mostly build websites, you work for a large corporates – mostly build websites, you freelance – you mostly build websites. I am well aware that I am generalising quite a bit, but do bear with me, I am going somewhere.

Eventually you get a little tired of it, as I did. Don’t get me wrong it can be fun and challenging work, providing opportunities to solve problems and interact with interesting people – I am happy to do it during work hours. But the thought of building yet more websites in my personal time has somewhat lost its luster – you begin to look for something more interesting/cool/fun, as – once again – I did. Some people gravitate to front-end technologies and graphical things – visual feedback is seductive – I was not one of them (I love a nice front-end as much as the next guy, but it doesn’t really excite me), which is why, when I was confronted with some search-related problems I decided to dig a little further. And this brings me back to the start of this story because as soon as I grabbed the first metaphorical shovel-full of search, I ran smack-bang into some math and realized exactly just how far my skills have deteriorated. Unlike riding a bike – you certainly do forget (although I haven’t ridden a bike in years so maybe you forget that too :)).

Broadening Horizons

Learning a little bit about search exposed me to all sorts of interesting software-y and computer science-y related things/problems (machine learning, natural language processing, algorithm analysis etc.) and now everywhere I turn I see math and so feel my lack of skills all the more keenly. I’ve come to the realization that you need a decent level of math skill if you want to do cool and interesting things with computers. Here are some more in addition to the ones I already mentioned – cryptography, games AI, compression, genetic algorithms, 3d graphics etc. You need math to understand the theory behind these fields which you can then apply if you want to write those libraries and tools that I was talking about – rather than just use them (be a producer rather than just a consumer – to borrow an OS metaphor :)). And even if you don’t want to write any libraries, it makes for a much more satisfying time building software, when you really understand what makes things tick, rather than just plugging them in and hoping they do whatever the hell they’re supposed to.

Boltzmann did a great deal to introduce statistics to science but most scientists of his generation fought against the idea and insisted that Newton’s mechanical view of the universe was correct. The true era of statistics began on December 14th of 1900, when Max Plank first presented the Planck postulate. Since that time, statistics have conquered science. If you were to list things that were dominant motifs of the 20th century, you might include skyscrapers, airplanes, rockets, penicillin and statistics.

Nevertheless, statistics are not the truth about our universe. They are a useful tool, but as a world view, they take us no closer to the truth than Newton’s mechanical universe. As I write these words, I’m thinking of what Kuhn said, that science adopts a new paradigm, not because it is a better representation of reality, but because it is useful to the work that scientists are now doing.

For these reasons, it is important to remember the many problems that science faces when relying on statistical methods:

Statistical problems also afflict the “gold standard” for medical research, the randomized, controlled clinical trials that test drugs for their ability to cure or their power to harm. Such trials assign patients at random to receive either the substance being tested or a placebo, typically a sugar pill; random selection supposedly guarantees that patients’ personal characteristics won’t bias the choice of who gets the actual treatment. But in practice, selection biases may still occur, Vance Berger and Sherri Weinstein noted in 2004 in ControlledClinical Trials. “Some of the benefits ascribed to randomization, for example that it eliminates all selection bias, can better be described as fantasy than reality,” they wrote.

Randomization also should ensure that unknown differences among individuals are mixed in roughly the same proportions in the groups being tested. But statistics do not guarantee an equal distribution any more than they prohibit 10 heads in a row when flipping a penny. With thousands of clinical trials in progress, some will not be well randomized. And DNA differs at more than a million spots in the human genetic catalog, so even in a single trial differences may not be evenly mixed. In a sufficiently large trial, unrandomized factors may balance out, if some have positive effects and some are negative. (See Box 3) Still, trial results are reported as averages that may obscure individual differences, masking beneficial or harm ful effects and possibly leading to approval of drugs that are deadly for some and denial of effective treatment to others.

“Determining the best treatment for a particular patient is fundamentally different from determining which treatment is best on average,” physicians David Kent and Rodney Hayward wrote in American Scientist in 2007. “Reporting a single number gives the misleading impression that the treatment-effect is a property of the drug rather than of the interaction between the drug and the complex risk-benefit profile of a particular group of patients.”

Another concern is the common strategy of combining results from many trials into a single “meta-analysis,” a study of studies. In a single trial with relatively few participants, statistical tests may not detect small but real and possibly important effects. In principle, combining smaller studies to create a larger sample would allow the tests to detect such small effects. But statistical techniques for doing so are valid only if certain criteria are met. For one thing, all the studies conducted on the drug must be included — published and unpublished. And all the studies should have been performed in a similar way, using the same protocols, definitions, types of patients and doses. When combining studies with differences, it is necessary first to show that those differences would not affect the analysis, Goodman notes, but that seldom happens. “That’s not a formal part of most meta-analyses,” he says.

Meta-analyses have produced many controversial conclusions. Common claims that antidepressants work no better than placebos, for example, are based on meta-analyses that do not conform to the criteria that would confer validity. Similar problems afflicted a 2007 meta-analysis, published in the New England Journal of Medicine, that attributed increased heart attack risk to the diabetes drug Avandia. Raw data from the combined trials showed that only 55 people in 10,000 had heart attacks when using Avandia, compared with 59 people per 10,000 in comparison groups. But after a series of statistical manipulations, Avandia appeared to confer an increased risk.

A theory that reduces all measurement by any kind of science to 4 basic types. I find this an interesting way to organize the different kinds of mental activities involved in the different branches of science.

Nominal scale

At the nominal scale, i.e., for a nominal category, one uses labels; for example, rocks can be generally categorized as igneous, sedimentary and metamorphic. For this scale some valid operations are equivalence and set membership. Nominal measures offer names or labels for certain characteristics. …

Ordinal scale

In this scale type, the numbers assigned to objects or events represent the rank order (1st, 2nd, 3rd etc.) of the entities assessed. An example of ordinal measurement is the results of a horse race, which say only which horses arrived first, second, third, etc. but include no information about times. Another is the Mohs scale of mineral hardness, which characterizes the hardness of various minerals through the ability of a harder material to scratch a softer one, saying nothing about the actual hardness of any of them…

Interval scale

Quantitative attributes are all measurable on interval scales, as any difference between the levels of an attribute can be multiplied by any real number to exceed or equal another difference. A highly familiar example of interval scale measurement is temperature with the Celsius scale. In this particular scale, the unit of measurement is 1/100 of the difference between the melting temperature and the boiling temperature of water at atmospheric pressure. The “zero point” on an interval scale is arbitrary; and negative values can be used. The formal mathematical term is an affine space (in this case an affine line). Variables measured at the interval level are called “interval variables” or sometimes “scaled variables” as they have units of measurement…

Ratio measurement

Most measurement in the physical sciences and engineering is done on ratio scales. Mass, length, time, plane angle, energy and electric charge are examples of physical measures that are ratio scales. The scale type takes its name from the fact that measurement is the estimation of the ratio between a magnitude of a continuous quantity and a unit magnitude of the same kind (Michell, 1997, 1999). Informally, the distinguishing feature of a ratio scale is the possession of a non-arbitrary zero value. For example, the Kelvin temperature scale has a non-arbitrary zero point of absolute zero, which is denoted 0K and is equal to -273.15 degrees Celsius. This zero point is non arbitrary as the particles that compose matter at this temperature have zero kinetic energy….

I stumbled upon this essay, apparently regarded by some as a classic. I have not read it all, though it looks like it answers a lot of my questions about the bizarre handling of floating point numbers that I’ve noticed in some situations. What Every Computer Scientist Should Know About Floating-Point Arithmetic.

I like this post a lot. People who are asked to fake randomness overdue the “randomness” – they fail to include long sequences where the same result comes over and over again. People naturally understand improbable nature of the coin tosses in Rosencrantz and Guildenstern Are Dead (where one of the characters flips a coin throughout the play, and always gets heads up) but when asked to fake randomness, people go too far in the other direction.

The way to distinguish real random sequences from human-generated ones is to look for a place on the list where there are at least six heads or tails entries in a row. Almost everyone who tries to fake the tosses fails to include a run of such length, yet it is almost a statistical certainty that it will occur in a sufficiently large number of tosses. Using 200 flips, roughly 98% of the entries should have such a sequence of at least six consecutive heads or tails.

This is actually nontrivial to compute (see bottom), but a quick-and-dirty calculation that ignores any conditional probability is as follows: at any given point, the probability of getting six of the same side in a row is (1/2)^6 = 1/64. Thus at any given point the probability of not having such a sequence occur in the next six tosses is (1 – 1/64), and thus the likelihood of this not happening over the entire run is (1 – 1/64)^195 = approximately 5%. (We use 195 not 200, since it is impossible to get six in a row in the last 5 flips). So the rough probability of the sequence happening is 95%. The actual result is even higher.

Hence almost every time we can expect to get a run of six or more, but the near-certainty of this from a probability standpoint does not mesh with our psychological picture of what random coin tosses are supposed to “look like”. In fact, what most people tend to write down is a sort of pseudo-alternation of heads and tails, which is anything but random. If you look at randomness from a compression or signal analysis standpoint, it is equivalent to white noise, meaning that no patterns can be usefully extracted (and no compression can usefully be done). The more a sequence resembles H,T,H,T,H,T…. throughout, the more it becomes nonrandom because it contains the pattern of “H,T alternating”.

Interesting post about getting young people to learn math:

Age-appropriate development and understanding of mathematical concepts does not advance at a rate fast enough to please test-obsessed lawmakers. But adults using test scores to reward or punish other adults are doing a disservice to the children they claim to be helping.

It does not matter the exact age that you learned to walk. What matters is that you learned to walk at a developmentally appropriate time. To do my job as a physicist I need to know matrix inversion. It didn’t hurt my career that I learned that technique in college rather than in eighth grade. What mattered was that I understood enough about math when I got to college that I could take calculus. –Joseph Ganem, American Physical Society

Some articles I want to remember for later:

How to handle the CSS differences between IE6, IE7 and IE8.

Mockflow is a new online mockup/wireframing tool. Worth a look.

Massively collaborative mathematics.

A list of math blogs and wikis. Useful stuff for when I want to delve further into statistics and data minining.

I like the idea of studying continuously through the course, so that at the end of a course you do not need to study for the final, because you already know it all:

Furthermore I was lucky enough to talk to some of my students about the experience a few months later. The general consensus was that the material really stuck. Furthermore nobody studied for the final. No joke. As one girl said, “I tried studying because I thought I should, but I gave up after a half-hour because I already knew it all.” That is how I think it should be – if you study properly through the course, then you won’t need to study for the final. Because you’ve already learned it. And you’ll have a leg up on the next course because you still remember the material that everyone else has forgotten.

So were there any downsides? Unfortunately there were some big ones. I had set goals around learning. I failed to set any around happiness. Having to pay attention during class was hard on the class. Also it motivated them to work hard. Since everyone worked hard and they thought that I was going to grade them on a curve, there was a lot frustration that they wouldn’t properly be recognized for their work. (In fact I gave half of them A’s in the end.) This frustration showed up the teacher evaluations at the end of the course. :-(

I also know how hard it is to pay attention and not day dream. That is what sunk me throughout my school career – I could not concentrate, I would always daydream instead.

Today I had a friend building a loft in her apartment. One of the walls was at a diagonal to the rest of the room. She needed to calculate the distance down that wall where she would cut into the dry wall. The other angles were all 90 degrees, so we were working with a right angle. She looked around online and found a great triangle calculator, which helped her figure out the correct lengths.

She had a laser tool that told her the angle between the front wall and the diagonal wall was 100 degrees. Her friend Sam suggested that we treat most of the room as a rectangle (which could be ignored), and so then she only needed to analyze the right triangle that existed outside of that rectangle. Look at the triangle at the above page to get a sense of how we labeled our angles. The 100 degree angle became a 10 degree angle once we decided to only look at that portion of the room that was outside of the rectangle that made up the rest of the room. So the angles were :

B – 10

C – 90

A – 80

She measured the wall along “a” and found that it was 78 inches. That was the only length she felt she could measure with precision, given some of the unevenness of the walls. We’d forgotten our basic geometry, so we had to look up the needed equation online. Basically, the length of 78 inches was divided by the the sine of the 80 degree angle:

78 / sine80 = 79.2 inches

I’m linking to the online triangle calculator so I can remember it the next time I’m doing home renovations in an oddly shaped room.

Anyone alive today grew up in the era of Heisenberg’s uncertainty principle, Max Plank and quantum mechanics. As such, it seems natural to us to think of probability when we think of physics. Einstein may have said “I can not believe that God is dicing with the Universe” but dicing is the view dominates physics today. We are taught these ideas in school. Because of this, we have a hard time remembering what a surprise probability was to scientists working at the end of the 1800s. By that point, Western science had spent 200 years working with Newton’s view of a mechanical universe, and physics was seen as the most pure of the sciences, and therefore the most mechanical, the most free of randomness. Who would have guessed, in 1880, that physics was about to transform from the science most free of probability to the one that depended on it in its most fundamental theories?

Consider this bit of history from “The Human Use Of Human Beings: Cybernetics And Society” written by by Norbert Wiener in 1950.

The beginning of the 20th century marked more than the end of a 100 year period and the start of another. There was a real change in point of view even before we made the political transition from the century on the whole dominated by peace, to the half century of war through which we have just been living. This was perhaps first apparent in science, although it is quite possible that whatever has affected science lead independently to the marked break which we find between the arts and literature of the 19th and those of the 20th centuries.

Newtonian physics, which had ruled from the end of the 17th century to the end of the 19th with scarcely an opposing voice, described a universe in which everything happened precisely according to law, a compact, tightly organized universe in which the whole future depends strictly upon the whole past. Such a picture can never be either fully justified or fully rejected experimentally and belongs in large measure to a conception of the world which is supplementary to experiment but in some ways more universal than anything that can be experimentally verified. We can never test by our imperfect experiments whether one set of physical laws or another can be verified down to the last decimal. The Newtonian view, however, was compelled to state and formulate physics as if it were, in fact, subject to such laws. This is now no longer the dominated attitude of physics, and the men who contributed most to its downfall were Bolzmann in Germany and Gibbs in the United States.

These two physicists undertook a radical application of an exciting, new idea. Perhaps the use of statistics in physics which, in large measure, they introduced was not completely new, for Maxwell and others had considered worlds of a very large number of particles which necessarily had to be treated statistically. But what Bolzmann and Gibbs did was to introduce statistics into physics in a much more thoroughgoing way, so that the statistical approach was valid not merely for systems of enormous complexity, but even for systems as simple as the single particle in a field of force.

Statistics is the science of distribution, and the distribution contemplated by these modern scientists was not concerned with large numbers of similar particles, but with the various positions and velocities from which a physical system might start. In other words, under the Newtonian system the same physical laws apply to a variety of systems starting from a variety of positions and with a variety of momenta. The new statisticians put this point of view in a fresh light. They retained indeed the principle according to which certain systems may be distinguished from other by their total energy, but they rejected the supposition according to which systems with the same total energy maybe clearly distinguished indefinitely and described forever by fixed causal laws.

There was, actually, an important statistical reservation implicit in Newton’s work, though the 18th century, which lived by Newton, ignored it. No physical measurements are ever precise; and what we have to say about a machine or other dynamic system really concerns not what we must expect when the initial positions and momenta are given with perfect accuracy (which never occurs), but what we are to expect when they are given with attainable accuracy. This merely means that we know, not the complete initial conditions, but something about their distribution. The functional part of physics, in other words, cannot escape considering uncertainty and the contingency of events. It was the merit of Gibbs to show for the first time a clean-cut scientific method for taking this contingency into consideration.

I’m reading Ian Stewart’s book, Letters To A Young Mathematician. The book is written as a series of a letters to a fictional “Meg”, who I think is a fictional niece. As Stewart says:

The letters, addressed to “Meg”, follow her career in roughly chronological order, from high school through to a tenured position in a university. They discuss a variety of topics, ranging from basic career decisions to the working philosophy of professional mathematicians and the nature of their subject.

Some ideas were new to me, such as the fact that math can prove that something is unprovable. For instance, there is no reliable method to trisect an angle, and there never will be. Math is unique among the sciences in that is able to know which things are impossible.

This bit about gender in math was especially interesting to me:

Mathematicians are proud to trace their academic lineage through their thesis advisers. Brian was my mathematical father, and Phillip Hall my grandfather.

…Talent must be passed to succeeding generations. I’ve been a thesis adviser to 30 students so far, 20 men and 10 women. Since 1985, the proportions are 50% men and 50% women. I know women are just as good at math as men because I’ve watched both at close quarters. I am particularly proud of my mathematical daughters, most of whom hail from Portugal, where mathematics has long been viewed as a suitable activity for women. All of my Portuguese daughters have remained in mathematics. In fact, most of my graduate students have remained in mathematics, and every single one of them earned a PhD. However, one is now an accountant, several work in computing, and one owns an electronics company, or at least he did last time I heard from him.

The rest of the world is now following Portugals’s lead. In July 2005 the American Mathematical Society released the results of its 2004 Annual Survey Of The Mathematical Sciences. Since the early 1990s, women have been receiving around 45% of all first degrees in math. Women received almost one-third of all US doctorates in the mathematical sciences in the academic year 2003-2004, and one-quarter of those awarded in the top 48 math departments. In all, 333 women received math PhDs that year, the largest number ever recorded.

The percent of computer scientists who are female and who are earning advanced degrees peaked in 1989 and has since retreated. As math is foundational to computer science, this is revealing. Computer science is nearly alone among the professions in seeing women’s participation retreat during the last 20 years. Women have made huge strides in catching up to men in business, law and medicine. Almost 50% of all new doctors are female.

I’d been assuming that women’s retreat from computer science had to be mirrored by weakness in math, but if women are making huge strides in math, then the anomaly of their retreat from computer science is even more puzzling than before.

Lately I’ve become interested in chaos theory. My friend Lark Davis pointed me in the direction of Leonard Smith’s book, Chaos, A Very Short Introduction. It is a short, somewhat technical book, and it dives into the details fast.

I took a step back from it and decided to re-read James Gleick’s book, Chaos, Making A New Science. It is aimed at a popular audience. I read it back in the 90s, though apparently I never understood it. All this time I’ve been thinking there are random elements in any chaotic system, and of course, the whole point of chaos theory is that you can get chaotic outcomes from deterministic equations.

This bit (starting on page 63) says it well:

An ecologist imagining real fish in a real pond had to find a function that matched the crude realities of life – for example, the reality of hunger, or competition. When the fish proliferate, they start to run out of food. A small fish population will grow rapidly. An overly large population will dwindle (they will starve).

…In the Malthusian scenario of unrestrained growth, the linear growth function rises forever upward. For a more realistic scenario, an ecologist needs an equation with some extra term that restrains growth when the population becomes large. The most natural function to choose would rise steeply when the population is small, reduce growth to near zero at intermediate values,and crash downward when the population is very large. By repeating the process, an ecologist can watch a population settle into its long-term behavior – presumably reaching some steady state. A successful foray into mathematics for an ecologist would let them say something like this: Here’s an equation; here’s a variable representing the reproductive rate; here’s a variable representing the natural death rate; here’s a variable representing the additional death rate from starvation or predation; and look – the population will rise at this speed until it reaches that level of equilibrium.

How do you find such a function? Many different equations might work, and possibly the simplest modification of the linear, Malthusian version is this:

x[next] = rx(1-x)

x[next] is the population next year.

Again, the parameter r represents a rate of growth that can be set higher or lower. The new term, 1-x, keeps the growth within bounds, since as x rises, 1-x falls. Anyone with a calculator could pick some starting value, pick some growth rate, and carry out the arithmetic to derive next year’s population.

For convenience, in this highly abstract model, “population” is expressed as a fraction between zero and one, zero representing extinction, one representing the greatest possible population of the pond.

So begin: Choose an arbitrary value for r, say, 2.7, and a starting population of .02. One minus .02 is .98. Multiply by 0.02 and you get .0196. Multiply that by 2.7 and you get .0529. The very small starting population has more than doubled. Repeat the process, using the new population as the seed, and you get .1353. The population rises to .3159, then .5835, then .6562 – the rate of increase is slowing. Then, as starvation overtakes reproduction, .6092. Then .6428, then .6199, then .6362, then .6249. The numbers seem to be bouncing back and forth, but closing in on a fixed number: .6328, .6273, .6312, .6285, .6304, .6291, .6300, .6294, .6299, .6295, .6297, .6296, .6296, .6296, .6296, .6296, .6296. Success!

[skipping to page 69]

Robert May was a biologist. His interests at first tended toward the abstract problems of stability and complexity, mathematical explanations of what enables competitors to coexist. But he soon began to focus on the simplest ecological questions of how single populations behave over time.

…Once, in fact, on a corridor blackboard he wrote the equation out as a problem for the graduate students. It was starting to annoy him. “What the Christ happens when lambda gets bigger than the point of accumulation?” What happened, that is, when a population’s rate of growth, its tendency toward boom and bust, passed a critical point. By trying different values of this nonlinear parameter, May found that he could dramatically change the system’s character. Raising the parameter meant raising the degree of nonlinearity, and that changed not just the quantity of the outcome, but also it quality. It affected not just the final population at equilibrium, but also whether the population would reach equilibrium at all.

When the parameter was low, May’s simple model settled at a steady rate. When the parameter was higher, the steady state would break apart, and the population would oscillate between two alternating values. When the parameter was very high, the system – the very same system – seemed to behave unpredictably. Why? What exactly happened at the boundaries between the different kinds of behavior? May couldn’t figure it out. (Nor could the graduate students.)

May carried out a program of intense numerical exploration into the behavior of this simplest of equations… It seemed incredible that its possibilities for creating order and disorder had not long since been exhausted. But they had not. He investigated hundreds of different values of the parameter, setting the feedback loop in motion and watching to see where – and whether – the string of numbers would settle down to a fixed point. He focused more and more closely on the critical boundary between steadiness and oscillation. It was as if he had his own fish pond, where he could wield fine mastery over the “boom-and-bustiness” of the fish. Still using the logistic equation x[next] = rx(1-x), May increased the parameter as slowly as he could. If the parameter was 2.7, then the population would be .6292. As the parameter rose, the final population rose slightly too, making a line that rose slightly as it moved left to right on the graph.

Suddenly, though, as the parameter passed 3, the line broke in two. May’s imaginary fish population refused to settle down to a single value, but oscillated between 2 points in alternating years. Starting at a low number, the population would rise and then fluctuate until it was steadily flipping back and forth. Turning up the knob a bit more – raising the parameter a bit more – would split the oscillation again, producing a string of numbers that settled down to four different values, each returning every fourth year. Now the population rose and fell on a regular four-year schedule. The cycle had doubled again – first from yearly to every two years, and now to four. Once again, the resulting cyclical behavior was stable; different starting values for the population would converge on the same four year cycle.

With parameters of 3.5, say, and a starting value of .4, then May would see a string of numbers like this:
.4000, .8400, .4704, .8719,
.3908, .8332, .4862, .8743,
.3846, .8284, .4976, .8750,
.3829, .8270, .4976, .8750,
.3829, .8270, .5008, .8750,
.3828, .8269, .5009, .8750,
.3828, .8269, .5009, .8750,
.3828, .8269, .5009, .8750.

As the parameter rose further, the number of points doubled again, then again, then again. It was dumbfounding – such a complex behavior, and yet so tantalizingly regular. “The snake in the mathematical grass,” as May put it. The doublings themselves were bifurcations, and each bifurcation meant that the pattern of repetition was breaking down a step further. A population that had been stable would alternate between different levels every other year. A population that had been alternating on a two year cycle would now vary on the third and fourth years, thus switching to period 4.

These bifurcations would come faster and faster – 4, 8, 16, 32… – and suddenly break off. Beyond a certain point, the “point of accumulation,” periodicity gives way to chaos, fluctuations that never settle down at all. Whole regions of the graph are completely blacked in. If you were following an animal population governed by this simplest of nonlinear equations, you would think the changes from year to year were absolutely random, as though blown about by environmental noise. Yet in the middle of this complexity, stable cycles suddenly return. Even though the parameter is still rising, meaning that the nonlinearity is driving the system harder and harder, a window will suddenly appear with a regular period: an odd period, like 3 or 7. The pattern of changing population repeats itself on a three year or seven year cycle. Then period doubling bifurcations begin again, at a faster rate, rapidly passing through cycles of 3, 6, 12… or 7, 14, 28…, and then breaking off once again to renewed chaos.

Lately I’ve gotten interested in chaos. I read James Gleick’s book about chaos back in the 90s, but apparently I did not understand it. Gleick’s book says clearly that chaos is not random, but all this time I’ve been assuming that chaos must have some randomness to it. Thus, I missed the big deal about chaos, which is that you can have a deterministic equation that generates a pattern that never repeats. Somehow I missed all that until now.

Anyway, I’ve a new found interest in the math of differential equations, and I’m finding a lot of stuff that is suddenly interesting to me:

The Navier-Stokes equations are differential equations which describe the motion of a fluid. These equations, unlike algebraic equations, do not seek to establish a relation among the variables of interest (e.g. velocity and pressure), rather they establish relations among the rates of change or fluxes of these quantities. In mathematical terms these rates correspond to their derivatives. Thus, the Navier-Stokes equations for the most simple case of an ideal fluid with zero viscosity states that acceleration (the rate of change of velocity) is proportional to the derivative of internal pressure.