Entropy and Software Projects

May 10, 2008 10:34 AM

The 2nd law of thermodynamics (entropy theory) has been applied to information theory, economics, natural selection and human behaviour. How do I relate it to software projects?

It seems from a variety of sources that entropy theory has been abstracted and used to model many areas of life other than the physical sciences. It is quite fascinating. I'd like to know how software development projects can be modelled using this theory, preferably in a mathematical sense, as opposed to metaphors.

Any ideas or links from the hive mind?

posted by gttommy to science & nature (14 comments total)

I'm not sure you want to try to shoehorn that theory into whatever field you're looking at at the moment. Yes, it's been applied to natural selection, but the applicants do so incorrectly. Creationists claim that for complex creatures to evolve from simpler ones would be increasing the order in a system. This is clearly not true.
posted by chrisamiller at 10:44 AM on May 10

Seconding chrisamiller. Don't bean-plate it (as the kids around here are wont to say.)

That said, the 2nd law is so poorly understood by most of us that you could easily bull shit your way through some meeting or presentation while using it. Glassy eyed creationist have made a cottage industry out of it...

"... and that, ladies and gentlemen, is why the software development department needs in-house massages and sushi lunches eaten off of a naked Asian woman. The Second Law! Learn it. Know it. Live it. Thank you. I'm out!"
posted by wfrgms at 10:55 AM on May 10

The previous posters are correct - you'll be attempting to cram one thing into something else where it doesn't belong. You might remember Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity - a wee prank played upon the post-modern crowd by a scientist fed up with all of the blather generated by people attempting to make art/lit/critical theory more, I don't know, legitimate by rubbing it all over with the various neat-sounding words of science.

However, if you're truly bent on this path, you can pick up one of those nice, inexpensive Dover books on thermodynamics and actually derive the second law of thermodynamics yourself, from basic statistics. One of its more amusing charms is that, while the law has consequences people find unpalatable (almost as much so as conservation of mass-energy or an upper boundary for the speed at which something made of normal matter can travel), it's not empirical - it wasn't observed, it wasn't tweaked ... it arises out of math and becomes a bit difficult to argue with. That doesn't stop people from trying, but it's always fun to watch people attempting to prove math wrong.

After deriving it (an exercise everyone with any interest should perform, the same as they should for coming up with "the quadratic equation"), you could then look for any connections to your idea. I suspect you won't come up with anything not already found in The Mythical Man-Month - adding more particles (developers) just increases the entropy (lateness) of the system (project). Unless you're dealing in formal proofs for software (and remember how non-trivial they are), I don't see quite how you'll be attaching the math part.

Certainly quantifying human behavior has been a frustrating task - aside from timing some relatively basic functions people can do (how fast can you put that round peg in that hole?), we don't have a lot of sufficiently hard numbers to work off of. Or you could end up with something so bloody abstract (Omega number, I'm looking at you!) that it wouldn't exactly be applicable to anything but writing grants for a compsci/math group.

Any model you attempt to create will be inextricably tied in with the specifics of the software, then you'd have to come up with some way to quantize things like "Exactly how flaky are our clients when it comes to changing their minds endlessly?" "Measure developer ego and correlate it to output, but also to inertia as it relates to changing project needs and friction with other developers."

In short, you'll end up something like a bunch of knobby particles, whizzing about at high velocity, banging against things but not really going anywhere - in other words, ideal gas as a bunch of hot air.
posted by adipocere at 11:12 AM on May 10 [2 favorites]

Dude, it is "the 2nd law of thermodynamics", not "the 2nd law of software development projects" (or "the 2nd law of evolution"). I would bet there are much better and more applicable theories out there.
posted by charlesv at 11:53 AM on May 10

Information entropy and thermodynamic entropy are the same thing. One isn't a metaphor for the other. (Actually, they're not exactly the same thing, particularly in the way they are used. There is an wikipedia article about their relationship).

Obviously information entropy is pretty important in software that deals directly with probabilities, like in machine learning and artificial intelligence.

But, trying to come up with a predictive theory of 'software entropy' based on the real theory of entropy would actually make you look, to be blunt, stupid. For one thing, software code is written logically, with everything being true or false. Everything does exactly what you ask it it, and so there is 'zero entropy' in the system.

Remember, the entropy is -∑(p_i*ln(p_i)). Since each variable in a software program is 1 or zero, in the case of a 1, then ln(1) = 0. So the total entropy in the entire system is zero.

One thing you could say is that people might forget exactly what a function does if a project gets to large, and so the probability of a function doing a particular thing might become less then 1 from a programmers perspective, but that would be hard to quantify.
posted by delmoi at 12:03 PM on May 10

"Over time, software will degrade in worth, performance, accuracy, etc. approaching and ultimately reaching zero.

Some software modifications, bug fixes, upgrades, service packs, etc. give the illusion of "improvement" but only delay the software's decline momentarily.

Stephen, former Database Administrator and Software Engineer.
posted by swarkentien at 12:07 PM on May 10

I agree with delmoi that it might be difficult to find entropy in well written code, but in the development process there's plenty. I think adipocere has it. The only thing I'd add to his examples is feature creep. I don't know anything about the theoretical applications of entropy though, although evidence of my familiary with its practical application can be found throughout my apartment.
posted by Deathalicious at 12:17 PM on May 10

Far from needing to be 'shoehorned in', I think thermodynamics, the crowning second law in particular, is surprisingly immanent in many, many things, and that your project is quite feasible-- even a necessary (tell me Vista isn't bumping up against thermodynamic-looking limits, sort of like a single celled organism an inch in diameter) ultimate development, perhaps, although an extremely ambitious one.

Consider, for example, the incredibly deep insights Stephen Hawking, Brandon Carter, and James Bardeen were able to wring out of thermodynamics as applied to black holes.

In order to apply the mathematical models of thermodynamics to software, you will need to find, or be able to invent software variables which can fit into the thermodynamic equations the way black hole physicists did:

The zeroth law is analogous to the zeroth law of thermodynamics which states that the temperature is constant throughout a body in thermal equilibrium. It suggests that the surface gravity is analogous to temperature. T constant for thermal equilbrium for a normal system is analogous to κ constant over the horizon of a stationary black hole.

But to understand software, I suspect you will also need to study the behavior of self-organizing systems, which are far from the thermodynamic equilibrium which is the main focus of classical thermodynamics.

My thermodynamic intuition, such as it is, got its biggest boost from P. W. Atkins' The Second Law.
posted by jamjam at 12:39 PM on May 10

What chrisamiller said -- just don't go there.

(Though there is one legit sort of application for entropy in natural selection, but not what you'd think -- one of the equations turns out to be useful in getting some proofs from the replicator dynamics in evolutionary game theory... see Weibull textbook for details.)
posted by paultopia at 4:53 PM on May 10

Entropy, along with redundancy is a parameter used in information theory (from my vague recollection of a cognitive science lecture I attended years ago) .

Information theory could be used model software projects (along with most other things if I understand it correctly). The challenge is working out what the parameters are. I suspect that code, documentation and process will be separate but interacting parts of the model.
posted by singingfish at 9:27 PM on May 10

Yes, it does sound far-fetched but I'd like to see what the relationship are between software projects and entropy, following previous examples of entropy theory being used creatively. This would involve inventing software project variables to find mathematical parallels, as mentioned by jamjam and singingfish, maybe something like risk, complexity, etc to determine project value or cost (entropy?) over time. If it works, it works, if not then it won't.

Thanks for the input so far, anymore answers are welcome!
posted by gttommy at 10:18 PM on May 10

There's a book called "The Mythical Man Month" that might help you with a conceptual approach.

In general I would think the approach you'd be looking for would be in parallel with thinking in software project management, which is pretty numbers-based, even though it deals with individual humans as resources.
posted by kalessin at 3:49 AM on May 11

Information theory could be used model software projects (along with most other things if I understand it correctly).

You don't. As I outlined above, the true entropy of a deterministic software program is always zero.
posted by delmoi at 3:52 PM on May 11

Just to clarify, software project not software program.
posted by gttommy at 4:06 PM on May 13