(laws of human stupidity)

November 16, 2008 6:33 AM

Why systems fail - Review of the book: Systemantics; how systems work... and especially how they fail by John Gall. New York, Pocket Books, 1978. {via}

At last those concerned with social change have a basic textbook to explain why "things generally are indeed not working very well" despite our many efforts. As is remarked on the cover: "Have you ever wondered why the unsinkable Titanic sank... or the poor in India eat better bread than the rich in America... or hospital patients are blamed for not getting well... or why, in general, things that don't work badly don't work at all?" Similar questions are of deep concern to those working in international organizations.

"The religious person may blame it on original sin. The historian may cite the force of trends such as population growth and industrialization. The sociologist offers reasons rooted in the peculiarities of human associations. Reformers blame it all on 'the system', and propose new systems that would, they assert, guarantee a brave new world of justice, peace, and abundance. Everyone, it seems, has his own idea of what the problem is and how it can be corrected. But all agree on one point - that their own system would work very well if only it were universally adopted.

"The point of view espoused in this essay is more radical and at the same time more pessimistic. Stated as succinctly as possible: the fundamental problem does not lie in any particular system but rather in systems as such. Salvation, if it is attainable at all, even partially, is to be sought in a deeper understanding of the ways of systems, not simply in a criticism of the errors of a particular system."

cf. Peter Klausler's Principles of the American Cargo Cult & Axel Boldt's Political Opinions and Other Thoughts

posted by kliuless (21 comments total)

Oh well, Who is John Gall?
posted by horsemuth at 6:50 AM on November 16, 2008 [3 favorites]

What, no "fail" tag? You could even do "epic fail" here.
posted by twoleftfeet at 6:56 AM on November 16, 2008

Some rather startling assertions in that Axel Boldt piece. Like: It is simply a fact of biology that a well-off fifty-year-old male is attracted to, and attracts, younger women, and will often leave his fifty-year-old wife for a girlfriend unless he can unproblematically buy sexual services from those younger women.
posted by Slothrup at 6:58 AM on November 16, 2008

As a programmer, I have seen some spectacular failures and completely nonfunctional multimillion-dollar software systems. It is quite an experience. Gall's axioms resonate with my own perception of why these failures occurred. Scope creep, Brooks's Law, and KISS are all aspects of this same issue. Interestingly, I have come across Gall's Law (Axiom 15) and subsets of the axioms in other contexts, but this is the first complete list I have seen with a reference to the book. I will have to try to find a copy.

Does it bother anyone else that the word "axiom" seems unsuited to describe a list of interdependent observations?
posted by pheideaux at 6:59 AM on November 16, 2008

The systematics part is a little overwrought methinks. Le Chatelier's Principle does not mean what I think he thinks it means.

That being said, I think the cargo cult essay is spot on.

In related news, I recommend The Logic of Failure for more on this sort of thing. They describe a bunch of computer simulations that describe various systems and what happened when they had a bunch of people come in and play with these systems. The take home message is, when you're dealing with something you're maybe not 100% familiar with, and it's performance is a bit off, cranking the dial from 3.5 to 10 first thing is probably not a good approach.
posted by Kid Charlemagne at 7:13 AM on November 16, 2008

But all agree on one point - that their own system would work very well if only it were universally adopted. ... Stated as succinctly as possible: the fundamental problem does not lie in any particular system but rather in systems as such.

See also: "Democrats vs. Republicans."
posted by ZenMasterThis at 8:40 AM on November 16, 2008

I'm interested in failure modes, but any writer who makes such broad statements as "Nothing complicated works" and "Systems in general work poorly or not at all" is unlikely to have written anything useful.

My lymphatic system works just fine, thanks.
posted by Xezlec at 9:11 AM on November 16, 2008 [2 favorites]

Oh and seeing the title "laws of human stupidity", I'll add that my Windows PC works just great, despite claims of Linux-heads everywhere.

Disproof by counterexample. Can we go home now?
posted by Xezlec at 9:15 AM on November 16, 2008

See also Normal Accidents by Charles Perrow. Perrow does a great job of telling why complex systems fail, and how to design them so they are more resistant to failure.
posted by Dr. Twist at 9:38 AM on November 16, 2008 [1 favorite]

Seconding Logic of Failure as a good read on this. To Engineer is Human by Petroski may also be of interest.
posted by ejaned8 at 9:53 AM on November 16, 2008

This book posits a more empirical, and less petulant, approach to the problem of systems.
posted by eustatic at 10:14 AM on November 16, 2008

I highly recommend Seeing Like a State, a book that examines the inevitable gaps between centralized, systematic, models and the reality of the people and systems they are trying to model.

Local practices are largely "illegible" to outsiders (who don't share the same context), and attempts to unify these customs into a common system that is legible will necessarily ignore subtleties that may be vital to the original system.

The author makes a strong argument for "the indispensable role of practical knowledge, informal processes, and improvisation in the face of unpredictability."
posted by findango at 10:49 AM on November 16, 2008 [1 favorite]

I've read Systemantics, found it a little too flippant. Designing Freedom by Stafford Beer is my favorite.
posted by natteringnabob at 10:56 AM on November 16, 2008

My lymphatic system works just fine, thanks.

It does, but certainly not forever. One slight bifurcation and a biological system moves towards failure. That also might sort of be the point. Your lymphatic system is also entirely natural, having adapted from simpler systems over millions of years. We're not all walking around with elephantiasis because, not to get all dao, nature harmonizes the chaos, at least to the extent we our systems can survive until we procreate, then die, or if we've failed, become extinct. Whether or not this is a harmony at all is another debate, but human-made systems built on non-dynamic, maladaptive principles, especially those that are gigantic and complex, are vulnerable to the slightest malfunction, and rarely, if ever, work in the harmonizing, static way we expect them to. There's just too many other conditions, it's why we can't for certainty predict the precise temperature anywhere on earth a week from now. It goes all the way back to probabilistic issues of outcome and inductive problems, and we'll be toiling with this incessantly for years to come.
posted by ageispolis at 10:57 AM on November 16, 2008

This book was published 30 years ago and nobody seems to have noticed. Does this make it a self-fulfilling prophesy?
posted by donfactor at 11:38 AM on November 16, 2008

The delicate nature of our lymphatic system has been recognized for ages. Orthodox Jews recite a set of blessings every morning, and the third blessing, the asher yatzar, is also recited every time one performs one's bodily functions.

Blessed are You, LORD our God, Master of the universe, who formed mankind in wisdom, and created in him all manner of openings and cavities [alternatively arteries, organs and glands]. It is manifest and known before the throne of your glory that if any one of them ruptured or were blocked [for even a short period of time], it would be impossible to survive and stand before You. Blessed are You, LORD, who heals all flesh and is wonderful in His acts


I'm not particularly religious, but I love that my tradition includes thanks for relief from ills such as constipation and stones. :-)

Back to the subject ... All religion aside, it is well to remain humble when confronted with complexity. Fools rush in, and all that.
posted by Araucaria at 2:35 PM on November 16, 2008

One of the things I've tried to get across to my coworkers and management in the IT field is the need for both simplicity and standardization, even in the face of "but this complex system is 'better'". The complex system might be faster, have more features, and even save a little money in the short term...but in the long term the complexity of such systems causes slowdowns, hidden bugs and costs more money in the long run.

Think of running a simple sales system in DOS (whichever one...Caldera DR-DOS or a variant if you believe that MS-DOS was crap) versus running in Windows Vista. Vista has more features ("you get to use a mouse!") but adds nothing to accomplishing the task that has been allocated to that machine.
posted by Kickstart70 at 3:11 PM on November 16, 2008

So, so, so true Kickstart70. It's amazing how easily people will write checks for huge software packages costing into the 7 figures that claim to "solve all your problems" in systems management but end up being these massive systems that go underutilized, misunderstood, and full of problems. Meanwhile, any good operations person can script/automate 80% of what the package would have done, and if they do it well, they can even automate the automation to approach that 100%- they just end up piping the various small, fast, simple tools they've written. It's cheaper, more flexible, and more easily understood because any one part is back-of-the-envelope simple. Huge-ass bloatware is not loosely coupled...
posted by hincandenza at 6:44 PM on November 16, 2008

Even though it's been flagged as "un-Wikipedia-like", I like Wikipedia's article on the book
posted by dylanjames at 4:59 PM on November 17, 2008

Whether or not this is a harmony at all is another debate, but human-made systems built on non-dynamic, maladaptive principles, especially those that are gigantic and complex, are vulnerable to the slightest malfunction, and rarely, if ever, work in the harmonizing, static way we expect them to.

That is a much more believable statement than "nothing complicated works," but I still don't know if that matches my experience in all areas. I'm harping on this because, as an engineer, I wouldn't have a job if nothing complicated worked. I help build gigantic, complicated systems that work (scientific signal-processing software), using sometimes-gigantic, complicated tools that work (i.e. gcc).

It's cheaper, more flexible, and more easily understood because any one part is back-of-the-envelope simple. Huge-ass bloatware is not loosely coupled...

Interesting. I guess I've always felt the opposite way. Loosely coupled software seems buggy to me. I've never seen a 100% working Linux system in my life. Things on my Linux box break every week or so, and I'm given no clue what broke or where, or in some cases even that anything is broken. More importantly, I usually can't figure out how to fix it and just have to reinstall the whole OS if I need it. These systems are also harder to understand, at least for me, because so much more of their internal complexity is exposed. I will agree with "flexible" and "cheaper" for sure, but I thought this was a tradeoff, and reliability and understandability are what you lose.

In Windows, I can do almost anything I want to do in 2 seconds. And it will never stop working. In Linux, I can do absolutely anything in 2 seconds, but only after 2 months of research, and it might suddenly break at any time, with the slightest disturbance.
posted by Xezlec at 6:52 PM on November 17, 2008

I wouldn't have a job if nothing complicated worked. I help build gigantic, complicated systems that work (scientific signal-processing software)

Scientific software is one of the rare types of complex systems where the behavior can be fully specified in advance, and if it works according to spec, then it works. I'd amend the rule to "Nothing complicated involving people works."
posted by fuzz at 1:11 AM on November 18, 2008