How do I, a non-scientist, recognize quackery in science and tech?
April 7, 2015 5:08 PM   Subscribe

Lately I've been running across a lot of highly theoretical science and technology information that I don't understand. I would like to be able to recognize quackery and fringe science, as well as when technologists (especially programmers) are reinventing the wheel and claiming to revolutionize things, so I don't get steered in the wrong direction. I'm looking for heuristics.

Some things are to me instantly recognizable as woo. The time cube guy, Singularitarians, anything that tries to connect anything "quantum" with consciousness -- these are the types of ideas that range from batshit insane to obviously suspect. I'm talking about ideas that are less immediately identifiable as wack. I visit Rational Wiki a lot, but they're more validating than enlightening.

For example, here's a guy who claims to have invented recursive lossless compression. To my understanding, this would be revolutionary if not impossible. His website looks a bit amateurish and he doesn't appear to have won any major prizes. He's split his data in a way that reminds me of Maxwell's Demon, a concept that is generally not thought to be workable in the real world. He goes on to give equations. Unfortunately my mathematical education ended in 11th grade so I have no way of knowing whether the wool is being pulled over my eyes. But something about it seems intuitively fishy. I would like to be able to articulate why other than "I dunno, it just seems off."

The above is just an example of what I run across a lot as I research science and tech on my own -- an egregious claim by someone with few credentials, or specious ones, who uses lots of big words and mathematical symbols to prove his point. Alas, I have no way of knowing for sure just how egregious it is. Perhaps I'm just too uneducated to understand it.

Related is the issue of how to tell whether someone in a field not my own is good at what they do. Specifically, I'd like to know who the good programmers are among my friends and acquaintances. I'm interested in learning programming and computer science, and people I know are always engaging me in talking about their pet projects. To a complete noob, all their projects sound really interesting and cool. But I really wouldn't know, because I don't know the heuristic for judging. And those I have encountered online, such as the Coding Horror blog's critical writings, presuppose a knowledge of programming that I don't have yet.

Metafilter, help me devise a way of evaluating information before I get caught up in someone else's evangelistic fervor. I don't want to end up tilting at windmills.
posted by Beethoven's Sith to Technology (16 answers total) 23 users marked this as a favorite
The most important heuristic is a very old one: "If it seems too good to be true, it probably is."
posted by Chocolate Pickle at 6:00 PM on April 7, 2015 [1 favorite]

Carl Sagan addressed this issue in his book The Demon Haunted World. There's a section called The Fine Art of Baloney Detection, and you can search that title for lots of references. Here is one page with a decent summary, in case you're not inclined to read the (quite worthwhile) book.
posted by Flexagon at 6:08 PM on April 7, 2015 [1 favorite]

The problem is that it often takes some effort to identify quackery. As a scientist, I can immediately tell you that when you see the words "free energy", the concept is nonsense. If you then say, "but the whole explanation makes so much sense, where is he lying?" I will get very annoyed, because finding the exact point of error is a much larger effort than just going with my general sense that this guy is wack, and doesn't add anything of value to the conclusion (i.e. I already knew it was nonsense).

Basically, I encourage you to hone your gut feeling of what's too good to be true, but not to get caught up in the technical details. Don't feel that you need to disprove it logically to know that it's wrong.
posted by aimedwander at 6:14 PM on April 7, 2015 [3 favorites]

Best answer: You might enjoy this list by Scott Aaronson. It is specific to theoretical computer science but some of his 10 criteria are more generally useful. (If someone in math, physics, or computer science doesn't use LaTeX, you really do need to be skeptical. Biology and other sciences, though, have different publication standards and I think frequently use Word.)

Also realize that even if you were an expert in whatever field, it could still take significant time and effort to read, make sense of, and refute these crackpot claims. In a sense, there's no reason why you should be able to do this.

I usually feel that if someone does really have a revolutionary breakthrough, I'll hear about it from a credible source eventually.

As for telling who's a good programmer: this is like asking how to tell if someone is a good writer. Read what they've written. You can teach yourself to code, and you can probably learn a lot even from "bad" programmers among your acquaintances. I wouldn't worry about it until then. Unlike scientific crackpots, you'll know right away whether or not you're looking at bad code.
posted by vogon_poet at 6:21 PM on April 7, 2015 [2 favorites]

This is one [of many] reasons that peer-reviewed journals exist. It's safe to say that if something is revolutionary but it hasn't been published in a major peer-reviewed journal in its field, then it has been deemed by experts in the field to be bunk (or at least not ready for primetime).
posted by telegraph at 6:25 PM on April 7, 2015 [5 favorites]

Best answer: When you don't know enough about the science in question, there are some social cues that can help you try to bet intelligently about whether the person is full of hot air or is someone speaking over your heard.

Some of the negative indicators in the example you linked to (indicators that suggest he is probably full of hot air):

It looks like a company site, but it was apparently last updated in 2012 and 2012 is when they claim to have achieved a working prototype. Prototypes are not final products. They are initial products, that typically get further modified after they are tried because just like "no plan of battle survives contact with the enemy," most theories don't remain completely intact either once tested against reality.

There are also statements on one of the pages that look contradictory to me in terms of claiming this is a done deal and then a paragraph later it talks about "theoretically."

In science, the words hypothesis and theory have strict meanings. A theory is a stronger idea than a hypothesis, with more evidence that it is probably correct -- which is why we have the phrase "Theory of Relativity". But people who are not scientists use words like "theoretically" to mean something very different from how scientists use such words. Colloquially, the word "theoretically" gets used to mean something like "it seems plausible".

I don't know that much about data science or programming, but I can tell you that the above site uses the word "theoretically" in a way that looks colloquial to me and that also contradicts the claim that this technology is a done deal. The fact that the timeline stops at "prototype" agrees that there is no finalized technology here.

Furthermore, the above site has a place where it declines giving a real answer on the basis of "that comes close to trade secret." But, really, I see no evidence that this a thriving business, making scads of money off their super secret algorithm. So I don't see any "trade secret" to protect.

None of that directly proves whether or not the individual in question actually came up with a potentially workable algorithm. But it does strongly suggest that the site is a house of cards -- all appearance and no real substance. There does not seem to be any means to get a "demonstration" even though the site says there is a prototype ready for demonstration. There is no "insert your data and see how it works."

Again, that does not in any way prove there is no algorithm, but it casts strong doubt on the trustworthiness of the individual. And that doubt makes it less likely that there really is an algorithm.

You could try to see if it is possible to look up, for example, the U.S. patent number given on the site under his credentials. I have no idea how that can be done, but you could look up A) if it is a real patent number B) if it is actually for the technology named and C) if it really has the name of the individual on the patent that the site claims has the patent. That would be something you should be able to verify. Most people won't, so claiming to have a U.S. patent is something that sounds good that most people won't verify.

Furthermore, the site claims to have "world wide" patents on other items. To my knowledge, there is no such thing as a world wide patent. I know of no global patenting body and, as far as I know, in order to hold such a thing, you would have to register it in every country on the planet and I strongly suspect, without actually researching it, that there are countries where that wouldn't be feasible because they lack the infrastructure for establishing a patent in that country.

So if you learn more about social stuff, that can help you figure out who is full of hot air and who is actually very knowledgeable in a way that is over your heard. Although social proof has a lot of inherent problems, there are reasons we persist in using it as a metric. There are some good practices that are meaningful in gauging social indicators.
posted by Michele in California at 6:27 PM on April 7, 2015 [5 favorites]

They are initial products, that typically get further modified after they are tried because just like "no plan of battle survives contact with the enemy," most theories don't remain completely intact either once tested against reality.

That should probably say "most hypotheses don't remain completely intact either once tested against reality."

Yeah, I use those words sloppily myself, proving my point about colloquial usage being different from how scientists use those words.

posted by Michele in California at 6:35 PM on April 7, 2015

I generally apply The Economic Argument. If a technology/algorithm/material/process is THAT revolutionary (and also real), some company would be using it to make gajillions of dollars.
posted by Dilligas at 6:46 PM on April 7, 2015

Similar to the Economic Argument, there's the "why me, why now" question. Look, you and I are basically unimportant schmucks. We are, admit it.

So, it's really a breakthrough, why are we the ones just stumbling over this? The guy peddling snake oil? Why is he pitching this to me? Why now?

The people with the really good ideas don't need to try hard to pitch.
posted by Cool Papa Bell at 7:46 PM on April 7, 2015

First, there is no way to be sure. You can't actually evaluate something if you aren't an expert. There are tip-offs someone is an idiot, but pretty much any guideline I can come up with would have had me sneering at plate tectonics or something.

That being said here's Martin Gardner's list in the classic Fads and Fallacies in the Name of Science (hoisted via an old Cosma Shalizi post on a specific bit of quackery)
  • He considers himself a genius.
  • He regards his colleagues, without exception, as ignorant blockheads. Everyone is out of step except himself....
  • He believes himself unjustly persecuted and discriminated against....
  • He has strong compulsions to focus his attacks on the greatest scientists and the best-established theories.
  • He often has a tendency to write in a complex jargon, in many cases making use of terms and phrases he himself has coined....
As I said, you need to accept you'll miss something, but if an idea is a good one, and is being propounded by a crank, likely someone will steal it and repackage it and it will get better advocates.

. . . how to tell whether someone in a field not my own is good at what they do.

Here I just want to point out how much you are asking for. The ability to identify talent without really knowing the subject or being that engaged in the work would be very valuable. If someone *does* give you an answer, write it down because you will be able to make a ton of money as a god among managers. But I am confident no one has ever figured this out. The two strategies boil down to the very imperfect "trust your gut" or "pay attention to them for a long time and look at their accomplishments."
posted by mark k at 8:08 PM on April 7, 2015 [5 favorites]

If they set themselves up as outside the mainstream and telling you what he mainstream don't want you to know, they are quacks. Mainstream scientists do want you to know the truth. They just don't want you listening to people who don't do proper science and have no proof of their miraculous claims.
posted by pracowity at 2:59 AM on April 8, 2015 [2 favorites]

I do have the technical background to spot the errors in some of these kinds of thing, but the method I follow doesn't normally involve using it.
It is hard to pin down the error in a faulty bit of reasoning, so I wouldn't bother trying; instead, you can look at the quality of their counterarguments to their critics:
if someone has a claim of perpetual motion, I would look for their rebuttals of the Second Law rather than reading their theories about how their thing does work.

In the example you give the FAQ does this; the equations are all bluster, but reading the FAQ immediately reveals his mistakes in Q10, Q11 and Q12:
  • he understands that his method cannot compress all inputs (Q12)
  • extra information is needed to know how to decompress the result (Q10) - the number of rounds of recursive compression
These taken together show that there are inputs for which the method expands the output, no matter the (largely irrelevant) equations on the front page. Maybe spotting the error (sneaking some more bits into a secondary output) requires some specialist experience to look out for - there is usually an analog in bogus mathematical or physical claims.
posted by larkery at 7:38 AM on April 8, 2015

Computer science math is a deep subject, and lots of good people work in that field. The chances that anyone outside academia will disprove an accepted result, or prove a major conjecture are very small.

Einstein was an outsider, though.
posted by SemiSalt at 4:19 PM on April 8, 2015 [1 favorite]

Anything that purports to tell you things that "they" don't want you to know is 100% sure to be a scam.
posted by KRS at 6:28 PM on April 8, 2015

QuackWatch's 7 signs of bogus science.
posted by Zed at 9:33 AM on April 9, 2015

I will caution you that using the rubric that "it is statistically unlikely" is an extremely poor measure for judging a specific individual, especially online. I interact with people with PhDs or who are Trans or otherwise statistical outliers of some sort far more online than I do IRL. You can run into literally anyone online. Betting the odds in that way is a broken model on the Internet and amounts to prejudice.

There are 7 billion people on the planet. If something has 1 in a million odds, then there are probably about 7000 people who fit that 1 in a million thing. On the Internet, it would be possible to bring a large number of them together at one time in a forum for this 1 in a million thing and talk to dozens or hundreds of those 7000 people at one time, even if they were scattered across the globe.

Furthermore, if you read biographies or articles about famous people, anecdotes about them being not recognized or not credited for their ability or whatever are a dime a dozen. Charlie Chaplin once entered a Charlie Chaplin look alike contest and came in third. Before Warren Buffett was famous, he once offered to invest money for his next door neighbor who turned him down. At the time, Buffett was running a private fund from home. He intentionally kept the number of members low so he wouldn't be subjected to federal regulations concerning public funds. So it wasn't very obvious that he was, in fact, raking in the dough behind closed doors for himself and a select group of people. I am sure the neighbor is still kicking himself.

As noted above, there are things you can seek to verify about the site you linked to. I have looked up the patent number provided on the site. It is a real patent number and it is for the technology he described. It is under the name provided and lists Melbourne Australia, the place he claims he was at one time. Linky

I still think "world wide patents" sounds specious (because patents are issued by governments and there is no world government, so there is no global patenting office, etc). On the other hand, Dr. Wong would speak English as a second language. Sometimes, intelligent, well educated people are sloppy about certain things and/or have specific blind spots. It happens. But the listed patent number checks out -- it exists, it is for the technology named, it is under the name listed on the site and assigned to a company in Hong Kong. All of that fits the claims of credentials on the site. Furthermore, here is a list of Dr. Wong's filed and mostly lapsed patents in Australia: Linky

Again, that does not prove the algorithm exists. But verifying that the U.S. patent exists and that there are, in fact, other patents previously filed by this man makes the information on the site more credible. So if you have an earnest interest in this algorithm (because it would fill some pressing need of yours), and not just a passing fancy in the form of "Huh, that would be neat," then it would be worth your while to check into it further. If it is just a case of "Huh, that would be neat." I suggest you mentally put it in the category of "Plausible, but not proven. Moving on."

So here are your two best means to judge a specific individual:

1) Check out the social indicators of veracity. Some of them can be verified. If he has a PhD, some university issued it. If he has a job at a prestigious institution or big company and is stating publicly he has a PhD, the odds are good the company verified it and he isn't making it up. (When I worked for a Fortune 500 company, they wanted copies of my transcripts. They did not just take my word for it that I had all these college classes and an AA and a Certificate in GIS. I imagine had I been unable to produce legitimate transcripts, I would have been fired for lying on my resume.) If he claims to have a US patent and provides the number, it can be readily looked up online. Etc.

2) Objective evidence.
Einstein's Theory of Relativity was eventually proven with photographic evidence taken during a total eclipse of the sun. It took years to get the evidence because total eclipses don't happen very often. There are two to three solar eclipses per year, but most are partial eclipses. We recently had a total solar eclipse this year. The last one before that was in 1999. As noted above, Einstein was an outsider. He couldn't even get a job in academia like he wanted. He worked at a patent office for years. But someone must have believed him enough to seek the evidence because he is not the person who took the photographs proving his theory. There were two teams on the first attempt. It failed, in part because it was during WWI and one of the teams was detained as spies or something. IIRC, there were six teams the second time around and, in spite of rain and clouds, one of them did succeed. Then, overnight, Einstein was world famous.

For the above algorithm, objective evidence would be things like if it was published or if a demo of the technology was provided. If he has a very niche business, he may have no need to prove anything to the general public. It could be that his market is some small set of industry insiders in some relatively small (but possibly important) industry. So you may never see objective evidence in this case, yet the algorithm might actually exist.

When you are dealing with specific individuals, saying "That's statistically unlikely" as your reason for dismissing them has you assuming that everyone you interact with is heterosexual, that anyone who tells you they are Trans is lying because that is uncommon and so on. That's an extremely bad way to go about handling such things. I really do not recommend it as a method.
posted by Michele in California at 11:10 AM on April 9, 2015 [1 favorite]

« Older What song features a chorus of "CEEEeeee JAAAaaay...   |   But I'm NOT The One Who Knocks! Newer »
This thread is closed to new comments.