What's the standard for g. i.e. general intelligence?

November 14, 2007 5:34 PM

Do intelligence tests measure g (i.e. general intelligence)? How do we know? With most testing standards, there's an arbitrary physical constant in the world for us to measure and quantify: the meter is defined in terms of a wavelength of light in a vacuum, for instance. However, intelligence tests seem more like the kilogram's platinum-iridium cylinder, which is a sort of moving target. What's the standard for g?

It seems like "psychometrics" always assume that analytical abilities as measured in math, reading comprehension, and various logic 'games' are highly correlated with g, but I'd like to know what sort of evidence there is for this claim. The field has a vast internal body of research on this to determine whether a particularly testing question tracks with the overall testing apparatus (they call it g-loading.) But where do they get their initial g assessments from?

I'm just trying to get my head around why this isn't a vicious circle.

posted by anotherpanacea to science & nature (18 comments total)

Forgive me if you already read this and found it unhelpful, but I thought the Wikipedia page on the subject laid it out pretty well.

I'm just trying to get my head around why this isn't a vicious circle.
You can make a good argument that that's exactly what it is. I think the reason it hasn't really progressed beyond this is that it's hard they can't really agree on how to define "intelligence" to begin with, so why not use the notion of -factor to make everyone happy.
posted by Brian James at 5:47 PM on November 14, 2007

Its made up of course. Its even less accurate than the kilogram, because it is just a construct. There's no objective definition of intelligence possible. The whole idea of intelligence is a construct in the first place.

The best you can say is that it is agreed upon by a general number of researchers. G is a fantasy.
posted by Ironmouth at 5:47 PM on November 14, 2007

Check out this piece: g, a statistical myth .

Basically, as I understood it, the article claims that all the statistical evidence people use to claim that g must exist can just as easily be explained by a large number of independent abilities, so long as those abilities are positively correlated (as you'd expect them to be, if they have anything at all to do with health or wealth etc.)
posted by wyzewoman at 5:59 PM on November 14, 2007

g is indeed a joke. Measuring it was something that someone thought would be a good idea once, and they kludged up a test for the purpose. Other people refined the tests.

The real problem that we have is that these measures of g (full scale IQ), which measure something that everyone agrees doesn't exist, correlate strongly with all kinds of things, like how much money you're going to make and how succesful you'll be in life. Figuring out what these correlations are all about is where the fun is at in these endeavors.
posted by ikkyu2 at 6:12 PM on November 14, 2007

As someone who was introduced to this topic by this post, I'm going to mostly agree with Ironmouth. However, I think it bears emphasis that all standards are going to be arbitrary. All concepts are abstractions used for differentiation in our minds, which are (according to some) what makes conscious thought possible beyond stimulus-response.

The fact that it is the tests themselves that determined the "constant" here doesn't seem troubling to me, since that's the only thing it could have meaningfully been based on--g, the abstract concept, is a measurable trait only insofar as it exists, and it does not exist until it is measured, and it can only be usefully measured through testing. It is, therefore, necessary that its definition relies upon the types of tests that tend to correlate best. According to wikipedia, "The abstraction of g stems from the observation that scores on all forms of cognitive tests correlate positively with one another."

If I'm understanding you correctly, you are asking how to determine whether a test measures cognitivity. Is that a fair reading?
posted by Phyltre at 6:12 PM on November 14, 2007

Re: wyzewoman--The very reason we came to suspect there was a general intelligence is because we can’t create tests of intelligence (or tests of any kind of cognitive capacity) that don’t correlate with each other. There’s something about our physical make-up where if we’re good at one cognitive activity, we tend to be good at another (the correlations can be very low, but they are never zero). Most respected researchers in the field accept that there is a general intelligence. The exact nature or what it means physiologically is still up for debate (don't even mention Howard Gardner or Stephen J Gould, I don't include them as respected researchers.)

To answer your main question: RB Cattell gave us the answer for how to locate g without tautology. He said that the best way to locate general intelligence was to also administer tests of different constructs like personality or moral values. In factor space, having known measures of other major constructs is the only way to locate g and know for sure what you've got.

G is a statistical construct, but the very fact that it does correlate with life success, with capacity for complexity, with school grades, with job performance, etc. indicates that we’re measuring something real and important. This is one of my favorite readings on the subject:

Gottfredson, Linda S. (1997). Why "g" Matters: The Complexity of Everyday Life.
Intelligence, 24(1), p79-132.
posted by parkerjackson at 7:14 PM on November 14, 2007 [2 favorites]

I don't think you quite understand the notion of General Intelligence. G is primarily an empirical result, not a theory. People's performance on ALL cognitive tasks correlate. Everything from reaction time to verbal analogies to short-term memory, it all correlates. G is simply a measure of the common correlation between tasks, as extracted by factor analysis.

Haven't you noticed that some of your classmates do well in all of their classes? These people have high G. Others that struggle in all their classes? Low G. Picture a person with down syndrome. Low G. Picture Albert Einstein. High G.

This result is often surprising to people because they don't understand correlations. Correlations don't have to be perfect to be real. Its not like a person at the 75th percentile on one test is at the 75th percentile on ALL tests. This is just a general tendancy we are talking about. The interpretation is more like: A person scoring above the 95th percentile on one test is almost sure to fall in the top half of the distribution on another test.

G-loading refers to the degree to which a particular task correlates with the general factor across all cognitive tasks. This is usually related to the task's complexity and the amount of reasoning involved.

You are right that G can seem a bit circular. We define G as test performance and then try to design tests that measure it well. However, this is simply an empirical result. I guarantee you if you design a test, any test, as long as that test taps into some kind of cognitive ability it is going to be correlated with G. The intentional effort goes into bumping up the test's correlation with G, not causing the correlation in the first place.

Why does G predict life success? Because life is a general test. People don't succeed based on their performance at any one little task, rather it is a summation of performance on all kinds of things. Getting through school, landing a job, getting promoted, investing wisely, etc. A G-loaded society is a good thing, it means we are a meritocracy. The only way to avoid it would be to set up artificial barriers to success like heriditary classes and good-ol-boy networks. The extent to which these sorts of barriers exist in our society explains why success isn't MORE G-loaded.

Why does G exist? There are many different theories. Level of mylination in the brain, working memory capacity, overally number of neurons or axon density, mutualism (the theory that every specific cognitive ability reinforces every other one, leading to G). Part of it is almost certainly gene-environment interaction. Smart people enrich their environment (read tons of books, post on metafilter), dumb people don't. This enhances the observed G factor.

What is G? This is an even more difficult question. It is some combination of reasoning ability, working memory capacity, and "cognitive flexibility" or the ability to learn novel tasks quickly. The primary common element in both definitions of intelligence and tests of intelligence is some type of reasoning requirement.

If you are interested in learning more about G I suggest you take parkerjackson's advice and dive into some scientific journals. Ignore the pop-psychology version as best you can and be skeptical of people who have built their careers on an anti-G crusade (Gould, Gardner, etc.).
posted by Paladin165 at 8:23 PM on November 14, 2007 [2 favorites]

be skeptical of people who have built their careers on an anti-G crusade (Gould, Gardner, etc.).

Oh, come on. There undoubtedly people who have "built their careers on an anti-G crusade" but to use Gould or Gardner as examples is to ignore the breadth of either man's talents.
posted by Neiltupper at 8:41 PM on November 14, 2007

You are right I didn't mean to criticize all of their work, just their opinions on g-theory. Gardner especially is probably worth reading. Gould I'm not so sure about. He has a lot of detractors in archeology and biology as well as psychology. I think it is fair to say both men have (had) larger gifts for writing than for research.
posted by Paladin165 at 10:41 PM on November 14, 2007

I think it is fair to say both men have (had) larger gifts for writing than for research.

No, it's not fair to say that about Stephen Jay Gould. What utter bullshit.
posted by grouse at 1:35 AM on November 15, 2007

I don't think you quite understand the notion of General Intelligence.

I don't think you do either, Paladin165, or you might have been able to explain it better. Is it really the case that "bumping up the correlation" isn't a way of gaming the tests? Surely there would be cases in which a "naturally existing" weak correlation would say much less about a given trait than intelligence tests which have "bumped up" that correlation would?

I'm also confused by how successful SJGould was. Shouldn't we pay attention to him because of that? We live in a meritocracy, after all.
posted by OmieWise at 5:20 AM on November 15, 2007

Thanks all for your responses. I'm most interested in the arguments made by g-proponents, since the anti-g accounts seem so intuitively obvious. As I suspected, there's a strong correlation between a belief that we live in a functioning meritocracy and the belief in g. Thanks Paladin165, especially, for the phrase: "Life is a general test." The denial of contingency, of statistical anomalies, of gaming and cheating, of social and economic factors, and of luck(!) inherent in that phrase is exactly what I was looking for.

Nonetheless, the elaborated accounts of this mysterious correlation between testing achievement has a persuasive force that I've been lacking up until now, so thank you.
posted by anotherpanacea at 7:06 AM on November 15, 2007

I don't think you do either, Paladin165, or you might have been able to explain it better. Is it really the case that "bumping up the correlation" isn't a way of gaming the tests? Surely there would be cases in which a "naturally existing" weak correlation would say much less about a given trait than intelligence tests which have "bumped up" that correlation would?

It isn't gaming the tests if g is what you are trying to measure! Sure there are tests for other traits, but tests measure multiple things at once. Part of what they measure will almost always be g, but of course psychometricians try to maximize the loading on whatever trait it is they are trying to measure.

I'm also confused by how successful SJGould was. Shouldn't we pay attention to him because of that? We live in a meritocracy, after all.

Once again, correlation doesn't imply perfect correlation. People do not earn success in direct proportion to their gifts, it is a much more random process. Second, just because someone is intelligent doesn't mean you have to automatically believe everything they say. Virtually all major academics, authors, and journalists are highly intelligent. We obviously can't listen to all of them, and they can't all be right about everything.
posted by Paladin165 at 7:13 AM on November 15, 2007

Thanks Paladin165, especially, for the phrase: "Life is a general test." The denial of contingency, of statistical anomalies, of gaming and cheating, of social and economic factors, and of luck(!) inherent in that phrase is exactly what I was looking for.

Some factors are fixed, others are random. Only the fixed effects will produce systematic effects that we can study. Most of the effects you mention are random. No one would deny the role of the factors you mention, but intelligence is the single largest individual differences variable in the prediction of career success. Also, many of the other important variables (family income, social class, geographic location, connections) are correlated with intelligence.
posted by Paladin165 at 7:17 AM on November 15, 2007

Paladin165: If you're conversant in statistics, you should check out wyzewoman's link, especially the section titled "How to make 2766 independent abilities look like one g factor." An argument of this nature was what I had primarily in mind when posting here.

Your assertion that a correlation between variables should be dispositive with regard to a causal relationship indicates that you're not actually very familiar with statistical argumentation. So consider this: we know that class, race, and connections play a role in career success, and yet we find that there is no predictive bias between different groups. In other words, given the legal standards of racism in the US, there should be a lot of smart black men who are doing less well than smart white men. And yet, we find the opposite: g predicts career success regardless of social context. This proves pretty conclusively that the measurements are building social biases into the metric: they're not really measuring g, but rather the social antecedents of success. Some citations: 1, 2.
posted by anotherpanacea at 8:05 AM on November 15, 2007

we know that class, race, and connections play a role in career success, and yet we find that there is no predictive bias between different groups

I am having difficulty parsing this. Can you clarify?
posted by topynate at 12:54 PM on November 15, 2007

anotherpanacea,

I think you're asking for more subtlety here than an internet posting conversation can really provide...but I will try to respond to your points.

Wyzewoman is right, of course g can be interpreted as a set of correlated abilities. That does not make it any less real. All kinds of macrophenoma can be decomposed into constituent parts. Are waves any less real because they can be described as “just molecules of water moving around in a way that happens to be correlated”? Anyway, correlation is usually taken as evidence of dependence, so arguing that these are highly correlated yet "independent" abilities is hard to do. The mutualism theory I mentioned, for example, holds that g is a result of thousands of small abilities reinforcing each other in a positive manifold. Under this theory g would be a result of dependencies between discrete cognitive abilities. As Wyzewoman’s link points out, you can also get G simply from assuming that any one test is really measuring hundreds of micro-abilities. It is not practically meaningful to call these test scores “independent” if the subsets of micro-abilities they load on are always overlapping, regardless of whether the micro-abilities themselves are in some sense “independent”.

The most commonly accepted current theory of cognitive abilities (Carroll, 1993) compromises between the single-ability and multiple-ability perspectives by positing a hierarchy of abilities. I know of no respected theory of intelligence that posits completely independent abilities. Such gross factors as genetic defects and nutrition would suffice to cause at least SOME dependency between abilities for some people so this is a fairly weak position at the population level. Also it is logical that suppose that many cognitive abilities are functionally dependant on one another. You can’t very well recall material from long term memory if you are unable to store it in the first place, and you can’t store visual material if you visual center isn’t working, etc.

I think you are attacking the theory of g as a "real" biological "organ" or something, as if a person with a 150 IQ has a bigger g-center in his brain than someone with a 120 IQ. I completely agree with you here. We have no idea whether anything like this is true or not. There is some evidence for things like differential neural pruning, mylination, brain size, metabolic efficiency, etc, but all of this is fairly speculative. The view of G I am defending is purely statistical/educational/practical. Psychometrics is a very empirical discipline, it is based on patterns of test scores, not on psychological theorizing or neuroscience. That is what makes it one of the more scientific branches of psychology (contrary to popular opinion). The fact is that if you walk into any classroom, or any workplace, you will find people that everyone describes as "slow" or "thick" and others that everyone views as "quick" or "smart". This is the G I am talking about. If you are looking to hire someone, you want to know if they have a lot of G purely because this translates into problem solving ability, creativity, and ultimately more profit for the company (other things being equal of course, and assuming the job actually requires the intelligence).

The validity of G as a theoretical entity in hard sciences like neuroscience is completely different from its validity for practical purposes. Neuropsychological tests typically attempt to measure extremely specific abilities to detect problems with specific brain regions. These specific abilities have almost no predictive validity in the real world for something like educational or career success, that is why those on the more practical side of things seem so G-happy (companies like Microsoft and organizations like the NFL use IQ tests regularly). By contrast, these specific abilities are much more theoretically interpretable for hard scientists than a big fuzzy construct like G.

G is much like physical fitness, or the "car size" factor mentioned in wyzewoman's link. It is simply a general measure of overall mental functioning. What's wrong with that?

I'm not sure I understand your point about race and career prediction. How could the tests "not really measure" g? Weren't we just saying in earlier posts that in some ways the tests define what G is? How does a simple numerical analogies test (e.g. items like [7:20::3:12::1:?] a highly g-loaded test) measure the "social antecedents of success"? Certainly not directly. Presumably you mean high scores on these tests and career success are both caused by a third variable like class and privilege, and this relationship is NOT mediated by intelligence. This is a difficult proposition to straight out disprove, but I would say it is weakened by data from a variety of sources. One of the best is a study done by Charles Murray looking at IQ-income relationships within families (http://www.eugenics.net/papers/murray.html). By looking at high and low IQ siblings he could control for the class and privilege factors you are talking about. Restricting his sample to just those born into the top quartile of the income distribution, he found that “bright” siblings made twice the amount of money as their “dull” siblings. Some of the most interesting research in recent years has been about disproving myths about the benefits of attending prestigious colleges or even college at all for that matter. It turns out that many of these “wealth and privilege” factors have almost no effect after controlling for initial IQ.

One of the problems that plagues this topic is the general lack of contact many educated Americans have with the bottom half of the bell curve. If you only look at the top half, both the importance of the g-factor and the IQ-income relationship decreases, mostly due to restriction of range. I think this explains why so many academics have trouble accepting the reality of general intelligence, they are stuck in a very tiny section of a much larger scatter plot. To better see the problems I have with the wealth-and-privilege argument, imagine a wealthy family that gives birth to a severely learning-disabled child. No amount of money or care is going to raise this child’s IQ. How is his privilege supposed to raise his test scores and land him a top-tier job? In order for what you are saying to be true it must be possible for money to “buy IQ” and this simply isn’t the case. There is no instructional method or intervention that has been proven to raise IQ in a meaningful and permanent way (besides things like breastfeeding and nutrition). If you come across one please email me about it!
posted by Paladin165 at 4:31 PM on November 15, 2007

Anotherpanacea-- I don’t find the section you mention in wyzewoman's link compelling. Yes, there could be 2,000 independent abilities, but given our observations of the way the brain works, it seems just as possible that there are 2,000 dependent abilities. That is a big assumption of that individual, and one that is not parsimonious. Why should there be 2,000 independent abilities and not 100 dependent abilities (all influenced by the infrastructure of the brain)? Scientists are supposed to always side with parsimony.

BTW, correlations are not mystical fabrications of science; they are simply quantifications of how similarly two tests rank-order people. If I give a group of 100 people a test of vocabulary and rank-order them on their knowledge, then to some degree, I can predict their rank order on another cognitive task, say mental arithmetic. The fact that they are rank-ordered similarly implies to me that someone who is good at one thing is good at another. The fact that I can do this with 100's of tests of all formats and sizes, tells me there's something pervasive causing those ranks to be similar.

This observation (as Paladin pointed out) is the experiential justification for our statistically-constructed g. If the only thing that intelligence tests predicted was other intelligence tests, then I would see the value of your argument, but it doesn’t—it predicts lots and lots of useful things in our academic, business, and social lives.

I think you may be confounding a belief in g with a belief in a FIXED intelligence. This is absolutely not the case. I, for one, believe there are many factors that contribute to g—luck, a good environment, etc. And on real tests, of course, there is cheating and gaming the system. Measurement error and developmental growth on the trait does not mean that there isn’t a real basis for g. We observe g in everyday life as the capacity to learn things quickly and to handle many cognitive tasks at once. This is something that can absolutely be impacted by poor nutrition, by unresponsive caregivers, by being lucky enough to be rich, etc. Don’t assume that people who believe in a general intelligence also believe that we’re born with a set intelligence OR that they believe in absolute Social Darwinism. I can believe in g and still believe we should feed the poor and provide opportunities to every child.
posted by parkerjackson at 4:47 PM on November 15, 2007 [1 favorite]