What do social scientists think of IQ?
April 10, 2017 10:36 PM Subscribe
How respectable and/or embarrassing is the IQ / 'g' literature in the halls of academia?
I put a pretty low value on the value of IQ testing, and the idea of a "general intelligence" jars with my personal experience (way above average at math, nearly pathologically stupid at facial recognition, etc.) I've read The Mismeasure of Man back in the day and know about the Flynn effect and pretty much everything I read from people I usually trust falls into the "extreme skepticism" camp for anything other than a few types of limited uses (like, epidemiological studies of lead pollution effects.)
But I've seen at least a few apparently credentialed types who insist IQ and g are on really solid ground and correlates wonderfully with fMRI, reaction time or other fancy crap that gives objective measures of something or other. And even if I knew enough I wouldn't have the patience to really dig into what's really going on in the literature.
So I'm curious to hear from anyone who's a researcher / grad student / etc. in an active area about what their colleagues actually think. If you went to a conference and it came up in conversation would everyone roll their eyes at how stupid the public is, on one side or another? Or would their be a heated discussion? Are the people who like 'g' cranks, or just more indifferent to having their research misinterpreted / criticized? Does it vary between departments or subdisciplines?
To be clear I'm not looking for a link that argues a position, or even the professional opinion of an individual however qualified you are. I'm already kind of overwhelmed by the number of opinions I can find. What I'm curious about here is if there is an active debate among people who understand this stuff and if so how the lines are drawn.
(Also, nothing about genetics or heritability please, which I assume can be separated out?)
I put a pretty low value on the value of IQ testing, and the idea of a "general intelligence" jars with my personal experience (way above average at math, nearly pathologically stupid at facial recognition, etc.) I've read The Mismeasure of Man back in the day and know about the Flynn effect and pretty much everything I read from people I usually trust falls into the "extreme skepticism" camp for anything other than a few types of limited uses (like, epidemiological studies of lead pollution effects.)
But I've seen at least a few apparently credentialed types who insist IQ and g are on really solid ground and correlates wonderfully with fMRI, reaction time or other fancy crap that gives objective measures of something or other. And even if I knew enough I wouldn't have the patience to really dig into what's really going on in the literature.
So I'm curious to hear from anyone who's a researcher / grad student / etc. in an active area about what their colleagues actually think. If you went to a conference and it came up in conversation would everyone roll their eyes at how stupid the public is, on one side or another? Or would their be a heated discussion? Are the people who like 'g' cranks, or just more indifferent to having their research misinterpreted / criticized? Does it vary between departments or subdisciplines?
To be clear I'm not looking for a link that argues a position, or even the professional opinion of an individual however qualified you are. I'm already kind of overwhelmed by the number of opinions I can find. What I'm curious about here is if there is an active debate among people who understand this stuff and if so how the lines are drawn.
(Also, nothing about genetics or heritability please, which I assume can be separated out?)
_(Also, nothing about genetics or heritability please, which I assume can be separated out?)_
I think it can't be -- that is, all of the active debating is involved with this issue. Once you separate that out, you're left with (1) tests of random mental skills do correlate with each other pretty well -- someone who does well on math tests will also probably do better than the average on vocabulary tests, and so on and so forth. Statistically, you can pull out a common factor, g, to express that correlation. And this is quite stable for any individual over their life. (2) Does that common factor mean anything intrinsic about the individual, as opposed to the sum of their environment, experiences, general health, and so on? No way to tell for sure without a physical mechanism, which we don't have.
The argumentation is all between people who are arguing some version of 'come on, it's got to be genetic, mostly. There's a physical mechanism there, gotta be,' and people saying 'no, really, no, we do not have good evidence for that at all.'
(This is an entirely unexpert opinion -- I just read about this stuff some.)
posted by LizardBreath at 5:34 AM on April 11, 2017 [1 favorite]
I think it can't be -- that is, all of the active debating is involved with this issue. Once you separate that out, you're left with (1) tests of random mental skills do correlate with each other pretty well -- someone who does well on math tests will also probably do better than the average on vocabulary tests, and so on and so forth. Statistically, you can pull out a common factor, g, to express that correlation. And this is quite stable for any individual over their life. (2) Does that common factor mean anything intrinsic about the individual, as opposed to the sum of their environment, experiences, general health, and so on? No way to tell for sure without a physical mechanism, which we don't have.
The argumentation is all between people who are arguing some version of 'come on, it's got to be genetic, mostly. There's a physical mechanism there, gotta be,' and people saying 'no, really, no, we do not have good evidence for that at all.'
(This is an entirely unexpert opinion -- I just read about this stuff some.)
posted by LizardBreath at 5:34 AM on April 11, 2017 [1 favorite]
It is considered a truism in organizational psychology that g is the single best predictor of job performance. Mind you, it is not the only predictor: there are numerous other factors, many of which account for significant additional variance.
posted by DrGail at 8:35 AM on April 11, 2017 [1 favorite]
posted by DrGail at 8:35 AM on April 11, 2017 [1 favorite]
I used to do cognitive psych/cognitive neuro research on human memory. My first publication, actually, was on the undeniable but mysterious correlation between episodic memory and WAIS IQ. Seconding everything forza said-- excellent summary.
I'll say something that more or less overlaps: IQ ain't intelligence. It's a general factor that can't be explained by a bunch of specific tests. (The theory behind factor analysis is key here.) When you ask a bunch of people a bunch of questions all designed to get at a single facet of personality or pathology, like the CES-D depression scale, or the extraversion domain items on the NEO-FFI, they should all be measuring the same thing, so when you do something like exploratory factor analysis (or PCA) on the questions across a bunch of people, they should all load on one main dimension. You call that extraversion, or depression, or whatever. (There's a serious chicken-and-egg problem with theoretical scales and items designed to get at them, which is why my most recent research group, for one, is trying to do away with questionnaires and just measure these factors in someone's naturalistic language.)
I bring this up because when you have a whole host of cognitive tasks that are each measured with some performance score, what happens when you dimensionally reduce them, or run factor analysis? They probably all have positive loadings on a single factor. That is, people who tend to do well on one do well on others. More than you'd expect by chance. That's why it's called the general factor.
An IQ test like WAIS is designed to have good coverage of all of the different cognitive tasks people might run across, so that the measures that come out of it have internal validity (all loading consistently on one factor, in the same direction) and ecological/external validity, that people who score well do better at everyday cognitive tasks. The second one is the main source of controversy about IQ testing. Like, does knowing who Howard Carter was mean you're more intelligent? Depends what you mean by intelligent. But that's the sort of detail you might need to retrieve in everyday life, so that's the sort of thing that IQ tests measure.
In summary: ¯\_(ツ)_/¯ Psychometrics (looking at it as a mostly-outsider) is weird, and now we're starting to have enough data and computational power that these unnatural batteries (both for measuring ability and other traits) aren't necessary. But they're still what new data-driven techniques are going to be measured against. That's my $0.02.
posted by supercres at 11:03 AM on April 11, 2017 [2 favorites]
I'll say something that more or less overlaps: IQ ain't intelligence. It's a general factor that can't be explained by a bunch of specific tests. (The theory behind factor analysis is key here.) When you ask a bunch of people a bunch of questions all designed to get at a single facet of personality or pathology, like the CES-D depression scale, or the extraversion domain items on the NEO-FFI, they should all be measuring the same thing, so when you do something like exploratory factor analysis (or PCA) on the questions across a bunch of people, they should all load on one main dimension. You call that extraversion, or depression, or whatever. (There's a serious chicken-and-egg problem with theoretical scales and items designed to get at them, which is why my most recent research group, for one, is trying to do away with questionnaires and just measure these factors in someone's naturalistic language.)
I bring this up because when you have a whole host of cognitive tasks that are each measured with some performance score, what happens when you dimensionally reduce them, or run factor analysis? They probably all have positive loadings on a single factor. That is, people who tend to do well on one do well on others. More than you'd expect by chance. That's why it's called the general factor.
An IQ test like WAIS is designed to have good coverage of all of the different cognitive tasks people might run across, so that the measures that come out of it have internal validity (all loading consistently on one factor, in the same direction) and ecological/external validity, that people who score well do better at everyday cognitive tasks. The second one is the main source of controversy about IQ testing. Like, does knowing who Howard Carter was mean you're more intelligent? Depends what you mean by intelligent. But that's the sort of detail you might need to retrieve in everyday life, so that's the sort of thing that IQ tests measure.
In summary: ¯\_(ツ)_/¯ Psychometrics (looking at it as a mostly-outsider) is weird, and now we're starting to have enough data and computational power that these unnatural batteries (both for measuring ability and other traits) aren't necessary. But they're still what new data-driven techniques are going to be measured against. That's my $0.02.
posted by supercres at 11:03 AM on April 11, 2017 [2 favorites]
Oh, I meant to add: the link that my study found is that people with high measured IQ can do better at remembering lists of words. Like, I'm going to show you 16 words, one at a time, on a computer screen. After a delay where I make you do as many simple arithmetic problems as you can, I want you to tell me as many of those words as you can remember, in any order. That correlates super-strongly with IQ, across a variety of experimental manipulations.
Why? Beats me. Correlates even when we're not looking at the working memory parts of the WAIS, which should theoretically be a different process anyway. Those other tasks predict ability in the experimental task. But something, maybe general cognitive efficiency, maybe comfort at being in test-like scenarios, maybe top-down strategizing, makes them correlate. That's what g/IQ is measuring. In theory. Again: ¯\_(ツ)_/¯
posted by supercres at 11:08 AM on April 11, 2017 [2 favorites]
Why? Beats me. Correlates even when we're not looking at the working memory parts of the WAIS, which should theoretically be a different process anyway. Those other tasks predict ability in the experimental task. But something, maybe general cognitive efficiency, maybe comfort at being in test-like scenarios, maybe top-down strategizing, makes them correlate. That's what g/IQ is measuring. In theory. Again: ¯\_(ツ)_/¯
posted by supercres at 11:08 AM on April 11, 2017 [2 favorites]
I have more experience than I'd like to admit with these tests.
IQ and g measure very specific things that the lay public tends to think mean more than they actually do.
Tests of IQ tend to have a lot of subtests. The Wechsler Adult Intelligence Scale IV, for example, has:
Verbal Comprehension: Similarities, Vocabulary, Information, Comprehension.
Perceptual Reasoning: Block Design, Matrix Reasoning, Visual Comprehension, Picture Completion, Figure Weights.
Working Memory: Digit Span, Arithmetic, Letter-Number Sequencing .
Processing Speed: Symbol Search, Coding, Cancellation.
People with higher full-scale IQ do better as a whole on all of these tests, but full-scale IQ is a gestalt and the real picture of someone's functioning is in the subscales. So, for example, person A and person B could have full-scale IQs of 130 but person A could have better spatial reasoning and person B could have better verbal skills. These tests also have ceilings and floors, which is part of why people who score at the extremes of these often get the run-around when trying to figure out what the hell is going on in their lives.
IQ tests also used to not be culture-fair, and their design has gotten better, but they're heavily biased toward Western concepts of intelligence. There's also sociocultural impacts on the average IQ, like the Flynn effect and the effect of development and the economy - richer countries have higher average IQs because you've got better conditions for growing brains.
IQ and g have correlations with things like years of education and number of degrees, income, and whether you stay out of jail, and these are pretty robust correlations, but it's one factor of many. There is, interestingly, a concept called socially optimal IQ where between the IQ ranges of 125 and 155 you're set up extraordinarily well to occupy a position of power in society because you're just smart enough to have an advantage over most people but just close enough to the average to not have gross social difficulties. Above this, you might as well be kind of an alien, and people with 160+ IQs have to deal with the sharpest damn double-edged sword there ever was.
So there's a robust and pretty well-accepted literature, but the view on IQ and g from what I understand from my psych colleagues is that there's a lot of nuance.
posted by actionpotential at 11:39 AM on April 11, 2017 [4 favorites]
IQ and g measure very specific things that the lay public tends to think mean more than they actually do.
Tests of IQ tend to have a lot of subtests. The Wechsler Adult Intelligence Scale IV, for example, has:
Verbal Comprehension: Similarities, Vocabulary, Information, Comprehension.
Perceptual Reasoning: Block Design, Matrix Reasoning, Visual Comprehension, Picture Completion, Figure Weights.
Working Memory: Digit Span, Arithmetic, Letter-Number Sequencing .
Processing Speed: Symbol Search, Coding, Cancellation.
People with higher full-scale IQ do better as a whole on all of these tests, but full-scale IQ is a gestalt and the real picture of someone's functioning is in the subscales. So, for example, person A and person B could have full-scale IQs of 130 but person A could have better spatial reasoning and person B could have better verbal skills. These tests also have ceilings and floors, which is part of why people who score at the extremes of these often get the run-around when trying to figure out what the hell is going on in their lives.
IQ tests also used to not be culture-fair, and their design has gotten better, but they're heavily biased toward Western concepts of intelligence. There's also sociocultural impacts on the average IQ, like the Flynn effect and the effect of development and the economy - richer countries have higher average IQs because you've got better conditions for growing brains.
IQ and g have correlations with things like years of education and number of degrees, income, and whether you stay out of jail, and these are pretty robust correlations, but it's one factor of many. There is, interestingly, a concept called socially optimal IQ where between the IQ ranges of 125 and 155 you're set up extraordinarily well to occupy a position of power in society because you're just smart enough to have an advantage over most people but just close enough to the average to not have gross social difficulties. Above this, you might as well be kind of an alien, and people with 160+ IQs have to deal with the sharpest damn double-edged sword there ever was.
So there's a robust and pretty well-accepted literature, but the view on IQ and g from what I understand from my psych colleagues is that there's a lot of nuance.
posted by actionpotential at 11:39 AM on April 11, 2017 [4 favorites]
I have a friend who has a PhD in cognitive psychology and works as a school psychologist. He's trained to administer IQ tests and we've discussed them a bit. My understanding is that the tests are used in a school psychology context to help diagnose and develop treatment plans for children who are struggling in school. There are probably some issues with how to adjust the scoring of the tests for kids, but in general, the field seems to take IQ tests pretty seriously and considers them useful as a diagnostic tool in that context.
posted by phoenixy at 12:51 PM on April 11, 2017 [2 favorites]
posted by phoenixy at 12:51 PM on April 11, 2017 [2 favorites]
Response by poster: Thanks to everyone who's answered so far. It's very interesting. I did want to clarify that if my wording discouraged people who are say, sociologists or anthropologists as opposed to psychologists or even just cognitive psychologists from answering, that wasn't the intent.
Facial recognition isn't tied to IQ, I'm 99% sure.
To be clear that's why I brought it up. If a useful cognitive ability like facial recognition really doesn't correlate with IQ than that seems to tie into the "IQ measures something but what that is is unclear" line forza and others take.
OT: I test well but am borderline pathological with faces. I mentioned this too my dad a couple years ago and it was an amazing conversation. He said as a kid he couldn't recognize his mother picking him up from school unless he made a note of remembering what she was wearing before he left. I had no idea. He's in his seventies, I was mid-forties, we'd never talked about it. Siblings didn't know. He's got it way worse than me but apparently we both had developed coping strategies.
posted by mark k at 10:03 PM on April 11, 2017
Facial recognition isn't tied to IQ, I'm 99% sure.
To be clear that's why I brought it up. If a useful cognitive ability like facial recognition really doesn't correlate with IQ than that seems to tie into the "IQ measures something but what that is is unclear" line forza and others take.
OT: I test well but am borderline pathological with faces. I mentioned this too my dad a couple years ago and it was an amazing conversation. He said as a kid he couldn't recognize his mother picking him up from school unless he made a note of remembering what she was wearing before he left. I had no idea. He's in his seventies, I was mid-forties, we'd never talked about it. Siblings didn't know. He's got it way worse than me but apparently we both had developed coping strategies.
posted by mark k at 10:03 PM on April 11, 2017
Seconding that face recognition seems to be a very specialized brain circuit/ability that doesn't have a lot to do with general intelligence. Oliver Sacks famously had prosopagnosia (face blindness). Conversely, there are also people who are not necessarily extraordinary geniuses in other aspects of their life whose facial recognition is so good that Scotland Yard uses them as part of a special investigations team.
That's not really evidence for or against the idea that a general intelligence factor (partially) maps to something measurable in biology, though. Modularity in the brain doesn't preclude the influence of overall general factors, or vice versa for that matter.
posted by en forme de poire at 2:28 AM on April 12, 2017
That's not really evidence for or against the idea that a general intelligence factor (partially) maps to something measurable in biology, though. Modularity in the brain doesn't preclude the influence of overall general factors, or vice versa for that matter.
posted by en forme de poire at 2:28 AM on April 12, 2017
Not only is facial recognition a distinct thing, but "normal" people vastly overestimate their ability to recognize faces. Studies show that we get this wrong about half the time if important contextual cues are missing.
So, people who do recognize faces and think you are terribly impaired are basically prejudiced.
Testing is a tool. It can serve a qualified professional well in their efforts to assess. But, when I was last involved regularly in discussions on such subjects with insider types, we have a problem in that newer tests tend to have lower ceilings. So, we are having problems in this area.
Also, the first test that led to IQ testing was not intended to measure intelligence. It was intended to measure school readiness for rural French kids who often lacked birth certificates, so you could not rely on age.
Also, IQ tests are heavily culturally dependent. This fact casts a lot of doubt on the very idea of general intelligence. The people with the right background know the right answers. This can be attributed to socialization.
This is not at all some kind of settled question, last I checked.
posted by Michele in California at 2:13 PM on April 12, 2017
So, people who do recognize faces and think you are terribly impaired are basically prejudiced.
Testing is a tool. It can serve a qualified professional well in their efforts to assess. But, when I was last involved regularly in discussions on such subjects with insider types, we have a problem in that newer tests tend to have lower ceilings. So, we are having problems in this area.
Also, the first test that led to IQ testing was not intended to measure intelligence. It was intended to measure school readiness for rural French kids who often lacked birth certificates, so you could not rely on age.
Also, IQ tests are heavily culturally dependent. This fact casts a lot of doubt on the very idea of general intelligence. The people with the right background know the right answers. This can be attributed to socialization.
This is not at all some kind of settled question, last I checked.
posted by Michele in California at 2:13 PM on April 12, 2017
« Older How does copyright permission work in the UK? | I took a new job at my company. Now, how do I take... Newer »
This thread is closed to new comments.
- Most people, I think validly, recognise that IQ measures something. It has extremely good psychometric properties. The test-retest validity is high, especially for a psychological measure -- what I mean by that is that in general, people will have similar IQ scores if they take a properly curated and designed test at two different times, even if those times are separated by years. It's hard to see how this could happen if it wasn't measuring something that was a somewhat stable trait.
- Also, most people acknowledge that IQ is predictive of various things in our society that seem to matter to people, like health outcomes, success (measured in terms of money / happiness / SES), etc. It is very hard to disentangle what is cause and what is effect here, because higher IQs are also demonstrably caused by growing up in better conditions and those also relate to better health outcomes and success -- so there's a lot of argument about what is driving what. Also, it may be that IQ predicts some of these because we use IQ tests (or analogues, like SATs) as a way of deciding who gets accepted to what. So what those relationships mean is very fraught and very debated. But people agree that these relationships exist.
- Everyone agrees that most IQ tests you find online are crap. Unless it is administered by a properly trained person, and the test itself has been properly normed, it doesn't have the above properties and you will get eye rolls. Similar eye rolls occur when someone feels the need to declaim about how awesome their IQ is, or even tell anyone.
- People recognise that IQ tests have ceilings and floors which means they are pretty useless outside of a range of +- three standard deviations or so. And even within that range, they are good at reliably reflecting whatever it is they measure, but the crux of the issue is knowing what that is.
- Building on that, here's an issue: even if IQ measures something, is it something meaningful that we should care about? My sense is that most everyone in the field strongly would resist the implication that IQ means anything about how worthy you are as a person, about what rights or opportunities you should have, or even means much about your "genetic endowment." Probably genes play some role, but IQ is demonstrably shaped by environmental factors, including epigenetics and things like chemical exposures etc. Also, given the social fraughtness of this topic, my experience is that wading into that territory is something many people are hesitant to do at all. My own personal sense is that our society over-values IQ because our society values things that can be measured, and IQ can be measured. I'm sure if we were to manage to develop a test of, say, charisma that had similarly strong psychometric properties to IQ, then it would suddenly become A Thing in the way IQ is. That wouldn't mean that charisma is suddenly more important than it was, but nor does it mean it's useless.
- The Flynn effect is well-known and it has interesting implications for what exactly IQ is measuring and how (and in what way) it is susceptible to environmental influences. People study and discuss this in a nuanced way.
I could probably list more, and like I said I don't study this directly, but I've read much of the literature on it. I'd be interested to what extent others' perceptions differ from mine but this is my general sense.
posted by forza at 12:12 AM on April 11, 2017 [73 favorites]