Wednesday, October 23, 2013

Stereotype threat

Putting it bluntly, Stereotype threat is an invented process to explain why blacks do poorly on IQ tests.  If blacks know that they are expected to do badly they allegedly get all anxious and do even worse than they otherwise would.  But shouldn't the knowledge that they are expected to do badly energize them and make them try harder -- just to prove the stereotype wrong?  I would have thought so but I am not a Leftist.

I have had a bit of a laugh at the theory before (e.g. here) and also see here

The theory has also been used to explain away the fact that women on average do badly on mathematical tasks (those nervous ladies!) and there has recently been some interesting work suggesting that the theory is wrong in that field too.  Steve Sailer summarizes:

"Although the social sciences are considered a bastion of progressivism, it's remarkable how few data-driven ideas they generate in support of their ideology. We can get a feel for this by noting how rare are the "exceptions to the rule" studies that become immensely popular due to bolstering the dominant worldview, such as Hart & Risley's finding that black people don't talk enough and Claude Steele's little study of Stereotype Threat in which he induces black students at Stanford to score lower on a low stakes test of his devising than their high stakes SAT scores would predict. (I wrote about Stereotype Threat in in 2004, suggesting it's not hard to get across the message to black or female students that the professor wants them to not exert themselves fully on this meaningless test. That you can "prime" groups of people to work less hard on an unimportant test does not prove that you know how to make them score higher on an important test.)

Lately, the evidence has been mounting that the existence of Stereotype Threat is quite dependent upon the file drawer function: studies finding its existence are quickly published while studies not finding its existence are in much less demand. A recent article:

An Examination of Stereotype Threat Effects on Girls' Mathematics Performance

By Colleen M. Ganley et al.

... Conclusion

Taken together, the findings from published research, unpublished articles, and the present studies reveal inconsistency in the effects of stereotype threat on girls’ mathematics performance. The discrepancy in results from published and unpublished studies suggests publication bias, which may create an inaccurate picture of the phenomenon. A recent review suggests that this publication bias may also be an issue in the literature on stereotype threat in adult women (Stoet & Geary, 2012). Overall, these results raise the possibility that stereotype threat may not be the cause of gender differences in mathematics performance prior to college. Although we feel that more nuanced research needs to be done to truly understand whether stereotype threat impacts girls’ mathematics performance, we also believe that too much focus on this one explanation may deter researchers from investigating other key factors that may be involved in gender differences in mathematics performance. For example, there are a number of factors (e.g., mathematics anxiety, mathematics interest, spatial skills; see Ceci & Williams, 2010) that have been shown to be consistently related to mathematics performance and mathematics-and science-related career choices and may warrant more research attention than does stereotype threat."


Monday, October 21, 2013

Kees Jan can't

Kees-Jan Kan, a young Dutchman, has recently rediscovered one of the most basic facts in IQ testing: That it's easiest to detect IQ differences if the people you are studying (Ss) have a common background.  So if the Ss are all in the same class at school, for instance, a vocabulary test (finding out how many hard words they know) will give you a quick and easy way to sort them out.  And you will find that the guys who know lots of words are also good at a whole range of puzzles, even mathematical ones. 

So a common background optimizes your chances of assessing IQ accurately. And to be a bit technical, vocab loads highly on 'g' (the general factor in intelligence), meaning that, where it can be used, it is a powerful predictor of other abilities.  Vocab is however convenient rather than essential in IQ measurement.  Tests designed for use among people who do not have a common background (such as the Raven PMs) don't use it but still work perfectly well.

On those basic facts, KJK has erected an elaborate theory, which comes to the conclusions that IQ is mostly cultural, with a genetic component much smaller that is generally thought.  And it is the cultural part which is hereditary.

To arrive at that, KJK goes via the concept of the "cultural load" of each IQ question -- which he assesses by looking at how often a question has to be altered when you are adminstering it to a new and different population.  And he finds that by removing (statistically) the influence of cultural load, all other correlations are much reduced.

When we look more closely at his data, however (e.g. Table 3.1 in KJK's doctoral dissertation) we find that only two out of 11 question types have a high cultural load:  Vocab and general knowledge.  And the cultural dependency of those two question types has been obvious to everyone since the year dot.

What is interesting however is that the remaining 9 question types have low to negligible cultural load.  In other words, we could remove the vocab and knowledge subtests from the overall test and still have a robust test.  So my conclusion is that what KJK should have done from the beginning is to remove those two flawed item types from his calculations altogether.  Once you do that all his exciting findings melt away.  His findings rely on items that he himself knows to be flawed.

There is a summary of KJK's dissertation at  The Unscientific American -- JR