The Situationist

Posts Tagged ‘IAT’

Tierney’s Skepticism at the New York Times

Posted by Jerry Kang on November 19, 2008

Recently, John Tierney who writes a Science column in the New York Times has shown great skepticism about the concept of implicit bias, how it might be measured (through the Implicit Association Test), and whether it predicts real-world behavior. See, e.g.,  Findings column (Nov. 17, 2008).    I write to make provide praise, critique, and cultural commentary.

First, praise.  I praise Tierney’s skepticism, which is fundamental to critical inquiry generally and good science especially.  Serious, critical inquiry is why most of us got into academics, and it’s why you the reader are reading this blog.

Second, critique.  But skepticism should not be one-sided.  Tierney’s columns suggest that one side is just asking for good, skeptical science, whereas the other side is pushing along a politically correct agenda recklessly.  That is hardly fair and balanced.  To take one example, Tierney gives prominent weight to Prof. Phil Tetlocks’ criticisms of the implicit bias research.   But let’s probe further.  In an article by Tetlock and Prof. Gregory Mitchell (UVA Law) attacking the science, the authors suggest that one of the reasons that Whites may perform worse on the Black-White IAT is because of a phenomenon called “stereotype threat”.  They write that Whites “react to the identity threat posed by the IAT by choking under stress–and performing even worse on the IAT, thus confirming the researchers’ original stereotype of them.” 67 Ohio St. L.J. 1023, 1079 (2006).

For this “choke under threat” explanation, Tetlock and Mitchell cite a single study.  Moreover, they do not turn their powerful skepticism against this body of work, launched by Prof. Claude Steele at Stanford, which explains why negative stereotypes can depress test performance.  This body of work, if taken as seriously as Tetlock and Mitchell do in a throw-away line, challenges the use of standardized examinations in university admissions.  But I doubt that that’s what Tetlock and Mitchell would call for, as a matter of policy.  So why not be methodologically pure and go after the stereotype threat work with equal vigor and skepticism?  Instead, they deploy “stereotype threat” science without raising an eyebrow, since it fits their arsenal of critique of the “implicit bias” science.

The general point is that it’s facile to think that one side has the scientific purists — just seeking good data and good science, and the other side has the political hacks.  And self-serving reasoning no doubt infects us all, on both sides.  This is why we should trust long-run scientific equilibrium and be skeptical of both aggressive claims and their backlashes.

Third, cultural commentary.  The readers’ comments to the Tierney articles are fascinating because they largely give no deference to scientific expertise.  From the large N of 1, those who have taken the IAT conclude that the test must be nonsense and raise myriad confounds (without bothering to read the FAQs that explain how stimuli are randomized, etc.)  If geneticists were debating the meaning of some expressed sequence tags or if astrophysicists were debating new evidence of dark matter, I wonder if readers would bother to chime in aggressively with their views.  “I have plenty of genes, and that view about inheritability is nonsense!”  “I’ve seen stars, and if I can’t see ‘em they must not exist!”

I suggest that we feel so personally connected to race and to gender (most of the comments focus on race) and are so personally invested in not being biased that we feel compelled toward such participation.  Again, if some “coffee increases likelihood of ulcers” study came out, would people write in:  “I drink coffee, and I don’t have an ulcer!!!”  I don’t think so.   What does that say about our current cultural moment?  Perhaps it reveals a sort of intellectual prejudice­-a proclivity not to take race research seriously, as nothing more than personal opinion, regardless of the scientific and statistical bona fides.

* * *

Look, science always involves conflict.  And in the long run, there’s no reason to think that this controversy won’t be resolved through the traditional scientific method and reach a long-run equilibrium consensus.  But getting there has already been rocky and will continue to be.   Maybe the implicit bias work, which is far more extensive than just the implicit association test (IAT), will turn out to be nothing more than “intelligent design”–just ideology (in that case religious) wrapped up in pseudo-science.  Or, and I think this is far more likely, it will be another inconvenient truth that is established, as global warming ultimately was:  We are not as colorblind as we hope to be, and on the margins, implicit associations in our brain alter our behavior in ways that we would rather they not.  Certainly the balance of peer-reviewed studies in number and quality point in that direction.

In the end, time truly will tell.  The real question is which side will maintain its scientific integrity when the results come in.

Full disclosure:  I’m a co-author of Mahzarin Banaji, whose work is discussed in Tierney’s pieces.  You can read my implicit bias work at:

http://jerrykang.net/Research/Race

Posted in Implicit Associations, Law, Naive Cynicism, Social Psychology | Tagged: , , | 1 Comment »

Do You Implicitly Prefer Markets or Regulation?

Posted by The Situationist Staff on October 23, 2008

To find out, you can take the “Policy IAT 1.0″ (a roughly 15-minute task), click here.

For those of you on IBM-compatible computers, you can now access our new version, “Policy IAT 2.0,”  by clicking here.

Posted in Ideology, Implicit Associations | Tagged: , , , , | Leave a Comment »

The Cognitive Costs of Interracial Interactions

Posted by The Situationist Staff on August 25, 2008

Ann Conkle has an article, titled “Investigating Interracial Interactions,” (in the current APS Observer) summarizing Jennifer Richeson‘s presentation at the APS 20th Annual Convention at which Richeson described her remarkable research on interracial interactions.

* * *

[Richeson] . . . presented her recent findings in the During an encounter between people of different races, if one or both parties are worried about the possibility of expressing or being thought to express prejudice, they may experience some level of anxiety or self-consciousness and may even induce physiological responses to stress, like an increased heart rate and constriction of blood vessels. These reactions can also be cognitively costly. After being primed with a racial situation, like discussing racial profiling with someone of a different race, study participants perform worse on the classic Stroop task. And those with higher bias perform even more poorly. Richeson believes that because they are aware of possible bias and dealing with physiological arousal, individuals must actively self-regulate during interracial interactions. This self-regulation uses cognitive resources and leads to depletion in other areas.

Richeson conducted two further studies to investigate these interactions. [More here.] . . .

In order to investigate what occurs in actual interactions, Richeson and collaborator Nicole Shelton, Princeton University, recruited black and white participants to take a race IAT and then engage in a 10-minute conversation about racial issues. White participants engaged in a conversation with another white participant or with a black participant. In same-race interactions, those with higher bias IAT scores were rated lower by their counterparts on likability. But, ironically, in interracial interactions, black participants liked higher-biased white participants more than lower-biased white participants and found them to be more engaged in the interaction. It seems that higher-biased individuals are aware that they could seem biased and so they self-regulate and in a sense turn on the charm to ensure that the interaction goes smoothly.

So, as Richeson said, the “intra-personal costs may come with inter-personal benefits.” Although it may seem like a positive thing that interracial interactions appear to go well, even when bias is involved, it should not overshadow the fact that interracial interaction is cognitively and physiologically draining. This could cause individuals, especially in those who are most biased, to avoid interracial interactions when possible, a situation which leads to less interracial interaction and undermines the interracial ideals to which our society aspires.

* * *

The whole article is here. For some related Situationist posts, see “Guilt and Racial Prejudice,” “Perceptions of Racial Divide,” and “Banaji & Greenwald on Edge – Part IV.” To review all of the previous Situationist posts discussing implicit associations click on the “Implicit Associations” category in the right margin or, for a list of such posts, click here.

Posted in Choice Myth, Emotions, Ideology, Implicit Associations, Social Psychology, Uncategorized | Tagged: , , , , , | 1 Comment »

The Interior Situation of Suicide

Posted by The Situationist Staff on July 27, 2008

The Boston Globe Sunday Magazine included an article by Peter Bebergal, titled “On the Edge.” (The teaser reads as follows: “Can a test reveal if a person has a subconscious desire to kill himself? Peter Bebergal, who lost a brother to suicide, goes inside Mass. General, where Harvard researchers are trying to find out.”) Here are a few excerpts.

* * *

Four years after my brother’s death, Harvard researchers at MGH are experimenting with a test they think could help clinicians determine just that. It focuses on a patient’s subconscious thoughts, and if it can be perfected, these researchers say it could give hospitals more of a legal basis for admitting suicidal patients.

* * *

This missing piece in the suicidal puzzle is what prompted the innovative research study now in its final phase at MGH. The study, led by Dr. Matthew Nock, an associate professor in the psychology department at Harvard University, is called the Suicide Implicit Association Test. It’s a variation of the Implicit Association Test, or IAT, which was invented by Anthony Greenwald at the University of Washington and “co-developed” by [Situationist Contributor] Dr. Mahzarin Banaji, now a psychology professor at Harvard who works a few floors above Nock on campus. The premise is that test takers, by associating positive and negative words with certain images (or words) – for example, connecting the word “wonderful” with a grouping that contains the word “good” and a picture of a EuropeanAmerican – reveal their unconscious, or implicit, thoughts. The critical factor in the test is not the associations themselves, but the relative speed at which those connections are made. (If you’re curious, take a sample IAT test online at implicit.harvard.edu/implicit/.)

The IAT itself is not new – it was created in 1998 – and has been used to evaluate unconscious bias against African-Americans, Arabs, fat people, and Judaism. But critics question whether the test is actually practical, and up until now no one has tried to apply it to suicide prevention. As part of his training, Nock worked extensively with adolescent self-injurers – self-injury, such as cutting and burning, is an important coping method for those who engage in it, though they are often unlikely to acknowledge it. Nock thought that the IAT could serve as a behavioral measure of who is a self-injurer and whether such a person was in danger of continuing the behavior, even after treatment. In their first major study, Nock and Banaji asserted that the IAT could be adapted to show who was inclined to be self-injurious and who was not. And more important, they said, the test could reveal who was in danger of future self-injury.

The next step, Nock realized, was to use the test to determine, from a person’s implicit thoughts, whether someone who had prior suicidal behavior was likely to continue to be suicidal. It would give doctors a third component, along with self-reporting and clinician reporting, and result in a more complete picture of a patient. Nock doesn’t assume that a test like the IAT would be 100 percent accurate, but he believes it would have predictive ability. “It is not a lie detector,” he says. “But in an ideal situation, a clinician who is struggling with a decision to admit a potentially suicidal patient to the hospital, or with an equally difficult decision to discharge a patient from the hospital following a potentially lethal suicide attempt, the IAT could provide additional information about whether the clinician should admit or keep that patient in the hospital.”

Over two years, researchers at MGH asked patients who had attempted suicide if they would be willing to participate in the test. About two-thirds of them agreed (some 200 patients) – even though some had tried killing themselves just hours before – and after answering a battery of questions about their thoughts, sat with a laptop and took the IAT.

During one test, a person was shown two sets of words on a screen, one in the upper left corner, one in the upper right. A single word then appeared in the center, and the test taker was asked to indicate with a keystroke the corner containing the word that connected to the center word. The corner sets were drawn from two groups of words (one group was “escape” and “stay,” and another was “me” and “not me”). In one version, the sets were “escape/not me” and “stay/me,” and the series of words that appeared in the center included, among others, “quit,” “persist,” “myself,” and “them.” The correct answers called for “quit” to be associated with the side that had “escape,” for “myself” to be matched with the side that had “me,” and so forth. In theory, a delay in answering on “quit,” even if the person got it right, could reveal that he was associating the idea of “quit” with the idea of himself. The word sets varied depending on the test, and bias could emerge in a positive or negative way. For example, if the sets were “escape/me” and “stay/not me” and a person hesitated in correctly matching “myself” to the side with “me,” it could reveal that he was associating himself with the idea of “stay.”

For about the next five months, Nock and his research team at Harvard will analyze all the data collected from MGH. If they think their findings show promise, they will follow up and run their experiment again to see if it yields similar results. If it does, they may seek to implement the test at an area hospital. For now, following up with patients will be pivotal in assessing the test’s effectiveness. Tragically, though, the only way researchers will know for sure whether the test can predict behavior is if a key number of patients attempt suicide again.

* * *

Nock says it’s still too early to tell how well the test will predict someone’s likelihood of engaging in suicidal behavior. But he says the hope is that the IAT will be able to record subtle distinctions between those who are at risk and those who aren’t by measuring how “positively or negatively people value the option of suicide as a potential response to their intolerable distress.

* * *

We recommend the entire article, which you can link to here. For a collection of Situationist posts about implicit associations, click here.

Posted in Implicit Associations, Life | Tagged: , , , , , | 1 Comment »

Banaji & Greenwald on Edge – Part III

Posted by The Situationist Staff on March 19, 2008

Mahzarin Banaji & Anthony GreenwaldThere is a great video interview of Tony Greewald and Situationist contributor Mahzarin Banaji on Edge. We’re posting parts of the transcript in several bite-sized installments. Part I is here; Part II is here.

* * *

GREENWALD: The race IAT was critical. It is the version of the IAT that got the widest attention, and it is the one we would most often demonstrate in lectures. That is another of the IAT’s virtues: you can give a group demonstration and show a room full of people all at the same time that they have this shared difficulty in giving the same response to Category A and to Category B, but if you the switch the sides of the two categories, it becomes very easy.

We, and also many of our colleagues, have been very motivated by the discovery that the IAT reveals these associations with race effect in our own heads. The effect does not show up in everyone, but in perhaps 70 to 75 percent of those who have tried the test. Some are upset by this observation. They are concerned that the test is, in effect, accusing them of being prejudiced. That reaction has led us to take pains to distinguish what this test measures from what is ordinarily meant by prejudice.

Prejudice is ordinarily understood as a state of mind that leads to intentional discriminatory behavior. People who have no intent to discriminate and no dislike of a racial group might think that their IAT result was characterizing them as prejudiced. We have repeatedly said, no, that it is not what the result means. In describing what the IAT measures, we have been careful to use phrases such as ‘implicit attitude,’ ‘implicit preference,’ and ‘automatic preference,’ while avoiding the unqualified word ‘prejudice’. Incidentally, if we did describe the test as measuring prejudice, then we would be accusing ourselves of being prejudiced.

BANAJI: One of the great insights this test has provided for me is the ability to look at what might be evolutionarily “old ways” in which we tend to behave when left to our own devices. Our social preferences must have some roots in our early social groups and interactions. From having evolved in a world where people on the other side of the river were either people you killed or people who would kill you. We now live in a world where we have to outsource to those same people on the other side of the river! We have to be friends enough with them to understand their culture so that we can get them to do things with us and to think about our common fate.

It is one thing to say, “The law says you should do this or that.” It is quite another thing to say, Well, if we are really smart, and if we are really the adaptive creatures that we are, we are going to look at the ways in which we behave and see that they are not necessarily to our advantage. And as we learn we will change. And we will change in all of the ways in which we are going to need to. Like the work we need to do and will do to solve our environmental problems. Or our health problems. Eating too much is a problem because our bodies evolved in a world where food was far less abundant that it has come to be (for many people in the world). We cope by thinking about calorie intake and output in new ways. I think one of the tests of human intelligence will be whether we can take insights that are inconvenient truths about our minds, turn them around, and use such knowledge to create a better society – by which I mean, one that is line with our consciously chosen aspirations, rather than one we are being driven toward out of ignorance of who we are and our past.

GREENWALD: The IAT provides a useful window into some otherwise difficult-to-detect contents of our minds. In some cases, we find things we did not know were there. It may be “an inconvenient truth” that what’s there is not what we thought was there or want to be there. But I think it is generally something we can come to grips with.

BANAJI: We have to believe that everyone is to blame, or rather everybody is responsible. It is not just the media. It is not just your parents. Instead, we find to be attractive those metaphors that come from air borne pollutants. What our minds acquire comes from the stuff that is hanging around in the atmosphere. It is in the water. It is in the air. When that is the case, you cannot hold individual people responsible. You can hold larger units and larger groups of people responsible in the same way that we do in order to solve the problem of environmental damage.

To come back to the test, I think an intuitive way to understand what it does is to imagine working with a deck of playing cards. If I ask you to sort the cards into two piles—red cards on one side; black cards on the other—you should be able to do so relatively easily. Let’s say I measure the time it takes you to put the red cards to the right and the black cards to the left and use a stopwatch to time you. Then I say, “All right, I am going to time you again. This time put the hearts and clubs to one side, and the spades and diamonds to the other”, and I start the stopwatch again. . We would all understand intuitively that the second sorting should take longer than the first. And the difference in time is indicative of the cognitive ease of the first relative to the second.

Like that, the IAT is trying to capture the difference between two sortings. When we encounter two things that have not been paired together very much in our experience, it takes a little longer to put them together because they are strangers to each other, making the task difficult. Working from this assumption, the IAT requests pairings of say Obama and good and Hillary and good and looks at the relative speed of judgment and the error rates. What is nifty is that you can replace Obama and Hillary with anything you want. If you are interested in looking at your preference for Coca-Cola versus Pepsi-Cola, you can adapt the test to do that.

* * *

For a eight-minute review of IAT research by Scientific American (including clips of both Mahzarin Banaji and Brian Nosek, who each explain how the biases can operate outside of, or contrary to, one’s intentions), watch the video below.

Posted in Implicit Associations, Video | Tagged: , , , | Leave a Comment »

Banaji & Greenwald on Edge – Part II

Posted by The Situationist Staff on March 14, 2008

There is a great video interview of Tony Greewald and Situationist contributor Mahzarin Banaji on Edge. We’ll post parts of the transcript in severalMahzarin Banaji & Anthony Greenwald bite-sized installments. Part I is here.

* * *

GREENWALD: At various points in my career I have worked on some very interesting questions, such as: How is voluntary action controlled? What is the self? How does the mind function unconsciously? On that last one, I surprised myself by discovering that visual stimuli, flashed so briefly as to remain consciously unseen, can influence behavior. In starting this research I had no idea that was true. In doing the research, I found that we can learn new things by creating new methods. I had created new methods for studying subliminal perception, and shortly after than Mahzarin and I started working on developing methods to investigate things going on outside of awareness that could influence social behavior.

We were very influenced by cognitive psychologists’ work on implicit cognition—which is the label for knowledge expressed implicitly in behavior even when the person performing the behavior was unaware of having the knowledge. We were looking for ways to study the implicit aspects of social behavior. An important moment came in 1995, when we performed an experiment that we had conceived a few years earlier and written into one brief paragraph of a proposal to the National Science Foundation.

In that experiment subjects gave a response on a computer keyboard with the index finger of the right hand to words that named pleasant things and to names of flowers. With left hand they were to respond to another two categories—words that named unpleasant things and insect names. This was a very easy task. Then we made one minor change: We switched hands for the flower and insect names. Now subjects had to give the same response to pleasant words and insect names and a different response to unpleasant words and flower names. Immediately the task became hugely difficult. The slowing on a response-by-response basis was on the order of 300 milliseconds, which was a magnitude of impact nobody could have expected. We certainly did not expect it.

I was the first subject in this experiment. When I experienced this slowing I found to my surprise that I could not overcome it—repeating the task did not make me faster. If I tried to go faster, I just started making errors when I was trying to give the same response to flower names and unpleasant words. This was a mind-opener.

About a month later, I modified the task by replacing the flowers and insects with the names of famous White and Black people. My thought was, if this works for flowers and insects, maybe we could use the same task to measure something that we had not yet begun to call implicit race attitudes. To my dismay, but also with some excitement, I discovered that the names of famous Black people were functioning much like the names of insects. I had a difficult time responding rapidly when I had to give the same response to famous Black people and to pleasant words. Shortly thereafter, I persuaded Mahzarin to start using this task in her lab at Yale. One of the students whom she brought along on this work was [Situationist contributor] Brian Nosek, who ever since has been a very important collaborator in this research.

BANAJI: What is remarkable about this test, which is called the Implicit Association Test—the IAT—is that it allows you to be a subject in your own experiment. Most scientists do not have the remarkable experience of being the object of study in their own research.

If you are a physicist, you cannot be the material object that you are studying. If you are a psychologist, you cannot be the subject because you know the hypothesis. But when you study the unconscious aspects of the mind, you can! You can have the experience of having your hand NOT do what your mind is willing it to do. That may be part of the appeal of the test.

How does the test work? Well, I intend to associate white with good as quickly as I associate black with good—that is my conscious goal—but I fail to do that. [See brief video above for quick summary of how IAT works.] It has to be fascinating to anybody to discover that they cannot do something simple that they will themselves to do. When we failed our own tests, we decided that this was of course for the scientific journals, but also for wider access. When in 1998 we decided that this was probably worth sharing with a wider audience than our own students, we asked, What would Galileo do, and we put it on the web.

We were stunned that within the first month with no advertising on our part, just media coverage — we had 40,000 completed tests. We had clearly struck a chord or a nerve (depending on the participant’s response!).

The test is unusual in that it provokes a reaction of surprise, even astonishment. It is both a tool to understand what goes on invisibly in our minds but it is also a catalyst for insight. And once it has suggested to us and we may contain the views of multitudes, we can ask “Am I leading my life the way I want to?” It is a unique test in that regard.

Posted in Implicit Associations, Social Psychology, Video | Tagged: , , , | 2 Comments »

 
Follow

Get every new post delivered to your Inbox.

Join 846 other followers

%d bloggers like this: