Face Reading

[Crafty_Dog]

---------------------------------------
August 5, 2002, The New Yorker
ANNALS OF PSYCHOLOGY
The Naked Face
Can you read people's thoughts just by looking at them?
http://www.gladwell.com/2002/2002_08_05_a_face.htm

1.

Some years ago, John Yarbrough was working patrol for the Los Angeles County Sheriff's Department. It was about two in the morning. He and his partner were in the Willowbrook section of South Central Los Angeles, and they pulled over a sports car. "Dark, nighttime, average stop," Yarbrough recalls. "Patrol for me was like going hunting. At that time of night in the area I was working, there was a lot of criminal activity, and hardly anyone had a driver's license. Almost everyone had something intoxicating in the car. We stopped drunk drivers all the time. You're hunting for guns or lots of dope, or suspects wanted for major things. You look at someone and you get an instinctive reaction. And the longer you've been working the stronger that instinctive reaction is."

Yarbrough was driving, and in a two-man patrol car the procedure is for the driver to make the approach and the officer on the passenger side to provide backup. He opened the door and stepped out onto the street, walking toward the vehicle with his weapon drawn. Suddenly, a man jumped out of the passenger side and pointed a gun directly at him. The two of them froze, separated by no more than a few yards. "There was a tree behind him, to his right," Yarbrough recalls. "He was about seventeen. He had the gun in his right hand. He was on the curb side. I was on the other side, facing him. It was just a matter of who was going to shoot first. I remember it clear as day. But for some reason I didn't shoot him." Yarbrough is an ex-marine with close-cropped graying hair and a small mustache, and he speaks in measured tones. "Is he a danger? Sure. He's standing there with a gun, and what person in his right mind does that facing a uniformed armed policeman? If you looked at it logically, I should have shot him. But logic had nothing to do with it. Something just didn't feel right. It was a gut reaction not to shoot? a hunch that at that exact moment he was not an imminent threat to me." So Yarbrough stopped, and, sure enough, so did the kid. He pointed a gun at an armed policeman on a dark street in South Central L.A., and then backed down.

Yarbrough retired last year from the sheriff's department after almost thirty years, sixteen of which were in homicide. He now lives in western Arizona, in a small, immaculate house overlooking the Colorado River, with pictures of John Wayne, Charles Bronson, Clint Eastwood, and Dale Earnhardt on the wall. He has a policeman's watchfulness: while he listens to you, his eyes alight on your face, and then they follow your hands, if you move them, and the areas to your immediate left and right? and then back again, in a steady cycle. He grew up in an affluent household in the San Fernando Valley, the son of two doctors, and he is intensely analytical: he is the sort to take a problem and break it down, working it over slowly and patiently in his mind, and the incident in Willowbrook is one of those problems. Policemen shoot people who point guns directly at them at two in the morning. But something he saw held him back, something that ninety-nine people out of a hundred wouldn't have seen.

Many years later, Yarbrough met with a team of psychologists who were conducting training sessions for law enforcement. They sat beside him in a darkened room and showed him a series of videotapes of people who were either lying or telling the truth. He had to say who was doing what. One tape showed people talking about their views on the death penalty and on smoking in public. Another featured a series of nurses who were all talking about a nature film they were supposedly watching, even though some of them were actually watching grisly documentary footage about burn victims and amputees. It may sound as if the tests should have been easy, because we all think we can tell whether someone is lying. But these were not the obvious fibs of a child, or the prevarications of people whose habits and tendencies we know well. These were strangers who were motivated to deceive, and the task of spotting the liars turns out to be fantastically difficult. There is just too much information?words, intonation, gestures, eyes, mouth?and it is impossible to know how the various cues should be weighted, or how to put them all together, and in any case it's all happening so quickly that you can't even follow what you think you ought to follow. The tests have been given to policemen, customs officers, judges, trial lawyers, and psychotherapists, as well as to officers from the F.B.I., the C.I.A., the D.E.A., and the Bureau of Alcohol, Tobacco, and Firearms? people one would have thought would be good at spotting lies. On average, they score fifty per cent, which is to say that they would have done just as well if they hadn't watched the tapes at all and just guessed. But every now and again? roughly one time in a thousand?someone scores off the charts. A Texas Ranger named David Maxwell did extremely well, for example, as did an ex-A.T.F. agent named J.J. Newberry, a few therapists, an arbitrator, a vice cop? and John Yarbrough, which suggests that what happened in Willowbrook may have been more than a fluke or a lucky guess. Something in our faces signals whether we're going to shoot, say, or whether we're lying about the film we just saw. Most of us aren't very good at spotting it. But a handful of people are virtuosos. What do they see that we miss?

2.

All of us, a thousand times a day, read faces. When someone says "I love you," we look into that person's eyes to judge his or her sincerity. When we meet someone new, we often pick up on subtle signals, so that, even though he or she may have talked in a normal and friendly manner, afterward we say, "I don't think he liked me," or "I don't think she's very happy." We easily parse complex distinctions in facial expression. If you saw me grinning, for example, with my eyes twinkling, you'd say I was amused. But that's not the only way we interpret a smile. If you saw me nod and smile exaggeratedly, with the corners of my lips tightened, you would take it that I had been teased and was responding sarcastically. If I made eye contact with someone, gave a small smile and then looked down and averted my gaze, you would think I was flirting. If I followed a remark with an abrupt smile and then nodded, or tilted my head sideways, you might conclude that I had just said something a little harsh, and wanted to take the edge off it. You wouldn't need to hear anything I was saying in order to reach these conclusions. The face is such an extraordinarily efficient instrument of communication that there must be rules that govern the way we interpret facial expressions. But what are those rules? And are they the same for everyone?

In the nineteen-sixties, a young San Francisco psychologist named Paul Ekman began to study facial expression, and he discovered that no one knew the answers to those questions. Ekman went to see Margaret Mead, climbing the stairs to her tower office at the American Museum of Natural History. He had an idea. What if he travelled around the world to find out whether people from different cultures agreed on the meaning of different facial expressions? Mead, he recalls, "looked at me as if I were crazy." Like most social scientists of her day, she believed that expression was culturally determined? that we simply used our faces according to a set of learned social conventions. Charles Darwin had discussed the face in his later writings; in his 1872 book, "The Expression of the Emotions in Man and Animals," he argued that all mammals show emotion reliably in their faces. But in the nineteen-sixties academic psychologists were more interested in motivation and cognition than in emotion or its expression. Ekman was undaunted; he began travelling to places like Japan, Brazil, and Argentina, carrying photographs of men and women making a variety of distinctive faces. Everywhere he went, people agreed on what those expressions meant. But what if people in the developed world had all picked up the same cultural rules from watching the same movies and television shows? So Ekman set out again, this time making his way through the jungles of Papua New Guinea, to the most remote villages, and he found that the tribesmen there had no problem interpreting the expressions, either. This may not sound like much of a breakthrough. But in the scientific climate of the time it was a revelation. Ekman had established that expressions were the universal products of evolution. There were fundamental lessons to be learned from the face, if you knew where to look.

Paul Ekman is now in his sixties. He is clean-shaven, with closely set eyes and thick, prominent eyebrows, and although he is of medium build, he seems much larger than he is: there is something stubborn and substantial in his demeanor. He grew up in Newark, the son of a pediatrician, and entered the University of Chicago at fifteen. He speaks deliberately: before he laughs, he pauses slightly, as if waiting for permission. He is the sort to make lists, and number his arguments. His academic writing has an orderly logic to it; by the end of an Ekman essay, each stray objection and problem has been gathered up and catalogued. In the mid-sixties, Ekman set up a lab in a ramshackle Victorian house at the University of California at San Francisco, where he holds a professorship. If the face was part of a physiological system, he reasoned, the system could be learned. He set out to teach himself. He treated the face as an adventurer would a foreign land, exploring its every crevice and contour. He assembled a videotape library of people's facial expressions, which soon filled three rooms in his lab, and studied them to the point where he could look at a face and pick up a flicker of emotion that might last no more than a fraction of a second. Ekman created the lying tests. He filmed the nurses talking about the movie they were watching and the movie they weren't watching. Working with Maureen O'Sullivan, a psychologist from the University of San Francisco, and other colleagues, he located people who had a reputation for being uncannily perceptive, and put them to the test, and that's how Yarbrough and the other high-scorers were identified. O'Sullivan and Ekman call this study of gifted face readers the Diogenes Project, after the Greek philosopher of antiquity who used to wander around Athens with a lantern, peering into people's faces as he searched for an honest man. Ekman has taken the most vaporous of sensations? the hunch you have about someone else? and sought to give them definition. Most of us don't trust our hunches, because we don't know where they came from. We think they can't be explained. But what if they can?

3.

Paul Ekman got his start in the face-reading business because of a man named Silvan Tomkins, and Silvan Tomkins may have been the best face reader there ever was. Tomkins was from Philadelphia, the son of a dentist from Russia. He was short, and slightly thick around the middle, with a wild mane of white hair and huge black plastic-rimmed glasses. He taught psychology at Princeton and Rutgers, and was the author of "Affect, Imagery, Consciousness," a four-volume work so dense that its readers were evenly divided between those who understood it and thought it was brilliant and those who did not understand it and thought it was brilliant. He was a legendary talker. At the end of a cocktail party, fifteen people would sit, rapt, at Tomkins's feet, and someone would say, "One more question!" and they would all sit there for another hour and a half, as Tomkins held forth on, say, comic books, a television sitcom, the biology of emotion, his problem with Kant, and his enthusiasm for the latest fad diets, all enfolded into one extended riff. During the Depression, in the midst of his doctoral studies at Harvard, he worked as a handicapper for a horse-racing syndicate, and was so successful that he lived lavishly on Manhattan's Upper East Side. At the track, where he sat in the stands for hours, staring at the horses through binoculars, he was known as the Professor. "He had a system for predicting how a horse would do based on what horse was on either side of him, based on their emotional relationship," Ekman said. If a male horse, for instance, had lost to a mare in his first or second year, he would be ruined if he went to the gate with a mare next to him in the lineup. (Or something like that? no one really knew for certain.) Tomkins felt that emotion was the code to life, and that with enough attention to particulars the code could be cracked. He thought this about the horses, and, more important, he thought this about the human face.

Tomkins, it was said, could walk into a post office, go over to the "Wanted" posters, and, just by looking at mug shots, tell you what crimes the various fugitives had committed. "He would watch the show "To Tell the Truth,' and without fault he could always pick the person who was lying and who his confederates were," his son, Mark, recalls. "He actually wrote the producer at one point to say it was too easy, and the man invited him to come to New York, go backstage, and show his stuff." Virginia Demos, who teaches psychology at Harvard, recalls having long conversations with Tomkins. "We would sit and talk on the phone, and he would turn the sound down as Jesse Jackson was talking to Michael Dukakis, at the Democratic National Convention. And he would read the faces and give his predictions on what would happen. It was profound."

Ekman's most memorable encounter with Tomkins took place in the late sixties. Ekman had just tracked down a hundred thousand feet of film that had been shot by the virologist Carleton Gajdusek in the remote jungles of Papua New Guinea. Some of the footage was of a tribe called the South Fore, who were a peaceful and friendly people. The rest was of the Kukukuku, who were hostile and murderous and who had a homosexual ritual where pre-adolescent boys were required to serve as courtesans for the male elders of the tribe. Ekman was still working on the problem of whether human facial expressions were universal, and the Gajdusek film was invaluable. For six months, Ekman and his collaborator, Wallace Friesen, sorted through the footage. They cut extraneous scenes, focussing just on closeups of the faces of the tribesmen, and when the editing was finished Ekman called in Tomkins.

[Crafty_Dog]

The two men, prot?g? and mentor, sat at the back of the room, as faces flickered across the screen. Ekman had told Tomkins nothing about the tribes involved; all identifying context had been edited out. Tomkins looked on intently, peering through his glasses. At the end, he went up to the screen and pointed to the faces of the South Fore. "These are a sweet, gentle people, very indulgent, very peaceful," he said. Then he pointed to the faces of the Kukukuku. "This other group is violent, and there is lots of evidence to suggest homosexuality." Even today, a third of a century later, Ekman cannot get over what Tomkins did. "My God! I vividly remember saying, "Silvan, how on earth are you doing that?' " Ekman recalls. "And he went up to the screen and, while we played the film backward, in slow motion, he pointed out the particular bulges and wrinkles in the face that he was using to make his judgment. That's when I realized, "I've got to unpack the face.' It was a gold mine of information that everyone had ignored. This guy could see it, and if he could see it, maybe everyone else could, too."

Ekman and Friesen decided that they needed to create a taxonomy of facial expressions, so day after day they sat across from each other and began to make every conceivable face they could. Soon, though, they realized that their efforts weren't enough. "I met an anthropologist, Wade Seaford, told him what I was doing, and he said, 'Do you have this movement?'" ?and here Ekman contracted what's called the triangularis, which is the muscle that depresses the corners of the lips, forming an arc of distaste? "and it wasn't in my system, because I had never seen it before. I had built a system not on what the face can do but on what I had seen. I was devastated. So I came back and said, 'I've got to learn the anatomy.' " Friesen and Ekman then combed through medical textbooks that outlined each of the facial muscles, and identified every distinct muscular movement that the face could make. There were forty-three such movements. Ekman and Friesen called them "action units." Then they sat across from each other again, and began manipulating each action unit in turn, first locating the muscle in their mind and then concentrating on isolating it, watching each other closely as they did, checking their movements in a mirror, making notes of how the wrinkle patterns on their faces would change with each muscle movement, and videotaping the movement for their records. On the few occasions when they couldn't make a particular movement, they went next door to the U.C.S.F. anatomy department, where a surgeon they knew would stick them with a needle and electrically stimulate the recalcitrant muscle. "That wasn't pleasant at all," Ekman recalls. When each of those action units had been mastered, Ekman and Friesen began working action units in combination, layering one movement on top of another. The entire process took seven years. "There are three hundred combinations of two muscles," Ekman says. "If you add in a third, you get over four thousand. We took it up to five muscles, which is over ten thousand visible facial configurations." Most of those ten thousand facial expressions don't mean anything, of course. They are the kind of nonsense faces that children make. But, by working through each action-unit combination, Ekman and Friesen identified about three thousand that did seem to mean something, until they had catalogued the essential repertoire of human emotion.

4.

On a recent afternoon, Ekman sat in his office at U.C.S.F., in what is known as the Human Interaction Laboratory, a standard academic's lair of books and files, with photographs of his two heroes, Tomkins and Darwin, on the wall. He leaned forward slightly, placing his hands on his knees, and began running through the action-unit configurations he had learned so long ago. "Everybody can do action unit four," he began. He lowered his brow, using his depressor glabellae, depressor supercilli, and corrugator. "Almost everyone can do A.U. nine." He wrinkled his nose, using his levator labii superioris, alaeque nasi. "Everybody can do five." He contracted his levator palpebrae superioris, raising his upper eyelid.

I was trying to follow along with him, and he looked up at me. "You've got a very good five," he said generously. "The more deeply set your eyes are, the harder it is to see the five. Then there's seven." He squinted. "Twelve." He flashed a smile, activating the zygomatic major. The inner parts of his eyebrows shot up. "That's A.U. one? distress, anguish." Then he used his frontalis, pars lateralis, to raise the outer half of his eyebrows. "That's A.U. two. It's also very hard, but it's worthless. It's not part of anything except Kabuki theatre. Twenty-three is one of my favorites. It's the narrowing of the red margin of the lips. Very reliable anger sign. It's very hard to do voluntarily." He narrowed his lips. "Moving one ear at a time is still the hardest thing to do. I have to really concentrate. It takes everything I've got." He laughed. "This is something my daughter always wanted me to do for her friends. Here we go." He wiggled his left ear, then his right ear. Ekman does not appear to have a particularly expressive face. He has the demeanor of a psychoanalyst, watchful and impassive, and his ability to transform his face so easily and quickly was astonishing. "There is one I can't do," he went on. "It's A.U. thirty-nine. Fortunately, one of my postdocs can do it. A.U. thirty-eight is dilating the nostrils. Thirty-nine is the opposite. It's the muscle that pulls them down." He shook his head and looked at me again. "Oooh! You've got a fantastic thirty-nine. That's one of the best I've ever seen. It's genetic. There should be other members of your family who have this heretofore unknown talent. You've got it, you've got it." He laughed again. "You're in a position to flash it at people. See, you should try that in a singles bar!"

Ekman then began to layer one action unit on top of another, in order to compose the more complicated facial expressions that we generally recognize as emotions. Happiness, for instance, is essentially A.U. six and twelve? contracting the muscles that raise the cheek (orbicularis oculi, pars orbitalis) in combination with the zygomatic major, which pulls up the corners of the lips. Fear is A.U. one, two and four, or, more fully, one, two, four, five, and twenty, with or without action units twenty-five, twenty-six, or twenty-seven. That is: the inner brow raiser (frontalis, pars medialis) plus the outer brow raiser (frontalis, pars lateralis) plus the brow-lowering depressor supercilli plus the levator palpebrae superioris (which raises the upper lid), plus the risorius (which stretches the lips), the parting of the lips (depressor labii), and the masseter (which drops the jaw). Disgust? That's mostly A.U. nine, the wrinkling of the nose (levator labii superioris, alaeque nasi), but it can sometimes be ten, and in either case may be combined with A.U. fifteen or sixteen or seventeen.

Ekman and Friesen ultimately assembled all these combinations?and the rules for reading and interpreting them? into the Facial Action Coding System, or FACS, and wrote them up in a five-hundred-page binder. It is a strangely riveting document, full of details like the possible movements of the lips (elongate, de-elongate, narrow, widen, flatten, protrude, tighten and stretch); the four different changes of the skin between the eyes and the cheeks (bulges, bags, pouches, and lines); or the critical distinctions between infraorbital furrows and the nasolabial furrow. Researchers have employed the system to study everything from schizophrenia to heart disease; it has even been put to use by computer animators at Pixar ("Toy Story"), andat DreamWorks ("Shrek"). FACS takes weeks to master in its entirety, and only five hundred people around the world have been certified to use it in research. But for those who have, the experience of looking at others is forever changed. They learn to read the face the way that people like John Yarbrough did intuitively. Ekman compares it to the way you start to hear a symphony once you've been trained to read music: an experience that used to wash over you becomes particularized and nuanced.

Ekman recalls the first time he saw Bill Clinton, during the 1992 Democratic primaries. "I was watching his facial expressions, and I said to my wife, 'This is Peck's Bad Boy,' " Ekman says. "This is a guy who wants to be caught with his hand in the cookie jar, and have us love him for it anyway. There was this expression that's one of his favorites. It's that hand-in-the-cookie-jar, love-me-Mommy-because-I'm-a-rascal look. It's A.U. twelve, fifteen, seventeen, and twenty-four, with an eye roll." Ekman paused, then reconstructed that particular sequence of expressions on his face. He contracted his zygomatic major, A.U. twelve, in a classic smile, then tugged the corners of his lips down with his triangularis, A.U. fifteen. He flexed the mentalis, A.U. seventeen, which raises the chin, slightly pressed his lips together in A.U. twenty-four, and finally rolled his eyes?and it was as if Slick Willie himself were suddenly in the room. "I knew someone who was on his communications staff. So I contacted him. I said, 'Look, Clinton's got this way of rolling his eyes along with a certain expression, and what it conveys is "I'm a bad boy." I don't think it's a good thing. I could teach him how not to do that in two to three hours.' And he said, 'Well, we can't take the risk that he's known to be seeing an expert on lying.' I think it's a great tragedy, because . . ." Ekman's voice trailed off. It was clear that he rather liked Clinton, and that he wanted Clinton's trademark expression to have been no more than a meaningless facial tic. Ekman shrugged. "Unfortunately, I guess, he needed to get caught?and he got caught."

5.

Early in his career, Paul Ekman filmed forty psychiatric patients, including a woman named Mary, a forty-two-year-old housewife. She had attempted suicide three times, and survived the last attempt?an overdose of pills?only because someone found her in time and rushed her to the hospital. Her children had left home and her husband was inattentive, and she was depressed. When she first went to the hospital, she simply sat and cried, but she seemed to respond well to therapy. After three weeks, she told her doctor that she was feeling much better and wanted a weekend pass to see her family. The doctor agreed, but just before Mary was to leave the hospital she confessed that the real reason she wanted to go on weekend leave was so that she could make another suicide attempt. Several years later, a group of young psychiatrists asked Ekman how they could tell when suicidal patients were lying. He didn't know, but, remembering Mary, he decided to try to find out. If the face really was a reliable guide to emotion, shouldn't he be able to look back on the film and tell that she was lying? Ekman and Friesen began to analyze the film for clues. They played it over and over for dozens of hours, examining in slow motion every gesture and expression. Finally, they saw it. As Mary's doctor asked her about her plans for the future, a look of utter despair flashed across her face so quickly that it was almost imperceptible.

Ekman calls that kind of fleeting look a "microexpression," and one cannot understand why John Yarbrough did what he did on that night in South Central without also understanding the particular role and significance of microexpressions. Many facial expressions can be made voluntarily. If I' m trying to look stern as I give you a tongue-lashing, I'll have no difficulty doing so, and you' ll have no difficulty interpreting my glare. But our faces are also governed by a separate, involuntary system. We know this because stroke victims who suffer damage to what is known as the pyramidal neural system will laugh at a joke, but they cannot smile if you ask them to. At the same time, patients with damage to another part of the brain have the opposite problem. They can smile on demand, but if you tell them a joke they can't laugh. Similarly, few of us can voluntarily do A.U. one, the sadness sign. (A notable exception, Ekman points out, is Woody Allen, who uses his frontalis, pars medialis, to create his trademark look of comic distress.) Yet we raise our inner eyebrows all the time, without thinking, when we are unhappy. Watch a baby just as he or she starts to cry, and you'll often see the frontalis, pars medialis, shoot up, as if it were on a string.

Perhaps the most famous involuntary expression is what Ekman has dubbed the Duchenne smile, in honor of the nineteenth-century French neurologist Guillaume Duchenne, who first attempted to document the workings of the muscles of the face with the camera. If I ask you to smile, you' ll flex your zygomatic major. By contrast, if you smile spontaneously, in the presence of genuine emotion, you' ll not only flex your zygomatic but also tighten the orbicularis oculi, pars orbitalis, which is the muscle that encircles the eye. It is almost impossible to tighten the orbicularis oculi, pars lateralis, on demand, and it is equally difficult to stop it from tightening when we smile at something genuinely pleasurable. This kind of smile "does not obey the will," Duchenne wrote. "Its absence unmasks the false friend." When we experience a basic emotion, a corresponding message is automatically sent to the muscles of the face. That message may linger on the face for just a fraction of a second, or be detectable only if you attached electrical sensors to the face, but It's always there. Silvan Tomkins once began a lecture by bellowing, "The face is like the penis!" and this is what he meant?that the face has, to a large extent, a mind of its own. This doesn't mean we have no control over our faces. We can use our voluntary muscular system to try to suppress those involuntary responses. But, often, some little part of that suppressed emotion?the sense that I' m really unhappy, even though I deny it?leaks out. Our voluntary expressive system is the way we intentionally signal our emotions. But our involuntary expressive system is in many ways even more important: it is the way we have been equipped by evolution to signal our authentic feelings.

"You must have had the experience where somebody comments on your expression and you didn't know you were making it,"Ekman says. "Somebody tells you, "What are you getting upset about?' "Why are you smirking?' You can hear your voice, but you can't see your face. If we knew what was on our face, we would be better at concealing it. But that wouldn't necessarily be a good thing. Imagine if there were a switch that all of us had, to turn off the expressions on our face at will. If babies had that switch, we wouldn't know what they were feeling. They' d be in trouble. You could make an argument, if you wanted to, that the system evolved so that parents would be able to take care of kids. Or imagine if you were married to someone with a switch? It would be impossible. I don't think mating and infatuation and friendships and closeness would occur if our faces didn't work that way."

Ekman slipped a tape taken from the O.J. Simpson trial into the VCR. It was of Kato Kaelin, Simpson's shaggy-haired house guest, being examined by Marcia Clark, one of the prosecutors in the case. Kaelin sits in the witness box, with his trademark vacant look. Clark asks a hostile question. Kaelin leans forward and answers softly. "Did you see that?" Ekman asked me. I saw nothing, just Kato being Kato? harmless and passive. Ekman stopped the tape, rewound it, and played it back in slow motion. On the screen, Kaelin moved forward to answer the question, and in that fraction of a second his face was utterly transformed. His nose wrinkled, as he flexed his levator labii superioris, alaeque nasi. His teeth were bared, his brows lowered. "It was almost totally A.U. nine," Ekman said. "It's disgust, with anger there as well, and the clue to that is that when your eyebrows go down, typically your eyes are not as open as they are here. The raised upper eyelid is a component of anger, not disgust. It's very quick." Ekman stopped the tape and played it again, peering at the screen. "You know, he looks like a snarling dog."

Ekman said that there was nothing magical about his ability to pick up an emotion that fleeting. It was simply a matter of practice. "I could show you forty examples, and you could pick it up. I have a training tape, and people love it. They start it, and they can't see any of these expressions. Thirty-five minutes later, they can see them all. What that says is that this is an accessible skill."

Ekman showed another clip, this one from a press conference given by Kim Philby in 1955. Philby had not yet been revealed as a Soviet spy, but two of his colleagues, Donald Maclean and Guy Burgess, had just defected to the Soviet Union. Philby is wearing a dark suit and a white shirt. His hair is straight and parted to the left. His face has the hauteur of privilege.

"Mr. Philby," he is asked. "Mr. Macmillan, the foreign secretary, said there was no evidence that you were the so-called third man who allegedly tipped off Burgess and Maclean. Are you satisfied with that clearance that he gave you?"

Philby answers confidently, in the plummy tones of the English upper class. "Yes, I am."

"Well, if there was a third man, were you in fact the third man?"

"No," Philby says, just as forcefully. "I was not."

Ekman rewound the tape, and replayed it in slow motion. "Look at this," he said, pointing to the screen. "Twice, after being asked serious questions about whether he's committed treason, he's going to smirk. He looks like the cat who ate the canary." The expression was too brief to see normally. But at quarter speed it was painted on his face?the lips pressed together in a look of pure smugness. "He's enjoying himself, isn't he?" Ekman went on. "I call this?duping delight? the thrill you get from fooling other people." Ekman started the VCR up again. "There's another thing he does." On the screen, Philby was answering another question. "In the second place, the Burgess-Maclean affair has raised issues of great"? he pauses? "delicacy." Ekman went back to the pause, and froze the tape. "Here it is,"he said. "A very subtle microexpression of distress or unhappiness. It's only in the eyebrows? in fact, just in one eyebrow." Sure enough, Philby's right inner eyebrow was raised in an unmistakable A.U. one. "It's very brief," Ekman said. "He's not doing it voluntarily. And it totally contradicts all his confidence and assertiveness. It comes when he's talking about Burgess and Maclean, whom he had tipped off. It's a hot spot that suggests, 'You shouldn't trust what you hear.' "

[Crafty_Dog]

A decade ago, Ekman joined forces with J. J. Newberry?the ex-A.T.F. agent who is one of the high-scorers in the Diogenes Project? to put together a program for educating law-enforcement officials around the world in the techniques of interviewing and lie detection. In recent months, they have flown to Washington, D.C., to assist the C.I.A. and the F.B.I. in counter-terrorism training. At the same time, the Defense Advanced Research Projects Agency (DARPA) has asked Ekman and his former student Mark Frank, now at Rutgers, to develop experimental scenarios for studying deception that would be relevant to counter-terrorism. The objective is to teach people to look for discrepancies between what is said and what is signalled?to pick up on the difference between Philby's crisp denials and his fleeting anguish. It's a completely different approach from the shouting cop we see on TV and in the movies, who threatens the suspect and sweeps all of the papers and coffee cups off the battered desk. The Hollywood interrogation is an exercise in intimidation, and its point is to force the suspect to tell you what you need to know. It does not take much to see the limitations of this strategy. It depends for its success on the co?peration of the suspect?when, of course, the suspect's involuntary communication may be just as critical. And it privileges the voice over the face, when the voice and the face are equally significant channels in the same system.

Ekman received his most memorable lesson in this truth when he and Friesen first began working on expressions of anger and distress. "It was weeks before one of us finally admitted feeling terrible after a session where we' d been making one of those faces all day," Friesen says. "Then the other realized that he'd been feeling poorly, too, so we began to keep track." They then went back and began monitoring their body during particular facial movements. "Say you do A.U. one, raising the inner eyebrows, and six, raising the cheeks, and fifteen, the lowering of the corner of the lips," Ekman said, and then did all three. "What we discovered is that that expression alone is sufficient to create marked changes in the autonomic nervous system. When this first occurred, we were stunned. We weren't expecting this at all. And it happened to both of us. We felt terrible . What we were generating was sadness, anguish. And when I lower my brows, which is four, and raise the upper eyelid, which is five, and narrow the eyelids, which is seven, and press the lips together, which is twenty-four, I' m generating anger. My heartbeat will go up ten to twelve beats. My hands will get hot. As I do it, I can't disconnect from the system. It's very unpleasant, very unpleasant."

Ekman, Friesen, and another colleague, Robert Levenson, who teaches at Berkeley, published a study of this effect in Science. They monitored the bodily indices of anger, sadness, and fear?heart rate and body temperature?in two groups. The first group was instructed to remember and relive a particularly stressful experience. The other was told to simply produce a series of facial movements, as instructed by Ekman? to "assume the position," as they say in acting class. The second group, the people who were pretending, showed the same physiological responses as the first. A few years later, a German team of psychologists published a similar study. They had a group of subjects look at cartoons, either while holding a pen between their lips?an action that made it impossible to contract either of the two major smiling muscles, the risorius and the zygomatic major? or while holding a pen clenched between their teeth, which had the opposite effect and forced them to smile. The people with the pen between their teeth found the cartoons much funnier. Emotion doesn't just go from the inside out. It goes from the outside in. What's more, neither the subjects "assuming the position" nor the people with pens in their teeth knew they were making expressions of emotion. In the facial-feedback system, an expression you do not even know that you have can create an emotion you did not choose to feel.

It is hard to talk to anyone who knows FACS without this point coming up again and again. Face-reading depends not just on seeing facial expressions but also on taking them seriously. One reason most of us?like the TV cop? do not closely attend to the face is that we view its evidence as secondary, as an adjunct to what we believe to be real emotion. But there's nothing secondary about the face, and surely this realization is what set John Yarbrough apart on the night that the boy in the sports car came at him with a gun. It's not just that he saw a microexpression that the rest of us would have missed. It's that he took what he saw so seriously that he was able to overcome every self-protective instinct in his body, and hold his fire.

6.

Yarbrough has a friend in the L.A. County Sheriff's Department, Sergeant Bob Harms, who works in narcotics in Palmdale. Harms is a member of the Diogenes Project as well, but the two men come across very differently. Harms is bigger than Yarbrough, taller and broader in the chest, with soft brown eyes and dark, thick hair. Yarbrough is restoring a Corvette and wears Rush Limbaugh ties, and he says that if he hadn't been a cop he would have liked to stay in the Marines. Harms came out of college wanting to be a commercial artist; now he plans to open a bed-and-breakfast in Vermont with his wife when he retires. On the day we met, Harms was wearing a pair of jean shorts and a short-sleeved patterned shirt. His badge was hidden inside his shirt. He takes notes not on a yellow legal pad, which he considers unnecessarily intimidating to witnesses, but on a powder-blue one. "I always get teased because I'm the touchy-feely one," Harms said. "John Yarbrough is very analytical. He thinks before he speaks. There is a lot going on inside his head. He's constantly thinking four or five steps ahead, then formulating whatever his answers are going to be. That's not how I do my interviews. I have a conversation. It's not "Where were you on Friday night?' Because that's the way we normally communicate. I never say, "I'm Sergeant Harms.' I always start by saying, "I'm Bob Harms, and I'm here to talk to you about your case,' and the first thing I do is smile."

The sensation of talking to the two men, however, is surprisingly similar. Normal conversation is like a game of tennis: you talk and I listen, you listen and I talk, and we feel scrutinized by our conversational partner only when the ball is in our court. But Yarbrough and Harms never stop watching, even when they're doing the talking. Yarbrough would comment on my conversational style, noting where I held my hands as I talked, or how long I would wait out a lull in the conversation. At one point, he stood up and soundlessly moved to the door? which he could have seen only in his peripheral vision?opening it just before a visitor rang the doorbell. Harms gave the impression that he was deeply interested in me. It wasn't empathy. It was a kind of powerful curiosity. "I remember once, when I was in prison custody, I used to shake prisoners' hands," Harms said. "The deputies thought I was crazy. But I wanted to see what happened, because that's what these men are starving for, some dignity and respect."

Some of what sets Yarbrough and Harms and the other face readers apart is no doubt innate. But the fact that people can be taught so easily to recognize microexpressions, and can learn FACS, suggests that we all have at least the potential capacity for this kind of perception. Among those who do very well at face-reading, tellingly, are some aphasics, such as stroke victims who have lost the ability to understand language. Collaborating with Ekman on a paper that was recently published in Nature, the psychologist Nancy Etcoff, of Massachusetts General Hospital, described how a group of aphasics trounced a group of undergraduates at M.I.T. on the nurses tape. Robbed of the power to understand speech, the stroke victims had apparently been forced to become far more sensitive to the information written on people's faces. "They are compensating for the loss in one channel through these other channels," Etcoff says. "We could hypothesize that there is some kind of rewiring in the brain, but I don't think we need that explanation. They simply exercise these skills much more than we do." Ekman has also done work showing that some abused children are particularly good at reading faces as well: like the aphasics in the study, they developed "interpretive strategies"?in their case, so they could predict the behavior of their volatile parents.

What appears to be a kind of magical, effortless intuition about faces, then, may not really be effortless and magical at all. This kind of intuition is a product of desire and effort. Silvan Tomkins took a sabbatical from Princeton when his son Mark was born, and stayed in his house on the Jersey Shore, staring into his son's face, long and hard, picking up the patterns of emotion?the cycles of interest, joy, sadness, and anger?that flash across an infant's face in the first few months of life. He taught himself the logic of the furrows and the wrinkles and the creases, the subtle differences between the pre-smile and the pre-cry face. Later, he put together a library of thousands of photographs of human faces, in every conceivable expression. He developed something called the Picture Arrangement Test, which was his version of the Rorschach blot: a patient would look at a series of pictures and be asked to arrange them in a sequence and then tell a story based on what he saw. The psychologist was supposed to interpret the meaning of the story, but Tomkins would watch a videotape of the patient with the sound off, and by studying the expressions on the patient's face teach himself to predict what the story was. Face-reading, for those who have mastered it, becomes a kind of compulsion; it becomes hard to be satisfied with the level and quality of information that most of us glean from normal social encounters. "Whenever we get together," Harms says of spending time with other face readers, "we debrief each other. We're constantly talking about cases, or some of these videotapes of Ekman's, and we say, "I missed that, did you get that?' Maybe there's an emotion attached there. We're always trying to place things, and replaying interviews in our head."

This is surely why the majority of us don't do well at reading faces: we feel no need to make that extra effort. People fail at the nurses tape, Ekman says, because they end up just listening to the words. That's why, when Tomkins was starting out in his quest to understand the face, he always watched television with the sound turned off. "We are such creatures of language that what we hear takes precedence over what is supposed to be our primary channel of communication, the visual channel," he once said. "Even though the visual channel provides such enormous information, the fact is that the voice pre?mpts the individual's attention, so that he cannot really see the face while he listens." We prefer that way of dealing with the world because it does not challenge the ordinary boundaries of human relationships. Ekman, in one of his essays, writes of what he learned from the legendary sociologist Erving Goffman. Goffman said that part of what it means to be civilized is not to "steal" information that is not freely given to us. When someone picks his nose or cleans his ears, out of unthinking habit, we look away. Ekman writes that for Goffman the spoken word is "the acknowledged information, the information for which the person who states it is willing to take responsibility," and he goes on:

When the secretary who is miserable about a fight with her husband the previous night answers, "Just fine," when her boss asks, "How are you this morning?"?that false message may be the one relevant to the boss's interactions with her. It tells him that she is going to do her job. The true message?that she is miserable?he may not care to know about at all as long as she does not intend to let it impair her job performance.

What would the boss gain by reading the subtle and contradictory microexpressions on his secretary's face? It would be an invasion of her privacy and an act of disrespect. More than that, it would entail an obligation. He would be obliged to do something, or say something, or feel something that might otherwise be avoided entirely. To see what is intended to be hidden, or, at least, what is usually missed, opens up a world of uncomfortable possibilities. This is the hard part of being a face reader. People like that have more faith in their hunches than the rest of us do. But faith is not certainty. Sometimes, on a routine traffic stop late at night, you end up finding out that your hunch was right. But at other times you'll never know. And you can't even explain it properly, because what can you say? You did something the rest of us would never have done, based on something the rest of us would never have seen.

"I was working in West Hollywood once, in the nineteen-eighties," Harms said. "I was with a partner, Scott. I was driving. I had just recently come off the prostitution team, and we spotted a man in drag. He was on Sunset, and I didn't recognize him. At that time, Sunset was normally for females. So it was kind of odd. It was a cold night in January. There was an all-night restaurant on Sunset called Ben Franks, so I asked my partner to roll down the window and ask the guy if he was going to Ben Franks? just to get a reaction. And the guy immediately keys on Scott, and he's got an overcoat on, and he's all bundled up, and he starts walking over to the car. It had been raining so much that the sewers in West Hollywood had backed up, and one of the manhole covers had been cordoned off because it was pumping out water. The guy comes over to the squad car, and he's walking right through that. He's fixated on Scott. So we asked him what he was doing. He says, "I was out for a walk.' And then he says, "I have something to show you.'"

Later, after the incident was over, Harms and his partner learned that the man had been going around Hollywood making serious threats, that he was unstable and had just attempted suicide, that he was in all likelihood about to erupt. A departmental inquiry into the incident would affirm that Harms and his partner had been in danger: the man was armed with a makeshift flamethrower, and what he had in mind, evidently, was to turn the inside of the squad car into an inferno. But at the time all Harms had was a hunch, a sense from the situation and the man's behavior and what he glimpsed inside the man's coat and on the man's face? something that was the opposite of whatever John Yarbrough saw in the face of the boy in Willowbrook. Harms pulled out his gun and shot the man through the open window. "Scott looked at me and was, like, "What did you do?' because he didn't perceive any danger," Harms said. "But I did."
Copyright 2002, Malcolm Gladwell

[C-Kumu Dog]

Wow, that is a fascinating article.? I googled the term FACS and was able to find http://face-and-emotion.com which has some good reading material as well as as facial diagrams.

[Crafty_Dog]

AI-Generated Face Swapping Video Technology Rampant in China
By Shawn Lin April 22, 2022 Updated: April 22, 2022biggersmaller Print



Do you ever worry about identity theft? Well, now there’s “face swapping,” the latest AI technology that could get us all into a whole lot more trouble. It’s not only fast, but can also mix the spurious with the genuine. The barriers to setting up this technology are getting lower and lower, and a fake video of you doing or saying something can be synthesized in China for a small fee.

According to an April 14 report by Shanghai-based media The Paper, it is now very easy to generate and make dynamic videos of human faces on the computer. Last year in Hefei City, Anhui Province, the police found that several people were able to create a dynamic video from static photos. The characters produced through this simulation not only nod and move their heads, but also make expressions like winking, opening the mouth, frowning, etc. The effect is extremely realistic.

Chinese police found more than a dozen gigabytes of face data on one group’s computers, with face photos and ID photos stored in different folders according to different categories. “Photos of the front and back of the ID card, photos of people holding ID cards, self-portraits, etc., are called a set.” A complete set of photos is called “materials.” Due to the ease and simplicity of production, the cost of a video is only 2 to 10 yuan (about $0.30 to $1.55).

The eight people involved in this case were said to not be highly educated, some had not even finished high school. They downloaded the software according to online tutorials and spent months teaching themselves to use it.

In addition to obtaining other people’s photos, there are also people who buy other people’s voices and other “materials.” Only a small audio and video sample is needed to synthesize fake audio and video, which is comparable to real images and sounds.

Therefore, as long as you get data such as a person’s photo and voice recording, video and audio of this person can be generated on the computer. In other words, a video and audio of anyone may not be authentic or real.

Zhu Jun, the director of the Basic Theory Research Center of the Institute of Artificial Intelligence of Tsinghua University, said that the rapid development of deep synthesis technology has made “seeing no longer believing.”

There is even a “face changing tutorial” on YouTube. The presenter uses a photo of President Joe Biden and inserts a video of a Chinese singer. It only takes a few minutes to generate a video of President Biden singing songs in Chinese. It is difficult to determine the authenticity from the realistic way the lips move.

A popular science article in China’s Shanghai Science and Technology Museum introduces the specific method of AI face changing. The article said that in addition to facial features, factors such as age, gender, personality, and emotions will be reflected on the face. All these factors are considered parameters in the artificial intelligence neural network model. The computer obtains these parameters by learning and summarizing a large number of pictures—a process called “training.”

The article said that in order to make the pictures or videos look more realistic, AI face changing is usually realized with the help of “Generative Adversarial Networks” (GAN) and usually consist of two parts. One is the Generative Model (G for short); the other is the Discriminative Model (D for short), which discriminates the faces synthesized by the generative model. The two are trained using a large number of pictures, and they try to challenge each other. Once the discriminant model cannot judge the authenticity anymore, it means that the training is relatively mature. When the deep model is trained and matured, one can input a human face, generate various expressions on the face, and add in other pictures or videos to achieve seamless integration.

This technology was originally used for the post-production of film and television works. Later, some people used the technology to play pranks, using photos of celebrities and politicians to “face-swap” them into videos, and to synthesize speeches that didn’t happen. In addition, some pornography websites also swap faces of celebrities into photos or videos, causing them a lot of trouble.

In China, it was discovered that some people have used the technology to register mobile phone cards or deceive payment systems with fake IDs. They do this to evade or counter the Chinese authorities’ large-scale surveillance system. In China, real-name registration is required to purchase a mobile phone card, and real-name authentication is required to speak online or even buy a kitchen knife. Thus, by changing faces one can evade the supervision of the CCP.