Friday, March 1, 2013

Text Analytics: Smarter Than You May Realize


Have you ever gone through a trauma or personal problem of some kind, and resorted to keeping a journal in order to try to steady your emotions? Writing a daily journal entry is often recommended as a kind of therapy for helping yourself heal a wound or redress a problem. Research, however, shows that not all journal writing is equally restorative. In fact, those who use either too many or too few negative emotion words tend not to heal as fast (because either they aren’t fully acknowledging the problem or they are wallowing in it).
The fastest healing comes when journal entries are characterized not only by a moderate amount of negative emotion, but also by an alternating concern for self and others, as indicated by the use of personal pronouns (I, me, my) and third-person pronouns (he, she, they). Or, as James W. Pennebaker puts it in his book The Secret Life of Pronouns: What Our Words Say About Us (Bloomsbury USA, 2007), “[H]ealthy people say something about their own thoughts and feelings in one instance and then explore what is happening with other people before writing about themselves again.”
This is just one aspect of how our use of pronouns in everyday speech patterns reveals the nature of our egos, our personalities, and our sense of self-worth. The conclusions in Pennebaker’s book are all the more remarkable because they are based on solid, quantitative analysis of word frequency – particularly pronouns. His entire book, which is the subject of this Friday Book Share, is based on a kind of text analytics.
And while our conscious minds might come to one conclusion about how the pronouns someone uses reflect their personality, the numbers tell a completely different story. In the middle of his 2004 run for the presidency against George W. Bush, for instance, John Kerry’s advisors felt he had developed an image that was too distant and stand-offish, compared to his opponent. Faced with this problem, they began peppering his speeches with pronouns like “we” and “our” and “us,” while minimizing the use of “I” and “me” and “my.”
But this was exactly the wrong thing to do if the goal was to make Kerry seem more warm and personal. According to Pennebaker, “use of I-words is associated with being honest and personal, and when politicians use them, we-words sound cold, rigid, and emotionally distant. At the time, Kerry was already using we-words at twice the rate of Bush and I-words at half Bush’s rate. Kerry’s advisers, who were some of the smartest people in the country, failed to understand how invisible stealth words worked.”
Pronoun usage also varies by gender. While there are exceptions, of course, to this and every other statistical generalization, Pennebaker's research does show that “Males categorize their worlds by counting, naming, and organizing the objects they confront. Women, in addition to personalizing their topics, talk in a more dynamic way, focusing on how their topics change. Discussions of change require more verbs.” As evidence for this, he includes a handy table of male-female comparisons, based on quantitative word counts:
No matter how literate a writer you are, you cannot escape your own personality, and your personality is driven in large measure by your gender. Investigating the gender angle a bit further, Pennebaker did a quantitative analysis of Shakespeare’s Romeo and Juliet and compared it to Nora Ephron’s Sleepless in Seattle. He was interested in finding out whether great writers could accurately mimic the way characters of the opposite gender used words. What he found, however, was that “[b]oth Mr. Romeo and Ms. Juliet talk like men, and both Ephron’s main characters talk a lot like women.”
And lest you think this is all just a lot of pointless analysis for not much gain, the fact is that lies can often be distinguished from truth simply by paying closer attention to the types of words and phrasing used. Pennebaker devotes a lot of attention to this subject, but without going into too much detail, suffice it to say that liars tend to use shorter, less complex sentences, fewer "I" words, and (surprisingly) more positive emotion words, than truth-tellers do.
“Many forms of deception are associated with optimism and overconfidence,” according to Pennebaker. “The person trying to sell you a new rug, a new religion, or a new war often brims with the certainty of truth. Part of the effective salesperson’s approach is convincing you that once you buy the product there is absolutely no doubt that you too will be as happy and confident as the salesperson. Counter to common sense, people who are deceptive make more references to other people and rely on more positive emotion words.”
Sound like any sales pitch you’ve heard recently?
If you think you really know what people mean when they are speaking to you, you might be humbled by taking the online “i-test” provided as a kind of “final exam” for readers of this book. Simply go to www.secretlifeofpronouns.com/itest and answer the ten questions. And remember – the actual answers are all factually true, based on statistical analysis. These aren't judgment calls. They are reality.
It really shouldn't be long now before there's a smartphone app for alerting us when someone's words indicate either genuine concern or an intent to deceive.

No comments:

Post a Comment