by Barry A. Liebling
A wealthy, but unsophisticated man yearns to be accepted by the reigning aristocracy in seventeenth century France. He buys expensive clothes and hires tutors to school him in the ways of the elite. In the most memorable part of Molière’s play, The Bourgeois Gentleman, he learns to his astonishment and great pleasure that for forty years he has been “speaking in prose” without knowing it.
The scene is funny for at least two reasons. The audience knows exactly what prose is and can enjoy the display of innocence of an adult who has only recently come to understand the concept. More profound, the episode suggests that you can engage in an activity for a lifetime without appreciating what it is and how it is significant.
Recently the issue of human experimentation on the internet has been in the news. The management of Facebook conducted an experiment where it filtered the News Feeds of some of its members. In one condition users were presented with News Feeds that had more positive expressions, while in a second condition Facebook users were given News Feeds that had more negative expressions. As expected, contagion was observed. There was a (very slight) tendency for users to make more positive posts in the first condition and more negative posts in the second condition. http://www.pnas.org/content/111/24/8788.full.pdf+html?sid=48be5fa0-7f65-49eb-9cff-437344f2fc53
On cue, shocked critics lambasted Facebook for conducting an experiment on its members without first obtaining their explicit permission. Note that no false fabrications were used in the Facebook experiment, and the News Feeds in the study were a subset of what the members would have received if the research were not conducted.
In defense of Facebook, as a preemptive strike against critics, and as a means of promoting his new book, Dataclysm, Christian Rudder one of the founders of OkCupid wrote an article that described three “of the more interesting experiments OkCupid has run.” http://blog.okcupid.com/index.php/we-experiment-on-human-beings/
Before reviewing Mr Rudder’s comments let’s be clear on what constitutes a human experiment. Like the fictional Molière gentleman who was amazed when he learned he was speaking in prose, many people would be surprised to learn that nearly everyone routinely conducts experiments and is the subject of experiments. Every time you say or do something and then check to see how someone else responds you are running an experiment. Do you smile and say hello to your neighbor when you see him on the street or do quickly walk past him? You are an experimenter if you have tried both and detected a difference in your neighbor’s actions. When you want someone to confide in you do you deliberately reveal something personal about yourself? If so, have you noticed he is more likely to reciprocate? You have conducted an experiment.
Think of how common it is to be a participant in a business experiment. Some clothing stores play various types of music and check to see how it affects sales. When a restaurant offers its customers a free dessert you can be sure the proprietor is measuring how much, if any, extra revenue the eatery obtains. Appliance stores are constantly testing whether you are more likely to buy if they advertise a 20 percent discount from the marked price or, alternatively, offer no discounts but have lower base prices.
Notice that these examples are not dependent on the internet. Advances in technology have made it easier and more economical to experiment, but the basic principles do not change. Human experiments that are done online are not fundamentally different from those that are conducted in other settings (such as in person, by mail, or on the phone). Note also that most experiments do not require the explicit informed consent of the participant. Some do, of course, especially when there is the potential for harm – such as medical experiments where volunteering subjects might be given a treatment that may not be effective or given a placebo – no real treatment at all.
What are the standards for recognizing whether it is acceptable to perform an experiment on humans? The same rules apply that should be used for all interactions: Deal by mutual consent (which means that coercion is strictly prohibited) and act honestly (which rules out fraud and deliberate deception).
Let’s examine Mr Rudder’s interesting OkCupid experiments. OkCupid is one of the largest and most successful dating sites. Members fill out a questionnaire with items regarding personal habits, opinions, and attitudes. They indicate not only their own answers but also what answers they regard as acceptable in a potential match. From these data OkCupid calculates a “match percentage” indicating how compatible any two members are likely to be. OkCupid asserts that the “match percentage” is valuable information in deciding whom to contact.
In Experiment 1 OkCupid temporarily removed all the pictures from the site to see how this affected users’ propensity to contact one another. As expected, members adjusted their interactions to take advantage of the information that was available. When the pictures were put back on the site members reverted to their original habits.
In Experiment 2 investigators tested how much user ratings are influenced by pictures compared to profiles (written self-descriptions). They found – not surprising to OkCupid management – that the picture is more powerful than the words.
Experiment 3 is qualitatively different from the other two. In an effort to test how effective the “match percentage” is, OkCupid deliberately told people who were good matches that they were bad matches. Then it told people who had bad “match percentage” scores that they were good matches. The investigators found that the “match percentage” works as intended but that merely telling someone they are a good match or a bad match (regardless of the actual score) also influences how many messages users send to one another. Note that at the end of the experiment OkCupid informed the participating members what the correct “match percentage” was.
That Experiment 3 employed deception cannot be overlooked. It is not OK, and it is a corporate blunder. Even if no harm was done to members, presenting users with false information damages the credibility of OkCupid. The lesson that observers learn is that OkCupid might intentionally tell you something that is false but the company will set the record straight at a later time. The message to the wise is “withhold your trust from this company.”
Apologists for OkCupid might argue that, in the big picture, the misrepresentation of “match percentage” scores is not so important. But, in fact, “match percentage” is a vital feature that sets OkCupid apart from its competitors. The site explicitly states, “By all our internal measures, the ‘match percentage’ we calculate for users is very good at predicting relationships. It correlates with message success, conversation length, whether people actually exchange contact information.” This is not something management should fool with.
Incidently, if OkCupid management wants to test the effectiveness of its “match percentage” score it can find ways to do so without resorting to deception. For example, users could be given no information at all as to whether they are a good or bad match and OkCupid could examine how their frequency of messages relates to their actual (but masked) “match percentage” scores.
Performing human experiments – like speaking in prose – is a normal part of life. And – like composing prose – experimentation can be done well or poorly. OkCupid is right to make experimenting an integral part of its business practices, and as long as it adheres to the appropriate rules it will be OK.
*** See other entries at AlertMindPublishing.com in “Monthly Columns.” ***