Unreliable Psychology Research (2015 Oct)

by Barry A. Liebling

There is troubling news for a large number of government policy makers and mainstream academics. They are in the habit of boasting that their views, preferences, and recommendations that relate to psychology and social science are not merely personal opinions. Instead, their world view is legitimized by “scientific research” that has been scrutinized by “experts in the field” and published in peer review journals.

The problem is that many of the studies that are regarded as evidence for the cultural mainstream’s outlook are unreliable. Recently Science published follow-up research conducted by academic psychologists where 100 experimental and correlational studies that appeared in academic journals were replicated. The investigators took pains to use the same methodologies as the original authors. The results were that most of the replications produced weaker results than what was reported in the original paper. Put plainly – a lot of the research findings simply did not hold up. http://www.sciencemag.org/content/349/6251/aac4716

Why is it so difficult to replicate psychological science research? It is because the phenomena investigators often choose to study is inherently weak and is overshadowed (in the real world) by an individual’s deliberate actions. Most psychology experimenters take pains to assure that the participating subjects are not informed of the purpose of the research or what is really being studied. Experimenters know that participants who are aware of what the study is really about may decide on their own either to sabotage the results or (equally damaging) to attempt to please the investigator and exaggerate their responses.

Note that if you are studying phenomena that is robust it is easy to demonstrate it with an experiment. You do not need a large number of participants or arcane statistics to show convincingly that your results are genuine. Imagine a simple experiment where people with staphylococcus infections are either given oral antibiotics or sugar pills. The medicine condition should yield a strong result that will immediately be apparent. The magnitude of the effect would be so large that a conventional statistical test would reveal that the probability that the results are attributable to chance is tiny.

Or, let’s return to psychology. In the classic 1951 Asch experiment participants were asked to judge the length of lines. When other people in the room (confederates of the experimenter) voiced judgements that were obviously incorrect a proportion of the participants (more than one-third, fewer than one-half) caved in, conformed to the group, and slavishly gave incorrect answers. Imagine running the Asch experiment again where participants are given a full explanation of the original experiment. Few, if any, participants would conform. Fancy statistics would not be necessary to evaluate the results.

Why are academic psychologists focusing their attention on weak phenomena? The bias can be traced to flawed philosophical assumptions that underlie most psychological research. In much of mainstream academic psychology humans are conceptualized as essentially machines that are controlled by external circumstances and by their hard-wired, genetically determined dispositions. To an investigator with this viewpoint the job for the researcher is to find situations where subtle (unrecognized by the participant) changes in the environment “nudge” people to behave a certain way or to have a particular attitude. A large proportion of academic research is interpreted by its fans as “confirming evidence” that people are foolish, generally make irrational decisions, and probably cannot be trusted to manage their own lives well.

And it is noteworthy that this view of humanity is embraced by elite members of the political left. Since common people are incompetent to run their own lives, wiser experts should decide for them what they should do. Of course, leftist “intellectuals” always nominate themselves as the candidates most qualified to direct the affairs of others. And, significantly, government agencies that fund most academic research are dominated by professionals who are enthusiastic supporters of leftist politics.

The alternative, correct conceptualization, is that humans have free will which, when activated, supercedes thoughtless reflexes and gives humans agency to decide what to do and how to do it. From this perspective, much of academic psychology research consists of parlor tricks. Investigators concoct artificial situations designed to show that humans are shoved around by forces they do not notice and cannot resist. To a mainstream researcher human agency is noise in the system to be minimized and discounted.

Of course there are hundreds of research studies published each year, and not all of them are tainted. But the charge applies to a sizable chunk of them. The investigators that selected 100 studies to replicate did not do so at random. They focused their gaze on articles they judged to be important. And the esteemed New York Times identified three research efforts as noteworthy. What do these studies reveal about the state of academic psychology? http://www.nytimes.com/2015/08/28/science/many-social-science-findings-not-as-strong-as-claimed-study-says.html?_r=0

In the first study the authors argue that a belief in determinism (as contrasted with free will) encourages people to cheat. The investigators had participants read either several paragraphs that touted free will or some text that pushed determinism. They performed two experiments and defined cheating as either failing to press a bar that conceals the correct answer on a test or taking too much money for a task where they were supposed to pay themselves for correct answers. According to the authors in neither experiment did they ever measure or document any individual participant cheating. Yet this did not deter them from claiming they demonstrated a general principle of human behavior. And the investigators were careful to separate themselves from people who understand and appreciate free will. They explicitly state that they doubt free will exists and,”it is possible that free will is an illusion that nevertheless offers some functionality.” So much for how well they understand the essential importance of human agency.

The second study purports to show that human judgement is so fickle that it is easily distorted by routine (apparently not related) experiences. The authors had participants draw either long lines or short lines. Afterwards those who drew the longer lines (according to this study) reflexively “distanced themselves” from what ever the experimenter showed them. They were less affected by an embarrassing story, less distressed by violent media, had lower estimates of the calories in fattening foods, and reported weaker emotional attachments to their family members. There is no doubt that if an experimenter is clever enough and sets up the situation just right he can get at least some people to respond this way. But the shrewd experimenter is acting exactly like a stage magician who puts on a show – taking pains to rig the situation so that subjects are “nudged” in the desired direction. It is no wonder that this research is difficult to replicate.

The third study focused on women who were either fertile or infertile. The investigators asked women to rate the attractiveness of photographs of men who were described as either attached or not attached. Fertile women tended to prefer men who were not attached (because they would be more willing sex partners according to the authors), while unfertile women favored the pictures of attached men. Again, even if the effect being reported is real, it is small and easily overpowered by the deliberate judgments of women in the real world. But this study, like many psychology experiments, regards rational, volitional actions as less important and less interesting than behavior that is influenced by subconscious stimuli.

Some observers have noted the failure to replicate psychology findings and have concluded that what is needed is more careful research – with larger numbers of participants to assure statistical reliability and more attention to detail in conducting experiments. But the more important lesson is that psychologists need to reconsider their premises. A valid science of psychology must be based on what defines human beings – the property of free will and the ability to act rationally.

*** See other entries at AlertMindPublishing.com in “Monthly Columns.” ***

Comments are closed.