If you wanted to design the perfect LSAT question to drive me crazy, it would look pretty close to this:
Cognitive psychologist: During frightening experiences, the body releases elevated levels of the hormone adrenaline. People tend to remember experiences better when the body has released higher levels of this hormone. Therefore, on average, people will retain the details of frightening experiences better than the details of nonfrightening experiences.
Which of the following statements, if true, would most significantly undermine the psychologist’s argument?
A. The body won’t release extra adrenaline when a hazardous situation isn’t perceived as frightening.
B. Frightening experiences are much less common than nonfrightening experiences.
C. Experiences can be so extreme that they impair normal memory retention for details.
D. The details of an experience are usually only retained when relevant to one’s goals.
E. As with frightening experiences, the body releases elevated levels of adrenaline during very pleasurable experiences.
In fact, this is a real problem (PT156 S2 Q22), rewritten slightly to avoid copyright violations. About psychology, no less!
Given its place in the “logical reasoning” (LR) section of the test, this question is designed to test one’s ability to understand the logical structure of an argument and strength of inference. The logical understanding that’s expected for the test is sometimes casually referred to as “lawgic” — basic knowledge of introductory concepts like necessary and sufficient conditions and predicates, such as “some” and “all.” A formal logic class in philosophy might help test takers perform slightly better, but is not required.
Although the LSAT writers don’t want the test to depend on serious training, this poses a potential problem when the test encroaches into the territory of scientific reasoning. Can you think logically about science without at least some technical understanding of probability and causal inference? The LSAT question above is an interesting test case. The researcher’s argument is inductive; the conclusion is probabilistic and based on statistical regularities. People may have a heuristic understanding of the argument, but heuristics can fail.
That’s why I believe the LSAT writers misfired here. They claim the correct answer is E, presumably because it sounds like the most direct challenge to the causal hypothesis the researcher is offering about frightening experiences. But I will argue that if you apply the correct formal tools to the argument, you will find that the statement in E doesn’t actually weaken the researcher’s argument. In contrast, answer choice C highlights a key challenge to the logic.
Of course, I may just be confused. If you think so, please reach out!
A (way too) detailed analysis
Let’s break the cognitive psychologist’s argument into its component premises and conclusion.
The conclusion of the psychologist’s argument is that the details of frightening experiences will tend to be remembered better than those of nonfrightening experiences. Note that this is a probabilistic statement; it doesn’t have to be the case that frightening experiences are always remembered better than nonfrightening ones.
This conclusion is supported by two premises:
Premise #1: Frightening experiences cause the elevation of adrenaline levels.
Premise #2: More adrenaline is associated with better experiential memory.
In essence, these two premises combine to form a mediated causal pathway from frightening experiences to improved memory via adrenaline:
\[ \text{Frightening Experience} \rightarrow \text{Extra Adrenaline} \rightarrow \text{Better Memory}. \]
The first premise supports the arrow on the left, and the second premise supports the arrow on the right. If we follow the chain all the way from left to right, the conclusion seems to be at least partly justified (more on this later).
However, the question asks us to consider how the argument might be undermined. Before looking at the answer choices, let’s consider a few possible weaknesses:
The conclusion makes a comparative claim about frightening versus nonfrightening experiences. Could nonfrightening experiences likewise cause physiological changes that improve memory? Perhaps these physiological changes are even stronger predictors of better memory than adrenaline is, in which case the psychologist’s conclusion would be false.
The mediating pathway above may be overly simplistic. Adrenaline secretions are a continuous variable, and the relationship between the amount of the hormone released and memory retention for details could be nonlinear. Perhaps a little bit of extra adrenaline improves memory, but too much of it impairs memory.
Being frightened could impact memory retention through mechanisms other than adrenaline levels. For example, if someone is extremely frightened, they may be too stressed and distracted to attend to details that would normally get encoded into memory. So, even if adrenaline aids memory, other factors could counteract this benefit. If the negative influence of these other factors outweighs the positive effect of adrenaline, the researcher’s conclusion is false.
Now, looking at the possible answers, we see that choice E (the officially correct answer) seems representative of the first type of weakness, and choice C (my selected answer) seems representative of the last type of weakness. It could also be consistent with the second type insofar as adrenaline is what causes memory impairment in extreme cases.
At this point, folk lawgic doesn’t seem capable of quantifying the magnitude of these two types of weakeners. Perhaps answer choice C looks worse because it relies on an implicit (albeit highly intuitive) assumption that frightening experiences are more likely than average to be “extreme,” and answer choice E looks better because it invokes adrenaline. In the LSAT universe, this is all that matters for determining which answer is the strongest. But looks can be deceiving.
Peaking under the hood
A full analysis of the problem will require some probability theory (see here for a review). I hope to persuade you that this is a worthwhile sacrifice to get a deeper understanding. Keep in mind that I am not suggesting that pre-law students actually learn the formalism of probability theory for the sake of the LSAT. This was not the approach I took when I encountered this problem on a timed practice test. Ideally, people would develop an intuitive statistical and causal understanding of these types of problems, and the test would assess this type of reasoning properly. To develop good intuitions, however, it can be helpful to look in more depth.
Setup
We start by translating the researcher’s conclusion into a probabilistic equation. Letting \(R\) stand for “remember details” and \(F\) stand for “frightening experience,” the conclusion states that details will tend to be remembered better when an experience is frightening compared to when it is not frightening; i.e.,
\[ P(R \mid F) > P(R \mid \neg F). \]
Note that this equation doesn’t entail that \(P(R \mid F)\) is high in absolute terms or that the difference between the two probabilities is large. Thus, to weaken the conclusion, we need a reason to believe that the inequality itself is false — either because the two probabilities are roughly equal or because experiential memory is worse for frightening experiences.
In support of this conclusion, Premise #1 states that frightening experiences lead to increased adrenaline release. As written, it’s not clear whether this should be taken as probabilistic or deterministic. At the very least, the researcher suggests that adrenaline (\(A\)) is reliably produced after a frightening experience, i.e.,
\[ P(A \mid F) > \tau, \]
where \(\tau\) is some probability close to 1. More importantly, the word “elevated” entails that
\[ P(A \mid F) > P(A). \]
In other words, adrenaline levels are more likely to be high during a frightening experience than during a randomly selected experience. Otherwise, how could adrenaline be “elevated” relative to baseline? Elevation implies an amount that’s “above average.”
Premise #2 statistically links adrenaline to better experiential memory:
\[ P(R \mid A) > P(R \mid \neg A). \]
Again, we don’t know what these probabilities are in absolute terms; we just know that the presence of elevated adrenaline levels predicts better memory for details relative to a lack of elevated adrenaline.
An important corollary
If we accept Premise #1, we must also accept that
\[ P(A \mid F) > P(A \mid \neg F). \]
This follows from the law of total probability, which states that \(P(A)\) is a weighted average of conditional probabilities:
\[ P(A) = P(F)P(A \mid F) + (1 - P(F))P(A \mid \neg F). \]
Specifically, the above equation entails that \(P(A)\) is between \(P(A \mid F)\) and \(P(A \mid \neg F)\) because a convex combination of two numbers is always intermediate between those numbers. Since we know from Premise #1 that \(P(A \mid F)\) is larger than \(P(A)\), it follows, a fortiori, that \(P(A \mid F)\) is larger than \(P(A \mid \neg F)\). So, granting Premise #1, we know that elevated adrenaline is more common during frightening experience than during nonfrightening experiences.
Importantly, this inequality can hold even if some nonfrightening experiences — such as the highly pleasurable experiences referenced in answer choice E — boost adrenaline levels. By the law of total probability, the probability \(P(A \mid \neg F)\) must be a weighted average of (a) the probability of elevated adrenaline for highly pleasurable experiences (which are assumed to not also be frightening) and (b) the probability of elevated adrenaline for experiences that are both not highly pleasurable and not frightening. Even if the former probability is as high as \(P(A \mid F)\), as implied by answer choice E, Premise #1 entails that the latter probability must be lower, thereby pulling down the average.
If this is confusing, consider an analogous argument. I tell you that eating steak raises cholesterol levels and that elevated cholesterol is associated with increased risk of heart attack. Therefore, eating steak is likely to increase your risk of having a heart attack, relative to eating foods that aren’t steak. You challenge the argument by claiming that eggs — an example of a food that’s not steak — raise cholesterol levels by the same amount as steak. Does this contradict or weaken my conclusion? No, because eggs are just a small subset of the set of foods that aren’t steak. Many other foods lower cholesterol or raise it to a lesser degree than steak, and my premise that “eating steak raises cholesterol levels” merely implies a raise relative to an average diet.
That’s the core intuition for why answer choice E doesn’t weaken the researcher’s argument. Continue on for a more detailed justification.
Mediation math
The psychologist’s argument implicitly posits a mediating pathway from frightening experiences to better memory, via increased adrenaline. That is, an experience’s being frightening predicts a higher probability of elevated adrenaline, and elevated adrenaline predicts a higher probability of clear memory for details. Moreover, if this is the only mediating pathway from experiences to memory, there is conditional independence between these two variables: frightening experiences don’t predict a higher probability of detailed memory once adrenaline levels are accounted for. This is a key assumption, which we will challenge in the next section.
Under this simple mediation model, there is a standard way to express the probability of remembering details during a frightful experience:
\[ P(R \mid F) = P(A \mid F)P(R \mid A) + (1 - P(A \mid F))P(R \mid \neg A). \]
That is, this probability is a weighted average of the effect of frightening experiences that don’t elevate adrenaline and the effect of frightening experiences that do elevate adrenaline. Given Premise #2, \(P(R \mid F)\) will increase with \(P(A \mid F)\), i.e., the reliability with which frightening experiences boost adrenaline. As this probability approaches 1, \(P(R \mid F)\) approaches \(P(R \mid A)\).
We can write the same expression for nonfrightening experiences. These experiences have some lower probability of increasing adrenaline levels, which will in turn improve memory via the same mechanism:
\[ P(R \mid \neg F) = P(A \mid \neg F)P(R \mid A) + (1-P(A \mid \neg F))P(R \mid \neg A). \]
If \(P(R \mid F)\) is larger than \(P(R \mid \neg F)\), as the researcher concludes, then
\[ P(R \mid F) - P(R \mid \neg F) > 0. \] Subtracting the two equations above, we’re left with the product of two differences:
\[ P(R \mid F) - P(R \mid \neg F) = (P(A \mid F) - P(A \mid \neg F))(P(R \mid A) - P(R \mid \neg A)). \]
The first difference,
\[ P(A \mid F) - P(A \mid \neg F), \]
is positive via the corollary proved earlier. The second difference,
\[ P(R \mid A) - P(R \mid \neg A), \]
is positive because of Premise #2.
Thus, because the product of two positive numbers must be positive, the difference between \(P(R \mid F)\) and \(P(R \mid \neg F)\) must be positive, rendering the researcher’s conclusion true:
\[ P(R \mid F) > P(R \mid \neg F). \]
In layman’s terms, this math shows that, if we take the researcher’s premises to be true and we assume that adrenaline is the only mediator of experience on memory, then the researcher’s conclusion must also be true. Answer choice E can’t undermine this conclusion on its own. Granted, if we interpret the researcher’s conclusion more colloquially, choice E could shrink the implied size of the difference between \(P(R \mid F)\) and \(P(R \mid \neg F)\) by suggesting that \(P(A \mid F)\) and \(P(A \mid \neg F)\) are closer together than expected. Still, keep in mind that the researcher doesn’t draw any conclusion about the magnitude of the difference between the memory effects of frightening versus nonfrightening experiences. Even if that difference is small, the conclusion that “on average, people will retain the details of frightening experiences better than the details of nonfrightening experiences” will hold.
Alternative mediators
There is, however, a different method of weakening the argument: consider alternative mediators other than adrenaline. This is what answer choice C does, by suggesting a distinct way in which frightening experiences could impair memory.
If we accept that frightening experiences are more likely, on average, to be extreme experiences compared to a random other experience, and that the extreme stress1 associated with frightening experiences can impair memory, then there’s a new mediating pathway from fear to memory:
\[ \text{Frightening Experience} \rightarrow \text{Extreme Stress} \rightarrow \text{Worse Memory}. \]
In order to model the effect of frightening experiences on memory, we need to account for the fact that such an experience is more likely to both release adrenaline (\(A\)) and cause stress (\(S\)). The equation gets a little bit longer, but the basic logic stays the same; to calculate the probability \(P(R \mid F)\), we take a weighted average over all possible combinations of adrenaline and stress:
\[ \begin{align} P(R \mid F) = {}& P(A \land S \mid F)P(R \mid A, S) \\ & + P(\neg A \land S \mid F)P(R \mid \neg A, S) \\ & + P(A \land \neg S \mid F)P(R \mid A, \neg S) \\ & + P(\neg A \land \neg S \mid F)P(R \mid \neg A, \neg S). \end{align} \]
The equation for \(P(R \mid \neg F)\) is the same, substituting \(F\) for \(\neg F\) everywhere.
Answer choice C implies the following inequality about how memory is related to stress:
\[ P(R \mid S) < P(R \mid \neg S). \]
While the answer choice doesn’t specify the magnitude of the stress impairment, it leaves open the possibility that
\[ P(R \mid A,S) < P(R \mid \neg A, \neg S). \]
In other words, experiences without increased adrenaline or extreme stress may be remembered better than those with both. If that’s the case, it’s now possible that
\[ P(R \mid F) < P(R \mid \neg F). \]
Translation: because frightening experiences are more likely to be both stressful and adrenaline boosting than a random nonfrightening experience, and stress combined with adrenaline may actually worsen memory overall, the researcher’s conclusion is possibly undermined. Thus, unlike answer choice E, answer choice C provides a route for the researcher to be wrong.
Intuitive statistics is lawgical
While it’s impractical, and possibly inequitable, to expect law school applicants to learn probability theory for the LSAT, the test shouldn’t encourage biased methods of reasoning, as I believe it has done here. Instead, the writers should recognize that scientific logic plays an essential role in the law — for example, for assessing evidence in a trial and deciding whether expert testimony is credible and compelling.
Fortunately, I believe it’s possible to learn good inductive reasoning without hardcore math. This topic deserves a much more detailed discussion than what I can offer here. But to whet your appetite in concluding this post, I’ve written my own LSAT-style question to suggest one way to test good thinking about statistical evidence. Give it a shot before checking the answer in the footnote.2
Lawyer: My neuroscience expert will testify that my client has a brain lesion that’s common in people with psychosis, supporting our contention that the defendant was insane at the time of the murder.
Judge: Your expert’s studies have found that the brain lesion in question is present in only 20% of psychotic individuals, and it’s therefore much more common not to have the lesion. Your line of argument can’t possibly support an insanity defense.
The judge’s response is flawed in that it
A. Relies on an equivocal usage of “common.”
B. Focuses on the lawyer’s motives rather than the lawyer’s argument.
C. Fails to consider whether the brain lesion in question could be more common for mental illnesses that don’t cause psychosis.
D. Rules out the possibility that the brain lesion in question is even less common among nonpsychotic individuals.
E. Assumes that only statistical factors can support an insanity defense.
The answer choice doesn’t mention stress specifically, but I mention it for ease of presentation. This is not a requirement for the argument. Any mechanism by which extremity impairs memory works, so long as extremity is correlated with fright.↩︎
Correct answer: D (highlight the text to reveal the answer).
Explanation: To assess whether some fact is evidence, we need to evaluate the ratio of the probability of observing this data if the hypothesis is true to the probability of observing this data if the hypothesis is false. Thus, even if only 20% of psychotic people have the lesion, having the lesion could be evidence of psychosis if the base rate of the lesion in the general population is much lower. (By analogy, consider the fact that winning the lottery can be extremely strong evidence that you bought a lottery ticket even though winning the lottery is extremely unlikely.)
The other answers are wrong for the following reasons:
A. The lawyer and judge may be using the word “common” in slightly different ways, but the judge isn’t making a semantic argument that hinges on the meaning of that word.
B. The judge doesn’t attack the lawyer’s motives.
C. This is only indirectly related to the diagnostic value of the lesion for demonstrating psychosis. And if the judge found support that the lesion is common for nonpsychotic mental illnesses, this would actually strengthen the judge’s case that the lesion is not evidence of psychosis.
E. The judge is arguing that a particular statistical argument is inadequate, not making a general point about the adequacy of (non)statistical evidence.↩︎
