Start here

One possible explanation of the Marine Corps gender difference data: Study statistics more!

This morning I heard this piece (linked below) on NPR.  The thrust was that Marine units with women in them performed worse in combat simulations than did those with men only.  There could be many explanations of this.  I want to flag one that was not raised in the NPR piece.

My understanding from this article is that some of the performance differences were likely due to muscle mass.  How much weight can people carry around for how long?  Probably those with more muscle mass can carry more weight.  I am no expert on the following, but my best read is that men tend to have more muscle mass.  I would not be surprised if the men, on average, could carry more.  But…someone women doubtless could carry more than some men. I’ll elaborate below.

I could not find the distributions of muscle mass for men and women, but I did find one for height, and I can use this to illustrate the point.  Here’s the link:

Men tend to be taller than women.  If we draw a particular cutoff for inclusion in our group (say “the Marines!”) (say, 169 cm in the graph), the height of men above the cutoff will be greater than the height of the women above the cutoff.  All have to be above 169 cm, but the average man taller than 169 will still be taller than the average woman above 169 cm.  So…if they are doing a height-related task men will do better than women, even though the same cutoff is being used to let someone in.

Is this the same thing with the Marine Corps?  If so, a uniform standard for muscle mass (or how much weight one can carry, for that matter) will leave a group included in the Marines in which women tend to underperform relative to men who have surpassed that uniform standard.

But…and this is key…by including women you can raise the standard and do better overall.  Some women will carry more than some men and those men can be displaced from the group leaving you better off overall.

Look back at the height chart. If I need a particular number of people and I want the tallest, I can reach the requisite number of tall people at a higher cutoff if I include women than if I do not.  Thus inclusion of women leaves us with two things (a) a group that overall performs better on height-related tasks (which is what should matter!); and (b) a group in which women underperform relative to men on height-related tasks.

If the analogy holds, the Marines should not worry a whit if the groups with women underperform relative to groups with men.  That’s not the relevant question.  What they should worry about is whether they can be more exclusive (that is, set a higher standard for inclusion in combat service) by including women, and thus as a whole perform better.

Obviously this is a more complicated situation than I described.  But I was stunned when I listened that no one discussed this as an issue.  These are not stupid people. But I wonder if they have spent any serious amount of time thinking about stats.

If you want to think clearly, please, please, please spend a good chunk of time learning, at a deep level, about basic statistics.

Inigo Montoya and psychology–You keep using that word. I do not think it means what you think it means. (Psych Favorites,post 3)

I am starting this series with three important stories about how little psychology can tell you.  In the first two I’ve noted that we can’t tell you why you should get out of bed in the morning and that we very rarely know much about why things happen.

Today I am focused on the fact that we oftenuse words in ways that are misleading.

Take, for instance, “happiness.”  I just googled “psychology and happiness” and I see the last story posted on a psych study about happiness is from 2 hours ago, in New York Magazine.

The story has this quote: “[Y]our middle-age happiness can be predicted by two things: the quantity of friends in your 20s, and the quality of friendships in your 30s.”  They measured happiness, right!  What did you think when you saw the word? Did you wonder what they meant by happiness or did you just fill in with your sense of what happiness was?  They seem to have combined four self-report measures.  For instance, they asked subjects the degree to which they agree with statements like “In general I am in charge of the situation in which I live” and “I tend to worry what other people think of me.”  Is that happiness?  Is that what you thought they meant when they said “happiness?” You can make an argument that it is, but there are all sorts of reasons to be hesitant.  People have trouble reporting on their experiences.  (What has your day been like?  As you answer did you go back to every moment of the day then somehow average it?  If so I am impressed because that seems like an impossible task!) So maybe they think they were happy but were in fact not happy.   Is happiness about the experience of pleasure?  Or is it about meaning (“this diaper I am changing is really rancid, but despite this misery my life has purpose!”)  Is either of those captured by the questions above? The article you read just said “happiness.”  And you likely filled in what they meant rather than wondering if the authors really got at happiness.  (Having said that, New York Magazine described a very cool study!)

That’s an easy example.  But think of any psychology study you read about in the news.  Does going to college enhance quality of life?  Some newspaper accounts will say yes, others no, but do any of the studies really address anything that gets at what you think of as “quality of life!”  Other studies will talk about doing things that lead to success, but what is success?

Or as another example, people want mental health.  But what the heck is it?  Over time the definition has changed. Across time the definition has changed.  At one point the methods that psychologists used to classify were wildly unreliable.  But once the phrase “mental health” has been uttered by someone with a doctorate how hard is it to keep an open mind about whether the person is actually talking about mental health?  And so we read articles about mental health and do not realize the challenges of actually measuring mental health.

I think every graduate student who works with me has a moment in which she says “I have here a measure of X.”  I say “Are you sure it is measuring X?  Read the items to me.”  The student reads the items and then says….”hey, that doesn’t sound like X at all!”  Because life is short and if you are a student looking for a measure of X and someone says they have a measure of X who are you to say they do not and what time do you have to really worry about it. So you think you have a measure of X even if it measures something else entirely.

But saying something is X does not make it X.  That word often does not mean what we think it means.

Over time when we as psychologists try to assess something like “happiness” we try to do so in a bunch of different ways.  We ask people to tell us what they think. We look at what they do. We ask their friends to report on them. And so on.  If we find that for every way we can think of measuring “happiness” doing a particular exercise increases happiness then we start to think the exercise causes happiness. (Start!  Maybe we have missed something important that won’t be discovered til later!)

As a reflex, then, I am skeptical when I hear someone say “I have measured X.”  But I have developed that reflex from years of study.  Mostly people absorb their psychology with less training.  And less time to think.  Those communicating in media often have reason to use shorthand.  (“Hey–happiness!”)  Particularly these days (perhaps I will write sometime on why I say that) there is great pressure on academics to fudge a bit (“Hey-happiness!”) rather than hedging about how uncertain they are that they have measured what they hoped to measure.

Given this, what can you do as you read the paper, listen to the news, scroll through your news feed.  I sense that two extremes are likely–believing thoroughly what is written or rejecting psych research as a fraud. I would encourage a third way, Perhaps say “That’s interesting.  Tell me more. And let’s watch how this plays out over the years to see if it has legs.”  That might be less satisfying than just accepting or rejecting the story.  But it is also likely more in keeping with what we know. And it can help to cultivate something that I personally find compelling–a sense of wonder.

Would you eat your pet?

Well, would you? If you were not vegetarian, if the pet had died of natural causes, if you had received assurance from a veterinarian that it was healthy, and if you had a nice barbeque sauce for it?

If you are like my students you almost certainly said “NOOO!”


That’s the interesting question.  Can you articulate a reason?  Did you articulate that reason and conclude logically from that reason that you should answer “NOOO!”?  When I ask my students I generally get to the point at which they say “I don’t know why, but it just feels wrong!”  This is much the answer Jon Haidt has gotten, and I cribbed the example from him.

Haidt’s view is that in many cases our moral judgment is intuitive. We have a gut sense that something is wrong and we then come up with reasons to justify the gut sense.  So we might say “but it might not be healthy to eat Rover” but when upon hearing that the vet assures us, and we believe it would be healthy we still think “NOOO!”  Perhaps much of our moral life is driven by our gut sense and our subsequent efforts to justify our gut sense.  Our lofty moral explanations, then, are often just stories we tell ourselves so we don’t feel like idiots with strong moral intuitions not grounded in reason.

Haidt suggests that we have five (ok, now it’s six, but the link below focuses on the older version) foundations from which arise these moral judgments.  Is there harm?  Is it fair?  Does it obey authority?  Does it violate loyalty to the person’s group?  Does it violate purity/sanctity?

Much of his work suggests that conservatives are driven by all five foundations whereas liberals are driven mostly by harm and fairness.  Thus liberals often do not understand conservatives’ moral judgments because they don’t value those other three foundations (authority/loyalty/sanctity).  And both liberals and conservatives often work from intuitions but hold forth publicly using the stories they tell themselves, which are not the real reasons for their moral judgment (“It would be unhealthy!  Oh, it would be healthy?  Well, I still think it would be wrong. Give me some time to figure out why!”)  And so we have really awful public discussions of moral decisions in which we understand neither our own basis for morality, nor that of others, and we at best struggle to see the other as someone who is neither a dolt nor evil.

The linked piece (below, from Bloomberg) cites some new research suggesting that liberals might actually use those other three dimensions after all.  For instance, if someone paints an arrow on a mountain it evokes in liberal mountain climbers a reaction like that arising in conservatives who see an arrow painted on an American flag.  This seems not to be due to harm to the mount, but more a sense that the mountain is sacred and the paint a violation.  Similarly are there authorities whom liberals feel should be obeyed (environmentalists, for instance), such that it is only conservative authorities liberals do not think should be obeyed?  In his wonderful initial TED talk Haidt (link below) intimated that liberals might actually value sanctity/purity (organic food!).  But I had not seen data supportive of that til these articles described in the Bloomberg piece.

Why should we care?  What if we all (liberals and conservatives alike) have intuitions about the sacred, about authorities to which we defer, about the ingroups to which we should be loyal, but what if we often are unaware of these intuitions?  Things (like eating the pet) will seem wrong to us. We will come up with stories about our reasons that are just stories, not the real reasons for our intuitions, and we will be appalled by the badness of others whose intuitions differ from our own.  As McArdle (below) puts it: “Coming at someone with utilitarian math when the problem is actually that you’ve desecrated their sacred space is a recipe for bitter and unresolvable conflict — and perhaps, for a culture war that no one is going to win.”

But what if we understood ourselves to see some things as sacred, even if we do not fully understand what those are, even if we think at some level that “sacred” is just a superstitious idea?  Perhaps we could understand, at least a little, others who have views of the sacred as well.  Perhaps we could talk to them about the sacred, rather than about harm (when their intuition is about the sacred).  Perhaps then our disagreements will be less sulfurous.

Postscript: I find Haidt’s work beautiful to reflect upon.  How were my moral intuitions formed?  Likely they developed long ago and I often lack the language to articulate those morals.  How might I shape my intuitions to be somewhat more in line with my reason?  I do not have an empirical answer to that question.  But perhaps by reading moral philosophers and moral theologians with both an intention to be open and the knowledge that I am in some ways a stranger to myself.

You know nothing about causality, Jon Snow! (Psych favorites, post 2)

(2nd in my series of most important things to know about or from psychology)

Psychologists study why people do what they do.  They know almost nothing about why people do what they do.

Today I’ll write about tive problems in inferring cause.  1) We mistake correlation for cause.  2) We forget that actions can have more consequences than the ones we study.  3) We study linear causes when people are embedded in systems. 4) It’s really hard to study a lot of possible causes. 5) We reason backwards from effect to cause.  This piece ran long, so I numbered them if you want to skip ahead.

1) The amount of ice cream consumed on a given day is correlated with the number of homicides on that day. So eating ice cream makes people homicidal, right?  You of course know better.  Killing people makes people crave ice cream!  We had it backwards.  Or perhaps a third way of looking at this is better.  Hot days prompt both ice cream consumption and homicides.  You know correlation is not cause, so hopefully you followed this example.  You know that we need to do experiments (manipulating one variable and holding the others constant) to determine if something is a cause.  If you want to know if eating ice cream causes homicides have some people eat ice cream and others not and see if innocent bystanders are more likely to die at the hands of the former.  I’m no longer IRB Chair, so I don’t even have to review your study for its ethics!

But do you remember to question whether those headlines you read in the news about cause are based on correlations rather than experiments?  You should.  For instance, one British study found that 40% of press releases exaggerated the meaning of the studies they described, with confusion of correlation and cause one common problem.  I just went to the Washington Post health/science site and found a piece from only 12 hours with this: “How much money people have to fork over when they go to the doctor can make a big difference in how satisfied they are with their health plan, a recent study suggests.”  Those with more out of pocket costs were less satisfied, and this reads like they know cause and effect, right? But did the costs actually “make a big difference”?  Who knows?  It’s not an experiment.  It could be that something makes people choose plans with more out of pocket costs and separately makes them less satisfied with their care.  (Income?)  We can’t know from the study as described.

With the election coming up you will see lots of statements from politicians about how what they propose will cause particular outcomes. Reflect on how many of these are backed by experiments.  Without that why should you buy into their predictions?

2) Actions have multiple effects.  If I buy a donut I not only end up with a donut but also with less cash in my wallet.  But science is often conducted by measuring only one outcome, for instance, counting the donuts I have after the purchase but not the shortage of cash.  Consider research on treatments for depression.  You will find experiments (studying a bunch of different treatments!) with reasonably good control groups.  You will see that the treatment affects depression. What you won’t see is an examination of all the other variables that might also be affected by the treatment.  For instance, might drug treatments for depression both reduce depression symptoms and also create other unwanted effects?

This strikes me as particularly problematic when the unwanted effects are difficult to measure.  One of the prime examples of this is the misguided (to my mind) emphasis on high stakes testing.  Folks with a very reasonable concern (how do we know kids are learning?) introduce testing so they can see if learning actually does increase.  But what other effects might high stakes testing have?  Might it lead to teaching to the test so that other things (like learning wonder, cooperation, and survival tactics for boredom) fall by the wayside?  Might some treatments for psychological disorder cause short term improvement at long term cost?  Might some forms of bucking up support for our favorite political candidate help win an election but also undermine the long term survival of democracy?  Might sparing people from challenge today leave them helpless a year from now?

Next time you read an article reporting that an experiment shows that X causes Y ask yourself what the Z’s might be that X also causes but that might not have been studied.

3) Have you ever left home, changed yourself, then returned home to find yourself returning to your old ways?  If so, this hints a the powers of systems, of the ecologies in which we live.  Psychology prizes (for many good reasons) experiments that involve manipulating X and looking at the consequences.  It is the way we get at cause.  But we only study people for so long, then they return to their homes.  So we might via experiment find a great way to get someone to give up alcohol off at our secluded treatment center. But when they return home their sobriety has a consequence-friends desert them!  And this serves to bring people back to the drink.  People live in a web of causes that can tend to hold things in balance.  Push on one factor and other factors will subsequently push back to return things to the status quo ante.  Lasting change might be better served by understanding this.

How much has our emphasis on the experiment caused a picture of the human that misses the centrality of our complete ecology?

4) If we can only know about cause through experiments then the set of causes of human behavior that we will know will be those causes that are easily manipulated.  If you want to know if going to college actually has an effect on something you need to randomly assign some people to go to college and others not, then look for the difference. This is insanely hard so I really have no clue if college actually causes good outcomes.  (Sorry if I just cost a lot of colleges tuition money!)  And I live my life understanding that I do not actually know if I am helping or hurting my students.   I can’t do the experiment.

Perhaps a treatment for X that takes 2 years will be much more effective than a treatment that only takes 2 months.  But you could only find out if you could do a study that actually delivers 2 years of treatment. Good luck with that! It will often be enormously expensive and logistically excruciating a study that takes that long.

How much does this limitation of science shape our understanding of what can cause good outcomes?  You have before you the rest of your life. You could imagine embarking on an experiment with it to see what happens.  How could I tell you that the life path you choose helps or hurts?  The experiment can’t be done.

5) Often the question we have is why something happened. The science can often only tell you what some causes of a given outcome are.  We can say that blowing up a thermonuclear device within fifty feet of you causes bad things for your health.  But just because your health is bad does not mean that you just had a thermonuclear device go off nearby.  We can speculate about the things that caused us to be where we are now, but it will mostly just be speculation.

People want to know why relationships ended (or thrived!), why jobs are lost (or found!), why their children are learning (or slumping).  While we know some factors that affect relationships, jobs, and children, it is awfully difficult to know about what in the history of a particular person has been an important cause.

6) A concluding thought.

All of this (and more!) says that most of what I would like to know about why people do what they do I cannot know.  Paraphrasing Gleitman (I think), psychology consists of small islands of coherence in a vast sea of chaos.

And in this I come to one of the main stories of psychology.  I have before me a choice.  I can pretend to know things I do not.  Or I can despair from knowing how little I know.  Or I can understand the limits of what I know and then act, knowing the act is at least to some degree a leap of faith.  What do we think of “faith” in modern America?  My sense is that we value certainty, instead.  Might understanding the limits of what we can know about people, and noticing that our actions then are leaps of faith rather than acts born of certainty help us find new appreciation for that word “faith?” And new tolerance of ourselves and others when our actions do not bring us to the verdant pastures we sought?

And for the field of psychology–can we neither claim more than we know nor despair in that which we do not know, but rather do the next study, in the faith that our collective enterprise will turn those small islands of coherence into somewhat larger ones?

Why get out of bed in the morning? Don’t ask psychology. (Psych Favorites, post 1)

In addition to periodic reports of new findings I am going to write a series of posts on some of what I believe to be the most important things to know about or from psychology.  I will start with some of what psychology can and cannot do, then turn to some specific findings. This is the first of these posts.

I have taught introductory psychology for over a quarter century. Many students take the course to discover the meaning of life, to decide why they should get out of bed each morning.

My first class meeting I tell students that we can’t answer their question.  It’s not our job.  They should turn to philosophy or theology (or perhaps the Hitchhiker’s Guide) if they want an answer to the question of the meaning of life.


To understand this is to understand what psychology does. A rough definition is that psychology is the study of behavior and mental processes.  Some might shorten this to the study of behavior given that mental processes are just one sort of behavior.  People do things.  We run laps and research studies, go to concerts and through existential crises, walk dogs and yoyos, watch leaves change and The West Wing, fall in love and in the pit…and I could go on.  Psychology can describe what some of these things are that people do.  (As I write this my behavior could be defined by the movements of my fingers, by my construction of words, by my attempt to write a cogent essay, by my attempt to lead a good life, and so on.  How shall we describe it?)  It can then talk about factors that help us or hinder us in these actions.  (It’s easier to type with none of my fingers in a splint, and it’s easier to be cogent if I am not severely sleep deprived.)  Causes can be explored at many levels.  (What chemicals are doing what in my brain?  Who just walked down the hall outside my office?  What is my intention?  What were my surroundings growing up?)

None of that description and explanation of behavior says a whit about why we should get out of bed in the morning, or more generally what the meaning of life is.  To answer that question you have to have a concept of the “good.”  Psychology can certainly describe what people think “good” is.  (Human sacrifice?  Veganism?  Total immersion in politics?  Excitement? Tranquility?  Watching Silence of the Lambs?)  We can tell you who has the most popular definition at any given moment.  But we cannot tell you which definition is the right one.

Not our job.  Not doable by our methods.  How could it be?  We can describe, including description of cause. That’s it.  I like the comparison to auto mechanics.  They can describe cars very well. They can make them work because they understand an important subset of the causes involved in being a car that travels.  But a mechanic can’t tell you why you should prefer to drive to New Mexico vs. New York.  Well, I guess they can, but why on earth should you privilege their answers over the answers of others?

I think about this a lot as I teach my course on Psychology of Happiness.  At some level I cannot even say what happiness is, at least not as a psychologist.  Is happiness leading a pleasurable life?  (I am drinking a nice oolong tea as I write this.)  Is it being deeply engaged in what one is doing?  (As I wrote that sentence I suddenly realized how focused I had been on writing this.)  Is it doing something meaningful?  (I find meaning in sharing with others that beauty of psychology that I know.)  Different people give different definitions.  Who am I to judge who has the best?

We do not train psychologists to understand the meaning of life.  We don’t have the skills to do that.  Nonetheless, my sense is that it is to psychologists that people turn.

If you want to decide where to drive this summer read about different locations.  Talk to those whose tastes are like yours.  Reflect on your life and your values.  I’d recommend any of those over talking to a random auto mechanic.

If you want to understand the meaning of life read some philosophy.  Read some theology.  Reflect.  Once you have a definition you might learn some psychology to help you get to the end that you desire.

How much would you play for that beer?

Imagine you are at the beach on a sunny day and a friend offers to pick up a beer and bring it back to you.  How much would you pay?  How do you make the decision?  Does the amount you would pay depend on whether it comes from a grocery store or an expensive resort?  Does it depend on how much money you make?

In an April Psych Science paper, Shah, Shafir, and Mullainathan describe a bunch of studies with more than 4000 subjects.  Across a variety of tasks those who were poor (whether in money or time) were more likely to make their decisions based on tradeoffs (“If I buy the beer there’s something else I can’t buy…”), and so were more likely to be unaffected by frame (so, for instance, they will pay the same for the beer whether it is from a grocery store or a resort).  Wealthy people do not report thinking in terms of tradeoffs and they are willing to pay more for the beer from the resort than the grocery store.

The paper focuses on the implications for economics.  “Economics makes concrete predictions about how preferences should unfold, whereas psychology and behavioral economics have identified several ways in which those predictions break down.  Economics makes those predictions because it is built on the (correct) assumption that humans navigate a world of scarcity and regularly make tradeoffs. Remarkably, however, when people experience sufficient abundance, those trade-offs recede from attention.”

I wonder about which factors bring about or alleviate a scarcity mentality.  I study gratitude, and one part of the practice of gratitude seems to be recalling that our lives are abundant.  Take a second to recall the last hour or day. What good things have happened?  How full is your life when you stop to think about it?  Would the practice of gratitude reduce scarcity effects?  I just did a PsycInfo search and found no studies on this.  (“Gratitude” and “scarcity” produced only 6 hits, none on topic.)

But if gratitude reduces scarcity effects that would seem to make people less rational in an economic sense.  But…gratitude seems to be good for us. (The evidence is yet young so I’m not entirely convinced.  See my chapter with Courtney Forbes for more detail.  Still, it looks promising.)  How does one reconcile this?  Is economic irrationality good for us?

I’m not sure what I think, and I suspect my thoughts are at this point incoherent.  I’m mostly just curious.  But perhaps it says something about what rationality is when there is real scarcity (when, for instance,it is unclear where the next meal will come from) as opposed to rationality when there is abundance (when, for instance, we might focus on the needs of others, since our own are met).

Here in the US we live in one of the wealthiest countries in history, yet the psychology in which so many of us live seems one of scarcity.  How much does that lead us to think in terms of specific tradeoffs (if I pay too much for this beer I won’t be able to buy lunch), and how much does it lead us not to think in other ways (the cost of the beer is just not that important to me–what is more important in my life?)

Here is a link to a piece on the Shah et al. article.

Mindful eating

Are you reading this as you eat?  If so, how much are you enjoying your food?

I’m returning home from the Association for Behavioral and Cognitive Therapies (ABCT) convention and one of the talks, by Colorado’s Joanna Arch, was on mindfulness and eating.  She has had people eat just one of something (in two studies a raisin, in the third a chocolate chip), five times, with two or three minute breaks in between.  During the breaks subjects search for words in a jumble. Some are given mindfulness instructions (like the raisin exercise for those who know it). Others are given no instructions or instructions to keep working in the word search as they eat.  The mindfulness exercise leads to more enjoyment of the food.  In the third study subjects could then eat as much as they wanted of six foods (pretzels, M & M’s, potato chips, carrot sticks..I’ve forgotten the others), as a “thanks” since they had fasted for three hours.  Those who had eaten mindfully now ate less of the salty, sugary, and high-fat foods, but just as much of the “healthy” (e.g., carrot sticks) food.  So mindfulness both increased enjoyment of tasty food and reduced the amount of eating.

Perhaps slowing down as we eat will help us to enjoy the food before us and, despite this increased enjoyment, eat less of it.

I imagine the “eating less” could happen in various ways….perhaps there’s less of a need after mindfulness. Perhaps there is more intentionality. Perhaps some of the not eating is aversive. (“I love this food, but it’s scary, and I’m paying attention now, so I realize it’s scary.”) So there’s more more work to be done.  But the increased enjoyment seems consistent and reasonable.  And I very much like the idea that attention to that which we have before us will help us feel sated and content, and lead us to chase most after that which we really need in the moment rather than consuming without even knowing it.


Get every new post delivered to your Inbox.