The workforce crisis part 6 – What do the results of the ‘Mark’ and ‘Elizabeth’ study mean?

The collaborative research project between the University of Exeter and the British Veterinary Association on workforce issues in the veterinary profession has prompted many questions. In this series of blog posts researcher Chris Begeny will explain why their team framed their study as they did and how they interpreted the results.

You find that employers and managers who believe female vets still face discrimination do not significantly differ in the salary advised for ‘Mark’ or ‘Elizabeth.’ This is shown in a figure in the BVA report specifically, the bars in the right panel.

But those bars look different. So what does it mean for differences to be (not) significantly different from one another, statistically?

What can we conclude?

Note upfront: Below are some general and illustrative descriptions of what inferential statistical analyses allow us to conclude – these are not literal or technical descriptions of how the analyses work. If you want more information about these types of analyses there are some very useful videos on YouTube from Brandon Foltz and other statisticians.

The basic idea is this: imagine that we were to put the data we’ve collected aside and re-run this study – collecting data from another 250+ managers and employers (who have no prior knowledge of this report, the associated LVS presentation, or any other information that’s been disseminated about this study, as this could influence how they respond to the study).

The analyses of the data we’ve actually collected would suggest, with over 95% confidence, that when these new data were analysed, the bars in the left panel would end up looking more or less the same as they currently do – a significant “pro-Mark” pay disparity would persist among those who believe discrimination against women in the profession is no longer a problem.

By comparison, we can’t say with any reasonable level of confidence that the bars in the right panel would look the same. They could end up looking the opposite of what they currently look like. They might also look the same as they do now, but they might also look roughly level to one another, or they could show any other conceivable pattern of results. And that's the point. The statistical analyses tell us that among this subset of respondents – those who believe discrimination against women in the profession is still an issue – there is no reliable or consistent “pro-Liz” or “pro-Mark” pattern of advised salaries.

Here’s another way of thinking about it: while the two bars in the right panel might on the face seem to suggest that ‘Elizabeth’ gets offered more money than Mark, statistical analyses allow us to empirically assess whether this seeming difference is a reliable one – one that would likely show up again and again if we re-ran the study – or if instead, it is just a fluke. The analyses do this in part by looking at the amount of variability among respondents’ advised salaries for ‘Mark’ and ‘Elizabeth.’ And the results show that there is a lot of variability among these respondents (those who believe female vets still face discrimination) in terms of how much they would pay ‘Elizabeth’ or ‘Mark’ – so much so that there is no clear, consistent or discernible pattern suggesting a “pro-Liz” (or "pro-Mark") pay disparity.

In other words, within this subset of respondents, there are some individuals who offer Elizabeth a relatively large salary, but there are others who offer Mark a relatively large salary – offering him more than a lot of other respondents in this same subset offer to ‘Elizabeth.’ Still, other individuals are offering Mark a salary that is on par with what others offer Elizabeth in this subset. So again, the point is that there is a lot of variability within this subset of respondents (those who believe discrimination against women is alive and well). 

Statistically, this means there is no clear, consistent or statistically reliable pattern that would allow one to conclude with any meaningful level of confidence that employers/managers who believe women still face discrimination would in fact systematically offer different salaries to equally qualified male and female vets. By comparison, we can say with a great deal of confidence that employers and managers who believe women do not face discrimination would systematically pay a male vet more than an equally qualified female vet.

To be clear, this doesn’t mean that every single manager/employer who thinks gender discrimination is a thing of the past is or will discriminate against female vets. But it does mean that if you were to pick out an individual from a collection of employers/managers who have just one thing in common – they all think discrimination against women is a thing of the past – far more often than not you will find an individual that does discriminate.

Have similar gender discrimination studies been conducted by other professions and how do vets compare?

Similar studies have been conducted in a variety of contexts and professions. For an overview, you might look at the meta-analysis carried out by Koch and colleagues in 2015, which includes 136 independent study samples and 22,348 respondents total. Note however that by virtue of it being a meta-analysis it is empirically somewhat ‘clunky’ – not well-suited to examine or identify important nuance underlying patterns of gender bias, but it is helpful for seeing broader patterns of bias that are evident. More generally, it provides a useful list of several (experimental) studies that have been conducted over the years.

That said, I think it is difficult to make direct comparisons to other professions and contexts. Partly this is because there are notable (often nuanced) differences between professions and contexts, and partly because studies conducted vary in the specific methodology used, the population sampled, the specific outcomes assessed. The article noted above ( Koch et al., 2015) will give you a flavour of just some of this variability. All to say, I think it is prudent to avoid making too many direct comparisons between the veterinary profession and other professions.

That said, there are some particularly notable similarities between what we found in the vet profession and what Moss-Racusin and colleagues (2012) have found in the academic sciences, focusing in particular on the biological and physical sciences. I think this context has some notable parallels to the veterinary context, including that both are traditionally male-dominated yet with evidence of an increasing number of women represented, at least at certain levels within these fields.

As in our study, Moss-Racusin and colleagues find a pattern of gender discrimination that:

1. Favours men.
2. Is rooted in differences in the perceived competence of (equally qualified) men and women.
3. Is most pronounced among those who don’t believe women continue to experience discrimination (as we explain in our report, and as Moss-Racusin discusses, this is part of a more general manifestation of sexism/sexist beliefs).

Though again I hesitate to make too many parallels between professions, I think this consistent pattern of findings can be taken to suggest, at the very least, that the vet profession is not unique. The persistent gender biases found in the vet profession are seemingly alive and well in various other professions too.

Related research

• Gender bias in leader evaluations: Merging implicit theories and role congruity perspectives (Personality and Social Psychology Bulletin)
• A meta-analysis of gender stereotypes and bias in experimental simulations of employment decision making. (Journal of Applied Psychology)
• Science faculty’s subtle gender biases favor male students. (Proceedings of the National Academy of Sciences)
• Meta-analysis of field experiments shows no change in racial discrimination in hiring over time. (Proceedings of the National Academy of Sciences)