In areas of my and the Bibliovore's life including work, reading, social interaction, and media consumption, we notice and are bothered by the Dunning-Kruger effect. I've been wanting to explore this topic fo a while--and am hoping to avoid a rant! Therefore, the sources for this post will be more on the academic/technical side than what I usually cite. I will also hone in on the blogosphere aspect, as a means of self-reflection.
What is the Dunning-Kruger Effect (DKE)?
Named for its namers, David Dunning and Justin Kruger, the Dunning-Kruger effect (abbreviated DKE throughout this post) is one of many possible cognitive biases. Not all biases are bad, but the DKE is fairly problematic.
Definition
From one of my favorite references, Encyclopedia Britannica, the DKE was described and defined in 1999 by Dunning and Kruger based on analysis of students' self-assessment scores compared to their self-ranking scores (within a peer group) and their actual competence scores in several areas including logical reasoning. While low-performing students will reliably recognize that they do not have the same level of competence as do experts in a skill or knowledge area, by the DKE they will consistently overestimate their own competence or knowlege in that area relative to "average" peers.
The DKE has been recognized for centuries before its description in 1999. I'm reminded of the "Lake Wobegon Effect" where "all the children are above average." These two things aren't technically the same, but rather the DKE could be phrased as "all the children think they're above average."
Traditional Applications and Solutions
A brief blog post from University of Michigan links to several multimedia resources on the DKE. As the 1999 paper by Dunning and Kruger stated, logical thinking, humor, and partisan settings (e.g., alignment with political parties) were three areas where the effect was most pronounced. Because cognitive biases are part of being human, it is highly likely that each person is somewhere susceptible to the DKE.
Dunning did a systematic review in 2011 of applications of the effect, extending the original 1999 work. I was unable to access the entire article, but this quote from the abstract summed up the DKE well:
"Poor performers in many intellectual and social domains [have] a double burden--not only does their incomplete and misguided knowledge lead them to make mistakes but those exact same deficits also prevent them from recognizing when they are making mistakes and others are choosing wisely."
Work that has been done by Dunning, Kruger, and others cites the best solution for the DKE as improvement of personal competence. This will, in turn, improve metacognition ("thinking about thinking"), which my colleagues and I focus heavily on in our graduate students. The trouble is, some students with a severe case of the DKE may resist evidence-based efforts to improve their competence, or complain when those methods are foisted on them (e.g., distributed practice).
What are Other Explanations for the DKE?
Sources for this section are Science Alert, Scientific American, and Nuhfer et al. 2017. I found background research for this section particularly helpful, because I've been susceptible in my own teaching and thinking to mischaracterizing the DKE as something akin to the following graphic:
It is correct to say that intellectual humility--acknowledging one's limitations--is an important part of wisdom. However, the best explanation for the DKE is that it applies to distinct fields of skill or knowledge, and should not be classified as "overconfidence in oneself." (As a side note, another area where many people incorrectly apply the principles to individuals rather than to populations is evolution. More on that in other posts!)
It is also correct to say that the underlying cause of the DKE is lack of insight into oneself, rather than lack of insight into others. However, again, this does not excuse the need to be sincerely interest in others and learn about how they work. I often see a personal overconfidence paired with an unwillingness to listen to or acknowledge the validity of the other person's point of view. Just because a major aspect of the other's view is wrong does not mean that every aspect of their view is wrong.
Given the previous correct statements, other academics have still found alternative mathematical explanations for the DKE. Some studies classify the effect under regression toward the mean. Another explanation, quite interesting to me although I didn't take enough math classes to completely follow the reasoning, is Nuhfer et al. 2017 (linked above).
This study used randomly generated data to mimic the DKE. It confirmed that, contrary to most representations of the effect, poor performers can accurately estimate their own absolute performance, but can't accurately estimate how well they do compared to others. Additional findings, based on the high noise-to-signal ratio of self-assessment assessments (imprecise data), include that self-assessment abilities do improve in experts and women versus novices and men. Overall, though, people do self-assess fairly accurately how they compare to the norm.
Overview of Recent Literature on the DKE
Because the DKE is classified as a psychological phenomenon, I chose to use Google Scholar rather than PubMed as my base database for looking for review articles (literature reviews, systematic reviews). PubMed is associated with the National Institutes of Health, which has foci much broader than psychology alone--Google Scholar was likely to have more relevant literature in the first pages of search results without too much sifting.
While I usually advise my students that Google Scholar is most appropriate as an initial database to see what kinds of articles are extant on a given subject, I judged it as a good choice for supporting this blog post because it is designed to show articles that anyone can access, at least in part. If you are a Chrome user and install the Unpaywall extension on your browser, you will be directed to legally accessible PDF versions of articles that have such versions available.
There are hundreds of review-type articles on the DKE, its applications, and its alternative explanations, most of which likely use the same underlying individual studies. (Here's a link to the evidence hierarchy that I reference in my courses to help you see why I prefer review articles. Note that expert opinion, which includes many podcast interviews, blog posts, and similar types of information, is at the very bottom of the pyramid.)
I selected 5 systematic reviews that looked particularly interesting. Here are their summaries.
- Mahmood 2016 - in a systematic review pertinent to librarians, the DKE severely affects people's information literacy skills versus their own self-rating. Side comment: One of the skills I and other faculty emphasize in the DPT program I teach in is evidence location and appraisal. This article emphasizes the need for ongoing personal guidance of students as they practice finding and using literature in their projects and coursework.
- Huang 2013 - in the peer-review process, the DKE is prevalent even among relative experts in the literature on a given submitted manuscript. Knowing what one doesn't know is a description for metacognition. Side note: I participate in peer review for several manuscripts every year; I certainly feel the unsurety even when the topic of the manuscript is entirely within my expertise and knowledge of the current published literature!
- Celine et al. 2023 - many beauty bloggers who post recipes for their homemade cosmetics aren't qualified to do so and thus spread misinformation. Side comment: the more qualified one is, in my experience, the more likely one is to reference others' work--such as published journal articles--more than one's own.
- Sanchez & Dunning 2023 - people with "just enough" science training don't answer "I don't know" when they should still be doing so. Side note: I think the reasons for this go beyond the DKE. See note on intellectual humility, above.
- Gabbard & Romanelli 2021 - at least in pharmacy education, student perception surveys are an unreliable means of measuring true learning. Use competencies instead. Side comment: I strongly agree with these results, especially since the university I work for is recommending a move away from student experience surveys.
How the Bibliovore's Wife Succumbs to the DKE
Most obviously, my previous blog (The Renaissance Biologist), especially pre-2013, showcases numerous areas in which I was overconfident in my own knowledge and the rightness thereof. I've been prone to bolder-than-warranted assertions in (hyper)conservatism, libertarianism, young-earth creationism, and theology in general. Thus far, I've been able to avoid some areas of blogging in which I see a strong DKE, especially the narrative in the healthy-living niche that labels all synthesized substances as "toxic chemicals." This tendency irritates me because (1) I have a chemistry minor and thus more chemistry knowledge than most of the labelers, (2) everything in the material world is made up of . . . chemicals, and (3) any chemical can be toxic at the proper dosage.
Currently, I like to think that I'm not as prone in as many areas to the DKE. My stubborn streak, though, is still learning how to graciously lose an argument, withhold an opinion, and figure out where it's realistic for me to learn more. Thus, in topics where the Bibliovore is better versed than I, or in areas where my students do know more though I refuse to admit it, or simply when Toddler expresses a valid but different opinion that's not on my timetable, I will still tend to think that I know better than the other party in the discussion.
Comments
Post a Comment