How to overcome the ‘curse of knowledge’ and write non-fiction that readers will devour

Have you ever glazed over in boredom as a computer salesperson waxed lyrical about JavaScript, when all you wanted to know was the main pros and cons of Windows and Mac laptops? Or experienced the frustration of wading through an instruction manual for a new microwave or washing machine that leaves your brain reeling in confusion?

Both salesperson and manual writer fell prey to the ‘curse of knowledge’. This refers to the fact that once you know about a subject, whether that's coding, tennis or music theory, it’s hard to imagine what it’s like not to know it. It’s a common cause of misjudging audiences that impedes good communication. To produce compelling non-fiction you need to avoid it in your own writing.

The curse of knowledge was investigated in a famous experiment by a Stanford University psychologist. A group of people were divided into ‘tappers’ and ‘listeners’. The tappers were asked to choose a famous song such as Happy Birthday and to beat out its rhythm on the table. The listeners had to guess the song. The listeners guessed successfully 2.5% of the time. But the tappers predicted that the listeners would guess correctly half of the time. Why the huge discrepancy? When the tappers beat out their rhythm they heard the song in their heads and so to them the taps very obviously matched the song. To the listeners this was not at all obvious, but the tappers assumed that it would be.

If the curse of knowledge can influence people taking part in an experiment lasting just an afternoon, imagine how much it afflicts experts who have become intimately familiar with a field of knowledge over many years. If, as an expert, you write for a non-expert without understanding where they’re coming from, you'll leave your reader confused, frustrated and bored. They’ll never likely make it more than a few pages into your book before giving up.

The very things that help experts perform at a high level make it harder for them to understand the perspective of non-experts. As you practice over and over, certain parts of a skill become automatic. You no longer have to think about them. Experts work by feel and intuition and no longer have to proceed step-by-step as a beginner does. Once you’ve driven a car for many years, you no longer have to consciously think about when to release the clutch and when to press the accelerator as you did during your first few lessons. When a skill is so familiar that it becomes automatic, it can be harder to explain exactly what you are doing. Imagine trying to write down which muscles to expand and contract in order to take a breath or to walk a few paces? Even in fields such as chess or mathematics, which you might assume are all about the methodical following of a clear set of rules, proficient practitioners seem to magically spot patterns and work their way towards solutions using shortcuts and gut instinct.

The curse of knowledge often lies behind readers' complaints about cookbooks. If you're a chef, it's easy to assume people know what you mean by 'sweat the onions' or 'skim the stock'. But unless you tell them what to look for (that the onions should become soft and translucent, but not brown; that you should carefully lift off any scum or froth from the surface of the liquid with a large spoon), beginner cooks may be left floundering. Of course, it depends on the target audience; experienced cooks would have no problem with this.

The other consequence of the curse of knowledge is that experts often underestimate the value of their knowledge to their target readers. So much of what you know seems obvious to you, and you assume that it will be obvious to everybody else. But what is obvious to you, to your readers might be a valuable new insight or piece of practical knowhow that needs to be included in your book.

So the curse of knowledge leads people to overestimate what others know and to underestimate the value of their own knowledge. Both can seriously impede successful writing. Here are a few strategies for combating the curse of knowledge in your own writing:

Put yourself in the shoes of your readers

That garrulous computer salesperson likely didn’t even realise that they were speaking way over your head. The first step is simply to be aware of the curse of knowledge and to realise that it is a potential pitfall.

Then try to put yourself in the shoes of your target reader. If you’re writing a basic introduction to using a computer, what kind of things will your reader want to know, and what do they probably know already? A useful way of making this more concrete is to picture your target reader in the form of a real person – a friend, relative or colleague – someone who you think would be the typical reader of your book. What kinds of things are they likely to think are important in this area? What are they likely to know already? What kind of language are they used to? Then as you plan and begin to write your book, test your material by asking these kinds of questions to see what material needs to be included and what needs to be left out and how that information should be communicated.

Once you’ve written your book, it can be really useful to test it out on a reader. You might even use that friend or relative who you imagined as your ideal reader. Ask them to read the text and underline any sentences or paragraphs that confused or bored them. It can be useful to do this before you’ve finished the book, perhaps after you’ve completed a few chapters. The information that your test reader gives you can be a great way of tweaking your approach. When you’ve finished your book, working with an editor who is not an expert in your field can actually be an advantage because they are likely to understand the perspective of your audience more easily than an expert.

The key thing here is to keep your reader at the forefront of your mind. This is the difference between non-fiction that hits the target and that which leaves readers cold.

Focus on motivation and relevance

Facts, theories or practical advice mean nothing to a reader in isolation. Why should a reader care about the details of the gold standard, tennis scoring statistics or the chemical composition of rocks? Because they are essential to the story you’re telling, the case you’re arguing or the skill that you’re trying to teach your reader. Being clear about the motivation for the information that you’re including is essential. Otherwise readers will wonder why they are having to wade through a load of detail whose relevance they don’t understand.

It may well be that that motivation turns out to be a bit fuzzy in your own mind, in which case you need to work on refining and clarifying your line of argument to yourself. Being clear on this motivation then acts an essential filter for information that stops you from trying to include too much. In a non-fiction book the aim is rarely to be encyclopaedic, but instead to include the information that is relevant to your story and to your readers’ needs. It's not an opportunity to download everything you know about a particular subject.

Find missing steps

Experts have internalised the steps in their processes so completely that they no longer have to think about them, and this can make it harder for them to articulate the specific steps that a beginner needs to go through, whether it's understanding an area of knowledge or picking up a practical skill. If you’re writing a book that aims to teach knitting to beginners, begin by breaking down the process into steps. Are there steps within those steps that you have missed because you no longer have to think about them when you knit? What have you left implicit in your explanations? Think back to the kinds of things you found difficult when you first began learning.

Use jargon on a need-to-know basis

The overuse of jargon is a common symptom of the curse of knowledge. Experts and academics often spend a lot of time talking to each other and speak in a shared specialised language. But if they want to communicate with a broader, non-expert audience they have to shift into a new mode of expression that depends less on the jargon of their discipline.

Does that mean that you should never use jargon? Not at all. An important part of learning a subject is becoming familiar with its terminology. It would be cumbersome to write about tennis strategy without using basic terms such as backhand and volley. But there are a number of requirements for doing this successfully. When you introduce a piece of jargon ask yourself whether it is really necessary to the bigger story or skill that you’re trying to communicate. How does it relate to the broader motivation rooted in your story or argument? If it is necessary, then explain it using clear language that doesn’t depend on other bits of jargon. And remember that readers can quickly feel swamped by information and terminology. Even if they’re clearly defined, twenty new terms introduced over ten pages is likely to overload them.

Make use of examples

Experts often talk using abstractions that summarise and distill their broad knowledge of a field. Non-experts like concrete explanations. Can you find vivid examples that help to illustrate a concept or a new term? Analogies from everyday life that readers can relate to are a powerful way of demystifying a concept and showing why it’s important. Here the advice ‘show, don’t tell’, which is commonly given to fiction writers, can be just as useful to writers of non-fiction.

Leave a comment below to let us know how you get on.