How Creativity is Measured—And Why It’s So Difficult

How would the world be different if people had tails?

It’s an open-ended question that doesn’t have a right answer. That’s why Marjorie Taylor, a psychologist and professor emerita at the University of Oregon, uses it to measure the creativity of children. The creativity of kids’ responses—which range from “pants would have holes in them” to “people would hold tails instead of hands”—are evaluated by adults, and those results in turn fuel research.

But, as Taylor will tell you, creativity isn’t a single set thing that you can measure by simply asking one open-ended question. Rather, creativity is challenging to evaluate uniformly, and psychologists have developed numerous different methods, all with their own advantages and drawbacks, to study the elusive, coveted quality.

The wide range of approaches to creativity—from psychoanalytic, to psychological, to neurobiological—generally reveals the diversity of the field, but has led some to describe it as “a degenerating research program,” as Mark Batey, a senior lecturer in organizational psychology at the University of Manchester, wrote in a 2012 article on measuring creativity.

Journalists (myself included) don’t always help matters. Boiling down a study on creativity to a headline potentially gives the impression that researchers have a universally agreed upon way of measuring creativity, and that this measure aligns with a reader’s own idea of what defines creativity. This isn’t solely an academic discussion. Today, as rote work becomes increasingly automated, creativity is being billed as a marketable professional skill across many industries. So, as individuals actively pursue methods to increase their own creativity, they should understand how researchers are defining it.

“When you’re talking about creativity, you have to understand you’re talking about a broad spectrum of abilities in different domains,” Taylor said. “It’s hard to measure.”

A recent literature review, published in December 2017, gives an overview of how psychologists have approached measuring creativity. Conducted by Sameh Said-Metwaly, a PhD student at KU Leuven in Belgium, along with professors Eva Kyndt and Wim van den Noortgate, the review looked at 152 papers published through 2016 and found that there was no common framework.

“Researchers generally agree that creativity involves the production of something that is both new and suitable for a particular purpose or use,” Said-Metwaly wrote to Artsy in an email. “In other words, creativity is the ability to be different in a useful way.”

But what does “useful” mean? If a researcher thinks it means utility to society as a whole, they may take a different approach to measuring creativity than another researcher who treats it as only existing within the context of a laboratory study, Batey noted.

Generally in psychology, there are four very broad aspects of creativity that researchers examine called the “four Ps,” and they deploy a host of different methods to do so.

The first “P” is the cognitive process element, which seeks to find out if you think in a creative way. The second is the personal element, which focuses on traits (like high energy or risk-taking) that are associated with creativity in individuals. Then there’s the product, which aims to find out if the ultimate outcome or solution by a person is creative. And finally, the press (or environment), which is meant to determine if the context in which something occurs is creative.

The cognitive process element was the most commonly studied element in the research reviewed by Said-Metwaly and his colleagues, appearing in 52.6 percent of the papers. Researchers looking into cognitive creativity typically use methods that test a user’s divergent thinking, which involves generating a range of possible solutions to a prompt, rather than a single correct answer.  

Among the most famous examples of tests to measure divergent thinking is the “unusual uses” test. How many uses can you think of for a brick? Or a milk carton? Researchers will often put this question (or a variation of it) to participants, take their answers, and then analyze them, looking at the number of answers (or uses), originality, flexibility, and level of detail.

While process tests like this one are broadly adopted and used in a consistent way, it’s easy to see their limitations. For one thing, they’re predicated on a very general notion of creativity, rather than how creativity functions in the context of a specific field (for example, creativity likely means something different, and requires different skills, for mathematicians and artists). Similarly, such tests don’t involve real-world situations, nor do they occur in real-world contexts. And kids, who are not used to such repetitive questioning, find the test particularly strange, Taylor noted.

“For a lot of studies, creativity gets boiled down to the unusual uses task,” she said, adding that while she uses the test, she’s not a huge fan. “It’s very narrow in terms of focusing in on creativity as manipulation of a physical object.”

Taylor prefers a test in which the researcher will start a story and ask children to create the ending: For example, you’re walking down a path in the woods when you notice a key on the ground, and a door, then what? Their responses are then rated by adults for their creativity.

This story completion test is broadly in line with product approach to measuring creativity, which is about outcomes. Researchers using this approach take a person’s product (a painting, or a mathematical formula, for example) and give it to a group of experts, who then rate it for creativity. The underpinning theory is that people in a specific field are best equipped to evaluate the extent to which something is creative.

The drawbacks here are that certain people who are considered extremely creative are not seen as such in their own time. Additionally, it can be costly to bring in experts, and the personality of judges can impact the outcome. Plus, sometimes they seem to get it wrong. One 1991 study, cited by Said-Metwaly in the literature review, asked judges, of various ages with differing experience with art, to rate the creativity of mass-produced art and art found in museums. In the end, they rated both categories as being similarly creative.

The person approach relies generally on identifying certain qualities associated with creative achievement (high energy, behavioral flexibility, intuition, emotional variability, self esteem, risk-taking, perseverance, independence, introversion) and then asking people about them, using self-reported surveys. These questionnaires are easy to administer and standardized, allowing for the data to be easily interpreted, noted Said-Metwaly’s literature review.

But of course, the presence of these traits doesn’t guarantee that a person is creative or adept at divergent thinking tasks. And traits that may be linked to creativity in one field may not be found among creative individuals in another field. A pair of studies, conducted in 1998 and ‘99 by San Jose University psychology professor G. J. Feist, and cited in Said-Metwaly’s review, found that while certain traits are linked to creative individuals broadly (like openness to experience, hostility, and impulsiveness), specific professions see some variance. So while creative scientists are self-confident, autonomous, and arrogant, creative artists tend to be anxious, imaginative, and emotionally sensitive, according to Said-Metwaly’s overview.

The least studied element of creativity, appearing in only 4.1 percent of papers examined by Said-Metwaly, is the environmental factor. This includes research into how organizational structure, as well as things like school environment, relate to creativity.

Ultimately, the goal is not to pick one of the “four Ps” and trumpet it above all the others, nor to devise the single way to study the creative element that a researcher wants to explore. As Batey wrote in his paper, it is worth developing frameworks that mesh the different approaches to studying creativity together, in order to get a more holistic picture.

Creativity, noted Said-Metwaly, is an extremely multifaceted quality. It makes sense that there’s not only one method of measurement, but he argued that more needs to be done to reach a consensus on what creativity is and how to measure it, along with more research into the domain-specificity of creativity in various fields. But broadly, one thing researchers are united on is the need to continue to examine creativity.

“Creativity is so important and so interesting and so ubiquitous,” said Taylor. “It’s really important to try and understand it scientifically.”