15 min read
What You Will Learn in this Chapter
In this chapter you will learn about the process of developing criteria and how we need to pay attention to both communal and personal expectations about a topic. All evaluation is guided by a set of criteria or expectations, and many of those expectations are created by the community that engages with a topic. However, it is up to the evaluator to revise, apply, and explain those criteria in a way that is clear to the audience, even if the audience ends up disagreeing with the evaluator.
- Secondary sources
In sports fandom, there are a lot of debates. Debates about which team is playing better, which player deserves what award, how rules should be applied or changed, what a team’s uniforms should look like, and countless others. Sometimes these debates can be resolved easily or can achieve a sort of consensus among fans, but more often than not, fans will continue to disagree on what they think about their teams or sports.
One less serious debate that rarely reaches a consensus is about which athlete is the Greatest of All Time in their sport or even in all sports. The Greatest of All Time or GOAT conversation rarely resolves because fans typically don’t value the same things. Some fans watch sports for the excitement of a close game, while some want their team to crush every opponent and the outcome of the game is never in doubt. Some fans track stats while others respond more to well-structured narratives. In GOAT discussions, Fans have a tough time agreeing because they have different expectations of what a GOAT should be. In fact, more often than not, the GOAT debate is usually just as much about the criteria we use to define the GOAT as it is about who fits that criteria. This is true of other similar debates about the best album of the year, best restaurant in town, or best video game console. Creating good criteria for these conversations is almost a bigger task than deciding which person, album, restaurant, or console fulfills them.
So if criteria are so important to these conversations, what exactly are they?
When we talk about criteria, we often think about them as strict rules, and sometimes they are. For example, if an individual WNBA player scores the most total points across all games during the regular season, they win the “Scoring Title.” The same is true for players in the NBA. The criteria are simple: score more points than any other player that season. Or when trying to make a particular grade in a class, a student will usually need to earn a certain number of points to get an A or B. However, criteria are typically not this simple.
Criteria are more like expectations. What do we expect from the album of the year? What do we expect from the greatest basketball player of all time? What do we expect from a frozen waffle we reheated in a toaster or microwave? These expectations are usually formed collaboratively between individuals and the communities that care about whatever the criteria is about. So, in the case of athletics: fans, sports media, coaches, athletes, academics, owners, and others involved in a sport are the ones who have some control over what we expect from an athlete. Each person provides input in some way into what they think is important and then eventually a general set of expectations takes root.
But how can we tell what those general expectations are, especially if we’re new to a topic?
Criteria are communal
One of the key ways to discover criteria is to listen. We say listening here, we’re not just talking about what you hear, but of purposefully being aware of what people say, writer, or do about a topic.
For example, if you wanted to figure out what people generally expect from a good romantic comedy, you might start by watching a bunch of romantic comedies. By watching the trends and repeated tropes in these movies, you can start to see what people expect from these films in an indirect way. You can formulate a lot of useful ideas from examining these primary sources, or the actual subjects of the conversation, but it might be a slower process at understanding what fans of romantic comedies think is important. To get to that idea more directly, you need to go to secondary sources, or look at what people say about or in response to romantic comedies. You could read reviews, watch videos that present their top 10 romantic comedies of all times, or even engage in direct conversation with fans of the genre. Regardless of where you start, you want to figuratively cast a wide net and get a lot of different perspectives to really get a sense of what fans think makes a good romantic comedy and what makes a bad one.
In a similar sense, if you wanted to figure out with men’s basketball player is the greatest of all time, you could watch video of professional players since the creation of the NBA, but this would be a slow process and might still not give you as direct of a picture of what fans think makes a player the GOAT. So you would again need to turn to secondary sources and read articles, listen to sports commentators and analysts debate, talk to fans from different teams, etc. Through this process you would begin to see a clearer picture about what fans value and what they don’t. As you dig into secondary sources for any topic, you will want to watch for common trends. What ideas or people are mentioned frequently and which ones aren’t? What are common expectations and which ones are less frequent?
If you examined sources in 2021 about who the GOAT of men’s basketball is, you would at this moment see names like Michael Jordan, LeBron James, Kobe Bryant, Kareem Abdul-Jabbar, and Wilt Chamberlain mentioned more commonly than other players with Michael Jordan and LeBron James being the most common. Ideas that would typically come up are how many points each players scored, how many times they won Most Valuable Player, how many championships they won, and how much they changed the trajectory of each team they were on. Reputation can sometimes point you to what fans value about players, but it can also create blind spots. After all, most sports were not integrated for much of their history, so some of the best non-white players have not received the same amount of exposure even if they were superior to their more popular white counterparts. Because these non-white athletes played in non-white leagues or non-white teams that were largely ignored by the press, their reputations are less developed even though they may fit the criteria for being a great player. Many women’s teams still don’t received the same level of accolades or praise that their male counterparts received. For example, women were playing American football in the 1930s and 40s and leagues were beginning to form as the men’s sport was asserting itself; however, some administrators and politicians found the sport to be “too masculine” for women and shut down or banned women from playing the game. Even now as women’s leagues and sports are slowly gaining public sports, women’s football is largely unaccepted because of generally held values by fans of the sport and the United States about which sports men and women should play. There are many other instances of athletes being ignored due to their activism, their politics, or because a handful of powerful people decided they didn’t like the athlete/team/sport. So while paying attention to trends and commonly brought up names and talking points can give you a sense of expectations, these expectations and criteria are not neutral or formed in neutral environment. No set of communally formed criteria will be without some of the bias and historical prejudice of the community that forms it.
Secondary sources and community expectations can be an incredibly useful step in developing criteria, but they can have limitations which means we must also shape criteria from our own personal experience and analysis.
Criteria are personal
Even though criteria are formed collaboratively with others, we still need to personally decide how we want to use those expectations and where we want to change them or insert our own. As we discussed above, sometimes the criteria created by a community has prejudices or unfair expectations built into it which can mean following those criteria exactly would continue those prejudices. While criteria naturally update over time, we can also change them intentionally though we still have to be able to articulate why these new criteria are good ones to use.
For example, let’s return to the conversation about the GOAT of men’s basketball. As we discussed above, when reading sources, you will come across common names like Micheal Jordan and common requirements like championships. However, you still need to determine and explore how to approach those criteria and if you think they are being applied correctly. For example, Michael Jordan played for a team called the Chicago Bulls and after he joined the team, the Bulls had gone from losing most of their games to winning most of their games, and starting in 1990 Jordan and the Bulls won 6 of the next 8 NBA championships. LeBron James, on the other hand, has led 3 different teams to 4 championships, including the Cleveland Cavaliers whose only championship in their 51 year existence came as a direct result of James’s presence on the team. Kobe Bryant won 5 championships though he played for the Los Angeles Lakers, who had an established history of success. Kareem Abdul-Jabbar (6 championships) and Wilt Chamberlain (2 championships) also played for the Lakers for a significant part of their careers, but they helped establish the success of that team for players like Kobe Bryant to take advantage of in later years. However, if you base your discussion solely off of championships won, none of these players would even be considered as the GOAT. Instead, this would go to Bill Russell who won 11 NBA titles or Sam Jones who was on the same team as Russell and won 10. Both of these players should be considered in that conversation, but they are often overlooked because of how long ago they played (1950s & 60s) compared to players like Michael Jordan (1980s-00s) or LeBron James (2000s-20s). Thus, even this simple set of criteria conflicts in some ways from the common conversation around the issue.
You could stop there and consider only championships and numerical measures like wins and losses, but that would likely give you an incomplete picture of who is the GOAT. After all, an excellent player can lose a lot in a team sport while still being one of the best players of all time. Someone like US gymnast Simone Biles could fall short of winning a team gold medal at the Olympics because her teammates may have struggled to record high enough scores to win gold, but Biles could still win individual gold and be individually incredible. Or in sports like American football or soccer where teams are much bigger and require synergy between even more players than gymnastics or basketball, many of the players with the record setting individual statistics have only won few if any championships at the professional and college level. However, in an individual sport like tennis, wins and titles can be more representative of individual greatness. Tennis player Serena Williams has won more Grand Slam tournaments, the most competitive and difficult tournaments to win, than any other tennis player in the last 45 years. It doesn’t mean that another individual player could not have beaten her or even been better than her at times, but we can read her statistics a little differently than we would someone like Michael Jordan or Simone Biles who had to depend on teammates for some of their success. Thus we would need to figure out how to balance team accomplishments vs. individual accomplishments as we consider who the greatest men’s basketball player of all time is.
While all of these examples come from sports, these questions and concerns can apply to evaluating other things, even something as ubiquitous as fast food. Instead of considering teams vs individuals we might think or brand reputation vs specific locations or meals. How much does the reputation of a chain restaurant like McDonald’s effect how much we enjoy their food on a single occasion? Or In-and-Out? Or Whataburger? Or Starbucks? How much loyalty do we have to certain brands and how much does that make us excuse the shortcomings of their products? Does it matter more that our phones and computers all run the same operating system or that they all work exactly how we want them to individually? These and other similar questions are one we will need to grapple with as we create our criteria for anything, especially if we are going to argue that our criteria is reasonable. How much were communal criteria decided on by people who don’t share our experiences or needs? What do other people say about this topic, and how does that fit with what I see or experience? And alternatively, how much does my own personal taste change my criteria and where is there room for disagreement?
To form a clear set of criteria for even something as low stakes as the GOAT of men’s basketball, we would need to dissect and consider this and many other questions about what criteria to use in this debate. Even in instances where we take up the common criteria used by most people, our specific application of those criteria will require us to make choices about how to apply those criteria and how to explain that application to our audience.
When building criteria, you need to be able to clearly articulate what your reasons are for the criteria you have. You may never include all of those reasons in your writing, but having a clear vision of your criteria makes composing more straightforward. It can be much easier to compose an evaluation if you understand your criteria in advance as opposed to figuring them out as you go. Let’s take a look at one example of what that might look like.
To deviate a little from the sports metaphors, let’s instead talk about how we would determine the best pizza in town. You can approach this from a lot of different directions, but if we’re going to focus primarily on the food itself, usually the areas we would assess are: crust, sauce, toppings, and cheese. But those 4 areas aren’t criteria yet since they don’t give us an expectations yet, so we’ll need to determine what expectations we have for a good pizza in those 4 categories.
In the category of crust, people tend to prefer many different styles of crust, so making criteria like “crust needs to be thin and crispy” or “crust needs to be thick with air pockets like bread” would make the assessment too narrow. Instead, we should discuss a quality that can be assessed, something like flavor. Something like “crust should have a light flavor but not be tasteless” might work better for our purposes because it gives us a scale to work on. If the crust flavor is overpowering or absent, something has gone wrong and we can notice that fairly easily while eating a wide variety of pizzas.
We could use a similar format for the other 3 categories where instead of prescribing a particular kind of pizza, we look for qualities that extend across most types. We could assess how much sauce there is and if the sauce has a good balance of sweet and savory, we could ask if there is enough cheese to blanket the sauce and if it has enough flavor to be good without any other toppings, and we could check on if the toppings are fresh and mesh well with the other components of the pizza. Now our criteria have gone from the 4 categories of crust, sauce, cheese, and toppings to:
- Crust should have a light flavor but not be tasteless.
- Sauce should cover entire pizza but not be so much that it makes eating the pizza difficult. Sauce should have a good balance of sweet and savory.
- Cheese should cover entire middle of pizza and have enough flavor to stand on its own without any other toppings.
- Toppings of any kind should be fresh and should work well with other ingredients.
You can, of course, disagree with these criteria, but we can see how these criteria came into existence and how you could begin to apply them. You can also notice from the criteria how much they privilege balance and cohesion in the way the pizza is made. Nothing in these 4 areas should overpower or overshadow anything else so drastically that it distracts, but they all unify into a single concept, but you could make a good argument that pizzas shouldn’t be balanced in every area but should have toppings be the star. You might also give each criteria a certain weight, so a pizza with bland crust wouldn’t be that important if it excels in the other 3 criteria, but maybe a pizza with bad toppings would be awful no matter how good the other 3 components are. You could also add more criteria that’s more specific around texture, aesthetics, innovative use of ingredients, pizza boxes, cooking method, and countless other areas related to the experience of eating a pizza.
However you build the criteria for a pizza or any other topic, you need to make sure you clearly outline what your expectations are in a way your reader can understand, even if they will ultimately disagree with you. Or, to return to the GOAT debate around men’s basketball, your friends may not agree with you that Dennis Rodman is the GOAT, but they will at least understand why you think that.
- Pick one of your hobbies or things you do that isn’t academic. What are some of the expectations/criteria you have for that hobby/topic? How do they differ from other people’s criteria about that topic? Why do think that is?
- When curating criteria, it’s important to listen to a wide variety of secondary sources. How would you go about finding and determining which sources to read and take seriously, specifically about the topic you described in Q1 above?
- Find a source you disagree with and briefly describe what causes you to disagree with it. Are the criteria they are using poorly explained or somehow not the best for the topic? Do you disagree with how they apply their own criteria?