A few months, my coworkers and I were chatting around the lunch table about the merits of various desserts. What’s the best ice cream flavor? Does bubble milk tea count as a dessert? When the conversation got serious, we drew up a March Madness-style bracket to find the winning dessert.
The details of the bracket aren’t the main point of this post. If you’re curious, you can find the completed bracket here. What matters is that in the finals, we had chocolate chip cookies versus New York cheesecake. To make the final voting data-driven (and delicious), we sampled both options.
But I wasn’t just going to make some chocolate chip cookies. I used the audience to taste five batches and find the ultimate chocolate chip cookie recipe.
Methods
Cookies are best fresh. Some cookies age better than others, but they all tend to get worse with time, and that can affect taste significantly. For a fair comparison, I wanted to bake the cookies all at the same time.
This actually ended up being somewhat tricky because different cookies have different timelines. Some are baked immediately. Some need to rest. Some should be baked from room temperature, while others from the fridge.
I did my best to follow the instructions for the recipe as given. I made the dough over the course of the week, rested them in the fridge as much or as little as instructed, then put them into the freezer. Two days before serving, I pulled out one baking sheet’s worth of each batch to defrost in the fridge. The night before serving, I pulled the dough balls out of the fridge to match the schedule needed.
Complications and variables
So what’s the difference between recipes anyways? Besides the ratios of ingredients, common differences in recipes include:
- Size. The largest cookies were more than twice the weight of the smallest
- Chocolate. Most recipes just call for chocolate chips, which usually means semi-sweet. However, some recipes call for chopped up bars, with big differences in percentages
- Brown butter. I thought browning butter was a pain, but many recipe authors swear by it
- Resting. Some authors insist that resting dough improves the flavor. Some just bake the dough immediately after mixing
- Salt. Sprinkling salt on top of cookies is trendy these days
And those differences and more also made some recipes much harder than others to make. However, I was generally faithful to the concept, and in a few hours of baking, I had made about 40 cookies.
Presentation
For the taste test, I cut up the cookies into roughly equal sized pieces, ranging from fours to eights. I provided each taster with a piece of paper on which they would write:
- Relative ranking (from 1-5)
- Absolute rating (from 1-10)
- Notes
We discussed during the tasting, and then afterwards, I revealed which recipe each number corresponded to and some comments on the composition.
And because more data is better, I did the tasting two more times with other tasters. It wasn’t much work because I still had the frozen dough balls. However, I did include one more recipe for a total of 6 recipes in consideration.
Here’s how they all compared.
The Unloved: Food Lab’s Chocolate Chip Cookies
I’m not sure why, but this batch was consistently ranked below the other cookies. Some cited the salt. Some cited the chocolate. However, neither of those factors were specific just to this recipe. Its only unique factor was the emphasis on a a caramel flavor.
As the baker, I found this recipe somewhat finicky. It calls for adding an ice cube to compensate for lost moisture with the browned butter, but I wasn’t sure how big an ice cube is. It calls for resting and tearing the dough balls for craggier edges. And it bakes very cool at 325F.
Finally, I baked these cookies twice, and the shape and spread was very different between each batch. I have no idea what happened.
If the results had been more popular, I might have stuck with it, but in the end, I don’t quite think it’s worth the effort.
The Dark Horse: Claire Saffitz’s Chocolate Chip Cookies
I thought I had made a terrible mistake with this recips. It calls for 18 minutes to bake, but by 13 minutes, the cookies were very, very dark.
My first hypothesis is that I had only baked one sheet. Saffitz calls for baking two sheets at a time, so when I made the second batch, I also put an empty sheet pan in the oven.
It didn’t help enough. I still pulled them out about 13 minutes in, and they were still dark (but not as dark). As such, I had low expectations for these cookies.
The ratings for this recipe were bimodal. Some were neutral, but the rest loved this cookie. The tasters liked both dark chocolate and crispy cookies.
Additionally, this recipe calls for browned butter and resting, so it also has a nice caramel flavor. The cookies are also quite large.
Nostalgia: Hershey’s Chocolate Chip Cookies
(These are the cookies on the left on the picture above)
The Hershey’s recipe is 1/2 tsp salt away from being the Nestle Tollhouse recipe, so it’s the most famous homemade cookie. And if it’s the cookie you remember, it might well be the one you like.
It’s by far the simplest to make. It has no special ingredients or techniques and is baked straight from mixing. Surprisingly, it ends up making visually exceptional cookies. They are the smallest, and they’re so thin that the chips poke out from the cookies.
These cookies were also very popular. I think that these taste the most straightforward: whereas other cookie recipes were trying to get a caramel flavor or blend the cookie with the chocolate, this recipe just tastes like chips plopped into a generic cookie recipe. And that can be a good thing.
It isn’t my personal favorite, but as the baker, it’s hard to beat the convenience. If tasters are perfectly happy with this cookie, why work harder?
The Exemplar: Cook’s Illustrated Perfect Chocolate Chip Cookies
In isolation, all of the cookies look like chocolate chip cookies. However, looking at all of them together, these cookies really look like chocolate chip cookies.
They’re thicker with distinctive chocolate chips (because it actually calls for chocolate chips). They come in a bit heavy and are very chewy. The recipe calls for browned butter, dark brown sugar, and one whole egg plus one egg yolk. The browned butter is some work, but the rest comes together quite easily. In fact, it doesn’t even use a mixer.
The opinions on this cookie were all over the place. Some found it very “basic”. Everyone agreed that they were chewy. Taken together, though, the ratings were generally quite positive.
I personally liked these cookies a lot. I was surprised because in general, I prefer crispy edges, but I guess this cookie got just enough crisp.
The Crunch: Sarah Kieffer’s Pan-Banging Chocolate Chip Cookies
I stand by everything I last wrote about these cookies. It was nice having the extra practice. This time, I knew how to do the pan-banging and what to look for.
As expected by the requested technique, this cookie gets nice crispy edges to contrast with the inside. However, this cookie loses a lot with age.
Overall, this cookie ends up with a result similar to Claire Saffitz’s cookie, and yet, Saffitz’s got slightly better ratings across the board. Tasters identified the crispy edges and salt as positives.
Latecomer – Sarah Kieffer’s Perfect Chocolate Chip Cookies 2.0
When I was looking for the Sarah Kieffer’s pan-banging recipe online, I stumbled across her new chocolate chip cookie recipe. Despite having several chocolate chip cookie recipes in her cookbook, she has declared this one as the new favorite.
These cookies are straightforward to put together and generally versatile. She specifies mixing dark and semi-sweet chocolate, then using a spatula to flatten the top after baking.
The result is a cookie both crispy on the outside and gooey on the inside with a very classic look and shape. It sounds like a great type of cookie. However, it didn’t get great ratings from tasters. They identified exactly the desired qualities (crispy and gooey), but it just didn’t do it for them.
General observations
Before actually picking the best, I will first say that the cookies were all generally well-liked. Except for the Food Lab (scoring 6.375 out of 10), the rest of the cookies were all in the 7.something on a one to ten scale.
Based on the notes, I think there were just a few properties that would decide how someone voted.
First, dark chocolate. Some tasters loved dark chocolate, while others found it almost problematic.
Second, salt. Again, some loved the sprinkle on top, while others hated it.
Third, crispy or chewy. Or maybe both.
What didn’t tasters identify? Size (though I did cut them up), chocolate bars versus chips, caramel-y flavor.
Given all of those properties, I could make a case for baking any batch of cookies based on the stated preferences of the cookie consumer. I could also make some small alterations for my own convenience and better tuning.
Still, which recipe wins?
Picking winners
If I was baking for myself, I would bake the Cook’s Illustrated cookies. Oddly, I thought I was a crispy cookie person, and these cookies were large and chewy. As such, I learned something surprising about myself.
If I felt fancy, I would bake Claire Saffitz’s cookies. I still need to fix the bake, and I would use slightly less dark chocolate, but they overall worked really well.
But if I want to make some cookies, I would bake the Hershey’s/Tollhouse cookies. This feels like a cop-out and almost disappointing answer. However, tasters really like them, and they’re really easy to make. It’s just tough to beat the effort-to-payoff ratio.
Final thoughts
If you enjoyed this post, I recommend The Pancake Princess’s chocolate chip cookie bake-off. She goes into much more detail with more stats, comments, and recipes.
I love doing these taste tests and comparisons. It’s a fun way to just think more about what we eat. However, the results consistently have little impact. Most things that I make or eat are fine, and even comparing side-by-side, it’s often hard to pick a winner.
I’ll reiterate my takeaway from my year of cookies: if you mostly follow the recipe and don’t burn your cookies, people will love them. Don’t worry about what could have been: just enjoy the cookies.