Is Time Estimation in Software Engineering a System 1 or 2 Task?

Software engineers are notoriously bad at time estimation. When they receive a new bug report or product feature to work on, engineers are often asked by their project manager to guess how long it will take. It’s a very reasonable request. For example, if your website goes down, public relations needs to know how long it will take to put together a response. If you’re working on the next version of your app, product management needs to know what features are possible before the ship date. Continue reading “Is Time Estimation in Software Engineering a System 1 or 2 Task?”

The Effect of a Placebo-based Policy

(Disclaimer: I don’t actually know that much about medical ethics or public policy, so I welcome all comments to educate me on the issues here. Also, I will miss citing things because I’m a lazy blogger, so I recommend that if you’re interested, you do your own research)

The Placebo Effect is a well-known phenomenon where a patient actually improves even when given a non-effective treatment that  he or she believes is effective. The perception of efficacy alone not only makes the patient feel better, but actually makes them better.

This effect manifests in other, similar manners, where the perception overrides the actual sensation. For example, it turns out that people will enjoy (and report as enjoying) wine more if it is marked as being more expensive, independent of the actual wine itself. Though maybe if you’re a hipster cheapskate like me, you might perceive cheaper as being better, like a hole-in-the-wall Chinese restaurant.

In both of these cases, the subject is being tricked into something, but the effect on them is real. Given that, the big question I have is: do you think it’s ethical for doctors, marketers, and other professionals to use the Placebo effect?

Big Yak Mountain, the book club I am in, just read Dan Ariely’s Predictably Irrational. Ariely is a behavioral economist at Duke, and in this book, he describes several quirks about human decision-making that show how irrational we are. He discusses how uncertain we are in valuing things, to the point where we don’t even know whether we should pay or be paid for some experience. He talks about the gulf between free versus really cheap. He explains why I would happily help you move a couch out of the goodness of my heart but would be unwilling to do it if you offered me $5. Overall, he covers roughly 15 different topics, one per chapter, in an accessible manner. Although I think that Thinking, Fast and Slow is still better, I can understand why you might want to save yourself a few hundred pages and explanation of experimental methods.

When I was putting together notes for the discussion, the question I kept coming back to was, “Is it okay that we’re so irrational, or shoulwe be more rational, more economically logical, in reasoning?” From our discussion, we roughly agreed that these phenomenon are fascinating and good to know, but we were also completely satisfied to experience our blissful irrationality as it is. The bigger question, however, is how this should impact policy, and this is where I am going to do the very bad thing of generalizing narrow experimental results into my worldview.

I asked one policy question above, and I have a few more for you along the same lines. Here’s an easy one: should the use of “free” in advertising campaigns be regulated? I think most of us would say no, despite the fact that we’re totally irrational about free and will go way out of our way to get a free bagel to a degree far beyond that of a 1 cent bagel, even if the free bagel is just a way to lure us into a store to make a large purchase.

Here’s a more contentious one. Ariely spends a chapter on how irrational we are when aroused. If you’re unfamiliar with the study, I’ll save the joy of the experimental methods for when you read the book, but the result is that we act less rationally when emotionally aroused. Since people can be emotionally aroused by anger, one policy is a “cooling off” period between the purchase and acquisition of a gun to prevent people from doing something rash. What do you think of cooling off periods?

The last one is quite broad, but a big issue today is government surveillance. It appears that the NSA has been spying on just about everyone to a much greater degree than we thought. I’m a trusting guy and believe that this was done for our own good and not because someone is reading my phone meta-data to hurt me. If we momentarily ignore the ethics of the surveillance itself, we have been deceived about the activity itself. This deception sounds a lot like the Placebo effect, where an authority gave the perception of something to make us feel better and improve our situation, though they were technically lying at the time. Given my biased framing, how do you feel about the government deceiving us about surveillance for our benefit?

I haven’t quite worked through the details and nuances of this, and specific issues deserve more thought, but I personally am pretty liberal (socially and economically), and I think my belief in human irrationality (and by inclusion, the placebo effect) plays a big role in that. Simply, I, like most other people in this country, am just one person who can’t adequately grasp what is ultimately good for me, and to some degree, I need someone (and not just market forces) to be watching my back.

I need financial regulations because I have no idea what the banks are doing. I need environmental regulations because I don’t know what every supplier is doing. And even if I could, I couldn’t resist buying the cheaper detergent to make optimal purchases. I believe in universal health care because choice is good, but it’s a lot ask every individual to know how best to keep themselves healthy. And assuming that Congress is well-informed (which apparently has not been true so far), I’m generally pretty okay with government surveillance because there are things that I’m better off not knowing, and I trust an entity with the goal of protecting me more than businesses with the goal of being profitable.

So my wildly extrapolated beliefs based in science suggest that a stronger hand in policy is good for us. Of course, there’s always the looming threat of totalitarianism, but a healthy debate and oversight from our legislature should keep it in check. At least in contrast to current trends in libertarianism and small government, I would like to see government stay involved. Some people believe that they deserve complete transparency, that markets are self-correcting, that government should be limited, and that individuals should have tremendous personal freedom. I definitely believe in personal freedom on social issues where it’s more obvious (to me) how freedom affect one’s happiness. On the other issues, my understanding of human cognition is that given complete information and choice, people can still be quite bad at ultimately making themselves happy. And I think that public policy has that mandate as well.

“Thinking, Fast and Slow” Review (prefaced with ranting)

I have had a painful move out of psychology. Since I did research while I was still in school, I considered myself reasonably critical of experiments, basically fluent in psychological phenomenon, and quite up-to-date on recent findings. After I left school, I tried to keep in touch primarily through science writing, which I have actually found quite disappointing.

I’m generally disappointed by science reporting, and it’s entirely my own fault. Writers do report on recent findings, but I am rarely satisfied reading their accounts. I wonder how accurate their reporting is and whether they truly understood the paper. Even if they understood it and report it honestly, I wonder whether they have delivered a complete report of the paper. And even if they got all of that right, I wonder whether it’s properly contextualized in the body of knowledge and what else I’m missing. Of course, none of this is the fault of the journalist: they write for a wider audience, and they might actually have done everything correctly. It’s my problem that I don’t trust 3rd party reports. I would only feel comfortable having read the original paper and done the background myself, and I’m clearly unwilling to do that, or else I would still be in academia.

The other popular psychology writing comes in longer books or pieces from writers like Malcolm Gladwell or Jonah Lehrer and programs like Radio Lab. For awhile, I let these carry me, but the revelation that Lehrer self-plagiarized and fabricated quotes (see wikipedia’s references) made me re-evaluate this entire process, and I ended up being turned off by all of that as well. All of my above concerns are actually amplified here. The difference is that these pieces typically tell a “story” about what’s going on. A book or radio piece needs a compelling theme, and to do that, the writers pick and choose their science to fit their narrative, which can be quite misleading. Examples off the top of my head are Lehrer reporting on only 1 result from a paper and Radio Lab reporting on unfinished work.

So what’s left is science writing by actual scientists, and thankfully, I read Daniel Kahneman’s Thinking, Fast and Slow. Kahneman is a Nobel Award-winning psychologist whose work in heuristics and biases is taught in  Psychology 101 classes. In this book, he shares key findings in his research areas over his career. He starts out by describing the 2 systems of thinking, the “fast” (automatic) and “slow” (deliberate) from the title. From there, he continues into how our automatic processing compels us towards statistically incorrect and inconsistent decision-making.

Beyond his credentials, Kahneman presents the material in what I believe is a compelling, accessible, and most importantly, honest manner. The 2 systems of thinking are the theme to this book, but he uses them out of necessity and accessibility, not artificial unity, which he is careful to point out. He presents studies individually and offers some of the alternative explanations for the phenomenon. Although other writers may take broad themes from these results, he understands and limits his conclusions and lessons to the scope of the study.

Even better, Kahneman raises meaningful and thoughtful questions. Instead of leaving the reader with a sense of awe for the science and some vague speculation, Kahneman finds the actual limits of the science and discusses the impact of it on daily life to public policy.

The catch of the book is that it is quite long at almost 500 pages, and it can get quite dense. Despite that, Kahneman avoids the pedantic writing of true academia and mixes in real life lessons for the reader and uses anecdotes to enliven (but not prove) his points. If his Nobel Prize didn’t do it, his writing this book should convince you of how seriously and deeply he has thought about his research and its broad consequences for the rest of us. If you’re human, it’s worth the read.

Discussion Questions for “Alone Together”

For November, my book club* read Sherry Turkle’s Alone Together. In it, Turkle, a professor at MIT, investigates how our relationship with technology has developed. In part 1, she writes about robots, from tamagotchis and furbies to robots for elderly care, and how we interact with and feel about them. In part 2, she writes about how Facebook, texting, and online communities have changed our relationships with each other.

Since it can be heavy reading, we focused primarily on the second half of the book. In all, we were pretty neutral about the book. Although she had many good points, some of her examples seemed somewhat extreme, and we also felt that there was a lot of bias. I think we spent most of the discussion disagreeing with her, but she brings up some very interesting topics for discussion. Below are the discussion questions we worked from, which I think are interesting enough even having not read the book. I have, however, added a bit more context for it.

  1. Turkle writes about how we develop an online self  that is independent of our real life self. What do you think of that?
    1. Pete spends significant about of time on Second Life talking Jade, a sort of second wife. He notes that it’s easier to talk to Jade about some issues than his real wife because he doesn’t have to be concerned with Jade worrying about him like his wife would. Thoughts? What is it about the medium that makes this work?
    2. Adolescents seem worry a lot about their Facebook profiles and the sort of person that it portrays. This includes the obvious things such as profile pictures, but also minor things such as the order of artists in “favorite music”. Have you cared so much about your online presentation?
  2. Turkle suggests that thee digital world can’t offer the same opportunities for relationships that real life can.
    1. Is this a problem? Is there a place for this type of relationship?
    2. How does this work for adolescents or others who aren’t fully capable of developing relationships?
  3. We love multitasking because it makes us feel productive. We love technology because it allows us to be on-call at all times to respond to things. Neither of these, however, are working well for us. Multitasking makes us less efficient at all of the things we’re working on , and being more able to quickly deal with problems leads to more stress
    1. Are you okay with this change?
    2. What is it about this method of working that appeals so strongly to us?
    3. If you think it’s bad, how do we reverse this trend?
  4. Time is a big theme. Technology leads us to expect faster responses to each other. Technology also makes us expect things to develop quickly, whereas many things in real life (relationships in particular) happen incrementally over a long time.
    1. Adolescents today are growing up like this. How do you see this affecting us in the future?
    2. How have you expectations changed about response times with technology?
  5. Being disrupted sounds bad, but really, we love it. We check our email and Facebook obsessively for updates. We love the ping of push notifications like texts and emails, and it’s all instant gratification
    1. It may seem intrinsic to the technology, but products are designed to get us addicted to using it. Do ou think this is good?
    2. How does this change our reward system?
  6. The phone call is dying. These days, we don’t use the phone as much because we’re often not up to the demands of it. Many adolescents love the fact that texts can be carefully constructed, whereas phone calls
  7. because we can’t meet the demands of it. There’s a lot of talk about being able to control things better when it’s done via text, where you can edit your message exactly as you like.
    1. How important is it to you to be able to control the message you present when talking to others?
    2. Do you see phone calls as a lot of pressure?
    3. Is texting more or less efficient than talking?
  8. Another benefit to texting is that it’s asynchronous. Suddenly, we see phone calls as a possible inconvenience to others as we demand their attention immediately, and maybe they’re busy
    1. Before, we mentioned that we like being disrupted, though. How do we resolve this?
    2. Since when did we get so sensitive about bothering others?
  9. PostSecret and communities. The internet is supposed to be our liberation as free flow of information allows us to develop the exact communities that really fit us.
    1. Are these communities good substitutes for what we have in reality?
    2. Turkle seems to believe that these aren’t real communities in that you can run from the bad stuff. Is she right?
    3. What about authenticity in these? Is it okay that a good portion of PostSecret is fake? Or FML?
  10. Technology is supposed to make us feel better. We’re constantly connected so if bad things happen, we have a safety blanket. It also, however, makes us anxious as we’re tied to it
    1. Is this inherent in the design of these things?

From Marshmallows to Tracking in the American West

(Update 07/31/12: I just heard that Jonah Lehrer fabricated Bob Dylan quotes for “Imagine”, the book that inspired this post. Although I didn’t really rely on his work for this post, it seems prudent for you to keep that fact in mind while reading this. Let this be a reminder that science writing can sometimes be worse than misleading, and we should always maintain a sense of skepticism in reading.)

I’m currently reading Jonah Lehrer’s “Imagine”, a new book on what creativity is and where it comes from in the brain. He explores many, often contradictory, concepts and elucidates parts of creativity, which may be the quintessentially mysterious force of the universe. He talks about artists, such as Bob Dylan, who needed to retreat from a hectic tour to the middle of nowhere in Woodstock, New York to escape the musical world and reach his creative genius. He talks about academics and inventors, such as Paul Erdos, who needed amphetamines and caffeine tablets to focus intensely upon the problem at hand to be creative. Altogether, the book really is a great exploration of topics that can truly affect your approach to life, so even if you don’t get around to reading it, I recommend you listen to this hour-long segment on the local NPR affiliate where Jonah talks through many ideas and answers questions about creativity.

I’m a big fan of Jonah’s work, and as hipster cred, I was reading his blog, The Frontal Cortex, since before he published his first book, “Proust was a Neuroscientist”. His work is everywhere and is serious enough that I got a reference to it from my cognitive psychology class. We were talking about cognitive control, and psychologists love to use the marshmallow test as an accessible example of its importance. The test is simple: put a child in a room in front of a marshmallow and tell them that if they can resist eating it for 15 minutes, they will receive another marshmallow for 2 in total. Then, leave the room and see what happens.

They recently replicated the experiment, and you can watch a fun video of children squirming and sometime succumbing. Jonah had quite a good, albeit somewhat long, article in the New Yorker about the marshmallow test, where he goes into further detail for why people still care about a study done decades ago. Walter Mischel, who conducted the original test, surveyed the participants decades later, and discovered that those who could keep themselves from devouring the single marshmallow had higher SAT scores, better social skills, and a bunch of other presumably good traits. For a 15 minute test, it had incredible predictive power.

Upon analysis, they determined that the key was self-control. The ability to delay gratification and resist temptation was how those children avoided the trap of the marshmallow and kept up their study skills years later. Although these lasting effects from a young age may sound like a genetic predisposition, psychologists (last I heard) didn’t have a conclusive answer on whether self-control could be effectively trained as well. For the children in the test, the trick to success was some tactic other than sheer will. That might be imagining a glass box around the marshmallow or sitting on one’s hands.

A few years ago, self-control was the chic, super-finding in cognitive psychology that would make us all geniuses. All we had to do was figure out how to teach self-control. But psychology, being a common topic in popular science, has its own fads. Being a fad doesn’t necessarily make the finding any less true, but it may mislead the casual reader. For example, I think bilingualism was a psychology fad awhile ago: it’s still good for you, but it just isn’t the big new secret right now.

Right now, I think Jonah continues to lead the charge, and grit is up and coming. Grit is the fixation on a difficult goal, the will to overcome failure many times over, the constant push to learn against adversity. Jonah discusses it in “Imagine” as the flip side to classic notions of creativity. Exciting creativity is the moment of insight, like turning falling apples into a theory of gravity and burrs in clothing into velcro. Prosaic creativity is Beethoven rewriting the same bit of music dozens of times and Steve Jobs iterating through many designs of Apple products. It’s the latter case of single-mindedness where grit produces creativity.

So Jonah argues that grit is good for creativity, but he cites Angela Duckworth‘s work on it. She started in grit by trying to find the best predictor for the retention rate of West Point cadets through the first 2, very intense summer months of training. And as you might predict by my setup, grit best predicted success. It was a better predictor than SAT scores, self-control, school rank, leadership potential, and physical aptitude. These measures, which intuitively seem like the exact talents one needs to be successful, somehow don’t pan out as well as the ability to just “stick with it”, and further research by Duckworth has extended this finding into other domains.

This finding, at first glance, sounds like psychologists again telling us something we already knew. We’ve all heard the story of Michael Jordan getting cut from his high school team, practicing hard, and becoming the greatest ever. But even as we all agree, we still determine college admission by SAT scores, NFL draft picks by the Scouting Combine, and job offers from short technical questions in an interview. Clearly we need to refocus our society towards grit, and teach children to become grittier. Grit and self-control sound so similar; we should be able to roll it all into one.

But they’re not quite the same. Duckworth found a correlation of .66 between self-control and grit scores in National Spelling Bee contestants, which is strong, but not perfect. The difference between them is time. Self-control keeps you from reaching for an extra scoop of ice cream. Grit keeps you on your diet for years. It’s subtle, and I myself didn’t quite believe the difference until I reflected upon myself and saw them come apart.

I think I’m a pretty disciplined person. I don’t have any difficulty waking up with my alarm in the morning and don’t mind doing chores before it becomes critical. Like the children who best resisted the marshmallows, I use tricks to keep myself honest. My general approach is to be cynical about my future self. Today, I want to be productive tomorrow. Tomorrow, I’ll want to be lazy. To avoid falling into those traps, I put a lot of effort into tying my hands on things. I’ll make plans for a meal to force myself to go grocery shopping, put reminders all along my path, write extensive to do lists, or simply do things ahead of time. Overall, it works to keep me on-track day to day.

As proud as I am about that fact, it was a somewhat disappointing realization that I’m not a gritty person. I’m fortunate that I have been successful and happy with a lot of things that I do, yet I have let myself drop many passions after encountering adversity.

I was naturally gifted at tuba when my band director asked to consider switching freshmen year of high school, and I had a good run. I almost immediately got promoted to 1st chair in the top band and subsequently made region band the next 3 years, narrowly missing area my senior year. But then I came to college, auditioned for both the orchestra and wind ensemble, and made it to neither of them. I dallied around with jazz bass trombone, vaguely kept practicing tuba, and even played in the orchestra for a summer. But I mostly gave that up after those auditions.

In high school, I studied for the AP psychology exam out of interest and nailed that one. I came to college interested in psychology, only took my first psychology course junior year, then did research that summer. I applied for PhD programs the following winter, and was rejected from either 6 or 7 different programs. Now, I’m a software engineer who loves psychology but can’t really imagine enduring the same grad school application process again.

In a better life, I would be a grittier person, and even now, I feel as though simply being aware of this problem makes me better prepared to deal with adversity in the future. I know that this trait is a weakness of mine, and when things get tough, I just need to steel my mind with the discipline I have to do push through.

The close to this post is going to be somewhat awkward and may seem like a rationalization, but it’s mostly just rage at the half-truths of science journalism.

I gave you an amazing story about the importance of grit. Particularly, the West Point part had grit triumph over the classic winners (smarts and raw talent) as well as the recent incumbent (self-control) as the best predictor for success. Well, the study actually said that it was the best predictor for getting through the 2 months. The rest of the story takes a little more explaining.

In the study, Duckworth et al. measured performance not only on summer retention, but also on GPA the following spring and Military Performance Score, or MPS. MPS was aggregated from performance ratings on military-related activities, both academic and non-academic. The 3 different predictors used were grit, self-control, and Whole Candidate Score (or WCS, an acronym that unusually was not used in the original paper), which combined school rank, SAT scores, leadership potential, and physical aptitude.

As reported, grit best predicted summer retention, with self-control coming in next, and WCS being non-predictive. On GPA and MPS, however, grit was not the best predictor. WCS, which aggregates measures of aptitude, was substantially more predictive of these 2 measures, with self-control coming in 2nd on both (but not very strongly), and grit not correlating for much.

This isn’t a deathblow to grit: there are several other studies in the paper that demonstrate the importance of grit, and nothing said so far is untrue. The point here is that it’s important not to compare apples and oranges, and if you do, not to overstate what the results mean. Grit predicted retention because that’s what grit is: it’s hanging tough. WCS predicted MPS and GPA because that’s what WCS is: a weighted measure of past GPA and other performance ratings. Whether one result is a better definition for “success” is beyond me, but it’s important to know what the science says.

This post has wandered a lot, but I guess the short version is to be gritty, but don’t get too excited about it. It’s a funny feeling for me to find out that it took science to convince me to be a better person. Although I had questioned my persistence before, I only really believed it to be important when I read a paper that gave me specific results. I guess the self-help section dominates the popular psychology section in bookstores for a reason, but I think I’ll keep telling myself that I’m just in it for the science.

Project Presentation for Natural Language Understanding

If you’re interested in what I’ve been doing for classes this quarter, I recorded myself rehearsing a final presentation for CS224U, Natural Language Understanding. I feel like I really fly through this material, but it might give you a sense of my work.

Embedded below, and also a link here. And if you’re interested in the code and results itself, it’ll all in github.

Using the Web to Make Academic Work Useful

For several stretches of several academic school years, I have allowed my class-related work to become my blog content. Sometimes it’s more natural, such as the final essay for Creative Nonfiction, and sometimes it’s less natural, such as short critiques for Moral Philosophy. Most of my motivation for posting this work is pure laziness: it’s really hard to will myself to write a blog post after having worked through an essay. A smaller point, which is the crux of this post, is that it seems a shame that I should spend so much time on classwork that will ultimately be seen by only one grader.

Not all classwork is valuable beyond its own context. Mechanical math problems and proofs of known results are obvious examples of classwork for its own sake, so I hope you won’t be offended if I avoid posting addition and multiplication worksheets. A lot of other classwork, however, emphasizes critical thinking, synthesis, and creativity in research and projects.

I’ve tried to make most of my original and less embarrassing writing available on this site, either in blog posts or on my Writing page. Despite its pedagogical purpose, classwork can still be original and contribute to knowledge as a whole, especially given how sparse some of the content may be. For example, Google Analytics tells me that my essays and responses for Moral Philosophy are some of the most popular content from google searches on specific philosophers and philosophies. Posting this content is cheap and easy for me, and it may be extremely valuable to anyone else who ends up researching similar topics to those papers.

Even so, most of my work has been relatively simple, and I have often been frustrated by how difficult it can be to find similar resources for some actual published papers. Many researchers have released open frameworks for their work, but more often than not, the details aren’t available. Datasets, stimuli, program code, and all sorts of other work are poured over by researchers for months and sometimes years, yet are basically forgotten after being summarized and presented in a paper. I’m not a full-fledged researcher and don’t understand most of the logistics, red tape, and politics that probably drive most of the reasoning behind the process, but in the pursuit of knowledge, it only seems right to make as much known as possible.

Along those lines, I’ve taken that first step and released several of my projects on GitHub, where you can view much of the code for research I’ve done, along with some results and write-ups. As you might expect, the code is something of a mess, though should anyone want to use or understand it, I would be happy to clean it up. In all likelihood, the repositories will likely sit on the web, unseen and unimportant, but for how much I complain about not being able to find things, I can at least say that, “I tried.”

For many of my peers who have also worked on various projects, I recommend that you do the same. I’ve seen some really impressive work come out of class projects, and it would be a shame for that to be the end of it. And use it for current projects as well. Should you be doing any coding or research, you should be using a version control system anyways, so you might as well make it publicly available as well. In academia, we’re always all collaborating with everyone.

4 things I learned at Bing

This past quarter, I spent one morning every week in a classroom at Bing Nursery School, basically working as an assistant teacher for a class I was taking. The class focused on the development of young children (3 to 5 years old), and our journal entries (posted on this blog) and weekly discussions were structured around our experiences in the classroom. Going into the class, I only cared about the children’s cognitive development: their ability to make inductive leaps, learn new skills, and progress towards literacy. I quickly learned that there’s a lot more to school than just how children think: they also interact with each other, develop emotionally, and become a complete person. As a teacher, I learned a few techniques with dealing with different situations. Some are a little cliche, others depends on the naivety of children, but all are good to know for children, if not people in general. I don’t know whether these are really useful in real life, but you can kind of see where I think the extensions are.

1. Don’t pose a question if you don’t intend on giving them a choice.

It’s typical politeness, I think, for people to pose orders as questions. “Would you open the door for me?” “Can you grab that book off of the shelf?” It turns an order into a request, and you expect them to be polite enough to follow through with it. Often, it actually is a request, but in a classroom, there’s a hierarchy, and orders are meant to be followed. When posed as a question, however, an order might be declined. “Would you like to clean this up?” “No.” Well, there’s not much left after that.

Instead, give a limited choice. Children live in a controlled environment, and they do appreciate having choices or power*. To appeal to this and make them far more compliant, give a smaller choice. “Would you like to walk to the toilet, or would you like to fly to the toilet?” It sounds silly, but it totally works. At the end of the day, we needed to clean up the blocks, and the children had built 2 prisons. Instead of telling them to clean up directly again (they had already been told), I instead asked, “Would you rather put away this big prison first, or the small prison first?” Things got cleaned up quickly.

2. Don’t give judgmental feedback on creative activities.

Bing is an unstructured, play environment, so children are free to move between activities. Within those activities, many are creative in nature, and even in more structured activities, they’re allowed to make what they want of it. We’re very careful never to assume or judge a painting or other piece of art. I think the classic joke is a child showing a parent a painting, and when the parent says, “What a nice cat!”, the child responds, “It’s a dog.”

Admittedly, it’s sometimes very hard to tell, and frankly, not everything is good. To tell them, “Your picture is amazing!” or something positive like that sends the wrong message. In creative activities, they aren’t working towards receiving praise; they should be working towards developing their own abilities and creativity.

Instead, make observations about what they have accomplished. Since Bing is so focused on developing skills, you can absolutely respond to what they show you, or engage with them during creation, by discussing what they’re doing. “I noticed you blended the colors along the edge here.” “Your circles are rounder here than they are over here.” “You used a lot more finer lines around the head here.” This way, the children are attending to what they’ve done and the specific techniques that they can compose into a creative piece.

3. Always explain why.

Schools have a lot of policies, and Bing’s #1 policy is to always be safe. Different teachers have different comfort levels for what the children do, but when they hit a boundary, we have to be firm, but also explain why. We don’t want children running inside because it’s crowded and there are lots of things they may break or hurt themselves on. We don’t take toys that other children are playing with because it’s not a sustainable way to play, especially if it happened to the taker his or herself. And for the teachers, it’s a good reminder why policies and rules exist.

An interesting extension of this is that rules don’t always have to be consistent in all situations. Many books recommend absolute consistency, but frankly, the world isn’t consistent, and rules exist in context. For example, running is bad above for the reasons mentioned, but is fine outside in the grass area. The children seem to do pretty well with these rules as long as they have a reason.

4. Don’t worry about screwing up.

In the first 2 weeks, we got lots of suggestions on how to deal with the children, and most of them were “don’ts,” not “dos.” I myself ended up being very paralyzed by this and slowly needed to recover a more natural interaction with the children. When I mentioned this to one of the instructors for the course later, she recommended simply that I not worry about it.

A lot of what I posed above are also “don’ts,” but frankly, most of the time, it doesn’t matter. If you say that a sand castle is “very nice,” you’re not going to permanently hurt the child. If you can’t resolve a conflict between two children or tell a child something that isn’t true, it’s fine. They’ll get over it, and it’s life.


So those were some of the things I learned. It was a great experience being able to discuss techniques and topics, then immediately try them out in the classroom, and I appreciate the opportunity to do that. For any Stanford students, psych 147 is highly recommended.

*in early education, people seem to agree that boys pretending to play with guns is a manifestation for this need for power; I don’t entirely know how I feel about that

Does Theory of Mind Emerge in Pretend Play?

(This is the final paper for the class that I have been writing all these journal entries for. It’s probably a little spotty, and I’m not really proud of it, but I think it hits the highlights in the literature)

At Bing, children as young as 3 regularly engage in pretend play, and the school’s philosophy and design encourage it. In west room, a kitchen area lies in the middle of the classroom with toy babies, costumes, and various utensils. Outside, playhouses are equipped with pots and pans, sinks, and brooms where, on most days, children will be outside making mud pies out of sand and water. Sometimes they play alone, but often they’ll play with other children, requiring coordination and maintenance of their play. In their pretend play, children pretend to be in fictional worlds and act out events. They might think of themselves as airplanes, use sand as flour in cooking, or play as a family with parents and a baby. Through play, children may engage in social negotiation, work through emotional problems, reason through complex situations, and more. One particular skill related to pretend play is theory of mind.

Theory of mind is the ability to impute mental states to the self and others (Premack & Woodruff 1978). Consequently, reality alone doesn’t determine one’s mental state, and different people may have different representations of the world. Previous research suggests that children don’t establish this skill until they are 4 or 5, yet children in engage in pretend play as young as 3. Children may pretend that a stick is a gun, which requires a conception of an object than one determined strictly by reality. In play with others, children interact with the imagined worlds that others have in mind. Pretend play seems to require skills similar to theory of mind, yet their development doesn’t occur simultaneously.

Although some argue that children have already developed theory of mind before engaging in pretend play, others argue that children only exhibit a primitive version of theory of mind in pretend play. Bing’s philosophy of play as learning believes that pretend play allows children to practice and learn about theory of mind. In this paper, I will explore some of the issues and competing theories in how pretend play and theory of mind are related and the evidence for these claims.

Experimental paradigms for theory of mind

Before discussing the competing theories, I want to give an overview of common methods of testing children for theory of mind and point out some of characteristics of these tests.

First, the appearance-reality task tests whether children can distinguish between the appearance and true nature of an object. For example, a child is presented with an object that looks like a rock. After the child states that it’s a rock, he or she then learn that it is actually a sponge. Studies have found that children younger than 4 ½ would respond to the question, “What does it look like?” with sponge more often than rock. This result seems to show that young children can’t maintain distinct mental models for the true identity of an objects and its apparent appearance.

A more recent study by Hansen and Markman (2005), however, discovered that this result came from task demands. Children often have difficulty understanding and remembering details in experiments, and in this case, questions about what an object “looks like” can be understood as a reference to reality as well. After correcting for these difficulties, they found that children as young as 3 could distinguish between appearance and reality. Thus, by this measure, children demonstrate theory of mind by the time they are engaging in pretend play as well.

Second, the false belief task tests whether children can maintain a counterfactual state of the world. For example, a child is presented with a Band-Aid box. After the child states a belief that there are Band-Aids inside, the experimenter shows that crayons are in the box. When 3 year old children are asked, they report that their mother would know that there are crayons inside. This result seems to show that the 3 year olds don’t understand that others can have a mental model of the world that’s inconsistent with reality.

Third, the interpretive diversity task is related to the previous task. The same child is also asked what he or she thought was in the box before, and these same children will fail and say that they thought crayons were in the box. This result seems to show that they can’t reconcile their previous mental state with the facts of reality and revise their memory for one consistent interpretation.

These particular tests don’t necessarily measure the same type of theory of mind that we see exhibited in pretend play. Even so, several of the following studies discussed will use and build upon these tests.

Do children see pretend play as theory of mind?

In the introduction, I noted the similarities in the characteristics of pretend play and theory of mind. Even so, the standard measures above indicate that 3 year olds haven’t developed theory of mind yet. A possible explanation may be that children don’t treat their pretend play as having emerged from mental states.

A key factor of theory of mind is that a person must understand that his or her actions come from a mental state. One interpretation is that pretend play gives children practice with these mental representations, which Lillard (2001) calls the Metarepresentational Model of Pretense. A common method of testing understanding of pretense is to describe an actor with contradictory mental states and actions, then ask the child which corresponds to the actor’s pretense. Lillard considered the role of knowledge in pretend play with several experiments. In one, a child watched a doll jumping up and down like a rabbit but was told that the doll knew nothing about rabbits. When asked if the doll was pretending to be a rabbit, 4 and 5 year olds correctly said no, while 3 year olds said yes. This suggests that 3 year olds don’t treat pretense as a product of a mental state. Instead, their interpretation from reality alone fails to account for this ignorance on the part of others.

However, other studies have disputed this and similar claims, mainly on the difficulties of task demands. Some concerns are that the relative saliences of the mental and real states aren’t balanced, and that the wording of the questions has some bias. Custer (1996) responded to the above study by changing the paradigm by presenting the pretense and asking about the mental state (instead of presenting the mental state and asking about the pretense). For example, a child was shown a boy pretending to fish with a boot, then asked whether there was a boot or a fish on the end of his hook in his mental state. The results showed that 3 year olds did better on this pretend task than they did on a corresponding false belief task. Given that, children show more advanced understanding of the mental nature of pretense than they can on typical theory of mind tasks.

Lillard (2001) pointed out that this result may only demonstrate that children are aware that pretense and reality are different; knowing that, they simply pick the alternative, which happens to be the mental state. Instead, she proposed the Twin Earth model of pretend play, where children are in an imagined world similar to, but not precisely the same as, the real world. In this theory, children haven’t necessarily developed a mental representation of the world of their play and only realize that their pretense is something other than reality.

This debate about whether children actively use mental representations in pretend play (and are simultaneously developing theory of mind) continues as no study has conclusively argued in either direction. Common difficulties with these studies are the numerous other interpretations of children’s behavior and other possible cues that may lead to it. Even so, further research should work towards where causality lies between pretend play and theory of mind.

How does pretend play correlate with theory of mind?

Even if we can’t explicitly link theory of mind to pretend play, we can try to find correlations between them. Taylor and Carlson (97) correlated theory of mind and pretend play in a study of 152 children, 3 and 4 years old. For theory of mind, each child was measured on false belief, appearance-reality, representational change, and interpretive diversity tasks. For their level of pretense, each child and their parents were interviewed on the presence of imaginary preferences, and each child was tested for preferences in toys. From these measures, children were grouped into high and low fantasy groups, which roughly described their engagement in fantasy in play.

They found that high fantasy correlated significantly with ability on theory of mind tasks for the 4 year olds, but they didn’t find any such effect in 3 year olds. One explanation offered by the authors is that the measures used weren’t appropriate for 3 year olds, a common difficulty discussed earlier. Still, I noticed other issues with this study. One is that the measures for pretend play were indirect: instead of an ecologically valid method, such as observing children in a classroom, the preferences were determined in experiment rooms without actual play. A second issue is that children were tested individually, which avoids any social factors in play. Even so, the correlation in 4 year olds is significant and suggests that these abilities aren’t entirely disparate.

Given how broad the types of pretend play are, we can better understand the importance of pretend play in theory of mind by considering specific aspects of play. One particular factor is how children play together. To construct a fictional world with others, children must form some idea in the minds of others, and working flexibly with those representations helps to perpetuate play. For example, I once observed Cyrus and Jackson in a tree playing “clones.” Although they were fighting other children, Cyrus also found “clone berries” that he had picked off the tree, giving them to Jackson and me to eat. In the middle of our meal, however, several children came nearby, and instantly, Cyrus used the “clone berries” as “clone grenades,” and Jackson immediately understood and began throwing them as well. When Olivia came and wanted to join their play, Cyrus made her a princess and assigned her the role of collecting the very same “clone berries” that he was using as grenades. At this moment, Cyrus wasn’t simply pretending the berries to be something else; he was actively maintaining 2 different fictional identities of the berries for 2 different people who each construed them to be different things.

A study by Schwebel, Rosen, and Singer (99) considered the social factor specifically in how pretend play might lead to development of theory of mind. Although most of the discussion above focused upon the role of mental representation in pretend play, this study focused on the difference between social and solitary pretend play. They observed 85 preschoolers, coding and rating their ability in play. They then tested them on the appearance-reality task. They found that, even after controlling for verbal intelligence, social play correlated positively with performance on the appearance-reality task. We discussed the limitations of the appearance-reality task above, but controlling for verbal intelligence helps to mitigate: better verbal intelligence should have accounted for difficulty in understanding the task demands.

The first result that solitary play didn’t correlate with theory of mind suggests that simply construing objects as something different than reality doesn’t effectively indicate any awareness of mental states. Though this theory alone isn’t surprising, it’s interesting that this came from the appearance-reality task as the measure for theory of mind. The result from social play, however, is more encouraging, and it places special importance on this type of play. Overall, this study suggests that specific properties of pretend play are more useful than others in developing theory of mind, and pinpointing those may help us understand what this relationship is.


We have discussed some of the competing theories and evidence to understand how theory of mind and pretend play might be related. Pretend play seems to show some of these skills in understanding the mental representations of others, and although correlations between these have been found, the direction of causality still isn’t known. We hope that pretend play encourages children to learn theory of mind, but it’s also possible that theory of mind is a prerequisite for pretend play, and studies so far simply haven’t tested children in the correct manner. In spite of this uncertainty, pretend play offers the most complete experience children receive in flexibly using these skills, and the earlier emergence of theory of mind-like abilities in pretend play can help us understand how it develops.


Custer, W. L. (1996). A Comparison of Young Children’s Understanding of Contradictory Representations in Pretense, Memory, and Belief. Child Development, 67 (2), 678-688.

Hansen, M. B., & Markman, E. M. (2005). Appearance questions can be misleading: A discourse-based account of the appearance-reality problem. Cognitive Psychology, 50, 233-263.

Lillard, A. (2001). Pretend Play as Twin Earth: A Social-Cognitive Analysis. Developmental Review, 21, 495-531.

Premack, D. & Woodruff, G. (1978). Does the chimpanzee have a theory of mind? Behaviour & Brain Sciences, 4, 515–526.

Schwebel, D. C., Rosen, C. S., & Singer, J. L. (1999). Preschoolers’ pretend play and theory of mind: The role of jointly constructed pretence. British Journal of Developmental Psychology, 17, 333-348.

Taylor, M., & Carlson, S. M. (1997). The Relation between Individual Differences in Fantasy and Theory of Mind. Child Development, 68(3), 436-455.

Journal Entry 8 for Development in Early Childhood

(This is my last journal for the class)

This week, I was in the language area, but with low interest in writing stories and having Christian nearby as well, that ended up being a small part of my actual focus as I drifted around. Instead, I want to focus on a longer interaction I had with A..

I was initially in the music room looking at the chicks with C. when she said that she was going to go out to the monkey bars. At that point, A. asked me, “Do you know what the monkey bars are?” I was curious as to why he would ask me such as a question, since I would assume that he would think I must know, so instead of just saying know, I instead just asked him if he could tell me what they are. He explains in reasonable terms about what they are and how children interact with them, and we briefly discuss his own ability with the monkey bars. Although this initially left me confused, my interpretation now is that this was how A. was initiating conversation. Aware that C. would no longer be the focus of my attention, he picked up on the most immediate discussion point and threw it out there for me to engage with him on. This particular question was somewhat awkward as a starting point, but A. did demonstrate an ability to discuss in a context-sensitive manner. Forman points out that “…we need to make a shift in what we believe a question to be…,” and he continues to discuss one possibility that a question is “an implied failure to communicate” (79). Between adults, A.’s question would simply whether one knew what monkey bars are, but to A., this question had a much different purpose, being primarily communicative instead of focused on the actual knowledge. Moreover, it worked as we began to talk as we walked out of the music room.

A. was holding a photo from a vacation in his hand, and we walked over to the mailbox on the shelves, and he put it in. He tried to close it, but the flap flopped back open, and he soon realized that he couldn’t stand there forever to hold the mailbox closed. He first asked me how it closes, so I explained that most mailboxes have clasps on them. From there, he pointed out the one clasp on the flap, and I agree. He then commented that the clasp has nothing to attach to, so he put together all of the details to understand exactly why the mailbox wasn’t staying closed.

Next, he asked me why the mailbox had nails sticking out of the bottom of it. I honestly said that I didn’t know in the moment. He responded relatively quickly and told me that they’re to attach the mailbox to the wooden support below it. I was again very impressed by his ability to demonstrate this knowledge, though I’m still not entirely sure where his comment came from. First, it could’ve been prior knowledge. A pattern I noticed in his speaking manner was that he often asked “why” or “what” questions for things he asked seemed to know. Like how he began our conversation about monkey bars, perhaps those are the sentence forms that he feels comfortable speaking with. Second, he could’ve reasoned through it in the moment, which I think would be a great inductive leap to make in understanding the physical relationship between the parts and the purpose of the nails. In either case, he was able to carry on at a very high level of functioning.

From there, I saw an opportunity to scaffold his engineering talent and asked him if we could use anything to close it. He immediately said tape but then retracted it. When I asked him why, he pointed out that if it’s taped, we can’t get anything into the mailbox. When he floundered on other solutions, I suggested that maybe we can use string, so we went over to the craft area to get some string. Meanwhile, he still pulled two pieces of tape, and we returned to the mailbox to close it.

He told me to pull the string through, and I talked through with him exactly what he wanted, which turned out to be the string through a small hole on the side. Because of how the yarn is braided, I ended up pushing the first bit through, then allowing him to complete the rest of it. After that, he taped the flap anyways so that the flap remained partially open. Although I had had a particular solution in mind, he clearly still had his own vision in mind that he executed.

Later on, he said to me, “The string is just hanging there. Do we even need it?” On that point, I agreed with him, and he removed the piece of string. When I asked him how someone could put something into the mailbox if it’s taped, he simply said that one can just take the tape off.

Overall, I’m impressed by his ability to work through a particular goal. Although his method didn’t quite match up with what he had said, he continued to work towards it and think critically about the purpose of various objects present. From my end, I think I ended up being the most helpful to him in my confusion about his mental state and purpose and in my ignorance about how to proceed, so I feel like I did the right things if perhaps for the wrong reasons. My confusion drove me to ask him questions that demonstrated his thought process, which hopefully helped to solidify those. My ignorance gave him the opportunity to think and work through various problems instead of me just telling him what he needed to know or what he needed to do.

I don’t know if I can aptly distill my experience at Bing into words other than to say that it has been very good. As I mentioned early on, my interest in children is almost exclusively on cognitive development, but interacting with them has reminded me that there are other important parts of development as well, such as social skills and emotional maturity. Although my mindset is still to compartmentalize these ideas, there really is a lot of connection between these different topics that gives a better total understanding of where a child is overall and how he or she can best be supported.

I’m not sure if I had expectations of the capabilities of 3, 4, and 5 year olds, but whatever they were, they were shattered. Perhaps the only thing that I feel I had right coming in is that children are strangely competent and incompetent in various areas, sometimes at the same time on what seem like very similar things. The best example in my mind is their language development, which is in many ways very sophisticated, but is also often not at all translated in writing and reading. They can recognize the first letter of another child’s name, but not be able to spell or even write the letters for their own name.

I feel like I’ve grown quite a bit from this experience, if only to have bought in to Bing’s philosophy about a play-based education. Even beyond the ability to understand children better, both in academic and real-life settings, and use that in the future, the basic skills to engage with and help the children feel like they’re more generally useful in interacting with all people, adults or children. Although I don’t think I’ll ever go through conflict resolution in quite the same way with my roommate, I think the mindset and determination to really understand the motivation of others and express those very explicitly makes sense. These journals have caused me to think harder about why certain interactions happened as they did and try to do better in the future. After spending the past few years at college focusing so specifically on my own cognitive development in classes, this class was a good reminder that just like the children, I have other things I need to learn along the way, too.