I Learned How Computers Work

Despite having worked as a software engineer for over a decade and received a graduate degree in Computer Science, I had a serious gap in my knowledge: I didn’t actually know how computers worked.

Of course, I exaggerate. I knew how logic gates and Boolean algebra worked. I knew that compilers were related to formal grammars. However, I didn’t really understand how keyboard or screens fit into the flow of how a CPU worked to run a program. I didn’t know how code actually got run as 1s and 0s. I just scraped by trusting the my high-level Python code would get interpreted and executed on hardware somewhere.

Seeing as this quarantine has been my opportunity to correct long-running embarrassments, I decided to do the nand2tetris as provided by coursera. Someone on hackernews had previously recommended it, and frankly it sounded like a good if ambitious idea. Could the class really go from logic gates to Java-like language in 12 weeks?

The Source of the Problem

This gap in my knowledge was a deliberate choice in my education. I avoided the two big systems classes in the major: CS 143 Compilers and CS 140 Operating Systems. These classes were difficult with time-consuming and frustrating programming assignments. Students would build their course load around these classes. Since neither sounded interesting to me, I graduated with a systems introduction class instead.

Although these topics are quite difficult, nand2tetris teaches compilers in two weeks and operating systems in one. Despite my skepticism, I started watching the lectures at 2x speed and steadily worked through the assignments and stuck to the week-by-week structure, neither falling ahead or behind by much.

And it was great. I haven’t taken a class in awhile, but I feel more knowledgeable and accomplished after this class than most I can remember. I have a few ideas for what this class did differently.

Vertical Integration of Knowledge

In my experience, there are two popular structures for classes: survey classes and depth classes.

Survey classes cover a variety of related topics around a theme. They are often introductory classes since they provide exposure to a field through interesting ideas. Unfortunately, they may also lack cohesion if the topics aren’t linked well.

Depth classes angle towards some big idea at the very end. They typically start with prerequisites to build up relevant ideas, and then these form deep of understanding on one topic.

Unlike exactly these structures, nand2tetris builds a vertical stack of knowledge covering a variety of topics. Each topic could be a course itself (like a survey class), but the knowledge all builds (like a depth class).

By the end, the class felt like a story. We remember stories better than facts because we can bridge your knowledge from one thing to another through cause and effect and intent. It’s a very effective teaching technique: for example, history makes more sense when you can weave together the progress over time rather than memorizing as random events, people, and dates.

nand2tetris covered many random bits of familiar knowledge, but it only made sense when put together. They drew connections and design across topics that fit together nicely both in topics and my mind. I can’t say I could write the HDL for a multiplexer right away or how to generate assembly for function calls, but I can see how the design of it would be necessary.

Clever Code

In college, I skipped these classes because they sounded uninteresting. I was interested in big ideas in AI and clever algorithms there and was happy to leave the optimizations and tricks to others.

Well having just scratched the surface of the design of systems and optimizations, this is deeply fascinating.

This is the Arithmetic Logic Unit that we built in nand2tetris. When I first saw the inputs and outputs here, it looked completely ridiculous: how is this random set the cleanest implementation for any computation we might ever want to do? What even is “zy” for?

Having emerged on the other side of it, however, it’s brilliant and elegant in its design. And I have been led to believe that this is only a toy specification for an ALU (perhaps in the same way that a real car engine is much more involved than what you see in an instructional video), and I’m curious how those improvements work.

Satisfying Assignments

I like games. And many great games feel like work because I’m thinking hard and making big choices in the face of uncertainty, where I need a breakthrough towards the end to succeed.

That, of course, sounds like problem solving, which is how many people get hooked into programming. And a well-designed assignment for a programming class hopefully has that feel: it should seem challenging up front, but with steady progress, few frustrations, and a clear goal at the end, it can really work.

And they did a good job of it with nand2tetris. Julie said she enjoyed listening to me talk about how I finished each assignment. I suspect that the details of the assignment were intrinsically arcane or uninteresting, but I certainly talked about them with great gusto.

As I pointed out above, it was really satisfying to see the parts of the computer all come together as I built it up bit-by-bit. There were definitely challenging and confusing parts along the way, but those ended up being some of the most satisfying things to figure out.

There were frustrations along the way. The most frustrating parts for me were somewhere in the middle of the stack. The low level parts were easier to work with since the parts are more elementary. The high level parts are closest to my day job, so those felt more familiar. In the middle, I had to work with virtual machine stack code and assembly: it was quite abstract, and I really got lost understanding my own code.

But epic wins only come when least expected, and those were epic wins when I found my last bugs and had fully functional code. Maybe the debugging tools could have been better, but I was able to figure it all out in the end.

Final Thoughts

I’m sure I will forget much of what I learned. The lecturers insisted that this class will make me a better programmer, but I’m not sure about that. I feel like I still will write code the same way.

However, I have have the satisfaction of knowing how things work mechanistically, and that’s cool. I have rid myself of a non-programmer’s idea of what an operating system is and now really get what Linux or Windows provides for developers.

I can’t say I recommend this particular class if you’re not a programmer. Part 1 is doable, but part 2 is completely inaccessible if you’re not a programmer. However, if you were like me and happily writing your Java, Python, or JavaScript with no idea what was happening under the hood, I think it’s fun to know how computers work.