What Is Physical Computing?

Physical Computing is an approach to learning how humans communicate through computers that starts by considering how humans express themselves physically.A lot of beginning computer interface design instruction takes the computer hardware for given — namely, that there is a keyboard, a screen, perhaps speakers, and a mouse — and concentrates on teaching the software necessary to design within those boundaries. In physical computing, we take the human body as a given, and attempt to design within the limits of its expression.

This means that we have to learn how a computer converts the changes in energy given off by our bodies, in the form of heat, light, sound, and so forth, into changing electronic signals that it can read interpret. We learn about the sensors that do this, and about very simple computers, called microcontrollers, that read sensors and convert their output into data. Finally, we learn how microcontrollers communicate with other computers.

Physical computing takes a hands-on approach, which means that you spend a lot of time building circuits, soldering, writing programs, building structures to hold sensors and controls, and figuring out how best to make all of these things relate to a person’s expression.

Cool. So you build all kinds of robots and sharks with frickin’ lasers on their frickin’ heads?

Not quite. While the hardware skills used in physical computing are similar to those used in robotics, the concepts are a bit different. When you build robots, you’re usually focused on making something that’s autonomous, and can ignore people and navigate through the world on its own. That takes a lot of work. Physical computing applications tend to depend more on people for input, and amplify that input into another form, like an animation, a sound, or motion. Robotics tends towards duplicating what humans can do, replacing the human brain with a computer brain; physical computing tends to extend what they can do, leaving the human brain and body at the center of it all.

I want to take a physical computing class; what do I need to know?

First, you’ll need time. Most of the real work happens outside of class, both in the shop building and programming, and in the everyday world watching people and figuring out what they do that you’re interested in sensing and interpreting.

Second, if you’ve never programmed a computer before, you’ll need to know a little bit about that. In particular, it helps if you’ve done some graphic interface programming, because later in the course, we learn how to control screen graphics and sound with the output from sensors. At the very least, you should know and be comfortable with the following:

• What an if statement is, and how to use one.
• What variables are in a computer program, and how they’re used
• What a repeat loop is, and how to use them. If you’re comfortable with statements like for-next, while-wend, or repeat while… -end repeat, then you’re in good shape

If you’re a full-time student at the Interactive Telecommunications Program at the Tisch School of the Arts, NYU, then you’ll be taking an introductory programming course at the same time as you take physical computing. If you’re a summer student, or you’re learning physical computing on your own, you might want to learn a little programming first, or at the same time.

I know HTML pretty well; is that enough programming?

Not really. HTML is a markup language, not a programming language. It describes how a web page should be displayed, but it leaves it to the browser to display it. It’s sort of a language without verbs. Programming languages have verbs, or commands that can initiate action.

Can I use Macromedia Flash in physical computing?

Yes and no. On the one hand, the programming concepts in ActionScript are similar to the ones you’ll need in physical computing. On the other hand, Flash (currently) can’t talk to a computer’s serial port, which is essential for the kinds of things you’ll do in physical computing. There are rumors that Flash will have a serial port tookit real soon now, and Dan O’Sullivan has written a niceJava applet that allows Flash to connect to the serial port through an XMLSocket. So while you may find it useful in conjunction with other programming languages, you’ll need a bit more than Flash to get going.

So what programming environments are used in physical computing?

Just about any language that can access a computer’s serial ports. In the ITP class, we use Macromedia’s Director, Processing, Java, or Max/MSP for our in-class examples. We’ve had students who knew and used Visual Basic, RealBasic, C/C++, Perl, and Python, among others.

What about all this electronics stuff; am I going to have to do that too?

Yep. We try to teach you how to do as little as you can get away with to realize your ideas, but there are a few basics that you’ll use all the time. It’s not too hard, though; you probably slept through most of what you need to know in high school.

So where do I get started?

If you’re an NYU student and you’re interested in taking a class, start by contacting ITP. The administrative staff can let you know when courses are available, and which ones are open to non-ITP students. If you’re not an NYU student, our courses are often offered in the summer for general enrollment. If you’re looking to learn it on your own, you can start with these online notes. Check out Dan O’Sullivan’s original notes on the course too. He and I also wrote this book on the subject.

 -Tom Igoe
4 June 2004