Tom's Main Menu

Physical Computing Home

Intro to Physical Computing Syllabus

Networked Objects

Sustainable Practices



code, circuits, & construction

my links


Balancing Input and Output

In any system that you design to interact with people, the people are your most important consideration. You want to design everything with the response times and attention spans of human beings in mind. This becomes even more critical in a physical computing application, because your users are usually less sedentary and more actively engaged than desktop users, so their reaction times and expectations are higher.

In desktop applications, interaction designers have many schemes for keeping the user engaged when the computer needs to do things other than listen to input. The splash screen, the progress bar, the watch cursor and various other tools have been created to tell the user to wait while the computer does something else.

For more on basic screen interaction, see Chris Crawford's Understanding Interactivity, Brenda Laurel's (ed.) The Art of Computer-Human Interface Design, or any of many other texts on the field. These notes owe much to Crawford's book in particular.
In physical computing applications, we don't have such conventions to work with. Furthermore, physical computing applications often engage the user in a more physically intense manner than desktop applications, so the person's reaction times are generally heightened, and their patience for delays is shortened. Electronics and other appliances and tools have traditionally been simple enough that we encounter no significant delay times when using them. Because of this, people expect the same of most any tool that does not appear to be a desktop computer. So when we design physical computing applications, whether there's a computer involved or not, we have to plan the interaction such that there are no significant delay times.

In any interactive project, there are three basic tasks you have to handle: input, output, and the processing necessary to mediate between the two. All of these tasks take some of the time of your system, whether it's a desktop computer, microprocessor, or some combination of several elements. Juggling the system's time to handle all of these tasks smoothly is the central work you do in designing the system.

These three tasks are similar to the basic processes of conversation, namely, listening, thinking, and talking. However, in a conversation, human beings are capable of talking and listening at the same time. So when a listener wants to interrupt a talker, the talker knows to stop talking and listen. There are also natural pauses in a conversation for the listener to digest information and prepare a response. We know intuitively that when we present information in conversation, we have to give our listeners time to digest the information. Information is handed out in digestible chunks, with pauses in between, and even when we're talking, we're also listening for an interruption.

Computers don't have the ability to listen and talk at the same time. Fortunately, they execute instructions very fast, on the order of a few dozen microseconds per command at the slow end. So you often find yourself having to slow down the speed of the computer to give the output to humans in digestible chunks. This is useful, because what we can do is make the computer listen to its inputs for short periods, then control its output for short periods. If all of these periods are shorter than the human reaction time, it can appear to a human being as if the computer is speaking and listening at the same time.

One approach to this problem is to have separate processors handle the separate tasks. This could involve things like using a separate motor driver processor in a moving sculpture, or letting a MIDI sampler handle the playback of music in a sound installation, or letting a desktop computer or server handle internet connectivity in a networked project. This is the idea behind parallel processing: you take each part of a big job and give it to an individual processor. All the processors talk to each other only when needed, such as when an event occurs that another processor needs to handle. Their attention to each other is minimal relative to the attention they give to their tasks.

All of these are higher level solutions that save you the trouble of having to coordinate tasks at a programming level on one processor. They come at a cost, however, whether it's cost of equipment or time costs in connecting the components. You always have to balance these costs against the costs of building your own complex programming and electronics. Sometimes the best solution is to manage everthing on one central processor.

The balancing act on on one processor

When you first learn to program, you think about the output first. In fact, the first program most people learn to write is a simple one that outputs "Hello world!" On a microprocessor, where we have no screen to print to, we usually say hello to the world by blinking an LED. Of course, we can blink that LED very fast, so we slow it down by adding a command to pause the computer. We can get very precise timing of our output by using pauses in this way.

In order to listen to the user, we have to some of use the processor's time to read inputs. If we're reading a switch or an analog-to-digital converter, we can do this a few hundred or thousand times a second, if we're doing nothing else with the processor, so we've got pretty fine control over our input. However, when we try to combine expressing ourselves to the user and listening to the user, we get in trouble.

Let's say we want to make an LED blink three times every time a person presses a button, with a half second betweek blinks. We might write code something like this (in PicBasic Pro: BASIC):

    if portb.0 = 1 then
        for x = 1 to 3
            high portb.1
            pause 500
            low portb.1
            pause 500
goto main

It takes one and a half seconds to finish our response to a button push. That means that we only listen to the user every one and a half seconds. Human response time is a lot faster than that, so it's possible for them to press the button and have to wait for a response. There's the trouble.

Most users are as impatient with the delays involved with desktop computer action as they are with any other artificial delay, but many designers of desktop programs have convinced themselves that spinning cursor will placate even the most impatient user. Go figure.
We need a way to make sure that we're listening to the user at a rate faster than human response time, while still controlling our blinking LED at a rate that's what we want. To do this, we might try using several small delays, perhaps a thousandth of a second each, instead of just two big ones. That way, we could listen for user response on the button very fast, perhaps after each tiny pause, and blink the LED slower, perhaps after every couple hundred pauses. One way to do this is by establishing the smallest possible pause, then using a counter to count pauses, and then performing our various tasks after the correct number of pauses has passed. Here's an example:

PicBasic Pro:

' yellow LED on portb.0
' green LED on portb.1
' red LED on portb.2
' Connect analog input to channel-0 (RA0)

INCLUDE "modedefs.bas"
' Set Debug pin port
' Set Debug pin BIT
' Set Debug baud rate
' Set Debug mode: 0 = true, 1 = inverted

' Define ADCIN parameters
DEFINE  ADC_BITS        10     ' Set number of bits in result
DEFINE  ADC_CLOCK       3     	' Set clock source (3=rc)
DEFINE  ADC_SAMPLEUS    50    	' Set sampling time in uS 

pauseTimeVar var word
adcVar var word
ticks var word
pauseCounterVar var word

' our minimum pause:
pauseCounterVar = 1

' start with 0.5 second delay:
pause 500 

	for pauseCounterVar = 1 to 1000	
		if pauseCounterVar // 200 = 0 then
			' anything in this if statement 
    		' will happen once 200 pauses. 
			toggle portb.0
		if pauseCounterVar // 100 = 0 then
			' anything in this loop happens 
    		' twice var frequently 
        	' var the last loop
			toggle portb.1
		if pauseCounterVar // 50  = 0 then 
			toggle portb.2

   	' anything you put here 
   	' happens every loop:

		adcin 0, adcVar 
		debug DEC adcVar, 10, 13
		pause pauseCounterVar
goto main


dim pauseTimeVar as single
dim yellowState as byte
dim greenState as byte
dim redState as byte
dim adcVar as integer
dim ticks as single

Sub Main()
	dim pauseCounterVar as integer

' start with 0.5 second delay:
	call delay(0.5) 

' our minimum pause:
	pauseTimeVar = 0.0001	

		for pauseCounterVar = 1 to 10000	
			  if pauseCounterVar mod 200 = 0 then
				   ' anything in this if statement 
    			' will happen once 200 pauses. 
				    call togglePin(19, yellowState)
			  end if		
			  if pauseCounterVar mod 100 = 0 then
				    ' anything in this loop happens 
    				' twice as frequently 
        ' as the last loop
				    call togglePin(17, greenState)
			end if				
			if pauseCounterVar mod 50  = 0 then 
			   call togglePin(15, redState)
			end if

   ' anything you put here 
   ' happens every loop:

			adcVar = getADC(13)
		 debug.print cstr(adcVar)
			call delay(pauseTimeVar)

end sub

sub togglePin(byVal pinNumber as byte, byRef stateVar as byte)
		' example of a subroutine 
  ' that uses passed paramters

  if stateVar = 0 then
		stateVar = 1
		stateVar = 0
	end if
	call putPin(pinNumber, stateVar
end sub

We can use a loop like this to keep our LED blinking at a steady rate while reading the switch every few milliseconds or microseconds. See the BX-24 timing loop example for details.

We have to factor in the time taken to run the rest of our program as well. So we might have to make our pause smaller, or change our counting. Perhaps we'd change to pausecounterVar mod 40 instead of mod 50, which would execute our output on every 40th pause instead of every 50th. There's usually not a way to calculate this in advance, so you end up starting with an arbitrary value that you think is close enough, then changing the program until you find a value that works.

Certain commands that you give a processor will take more time than others. For example, debug statements will slow down a program quite a bit. Even the if-then statements in the example above slow the processor down somewhat. The surest way to find out how a given command affects your program is to try it in practice. Your interaction with the user and with your output will be helped by removing any unnecessary commands, so when you've learned what you need from a debug statement, for example, you should comment it out or remove it.

This example uses the modulus function. The modulus of two number is the remainder of their division. For example, 4 divided by 3 is 1, remainder 1. So 4 mod 3 is 1. Similarly, 5 divided by 3 is 1 remainder 2, so 5 mod 3 is 2. In general, if z = y mod x, z will vary from 0 to (x - 1). It's a nifty way to get a smaller counter inside a larger counter, as we're doing here
This timing loop is not the only way to balance timing between input and output. There are as many schemes for this as there are programmers, everyone has their own method. The key factor to keep in mind is that any time the processor is constrained to one task, like a pause, or a print statement, or any other command which takes time, then the system is not listening to the user. Whenever this happens, you must have a way of letting the user know that, and when it's appropriate for them to respond again.

Sometimes the problem of interaction is not one of precise timing, but of clear indication of expectations. Going back to our conversational model, if I enter into a long and detailed explanation of an idea that causes my listener to go silent for several minutes, I may need to cue him to respond at the end, perhaps through a silent pause and a questioning look. If he interrupts me, but I want to finish my sentence, I might raise my finger to acknowledge his interruption, but finish the sentence nonetheless.

It's wise to incorporate similar small indicators into any system to acknowledge user input. For example, turning an LED on when a button is pushed and off when the button is released takes negligible processing work, but gives the user a definite sign that the input was "heard" by the computer. The button may start or interrupt many other tasks besides lighting the LED, but lighting the LED lets the user know that those other tasks have been put in motion. Having been acknowldeged immediately, the user is prepared to wait for the other tasks for finish, as long as the wait is reasonable.

At other times, you might need to prompt user input. For example, if the user's footsteps trigger a complex series of audiovisual output that brings her to a standstill, then perhaps a subtle audio cue could be used to prompt her to respond again when response is appropriate. This is particularly true when the an output sequence has a subtle ending, or does not have a definite ending point.

In a well-designed system, the user will never consciously notice when the system's not listening. The interaction will be so balanced that the system takes advantage of natural pauses to do its work. If you plan the whole experience such that there are plenty of those pauses, and plan simple physical cues to make it clear to the user what's expected at any given moment, the implementation and programming of the system will flow much more simply from start to finish.

Some processors have a more advanced operating system, and can handle precise timing of multiple tasks for you. For example, the BX-24 has multitasking capability. Not many small microprocessors have such a capability built in, so you often have to find your own ways of handling it.

Other processors use hardware interrupts, which are input pins that are designed to stop the flow of a program when they receive an input. Interrupts are beyond the scope of these notes, but see the BX-24 documentation for more on interrupts and the BX-24.

Donald Norman writes about people's mental models of systems and how those models affect the interaction with the systems in The Design of Everyday Things. His explanation goes into more depth on the idea presented here.