Using a webcam to transmit and receive arbitrary binary data at 1 bit/s between two computers

Project executed on: June 4, 2014 // This summary written on: June 7, 2014 // Revised: November 14, 2014

At some point about a year ago, I became interested in telecommunications. One thing that particularly caught my attention was the fact that dial-up modems, fax machines, phones with Caller ID, and many other devices all rely on (very complex) old-school technology that through some mathematical magic turns binary data into sound. This got me wondering what else I could use to transmit binary data. I then had the idea to set up two computers with cameras pointed at each other's monitors so that one computer could "transmit" by blinking different colors on the screen, and the other could "receive" by reading image data from the camera at regular intervals and interpreting the colors in the image as data.

I didn't actually implement the idea until 6/4/14, when I didn't have much to do, and decided to finally tackle it. Here's a summary of the project, followed by more details:

Project Highlights

HTML and JavaScript

I wrote the program in HTML and JavaScript because it's easy and it's what I'm familiar with. The getUserMedia() API allows access to the computer's camera (with user permission, of course), which can then be written to a <canvas> for processing.

It should go without saying, but there are no frameworks involved here. Obviously. jQuery And Friends are definitely not needed for this.

Firefox, Desktop, Surface

The two machines I used (the two I have most readily at my disposal) were my main desktop PC, as well as my Surface Pro 2. On both machines, the program was run in Firefox 29. IE 11 doesn't support getUserMedia() and Chrome 35 seemed to have a bug that kept it from talking to the Surface's cameras, although it worked on my desktop.

Any kind of data

I tested this exclusively with UTF-8 text. The program as is has two text boxes: one for entering data to send, and another displaying what's been received. However, it could be extended to sending whole files, although it would be very slow - a 2KB file would take almost 4.5 hours to send!

1 bit per second

That's extremely slow, in case you didn't know. One bit is sent each second, meaning that one UTF-8 character (8 bits) takes 8 seconds to arrive on the other computer.

In Sync

While it is in "communication mode," the program alternates between two different "phases": the sending phase and the receiving phase. During the sending phase, the color of the screen is updated to match the next bit to be sent. During the receiving phase, the color that the camera sees is evaluated to determine the next bit that has been received. These phases alternate every half second. For the program to work properly, so that no bits at the beginning or end of a sequence are lost, the computers have to both be in the same phase at a given time. However, their phases do not necessarily have to start at the same time. To get the computers sufficiently in phase with each other, there is a handshake that occurs to make sure that the sending and receiving phases match, and that they start sending and receiving at the same time.

Diagram of the Setup

Here's a diagram showing what the setup looked like. I had to dismantle it because it tied up two computers and took up my whole desk.

Diagram

Challenges

There were some challenges involved in getting this project to work. They mostly revolved around the limitations of the cameras. In many cases, the cameras would not respond to sudden changes in light quickly enough for the speed to be any faster. For instance, I tried to double the speed, but when the colors were changing that frequently, the cameras were not able to keep up. Perhaps better cameras would allow faster transmission. Another idea that I tried to speed it up was to add additional colors that would have other meanings, such as transmitting multiple bits at once with different colors. This didn't work because of the low-quality cameras. These changes in color usually washed out the image, causing the program to see white instead of the intended color.

Other challenges related to the setup, not to the cameras themselves. Originally, I had the Surface facing the desktop's monitor, with the desktop's webcam below the monitor. This didn't work because the Surface's screen is very glossy, so the desktop kept seeing its own reflection. That was easy to solve, though, because the Surface also has a rear camera, so I could have them facing away from each other.

One last problem I had came from ambient light. Although the cameras did an adequate job of adjusting for different ambient light levels, there was an inconveniently positioned ceiling light above my desk that kept causing a glare on the Surface's glossy screen. So, on the day that I did this project, I spent the afternoon in the dark.

How it Works

White is a 1, black is a 0. Each send phase, both computers update their monitor colors to reflect the value of the next bit in the sequence. 500ms later, in the receive phase, each computer retrieves the color that it sees from the other computer's monitor. The bits received are stored in a buffer until there are eight of them, then they are strung together, converted from a character code, and appended to the output box. At each send phase, if there are no bits in the buffer to send, the input box is checked for a character. The first character is removed and converted to a character code which is then converted to binary, split into bits, and placed in the send buffer. If there are no characters to send, the screen is turned red to indicate that there is nothing to send.