Connecting your Pixy Camera to the NXT

I recently backed an interesting Kickstarter project by Charmed Labs: the Pixy Camera (a.k.a. CMU Cam 5). The Pixy Camera is a fast vision sensor you can quickly “teach” to find objects, and it connects directly to Arduino and other controllers. “Other controllers” sounds to me like “LEGO MINDSTORMS”. The Pixy Camera onboard processor relieves your controller of choice (Arduino, BeagleBone, Raspberry Pi, LEGO MINDSTORMS) from the computational burden of processing each frame. The many communication interfaces available, the possibility of replacing the firmware, and the cross-platform configuration tool make Pixy Camera one of the best vision systems out there.

Facts about the Pixy Camera

• Processor: NXP LPC4330, 204 MHz, dual core
• Image sensor: Omnivision OV9715, 1/4″, 1280×800
• Lens field-of-view: 75 degrees horizontal, 47 degrees vertical
• Lens type: standard M12 (several different types available)
• Power consumption: 140 mA typical
• Power input: USB input (5V) or unregulated input (6V to 10V)
• RAM: 264K bytes
• Flash: 1M bytes

• Fast blob detection (50 Hz)
• Interfaces: UART serial, SPI, I2C, digital out, or analog out
• Directly drives two R/C servos (for pan & tilt head)
• Hue-based color filtering algorithm to detect objects (instead of RGB)
• Pixy can learn up to 7 different color signatures (you can distinguish objects of 7 different colors)
• Pixy can detect and output coordinates of hundreds of objects per each frame
• Pixy can also detect color codes with their angle (combinations of adjacent color tags)

Compatibility between Pixy Camera and LEGO MINDSTORMS

By checking the specs of the Pixy Camera, you can see that it needs about 5V at 140 mA to work properly, and that it can output data on UART serial and I2C. By studying the LEGO MINDSTORMS NXT Hardware Development Kit, you can see that the NXT brick can deliver up to a total of 180mA on all the input ports, can communicate via I2C on all the input ports and via RS485 (a fast serial with differential levels) on input 4.

Using I2C Interface

At first, I tried to use I2C because interfacing the Pixy Camera to the NXT would have been very easy. But but but… The Pixy Camera has 10KΩ pull-up resistors onboard on the SDA and SCL lines, instead of the 82KΩ resistors that LEGO recommends. I desoldered those tiny SMD components (at the risk of damaging my Pixy forever!) and then I added the two bigger resistors on the breadboard I was using to develop the interface. Unfortunately, the Pixy is too fast for the NXT to read anything, so the communication failed. I even tried the low level communication with the ROBOTC I2C Test Utility. It always returns BUS COMMUNICATION ERROR code, even if the ROBOTC firmware supports the I2C high speed mode. Someone could try LejOS I2C high-speed mode and check if it works.

In short, I2C does not work (yet).

Using UART Interface

In order to connect any UART device to the NXT on the input port 4, you need a level adapter chip. Among the many available, I used the ST Microelectronics 1480 chip, equivalent to the MAX485.The chip needs has two inputs that set the direction of the data. I connected those pins (2,3) to 3.3v to set the direction from the Pixy Camera to the NXT. In other words, the NXT can only receive data, but cannot send anything to the Pixy, which is fine, as we expect data from the vision sensor. The power supply for the Pixy is taken directly from NXT input pin 4 (green wire). Using a voltmeter I got about 4.5V on this pin, that proved to be enough (even if not exactly 5V) to power up the Pixy Camera. The RS485 chip needs a 3.3V power supply. The simplest (even if not the best) way to get that voltage is to use two diodes to drop 1.4V. This solution works, however using a Zener diode and a resistor would have been better. I did not have that Zener diode available at that time. The connector for the Pixy is just a 2×3 pin header that allows you to use the ribbon cable that comes with the camera.

I assembled the adapter using a lot of patience, given the small size of the prototyping board, and the presence of a SMD chip. For your delight, here’s how the adapter layout would look if you want to make a PCB instead.

The ROBOTC test program

I wrote a test program that displays on the NXT Brick screen the bounding boxes of the 10 biggest blobs that the Pixy Camera sees, together with their color ID. I basically adapted the Arduino Library source code to decode the data stream. You can download the ROBOTC test program by clicking on the link that will appear after you like this content.

[l2g name=”Donwload the Pixy NXT test Program” id=”1329″]

Future projects using the Pixy Camera with LEGO

Things yet to be done:

• Making a robot that features the Pixy Camera as vision sensor.
• Porting the code above to NXC and LejOS.
• Developing an hardware adapter for the EV3 brick (the RS485 chip is not needed)
• Developing code for the EV3, using ROBOTC, LejOS, or making a custom EV3 block (the EV3 needs each digital sensor to introduce itself when first connected to an input port)