Ziro Logo

Home

Modules UDP Scripts Unity

Examples

Mobile: IMU Cloud: Alexa Arduino: PIXY Arduino: Sensors

PIXY-Arduino Example

pixy

This example lays the foundation to one of the core features of Ziro - Hacking via Hardware. As Ziro is Arduino and Raspberry Pi compatible, developers can connect Arduino/R-Pi to their Ziro robot and develop applications on top of it. Here we discuss how to use a popular camera module called Pixy, and develop a ball-following robot using Ziro.

The Pixy camera is connected to an ESP32 micro-controller (that can be programmed using an Arduino IDE). Developers will write the application logic on the ESP32 microcontroller that will communicate to the Ziro robot. The ESP32 code can be found at the Scripts section. The Pixy camera has intensive documentation and wiki that will get you started on how to use it and start playing with it. Please find more resources at the Pixy Quickstart.

The hardware required for this examples are as follows -

The hardware connection for the two needs to follow the following layout-

ball

Please download the latest Arduino IDE and set it up for the ESP32 board. As discussed below, we include the necessary libraries to enable Wifi communication with the Ziro robots. We also declare the Wifi credentials for this particular ESP32 to connect to our Ziro modules. Module-xxx-00 is the parent module to which we would connect the ESP32 to. Here xxx represents the KitId of the Kit, so please fill in the appropriate number there. The password for the Wifi connection is "zirokits".

  
#include "WiFi.h"
#include "AsyncUDP.h"
#include "Pixy2I2C.h"
Pixy2I2C pixy;


const char * ssid = "Module-xxx-00";  /// Module Credentials for Wifi Connection
const char * password = "zirokits";


After declaring some global variables, we define our setup(). This function starts the serial monitor for debugging - Serial.begin(19200); , tries to connect to the declared SSID about ten times - WiFi.begin(ssid, password); and prints a bunch of data when a successful connection has been made. We finally complete the function by initializing the pixy pixy.init(). Please note that this connection is a simple wifi connection to the parent motor module on port 2325.

  
const double pixy_x = 315 / 2;
const double pixy_y = 207 / 2;
int cntr = 0;

bool WifiConnected;
AsyncUDP udp;

void setup()
{
  Serial.begin(19200);
  while (WifiConnected == false && cntr < 10) {
    WiFi.mode(WIFI_STA);
    WiFi.begin(ssid, password);
    if (WiFi.waitForConnectResult() != WL_CONNECTED) {
      WifiConnected = false;
      Serial.println("WiFi Failed");
    }
    cntr++;
  }
  if (udp.listen(2325)) {
    Serial.print("UDP Listening on IP: ");
    WifiConnected = true;
    Serial.println(WiFi.localIP());
    udp.onPacket([](AsyncUDPPacket packet) {
      Serial.print("UDP Packet Type: ");
      Serial.print(packet.isBroadcast() ? "Broadcast" : packet.isMulticast() ? "Multicast" : "Unicast");
      Serial.print(", From: ");
      Serial.print(packet.remoteIP());
      Serial.print(":");
      Serial.print(packet.remotePort());
      Serial.print(", To: ");
      Serial.print(packet.localIP());
      Serial.print(":");
      Serial.print(packet.localPort());
      Serial.print(", Length: ");
      Serial.print(packet.length());
      Serial.print(", Data: ");
      Serial.write(packet.data(), packet.length());
      Serial.println();
      //reply to the client
      packet.printf("Got %u bytes of data", packet.length());
    });
  }

  pixy.init();
}


In our loop function, we check Wifi and initiate a Pixy loop which deals with the operations of handling the information from Pixy and translating them to motor commands. After we get the blocks from the pixy, we evaluate which object detected has the biggest footprint. This way we cancel out the noises and make the Pixy only follow the biggest detectable object i.e. the Ball and focus only on that object. We use Pixy's width feature to ignore objects that do not have a significant footprint in Pixy's frame.

  
  if (pixy.ccc.numBlocks)
  {
    Serial.print("Detected ");
    Serial.println(pixy.ccc.numBlocks);
    int max_w = 0;
    int ball;
    for (int i = 0; i < pixy.ccc.numBlocks; i++) {
      if (pixy.ccc.blocks[i].m_width > max_w) {
        max_w = pixy.ccc.blocks[i].m_width;
        ball = i;
      }
    }

Pixy also outputs the height of the object in the frame which translates to the z-axis distance of the object. We can use that feature to calculate how far the object is from the camera. Based on that value we can make the robot move forward or backward. Similarly, Pixy outputs the X and Y coordinates of the object from the frame. Based on the x and y coordinates of the tracked object, we can make the robot turn left or right and align itself so that the object is in the center of the Pixy's frame.


   if (pixy.ccc.blocks[ball].m_height < 70) {
      Serial.print("  block ");
      pixy.ccc.blocks[ball].print();
      Block block1 = pixy.ccc.blocks[ball];
      int x = block1.m_x;
      int y = block1.m_y;
      int turn_x = pixy_x - x;

      if (x > pixy_x + 40) {
        udp.broadcast("{""pType"":7,""m_T"":[45,45,0,0,0,0,0],""m_M"":[2,2,0,0,0,0,0]}");
      } else if (x < pixy_x - 40) {
        udp.broadcast("{""pType"":7,""m_T"":[45,45,0,0,0,0,0],""m_M"":[1,1,0,0,0,0,0]}");
      } else {
        udp.broadcast("{""pType"":7,""m_T"":[55,55,0,0,0,0,0],""m_M"":[1,2,0,0,0,0,0]}");
      }
    } else {
      udp.broadcast("{""pType"":7,""m_T"":[0,0,0,0,0,0,0],""m_M"":[0,0,0,0,0,0,0]}");
    }

  } else {
    udp.broadcast("{""pType"":7,""m_T"":[0,0,0,0,0,0,0],""m_M"":[0,0,0,0,0,0,0]}");
  }

  
  

The Pixy powered trike would function like this -

ball

Developers can leverage the ESP32 microcontroller to attach multiple sensors and hack the hardware to develop an interesting robotic application using Ziro Robotics platform. Following is the image that represents the various pinouts available to connect various breakout boards to the ESP32 for development purpose. Developers can use the same setup() discussed in this Pixy code to establish Wifi connection with Ziro and build their application from the loop(). A detailed explanation of the pinouts and usage for the ESP32 breakout board can be found here.

ESP32pintout