Consider having a pet that is smart, responds to your commands without training, and even takes pictures for you! You don’t even have to pick up after or buy kibble for this pet! This is a dream come true for those craving an animal companion but cannot afford the time, money, or maintenance required for real animals. Robot pets are one example of how technology can assist in scenarios where companionship and fun are needed, but the handling and upkeep of real pets isn’t possible; especially in healthcare or senior homes.
Apart from doing normal pet things such as moving around and performing a few tricks, it can also perform robot tasks, like capturing photos with the camera in its nose. It can also remember you and follow you around for as long as you like. You can also deactivate this pet for some privacy and activate it back again when you feel like you need its company.
Through its camera “eyes”, this robot pet uses a combination of machine learning and computer vision techniques to see and respond to commands given through its owner’s body gestures. The robot pet can track, recognize, and perform a task such as following the user. The user is thus able to control the pet from a distance, enabling full autonomy.
Below are the gestures included in the repository.
| Command | Gesture | Description | Output |
|---|---|---|---|
Activate |
 |
Robot starts functioning and responding to commands |  |
Deactivate |
 |
Robot goes to “sleep” state and can only be woken up by giving it activate command |  |
Move Forwards |
 |
Moves the robot forwards |  |
Move Backwards |
 |
Moves the robot backwards |  |
Spin Right |
 |
Spins the robot right |  |
Spin Left |
 |
Spins the robot left |  |
Take a Picture |
 |
Initiates the routine to take a photo |  |
Follow |
 |
Initiates the routine to follow the person who gave the follow gesture |  |
Stop Follow |
 |
Stops the follow routine if the robot is in follow mode |  |
-
Raspberry Pi LCD Screen (Kuman 3.5" Inch TFT LCD Display 480x320 RGB Pixels)
-
Miscellaneous:
- 3D printer
- Raspberry Pi Camera Ribbon Cable
- 2 Ethernet Cables
- Breadboard Jumper Cables
- Screws
- Build robot base without using the Arduino or the ultrasonic sensors, you can find the instructions here
- Connect robot base parts to Raspberry Pi (see pinout in RaspberryPi_robot_pet/server_v9.py)
- Connect Raspberry Pi Camera to CAMERA1 on the Atlas 200 DK, you can follow this guide
- Set the output voltage of the power bank to 12 V. (Hold the power button and then double click a few times.)
- The Atlas Board can be connected to the "DC-Out" port; the router and the Pi can be connected to the USB ports.
- Router connection (After you've completed 1st step of Atlas 200DK Setup and 2nd step of Raspberry Pi Setup)
- Raspberry Pi connected to Atlas 200DK via wireless connection
- Atlas 200DK connected to the router via Ethernet cord
- Laptop connected to the router via wireless connection
Development PC requires Ubuntu 18.04 system (can be virtual machine), and have CANN installed. Please setup follow the this official guide or this wiki
-
Setup the Atlas 200 DK board, following this wiki or the official guide
-
Power the Atlas DK board and set up the router connections. You can follow this guide
-
Attach raspberry Pi camera to the board (https://support.huaweicloud.com/intl/en-us/qs-atlas200dkappc32/atlased_04_0006.html).
-
Set up RC Car by following instructions here.
-
On Atlas 200 DK, git clone this repo (No internet access? Just try connecting Atlas 200 Dk to a router with Ethernet Cable. For details, check official document or our router connection guide)
git clone https://github.com/kylerhunag/gesture_controlled_robot_pet.git -
Install the required dependencies to run this project:
cd ~/gesture_controlled_robot_pet/Atlas_robot_petpip3 install -r requirements.txt
The Robot Operating System (ROS) is a set of software libraries and tools that help you build robot applications. In order to set up ROS on the Atlas 200 DK board refer to the following guide below:
Once the conda environment and ROS has been set up on the board we have to create a workspace and a ROS project such that we can run our code. The following code describes how to do so:
-
Once you have created and built your ROS package you can now copy paste the
scriptsandlaunchfolder from~/gesture_controlled_robot_pet/ROSto~/catkin_ws/src/<package_name>/ -
Here we used robot_pet as our package_name. Once all the scripts and launch files are copied into the catkin_ws we need to make the files executable and as such do
chmod +x <file name>for all the files that have been copied. Before running the ROS project ensure that the presenter server script is running and the Raspberry PI script is running. -
Activate the environment you just created. Install the dependencies listed in the
requirements.txt. And lastly run:
cd ~/catkin_ws
source ./devel/setup.bash
catkin_make to build the nodes.
- Install the latest version of Raspbian onto an SD card
Follow the steps in the link:
https://www.raspberrypi.org/documentation/installation/installing-images/ - Insert SD card into Pi and power on the Pi by plugging it into a 5V power source.
- Make sure you complete the Router connection
- Update the system by opening a terminal and entering:
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install python3-pip
sudo apt install unclutter - Retrieve the files to be downloaded onto the Raspberry Pi, which are located in the folder “RaspberryPi_robot_pet”, either from cloning this repository or from the GitHub repository directly. Make sure the following files are in a folder called “RaspberryPi_robot_pet”:
- Raspberry-Pi_robot_pet/happy.mp4
- Raspberry-Pi_robot_pet/neu.mp4
- Raspberry-Pi_robot_pet/white.jpeg
- Raspberry-Pi_robot_pet/body_img.jpg
- Raspberry-Pi_robot_pet/camera_img.jpg
- Raspberry-Pi_robot_pet/deactivate.jpg
- Raspberry-Pi_robot_pet/follow_img.jpg
- Raspberry-Pi_robot_pet/server_v9.py
- Raspberry-Pi_robot_pet/requirements.txt
- Navigate to the directory where the main Raspberry Pi program is located, for example:
cd /home/pi/Documents/Capstone/Raspberry-Pi_robot_pet/ - Install OpenCV and numpy for Python 3:
pip3 install -r requirements.txt - Install the LCD driver
Follow instruction under the link:
https://github.com/goodtft/LCD-show
- Start Raspberry Pi into Raspbian OS
- ssh into the Pi
- Navigate to the directory where the main Raspberry Pi program is located, for example:
cd /home/pi/Documents/Capstone/Raspberry-Pi_robot_pet/ - Ensure that Pi and the Atlas 200DK are in the same network. Run the main program for the robot control:
python3 server_v9.py
-
Login to Atlas 200 DK from PC (Refer to this guide on how to setup and access). Note, it is required to use
VScodewithRemote-SSHextension to login remotely, otherwise you might not get the video stream to display on your PC. -
Navigate to the presenter server directory and start the presenter server:
cd ~/gesture_controlled_robot_pet/Atlas_robot_pet/presenterserverbash run_presenter_server.sh body_pose.conf
The ROS project can now be run. In order to run the project refer to the following set of commands:
cd ~/catkin_ws
source ./devel/setup.bash
roslaunch <package_name> robot_pet_launch.launch
- Body Pose Model: Sample Body Pose https://github.com/Atlas200dk/sample_bodypose
- Object Detection Model: FairMOT model https://github.com/HardysJin/atlas-track
- Raspbian image: https://www.raspberrypi.org/documentation/installation/installing-images/
- Presenter server & ROS: https://github.com/Ascend-Huawei/HiFly_Drone
- LCD driver: https://github.com/goodtft/LCD-show