Skip to content

Commit

Permalink
Fix READMEs for ROS2 (#84)
Browse files Browse the repository at this point in the history
* Fix READMEs for ROS2

* Fix missing s in README

* Fix package.xml _msgs dependencies for odas_ros and audio_utils

* Fix _msgs in CMakeLists.txt

* More _msgs fixes in CMakeLists.txt

* Fix weird image display in GUI when size is odd
  • Loading branch information
philippewarren authored Jul 25, 2024
1 parent 4562990 commit 9414d7e
Show file tree
Hide file tree
Showing 10 changed files with 57 additions and 58 deletions.
41 changes: 22 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# opentera-webrtc-ros

Welcome to the opentera-webrtc-ros project. The goal of the project is to provide useful ROS nodes to stream audio/video/data through Google's WebRTC library wrapped in [opentera-webrtc](https://github.com/introlab/opentera-webrtc). Wrappers are written in C++ and Python and are compatible with ROS1 (Noetic) at the moment. We use the [signaling server](https://github.com/introlab/opentera-webrtc/tree/main/signaling-server) implementation provided by opentera-webrtc.
Welcome to the opentera-webrtc-ros project. The goal of the project is to provide useful ROS nodes to stream audio/video/data through Google's WebRTC library wrapped in [opentera-webrtc](https://github.com/introlab/opentera-webrtc). Wrappers are written in C++ and Python and are compatible with ROS2 (Humble) at the moment. Support for ROS1 (Noetic) is in the [`ros1`](https://github.com/introlab/opentera-webrtc-ros/tree/ros1) branch. We use the [signaling server](https://github.com/introlab/opentera-webrtc/tree/main/signaling-server) implementation provided by opentera-webrtc.

Here are the key features:

* [ROS Messages](opentera_webrtc_ros_msgs) adding compatibility with ROS and [OpenTera protobuf protocol](https://github.com/introlab/opentera_messages) used in[opentera-teleop-service](https://github.com/introlab/opentera-teleop-service).
* [ROS Messages](opentera_webrtc_ros_msgs) adding compatibility with ROS and [OpenTera protobuf protocol](https://github.com/introlab/opentera_messages) used in [opentera-teleop-service](https://github.com/introlab/opentera-teleop-service).

* [ROS Streaming nodes](opentera_webrtc_ros/README.md) capable of sending / receiving audio, video and data from WebRTC streams.

Expand All @@ -14,7 +14,7 @@ Here are the key features:

* Sound Source Localization / Tracking / Separation using [ODAS ROS](https://github.com/introlab/odas_ros).

* [Stand alone demonstrations](opentera_webrtc_demos/README.md) with simulated robot in gazebo.
* [Stand alone demonstrations](opentera_webrtc_demos/README.md) with simulated robot in Gazebo.

* [Robot side front-end (Qt)](opentera_webrtc_robot_gui/README.md) to display call information and remote user interaction.

Expand Down Expand Up @@ -42,48 +42,52 @@ The project is licensed with:

## Dependencies / Requirements

The procedure is written for Ubuntu 20.04 using ROS noetic. We assume ROS is already installed. If not, follow the [ROS Installation Instructions](http://wiki.ros.org/noetic/Installation/Ubuntu) first. The following packages must also be installed :
The procedure is written for Ubuntu 22.04 using ROS2 Humble. We assume ROS is already installed. If not, follow the [ROS2 Humble Installation Instructions](https://docs.ros.org/en/humble/Installation/Ubuntu-Install-Debians.html) first. A more recent CMake than the default on Ubunuttu 22.04 is required. If the installed CMake version is 3.22, follow the [CMake Installation Instructions](https://apt.kitware.com/). The following packages must also be installed :

```bash
# utilities
sudo apt install unzip rsync ros-dev-tools

# opentera-webrtc-ros packages
sudo apt-get install nodejs ros-noetic-turtlebot3 ros-noetic-turtlebot3-gazebo ros-noetic-cv-camera ros-noetic-dwa-local-planner ros-noetic-rtabmap-ros
sudo apt install ros-humble-camera-info-manager ros-humble-rtabmap-ros ros-humble-rqt-tf-tree ros-humble-turtlebot3-gazebo ros-humble-turtlebot3-description ros-humble-turtlebot3-navigation2 ros-humble-joint-state-publisher-gui

# protobuf
sudo apt-get install libprotobuf-dev protobuf-compiler python3-protobuf
sudo apt install libprotobuf-dev protobuf-compiler python3-protobuf

# python dependencies
sudo apt-get install python3-pip portaudio19-dev
sudo apt install python3-pip portaudio19-dev

# nodejs dependencies
sudo apt-get install nodejs npm
sudo apt install nodejs npm

# audio_utils packages
sudo apt-get install cmake build-essential gfortran texinfo libasound2-dev libpulse-dev libgfortran-*-dev
sudo apt install cmake build-essential gfortran texinfo libasound2-dev libpulse-dev 'libgfortran-*-dev'

# odas_ros packages
sudo apt-get install libfftw3-dev libconfig-dev
sudo apt install libfftw3-dev libconfig-dev

# gstreamer for hardware acceleration
sudo apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-good1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-tools

# qt submodules
sudo apt-get install libqt5charts5-dev
sudo apt install libqt5charts5-dev
```

## Installation

### 1 - Create a catkin workspace (if not already done)
### 1 - Create a colcon workspace (if not already done)

```bash
# Make sure ROS is installed first.
source /opt/ros/noetic/setup.bash
# Create the workspace and initial build files
# Create the workspace
mkdir -p ~/teleop_ws/src
cd ~/teleop_ws/
catkin_make
```

### 2 - Get all the required ROS packages

```bash
cd ~/teleop_ws/src
# cv_camera
git clone https://github.com/Kapernikov/cv_camera.git
# audio_utils
git clone https://github.com/introlab/audio_utils.git --recurse-submodules
# odas_ros
Expand All @@ -107,8 +111,7 @@ python3 -m pip install -r requirements.txt

```bash
cd ~/teleop_ws
source devel/setup.bash
catkin_make
colcon build --symlink-install --cmake-args -DPYTHON_EXECUTABLE=/usr/bin/python3 -DCMAKE_BUILD_TYPE=Debug --no-warn-unused-cli
```

## Running the demos
Expand Down
6 changes: 3 additions & 3 deletions face_cropping/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,9 @@ It can be enabled and disabled from the robot GUI by pressing the face cropping

### Subscribed Topics

- `enable_face_cropping` [std_msgs/Bool](http://docs.ros2.org/foxy/api/std_msgs/msg/Bool.html): The topic to enable or disable the node.
- `input_image` [sensor_msgs/Image](http://docs.ros2.org/foxy/api/sensor_msgs/msg/Image.html): The input image topic.
- `enable_face_cropping` [std_msgs/Bool](https://docs.ros.org/en/humble/p/std_msgs/interfaces/msg/Bool.html): The topic to enable or disable the node.
- `input_image` [sensor_msgs/Image](https://docs.ros.org/en/humble/p/sensor_msgs/interfaces/msg/Image.html): The input image topic.

### Published Topics

- `output_image` [sensor_msgs/Image](http://docs.ros2.org/foxy/api/sensor_msgs/msg/Image.html): The output image topic.
- `output_image` [sensor_msgs/Image](https://docs.ros.org/en/humble/p/sensor_msgs/interfaces/msg/Image.html): The output image topic.
2 changes: 1 addition & 1 deletion map_image_generator/package.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<depend>visualization_msgs</depend>
<depend>rtabmap_msgs</depend>
<depend>opentera_webrtc_ros_msgs</depend>
<depend>odas_ros</depend>
<depend>odas_ros_msgs</depend>

<depend>cv_bridge</depend>
<depend>framework</depend>
Expand Down
12 changes: 4 additions & 8 deletions opentera_webrtc_demos/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,17 @@ To launch the stand alone demo try :

```bash
# Load ROS environment
$ source ~/teleop_ws/install/setup.bash
# Needed for simulation
$ export TURTLEBOT3_MODEL=waffle
source ~/teleop_ws/install/setup.bash
# Run the demo
$ ros2 launch opentera_webrtc_demos demo.launch is_stand_alone:=true
ros2 launch opentera_webrtc_demos demo.launch.xml is_stand_alone:=true
```

To launch the stand alone demo with ODAS functionnality, you will need the [ReSpeaker 4-mic array](https://respeaker.io/usb_4_mic_array/) plugged-in your computer:
```bash
# Load ROS environment
$ source ~/teleop_ws/install/setup.bash
# Needed for simulation
$ export TURTLEBOT3_MODEL=waffle
source ~/teleop_ws/install/setup.bash
# Run the demo
$ ros2 launch opentera_webrtc_demos demo_odas.launch is_stand_alone:=true
ros2 launch opentera_webrtc_demos demo_odas.launch.xml is_stand_alone:=true
```

Once launched go to the following URL :
Expand Down
6 changes: 3 additions & 3 deletions opentera_webrtc_robot_gui/src/MainWindow.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -104,15 +104,15 @@ void MainWindow::localImageCallback(const sensor_msgs::msg::Image::ConstSharedPt
if (msg->encoding == "rgb8")
{
// Step #1 Transform ROS Image to QtImage
QImage image(&msg->data[0], msg->width, msg->height, QImage::Format_RGB888);
QImage image(&msg->data[0], msg->width, msg->height, msg->step, QImage::Format_RGB888);

// Step #2 emit new signal with image
emit newLocalImage(image.copy());
}
else if (msg->encoding == "bgr8")
{
// Step #1 Transform ROS Image to QtImage
QImage image(&msg->data[0], msg->width, msg->height, QImage::Format_RGB888);
QImage image(&msg->data[0], msg->width, msg->height, msg->step, QImage::Format_RGB888);

// Step #2 emit new signal with image
// Invert R & B here
Expand Down Expand Up @@ -239,7 +239,7 @@ void MainWindow::closeCameraWindow()
void MainWindow::peerImageCallback(const opentera_webrtc_ros_msgs::msg::PeerImage::ConstSharedPtr& msg)
{
// Step #1 Transform ROS Image to QtImage
QImage image(&msg->frame.data[0], msg->frame.width, msg->frame.height, QImage::Format_RGB888);
QImage image(&msg->frame.data[0], msg->frame.width, msg->frame.height, msg->frame.step, QImage::Format_RGB888);

// Step #2 Emit signal (will be handled in Qt main thread)
// Image will be automatically deleted when required
Expand Down
8 changes: 4 additions & 4 deletions opentera_webrtc_ros/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ find_package(std_msgs REQUIRED)
find_package(std_srvs REQUIRED)
find_package(sensor_msgs REQUIRED)
find_package(geometry_msgs REQUIRED)
find_package(audio_utils REQUIRED)
find_package(audio_utils_msgs REQUIRED)
find_package(opentera_webrtc_ros_msgs REQUIRED)
find_package(map_image_generator REQUIRED)

Expand Down Expand Up @@ -145,7 +145,7 @@ ament_target_dependencies(${PROJECT_NAME}_topic_streamer
std_srvs
sensor_msgs
geometry_msgs
audio_utils
audio_utils_msgs
)
target_link_libraries(${PROJECT_NAME}_topic_streamer
${opentera_webrtc_native_client_libs}
Expand All @@ -167,7 +167,7 @@ ament_target_dependencies(${PROJECT_NAME}_data_channel_bridge
std_srvs
sensor_msgs
geometry_msgs
audio_utils
audio_utils_msgs
)
target_link_libraries(${PROJECT_NAME}_data_channel_bridge
${opentera_webrtc_native_client_libs}
Expand All @@ -188,7 +188,7 @@ ament_target_dependencies(${PROJECT_NAME}_json_data_handler
std_srvs
sensor_msgs
geometry_msgs
audio_utils
audio_utils_msgs
)
set_target_properties(${PROJECT_NAME}_json_data_handler PROPERTIES OUTPUT_NAME json_data_handler PREFIX "")
install(TARGETS ${PROJECT_NAME}_json_data_handler DESTINATION lib/${PROJECT_NAME})
Expand Down
32 changes: 16 additions & 16 deletions opentera_webrtc_ros/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@ It also forwards images and audio received on the WebRTC stream to ROS.

#### Subscribes

- ros_image : `sensor_msgs::Image`
- audio_in : `audio_utils::AudioFrame`
- ros_image : `sensor_msgs/Image`
- audio_in : `audio_utils_msgs/AudioFrame`

#### Advertises

- webrtc_image : `opentera_webrtc_ros::PeerImage`
- webrtc_audio : `opentera_webrtc_ros::PeerAudio`
- audio_mixed : `audio_utils::AudioFrame`
- webrtc_image : `opentera_webrtc_ros_msgs/PeerImage`
- webrtc_audio : `opentera_webrtc_ros_msgs/PeerAudio`
- audio_mixed : `audio_utils_msgs/AudioFrame`

#### Default Parameters

Expand All @@ -40,15 +40,15 @@ It also forwards images and audio received on the WebRTC stream to ROS.
<param name="is_screen_cast" value="false"/> <!-- Is the image source a screen capture? -->
<param name="needs_denoising" value="false"/> <!-- Does the image source needs denoising? -->
</param>
<param value="signaling">
<param name="signaling">
<param name="server_url" value="http://localhost:8080"/> <!-- Signaling server URL used in stand_alone mode -->
<param name="client_name" value="streamer"/> <!-- Peer name as which to join the room -->
<param name="room_name" value="chat"/> <!-- Room name to join -->
<param name="room_password" value="abc"/> <!-- Room password used in stand alone mode -->
</param>
```

For usage exemple look at [ros_stream_client.launch](launch/ros_stream_client.launch).
For usage exemple look at [ros_stream_client.launch.xml](launch/ros_stream_client.launch.xml).

## RosDataChannelBridge

Expand All @@ -59,25 +59,25 @@ data channel. It also forwards messages received on the WebRTC data channel to R

#### Subscribes (RosDataChannelBridge)

- ros_data : `std_msgs::String`
- ros_data : `std_msgs/String`

#### Advertises (RosDataChannelBridge)

- webrtc_data : `opentera_webrtc_ros_msgs::PeerData`
- webrtc_data : `opentera_webrtc_ros_msgs/PeerData`

#### Default Parameters (RosDataChannelBridge)

```xml
<param name="is_stand_alone" value="true"/>
<param value="signaling">
<param name="signaling">
<param name="server_url" value="http://localhost:8080"/> <!-- Signaling server URL used in stand_alone mode -->
<param name="client_name" value="data_bridge"/> <!-- Peer name as which to join the room -->
<param name="room_name" value="chat"/> <!-- Room name to join -->
<param name="room_password" value="abc"/> <!-- Room password used in stand alone mode -->
</param>
```

For usage exemple look at [ros_data_channel_client.launch](launch/ros_data_channel_client.launch).
For usage exemple look at [ros_data_channel_client.launch.xml](launch/ros_data_channel_client.launch.xml).

## RosJsonDataHandler

Expand All @@ -87,21 +87,21 @@ Implement a ROS node that dispatch received JSON messages and forward them on th

#### Subscribes (RosJsonDataHandler)

- webrtc_data : `opentera_webrtc_ros_msgs::PeerData`
- webrtc_data : `opentera_webrtc_ros_msgs/PeerData`

#### Advertises (RosJsonDataHandler)

- cmd_vel : `geometry_msgs::Twist`
- cmd_vel : `geometry_msgs/Twist`

#### JSON Format (RosJsonDataHandler)

For usage exemple look at [ros_json_data_handler.launch](launch/ros_json_data_handler.launch).
For usage exemple look at [ros_json_data_handler.launch.xml](launch/ros_json_data_handler.launch.xml).

## libnavigation

### Description (libnavigation)

Shared library for `goal_manager` and `label_manager` to send navigation commands to `move_base` as well as navigation waypoints to `map_image_generator`.
Shared library for `goal_manager` and `labels_manager` to send navigation commands to the `nav2` stack as well as navigation waypoints to `map_image_generator`.
The library is embedded in both nodes, a single instance is not shared.

#### Subscribed topics (libnavigation)
Expand Down Expand Up @@ -138,7 +138,7 @@ Manages labels.
The stored labels are dependent on the `map` transform and the database needs to be cleaned if the map changes.
A label represents a name grouped with an associated pose and a description.
This node relies on a service provided by the `map_image_generator` package to convert the labels from image coordinates to map coordinates.
This node can also send a label as a goal to `move_base` by its name.
This node can also send a label as a goal to `nav2` by its name.

#### Subscribed topics (labels_manager)

Expand Down
2 changes: 1 addition & 1 deletion opentera_webrtc_ros/package.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<depend>std_msgs</depend>
<depend>std_srvs</depend>
<depend>sensor_msgs</depend>
<depend>audio_utils</depend>
<depend>audio_utils_msgs</depend>
<depend>opentera_webrtc_ros_msgs</depend>

<depend>map_image_generator</depend>
Expand Down
4 changes: 2 additions & 2 deletions opentera_webrtc_ros_msgs/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ find_package(ament_cmake REQUIRED)
find_package(std_msgs REQUIRED)
find_package(geometry_msgs REQUIRED)
find_package(sensor_msgs REQUIRED)
find_package(audio_utils REQUIRED)
find_package(audio_utils_msgs REQUIRED)
find_package(rosidl_default_generators REQUIRED)

rosidl_generate_interfaces(${PROJECT_NAME}
Expand Down Expand Up @@ -38,7 +38,7 @@ rosidl_generate_interfaces(${PROJECT_NAME}
"srv/ImageGoalToMapGoal.srv"
"srv/SetString.srv"

DEPENDENCIES std_msgs geometry_msgs sensor_msgs audio_utils
DEPENDENCIES std_msgs geometry_msgs sensor_msgs audio_utils_msgs
)

ament_package()
2 changes: 1 addition & 1 deletion opentera_webrtc_ros_msgs/package.xml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
<depend>std_msgs</depend>
<depend>geometry_msgs</depend>
<depend>sensor_msgs</depend>
<depend>audio_utils</depend>
<depend>audio_utils_msgs</depend>

<member_of_group>rosidl_interface_packages</member_of_group>

Expand Down

0 comments on commit 9414d7e

Please sign in to comment.