Skip to content

Commit

Permalink
Adding API overview document
Browse files Browse the repository at this point in the history
  • Loading branch information
dorodnic committed Jul 7, 2017
1 parent f7b7f50 commit 8a54cff
Show file tree
Hide file tree
Showing 7 changed files with 43 additions and 6 deletions.
36 changes: 36 additions & 0 deletions doc/api_arch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# API Architecture

RealSense APIs is clustered into three categories, aimed at different usages:
1. [High-Level Computer-Vision Pipeline API](#High-Level-Pipeline-API) - If you are getting started with RealSense and want do build something that would just work, Pipeline API will configure the device with the best recommended settings and will manage hardware resources as well as threading on your behalf. Recommended for **Researchers** and **App Developers**.
2. [Processig Building-Blocks](#Processing-Blocks) - If you need to take control over threading, spatial and temporal syncronization and memory management, we provide a set of building blocks you can easily arrange into custom processing flows. Recommended for **Game Developers**.
3. [Low-Level Sensor Access](#Low-Level-Sensor-API) - If you know what you are doing you can take direct control over the individual sensors composing the device to get the maximum out of the hardware. Recommended for **Framework and Tools Developers**, as well as Developers in emerging feilds like **VR/AR**.

## Low-Level Sensor API
RealSense devices are composed from sensors. Some are commonplace, like a regular RGB camera, some are more exotic, like RS400 Stereo module or SR300 Structured-Light sensor:
<img src="img/realsense.gif" width="400" height="200" />
Each sensor has its own power management and control. Standard sensors pass UVC / HID complience tests and can be used without custom software. Different sensors can be safely used from different applications and can only influence each other indirectly.
Each sensor can offer one or more streams. Streams must be configured together and are usually dependent on each other. For example, RS400 Depth stream depends on Infrared streams, so they must be initiated together with a single resolution and FPS.
All sensors provide streaming (not nesseserily of images) but each individual sensor can be extended to offer additional APIs. For example, most video devices can let the user configure region of interest for the auto-exposure algorithm. RS400 stereo module can offer **Advanced Mode** functionality, letting you control the various ASIC registers responsible for depth generation.
The user of sensor API provides a callback to be invoked whenever new data frame becomes avaialable. The callback runs immediately on the thread that received the frame from the OS providing best possible latency. Whenever possible librealsense will avoid any unnessesary copies of the frame data.

TODO: Sensor enumeration code snippet
TODO: Sensor streaming code snippet
TODO: Link to sensor class
TODO: Advanced Mode code snippet

## Processing Blocks

* If you are not interested in the individual sensors of the device, you can configure the device for streaming using `config` class.
* To marshal frames from OS callback thread to the main application we provide safe `queue` implementation.
* If you wish to syncronize any set of different asynchronous streams with respect to hardware timestamps, we offer `syncer` class.
* If you need to align streams to a single viewport, we offer `TODO` class for that. You can also use your own calibration data to align otherwise uncalibrated devices.
* If you need to perfom processing on the frame while benefiting from librealsense memory pooling, we offer `processing` class for that.
Putting all these together lets you define your own custom processing pipeline, best suited for your application.

## High-Level Pipeline API

To further simplify working with the camera, we introduce the concept of `middleware`. Each middleware knows the best configuration recommended for its use-case and knows what processing has to be done. You can use `pipeline` class to execute one or possibly many middlewares. The pipeline will make sure all syncronization and alignment requirements are met and will take care of threading and resource management.

TODO: Link to background removal
TODO: Link to measurement tool?
TODO: Code snippet with pipeline
File renamed without changes
Binary file added doc/img/sensors.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
2 changes: 1 addition & 1 deletion doc/installation_windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
**Note:** Due to the USB 3.0 translation layer between native hardware and virtual machine, the librealsense team does not recommend or support installation in a VM.

librealsense can be compiled on Windows using CMake and Visual Studio 2015:
![Windows CMake](./windows_cmake.png)
![Windows CMake](./img/windows_cmake.png)
Don't forget to check `BUILD_EXAMPLES` if you wish to use librealsense samples.

**Note:** When working on Windows 8.1, make sure you have [KB3075872](https://support.microsoft.com/en-us/kb/3075872) and [KB2919355](https://support.microsoft.com/en-us/kb/2919355) installed. These patches are addressing issues with 8.1 video drivers, resolved in Windows 10.
2 changes: 1 addition & 1 deletion doc/rs400_support.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Starting with the RS400 RealSense camera, Intel Perceptual Computing group is in
* Software support for RealSense™ will be split between two coexisting lineups of releases: librealsense 1.x and librealsense 2+.
* librealsense 1.x will continue to provide support for the following generations of RealSense devices: F200, R200 and ZR300.
* librealsense 2+ will support the next generation of RealSense devices, starting with the RS400.
![RS400 support](ds5_branches.png)
![RS400 support](img/ds5_branches.png)

## API Changes

Expand Down
9 changes: 5 additions & 4 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,11 @@ Developer kits containing the necessary hardware to use this library are availab

A comprehensive suite of sample and tutorial applications are provided in the `/examples` subdirectory. For new users, it is best to review the tutorial series of apps which are designed to progressively introduce API features.

* [C API](./include/librealsense/rs.h) - With doxygen-style API comments
* To build documentation locally from sources, on Ubuntu run the following commands:
* `sudo apt-get install doxygen`
* `doxygen doc/doxygen/doxyfile`
* ![ new ](https://img.shields.io/badge/*-new-yellow.svg?style=plastic)[API Architecture](./doc/api_arch.md) - Overview of the high-level concepts
* [C API](./include/librealsense/rs.h) - With doxygen-style API comments
* To build documentation locally from sources, on Ubuntu run the following commands:
* `sudo apt-get install doxygen`
* `doxygen doc/doxygen/doxyfile`
* [C API](./include/librealsense2/rs.h) - With doxygen-style API comments
* [Frame Management](./doc/frame_lifetime.md) - Frame Memory Management, Threading Model and Syncronization
* [Frame Metadata](./doc/frame_metadata.md) - support for frame-metadata attributes
Expand Down

0 comments on commit 8a54cff

Please sign in to comment.