From 779cfa48a01657b0fb7ac5473f6d1ae99a858b41 Mon Sep 17 00:00:00 2001 From: Steux Yoann Date: Fri, 22 Nov 2024 14:55:10 +0000 Subject: [PATCH] doc: refacto doc dev --- .../developer_guide/algorithm_conception.rst | 125 ++++++++ .../concepts}/application.rst | 18 +- .../concepts}/carsdataset.rst | 27 +- .../concepts}/cluster_mp.rst | 61 ++-- .../developer_guide/concepts/orchestrator.rst | 272 ++++++++++++++++++ .../developer_guide/concepts/plugin.rst | 181 ++++++++++++ .../concepts/software_conception.rst | 65 +++++ .../contributing_to_cars.rst} | 13 +- .../developer_guide/creating_a_plugin.rst | 5 + docs/source/developer_guide/faq_dev.rst | 5 + docs/source/developer_guide/index.rst | 15 + .../developer_guide/troubleshooting.rst | 6 + docs/source/index.rst | 3 +- docs/source/software_design/index.rst | 37 --- docs/source/software_design/interaction.rst | 11 - docs/source/software_design/orchestrator.rst | 243 ---------------- docs/source/software_design/plugin.rst | 175 ----------- 17 files changed, 740 insertions(+), 522 deletions(-) create mode 100644 docs/source/developer_guide/algorithm_conception.rst rename docs/source/{software_design => developer_guide/concepts}/application.rst (98%) rename docs/source/{software_design => developer_guide/concepts}/carsdataset.rst (91%) rename docs/source/{software_design => developer_guide/concepts}/cluster_mp.rst (89%) create mode 100644 docs/source/developer_guide/concepts/orchestrator.rst create mode 100644 docs/source/developer_guide/concepts/plugin.rst create mode 100644 docs/source/developer_guide/concepts/software_conception.rst rename docs/source/{contributing_the_project.rst => developer_guide/contributing_to_cars.rst} (99%) create mode 100644 docs/source/developer_guide/creating_a_plugin.rst create mode 100644 docs/source/developer_guide/faq_dev.rst create mode 100644 docs/source/developer_guide/index.rst create mode 100644 docs/source/developer_guide/troubleshooting.rst delete mode 100644 docs/source/software_design/index.rst delete mode 100644 docs/source/software_design/interaction.rst delete mode 100644 docs/source/software_design/orchestrator.rst delete mode 100644 docs/source/software_design/plugin.rst diff --git a/docs/source/developer_guide/algorithm_conception.rst b/docs/source/developer_guide/algorithm_conception.rst new file mode 100644 index 00000000..a2182ed2 --- /dev/null +++ b/docs/source/developer_guide/algorithm_conception.rst @@ -0,0 +1,125 @@ +.. role:: raw-html(raw) + :format: html + + +.. _algorithm_conception: + +==================== +Algorithm Conception +==================== + + +.. tabs:: + + .. tab:: Grid Generation + + :raw-html:`

Method

` + + + :raw-html:`

Limits of the method

` + + + :raw-html:`

Implementation

` + + + + + .. tab:: Resampling + + :raw-html:`

Method

` + + + :raw-html:`

Limits of the method

` + + + :raw-html:`

Implementation

` + + + .. tab:: Sparse matching + + :raw-html:`

Method

` + + + :raw-html:`

Limits of the method

` + + + :raw-html:`

Implementation

` + + + .. tab:: Grid correction + + :raw-html:`

Method

` + + + :raw-html:`

Limits of the method

` + + + :raw-html:`

Implementation

` + + + .. tab:: Disparity a priory computation + + :raw-html:`

Method

` + + + :raw-html:`

Limits of the method

` + + + :raw-html:`

Implementation

` + + .. tab:: DEM generation + + :raw-html:`

Method

` + + + :raw-html:`

Limits of the method

` + + + :raw-html:`

Implementation

` + + + .. tab:: Dense matching + + :raw-html:`

Method

` + + + :raw-html:`

Limits of the method

` + + + :raw-html:`

Implementation

` + + + .. tab:: Triangulation + + :raw-html:`

Method

` + + + :raw-html:`

Limits of the method

` + + + :raw-html:`

Implementation

` + + + .. tab:: Point cloud filtering + + :raw-html:`

Method

` + + + :raw-html:`

Limits of the method

` + + + :raw-html:`

Implementation

` + + + .. tab:: Rasterization + + :raw-html:`

Method

` + + + :raw-html:`

Limits of the method

` + + + :raw-html:`

Implementation

` + + + diff --git a/docs/source/software_design/application.rst b/docs/source/developer_guide/concepts/application.rst similarity index 98% rename from docs/source/software_design/application.rst rename to docs/source/developer_guide/concepts/application.rst index 06fa1a14..8ee88608 100644 --- a/docs/source/software_design/application.rst +++ b/docs/source/developer_guide/concepts/application.rst @@ -1,10 +1,14 @@ + +.. role:: raw-html(raw) + :format: html + .. _application: -Application -=========== +:raw-html:`

Application

` + + +**Overview** -Overview --------- An *application* is a main step of CARS 3D reconstruction framework. It contains algorithm methods. @@ -17,13 +21,13 @@ It is composed of: * Some abstract applications (each one defined a main 3d step) * Some subclass associated to each abstract application, containing specific algorithm -.. figure:: ../images/application_concept.png +.. figure:: ../../images/application_concept.png :align: center :alt: Applications -Example -------- +**Example** + Let's take an example of `dense_matching` application to describe the main steps: diff --git a/docs/source/software_design/carsdataset.rst b/docs/source/developer_guide/concepts/carsdataset.rst similarity index 91% rename from docs/source/software_design/carsdataset.rst rename to docs/source/developer_guide/concepts/carsdataset.rst index 2cdefd5f..45835977 100644 --- a/docs/source/software_design/carsdataset.rst +++ b/docs/source/developer_guide/concepts/carsdataset.rst @@ -1,10 +1,15 @@ .. _cars_dataset: -CarsDataset -=========== -Goals ------ +.. role:: raw-html(raw) + :format: html + +:raw-html:`

CarsDataset

` + + + +**Goals** + *CarsDataset* is the CARS internal data structure. The data used within CARS imposes some restrictions that the structure must manage: @@ -16,17 +21,17 @@ The data used within CARS imposes some restrictions that the structure must mana *CarsDataset* aims at defining a generic data structure that takes into account this constraints. -Details -------- +**Details** + -.. figure:: ../images/Carsdataset.png +.. figure:: ../../images/Carsdataset.png :align: center :alt: CarsDataset concept Here is an example of one dataset with all needed information. -Attributes -^^^^^^^^^^ +*Attributes* + * *type* : CarsDataset can manage `Xarray.dataset` or `pandas.DataFrame` * *tiles* : List of list of `Xarray.dataset` or `pandas.DataFrame`. Include overlaps. @@ -41,8 +46,8 @@ Attributes It is important to note that a tile, if even if you'decided to use `Xarray.dataset` or `pandas.DataFrame` could be a `delayed` or `future` related to `dask` definition. See next sections. -Functions -^^^^^^^^^ +*Functions* + *CarsDataset* integrates all functions for manipulating the data throughout the framework: diff --git a/docs/source/software_design/cluster_mp.rst b/docs/source/developer_guide/concepts/cluster_mp.rst similarity index 89% rename from docs/source/software_design/cluster_mp.rst rename to docs/source/developer_guide/concepts/cluster_mp.rst index 24b3c35e..c0fd8203 100644 --- a/docs/source/software_design/cluster_mp.rst +++ b/docs/source/developer_guide/concepts/cluster_mp.rst @@ -1,24 +1,23 @@ -.. _cluster_mp: -Cluster Multiprocessing -======================= +**Cluster Multiprocessing** -Goals ------ -The multiprocessing (MP) cluster facilitates the distribution of computing for the :ref:`application` and the management of :ref:`cars_dataset` data. +**Goals** -Details -------- +The multiprocessing (MP) cluster facilitates the distribution of computing for the application and the management of cars_dataset data. + + +**Details** + The MP cluster is built upon `Python's multiprocessing`_ module using the forkserver mode. In this mode, a pool of worker processes handles the parallel execution of functions. Each worker process is single-threaded, and only essential resources are inherited. By design, CARS utilizes disk-based registry for data storage, distributing data across the processes. If specified in configuration, data distribution can be done in memory, with degraded performance. .. _`Python's multiprocessing`: https://docs.python.org/3/library/multiprocessing.html -How it works ------------- +**How it works** + The main class is the MP Cluster, which inherits from the AbstractCluster class. It is instantiated within the orchestrator. @@ -26,38 +25,38 @@ Inspired by the Dask cluster approach, the MP cluster initiates a list of delaye Factorisation of tasks allows to reduce the number of tasks without losing any time. Reducing the number of tasks permits to reduce the number of dumps on disk and to save time. For each task that has available data (intermediate results input from the linked previous task), the MP cluster transforms the delayed task into an MpFutureTask. -Upon completion of these jobs, the results are saved on disk, and the reference is passed to the next job. The :ref:`refresh_task_cache` function serves as the primary control function of the MP cluster. +Upon completion of these jobs, the results are saved on disk, and the reference is passed to the next job. The refresh_task_cache function serves as the primary control function of the MP cluster. The next sections illustrates the architecture of the MP cluster, while the API provides detailed functions that offer more insight into interactions and operations. -Class diagram -^^^^^^^^^^^^^ -.. image:: ../images/mp_cluster.svg +*Class diagram* + +.. image:: ../../images/mp_cluster.svg :align: center -API detailed functions -^^^^^^^^^^^^^^^^^^^^^^ +*API detailed functions* + **init** -++++++++ + Cluster allocation using a Python thread pool. The worker pool is set up in forkserver mode with a specified number of workers, job timeouts, and wrapper configuration for cluster logging. **create_task_wrapped** -+++++++++++++++++++++++ + Declare task as **MpDelayed** within the cluster. **MpDelayed** are instantiated using the **mp_delayed_builder** wrapper builder. Furthermore, the wrapper provides parameters for the job logger. **start_tasks** -+++++++++++++++ + Factorize tasks with **mp_factorizer.factorize_tasks** and add future tasks in the cluster queue. The cluster processes tasks from the queue. Transform **MpDelayed** with rec_start to **MpJob**, and calculate task dependencies for each job. **mp_factorizer.factorize_tasks** -+++++++++++++++++++++++++++++++++ + Take as input a list of final **MpDelayed** and factorize all the dependent tasks that are *factorizable*. A task **t** of the class **MpDelayedTask** is *factorizable* if : @@ -81,7 +80,7 @@ will be replaced by output of **t1** and then **t2** will be computed. Thus, the **rec_start** -+++++++++++++ + Transform delayed tasks to MpJob and create MpFuture objects to retrieve results. For each task: @@ -98,9 +97,9 @@ For each task: .. _refresh_task_cache: +# refresh_task_cache + -**refresh_task_cache** -++++++++++++++++++++++ At each refresh: 1. Sleep (refresh time). @@ -137,41 +136,41 @@ At each refresh: **get_ready_failed_tasks** -++++++++++++++++++++++++++ + Retrieve the new ready tasks and failed tasks. **get_tasks_without_deps** -++++++++++++++++++++++++++ + A static method evaluates a list of tasks that are ready and lack dependencies, excluding those deemed as initial tasks. The initial tasks of the graph have no priority. In order to enhance disk usage efficiency, the cluster initiates with N initial tasks (where N equals the number of workers), assigning priority to the subsequent connected tasks. After finishing a segment of the task graph, the cluster introduces N new initial tasks to continue the process. **future_iterator** -+++++++++++++++++++ + Enable the initiation of all tasks from the orchestrator controller. **get_job_ids_from_futures** -++++++++++++++++++++++++++++ + Obtain a list of job IDs from the future list. **replace_job_by_data** -+++++++++++++++++++++++ + Substitute MpJob instances in lists or dict with their actual data. **compute_dependencies** -++++++++++++++++++++++++ + Compute job result dependencies from args and kw_args. **MpFutureTask** -++++++++++++++++ + A multiprocessing version of the Dask distributed.future. This class encapsulates data and references to job cluster threads. It also facilitates the sharing of references between jobs and cleaning cache operations. **log_error_hook** -++++++++++++++++++ + A custom Exception hook to manage cluster thread exceptions. diff --git a/docs/source/developer_guide/concepts/orchestrator.rst b/docs/source/developer_guide/concepts/orchestrator.rst new file mode 100644 index 00000000..e1129d64 --- /dev/null +++ b/docs/source/developer_guide/concepts/orchestrator.rst @@ -0,0 +1,272 @@ +.. _orchestrator: + +.. role:: raw-html(raw) + :format: html + +:raw-html:`

Orchestrator

` + + + +**Goals** + + +The *orchestrator* is the central element of CARS concepts. +Its role is to ensure the communication between the *computing technology*, the *applications* and the *CarsDatasets*. + +**Details** + + +The *orchestrator* is unique and instantiated for each pipeline: + + +.. sourcecode:: python + + with orchestrator.Orchestrator(distributed_conf=distributed_conf) as cars_orchestrator: + + +It is mainly composed of: + +* a *cluster* +* a *CarsDatasetRegistry* +* a dictionary, later saved in a json output file, containing information given by applications + +.. tabs:: + + .. tab:: Cluster + + .. _cluster: + + **Cluster** + + + The cluster is the component managing access to all the threads available to CARS for its calculations. + + .. sourcecode:: python + + class AbstractCluster(metaclass=ABCMeta): + + + ... + + @abstractmethod + def create_task(self, func, nout=1): + """ + Create task + + :param func: function + :param nout: number of outputs + """ + + @abstractmethod + def start_tasks(self, task_list): + """ + Start all tasks + + :param task_list: task list + """ + + @abstractmethod + def future_iterator(self, future_list): + """ + Iterator, iterating on computed futures + + :param future_list: future_list list + """ + + + + The two main functions are: + + * `create_task` to declare a task to the cluster. It returns a `delayed` object. + * `start_tasks` to compute each task that has been declared. + * `future_iterator`: iterate over the `future` objects + + There are already 3 plugins, each one representing a mode: + + * *dask* + + * *local_dask* + * *pbs_dask* + * *slurm_dask* + + * *mp* (for mutliprocessing) + * *sequential* : (note: in this mode, `delayed` objects do not exist. They will instead directly be of type `Xarray.dataset` or `Panda.Dataframe`) + + .. tabs:: + + .. tab:: Multiprocessing Cluster + + .. include:: cluster_mp.rst + + .. tab:: Dask Cluster + + .. tab:: Sequential Cluster + + + .. tab:: Registries + + The *CarsDatasetRegistry* is a class that enables the management of the list of *CarsDatasets* that the user wants to save. + It is mainly composed of: + + * a registry *CarsDataset* list + * an id associated to each registered *CarsDataset* + + There are some functions that allow a user to: + + * Add new *CarsDataset* to registry + * Obtain an ID for a CarsDataset + * Find a *CarsDataset* from an ID + * Manage saving tile by tile (i.e future by future, related to `dask` terms), by using the `SingleCarsDatasetSaver` that wraps `CarsDataset` save functions. + + + .. tabs:: + + .. tab:: Saver Registry + WIP + .. tab:: Replacer Registry + WIP + .. tab:: Unseen Registry + WIP + .. tab:: Compute Registry + + WIP + + **How it works** + + + 1. Instantiate *orchestrator* before every pipeline with a configuration file that defines cluster mode and output directory + + .. sourcecode:: python + + with orchestrator.Orchestrator(distributed_conf=distributed_conf) as cars_orchestrator: + + + *Cluster* and *CarsDatasetRegistry* are created + + .. sourcecode:: python + + def __init__(self, distributed_conf=None): + + """ + Init function of Orchestrator. + Creates Cluster and Registry for CarsDatasets + + :param distributed_conf: configuration of distribution + """ + + # out_dir + self.out_dir = None + if "out_dir" in distributed_conf: + self.out_dir = distributed_conf["out_dir"] + else: + logging.error("No out_dir defined") + + self.nb_workers = 1 + if "nb_workers" in distributed_conf: + self.nb_workers = distributed_conf["nb_workers"] + + # init cluster + self.cluster = AbstractCluster( # pylint: disable=E0110 + distributed_conf + ) + + # init CarsDataset savers registry + self.cars_ds_savers_registry = CarsDatasetsRegistry() + + # init saving lists + self.cars_ds_list = [] + + # outjson + self.out_json = {} + + + 2. *Orchestrator* is used in every applications which can add *CarsDataset* to save (*orchestrator* interacts with *CarsDatasetRegistry*) + + .. sourcecode:: python + + def add_to_save_lists( + self, file_name, tag, cars_ds, dtype="float32", nodata=0 + ): + """ + Save file to list in order to be saved later + + :param file_name: file name + :param tag: tag + :param cars_ds: cars dataset to register + """ + + self.cars_ds_savers_registry.add_file_to_save( + file_name, cars_ds, tag=tag, dtype=dtype, nodata=nodata + ) + + if cars_ds not in self.cars_ds_list: + self.cars_ds_list.append(cars_ds) + + + 3. *Orchestrator* can be used to obtain *CarsDataset* ID (see application) + + .. sourcecode:: python + + def get_saving_infos(self, cars_ds_list): + """ + Get saving infos of given cars datasets + + :param cars_ds_list: list of cars datasets + :type cars_ds_list: list[CarsDataset] + + :return : list of saving infos + :rtype: list[dict] + """ + + saving_infos = [] + + for cars_ds in cars_ds_list: + saving_infos.append( + self.cars_ds_savers_registry.get_saving_infos(cars_ds) + ) + + return saving_infos + + 4. At the end of the pipeline, the `__exit__` function is called automatically. It computes all `delayed` needed for saving *CarsDataset* + using `cluster.start_tasks` function` that returns `future` objects. + And the `save` function of *CarsDatasetRegistry* is called for saving by iterating on `future` objects. + + .. sourcecode:: python + + def __exit__(self, exc_type, exc_value, traceback): + """ + Function run on exit. + + Compute cluster tasks, save futures to be saved, and cleanup cluster + and files + + """ + + # run compute and save files + logging.info("Compute delayed ...") + # Flatten to list + delayed_objects = flatten_object( + self.cars_ds_savers_registry.get_cars_datasets_list() + ) + + # Compute delayed + future_objects = self.cluster.start_tasks(delayed_objects) + + # Save objects when they are computed + logging.info("Wait for futures results ...") + pbar = tqdm(total=len(future_objects), desc="Processing Futures ...") + for future_obj in tqdm(self.cluster.future_iterator(future_objects)): + # get corresponding CarsDataset and save tile + if future_obj is not None: + self.cars_ds_savers_registry.save(future_obj) + else: + logging.debug("None tile : not saved") + pbar.update() + + # close files + logging.info("Close files ...") + self.cars_ds_savers_registry.cleanup() + + # close cluster + logging.info("Close cluster ...") + self.cluster.cleanup() diff --git a/docs/source/developer_guide/concepts/plugin.rst b/docs/source/developer_guide/concepts/plugin.rst new file mode 100644 index 00000000..26ec1cd0 --- /dev/null +++ b/docs/source/developer_guide/concepts/plugin.rst @@ -0,0 +1,181 @@ +.. _plugin: + +.. role:: raw-html(raw) + :format: html + +:raw-html:`

Plugin

` + + +.. tabs:: + + .. tab:: Geometry Plugin + + Geometry plugins aim to enable the use of different geometry libraries, typically `libGEO `_ or `Shareloc `_ to perform CARS geometric operations which require the interpretation of the geometric models of the pairs to process. + + Those operations are: + + * The epipolar grids computation + * The direct localization operation + * The lines of sight triangulation + + `SharelocGeometry` is an internal geometry plugin used in the baseline installations of CARS. + In the current state, Shareloc has to be installed when using CARS as this class is imported in the step module. + + *Geometry abstract class* + + The CARS abstract geometry class, named `AbstractGeometry`, is defined in the core geometry module (`cars/core/geometry/__init__.py`). + Considering the geometry operations, CARS has its own internal implementation of the CARS abstract class using Shareloc. External plugins can be used if they are registered correctly : + + .. code-block:: python + + from cars.core.geometry.abstract_geometry import AbstractGeometry + + @AbstractGeometry.register_subclass("GeometryPluginName") + class GeometryPluginName(AbstractGeometry): + ... + + In order to make an external plugin work with CARS, it is mandatory to use the entry point `cars.plugins` at the setup of the package to register the AbstractGeometry object. + For example, if the AbstractGeometry object is defined in file `cars_geometry_plugin_name.cars_geometry_plugin_name`, this code has to present in the file `cars_geometry_plugin_name.setup.py` + + .. code-block:: python + + setup( + entry_points={ + "cars.plugins": [ + "plugin=cars_geometry_plugin_name.cars_geometry_plugin_name :GeometryPluginName" + ] + }, + ) + + *Mandatory methods* + + Currently, the `AbstractGeometry` class requires the implementation of the following mandatory methods and properties: + + * `conf_schema` which specify the user input json schema required by the geometric library. + + .. code-block:: python + + def conf_schema(self): + """ + Returns the input configuration fields required by the geometry plugin + as a json checker schema. The available fields are defined in the + cars/conf/input_parameters.py file + + :return: the geo configuration schema + """ + + * `check_products_consistency` which check if the geometrical model filled by the user is readable by the geometric library. + + .. code-block:: python + + def check_products_consistency(cars_conf) -> bool: + """ + Test if the product is readable by the geometry plugin + + :param: cars_conf: cars input configuration + :return: True if the products are readable, False otherwise + """ + + * `triangulate` which is a method performing the triangulation from a disparity map or a set of matching points (mode parameter). + + .. code-block:: python + + def triangulate( + sensor1, + sensor2, + geomodel1, + geomodel2, + mode: str, + matches: Union[xr.Dataset, np.ndarray], + grid1: str, + grid2: str, + roi_key: Union[None, str] = None, + ) -> np.ndarray: + """ + Performs triangulation from cars disparity or matches dataset + + :param sensor1: path to left sensor image + :param sensor2: path to right sensor image + :param geomodel1: path and attriutes for left geomodel + :param geomodel2: path and attriutes for right geomodel + :param mode: triangulation mode + (constants.DISP_MODE or constants.MATCHES) + :param matches: cars disparity dataset or matches as numpy array + :param grid1: path to epipolar grid of img1 + :param grid2: path to epipolar grid of image 2 + :param roi_key: dataset roi to use + (can be cst.ROI or cst.ROI_WITH_MARGINS) + :return: the long/lat/height numpy array in output of the triangulation + """ + + * `generate_epipolar_grids` which generates the left and right epipolar grids from the images of the pair and their geometrical models. + + .. code-block:: python + + def generate_epipolar_grids( + self, + sensor1, + sensor2, + geomodel1, + geomodel2, + epipolar_step: int = 30, + ) -> Tuple[ + np.ndarray, np.ndarray, List[float], List[float], List[int], float + ]: + """ + Computes the left and right epipolar grids + + :param sensor1: path to left sensor image + :param sensor2: path to right sensor image + :param geomodel1: path to left geomodel + :param geomodel2: path to right geomodel + :param epipolar_step: step to use to construct the epipolar grids + :return: Tuple composed of : + + - the left epipolar grid as a numpy array + - the right epipolar grid as a numpy array + - the left grid origin as a list of float + - the left grid spacing as a list of float + - the epipolar image size as a list of int \ + (x-axis size is given with the index 0, y-axis size with index 1) + - the disparity to altitude ratio as a float + """ + + * `direct_loc` which performs direct localization operations. + + .. code-block:: python + + def direct_loc( + self, + sensor, + geomodel, + x_coord: list, + y_coord: list, + z_coord: list = None + ) -> np.ndarray: + """ + For a given image points list, compute the latitudes, longitudes, altitudes + + Advice: to be sure, use x,y,z list inputs only + + :param sensor: path to sensor image + :param geomodel: path and attributes for geomodel + :param x_coord: X Coordinates list in input image sensor + :param y_coord: Y Coordinates list in input image sensor + :param z_coord: Z Altitude coordinates list to take the image + :return: Latitude, Longitude, Altitude coordinates list as a numpy array + """ + + Where `constants` corresponds to the `cars/core/constants.py` module. + + *Available methods* + + Some methods are available in the `AbstractGeometry` class that might be useful for any geometry plugin which would only perform the triangulation using sensor coordinates. + CARS' API only provides as inputs of the geometry plugin triangulation method the epipolar coordinates for each image of the pair. Thus the `matches_to_sensor_coords` method enables any plugin to convert those coordinates into the corresponding sensor ones. + + `AbstractGeometry` implements the method `image_envelope`. It computes the ground footprint of an image in sensor geometry by projecting its four corners using the direct localization method. This method can be overloaded by any geometry plugin if necessary. + + + .. tab:: Application Plugin + + .. tab:: Pipeline Plugin \ No newline at end of file diff --git a/docs/source/developer_guide/concepts/software_conception.rst b/docs/source/developer_guide/concepts/software_conception.rst new file mode 100644 index 00000000..9ba687be --- /dev/null +++ b/docs/source/developer_guide/concepts/software_conception.rst @@ -0,0 +1,65 @@ +.. _software_conception: + +.. role:: raw-html(raw) + :format: html + +:raw-html:`

Software

` + + + +CARS design aims a modular and customizable framework for multiview 3d reconstruction. +This design is organized around key concepts described in this section. + +.. warning:: + + Under construction with CARS design evolution. + + +The CARS framework can be introduced by the following diagram: + +.. figure:: ../../images/design_overview.png + :align: center + :alt: Cars Framework + +This section presents one by one the CARS key concepts and their interaction. + + * cars_dataset Input and output object of an application. Contains a calculated and potentially tiled data. + * application: Algorithmic methods that takes + * Orchestrator: It instantiates and interfaces with the cluster to which it provides the tasks to be processed. It is responsible for writing the data calculated by the cluster on the fly. + * plugin: library or external tools providing specific 3d functions. Under heavy reconstruction ! + * **Pipeline**: A chain of applications ( 3d reconstruction steps) from input to output with intermediate data (CarsDataset) controlled by orchestrator; + + +.. tabs:: + + .. tab:: CarsDataset + + .. include:: carsdataset.rst + + + .. tab:: Application + + .. include:: application.rst + + + .. tab:: Orchestrator + + .. include:: orchestrator.rst + + + + .. tab:: Plugin + + .. include:: plugin.rst + + + + +Detailed interaction between concepts +===================================== + +Now that all the concepts have been presented in details, we can draw a more technical diagram: + +.. figure:: ../../images/orchestrator_app_cluster_dataset.png + :align: center + :alt: Overview concepts details \ No newline at end of file diff --git a/docs/source/contributing_the_project.rst b/docs/source/developer_guide/contributing_to_cars.rst similarity index 99% rename from docs/source/contributing_the_project.rst rename to docs/source/developer_guide/contributing_to_cars.rst index 0fdfd2c2..1f1483c6 100644 --- a/docs/source/contributing_the_project.rst +++ b/docs/source/developer_guide/contributing_to_cars.rst @@ -1,6 +1,9 @@ -======================== -Contributing the project -======================== + +.. _contributing_to_cars: + +==================== +Contributing to CARS +==================== CARS is an open source software : don't hesitate to hack it and contribute ! @@ -112,7 +115,7 @@ To generate documentation, use: .. code-block:: console make docs - + The documentation is then build in docs/build directory and can be consulted with a web browser. Documentation can be edited in docs/source/ RST files. @@ -127,7 +130,7 @@ To generate a `Jupyter kernel `_ with CARS installa .. code-block:: console make notebook - + Follow indications to run a jupyter notebook. Kernel is created with following command (with cars-version updated): diff --git a/docs/source/developer_guide/creating_a_plugin.rst b/docs/source/developer_guide/creating_a_plugin.rst new file mode 100644 index 00000000..717f13dd --- /dev/null +++ b/docs/source/developer_guide/creating_a_plugin.rst @@ -0,0 +1,5 @@ +.. _creating_a_plugin: + +================= +Creating a plugin +================= \ No newline at end of file diff --git a/docs/source/developer_guide/faq_dev.rst b/docs/source/developer_guide/faq_dev.rst new file mode 100644 index 00000000..5f6c2609 --- /dev/null +++ b/docs/source/developer_guide/faq_dev.rst @@ -0,0 +1,5 @@ +.. _faq_dev: + +=== +FAQ +=== \ No newline at end of file diff --git a/docs/source/developer_guide/index.rst b/docs/source/developer_guide/index.rst new file mode 100644 index 00000000..d3530fa5 --- /dev/null +++ b/docs/source/developer_guide/index.rst @@ -0,0 +1,15 @@ +.. _developer_guide: + +=============== +Developer Guide +=============== + +.. toctree:: + :maxdepth: 2 + + concepts/software_conception + algorithm_conception + contributing_to_cars + creating_a_plugin + faq_dev + troubleshooting diff --git a/docs/source/developer_guide/troubleshooting.rst b/docs/source/developer_guide/troubleshooting.rst new file mode 100644 index 00000000..d93ee7d2 --- /dev/null +++ b/docs/source/developer_guide/troubleshooting.rst @@ -0,0 +1,6 @@ +.. _troubleshooting: + +=============== +Troubleshooting +=============== + diff --git a/docs/source/index.rst b/docs/source/index.rst index 0b4868cb..ec7c9918 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -39,9 +39,8 @@ CARS is designed for massive production and scalability |cars_isprs|. It aims to exploring_the_field/index.rst usage howto - software_design/index.rst troubleshooting_and_faqs - contributing_the_project + developer_guide/index.rst .. toctree:: diff --git a/docs/source/software_design/index.rst b/docs/source/software_design/index.rst deleted file mode 100644 index e0f1c57f..00000000 --- a/docs/source/software_design/index.rst +++ /dev/null @@ -1,37 +0,0 @@ -.. _software_design: - -=============== -Software design -=============== - -CARS design aims a modular and customizable framework for multiview 3d reconstruction. -This design is organized around key concepts described in this section. - -.. warning:: - - Under construction with CARS design evolution. - - -The CARS framework can be introduced by the following diagram: - -.. figure:: ../images/design_overview.png - :align: center - :alt: Cars Framework - -This section presents one by one the CARS key concepts and their interaction. - - * :ref:`cars_dataset` Input and output object of an application. Contains a calculated and potentially tiled data. - * :ref:`application`: Algorithmic methods that takes - * :ref:`Orchestrator`: It instantiates and interfaces with the cluster to which it provides the tasks to be processed. It is responsible for writing the data calculated by the cluster on the fly. - * :ref:`plugin`: library or external tools providing specific 3d functions. Under heavy reconstruction ! - * **Pipeline**: A chain of applications ( 3d reconstruction steps) from input to output with intermediate data (CarsDataset) controlled by orchestrator; - -.. toctree:: - :maxdepth: 2 - - carsdataset - orchestrator - cluster_mp - application - interaction - plugin diff --git a/docs/source/software_design/interaction.rst b/docs/source/software_design/interaction.rst deleted file mode 100644 index c4671259..00000000 --- a/docs/source/software_design/interaction.rst +++ /dev/null @@ -1,11 +0,0 @@ -.. _interaction: - -Detailed interaction between concepts -===================================== - -Now that all the concepts have been presented in details, we can draw a more technical diagram -than the one presented in the :ref:`software_design` section. - -.. figure:: ../images/orchestrator_app_cluster_dataset.png - :align: center - :alt: Overview concepts details diff --git a/docs/source/software_design/orchestrator.rst b/docs/source/software_design/orchestrator.rst deleted file mode 100644 index 16aa5006..00000000 --- a/docs/source/software_design/orchestrator.rst +++ /dev/null @@ -1,243 +0,0 @@ -.. _orchestrator: - -Orchestrator -============ - -Goals ------ - -The *orchestrator* is the central element of CARS concepts. -Its role is to ensure the communication between the *computing technology*, the *applications* and the *CarsDatasets*. - -Details -------- - -The *orchestrator* is unique and instantiated for each pipeline: - - -.. sourcecode:: python - - with orchestrator.Orchestrator(distributed_conf=distributed_conf) as cars_orchestrator: - - -It is mainly composed of: - -* a *cluster* -* a *CarsDatasetRegistry* -* dictionary for output json file containing information given by applications - -.. _cluster: - -Cluster -^^^^^^^ - -The cluster is the component which allows to realize the calculations. - -.. sourcecode:: python - - class AbstractCluster(metaclass=ABCMeta): - - - ... - - @abstractmethod - def create_task(self, func, nout=1): - """ - Create task - - :param func: function - :param nout: number of outputs - """ - - @abstractmethod - def start_tasks(self, task_list): - """ - Start all tasks - - :param task_list: task list - """ - - @abstractmethod - def future_iterator(self, future_list): - """ - Iterator, iterating on computed futures - - :param future_list: future_list list - """ - - - - The two main functions are: - -* `create_task` to declare a task to the cluster. It returns `delayed` object. -* `start_tasks` to compute each task that have been declared. -* `future_iterator`: iterate over the `future` objects - -There are already 3 plugins, each one representing a mode: - -* *dask* - - * *local_dask* - * *pbs_dask* - * *slurm_dask* - -* *mp* (for mutliprocessing) -* *sequential* : (note: `delayed` is note a real one, it is directly the data type, so `Xarray.dataset` or `Panda.Dataframe`) - - -CarsDatasetRegistry -^^^^^^^^^^^^^^^^^^^^^ - -The *CarsDatasetRegistry* is a class that allows to manage the list of *CarsDatasets* that user wants to save. -It is mainly composed of: - -* a registry *CarsDataset* list -* id associated to each registered *CarsDataset* - -There is some functions that allows to: - -* Add new *CarsDataset* to registry -* Obtain an ID for a CarsDataset -* Find a *CarsDataset* from an ID -* Manage saving tile by tile (i.e future by future, related to `dask` terms), by using the `SingleCarsDatasetSaver` that wraps `CarsDataset` save functions. - - -How it works -^^^^^^^^^^^^ - -1. Instantiate *orchestrator* before every pipeline with configuration file for defining cluster mode and output directory - -.. sourcecode:: python - - with orchestrator.Orchestrator(distributed_conf=distributed_conf) as cars_orchestrator: - - -*Cluster* and *CarsDatasetRegistry* are created - -.. sourcecode:: python - - def __init__(self, distributed_conf=None): - - """ - Init function of Orchestrator. - Creates Cluster and Registry for CarsDatasets - - :param distributed_conf: configuration of distribution - """ - - # out_dir - self.out_dir = None - if "out_dir" in distributed_conf: - self.out_dir = distributed_conf["out_dir"] - else: - logging.error("No out_dir defined") - - self.nb_workers = 1 - if "nb_workers" in distributed_conf: - self.nb_workers = distributed_conf["nb_workers"] - - # init cluster - self.cluster = AbstractCluster( # pylint: disable=E0110 - distributed_conf - ) - - # init CarsDataset savers registry - self.cars_ds_savers_registry = CarsDatasetsRegistry() - - # init saving lists - self.cars_ds_list = [] - - # outjson - self.out_json = {} - - -2. *Orchestrator* is used in every applications which can add *CarsDataset* to save (*orchestrator* interacts with *CarsDatasetRegistry*) - -.. sourcecode:: python - - def add_to_save_lists( - self, file_name, tag, cars_ds, dtype="float32", nodata=0 - ): - """ - Save file to list in order to be saved later - - :param file_name: file name - :param tag: tag - :param cars_ds: cars dataset to register - """ - - self.cars_ds_savers_registry.add_file_to_save( - file_name, cars_ds, tag=tag, dtype=dtype, nodata=nodata - ) - - if cars_ds not in self.cars_ds_list: - self.cars_ds_list.append(cars_ds) - - -3. *Orchestrator* can be used to obtain *CarsDataset* ID (see :ref:`application`) - -.. sourcecode:: python - - def get_saving_infos(self, cars_ds_list): - """ - Get saving infos of given cars datasets - - :param cars_ds_list: list of cars datasets - :type cars_ds_list: list[CarsDataset] - - :return : list of saving infos - :rtype: list[dict] - """ - - saving_infos = [] - - for cars_ds in cars_ds_list: - saving_infos.append( - self.cars_ds_savers_registry.get_saving_infos(cars_ds) - ) - - return saving_infos - -4. At the end of the pipeline, the `__exit__` function is called automatically. It computes all `delayed` needed for saving *CarsDataset* -using `cluster.start_tasks` function` that returns `future` objects. -And the `save` function of *CarsDatasetRegistry* is called for saving by iterating on `future` objects. - -.. sourcecode:: python - - def __exit__(self, exc_type, exc_value, traceback): - """ - Function run on exit. - - Compute cluster tasks, save futures to be saved, and cleanup cluster - and files - - """ - - # run compute and save files - logging.info("Compute delayed ...") - # Flatten to list - delayed_objects = flatten_object( - self.cars_ds_savers_registry.get_cars_datasets_list() - ) - - # Compute delayed - future_objects = self.cluster.start_tasks(delayed_objects) - - # Save objects when they are computed - logging.info("Wait for futures results ...") - pbar = tqdm(total=len(future_objects), desc="Processing Futures ...") - for future_obj in tqdm(self.cluster.future_iterator(future_objects)): - # get corresponding CarsDataset and save tile - if future_obj is not None: - self.cars_ds_savers_registry.save(future_obj) - else: - logging.debug("None tile : not saved") - pbar.update() - - # close files - logging.info("Close files ...") - self.cars_ds_savers_registry.cleanup() - - # close cluster - logging.info("Close cluster ...") - self.cluster.cleanup() diff --git a/docs/source/software_design/plugin.rst b/docs/source/software_design/plugin.rst deleted file mode 100644 index 521221fd..00000000 --- a/docs/source/software_design/plugin.rst +++ /dev/null @@ -1,175 +0,0 @@ -.. _plugin: - -Plugin -====== - -Geometry plugin -^^^^^^^^^^^^^^^^^ - -Geometry plugins aim to enable the use of different geometry libraries, typically `libGEO `_ or `Shareloc `_ to perform CARS geometric operations which require the interpretation of the geometric models of the pairs to process. - -Those operations are: - -* The epipolar grids computation -* The direct localization operation -* The lines of sight triangulation - -`SharelocGeometry` is an internal geometry plugin used in the baseline installations of CARS. -In the current state, Shareloc has to be installed when using CARS as this class is imported in the step module. - -Geometry abstract class -+++++++++++++++++++++++ - -The CARS abstract geometry class, named `AbstractGeometry`, is defined in the core geometry module (`cars/core/geometry/__init__.py`). -Considering the geometry operations, CARS have its own internal implementation of the CARS abstract class using Shareloc External plugins can be used if they are registered correctly : - -.. code-block:: python - - from cars.core.geometry.abstract_geometry import AbstractGeometry - - @AbstractGeometry.register_subclass("GeometryPluginName") - class GeometryPluginName(AbstractGeometry): - ... - -In order to make an external plugin work with CARS, it is mandatory to use the entry point `cars.plugins` at the setup of the package to register the AbstractGeometry object. -For example, if the AbstractGeometry object is defined in file `cars_geometry_plugin_name.cars_geometry_plugin_name`, this code has to present in the file `cars_geometry_plugin_name.setup.py` - -.. code-block:: python - - setup( - entry_points={ - "cars.plugins": [ - "plugin=cars_geometry_plugin_name.cars_geometry_plugin_name :GeometryPluginName" - ] - }, - ) - -Mandatory methods -++++++++++++++++++ - -Currently, the `AbstractGeometry` class requires the implementation of the following mandatory methods and properties: - -* `conf_schema` which specify the user input json schema required by the geometric library. - -.. code-block:: python - - def conf_schema(self): - """ - Returns the input configuration fields required by the geometry plugin - as a json checker schema. The available fields are defined in the - cars/conf/input_parameters.py file - - :return: the geo configuration schema - """ - -* `check_products_consistency` which check if the geometrical model filled by the user is readable by the geometric library. - -.. code-block:: python - - def check_products_consistency(cars_conf) -> bool: - """ - Test if the product is readable by the geometry plugin - - :param: cars_conf: cars input configuration - :return: True if the products are readable, False otherwise - """ - -* `triangulate` which is a method performing the triangulation from a disparity map or a set of matching points (mode parameter). - -.. code-block:: python - - def triangulate( - sensor1, - sensor2, - geomodel1, - geomodel2, - mode: str, - matches: Union[xr.Dataset, np.ndarray], - grid1: str, - grid2: str, - roi_key: Union[None, str] = None, - ) -> np.ndarray: - """ - Performs triangulation from cars disparity or matches dataset - - :param sensor1: path to left sensor image - :param sensor2: path to right sensor image - :param geomodel1: path and attriutes for left geomodel - :param geomodel2: path and attriutes for right geomodel - :param mode: triangulation mode - (constants.DISP_MODE or constants.MATCHES) - :param matches: cars disparity dataset or matches as numpy array - :param grid1: path to epipolar grid of img1 - :param grid2: path to epipolar grid of image 2 - :param roi_key: dataset roi to use - (can be cst.ROI or cst.ROI_WITH_MARGINS) - :return: the long/lat/height numpy array in output of the triangulation - """ - -* `generate_epipolar_grids` which generates the left and right epipolar grids from the images of the pair and their geometrical models. - -.. code-block:: python - - def generate_epipolar_grids( - self, - sensor1, - sensor2, - geomodel1, - geomodel2, - epipolar_step: int = 30, - ) -> Tuple[ - np.ndarray, np.ndarray, List[float], List[float], List[int], float - ]: - """ - Computes the left and right epipolar grids - - :param sensor1: path to left sensor image - :param sensor2: path to right sensor image - :param geomodel1: path to left geomodel - :param geomodel2: path to right geomodel - :param epipolar_step: step to use to construct the epipolar grids - :return: Tuple composed of : - - - the left epipolar grid as a numpy array - - the right epipolar grid as a numpy array - - the left grid origin as a list of float - - the left grid spacing as a list of float - - the epipolar image size as a list of int \ - (x-axis size is given with the index 0, y-axis size with index 1) - - the disparity to altitude ratio as a float - """ - -* `direct_loc` which performs direct localization operations. - -.. code-block:: python - - def direct_loc( - self, - sensor, - geomodel, - x_coord: list, - y_coord: list, - z_coord: list = None - ) -> np.ndarray: - """ - For a given image points list, compute the latitudes, longitudes, altitudes - - Advice: to be sure, use x,y,z list inputs only - - :param sensor: path to sensor image - :param geomodel: path and attributes for geomodel - :param x_coord: X Coordinates list in input image sensor - :param y_coord: Y Coordinates list in input image sensor - :param z_coord: Z Altitude coordinates list to take the image - :return: Latitude, Longitude, Altitude coordinates list as a numpy array - """ - -Where `constants` corresponds to the `cars/core/constants.py` module. - -Available methods -+++++++++++++++++ - -Some methods are available in the `AbstractGeometry` class that might be useful for any geometry plugin which would only perform the triangulation using sensor coordinates. -CARS' API only provides as inputs of the geometry plugin triangulation method the epipolar coordinates for each image of the pair. Thus the `matches_to_sensor_coords` method enables any plugin to convert those coordinates into the corresponding sensor ones. - -`AbstractGeometry` implements the method `image_envelope`. It computes the ground footprint of an image in sensor geometry by projecting its four corners using the direct localization method. This method can be overloaded by any geometry plugin if necessary. \ No newline at end of file