Merge pull request #29 from feelpp/9-reviewer2

Reviewer 2

article.hid2-ub-cicd.ppam24.tex

@@ -77,7 +77,7 @@
 
 
 \begin{abstract}
-The building sector in the European Union significantly impacts energy consumption and greenhouse gas emissions. The EU's Horizon 2050 initiative sets ambitious goals to reduce these impacts through enhanced building renovation rates. The CoE HiDALGO2 supports this initiative by developing high-performance computing solutions, specifically through the Urban Building pilot application, which utilizes advanced CI/CD methodologies to streamline simulation and deployment across various computational platforms, such as the EuroHPC JU supercomputers. The present work provides an overview of the Ktirio Urban Building framework (KUB), starting with an overview of the workflow and a description of some of the main ingredients of the software stack and discusses some current results performed on EuroHPC JU supercomputers using an innovative CI/CD pipeline.
+The building sector in the European Union significantly impacts energy consumption and greenhouse gas emissions. The EU's Horizon 2050 initiative sets ambitious goals to reduce these impacts through enhanced building renovation rates. The CoE HiDALGO2 supports this initiative by developing high-performance computing solutions, specifically through the Urban Building pilot application, which utilizes advanced CI/CD methodologies to streamline simulation and deployment across various computational platforms, such as the EuroHPC JU supercomputers. The present work provides an overview of the Ktirio Urban Building framework (KUB), starting with its workflow and a description of some of the main ingredients of the software stack, it then discusses some results performed on EuroHPC JU supercomputers using an innovative CI/
 
 \keywords{HPC, HPCOps, Urban building, City Energy Simulation.}
 
@@ -105,8 +105,14 @@ \section{Introduction}
 
 Implementing the proposed KUB workflow will facilitate more informed urban planning decisions, support policy development for energy efficiency, and contribute to reducing urban greenhouse gas emissions. The project also focuses on enhancing the interactions between different environmental models to provide a holistic view of urban ecosystems.
 
-This paper is organized as follows: \cref{sec:kub-workflow} discusses KUB's current workflow, \cref{sec:urban-building} provides an overview of some modeling and simulation components, and finally, \cref{sec:cicd-framework} delves into the CI/CD processes from standard DevOps to HPCops tailored for EuroHPC environments.
+This paper is organized as follows: 
+\begin{inparaenum}[\it (i)]
+    \item \cref{sec:kub-workflow} presents the KUB workflow, detailing the data handling and simulation processes, and the final analysis and urban scale energy evaluation;
+    \item \cref{sec:urban-building} provides an overview of the geometrical and physical modeling components of the urban building application;
+    \item \cref{sec:cicd-framework} delves into the CI/CD processes tailored for EuroHPC environments;
+    \item \cref{sec:conclusion} concludes the paper.
 
+\end{inparaenum}
 %Feel++\cite{christophe_prudhomme_feelppfeelpp_2024}
 
 \section{Ktirio Urban Building Workflow}
@@ -155,8 +161,6 @@ \subsection{Final Analysis and Urban Scale Energy Evaluation}
 \end{figure}
 
 
-The described streamlined workflow is critical for accurately simulating and understanding urban sustainability challenges, supporting our application's broader objectives of improving urban living conditions and environmental impact.
-
 
 \section{Overview of Urban Building Modeling and Simulation}
 \label{sec:urban-building}
@@ -182,22 +186,22 @@ \subsection{Geometry Reconstruction of the KUB Urban Model}
 \begin{figure}[ht]%{R}{1\textwidth} % 'R' or 'L' for right or left, and width
 \centering
 \subfloat[LOD-0: a building is represented by its bounding box]{%
-  \includegraphics[width=0.48\linewidth]{img-compressed-buildings-lod0.png}
+  \includegraphics[width=0.45\linewidth]{img-compressed-buildings-lod0.png}
   \label{fig:building-lod0}
-}
+}\hspace{0.05\linewidth}
 %\hfill % This ensures that the images are placed side by side
 \subfloat[LOD-1: a building is represented by its ground footprint elevated to its height]{%
-  \includegraphics[width=0.48\linewidth]{img-compressed-buildings-lod1.png}
+  \includegraphics[width=0.45\linewidth]{img-compressed-buildings-lod1.png}
   \label{fig:building-lod1}
 }\\ % Ends the line for the first row of figures
 
 \subfloat[LOD-2: a building in full detail using BIM. Note that LOD-2 and LOD-1 are mixed.]{%
-  \includegraphics[width=0.48\linewidth]{img-compressed-buildings-lod2.png}
+  \includegraphics[width=0.45\linewidth]{img-compressed-buildings-lod2.png}
   \label{fig:building-lod2}
-}
+}\hspace{0.05\linewidth}
 %\hfill
 \subfloat[LOD-2: A zoom on the LOD-2 building.]{%
-  \includegraphics[width=0.48\linewidth]{img-compressed-buildings-lod2-zoom.png}
+  \includegraphics[width=0.45\linewidth]{img-compressed-buildings-lod2-zoom.png}
   \label{fig:building-lod2-zoom}
 }
 
@@ -288,21 +292,21 @@ \subsubsection{Partitioning Strategies Depending on Simulation Use Cases}
 Data partitioning is a crucial stage in the deployment of the supercomputer simulator. We need to partition the geometry of the city in such a way as to distribute the simulation computation correctly.
 Different partitioning strategies are considered depending on the specific requirements of the simulation use cases :
 \begin{inparadesc}
-\item[Case 0] simulate simple scenarios where buildings do not interact with their environment, a basic listing and weighting strategy is sufficient;
-\item[Case 1] simulate buildings interact with environmental elements, a more complex partitioning strategy is necessary that considers both the buildings and their immediate non-building surroundings; the build meshes and environment meshes (terrain, vegetation) are integrated conformably; each component can be partitioned separately;
-\item[Case 2] simulate full interaction models where the partitioning strategy starts with the buildings and extends to the entire urban mesh, ensuring all elements are considered to minimize communication overhead;
-\item[Case 3] simulate scenarios with extreme partitioning and employ a multi-grid approach to  define coarse and fine meshes and manage computational resources efficiently; this multi-fidelity approach enhances the accuracy and applicability of urban models in simulations and ensures scalability across different computational platforms, making it a cornerstone of urban analysis.
+\item[Case 0] simulates simple scenarios where buildings do not interact with their environment, a basic listing and weighting strategy is sufficient;
+\item[Case 1] simulates buildings interact with environmental elements, a more complex partitioning strategy is necessary that considers both the buildings and their immediate non-building surroundings; the build meshes and environment meshes (terrain, vegetation) are integrated conformably; each component can be partitioned separately;
+\item[Case 2] simulates full interaction models where the partitioning strategy starts with the buildings and extends to the entire urban mesh, ensuring all elements are considered to minimize communication overhead;
+\item[Case 3] simulates scenarios with extreme partitioning and employ a multi-grid approach to  define coarse and fine meshes and manage computational resources efficiently; this multi-fidelity approach enhances the accuracy and applicability of urban models in simulations and ensures scalability across different computational platforms, making it a cornerstone of urban analysis.
 \end{inparadesc}
 
 The current version of our simulation framework has implemented case 0 and case 1. Cases 2 and 3 require a full conformal mesh, which is not yet available. Moreover, these partitioning strategies are costly, so we have planned to use other third-party tools, such as Zoltan2~\cite{the_zoltan2_team_zoltan2_nodate}, to improve efficiency.
-\begin{wrapfigure}{R}{0.6\textwidth}
+\begin{wrapfigure}{R}{0.7\textwidth}
 \centering
 \subfloat[Partitioning Case 0\label{fig:city-strasbourg-lod0-parts}]{%
-  \includegraphics[width=0.48\linewidth]{img-compressed-city-strasbourg-lod0-parts.png}
-}
+  \includegraphics[width=0.46\linewidth]{img-compressed-city-strasbourg-lod0-parts.png}
+}\hspace{0.02\linewidth}
 %\hfill % Ensures that the images are placed side by side
 \subfloat[Partitioning Case 1\label{fig:city-strasbourg-lod1-parts}]{%
-  \includegraphics[width=0.48\linewidth]{img-compressed-city-strasbourg-lod1-parts.png}
+  \includegraphics[width=0.46\linewidth]{img-compressed-city-strasbourg-lod1-parts.png}
 }
 %\subfloat[Partitioning Case 3: Large scale\label{fig:city-ny-largescale}]{%
 %  \includegraphics[width=0.4\textwidth]{img-compressed-city-newyork-largescale.png}
@@ -503,7 +507,15 @@ \subsubsection{Computing Shading Masks and View Factors with Feel++}
 \label{fig:solar-masks-vf}
 \end{figure}
 
-In \cref{fig:solar-masks-vf}, we illustrate \textit{(i)} the solar masks of building face oriented eastwise for a discretization of the sun position in \cref{fig:sm-building-east}, \textit{(ii)} the solar masks for an entire building for discretization of the sun position in \cref{fig:sm-whole-building}, \textit{(iii)} a visualization of the solar mask early morning of the city center of Strasbourg in \cref{fig:sm-strasbourg} and \textit{(iv)} an image of a standard benchmark~\cite{van_eck_surface_2016} for the computation of view-factors and solving the heat transfer problems between three building blocks in 2D subsequently in \cref{fig:view-factor}.
+
+In \cref{fig:solar-masks-vf}, we illustrate several aspects of solar masks: \textit{(i)} the solar masks of a building face oriented eastward, showing a discretization of the sun's position in \cref{fig:sm-building-east}; \textit{(ii)} the solar masks for an entire building, representing a detailed discretization of the sun's position throughout the day in \cref{fig:sm-whole-building}; \textit{(iii)} a visualization of the solar mask in the early morning for the city center of Strasbourg in \cref{fig:sm-strasbourg}; and \textit{(iv)} an image of a standard benchmark~\cite{van_eck_surface_2016} used for the computation of view factors and solving heat transfer problems between three building blocks in 2D in \cref{fig:view-factor}.
+
+The grayscale values in each solar mask represent varying levels of solar obstruction for different azimuth and altitude angles.
+In these masks, the scale ranges from 0 to 1, where 0 (white) indicates no obstruction and full exposure to sunlight, and 1 (black) represents full obstruction, meaning no sunlight reaches that part of the building face for the given sun position. 
+Intermediate shades of gray indicate partial obstruction, where a portion of sunlight is blocked by surrounding buildings or other urban features.
+The level of detail in the solar mask depends on the chosen level of discretization (LOD), as illustrated in \cref{fig:sm-building-east} and \cref{fig:sm-whole-building}.
+
+%In \cref{fig:sm-building-east} (LOD-0), we see a coarser discretization with larger, generalized angular segments, leading to more abrupt transitions between obstructed and non-obstructed areas. In contrast, \cref{fig:sm-whole-building} (LOD-1) provides a finer level of detail, capturing more granular variations in solar accessibility across different sun angles. These masks help quantify the solar exposure of building surfaces over time, which is critical for accurately modeling heat gains and understanding the thermal dynamics of urban environments.
 
 \subsubsection{Heat Transfer Modeling with Feel++ and Modelica}
 In the Urban Building Energy Model (UBEM), heat transfer analysis is crucial for predicting energy consumption, internal air temperature variations, and overall building energy performance. This analysis is facilitated by a combination of Feel++ and Modelica, enabling detailed simulations of heat dynamics within urban buildings.
@@ -665,18 +677,27 @@ \subsection{Benchmarking KUB}
 \cref{fig:execution-breakdown} reports the portion of the total execution taken by:
 \begin{description}
 \item[Pre-processing (Pre-proc)] The time elapsed in initialization before entering the time loop of the simulation
-\item[Simulation (Simulation)] The cumulative time spent calculating the new solution at each time step
+\item[Simulation (Simulation)] The cumulative time spent calculating the new solution at each time step of the simulation described in \cref{sec:heat-transfer-in-buildings}.
 \item[Post-processing (Post-proc)] The cumulative time spent exporting results, i.e., generating files containing the output of the UB model.
 \end{description}
 
 In~\cref{fig:combined-metrics}, the total elapsed time varies from 100 seconds to 1000 seconds in function of the number of nodes.
 All machines tested have an elapsed time of the same magnitude order.
 
-Pre-processing does not scale. However, it occupies only a small part of the total execution, and thus it is not performance-critical. On the other hand, as more nodes are employed and the time spent in the actual simulation is decreased, the post-processing stage dominates the execution. It becomes the main bottleneck, causing the previously observed performance degradation.
-This behavior is caused by the multiple files being written in parallel on the shared file system. More specifically, most of the writing time is spent in opening and closing files in parallel. We are investigating potential solutions, such as asynchronous writes, data caching, etc. Finally, as the project progresses, we expect the urban building models used in the simulation to become more complex, leading to an increase in the time occupied by the simulation part and, hence, to a reduction of the impact of post-processing on the total execution time of the pilot.
+
+Pre-processing does not scale efficiently, but it occupies only a small portion of the total execution time and is therefore not performance-critical.
+As more nodes are employed and the actual simulation time decreases, the post-processing stage becomes the dominant component of execution, posing a potential bottleneck and causing the previously observed performance degradation.
+This behavior is primarily due to multiple files being written in parallel to the shared file system, with a significant portion of time spent opening and closing files simultaneously.
+
+However, our end-user approach focuses not on the complete solution fields, unlike this benchmark, but rather on outputs of interest and automatically generated analysis reports, which occupy only a small fraction of the typical post-processing time.
+By targeting specific metrics and reports, we minimize the post-processing overhead and reduce data transfer and storage demands (see \cref{sec:final-analysis} for more information).
+
+We are nevertheless actively exploring potential solutions to further streamline the processing of solution fields, such as asynchronous writes, data caching, and other strategies to mitigate the impact of file operations. 
+As the project progresses, we anticipate that the urban building models used in the simulation will grow in complexity, leading to increased time spent in the simulation itself (\cref{sec:heat-transfer-in-buildings}) and subsequently reducing the relative impact of post-processing on the overall execution time of the pilot.
 
 
 \section{Conclusion}
+\label{sec:conclusion}
 
 We are developing the computational Ktirio Urban Building(KUB) framework: assembling this very compelling application encompasses challenges in mathematics --- scalable modeling and simulation, large-scale watertight robust mesh generation, advanced analysis including data simulation --- and computer science --- scalable framework, software architecture, modern development, testing and packaging environment including standard DevOps and now HPCOps.---
 The overall programming, integration, delivery, and deployment environment is critical to develop such an application. To our knowledge, this is the first application that can be automatically benchmarked and executed on EuroHPC JU systems thanks to CI/CD or HPCOps. Moreover, the workflow from Feel++ to KUB provides a considerable gain in terms of development and testing time, automated as much as possible, and enabling researchers and developers to focus their work better.