-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathconclusions.tex
19 lines (16 loc) · 4.14 KB
/
conclusions.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
\chapter{Conclusions and Future Work}
\section{Conclusions}
\label{subsec:conclusions}
Our contributions have improved both the usability and the performance of the COMPSs programming model. Our features have been succesfully tested and used with real use-cases written by real users, not only with our own test, synthetic programs.\\
\\
We have removed two big flaws in the COMPSs programming model: the impossibility to run tasks with many parameters due to the huge overhead from the metadata of parameters (and thus affecting map-reduce workflows, which happen to be very common) and the performance issues when running COMPSs applications in non-shared filesystems such as GPFS by offering the user an alternative based on storage implementations.\\
\\
\section{Future Work}
\label{subsec:future_work}
Let's consider the application from section \ref{subsubsec:matmul_redis}. In the product of two $2 \times 2$ matrices the dependency $mul(A_{1, 1}, B_{1, 1}, C_{1, 1}) \implies mul(A_{1,2}, B_{2, 1}, C_{1, 1})$ appears, when it is actually enough to ensure that no two tasks involving the same $C_{i, j}$ are executed concurrently. An open research line consists of developing a distributed mechanism to allow task commutativity. It was discussed to implement the option to assign each task a commutativity group, meaning that two tasks that belong to this group are mutually commutative. Task commutativity is already implemented in the OMPSs programming model (a COMPSs-like software but optimized for making applications run in a single node) \cite{duran2011ompss}, but it still remains as a challenge to implement an equivalent feature for a distributed programming model.\\
\\
As we mentioned in section \ref{sec:storage}, Redis allows to have replicas of some instances. Replicas introduce a dangerous tradeoff between time and space. A naive usage of replicas quickly leads to program crashes derived from not having enough RAM, or even disk space. However, this does not mean that it is entirely useless. During the development of this project this line was quickly discarded, as it was considered to be too challenging and time-consuming. This idea cannot be properly tested locally, so a lot of time should be devoted to development and debugging, and even more supercomputer time should be requested. The replica feature was left unexplored and marked as a possible future work line.\\
\\
Among all the proposed implementations regarding to collections (see section \ref{sec:tasks_and_time_planning}), the most important one that was left undone is the collection transfer optimization. This was left as future work for two reasons: it seems hard and it is a vague idea. The general notion is that the COMPSs Runtime can exploit the fact that a group of parameters go together in a collection to optimize transfers. For example, they could be packed, sent in a single connection and unpacked in their destination. However, it was not possible to reach a concrete idea or implementation, leaving this as a possible future work line. Another possible work line regarding collections is dynamic size. Dynamic, and possibly unknown, sizings bring many new challenges. For example, the future or promise representation of the contents of a dynamically sized collection is not as trivial as fixed size arrays, as we cannot know in advance how many future objects we must generate.\\
\\
Finally, the last future work line to mention is I/O and Threading. The main idea was to parallelize I/O operations when serializing and deserializing objects. The main problem with this idea is that I/O parallelization created a serious problem with memory usage. Serializing and deserializing an object can take twice the size of the object of memory. This issue does not affect most regular COMPSs applications if done sequentally, but causes a lot of crashes when done in parallel, as twice the size of \textit{all} objects is significantly more than twice the size of \textit{some} object. However, this does not mean that this idea should be totally discarded. Some kind of \textit{smart} system that parallelizes I/O while avoiding these issues with memory can be designed, and we leave this as a another possible future work line.