Merge pull request #241 from Arquisoft/monitoring

Adding load testing results and monitoring to project and documentation
Arquisoft · Apr 29, 2024 · 8b7f415 · 8b7f415
2 parents a5679e2 + 48f7d4b
commit 8b7f415
Show file tree

Hide file tree

Showing 338 changed files with 153,882 additions and 6 deletions.
diff --git a/docs/images/10users-secA.png b/docs/images/10users-secA.png
diff --git a/docs/images/10users-secB.png b/docs/images/10users-secB.png
diff --git a/docs/images/2-10users-secA.png b/docs/images/2-10users-secA.png
diff --git a/docs/images/2-10users-secB.png b/docs/images/2-10users-secB.png
diff --git a/docs/images/2-25users-secA.png b/docs/images/2-25users-secA.png
diff --git a/docs/images/2-25users-secB.png b/docs/images/2-25users-secB.png
diff --git a/docs/images/2-2users-secA.png b/docs/images/2-2users-secA.png
diff --git a/docs/images/2-2users-secB.png b/docs/images/2-2users-secB.png
diff --git a/docs/images/2-5users-secA.png b/docs/images/2-5users-secA.png
diff --git a/docs/images/2-5users-secB.png b/docs/images/2-5users-secB.png
diff --git a/docs/images/2users-secA.png b/docs/images/2users-secA.png
diff --git a/docs/images/2users-secB.png b/docs/images/2users-secB.png
diff --git a/docs/images/5users-secA.png b/docs/images/5users-secA.png
diff --git a/docs/images/5users-secB.png b/docs/images/5users-secB.png
diff --git a/docs/src/08_concepts.adoc b/docs/src/08_concepts.adoc
@@ -110,18 +110,71 @@ The chosen color palette strikes a balance between functionality, aesthetics, an
 
 === Development concepts
 
-==== Testing
-We test the application with these different tests:
+==== Testing and Monitoring
  - Unit Testing: We employed the SonarCloud tool to monitor the code covered by these tests, while Continuous Integration practices were implemented using GitHub Actions.
- - Load Testing: We used Gattling for measuring the performance of the application when many users are using it.
+ - Load Testing and Monitoring: We used Gattling for recording user simulations that consist on login into the application, playing the existing game modes and checking the statistics and leaderboards. Then, Gatling allowed us to also measure the performance of the application and the average response times when creating 2, 5, 10 and 25 users per second during 60 seconds performing the previously mentioned simulations, using 2 different Azure machines. 
+ Particularly, load testing has been performed using a 1-cpu machine with almost 1GB of RAM memory and also, using a 2-cpu machine with 8GB of RAM memory.
+
+  ** 1-cpu machine 
+
+  With this machine all requests were answered quickly when 2 users per second where created (120 users in total) with almost all requests responding in less than 1 second with a minimum response time of just 24 milliseconds and a maximum reponse time of 1.6 seconds.
+
+image::2users-secA.png["2 users per second in 60 seconds user-simulation overall results with 1-cpu machine"]
+image::2users-secB.png["2 users per second in 60 seconds user-simulation specific graph results with 1-cpu machine"]
+
+    When creating 5 users per second (350 users in total), the responses took almost all of them (around 80%) less than a second with a minimum response time of 24 milliseconds, but with a maximum response time of 10 seconds which is a huge amount of time for a web application.
+
+image::5users-secA.png["5 users per second in 60 seconds user-simulation overall results with 1-cpu machine"]
+image::5users-secB.png["5 users per second in 60 seconds user-simulation specific graph results with 1-cpu machine"]
+
+    Unfortunately, when creating a load of 10 user per second (600 users in total) 41% of the responses were failling and around 70% of the responses where failing or taking more than a second to be answered. For sure, when creating even more load, almost all response were going to fail.
+
+image::10users-secA.png["10 users per second in 60 seconds user-simulation overall results with 1-cpu machine"]
+image::10users-secB.png["10 users per second in 60 seconds user-simulation specific graph results with 1-cpu machine"]
+
+    With this 1-cpu and 1GB of RAM azure machine we could afford around 200 hundred users making constant resquests without having a denial of service and providing reasonable requests' response times.
+
+ ** 2-cpu machine 
+
+    With this machine all requests were answered quickly when 2 users per second where created (120 users in total) with almost all requests responding in less than 1 second with a minimum response time of just 24 milliseconds and a maximum reponse time of 1.8 seconds.
+
+image::2-2users-secA.png["2 users per second in 60 seconds user-simulation overall results"]
+image::2-2users-secB.png["2 users per second in 60 seconds user-simulation specific graph results"]
+
+    When creating 5 users per second (350 users in total), the responses took almost all of them (around 80%) less than a second with a minimum response time of 24 milliseconds, but with a maximum response time of 10 seconds which is exactly the same time we obtained with the other machine and a similar mean response time.
+
+image::2-5users-secA.png["5 users per second in 60 seconds user-simulation overall results with 2-cpu machine"]
+image::2-5users-secB.png["5 users per second in 60 seconds user-simulation specific graph results with 2-cpu machine"]
+
+    When creating a load of 10 user per second (600 users in total), no response failed although the maximum response time was 58 seconds with a mean response time of 2 seconds, which meant that almost every respone took less than a second.
+
+image::2-10users-secA.png["10 users per second in 60 seconds user-simulation overall results with 2-cpu machine"]
+image::2-10users-secB.png["10 users per second in 60 seconds user-simulation specific graph results with 2-cpu machine"]
+
+    Finally, when creating a load of 25 user per second (1500 users in total), just a 7% of the total amount of response failed and the maximum response time was 60 seconds with a mean response time of 5 seconds.
+
+image::2-10users-secA.png["25 users per second in 60 seconds user-simulation overall results with 2-cpu machine"]
+image::2-10users-secB.png["25 users per second in 60 seconds user-simulation specific graph results with 2-cpu machine"]
+
+    Using this 2-cpu and 8GB machine, response times are not enhaced when using more powerfull hardware and most of the work should be done programatically by improving our software. 
+    But better hardware allows us to support much more load on the application, which means more users playing at the same time. This time, we could support more 400, but less than 800 simultaneous users.
+    Taking a look at the specific graph results of each of thesimulations performed, most of load is always provoked at the beggining of the simulation when the users have to login. 
+    Then, when users are playing some games the amount of response is reduced a lot since all the information for playing the game is asked at the beggining.
+
+    As a general conclusion, following the azure payment plan for virtual machines: With low load requirements and just paying around 35$ each month we could afford a 1-cpu and 1GB RAM memory web server supporting around 200 simultaneous users using the application at the same time.
+    On the contrary, if higher load requirements are needed and paying around 100$ for a 2-cpu and 8GB of RAM memory server, the amount of supported users is more than duplicated supporting more than 400 simultaneous users. 
+
+
  - E2e Testing: We used behavior-driven development scenarios written in the Gherking language as a basis for our end-to-end tests.
 
 ==== Deployment
 The application is deployed using Docker.
 
 ==== Configurability
-The application will not have configurable features. An early idea was to include a "dark mode".
-Through the development we decided to postpone these ideas in order to focus on a better application in general.
+The application has simple configurable game features for selecting between two game modes (normal/usual and trivia game mode) and two difficulty levels (easy and hard difficulties).
+ - Normal mode game consists of 10 random questions with an amount of time to answer the question before losing the possibility to answer. Easy and hard modes differ on the amount of time that the user has to answer the question. 
+ - Trivia mode game consists of 10 questions, which are generated based on the resulting category of rolling a dice. There are 6 possible categories: sports, science, history, geography and entertainment.
+Additionally, there is an option at the main application view where random music can be played.
 
 ==== Data access
 The development team has followed two different approaches for supporting data access from the running application for development and production.