1.1.0 - 2024-12-10 ADD Total Number of Queries: track the total number of SQL queries executed during a single HTTP request, Artisan command, or CLI execution.

Author: lopadova

.env.example

@@ -9,3 +9,8 @@ QUERYMONITOR_BUILDER_MAX_EXECUTION_TIME=200
 QUERYMONITOR_BUILDER_METHOD_REGEX='^.*$'
 
 QUERYMONITOR_BUILDER_MAX_STACK_DEPTH=5
+
+QUERYMONITOR_TOTAL_QUERIES_ATTIVA=true
+QUERYMONITOR_MAX_TOTAL_QUERIES=500
+QUERYMONITOR_TOTAL_QUERIES_REGEX=
+

CHANGELOG.md

@@ -3,6 +3,10 @@
 All Notable changes to `laravel-querymonitor` will be documented in this file
 
 
+## 1.1.0 - 2024-12-10
+
+- ADD Total Number of Queries**: track the total number of SQL queries executed during a single HTTP request, Artisan command, or CLI execution.
+
 ## 1.0.0 - 2024-11-29
 
 - Initial release

README.md

@@ -2,17 +2,17 @@
 
 ![Laravel QueryMonitor](./resources/images/logo.webp)
 
-Laravel QueryMonitor is a package for Laravel that allows you to monitor and log:
-
 [![Latest Version on Packagist](https://img.shields.io/packagist/v/padosoft/laravel-querymonitor.svg?style=flat-square)](https://packagist.org/packages/padosoft/laravel-querymonitor)
 [![Software License](https://img.shields.io/badge/license-MIT-brightgreen.svg?style=flat-square)](LICENSE.md)
 [![CircleCI](https://circleci.com/gh/padosoft/laravel-querymonitor.svg?style=shield)](https://circleci.com/gh/padosoft/laravel-querymonitor)
 [![Quality Score](https://img.shields.io/scrutinizer/g/padosoft/laravel-querymonitor.svg?style=flat-square)](https://scrutinizer-ci.com/g/padosoft/laravel-querymonitor)
 [![Total Downloads](https://img.shields.io/packagist/dt/padosoft/laravel-querymonitor.svg?style=flat-square)](https://packagist.org/packages/padosoft/laravel-querymonitor)
 
-
+Laravel QueryMonitor is a package for Laravel that allows you to monitor and log:
+- 
 - **Slow SQL Queries**: Monitors the actual execution time of SQL queries on the database.
 - **Slow Eloquent Methods**: Monitors the total time taken by Eloquent methods, including PHP processing.
+- **Total Number of Queries**: track the total number of SQL queries executed during a single HTTP request, Artisan command, or CLI execution.
 
 ## Requirements
 - "php": ">=8.1",
@@ -51,13 +51,37 @@ return [
         'methodRegEx' => env('QUERYMONITOR_BUILDER_METHOD_REGEX', '^(get|first)$'),
     ],
 
+    'total_queries' => [
+    
+        /*
+         * Whether to enable total query monitoring.
+         */
+        'attiva' => env('QUERYMONITOR_TOTAL_QUERIES_ATTIVA', true),
+    
+        /*
+         * Maximum allowed total queries per request/command.
+         * If this threshold is exceeded, a warning is logged.
+         */
+        'maxTotalQueries' => env('QUERYMONITOR_MAX_TOTAL_QUERIES', 500),
+    
+        /*
+         * A regex to filter which contexts to monitor.
+         * - For HTTP requests, this regex will be matched against the full URL (including query string).
+         * - For Artisan commands, it will be matched against the command name.
+         * - For CLI contexts, it can be matched against the script name.
+         * If unset or empty, all contexts are monitored.
+         * Example: '^/api/.*$' to monitor only requests under /api/
+         */
+        'traceRegEx' => env('QUERYMONITOR_TOTAL_QUERIES_REGEX', null),
+    ],
 ];
 ```
 
 
 ## Usage
 
 Once installed and configured, the package will automatically monitor and log SQL queries and Eloquent methods based on the provided settings.
+If you setting query total count to true, the package will automatically monitor and log total queries count.
 
 ### Monitoring SQL Queries
 - **What it monitors**: The execution time of SQL queries on the database.
@@ -135,6 +159,128 @@ $users = User::all();
 - **Action**: Consider using pagination or limiting the selected fields.
 
 
+## Monitoring the Total Number of Queries
+In addition to monitoring slow queries and slow Eloquent methods, laravel-querymonitor allows you to track the total number of SQL queries executed during a single HTTP request, Artisan command, or CLI execution. 
+This helps you identify cases where, although each individual query may be performant, the total number of queries is excessively high, potentially causing performance bottlenecks.
+
+**Why Monitor Total Queries?**
+Even if every single query is fast, executing too many queries per request or command can cause unnecessary overhead. 
+By monitoring the total query count, you can quickly identify scenarios where your application issues an excessive number of queries (for example, 2,000 queries in a single request), 
+pinpointing areas that need optimization (e.g., using eager loading, caching, or refining data retrieval logic).
+
+**How It Works**
+- For HTTP requests, a middleware hooks into the Laravel request lifecycle. It resets a query counter at the start of the request and increments it every time a query is executed. 
+At the end of the request, if the total number of queries exceeds the configured threshold, a warning is logged.
+- For Artisan commands, the package listens to the CommandStarting and CommandFinished events. 
+It resets the counter before the command runs and checks the final count after the command completes.
+- For CLI contexts (other non-command CLI scripts), you can manually integrate by resetting and checking counts in your own script logic.
+
+### Configuration
+All configuration options are defined in the querymonitor.php config file under the total_queries key:
+```php
+'total_queries' => [
+
+    /*
+     * Whether to enable total query monitoring.
+     */
+    'attiva' => env('QUERYMONITOR_TOTAL_QUERIES_ATTIVA', true),
+
+    /*
+     * Maximum allowed total queries per request/command.
+     * If this threshold is exceeded, a warning is logged.
+     */
+    'maxTotalQueries' => env('QUERYMONITOR_MAX_TOTAL_QUERIES', 500),
+
+    /*
+     * A regex to filter which contexts to monitor.
+     * - For HTTP requests, this regex will be matched against the full URL (including query string).
+     * - For Artisan commands, it will be matched against the command name.
+     * - For CLI contexts, it can be matched against the script name.
+     * If unset or empty, all contexts are monitored.
+     * Example: '^/api/.*$' to monitor only requests under /api/
+     */
+    'traceRegEx' => env('QUERYMONITOR_TOTAL_QUERIES_REGEX', null),
+],
+```
+
+### Enabling Total Query Monitoring for HTTP Requests
+To track the total number of queries for HTTP requests, the package provides a `TrackTotalQueriesMiddleware`. 
+This middleware must be added as the first middleware in the global middleware stack. 
+
+By doing so, it can:
+
+- **Set up tracking before any other middleware or controllers run**, ensuring that all queries executed during the request lifecycle are counted.
+- **Perform a final check after the response is generated**, running last in the outbound cycle, so that it includes queries made by all subsequent middleware, controllers, and operations performed downstream.
+
+**How to add it:**
+
+In your `app/Http/Kernel.php`, ensure that `\Padosoft\QueryMonitor\Middleware\TrackTotalQueriesMiddleware::class` appears at the top of the `$middleware` array:
+
+```php
+protected $middleware = [
+                        \Padosoft\QueryMonitor\Middleware\TrackTotalQueriesMiddleware::class,
+                        // ... other global middleware ...
+                        ];
+```
+
+By placing the TrackTotalQueriesMiddleware first, you guarantee comprehensive coverage of every query executed during the request lifecycle. 
+Once the request is fully processed, the middleware checks the total query count and logs a warning if it exceeds the configured threshold.
+
+
+### Examples of Logs
+- **1. HTTP Request Exceeding the Query Limit**
+
+If a request executes more than the allowed number of queries, a log entry is created after the response is generated:
+
+```text
+[2024-12-09 10:23:45] local.WARNING: Exceeded maximum total queries: 2020 queries (max: 500). {"context":"request","url":"https://example.com/products?category=shoes","method":"GET"}
+```
+- context: request
+- url: The full URL of the request, including query parameters.
+- method: The HTTP method (e.g., GET, POST).
+
+- **2. Artisan Command Exceeding the Query Limit**
+
+If an Artisan command triggers more queries than allowed, a warning is logged once the command finishes:
+
+```text
+[2024-12-09 10:25:10] local.WARNING: Exceeded maximum total queries: 1200 queries (max: 500). {"context":"command","command":"cache:clear","arguments":["artisan","cache:clear"]}
+```
+- context: command
+- command: The Artisan command name.
+- arguments: The arguments passed to the command.
+
+- **3. CLI Context Exceeding the Query Limit**
+
+For CLI contexts (non-Artisan commands), if you set up tracking manually and the query count is exceeded, you'll see a log like:
+
+```text
+[2024-12-09 10:26:00] local.WARNING: Exceeded maximum total queries: 3000 queries (max: 500). {"context":"cli-service","script":"myscript.php","arguments":["--option","value"]}
+```
+- context: cli-service
+- script: The name of the CLI script being executed.
+- arguments: The arguments passed to the script.
+
+**Using the Regex Filter**
+If you provide a traceRegEx:
+
+- For HTTP requests, the package only monitors requests whose URLs match the regex.
+- For Artisan commands, only the commands that match the regex pattern are monitored.
+- For CLI contexts, only scripts whose name matches the regex are tracked.
+
+For example, if you set:
+
+```php
+'traceRegEx' => '^/api/.*$',
+```
+only requests matching /api/... URLs are monitored.
+
+**Summary**
+By configuring and enabling total query monitoring, you gain deeper insights into your application's performance, identifying excessive query usage patterns that can be addressed to improve overall efficiency. 
+This is especially useful in complex, large-scale projects where minor optimizations in query counts can lead to significant performance gains.
+
+
+
 ## Final Notes
 - **Performance Optimization**: Remember that enabling monitoring can impact performance. It's advisable to use it in development environments or carefully monitor the impact in production.
 - **Dynamic Configuration**: You can modify the settings in real-time using environment variables or by updating the configuration file.

config/querymonitor.php

@@ -78,6 +78,39 @@
         'methodRegEx' => env('QUERYMONITOR_BUILDER_METHOD_REGEX', '^.*$'),
     ],
 
+    /*
+   |--------------------------------------------------------------------------
+   | Total Number of Queries
+   |--------------------------------------------------------------------------
+   |
+   | These settings track the total number of SQL queries executed during
+   | a single HTTP request, Artisan command, or CLI execution..
+   |
+   */
+    'total_queries' => [
+
+        /*
+         * Whether to enable total query monitoring.
+         */
+        'attiva' => env('QUERYMONITOR_TOTAL_QUERIES_ATTIVA', false),
+
+        /*
+         * Maximum allowed total queries per request/command.
+         * If this threshold is exceeded, a warning is logged.
+         */
+        'maxTotalQueries' => env('QUERYMONITOR_MAX_TOTAL_QUERIES', 500),
+
+        /*
+         * A regex to filter which contexts to monitor.
+         * - For HTTP requests, this regex will be matched against the full URL (including query string).
+         * - For Artisan commands, it will be matched against the command name.
+         * - For CLI contexts, it can be matched against the script name.
+         * If unset or empty, all contexts are monitored.
+         * Example: '^/api/.*$' to monitor only requests under /api/
+         */
+        'traceRegEx' => env('QUERYMONITOR_TOTAL_QUERIES_REGEX', null),
+    ],
+
     /*
     |--------------------------------------------------------------------------
     | Miscellaneous Settings

src/Middleware/TrackTotalQueriesMiddleware.php

@@ -0,0 +1,77 @@
+<?php
+
+namespace Padosoft\QueryMonitor\Middleware;
+
+use Closure;
+use Padosoft\QueryMonitor\QueryCounter;
+use Illuminate\Support\Facades\Config;
+use Illuminate\Support\Facades\Log;
+
+class TrackTotalQueriesMiddleware
+{
+    protected function logExcessiveQueries(int $count, int $max, array $context): void
+    {
+        $message = "Exceeded maximum total queries: {$count} queries (max: {$max}).";
+        $extra = [
+            'context' => 'request',
+            'url' => $context['url'] ?? 'unknown',
+            'method' => $context['method'] ?? 'unknown',
+        ];
+
+        Log::warning($message, $extra);
+    }
+
+    public function handle($request, Closure $next)
+    {
+        $totalQueriesConfig = Config::get('querymonitor.total_queries', []);
+
+        if (empty($totalQueriesConfig['attiva']) || $totalQueriesConfig['attiva'] === false) {
+            return $next($request);
+        }
+
+        $traceRegEx = $totalQueriesConfig['traceRegEx'] ?? null;
+
+        $url = $request->fullUrl();
+
+        // Se c'è una regex e la url non match, non tracciamo questa request
+        if ($traceRegEx && !preg_match("/{$traceRegEx}/", $url)) {
+            // Non attiviamo il tracking, quindi niente reset
+            return $next($request);
+        }
+
+        QueryCounter::reset([
+            'type' => 'request',
+            'url' => $url,
+            'method' => $request->method(),
+        ]);
+
+        return $next($request);
+    }
+
+    public function terminate($request, $response)
+    {
+        $totalQueriesConfig = Config::get('querymonitor.total_queries', []);
+
+        if (empty($totalQueriesConfig['attiva']) || $totalQueriesConfig['attiva'] === false) {
+            // Il tracking non è attivo, non facciamo nulla
+            return;
+        }
+
+        // Controlla se il tracking è stato attivato controllando se getCount > 0 o se QueryCounter::getContextInfo() non è vuoto
+        $context = QueryCounter::getContextInfo();
+        if (empty($context)) {
+            // Non è stato attivato nessun tracking per questa request
+            return;
+        }
+
+        $count = QueryCounter::getCount();
+        $max = $totalQueriesConfig['maxTotalQueries'] ?? 500;
+
+        if ($count <= $max) {
+            // Non superiamo il limite, non facciamo nulla
+            return;
+        }
+
+        $this->logExcessiveQueries($count, $max, $context);
+    }
+}

src/QueryCounter.php

@@ -0,0 +1,42 @@
+<?php
+
+namespace Padosoft\QueryMonitor;
+
+class QueryCounter
+{
+    protected static int $queryCount = 0;
+    protected static array $contextInfo = [];
+
+    /**
+     * Resetta il contatore query e memorizza il contesto (es: request info, command info)
+     */
+    public static function reset(array $contextInfo = []): void
+    {
+        self::$queryCount = 0;
+        self::$contextInfo = $contextInfo;
+    }
+
+    /**
+     * Incrementa il contatore delle query
+     */
+    public static function increment(): void
+    {
+        self::$queryCount++;
+    }
+
+    /**
+     * Restituisce il numero totale di query eseguite
+     */
+    public static function getCount(): int
+    {
+        return self::$queryCount;
+    }
+
+    /**
+     * Restituisce le info di contesto salvate (url, metodo, command, ecc.)
+     */
+    public static function getContextInfo(): array
+    {
+        return self::$contextInfo;
+    }
+}