Skip to content

Commit

Permalink
articles written
Browse files Browse the repository at this point in the history
  • Loading branch information
amyasnikov committed Jan 24, 2025
1 parent 03ede84 commit 46cb142
Show file tree
Hide file tree
Showing 4 changed files with 76 additions and 9 deletions.
7 changes: 7 additions & 0 deletions docs/entities/pollers.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,10 @@ There are 3 ways to bind a Poller to Device:
* Set the Poller at **Device Type** level. Go to Device Type page at set the Poller via custom fields. This action applies this Poller to all the devices with this Device Type and overwrites the value from Manufacturer.

* Set the Poller at the individual **Device** level. Go to Device page at set the Poller via custom fields. This action applies this Poller to one specific Device only and overwrites the values from Device Type and Manufacturer.


# Custom Pollers

You can easily write your own polling backend, connect it to Validity and then use for polling.

Read more about this feature in the [Custom Pollers](../features/custom_pollers.md) article.
17 changes: 17 additions & 0 deletions docs/features/custom_pollers.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,23 @@ PLUGIN_SETTINGS = {
}
```

??? info
The same setting may be defined via dict as well
```python
# configuration.py

from my_awesome_poller import ScrapliPoller

PLUGIN_SETTINGS = {
'validity': {
'custom_pollers' : [
'class': ScrapliPoller, 'name':'scrapli', 'color':'pink', 'command_types'=['CLI']
]
}
}
```


PollerInfo parameters:

* **klass** - class inherited from `CustomPoller`
Expand Down
60 changes: 51 additions & 9 deletions docs/installation/plugin_settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,25 +37,34 @@ If the system creates a new Report and the overall reports count exceeds *store_
Validity uses threads to perform device polling. This setting defines the upper limit of these threads number. If you have extremely large amount of devices (e.g. 100 000), you may want to increase this number to speed up the overall polling process.


### runtests_queue
### custom_queues

*Default:* `"default"`
*Type:* `dict[str, str]`

*Type:* `str`

RQ queue name for running the tests. May be useful if you want to move tests execution into a separate queue (e.g. handled by separate set of workers)
This settings defines custom RQ queue names for Validity scripts. It may be useful if you want to move script execution into a separate queue (e.g. handled by separate set of workers)

!!! note
Default RQ worker serves 3 queues only: `high`, `default` and `low`. You have to reconfigure your RQ worker (or just start another one in parallel) in case of choosing a different queue name.

For instance, if your runtests_queue is set to `my_queue`, then RQ worker has to be started with this queue name as a parameter:
For instance, if you have two custom queues `my_q1` and `my_q2` and want a separate worker for them, then the worker has to be started with these queue names as ths parameters:

`./manage.py rqworker my_q1 my_q2`


`./manage.py rqworker my_queue`
Here are the custom queues which may be defined via this setting:

| Queue Name | Description | Default value |
|---|---|---|
| runtests | Queue for Run Tests script | default |
| backup | Queue for backing up individual Backup Points (Back Up button) | default |

!!! warning
The `runtests_queue` setting is deprecated **since version 3.1**. Use `custom_queues.runtests` instead


### script_timeouts

*Type:* `dict`
*Type:* `dict[str, str | int]`

This setting defines the timeouts for RQ jobs started by Validity (e.g. for running the tests). Timeout defines the maximum amount of time some job can run. If a job exceeds the timeout, it gets terminated.

Expand All @@ -66,11 +75,44 @@ As was mentioned in the [corresponding article](../entities/scripts.md#stages),
| runtests_split | 10m |
| runtests_apply | 30m |
| runtests_combine | 10m |
| backup | 10m |


### custom_pollers

*Type:* `list[validity.settings.PollerInfo]`

*Default:* `[]`

This setting allows to connect custom user-defined [Pollers](../entities/pollers.md) to Validity.

Read more about this feature in the [Custom Pollers](../features/custom_pollers.md) article.


### integrations

*Type:* `dict[str, dict]`

*Default:*

```python
{
'git': {'author': 'netbox-validity', 'email': 'validity@netbox.local'}
'S3': {'threads': 10}
}
```

This setting is responsible for various integrations with third-party services, mostly for [Config Backup](../features/config_backup.md) feature.

`threads` param is responsible for the level of parallelism (number of threads used) for multiple files uploading to S3.



## Settings Example

!!! note
The following example does NOT represent the recommended settings.

Here is the full example of Validity settings:

```python
Expand All @@ -81,7 +123,7 @@ PLUGINS_CONFIG = {
'result_batch_size': 300,
'store_reports': 7,
'polling_threads': 100,
'runtests_queue': 'validity_tests',
'custom_queues': {"runtests": "runtests_q"}
'script_timeouts': {
'runtests_split': '15m',
'runtests_apply': '1h',
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ markdown_extensions:
line_spans: __span
pygments_lang_class: true
- admonition
- pymdownx.details
- pymdownx.inlinehilite
- pymdownx.snippets
- pymdownx.superfences
Expand Down

0 comments on commit 46cb142

Please sign in to comment.