Skip to content

Commit

Permalink
docs fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
amyasnikov committed Jan 24, 2025
1 parent 46cb142 commit 6ca1672
Show file tree
Hide file tree
Showing 8 changed files with 107 additions and 77 deletions.
1 change: 1 addition & 0 deletions docs/entities/backuppoints.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ If this param is `True`, backup will be performed each time respective Data Sour
Defines protocol for uploading the Data Source.

Available options are:

* Git (over http(s))
* Amazon S3 (or any other S3-compatible service)

Expand Down
2 changes: 1 addition & 1 deletion docs/entities/serializers.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ ipv6:
```
### XML
This method translates input XML-formatted text into Python dict using [xmltodict](https://github.com/martinblech/xmltodict) library. It is mainly used together with [Netconf commands](./commands.md#typenetconf).
This method translates input XML-formatted text into Python dict using [xmltodict](https://github.com/martinblech/xmltodict) library. It is mainly used together with [Netconf commands](./commands.md#type-netconf).
**Input data:**
```xml
Expand Down
19 changes: 10 additions & 9 deletions docs/features/config_backup.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ Validity supports device configurations backups to remote **Git** or **S3** serv

1. Follow the steps described in the [Quickstart: Polling](../quickstart_polling.md) article to set up all the entities required for device polling.

2. Polling is done when the Data Source is synced. In most cases it will be the default **Validity Polling** Data Source, but you are free to create your own one with the type *device_polling*.
2. Polling is done when the Data Source is synced. In most cases it will be the default **Validity Polling** Data Source, but you are free to create your own one with the type *device_polling*.

3. Create a [Backup Point](../entities/backuppoints.md) which describes a place where the Data Source with the configs have to be uploaded.
3. Create a [Backup Point](../entities/backuppoints.md) which describes a place where the Data Source with the configs has to be uploaded.

Example using pynetbox:

Expand All @@ -34,7 +34,7 @@ There are several options to trigger the backup process:

* Press "Back Up" button on the Backup Point page.

* Specify `Backup After Sync: True` in the Backup Point settings. This will trigger the backup process each time the respective data source is being synced (no matter via button, API or somehow else).
* Specify `Backup After Sync: True` in the Backup Point settings. This will trigger the backup process each time the respective data source sync occurs (no matter via GUI button, API or somehow else).

* Execute [RunTests](../entities/scripts.md#run-tests) script with `Sync Data Sources: True` option (`Backup After Sync: True` is required as well).

Expand All @@ -43,13 +43,13 @@ There are several options to trigger the backup process:

There are several options to provision periodic configuration backup process (e.g. daily, each 6 hours, etc):

1. Provision [RunTests](../entities/scripts.md#run-tests) script to execute on a regular basis using **Interval** script parameter. This will run everything at once:
* poll the devices (perform data source sync)
* execute compliance tests
* back up the data source
* Provision [RunTests](../entities/scripts.md#run-tests) script to start regularly using **Interval** script parameter. This will run everything at once:
1. poll the devices (perform data source sync)
2. execute compliance tests
3. back up the data source


2. If you don't want to run the tests via scheduler and want backup only, you can create tiny [custom script](https://netboxlabs.com/docs/netbox/en/stable/customization/custom-scripts/) inside your NetBox and configure it to run periodically.
* If you don't want to run the tests via scheduler and want backup only, you can create tiny NetBox [Custom Script](https://netboxlabs.com/docs/netbox/en/stable/customization/custom-scripts/) inside your NetBox and configure it to run periodically.

```python
from core.models import DataSource
Expand All @@ -65,4 +65,5 @@ class SyncDataSource(Script):

Execution of this script triggers the sync of the DataSource (and hence the backup process for bound Backup points with `Backup After Sync: True`).

3. Wait till periodic Data Source sync is implemented in the core NetBox. There is an [issue](https://github.com/netbox-community/netbox/issues/18287) for it, upvotes are appreciated.

* Wait till periodic Data Source sync is implemented in the core NetBox. There is an [issue](https://github.com/netbox-community/netbox/issues/18287) for it, upvotes are appreciated.
21 changes: 14 additions & 7 deletions docs/features/custom_pollers.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,16 @@ from validity.models import Command


class ScrapliPoller(CustomPoller):
# this class/function will be supplied with poller parameters
driver_factory = Scrapli
host_param_name = 'host' # Scrapli expects "host" param containing ip address of the device
driver_connect_method = 'open' # This driver method (if defined) will be called to open the connection.
driver_disconnect_method = 'close' # This driver method (if defined) will be called to gracefully close the connection.

def poll_one_command(self, driver, command) -> str:
# Scrapli class expects "host" param containing ip address of the device
host_param_name = 'host'
# This driver method (if defined) will be called to open the connection.
driver_connect_method = 'open'
# This driver method (if defined) will be called to gracefully close the connection.
driver_disconnect_method = 'close'

def poll_one_command(self, driver: Scrapli, command: Command) -> str:
"""
Arguments:
driver - object returned by calling driver_factory, usually represents connection to a particular device
Expand Down Expand Up @@ -74,7 +78,10 @@ PLUGIN_SETTINGS = {
PLUGIN_SETTINGS = {
'validity': {
'custom_pollers' : [
'class': ScrapliPoller, 'name':'scrapli', 'color':'pink', 'command_types'=['CLI']
'class': ScrapliPoller,
'name':'scrapli',
'color':'pink',
'command_types'=['CLI'],
]
}
}
Expand All @@ -83,7 +90,7 @@ PLUGIN_SETTINGS = {

PollerInfo parameters:

* **klass** - class inherited from `CustomPoller`
* **klass/class** - class inherited from `CustomPoller`
* **name** - system name of the poller, must contain lowercase letters only
* **verbose_name** - optional verbose name of the poller. Will be used in NetBox GUI
* **color** - badge color used for "Connection Type" field in the GUI
Expand Down
2 changes: 1 addition & 1 deletion docs/installation/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Validity is the [NetBox](https://netboxlabs.com/oss/netbox/) plugin. So, before

| **Python** | **NetBox** |
|------------|-----------------|
| >=3.10 | 3.7 | 4.0 | 4.1 |
| >=3.10 | 4.0 | 4.1 | 4.2 |

## Installation steps
Once you have installed NetBox, you should follow these steps
Expand Down
19 changes: 17 additions & 2 deletions docs/installation/netbox_compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ In the table below you can find appropriate Validity version according to your N
<table class="tg"><thead>
<tr>
<th class="tg-r82u" colspan="2" rowspan="2"></th>
<th class="tg-c6ba" colspan="6">NetBox<br></th>
<th class="tg-c6ba" colspan="7">NetBox<br></th>
</tr>
<tr>
<th class="tg-5gjp">v3.4</th>
Expand All @@ -35,17 +35,19 @@ In the table below you can find appropriate Validity version according to your N
<th class="tg-5gjp">v3.7</th>
<th class="tg-5gjp">v4.0</th>
<th class="tg-5gjp">v4.1</th>
<th class="tg-5gjp">v4.2</th>
</tr></thead>
<tbody>
<tr>
<td class="tg-5iin" rowspan="4">Validity<br><br></td>
<td class="tg-5iin" rowspan="5">Validity<br><br></td>
<td class="tg-c6ba">v1.4</td>
<td class="tg-8g55">✅</td>
<td class="tg-8g55">✅</td>
<td class="tg-8g55">✅</td>
<td class="tg-8g55"></td>
<td class="tg-8g55"></td>
<td class="tg-8g55"></td>
<td class="tg-8g55"></td>
</tr>
<tr>
<td class="tg-c6ba">v2.2</td>
Expand All @@ -55,6 +57,7 @@ In the table below you can find appropriate Validity version according to your N
<td class="tg-8g55">✅</td>
<td class="tg-8g55"></td>
<td class="tg-8g55"></td>
<td class="tg-8g55"></td>
</tr>
<tr>
<td class="tg-c6ba">v2.3</td>
Expand All @@ -64,6 +67,7 @@ In the table below you can find appropriate Validity version according to your N
<td class="tg-8g55">✅</td>
<td class="tg-8g55">✅</td>
<td class="tg-8g55"></td>
<td class="tg-8g55"></td>
</tr>
<tr>
<td class="tg-c6ba">v3.0</td>
Expand All @@ -73,5 +77,16 @@ In the table below you can find appropriate Validity version according to your N
<td class="tg-8g55">✅</td>
<td class="tg-8g55">✅</td>
<td class="tg-8g55">✅</td>
<td class="tg-8g55"></td>
</tr>
<tr>
<td class="tg-c6ba">v3.1</td>
<td class="tg-8g55"></td>
<td class="tg-8g55"></td>
<td class="tg-8g55"></td>
<td class="tg-8g55"></td>
<td class="tg-8g55">✅</td>
<td class="tg-8g55">✅</td>
<td class="tg-8g55">✅</td>
</tr>
</tbody></table>
112 changes: 57 additions & 55 deletions docs/installation/plugin_settings.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,22 @@
# Plugin Settings


Validity has some settings which can be changed through [PLUGINS_CONFIG](https://docs.netbox.dev/en/stable/plugins/#configure-plugin) variable inside your `configuration.py`.
Validity has some settings which can be changed through [PLUGINS_CONFIG](https://netboxlabs.com/docs/netbox/en/stable/plugins/installation/) variable inside your `configuration.py`.

## Settings

### result_batch_size
### **custom_pollers**

*Default:* `500`

*Type:* `int`

Execution of the Tests and producing Test Results is carried out in batches. As soon as each batch reaches its maximum size (specified via this variable) all the Test Results within a batch will be uploaded into a DB.


### store_reports

*Default:* `5`

*Type:* `int`

How many [Reports](../entities/results_and_reports.md#reports) should the system store.

If the system creates a new Report and the overall reports count exceeds *store_reports*, then the oldest exceeding report(s) will be deleted.

!!! note
Test Results bound to the Report will be deleted as well.


### polling_threads
*Type:* `list[validity.settings.PollerInfo]`

*Default:* `500`
*Default:* `[]`

*Type:* `int`
This setting allows to connect custom user-defined [Pollers](../entities/pollers.md) to Validity.

Validity uses threads to perform device polling. This setting defines the upper limit of these threads number. If you have extremely large amount of devices (e.g. 100 000), you may want to increase this number to speed up the overall polling process.
Read more about this feature in the [Custom Pollers](../features/custom_pollers.md) article.


### custom_queues
### **custom_queues**

*Type:* `dict[str, str]`

Expand All @@ -59,10 +38,46 @@ Here are the custom queues which may be defined via this setting:
| backup | Queue for backing up individual Backup Points (Back Up button) | default |

!!! warning
The `runtests_queue` setting is deprecated **since version 3.1**. Use `custom_queues.runtests` instead
The `runtests_queue` setting is **deprecated since version 3.1**. Use `custom_queues.runtests` instead


### **integrations**

*Type:* `dict[str, dict]`

*Default:*

```python
{
'git': {'author': 'netbox-validity', 'email': 'validity@netbox.local'}
'S3': {'threads': 10}
}
```

This setting is responsible for various integrations with third-party services, mostly for [Config Backup](../features/config_backup.md) feature.

`threads` param is responsible for the level of parallelism (number of threads used) for multiple files uploading to S3.


### script_timeouts
### **polling_threads**

*Default:* `500`

*Type:* `int`

Validity uses threads to perform device polling. This setting defines the upper limit of these threads number. If you have extremely large amount of devices (e.g. 100 000), you may want to increase this number to speed up the overall polling process.


### **result_batch_size**

*Default:* `500`

*Type:* `int`

Execution of the Tests and producing Test Results is carried out in batches. As soon as each batch reaches its maximum size (specified via this variable) all the Test Results within a batch will be uploaded into database.


### **script_timeouts**

*Type:* `dict[str, str | int]`

Expand All @@ -78,40 +93,24 @@ As was mentioned in the [corresponding article](../entities/scripts.md#stages),
| backup | 10m |


### custom_pollers

*Type:* `list[validity.settings.PollerInfo]`
### **store_reports**

*Default:* `[]`

This setting allows to connect custom user-defined [Pollers](../entities/pollers.md) to Validity.

Read more about this feature in the [Custom Pollers](../features/custom_pollers.md) article.


### integrations

*Type:* `dict[str, dict]`

*Default:*
*Default:* `5`

```python
{
'git': {'author': 'netbox-validity', 'email': 'validity@netbox.local'}
'S3': {'threads': 10}
}
```
*Type:* `int`

This setting is responsible for various integrations with third-party services, mostly for [Config Backup](../features/config_backup.md) feature.
How many [Reports](../entities/results_and_reports.md#reports) should the system store.

`threads` param is responsible for the level of parallelism (number of threads used) for multiple files uploading to S3.
If the system creates a new Report and the overall reports count exceeds *store_reports*, then the oldest exceeding report(s) will be deleted.

!!! warning
Test Results bound to the Report will be deleted as well.


## Settings Example

!!! note
The following example does NOT represent the recommended settings.
The following example does NOT represent the recommended setting values.

Here is the full example of Validity settings:

Expand All @@ -120,10 +119,13 @@ Here is the full example of Validity settings:

PLUGINS_CONFIG = {
'validity': {
'integrations': {
'git': {'author': 'Anton'}
},
'result_batch_size': 300,
'store_reports': 7,
'polling_threads': 100,
'custom_queues': {"runtests": "runtests_q"}
'custom_queues': {"runtests": "runtests_q"},
'script_timeouts': {
'runtests_split': '15m',
'runtests_apply': '1h',
Expand Down
8 changes: 6 additions & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
site_name: Validity
repo_url: https://github.com/amyasnikov/validity/
theme: material
theme:
name: material
icon:
logo: material/checkbox-marked-circle-outline
nav:
- Intro: index.md
- Installation:
Expand All @@ -26,7 +29,8 @@ nav:
- How to work with Device State: features/state.md
- Dynamic Pairs: features/dynamic_pairs.md
- Webhooks: features/webhooks.md
- Config backup: features/config_backup.md
- Custom Pollers: features/custom_pollers.md
- Config Backup: features/config_backup.md

markdown_extensions:
- pymdownx.highlight:
Expand Down

0 comments on commit 6ca1672

Please sign in to comment.