Improve the way the download of exernal database is working

[arno-st]

In 'slow' internet access, the download of the different database can be slow.
And since it's 'inside' cacti process it can provoke some timeout on the poller.
23/12/2024 00:08:01 - POLLER: Poller[Main Poller] PID[1021061] WARNING: Cactid/Cron is out of sync with the Poller Interval! The Poller Interval is '60' seconds, with a maximum of a '60' second Cactid/Cron, but 419.4 seconds have passed since the last poll! 23/12/2024 00:07:15 - FLOWVIEW IRR UPDATE: Downloading arin.db.gz 23/12/2024 00:07:15 - FLOWVIEW IRR UPDATE: IRR Source:arin, Current Serial:4202309, Last Serial:4157315 23/12/2024 00:07:08 - FLOWVIEW IRR UPDATE: Downloading apnic.db.rtr-set.gz 23/12/2024 00:07:03 - FLOWVIEW IRR UPDATE: Downloading apnic.db.route6.gz 23/12/2024 00:07:01 - MAILER INFO: Mail successfully sent via SMTP from 'Cacti 1.2.x Prod <EMAIL>', to 'Administrator <EMAIL>', cc '', bcc '', and took 0.07 seconds, Subject 'Cacti System Warning' 23/12/2024 00:07:01 - POLLER: Poller[Main Poller] PID[1021029] WARNING: Cactid/Cron is out of sync with the Poller Interval! The Poller Interval is '60' seconds, with a maximum of a '60' second Cactid/Cron, but 359.4 seconds have passed since the last poll! 23/12/2024 00:06:58 - FLOWVIEW IRR UPDATE: Downloading apnic.db.route.gz 23/12/2024 00:06:53 - FLOWVIEW IRR UPDATE: Downloading apnic.db.route-set.gz 23/12/2024 00:06:48 - FLOWVIEW IRR UPDATE: Downloading apnic.db.role.gz 23/12/2024 00:06:43 - FLOWVIEW IRR UPDATE: Downloading apnic.db.peering-set.gz 23/12/2024 00:06:38 - FLOWVIEW IRR UPDATE: Downloading apnic.db.organisation.gz 23/12/2024 00:06:33 - FLOWVIEW IRR UPDATE: Downloading apnic.db.mntner.gz 23/12/2024 00:06:28 - FLOWVIEW IRR UPDATE: Downloading apnic.db.key-cert.gz 23/12/2024 00:06:23 - FLOWVIEW IRR UPDATE: Downloading apnic.db.irt.gz 23/12/2024 00:06:18 - FLOWVIEW IRR UPDATE: Downloading apnic.db.inetnum.gz 23/12/2024 00:06:13 - FLOWVIEW IRR UPDATE: Downloading apnic.db.inet6num.gz 23/12/2024 00:06:08 - FLOWVIEW IRR UPDATE: Downloading apnic.db.inet-rtr.gz 23/12/2024 00:06:03 - FLOWVIEW IRR UPDATE: Downloading apnic.db.filter-set.gz 23/12/2024 00:06:01 - MAILER INFO: Mail successfully sent via SMTP from 'Cacti 1.2.x Prod <EMAIL>', to 'Administrator <EMAIL>', cc '', bcc '', and took 0.06 seconds, Subject 'Cacti System Warning' 23/12/2024 00:06:01 - POLLER: Poller[Main Poller] PID[1020990] WARNING: Cactid/Cron is out of sync with the Poller Interval! The Poller Interval is '60' seconds, with a maximum of a '60' second Cactid/Cron, but 299.4 seconds have passed since the last poll! 23/12/2024 00:05:58 - FLOWVIEW IRR UPDATE: Downloading apnic.db.domain.gz 23/12/2024 00:05:53 - FLOWVIEW IRR UPDATE: Downloading apnic.db.aut-num.gz 23/12/2024 00:05:48 - FLOWVIEW IRR UPDATE: Downloading apnic.db.as-set.gz 23/12/2024 00:05:43 - FLOWVIEW IRR UPDATE: Downloading apnic.db.as-block.gz 23/12/2024 00:05:43 - FLOWVIEW IRR UPDATE: IRR Source:apnic, Current Serial:12769501, Last Serial:12760161 23/12/2024 00:05:01 - MAILER INFO: Mail successfully sent via SMTP from 'Cacti 1.2.x Prod <EMAIL>', to 'Administrator <EMAIL>', cc '', bcc '', and took 0.06 seconds, Subject 'Cacti System Warning'

Maybe if we can do a exec in the background, and have a flag to let flowview_poller_bottom, the donwload is done, the import can be done inline. Even if on this log I don't wee the import!

Maybe keep in settings table, the imported serial number, and the new serial number, so if the number is not the same, that an import has to be done.

or something like that.

[TheWitness]

It's supposed to be in background already.

[arno-st]

I come back on this one, maybe the title is wrong, but I do have an issue, and can't find where it is.
Every night between 00:00 and 00:10 my poller is screw up, and I'm getting a lot of system error telling me that the poller output table is not empty, poller is out of synch. like those one (3 different messages, but same night, same 10 minutes windows):
WARNING: There are 47 processes detected as overrunning a polling cycle for poller id 1, please investigate. . WARNING: Cactid/Cron is out of sync with the Poller Interval for poller id 1! The Poller Interval is 60 seconds, with a maximum of a 60 seconds, but 482 seconds have passed since the last poll! . Maximum runtime of 58 seconds exceeded for poller id 1. Exiting.

If I disable flowview, every thing is fine, so it's look like something is happening at midnight, but I can't figure out wat exactly.
is the download of external DB that is taking forever to insert into the DB that lock something for too long ?
Is it that another process from flowview is doing some cleaning and lock the table ?

How can I find where is the glitch ?

[TheWitness]

How large is your system (cores, threads, memory)? At midnight, it's downloading the Internet Router Registry databases from around the world. This process does a lot of I/O to the database. So, if you are on a tiny VM, you don't have enough memory or threads, the system will hang up. Also, if you have a "out of band" database back taking place, and it's not done in a suitable way, your Cacti system will additionally hang. If you are using mariabackup, you need to time that backout outside of the time that the IRR Database is being synced.

[arno-st]

Nothing is under VM.

My front end is:
CPU 2x10 core @ 2.2 GHz
RAM 256 GiB
System DISk : 1*440TB ( os )
DATA disk : 2 drive mirrored 2.91 TB

The database servers, 2 system with MariaDB 10.6.17, and Galera
CPU 2x10 core @ 2.2 GHz
RAM 256 GiB
system Disk : 1*440TB ( os )
DATA Disk: 4 drive mirrored 2x2.91 TiB, 5.9TB usable.

Cacti access the DB via MaxScale configured with 1 services for Cacti, 1 for FlowView:

[CactiService]
type=service
connection_keepalive=1000ms
max_connections=16384
router=readwritesplit
servers=lslmysp12,lslmysp11
max_slave_connections=0

[FlowViewService]
type=service
max_connections=16384
router=readwritesplit
servers=lslmysp11,lslmysp12
master_accept_reads=true