Graceful Restart procedures do not apply when hard-administrative-reset flag is disabled #18238

dawkopagh · 2025-02-24T07:04:14Z

Description

When sending CEASE/Administrative Reset (clear bgp) with N-bit is set and bgp hard-administrative-reset disabled routes are being withdrawn from kernel. This should not be the case per this description

Version

My reproduction uses 10.1.1, however the same happens on 10.3-dev

3d4364ce7392# show ver
FRRouting 10.1.1 (3d4364ce7392) on Linux(5.15.0-127-generic).

How to reproduce

I have provided an instruction in my public repo here, just follow README.md

Expected behavior

Graceful Restart procedures should apply and routes should be retained in kernel

Actual behavior

Routes are being withdrawn from kernel when the session goes down and re-added after session establishes again.

Additional context

No response

Checklist

I have searched the open issues for this bug.
I have not included sensitive information in this report.

The text was updated successfully, but these errors were encountered:

ton31337 · 2025-02-24T09:29:43Z

Could you show a full configuration?

Update: I see the config in your repo. Will test it.

piotrsuchy · 2025-02-25T09:22:30Z

I'll test it as @dawkopagh is out for a week and I'll let you know

ton31337 · 2025-02-25T13:37:37Z

Read monitor file, you should see bunch of deletes and then installs into kernel after the session comes back up.

When the session comes up the router resends the BGP table, hence you see withdraws + updates, that's kinda expected.

I don't really understand where is an issue yet.

piotrsuchy · 2025-03-03T14:13:18Z

The issue is that based on the docs:

"When N-bit is set and bgp hard-administrative-reset is turned off Graceful-Restart is activated and routes are retained."

This is the case for this setup, however there is no graceful restart - it just does a typical restart, withdrawing the routes and then adding them. Expected behavior based on the docs description would be:

No immediate withdrawal from the kernel.
Routes remain installed as “stale.”
When the session reestablishes, the daemon reconfirms those same routes (or updates them if changes occurred).

And this is what was happening in FRR8.

ton31337 · 2025-03-03T14:20:53Z

What happens with FRR 10.x (or master)? Because I checked with the master, and it works fine (stale routes are retained when hard-administrative-reset is off). Could you check this topotest? It's actually what it does.

TC2: Trigger `clear bgp` (Administrative Reset):
    `bgp hard-administrative-reset` disabled:
        a) Check if Administrative Reset notification was sent from R2
        b) Routes should be retained on R1

piotrsuchy · 2025-03-03T15:00:10Z

This is me checking it on frr master:

root@45816009ab17:~# vtysh -c 'show ver'
FRRouting 10.4-dev (45816009ab17) on Linux(5.15.0-119-generic).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--prefix=/usr' '--enable-multipath=64' '--localstatedir=/var/run/frr' '--sysconfdir=/etc/frr' '--enable-user=root' '--enable-group=root' '--enable-vtysh'
root@45816009ab17:~# ip r
default via 192.168.9.1 dev eth0
192.168.9.0/24 dev eth0 proto kernel scope link src 192.168.9.2
192.168.9.100 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
192.168.9.101 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
192.168.9.102 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
192.168.9.103 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
192.168.9.104 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
192.168.9.105 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
192.168.9.106 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
192.168.9.107 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
192.168.9.108 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
root@45816009ab17:~# vtysh -c 'show ip route'
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

IPv4 unicast VRF default:
K>* 0.0.0.0/0 [0/0] via 192.168.9.1, eth0, weight 1, 00:01:23
C>* 192.168.9.0/24 is directly connected, eth0, weight 1, 00:01:23
L>* 192.168.9.2/32 is directly connected, eth0, weight 1, 00:01:23
B>* 192.168.9.100/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:38
B>* 192.168.9.101/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:38
B>* 192.168.9.102/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:38
B>* 192.168.9.103/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:38
B>* 192.168.9.104/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:38
B>* 192.168.9.105/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:38
B>* 192.168.9.106/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:38
B>* 192.168.9.107/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:38
B>* 192.168.9.108/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:38
L * 192.168.100.1/32 is directly connected, lo, weight 1, 00:01:23
C>* 192.168.100.1/32 is directly connected, lo, weight 1, 00:01:23
root@45816009ab17:~# vtysh -c 'clear bgp 2001:db8::2'
## running the show ip route soon after clearing - those routes weren't there
root@45816009ab17:~# vtysh -c 'show ip route'
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

IPv4 unicast VRF default:
K>* 0.0.0.0/0 [0/0] via 192.168.9.1, eth0, weight 1, 00:01:39
C>* 192.168.9.0/24 is directly connected, eth0, weight 1, 00:01:39
L>* 192.168.9.2/32 is directly connected, eth0, weight 1, 00:01:39
L * 192.168.100.1/32 is directly connected, lo, weight 1, 00:01:39
C>* 192.168.100.1/32 is directly connected, lo, weight 1, 00:01:39
root@45816009ab17:~# vtysh -c 'show ip route'
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

IPv4 unicast VRF default:
K>* 0.0.0.0/0 [0/0] via 192.168.9.1, eth0, weight 1, 00:01:40
C>* 192.168.9.0/24 is directly connected, eth0, weight 1, 00:01:40
L>* 192.168.9.2/32 is directly connected, eth0, weight 1, 00:01:40
B>* 192.168.9.100/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:00
B>* 192.168.9.101/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:00
B>* 192.168.9.102/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:00
B>* 192.168.9.103/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:00
B>* 192.168.9.104/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:00
B>* 192.168.9.105/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:00
B>* 192.168.9.106/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:00
B>* 192.168.9.107/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:00
B>* 192.168.9.108/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:00
L * 192.168.100.1/32 is directly connected, lo, weight 1, 00:01:40
C>* 192.168.100.1/32 is directly connected, lo, weight 1, 00:01:40
root@45816009ab17:~# vtysh -c 'show ip route'
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

IPv4 unicast VRF default:
K>* 0.0.0.0/0 [0/0] via 192.168.9.1, eth0, weight 1, 00:01:41
C>* 192.168.9.0/24 is directly connected, eth0, weight 1, 00:01:41
L>* 192.168.9.2/32 is directly connected, eth0, weight 1, 00:01:41
B>* 192.168.9.100/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:01
B>* 192.168.9.101/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:01
B>* 192.168.9.102/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:01
B>* 192.168.9.103/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:01
B>* 192.168.9.104/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:01
B>* 192.168.9.105/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:01
B>* 192.168.9.106/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:01
B>* 192.168.9.107/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:01
B>* 192.168.9.108/32 [20/0] via 192.168.9.21, brv3 onlink, weight 1, 00:00:01
L * 192.168.100.1/32 is directly connected, lo, weight 1, 00:01:41
C>* 192.168.100.1/32 is directly connected, lo, weight 1, 00:01:41
root@45816009ab17:~# vtysh -c 'show run' | grep 'no bgp ad'
root@45816009ab17:~# vtysh -c 'show run' | grep 'no bgp'
 no bgp hard-administrative-reset
 no bgp default ipv4-unicast
root@45816009ab17:~#

And checking the rtmon logs:

root@45816009ab17:~# ip monitor fil $(hostname -s)_rtmon.log
...
earlier logs
...
Deleted 192.168.9.100 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:21 2025 764705 us
Deleted 192.168.9.101 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:21 2025 764710 us
Deleted 192.168.9.102 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:21 2025 764714 us
Deleted 192.168.9.103 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:21 2025 764748 us
Deleted 192.168.9.104 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:21 2025 764775 us
Deleted 192.168.9.105 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:21 2025 764803 us
Deleted 192.168.9.106 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:21 2025 764825 us
Deleted 192.168.9.107 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:21 2025 764847 us
Deleted 192.168.9.108 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:24 2025 769302 us
192.168.9.108 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:24 2025 769325 us
192.168.9.107 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:24 2025 769328 us
192.168.9.106 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:24 2025 769331 us
192.168.9.105 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:24 2025 769333 us
192.168.9.104 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:24 2025 769335 us
192.168.9.103 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:24 2025 769338 us
192.168.9.102 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:24 2025 769340 us
192.168.9.101 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Mon Mar  3 14:56:24 2025 769343 us
192.168.9.100 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink

In my opinion, because the routes have a new timer and there was a moment when they weren't there and they appear as deleted and readded in the logs, that it means that the graceful restart is not properly applied, even though the no bgp hard-administrative-reset is set.

ton31337 · 2025-03-03T20:34:20Z

Could you show the "show bgp ipv4 detail" after clear? And please show a full log before/after the clear (with "debug bgp updates", "debug bgp neighbor").

piotrsuchy · 2025-03-05T16:39:50Z

1-st scenario logs (for 10.1.1, exactly the same thing happening for master)

With logs:

2025/03/05 16:33:50 BGP: [H7QV4-WR3ZD] %NOTIFICATION: sent to neighbor 2001:db8::2 6/4 (Cease/Administrative Reset) ""
2025/03/05 16:33:50 BGP: [H461A-BCRD2] 2001:db8::2(5049e83d99a7): Peer flags updated to 0x3003412420, GR flags 0x6, GR mode PEER_GLOBAL_INHERIT
2025/03/05 16:33:50 BGP: [H7QV4-WR3ZD] %NOTIFICATION: sent to neighbor 2001:db8::2 6/4 (Cease/Administrative Reset) ""
2025/03/05 16:33:50 BGP: [H461A-BCRD2] 2001:db8::2(5049e83d99a7): Peer flags updated to 0x3003412420, GR flags 0x6, GR mode PEER_GLOBAL_INHERIT
2025/03/05 16:33:50 BGP: [H7QV4-WR3ZD] %NOTIFICATION: sent to neighbor 2001:db8::2 6/4 (Cease/Administrative Reset) ""
2025/03/05 16:33:50 BGP: [H461A-BCRD2] 2001:db8::2(5049e83d99a7): Peer flags updated to 0x3003412420, GR flags 0x6, GR mode PEER_GLOBAL_INHERIT
2025/03/05 16:33:50 BGP: [ZWCSR-M7FG9] 2001:db8::2 [FSM] BGP_Stop (Established->Clearing), fd 22
2025/03/05 16:33:50 BGP: [PXVXG-TFNNT] %ADJCHANGE: neighbor 2001:db8::2(5049e83d99a7) in vrf default Down User reset
2025/03/05 16:33:50 BGP: [V4R0W-D4WGF] 2001:db8::2(Unknown) Update Group Hash: sort: 2 sub_sort: 0 UpdGrpFlags: 0 UpdGrpAFFlags: 553648131
2025/03/05 16:33:50 BGP: [NVVBY-K8MCE] 2001:db8::2(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 18432 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2025/03/05 16:33:50 BGP: [X4CQ0-63QKB] 2001:db8::2(Unknown) Update Group Hash: addpath paths-limit: (send 0, receive 0)
2025/03/05 16:33:50 BGP: [Z8Q37-65KK3] 2001:db8::2(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: rs rmap out: (NONE)
2025/03/05 16:33:50 BGP: [SM2F3-HRYKP] 2001:db8::2(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2025/03/05 16:33:50 BGP: [V8B3M-T6VFC] 2001:db8::2(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 0
2025/03/05 16:33:50 BGP: [Y5EX3-GHT5V] 2001:db8::2(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2025/03/05 16:33:50 BGP: [X19K7-9V4K2] 2001:db8::2(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2025/03/05 16:33:50 BGP: [SQ314-QBJCR] 2001:db8::2(Unknown) Update Group Hash key: 2617206404
2025/03/05 16:33:50 BGP: [V4R0W-D4WGF] 2001:db8::2(Unknown) Update Group Hash: sort: 2 sub_sort: 0 UpdGrpFlags: 0 UpdGrpAFFlags: 553648131
2025/03/05 16:33:50 BGP: [NVVBY-K8MCE] 2001:db8::2(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 2048 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2025/03/05 16:33:50 BGP: [X4CQ0-63QKB] 2001:db8::2(Unknown) Update Group Hash: addpath paths-limit: (send 0, receive 0)
2025/03/05 16:33:50 BGP: [Z8Q37-65KK3] 2001:db8::2(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: rs rmap out: (NONE)
2025/03/05 16:33:50 BGP: [SM2F3-HRYKP] 2001:db8::2(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2025/03/05 16:33:50 BGP: [V8B3M-T6VFC] 2001:db8::2(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 1
2025/03/05 16:33:50 BGP: [Y5EX3-GHT5V] 2001:db8::2(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2025/03/05 16:33:50 BGP: [X19K7-9V4K2] 2001:db8::2(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2025/03/05 16:33:50 BGP: [SQ314-QBJCR] 2001:db8::2(Unknown) Update Group Hash key: 675778512
2025/03/05 16:33:50 BGP: [V4R0W-D4WGF] 2001:db8::2(Unknown) Update Group Hash: sort: 2 sub_sort: 0 UpdGrpFlags: 0 UpdGrpAFFlags: 553648259
2025/03/05 16:33:50 BGP: [NVVBY-K8MCE] 2001:db8::2(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 2048 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2025/03/05 16:33:50 BGP: [X4CQ0-63QKB] 2001:db8::2(Unknown) Update Group Hash: addpath paths-limit: (send 0, receive 0)
2025/03/05 16:33:50 BGP: [Z8Q37-65KK3] 2001:db8::2(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: rs rmap out: (NONE)
2025/03/05 16:33:50 BGP: [SM2F3-HRYKP] 2001:db8::2(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2025/03/05 16:33:50 BGP: [V8B3M-T6VFC] 2001:db8::2(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 0
2025/03/05 16:33:50 BGP: [Y5EX3-GHT5V] 2001:db8::2(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2025/03/05 16:33:50 BGP: [X19K7-9V4K2] 2001:db8::2(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2025/03/05 16:33:50 BGP: [SQ314-QBJCR] 2001:db8::2(Unknown) Update Group Hash key: 2997845311
2025/03/05 16:33:50 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Clearing established_peers 0
2025/03/05 16:33:50 BGP: [HKWM3-ZC5QP] 2001:db8::2 fd -1 went from Established to Clearing
2025/03/05 16:33:50 BGP: [ZWCSR-M7FG9] 2001:db8::2 [FSM] Clearing_Completed (Clearing->Idle), fd -1
2025/03/05 16:33:50 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Idle established_peers 0
2025/03/05 16:33:50 BGP: [HKWM3-ZC5QP] 2001:db8::2 fd -1 went from Clearing to Idle
2025/03/05 16:33:52 BGP: [ZQTB5-H8522] 2001:db8::2 [FSM] Timer (start timer expire).
2025/03/05 16:33:52 BGP: [ZWCSR-M7FG9] 2001:db8::2 [FSM] BGP_Start (Idle->Connect), fd -1
2025/03/05 16:33:52 BGP: [Z195V-FNKRK] 2001:db8::2 [Event] Connect start to 2001:db8::2 fd 22
2025/03/05 16:33:52 BGP: [G0837-S7QES] 2001:db8::2 [FSM] Non blocking connect waiting result, fd 22
2025/03/05 16:33:52 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Connect established_peers 0
2025/03/05 16:33:52 BGP: [HKWM3-ZC5QP] 2001:db8::2 fd 22 went from Idle to Connect
2025/03/05 16:33:52 BGP: [ZWCSR-M7FG9] 2001:db8::2 [FSM] TCP_connection_open (Connect->OpenSent), fd 22
2025/03/05 16:33:52 BGP: [RWZTG-AA74G] 2001:db8::2 open active, local address 2001:db8:1::1
2025/03/05 16:33:52 BGP: [XKJ09-9VTZ7] 2001:db8::2 Sending hostname cap with hn = 5ce23506af95, dn = (null)
2025/03/05 16:33:52 BGP: [HVWXV-RN5J7] 2001:db8::2: Sending GR Capability, Restart time 30 R-bit NOT-SET, N-bit SET
2025/03/05 16:33:52 BGP: [P927W-B5CB8] ... F-bit SET for IPv4 Unicast
2025/03/05 16:33:52 BGP: [P927W-B5CB8] ... F-bit SET for IPv6 Unicast
2025/03/05 16:33:52 BGP: [P927W-B5CB8] ... F-bit SET for L2VPN EVPN
2025/03/05 16:33:52 BGP: [YJSKD-N5GGC] 2001:db8::2 Sending Software Version cap, value: FRRouting/10.3-dev
2025/03/05 16:33:52 BGP: [G1CME-TC11B] 2001:db8::2(5049e83d99a7) fd 22 sending OPEN, version 4, my as 65001, holdtime 9, id 192.168.9.5
2025/03/05 16:33:52 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: OpenSent established_peers 0
2025/03/05 16:33:52 BGP: [HKWM3-ZC5QP] 2001:db8::2 fd 22 went from Connect to OpenSent
2025/03/05 16:33:52 BGP: [WNM1E-D314G] 2001:db8::2 rcv OPEN, version 4, remote-as (in open) 65002, holdtime 180, id 192.168.9.2
2025/03/05 16:33:52 BGP: [QG29C-5TSVS] 2001:db8::2 rcv OPEN w/ OPTION parameter len: 150
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has MultiProtocol Extensions capability (1), length 4
2025/03/05 16:33:52 BGP: [VXVH9-3MXR0] 2001:db8::2 OPEN has MultiProtocol Extensions capability for afi/safi: IPv4/unicast
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 8
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has Extended Next Hop Encoding capability (5), length 6
2025/03/05 16:33:52 BGP: [XCS8V-P66M1] 2001:db8::2 Received with afi/safi/next-hop afi: IPv4/unicast/2
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has MultiProtocol Extensions capability (1), length 4
2025/03/05 16:33:52 BGP: [VXVH9-3MXR0] 2001:db8::2 OPEN has MultiProtocol Extensions capability for afi/safi: IPv6/unicast
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has MultiProtocol Extensions capability (1), length 4
2025/03/05 16:33:52 BGP: [VXVH9-3MXR0] 2001:db8::2 OPEN has MultiProtocol Extensions capability for afi/safi: l2vpn/evpn
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has Route Refresh capability (2), length 0
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has Enhanced Route Refresh capability (70), length 0
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has 4-octet AS number capability (65), length 4
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has BGP Extended Message capability (6), length 0
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 14
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has AddPath capability (69), length 12
2025/03/05 16:33:52 BGP: [SBSKM-G6QBW] 2001:db8::2 OPEN has AddPath capability for afi/safi: IPv4/unicast, receive
2025/03/05 16:33:52 BGP: [SBSKM-G6QBW] 2001:db8::2 OPEN has AddPath capability for afi/safi: IPv6/unicast, receive
2025/03/05 16:33:52 BGP: [SBSKM-G6QBW] 2001:db8::2 OPEN has AddPath capability for afi/safi: l2vpn/evpn, receive
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 17
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has Paths-Limit capability (76), length 15
2025/03/05 16:33:52 BGP: [MSJ7S-E9MRG] 2001:db8::2 OPEN has Paths-Limit capability for afi/safi: IPv4/unicast limit: 0
2025/03/05 16:33:52 BGP: [MSJ7S-E9MRG] 2001:db8::2 OPEN has Paths-Limit capability for afi/safi: IPv6/unicast limit: 0
2025/03/05 16:33:52 BGP: [MSJ7S-E9MRG] 2001:db8::2 OPEN has Paths-Limit capability for afi/safi: l2vpn/evpn limit: 0
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 16
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has FQDN capability (73), length 14
2025/03/05 16:33:52 BGP: [MR6EG-NS6N7] 2001:db8::2 received hostname 5049e83d99a7, domainname (null)
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 16
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has Graceful Restart capability (64), length 14
2025/03/05 16:33:52 BGP: [V30NV-6FS3W] 2001:db8::2(5049e83d99a7) OPEN has GR capability, Restart time 30 R-bit NOT-SET N-bit SET
2025/03/05 16:33:52 BGP: [W0E2T-VG0J8] 2001:db8::2(5049e83d99a7) F-bit NOT-SET for IPv4 Unicast
2025/03/05 16:33:52 BGP: [W0E2T-VG0J8] 2001:db8::2(5049e83d99a7) F-bit NOT-SET for IPv6 Unicast
2025/03/05 16:33:52 BGP: [W0E2T-VG0J8] 2001:db8::2(5049e83d99a7) F-bit NOT-SET for L2VPN EVPN
2025/03/05 16:33:52 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 23
2025/03/05 16:33:52 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has Long-lived BGP Graceful Restart capability (71), length 21
2025/03/05 16:33:52 BGP: [SGV3H-AGFR8] 2001:db8::2 Addr-family IPv4/unicast(afi/safi) Long-lived Graceful Restart capability stale time 0 sec
2025/03/05 16:33:52 BGP: [SGV3H-AGFR8] 2001:db8::2 Addr-family IPv6/unicast(afi/safi) Long-lived Graceful Restart capability stale time 0 sec
2025/03/05 16:33:52 BGP: [SGV3H-AGFR8] 2001:db8::2 Addr-family l2vpn/evpn(afi/safi) Long-lived Graceful Restart capability stale time 0 sec
2025/03/05 16:33:52 BGP: [ZWCSR-M7FG9] 2001:db8::2 [FSM] Receive_OPEN_message (OpenSent->OpenConfirm), fd 22
2025/03/05 16:33:52 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: OpenConfirm established_peers 0
2025/03/05 16:33:52 BGP: [HKWM3-ZC5QP] 2001:db8::2 fd 22 went from OpenSent to OpenConfirm
2025/03/05 16:33:52 BGP: [ZWCSR-M7FG9] 2001:db8::2 [FSM] Receive_KEEPALIVE_message (OpenConfirm->Established), fd 22
2025/03/05 16:33:52 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Established established_peers 1
2025/03/05 16:33:52 BGP: [HKWM3-ZC5QP] 2001:db8::2 fd 22 went from OpenConfirm to Established
2025/03/05 16:33:52 BGP: [N9HHH-F8H1M] %ADJCHANGE: neighbor 2001:db8::2(5049e83d99a7) in vrf default Up
2025/03/05 16:33:52 BGP: [V4R0W-D4WGF] 2001:db8::2(Unknown) Update Group Hash: sort: 2 sub_sort: 0 UpdGrpFlags: 0 UpdGrpAFFlags: 553648131
2025/03/05 16:33:52 BGP: [NVVBY-K8MCE] 2001:db8::2(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 18432 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2025/03/05 16:33:52 BGP: [X4CQ0-63QKB] 2001:db8::2(Unknown) Update Group Hash: addpath paths-limit: (send 0, receive 0)
2025/03/05 16:33:52 BGP: [Z8Q37-65KK3] 2001:db8::2(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: rs rmap out: (NONE)
2025/03/05 16:33:52 BGP: [SM2F3-HRYKP] 2001:db8::2(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2025/03/05 16:33:52 BGP: [V8B3M-T6VFC] 2001:db8::2(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 0
2025/03/05 16:33:52 BGP: [Y5EX3-GHT5V] 2001:db8::2(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2025/03/05 16:33:52 BGP: [X19K7-9V4K2] 2001:db8::2(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2025/03/05 16:33:52 BGP: [SQ314-QBJCR] 2001:db8::2(Unknown) Update Group Hash key: 2617206404
2025/03/05 16:33:52 BGP: [V4R0W-D4WGF] 2001:db8::2(Unknown) Update Group Hash: sort: 2 sub_sort: 0 UpdGrpFlags: 0 UpdGrpAFFlags: 553648131
2025/03/05 16:33:52 BGP: [NVVBY-K8MCE] 2001:db8::2(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 2048 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2025/03/05 16:33:52 BGP: [X4CQ0-63QKB] 2001:db8::2(Unknown) Update Group Hash: addpath paths-limit: (send 0, receive 0)
2025/03/05 16:33:52 BGP: [Z8Q37-65KK3] 2001:db8::2(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: rs rmap out: (NONE)
2025/03/05 16:33:52 BGP: [SM2F3-HRYKP] 2001:db8::2(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2025/03/05 16:33:52 BGP: [V8B3M-T6VFC] 2001:db8::2(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 1
2025/03/05 16:33:52 BGP: [Y5EX3-GHT5V] 2001:db8::2(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2025/03/05 16:33:52 BGP: [X19K7-9V4K2] 2001:db8::2(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2025/03/05 16:33:52 BGP: [SQ314-QBJCR] 2001:db8::2(Unknown) Update Group Hash key: 675778512
2025/03/05 16:33:52 BGP: [V4R0W-D4WGF] 2001:db8::2(Unknown) Update Group Hash: sort: 2 sub_sort: 0 UpdGrpFlags: 0 UpdGrpAFFlags: 553648259
2025/03/05 16:33:52 BGP: [NVVBY-K8MCE] 2001:db8::2(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 2048 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2025/03/05 16:33:52 BGP: [X4CQ0-63QKB] 2001:db8::2(Unknown) Update Group Hash: addpath paths-limit: (send 0, receive 0)
2025/03/05 16:33:52 BGP: [Z8Q37-65KK3] 2001:db8::2(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: rs rmap out: (NONE)
2025/03/05 16:33:52 BGP: [SM2F3-HRYKP] 2001:db8::2(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2025/03/05 16:33:52 BGP: [V8B3M-T6VFC] 2001:db8::2(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 0
2025/03/05 16:33:52 BGP: [Y5EX3-GHT5V] 2001:db8::2(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2025/03/05 16:33:52 BGP: [X19K7-9V4K2] 2001:db8::2(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2025/03/05 16:33:52 BGP: [SQ314-QBJCR] 2001:db8::2(Unknown) Update Group Hash key: 2997845311
2025/03/05 16:33:52 BGP: [P3D3N-3277A] 2001:db8::2 [FSM] Timer (routeadv timer expire)
2025/03/05 16:33:53 BGP: [K9G3A-CJ5QQ] u7:s7 announcing routes upon coalesce timer expiry(1000 ms)
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=0.0.0.0/0, selected=0x5617c8c92090
2025/03/05 16:33:53 BGP: [K9G3A-CJ5QQ] u8:s8 announcing routes upon coalesce timer expiry(1000 ms)
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=::/0, selected=0x5617c8d09100
2025/03/05 16:33:53 BGP: [K9G3A-CJ5QQ] u9:s9 announcing routes upon coalesce timer expiry(1000 ms)
2025/03/05 16:33:53 BGP: [P3D3N-3277A] 2001:db8::2 [FSM] Timer (routeadv timer expire)
2025/03/05 16:33:53 BGP: [TN0HX-6G1RR] u7:s7 send UPDATE w/ attr: nexthop 192.168.9.1, origin ?, mp_nexthop ::, metric 0, path
2025/03/05 16:33:53 BGP: [H06SA-0JAPR] u7:s7 send MP_REACH for afi/safi IPv4/unicast
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u7:s7 send UPDATE 0.0.0.0/0 IPv4 unicast
2025/03/05 16:33:53 BGP: [WEV7K-2GAQ5] u7:s7 send UPDATE len 70 (max message len: 65535) numpfx 1
2025/03/05 16:33:53 BGP: [NNG4V-WFG7M] u7:s7 2001:db8::2 send UPDATE w/ mp_nexthop 2001:db8:1::1
2025/03/05 16:33:53 BGP: [TN0HX-6G1RR] u8:s8 send UPDATE w/ attr: , origin ?, mp_nexthop ::, aigp-metric 1024, metric 1024, path
2025/03/05 16:33:53 BGP: [H06SA-0JAPR] u8:s8 send MP_REACH for afi/safi IPv6/unicast
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u8:s8 send UPDATE ::/0 IPv6 unicast
2025/03/05 16:33:53 BGP: [WEV7K-2GAQ5] u8:s8 send UPDATE len 70 (max message len: 65535) numpfx 1
2025/03/05 16:33:53 BGP: [NNG4V-WFG7M] u8:s8 2001:db8::2 send UPDATE w/ mp_nexthop 2001:db8:1::1
2025/03/05 16:33:53 BGP: [ZP3RE-J4Q8C] send End-of-RIB for L2VPN EVPN to 2001:db8::2
2025/03/05 16:33:53 BGP: [ZP3RE-J4Q8C] send End-of-RIB for IPv4 Unicast to 2001:db8::2
2025/03/05 16:33:53 BGP: [ZP3RE-J4Q8C] send End-of-RIB for IPv6 Unicast to 2001:db8::2
2025/03/05 16:33:53 BGP: [WT375-N4KPV] EOR REQ 0, EOR RCV 0
2025/03/05 16:33:53 BGP: [G3V0W-FSWRC] bgp_best_path_select_defer: processing route for IPv4 Unicast : cnt 0
2025/03/05 16:33:53 BGP: [M59KS-A3ZXZ] bgp_update_receive: rcvd End-of-RIB for IPv4 Unicast from 2001:db8::2 in vrf default
2025/03/05 16:33:53 BGP: [XXWBM-V772F] 2001:db8::2(5049e83d99a7) rcvd UPDATE w/ attr: , origin ?, mp_nexthop 2001:db8::2, path 65002 65001
2025/03/05 16:33:53 BGP: [RZMGQ-A03CG] 2001:db8::2(5049e83d99a7) rcvd UPDATE about ::/0 IPv6 unicast -- DENIED due to: as-path contains our own AS;
2025/03/05 16:33:53 BGP: [XXWBM-V772F] 2001:db8::2(5049e83d99a7) rcvd UPDATE w/ attr: nexthop 192.168.9.21, metric 0, extcommunity RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb, path 65002
2025/03/05 16:33:53 BGP: [YCKEM-GB33T] 2001:db8::2(5049e83d99a7) rcvd RD 65002:3 [5]:[0]:[0]:[0.0.0.0] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [YCKEM-GB33T] 2001:db8::2(5049e83d99a7) rcvd RD 65002:3 [5]:[0]:[32]:[192.168.9.100] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [YCKEM-GB33T] 2001:db8::2(5049e83d99a7) rcvd RD 65002:3 [5]:[0]:[32]:[192.168.9.101] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [YCKEM-GB33T] 2001:db8::2(5049e83d99a7) rcvd RD 65002:3 [5]:[0]:[32]:[192.168.9.102] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [YCKEM-GB33T] 2001:db8::2(5049e83d99a7) rcvd RD 65002:3 [5]:[0]:[32]:[192.168.9.103] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [YCKEM-GB33T] 2001:db8::2(5049e83d99a7) rcvd RD 65002:3 [5]:[0]:[32]:[192.168.9.104] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [YCKEM-GB33T] 2001:db8::2(5049e83d99a7) rcvd RD 65002:3 [5]:[0]:[32]:[192.168.9.105] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [YCKEM-GB33T] 2001:db8::2(5049e83d99a7) rcvd RD 65002:3 [5]:[0]:[32]:[192.168.9.106] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [YCKEM-GB33T] 2001:db8::2(5049e83d99a7) rcvd RD 65002:3 [5]:[0]:[32]:[192.168.9.107] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [YCKEM-GB33T] 2001:db8::2(5049e83d99a7) rcvd RD 65002:3 [5]:[0]:[32]:[192.168.9.108] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [XXWBM-V772F] 2001:db8::2(5049e83d99a7) rcvd UPDATE w/ attr: , origin ?, mp_nexthop 2001:db8::2, metric 0, path 65002
2025/03/05 16:33:53 BGP: [YCKEM-GB33T] 2001:db8::2(5049e83d99a7) rcvd 2001:db8:1::1/128 IPv6 unicast
2025/03/05 16:33:53 BGP: [WT375-N4KPV] EOR REQ 0, EOR RCV 0
2025/03/05 16:33:53 BGP: [G3V0W-FSWRC] bgp_best_path_select_defer: processing route for L2VPN EVPN : cnt 0
2025/03/05 16:33:53 BGP: [M59KS-A3ZXZ] bgp_update_receive: rcvd End-of-RIB for L2VPN EVPN from 2001:db8::2 in vrf default
2025/03/05 16:33:53 BGP: [WT375-N4KPV] EOR REQ 0, EOR RCV 0
2025/03/05 16:33:53 BGP: [G3V0W-FSWRC] bgp_best_path_select_defer: processing route for IPv6 Unicast : cnt 0
2025/03/05 16:33:53 BGP: [M59KS-A3ZXZ] bgp_update_receive: rcvd End-of-RIB for IPv6 Unicast from 2001:db8::2 in vrf default
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=l2vpn, safi=evpn, p=[5]:[0]:[0]:[0.0.0.0]
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=[5]:[0]:[0]:[0.0.0.0], selected=0x5617c8c6e6c0
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=l2vpn, safi=evpn, p=[5]:[0]:[32]:[192.168.9.100]
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=[5]:[0]:[32]:[192.168.9.100], selected=0x5617c8c6de40
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=l2vpn, safi=evpn, p=[5]:[0]:[32]:[192.168.9.101]
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=[5]:[0]:[32]:[192.168.9.101], selected=0x5617c8d0f380
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=l2vpn, safi=evpn, p=[5]:[0]:[32]:[192.168.9.102]
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=[5]:[0]:[32]:[192.168.9.102], selected=0x5617c8c6f4e0
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=l2vpn, safi=evpn, p=[5]:[0]:[32]:[192.168.9.103]
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=[5]:[0]:[32]:[192.168.9.103], selected=0x5617c8c6f820
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=l2vpn, safi=evpn, p=[5]:[0]:[32]:[192.168.9.104]
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=[5]:[0]:[32]:[192.168.9.104], selected=0x5617c8c6d700
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=l2vpn, safi=evpn, p=[5]:[0]:[32]:[192.168.9.105]
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=[5]:[0]:[32]:[192.168.9.105], selected=0x5617c8d07560
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=l2vpn, safi=evpn, p=[5]:[0]:[32]:[192.168.9.106]
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=[5]:[0]:[32]:[192.168.9.106], selected=0x5617c8d07810
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=l2vpn, safi=evpn, p=[5]:[0]:[32]:[192.168.9.107]
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=[5]:[0]:[32]:[192.168.9.107], selected=0x5617c8c65f50
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=l2vpn, safi=evpn, p=[5]:[0]:[32]:[192.168.9.108]
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=[5]:[0]:[32]:[192.168.9.108], selected=0x5617c8c66200
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv6, safi=unicast, p=2001:db8:1::1/128
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=2001:db8:1::1/128, selected=0x5617c8d08900
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.108/32
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.108/32, selected=0x5617c8d08af0
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.107/32
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.107/32, selected=0x5617c8d08d70
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.106/32
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.106/32, selected=0x5617c8c6cde0
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.105/32
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.105/32, selected=0x5617c8c6d150
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.104/32
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.104/32, selected=0x5617c8c6c590
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.103/32
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.103/32, selected=0x5617c8c6c870
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.102/32
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.102/32, selected=0x5617c8c6cb50
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.101/32
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.101/32, selected=0x5617c8d081f0
2025/03/05 16:33:53 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.100/32
2025/03/05 16:33:53 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.100/32, selected=0x5617c8d084d0
2025/03/05 16:33:53 BGP: [TN0HX-6G1RR] u7:s7 send UPDATE w/ attr: nexthop 192.168.9.21, origin ?, mp_nexthop ::, extcommunity RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb, path 65002
2025/03/05 16:33:53 BGP: [H06SA-0JAPR] u7:s7 send MP_REACH for afi/safi IPv4/unicast
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u7:s7 send UPDATE 192.168.9.108/32 IPv4 unicast
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u7:s7 send UPDATE 192.168.9.107/32 IPv4 unicast
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u7:s7 send UPDATE 192.168.9.106/32 IPv4 unicast
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u7:s7 send UPDATE 192.168.9.105/32 IPv4 unicast
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u7:s7 send UPDATE 192.168.9.104/32 IPv4 unicast
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u7:s7 send UPDATE 192.168.9.103/32 IPv4 unicast
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u7:s7 send UPDATE 192.168.9.102/32 IPv4 unicast
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u7:s7 send UPDATE 192.168.9.101/32 IPv4 unicast
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u7:s7 send UPDATE 192.168.9.100/32 IPv4 unicast
2025/03/05 16:33:53 BGP: [WEV7K-2GAQ5] u7:s7 send UPDATE len 138 (max message len: 65535) numpfx 9
2025/03/05 16:33:53 BGP: [NNG4V-WFG7M] u7:s7 2001:db8::2 send UPDATE w/ mp_nexthop 2001:db8:1::1
2025/03/05 16:33:53 BGP: [TN0HX-6G1RR] u8:s8 send UPDATE w/ attr: , origin ?, mp_nexthop ::, path 65002
2025/03/05 16:33:53 BGP: [H06SA-0JAPR] u8:s8 send MP_REACH for afi/safi IPv6/unicast
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u8:s8 send UPDATE 2001:db8:1::1/128 IPv6 unicast
2025/03/05 16:33:53 BGP: [WEV7K-2GAQ5] u8:s8 send UPDATE len 83 (max message len: 65535) numpfx 1
2025/03/05 16:33:53 BGP: [NNG4V-WFG7M] u8:s8 2001:db8::2 send UPDATE w/ mp_nexthop 2001:db8:1::1
2025/03/05 16:33:53 BGP: [TN0HX-6G1RR] u9:s9 send UPDATE w/ attr: nexthop 192.168.9.21, extcommunity RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb, path 65002
2025/03/05 16:33:53 BGP: [H06SA-0JAPR] u9:s9 send MP_REACH for afi/safi l2vpn/evpn
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u9:s9 send UPDATE RD 65002:3 [5]:[0]:[0]:[0.0.0.0] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u9:s9 send UPDATE RD 65002:3 [5]:[0]:[32]:[192.168.9.100] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u9:s9 send UPDATE RD 65002:3 [5]:[0]:[32]:[192.168.9.101] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u9:s9 send UPDATE RD 65002:3 [5]:[0]:[32]:[192.168.9.102] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u9:s9 send UPDATE RD 65002:3 [5]:[0]:[32]:[192.168.9.103] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u9:s9 send UPDATE RD 65002:3 [5]:[0]:[32]:[192.168.9.104] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u9:s9 send UPDATE RD 65002:3 [5]:[0]:[32]:[192.168.9.105] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u9:s9 send UPDATE RD 65002:3 [5]:[0]:[32]:[192.168.9.106] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u9:s9 send UPDATE RD 65002:3 [5]:[0]:[32]:[192.168.9.107] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [HVRWP-5R9NQ] u9:s9 send UPDATE RD 65002:3 [5]:[0]:[32]:[192.168.9.108] label 3 l2vpn evpn
2025/03/05 16:33:53 BGP: [WEV7K-2GAQ5] u9:s9 send UPDATE len 441 (max message len: 65535) numpfx 10
2025/03/05 16:33:53 BGP: [MBFVT-8GSC6] u9:s9 2001:db8::2 send UPDATE w/ nexthop 192.168.9.21

Output of `show bgp ipv4 detail` after clear:

root@5ce23506af95:~# vtysh -c 'clear bgp 2001:db8::2'
root@5ce23506af95:~# vtysh -c 'show bgp ipv4 detail'
BGP table version is 48, local router ID is 192.168.9.5, vrf id 0
Default local pref 100, local AS 65001
BGP routing table entry for 0.0.0.0/0, version 30
Paths: (2 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Local
    192.168.9.1(5ce23506af95) from 0.0.0.0 (192.168.9.5)
      Origin incomplete, metric 0, weight 32768, valid, sourced, bestpath-from-AS Local, best (Weight)
      Last update: Wed Mar  5 16:18:17 2025
BGP routing table entry for 0.0.0.0/0, version 30
Paths: (2 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[0]:[0.0.0.0], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:33:53 2025
BGP routing table entry for 192.168.9.100/32, version 48
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.100], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:33:53 2025
BGP routing table entry for 192.168.9.101/32, version 47
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.101], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:33:53 2025
BGP routing table entry for 192.168.9.102/32, version 46
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.102], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:33:53 2025
BGP routing table entry for 192.168.9.103/32, version 45
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.103], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:33:53 2025
BGP routing table entry for 192.168.9.104/32, version 44
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.104], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:33:53 2025
BGP routing table entry for 192.168.9.105/32, version 43
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.105], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:33:53 2025
BGP routing table entry for 192.168.9.106/32, version 42
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.106], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:33:53 2025
BGP routing table entry for 192.168.9.107/32, version 41
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.107], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:33:53 2025
BGP routing table entry for 192.168.9.108/32, version 40
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.108], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:33:53 2025

Displayed 10 routes and 11 total paths

2nd scenario

Just as an additional idea, here's the output of show bgp ipv4 detail before clearing, then clearing, then output of the same command right after clearing then a few seconds later:

root@5ce23506af95:~# rtmon -family inet route file $(hostname -s)_rtmon.log &
[1] 91
root@5ce23506af95:~# vtysh -c 'show bgp ipv4 detail'
BGP table version is 10, local router ID is 192.168.9.5, vrf id 0
Default local pref 100, local AS 65001
BGP routing table entry for 0.0.0.0/0, version 1
Paths: (2 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Local
    192.168.9.1(5ce23506af95) from 0.0.0.0 (192.168.9.5)
      Origin incomplete, metric 0, weight 32768, valid, sourced, bestpath-from-AS Local, best (Weight)
      Last update: Wed Mar  5 16:18:16 2025
BGP routing table entry for 0.0.0.0/0, version 1
Paths: (2 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[0]:[0.0.0.0], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:18:28 2025
BGP routing table entry for 192.168.9.100/32, version 10
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.100], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:18:28 2025
BGP routing table entry for 192.168.9.101/32, version 9
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.101], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:18:28 2025
BGP routing table entry for 192.168.9.102/32, version 8
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.102], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:18:28 2025
BGP routing table entry for 192.168.9.103/32, version 7
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.103], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:18:28 2025
BGP routing table entry for 192.168.9.104/32, version 6
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.104], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:18:28 2025
BGP routing table entry for 192.168.9.105/32, version 5
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.105], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:18:28 2025
BGP routing table entry for 192.168.9.106/32, version 4
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.106], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:18:28 2025
BGP routing table entry for 192.168.9.107/32, version 3
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.107], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:18:28 2025
BGP routing table entry for 192.168.9.108/32, version 2
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.108], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:18:28 2025

Displayed 10 routes and 11 total paths


root@5ce23506af95:~# vtysh -c 'clear bgp 2001:db8::2'
root@5ce23506af95:~# vtysh -c 'clear bgp 2001:db8::2'


root@5ce23506af95:~# vtysh -c 'show bgp ipv4 detail'
BGP table version is 20, local router ID is 192.168.9.5, vrf id 0
Default local pref 100, local AS 65001
BGP routing table entry for 0.0.0.0/0, version 11
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Local
    192.168.9.1(5ce23506af95) from 0.0.0.0 (192.168.9.5)
      Origin incomplete, metric 0, weight 32768, valid, sourced, bestpath-from-AS Local, best (First path received)
      Last update: Wed Mar  5 16:18:16 2025

Displayed 1 routes and 1 total paths
root@5ce23506af95:~# ip monitor fil $(hostname -s)_rtmon.log
Timestamp: Wed Mar  5 16:28:45 2025 197568 us
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: vrfv3: <NOARP,MASTER,UP,LOWER_UP> mtu 65575 qdisc noqueue state UP group default
    link/ether da:10:b7:21:78:7e brd ff:ff:ff:ff:ff:ff
3: brv3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9060 qdisc noqueue master vrfv3 state UP group default
    link/ether 5a:a5:30:d5:ba:03 brd ff:ff:ff:ff:ff:ff
4: vxlan3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9060 qdisc noqueue master brv3 state UNKNOWN group default
    link/ether 5a:a5:30:d5:ba:03 brd ff:ff:ff:ff:ff:ff
27841: eth0@if27842: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:c0:a8:09:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
Timestamp: Wed Mar  5 16:29:23 2025 704536 us
Deleted 192.168.9.100 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:23 2025 704576 us
Deleted 192.168.9.101 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:23 2025 704580 us
Deleted 192.168.9.102 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:23 2025 704583 us
Deleted 192.168.9.103 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:23 2025 704599 us
Deleted 192.168.9.104 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:23 2025 704602 us
Deleted 192.168.9.105 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:23 2025 704606 us
Deleted 192.168.9.106 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:23 2025 704609 us
Deleted 192.168.9.107 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:23 2025 704612 us
Deleted 192.168.9.108 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:26 2025 68887 us
192.168.9.108 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:26 2025 68938 us
192.168.9.107 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:26 2025 68960 us
192.168.9.106 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:26 2025 68982 us
192.168.9.105 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:26 2025 69004 us
192.168.9.104 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:26 2025 69027 us
192.168.9.103 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:26 2025 69048 us
192.168.9.102 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:26 2025 69069 us
192.168.9.101 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink
Timestamp: Wed Mar  5 16:29:26 2025 69090 us
192.168.9.100 nhid 23 via 192.168.9.21 dev brv3 proto bgp metric 20 onlink


## a few seconds later after the routes were installed

root@5ce23506af95:~# vtysh -c 'show bgp ipv4 detail'
BGP table version is 29, local router ID is 192.168.9.5, vrf id 0
Default local pref 100, local AS 65001
BGP routing table entry for 0.0.0.0/0, version 11
Paths: (2 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Local
    192.168.9.1(5ce23506af95) from 0.0.0.0 (192.168.9.5)
      Origin incomplete, metric 0, weight 32768, valid, sourced, bestpath-from-AS Local, best (Weight)
      Last update: Wed Mar  5 16:18:16 2025
BGP routing table entry for 0.0.0.0/0, version 11
Paths: (2 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[0]:[0.0.0.0], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:29:25 2025
BGP routing table entry for 192.168.9.100/32, version 29
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.100], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:29:25 2025
BGP routing table entry for 192.168.9.101/32, version 28
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.101], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:29:25 2025
BGP routing table entry for 192.168.9.102/32, version 27
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.102], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:29:25 2025
BGP routing table entry for 192.168.9.103/32, version 26
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.103], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:29:25 2025
BGP routing table entry for 192.168.9.104/32, version 25
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.104], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:29:25 2025
BGP routing table entry for 192.168.9.105/32, version 24
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.105], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:29:25 2025
BGP routing table entry for 192.168.9.106/32, version 23
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.106], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:29:25 2025
BGP routing table entry for 192.168.9.107/32, version 22
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.107], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:29:25 2025
BGP routing table entry for 192.168.9.108/32, version 21
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  5049e83d99a7(2001:db8::2)
  Imported from 65002:3:[5]:[0]:[32]:[192.168.9.108], VNI 3  65002
    192.168.9.21(5049e83d99a7) from 5049e83d99a7(2001:db8::2) (192.168.9.2) announce-nh-self
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Extended Community: RT:65002:3 ET:8 Rmac:a2:1b:9b:5c:06:cb
      Last update: Wed Mar  5 16:29:25 2025

Displayed 10 routes and 11 total paths

So for a few seconds those bgp routes aren't in the rib, they aren't retained, only after they get installed they are there again.

ton31337 · 2025-03-05T19:47:19Z

So.. is this happening only for evpn imported routes?

dawkopagh · 2025-03-10T11:03:53Z

The same happens for ipv4 unicast (I assume you are asking because evpn imported routes are expected to be withdrawn due to this snippet?)

Tested on master:

a908e04dcc09# show ver FRRouting 10.4-dev (a908e04dcc09) on Linux(5.15.0-127-generic).

show bgp ipv4 detail (before clear):

BGP table version is 71, local router ID is 192.168.9.5, vrf id 0
Default local pref 100, local AS 65001
BGP routing table entry for 0.0.0.0/0, version 1
Paths: (2 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  Local
    192.168.9.1(a908e04dcc09) from 0.0.0.0 (192.168.9.5)
      Origin incomplete, metric 0, weight 32768, valid, sourced, bestpath-from-AS Local, best (Weight)
      Last update: Mon Mar 10 08:09:02 2025
BGP routing table entry for 0.0.0.0/0, version 1
Paths: (2 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002
      Last update: Mon Mar 10 10:41:02 2025
BGP routing table entry for 192.168.9.0/24, version 62
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:41:02 2025
BGP routing table entry for 192.168.9.100/32, version 63
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:41:02 2025
BGP routing table entry for 192.168.9.101/32, version 64
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:41:02 2025
BGP routing table entry for 192.168.9.102/32, version 65
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:41:02 2025
BGP routing table entry for 192.168.9.103/32, version 66
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:41:02 2025
BGP routing table entry for 192.168.9.104/32, version 67
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:41:02 2025
BGP routing table entry for 192.168.9.105/32, version 68
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:41:02 2025
BGP routing table entry for 192.168.9.106/32, version 69
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:41:02 2025
BGP routing table entry for 192.168.9.107/32, version 70
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:41:02 2025
BGP routing table entry for 192.168.9.108/32, version 71
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:41:02 2025

Displayed 11 routes and 12 total paths```

## show bgp ipv4 detail (after clear)
```root@a908e04dcc09:~# vtysh -c 'show bgp ipv4 detail'
BGP table version is 91, local router ID is 192.168.9.5, vrf id 0
Default local pref 100, local AS 65001
BGP routing table entry for 0.0.0.0/0, version 1
Paths: (2 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  Local
    192.168.9.1(a908e04dcc09) from 0.0.0.0 (192.168.9.5)
      Origin incomplete, metric 0, weight 32768, valid, sourced, bestpath-from-AS Local, best (Weight)
      Last update: Mon Mar 10 08:09:01 2025
BGP routing table entry for 0.0.0.0/0, version 1
Paths: (2 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002
      Last update: Mon Mar 10 10:51:04 2025
BGP routing table entry for 192.168.9.0/24, version 82
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:51:04 2025
BGP routing table entry for 192.168.9.100/32, version 83
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:51:04 2025
BGP routing table entry for 192.168.9.101/32, version 84
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:51:04 2025
BGP routing table entry for 192.168.9.102/32, version 85
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:51:04 2025
BGP routing table entry for 192.168.9.103/32, version 86
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:51:04 2025
BGP routing table entry for 192.168.9.104/32, version 87
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:51:04 2025
BGP routing table entry for 192.168.9.105/32, version 88
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:51:04 2025
BGP routing table entry for 192.168.9.106/32, version 89
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:51:04 2025
BGP routing table entry for 192.168.9.107/32, version 90
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:51:04 2025
BGP routing table entry for 192.168.9.108/32, version 91
Paths: (1 available, best #1, table default)
  Advertised to non peer-group peers:
  1686b70c47be(2001:db8::2)
  65002
    2001:db8::2(1686b70c47be) from 1686b70c47be(2001:db8::2) (192.168.9.2)
      Origin incomplete, metric 0, valid, external, bestpath-from-AS 65002, best (First path received)
      Last update: Mon Mar 10 10:51:04 2025

Displayed 11 routes and 12 total paths

logs:

2025/03/10 10:51:02 BGP: [H461A-BCRD2] 2001:db8::2(1686b70c47be): Peer flags updated to 0x23003412420, GR flags 0x6, GR mode PEER_GLOBAL_INHERIT
2025/03/10 10:51:02 BGP: [H7QV4-WR3ZD] %NOTIFICATION: sent to neighbor 2001:db8::2 6/4 (Cease/Administrative Reset) ""
2025/03/10 10:51:02 BGP: [H461A-BCRD2] 2001:db8::2(1686b70c47be): Peer flags updated to 0x23003412420, GR flags 0x6, GR mode PEER_GLOBAL_INHERIT
2025/03/10 10:51:02 BGP: [YMS6M-YHVP0] 2001:db8::2 [FSM] BGP_Stop (Established->Clearing), fd 23 for Outgoing
2025/03/10 10:51:02 BGP: [PXVXG-TFNNT] %ADJCHANGE: neighbor 2001:db8::2(1686b70c47be) in vrf default Down User reset
2025/03/10 10:51:02 BGP: [V4R0W-D4WGF] 2001:db8::2(Unknown) Update Group Hash: sort: 2 sub_sort: 0 UpdGrpFlags: 0 UpdGrpAFFlags: 16777219
2025/03/10 10:51:02 BGP: [NVVBY-K8MCE] 2001:db8::2(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 18432 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2025/03/10 10:51:02 BGP: [X4CQ0-63QKB] 2001:db8::2(Unknown) Update Group Hash: addpath paths-limit: (send 0, receive 0)
2025/03/10 10:51:02 BGP: [XHN6Y-FHQSC] 2001:db8::2(Unknown) Update Group Hash: Link-Local Next Hop capability: advertised
2025/03/10 10:51:02 BGP: [Z8Q37-65KK3] 2001:db8::2(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: rs rmap out: (NONE)
2025/03/10 10:51:02 BGP: [SM2F3-HRYKP] 2001:db8::2(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2025/03/10 10:51:02 BGP: [V8B3M-T6VFC] 2001:db8::2(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 0
2025/03/10 10:51:02 BGP: [Y5EX3-GHT5V] 2001:db8::2(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2025/03/10 10:51:02 BGP: [X19K7-9V4K2] 2001:db8::2(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2025/03/10 10:51:02 BGP: [JW3N0-RCE28] 2001:db8::2(Unknown) Update Group Hash: IPv6 nexthop-local unchanged: 0 IPv6 global 2001:db8:1::1
2025/03/10 10:51:02 BGP: [SQ314-QBJCR] 2001:db8::2(Unknown) Update Group Hash key: 3383409095
2025/03/10 10:51:02 BGP: [V4R0W-D4WGF] 2001:db8::2(Unknown) Update Group Hash: sort: 2 sub_sort: 0 UpdGrpFlags: 0 UpdGrpAFFlags: 16777219
2025/03/10 10:51:02 BGP: [NVVBY-K8MCE] 2001:db8::2(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 2048 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2025/03/10 10:51:02 BGP: [X4CQ0-63QKB] 2001:db8::2(Unknown) Update Group Hash: addpath paths-limit: (send 0, receive 0)
2025/03/10 10:51:02 BGP: [XHN6Y-FHQSC] 2001:db8::2(Unknown) Update Group Hash: Link-Local Next Hop capability: advertised
2025/03/10 10:51:02 BGP: [Z8Q37-65KK3] 2001:db8::2(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: rs rmap out: (NONE)
2025/03/10 10:51:02 BGP: [SM2F3-HRYKP] 2001:db8::2(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2025/03/10 10:51:02 BGP: [V8B3M-T6VFC] 2001:db8::2(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 1
2025/03/10 10:51:02 BGP: [Y5EX3-GHT5V] 2001:db8::2(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2025/03/10 10:51:02 BGP: [X19K7-9V4K2] 2001:db8::2(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2025/03/10 10:51:02 BGP: [JW3N0-RCE28] 2001:db8::2(Unknown) Update Group Hash: IPv6 nexthop-local unchanged: 0 IPv6 global 2001:db8:1::1
2025/03/10 10:51:02 BGP: [SQ314-QBJCR] 2001:db8::2(Unknown) Update Group Hash key: 2685669171
2025/03/10 10:51:02 BGP: [Y3C15-ECQV0] bgp_fsm_change_status : vrf default(0), Status: Clearing established_peers 0 for Unknown
2025/03/10 10:51:02 BGP: [ZZPVC-3AHMF] 2001:db8::2 fd -1 went from Established to Clearing for Unknown
2025/03/10 10:51:02 BGP: [YMS6M-YHVP0] 2001:db8::2 [FSM] Clearing_Completed (Clearing->Idle), fd -1 for Unknown
2025/03/10 10:51:02 BGP: [Y3C15-ECQV0] bgp_fsm_change_status : vrf default(0), Status: Idle established_peers 0 for Unknown
2025/03/10 10:51:02 BGP: [ZZPVC-3AHMF] 2001:db8::2 fd -1 went from Clearing to Idle for Unknown
2025/03/10 10:51:04 BGP: [SEJV2-PXFYK] 2001:db8::2 [FSM] Timer (start timer expire for Unknown).
2025/03/10 10:51:04 BGP: [YMS6M-YHVP0] 2001:db8::2 [FSM] BGP_Start (Idle->Connect), fd -1 for Unknown
2025/03/10 10:51:04 BGP: [Z195V-FNKRK] 2001:db8::2 [Event] Connect start to 2001:db8::2 fd 23
2025/03/10 10:51:04 BGP: [GNSWT-VK6XB] 2001:db8::2 [FSM] Non blocking connect waiting result, fd 23 for Outgoing
2025/03/10 10:51:04 BGP: [Y3C15-ECQV0] bgp_fsm_change_status : vrf default(0), Status: Connect established_peers 0 for Outgoing
2025/03/10 10:51:04 BGP: [ZZPVC-3AHMF] 2001:db8::2 fd 23 went from Idle to Connect for Outgoing
2025/03/10 10:51:04 BGP: [YMS6M-YHVP0] 2001:db8::2 [FSM] TCP_connection_open (Connect->OpenSent), fd 23 for Outgoing
2025/03/10 10:51:04 BGP: [ZR8QK-NW86V] 2001:db8::2 open active, local address 2001:db8:1::1 for Outgoing
2025/03/10 10:51:04 BGP: [XKJ09-9VTZ7] 2001:db8::2 Sending hostname cap with hn = a908e04dcc09, dn = (null)
2025/03/10 10:51:04 BGP: [HVWXV-RN5J7] 2001:db8::2: Sending GR Capability, Restart time 30 R-bit NOT-SET, N-bit SET
2025/03/10 10:51:04 BGP: [P927W-B5CB8] ... F-bit SET for IPv4 Unicast
2025/03/10 10:51:04 BGP: [P927W-B5CB8] ... F-bit SET for IPv6 Unicast
2025/03/10 10:51:04 BGP: [YJSKD-N5GGC] 2001:db8::2 Sending Software Version cap, value: FRRouting/10.4-dev
2025/03/10 10:51:04 BGP: [G1CME-TC11B] 2001:db8::2(1686b70c47be) fd 23 sending OPEN, version 4, my as 65001, holdtime 9, id 192.168.9.5
2025/03/10 10:51:04 BGP: [Y3C15-ECQV0] bgp_fsm_change_status : vrf default(0), Status: OpenSent established_peers 0 for Outgoing
2025/03/10 10:51:04 BGP: [ZZPVC-3AHMF] 2001:db8::2 fd 23 went from Connect to OpenSent for Outgoing
2025/03/10 10:51:04 BGP: [WNM1E-D314G] 2001:db8::2 rcv OPEN, version 4, remote-as (in open) 65002, holdtime 180, id 192.168.9.2
2025/03/10 10:51:04 BGP: [QG29C-5TSVS] 2001:db8::2 rcv OPEN w/ OPTION parameter len: 122
2025/03/10 10:51:04 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2025/03/10 10:51:04 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has MultiProtocol Extensions capability (1), length 4
2025/03/10 10:51:04 BGP: [VXVH9-3MXR0] 2001:db8::2 OPEN has MultiProtocol Extensions capability for afi/safi: IPv4/unicast
2025/03/10 10:51:04 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 8
2025/03/10 10:51:04 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has Extended Next Hop Encoding capability (5), length 6
2025/03/10 10:51:04 BGP: [XCS8V-P66M1] 2001:db8::2 Received with afi/safi/next-hop afi: IPv4/unicast/2
2025/03/10 10:51:04 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2025/03/10 10:51:04 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has MultiProtocol Extensions capability (1), length 4
2025/03/10 10:51:04 BGP: [VXVH9-3MXR0] 2001:db8::2 OPEN has MultiProtocol Extensions capability for afi/safi: IPv6/unicast
2025/03/10 10:51:04 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2025/03/10 10:51:04 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has Route Refresh capability (2), length 0
2025/03/10 10:51:04 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2025/03/10 10:51:04 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has Enhanced Route Refresh capability (70), length 0
2025/03/10 10:51:04 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2025/03/10 10:51:04 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has 4-octet AS number capability (65), length 4
2025/03/10 10:51:04 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2025/03/10 10:51:04 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has BGP Extended Message capability (6), length 0
2025/03/10 10:51:04 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 10
2025/03/10 10:51:04 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has AddPath capability (69), length 8
2025/03/10 10:51:04 BGP: [SBSKM-G6QBW] 2001:db8::2 OPEN has AddPath capability for afi/safi: IPv4/unicast, receive
2025/03/10 10:51:04 BGP: [SBSKM-G6QBW] 2001:db8::2 OPEN has AddPath capability for afi/safi: IPv6/unicast, receive
2025/03/10 10:51:04 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 12
2025/03/10 10:51:04 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has Paths-Limit capability (76), length 10
2025/03/10 10:51:04 BGP: [MSJ7S-E9MRG] 2001:db8::2 OPEN has Paths-Limit capability for afi/safi: IPv4/unicast limit: 0
2025/03/10 10:51:04 BGP: [MSJ7S-E9MRG] 2001:db8::2 OPEN has Paths-Limit capability for afi/safi: IPv6/unicast limit: 0
2025/03/10 10:51:04 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 16
2025/03/10 10:51:04 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has FQDN capability (73), length 14
2025/03/10 10:51:04 BGP: [MR6EG-NS6N7] 2001:db8::2 received hostname 1686b70c47be, domainname (null)
2025/03/10 10:51:04 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 12
2025/03/10 10:51:04 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has Graceful Restart capability (64), length 10
2025/03/10 10:51:04 BGP: [V30NV-6FS3W] 2001:db8::2(1686b70c47be) OPEN has GR capability, Restart time 30 R-bit NOT-SET N-bit SET
2025/03/10 10:51:04 BGP: [W0E2T-VG0J8] 2001:db8::2(1686b70c47be) F-bit NOT-SET for IPv4 Unicast
2025/03/10 10:51:04 BGP: [W0E2T-VG0J8] 2001:db8::2(1686b70c47be) F-bit NOT-SET for IPv6 Unicast
2025/03/10 10:51:04 BGP: [NVZPF-5ST3B] 2001:db8::2 rcvd OPEN w/ optional parameter type 2 (Capability) len 16
2025/03/10 10:51:04 BGP: [SCW43-WN4M1] 2001:db8::2 OPEN has Long-lived BGP Graceful Restart capability (71), length 14
2025/03/10 10:51:04 BGP: [SGV3H-AGFR8] 2001:db8::2 Addr-family IPv4/unicast(afi/safi) Long-lived Graceful Restart capability stale time 0 sec
2025/03/10 10:51:04 BGP: [SGV3H-AGFR8] 2001:db8::2 Addr-family IPv6/unicast(afi/safi) Long-lived Graceful Restart capability stale time 0 sec
2025/03/10 10:51:04 BGP: [YMS6M-YHVP0] 2001:db8::2 [FSM] Receive_OPEN_message (OpenSent->OpenConfirm), fd 23 for Outgoing
2025/03/10 10:51:04 BGP: [Y3C15-ECQV0] bgp_fsm_change_status : vrf default(0), Status: OpenConfirm established_peers 0 for Outgoing
2025/03/10 10:51:04 BGP: [ZZPVC-3AHMF] 2001:db8::2 fd 23 went from OpenSent to OpenConfirm for Outgoing
2025/03/10 10:51:04 BGP: [YMS6M-YHVP0] 2001:db8::2 [FSM] Receive_KEEPALIVE_message (OpenConfirm->Established), fd 23 for Outgoing
2025/03/10 10:51:04 BGP: [Y3C15-ECQV0] bgp_fsm_change_status : vrf default(0), Status: Established established_peers 1 for Outgoing
2025/03/10 10:51:04 BGP: [ZZPVC-3AHMF] 2001:db8::2 fd 23 went from OpenConfirm to Established for Outgoing
2025/03/10 10:51:04 BGP: [N9HHH-F8H1M] %ADJCHANGE: neighbor 2001:db8::2(1686b70c47be) in vrf default Up
2025/03/10 10:51:04 BGP: [V4R0W-D4WGF] 2001:db8::2(Unknown) Update Group Hash: sort: 2 sub_sort: 0 UpdGrpFlags: 0 UpdGrpAFFlags: 16777219
2025/03/10 10:51:04 BGP: [NVVBY-K8MCE] 2001:db8::2(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 18432 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2025/03/10 10:51:04 BGP: [X4CQ0-63QKB] 2001:db8::2(Unknown) Update Group Hash: addpath paths-limit: (send 0, receive 0)
2025/03/10 10:51:04 BGP: [XHN6Y-FHQSC] 2001:db8::2(Unknown) Update Group Hash: Link-Local Next Hop capability: advertised
2025/03/10 10:51:04 BGP: [Z8Q37-65KK3] 2001:db8::2(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: rs rmap out: (NONE)
2025/03/10 10:51:04 BGP: [SM2F3-HRYKP] 2001:db8::2(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2025/03/10 10:51:04 BGP: [V8B3M-T6VFC] 2001:db8::2(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 0
2025/03/10 10:51:04 BGP: [Y5EX3-GHT5V] 2001:db8::2(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2025/03/10 10:51:04 BGP: [X19K7-9V4K2] 2001:db8::2(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2025/03/10 10:51:04 BGP: [JW3N0-RCE28] 2001:db8::2(Unknown) Update Group Hash: IPv6 nexthop-local unchanged: 0 IPv6 global 2001:db8:1::1
2025/03/10 10:51:04 BGP: [SQ314-QBJCR] 2001:db8::2(Unknown) Update Group Hash key: 3383409095
2025/03/10 10:51:04 BGP: [V4R0W-D4WGF] 2001:db8::2(Unknown) Update Group Hash: sort: 2 sub_sort: 0 UpdGrpFlags: 0 UpdGrpAFFlags: 16777219
2025/03/10 10:51:04 BGP: [NVVBY-K8MCE] 2001:db8::2(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 2048 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2025/03/10 10:51:04 BGP: [X4CQ0-63QKB] 2001:db8::2(Unknown) Update Group Hash: addpath paths-limit: (send 0, receive 0)
2025/03/10 10:51:04 BGP: [XHN6Y-FHQSC] 2001:db8::2(Unknown) Update Group Hash: Link-Local Next Hop capability: advertised
2025/03/10 10:51:04 BGP: [Z8Q37-65KK3] 2001:db8::2(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: rs rmap out: (NONE)
2025/03/10 10:51:04 BGP: [SM2F3-HRYKP] 2001:db8::2(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2025/03/10 10:51:04 BGP: [V8B3M-T6VFC] 2001:db8::2(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 1
2025/03/10 10:51:04 BGP: [Y5EX3-GHT5V] 2001:db8::2(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2025/03/10 10:51:04 BGP: [X19K7-9V4K2] 2001:db8::2(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2025/03/10 10:51:04 BGP: [JW3N0-RCE28] 2001:db8::2(Unknown) Update Group Hash: IPv6 nexthop-local unchanged: 0 IPv6 global 2001:db8:1::1
2025/03/10 10:51:04 BGP: [SQ314-QBJCR] 2001:db8::2(Unknown) Update Group Hash key: 2685669171
2025/03/10 10:51:04 BGP: [GB3VH-EAZK8] 2001:db8::2 [FSM] Timer (routeadv timer expire for Outgoing)
2025/03/10 10:51:05 BGP: [K9G3A-CJ5QQ] u11:s11 announcing routes upon coalesce timer expiry(1000 ms)
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=0.0.0.0/0, selected=0x56359f32c290
2025/03/10 10:51:05 BGP: [K9G3A-CJ5QQ] u12:s12 announcing routes upon coalesce timer expiry(1000 ms)
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=::/0, selected=0x56359f30ea20
2025/03/10 10:51:05 BGP: [GB3VH-EAZK8] 2001:db8::2 [FSM] Timer (routeadv timer expire for Outgoing)
2025/03/10 10:51:05 BGP: [TN0HX-6G1RR] u11:s11 send UPDATE w/ attr: nexthop 192.168.9.1, origin ?, mp_nexthop ::, metric 0, path
2025/03/10 10:51:05 BGP: [H06SA-0JAPR] u11:s11 send MP_REACH for afi/safi IPv4/unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u11:s11 send UPDATE 0.0.0.0/0 IPv4 unicast
2025/03/10 10:51:05 BGP: [WEV7K-2GAQ5] u11:s11 send UPDATE len 70 (max message len: 65535) numpfx 1
2025/03/10 10:51:05 BGP: [NNG4V-WFG7M] u11:s11 2001:db8::2 send UPDATE w/ mp_nexthop 2001:db8:1::1
2025/03/10 10:51:05 BGP: [TN0HX-6G1RR] u12:s12 send UPDATE w/ attr: , origin ?, mp_nexthop ::, metric 1024, path
2025/03/10 10:51:05 BGP: [H06SA-0JAPR] u12:s12 send MP_REACH for afi/safi IPv6/unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u12:s12 send UPDATE ::/0 IPv6 unicast
2025/03/10 10:51:05 BGP: [WEV7K-2GAQ5] u12:s12 send UPDATE len 70 (max message len: 65535) numpfx 1
2025/03/10 10:51:05 BGP: [NNG4V-WFG7M] u12:s12 2001:db8::2 send UPDATE w/ mp_nexthop 2001:db8:1::1
2025/03/10 10:51:05 BGP: [ZP3RE-J4Q8C] send End-of-RIB for IPv4 Unicast to 2001:db8::2
2025/03/10 10:51:05 BGP: [ZP3RE-J4Q8C] send End-of-RIB for IPv6 Unicast to 2001:db8::2
2025/03/10 10:51:05 BGP: [XXWBM-V772F] 2001:db8::2(1686b70c47be) rcvd UPDATE w/ attr: , origin ?, mp_nexthop 2001:db8::2, metric 0, path 65002
2025/03/10 10:51:05 BGP: [YCKEM-GB33T] 2001:db8::2(1686b70c47be) rcvd 0.0.0.0/0 IPv4 unicast
2025/03/10 10:51:05 BGP: [XXWBM-V772F] 2001:db8::2(1686b70c47be) rcvd UPDATE w/ attr: , origin ?, mp_nexthop 2001:db8::2, path 65002 65001
2025/03/10 10:51:05 BGP: [RZMGQ-A03CG] 2001:db8::2(1686b70c47be) rcvd UPDATE about ::/0 IPv6 unicast -- DENIED due to: as-path contains our own AS;
2025/03/10 10:51:05 BGP: [XXWBM-V772F] 2001:db8::2(1686b70c47be) rcvd UPDATE w/ attr: , origin ?, mp_nexthop 2001:db8::2, metric 0, path 65002
2025/03/10 10:51:05 BGP: [YCKEM-GB33T] 2001:db8::2(1686b70c47be) rcvd 192.168.9.0/24 IPv4 unicast
2025/03/10 10:51:05 BGP: [XXWBM-V772F] 2001:db8::2(1686b70c47be) rcvd UPDATE w/ attr: , origin ?, mp_nexthop 2001:db8::2, metric 0, path 65002
2025/03/10 10:51:05 BGP: [YCKEM-GB33T] 2001:db8::2(1686b70c47be) rcvd 2001:db8:1::1/128 IPv6 unicast
2025/03/10 10:51:05 BGP: [XXWBM-V772F] 2001:db8::2(1686b70c47be) rcvd UPDATE w/ attr: , origin ?, mp_nexthop 2001:db8::2, metric 0, path 65002
2025/03/10 10:51:05 BGP: [YCKEM-GB33T] 2001:db8::2(1686b70c47be) rcvd 192.168.9.100/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [YCKEM-GB33T] 2001:db8::2(1686b70c47be) rcvd 192.168.9.101/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [YCKEM-GB33T] 2001:db8::2(1686b70c47be) rcvd 192.168.9.102/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [YCKEM-GB33T] 2001:db8::2(1686b70c47be) rcvd 192.168.9.103/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [YCKEM-GB33T] 2001:db8::2(1686b70c47be) rcvd 192.168.9.104/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [YCKEM-GB33T] 2001:db8::2(1686b70c47be) rcvd 192.168.9.105/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [YCKEM-GB33T] 2001:db8::2(1686b70c47be) rcvd 192.168.9.106/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [YCKEM-GB33T] 2001:db8::2(1686b70c47be) rcvd 192.168.9.107/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [YCKEM-GB33T] 2001:db8::2(1686b70c47be) rcvd 192.168.9.108/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [WT375-N4KPV] EOR REQ 0, EOR RCV 0
2025/03/10 10:51:05 BGP: [G3V0W-FSWRC] bgp_best_path_select_defer: processing route for IPv6 Unicast : cnt 0
2025/03/10 10:51:05 BGP: [M59KS-A3ZXZ] bgp_update_receive: rcvd End-of-RIB for IPv6 Unicast from 2001:db8::2 in vrf default
2025/03/10 10:51:05 BGP: [WT375-N4KPV] EOR REQ 0, EOR RCV 0
2025/03/10 10:51:05 BGP: [G3V0W-FSWRC] bgp_best_path_select_defer: processing route for IPv4 Unicast : cnt 0
2025/03/10 10:51:05 BGP: [M59KS-A3ZXZ] bgp_update_receive: rcvd End-of-RIB for IPv4 Unicast from 2001:db8::2 in vrf default
2025/03/10 10:51:05 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.0/24
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.0/24, selected=0x56359ed14120
2025/03/10 10:51:05 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv6, safi=unicast, p=2001:db8:1::1/128
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=2001:db8:1::1/128, selected=0x56359f30dad0
2025/03/10 10:51:05 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.100/32
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.100/32, selected=0x56359f30dbd0
2025/03/10 10:51:05 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.101/32
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.101/32, selected=0x56359f305c70
2025/03/10 10:51:05 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.102/32
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.102/32, selected=0x56359f30f3e0
2025/03/10 10:51:05 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.103/32
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.103/32, selected=0x56359f30d930
2025/03/10 10:51:05 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.104/32
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.104/32, selected=0x56359f305720
2025/03/10 10:51:05 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.105/32
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.105/32, selected=0x56359f30de00
2025/03/10 10:51:05 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.106/32
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.106/32, selected=0x56359f30e080
2025/03/10 10:51:05 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.107/32
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.107/32, selected=0x56359f30e390
2025/03/10 10:51:05 BGP: [Z38CW-7NYWG] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=192.168.9.108/32
2025/03/10 10:51:05 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=192.168.9.108/32, selected=0x56359f30fde0
2025/03/10 10:51:05 BGP: [TN0HX-6G1RR] u11:s11 send UPDATE w/ attr: , origin ?, mp_nexthop ::, path 65002
2025/03/10 10:51:05 BGP: [H06SA-0JAPR] u11:s11 send MP_REACH for afi/safi IPv4/unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u11:s11 send UPDATE 192.168.9.0/24 IPv4 unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u11:s11 send UPDATE 192.168.9.100/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u11:s11 send UPDATE 192.168.9.101/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u11:s11 send UPDATE 192.168.9.102/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u11:s11 send UPDATE 192.168.9.103/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u11:s11 send UPDATE 192.168.9.104/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u11:s11 send UPDATE 192.168.9.105/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u11:s11 send UPDATE 192.168.9.106/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u11:s11 send UPDATE 192.168.9.107/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u11:s11 send UPDATE 192.168.9.108/32 IPv4 unicast
2025/03/10 10:51:05 BGP: [WEV7K-2GAQ5] u11:s11 send UPDATE len 115 (max message len: 65535) numpfx 10
2025/03/10 10:51:05 BGP: [NNG4V-WFG7M] u11:s11 2001:db8::2 send UPDATE w/ mp_nexthop 2001:db8:1::1
2025/03/10 10:51:05 BGP: [TN0HX-6G1RR] u12:s12 send UPDATE w/ attr: , origin ?, mp_nexthop ::, path 65002
2025/03/10 10:51:05 BGP: [H06SA-0JAPR] u12:s12 send MP_REACH for afi/safi IPv6/unicast
2025/03/10 10:51:05 BGP: [HVRWP-5R9NQ] u12:s12 send UPDATE 2001:db8:1::1/128 IPv6 unicast
2025/03/10 10:51:05 BGP: [WEV7K-2GAQ5] u12:s12 send UPDATE len 83 (max message len: 65535) numpfx 1
2025/03/10 10:51:05 BGP: [NNG4V-WFG7M] u12:s12 2001:db8::2 send UPDATE w/ mp_nexthop 2001:db8:1::1

kernel withdrawals:

Timestamp: Mon Mar 10 10:50:59 2025 273444 us
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
5699: eth0@if5700: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:c0:a8:09:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
Timestamp: Mon Mar 10 10:51:02 2025 736462 us
Deleted 192.168.9.100 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:02 2025 736534 us
Deleted 192.168.9.101 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:02 2025 736537 us
Deleted 192.168.9.102 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:02 2025 736540 us
Deleted 192.168.9.103 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:02 2025 736543 us
Deleted 192.168.9.104 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:02 2025 736546 us
Deleted 192.168.9.105 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:02 2025 736549 us
Deleted 192.168.9.106 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:02 2025 736551 us
Deleted 192.168.9.107 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:02 2025 736554 us
Deleted 192.168.9.108 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:05 2025 738800 us
192.168.9.100 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:05 2025 738849 us
192.168.9.101 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:05 2025 738853 us
192.168.9.102 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:05 2025 738875 us
192.168.9.103 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:05 2025 738879 us
192.168.9.104 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:05 2025 738882 us
192.168.9.105 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:05 2025 738886 us
192.168.9.106 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:05 2025 738889 us
192.168.9.107 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20
Timestamp: Mon Mar 10 10:51:05 2025 738892 us
192.168.9.108 nhid 20 via inet6 2001:db8::2 dev eth0 proto bgp metric 20

negotiated graceful-restart settings between peers:


Codes: GR - Graceful Restart, * -  Inheriting Global GR Config,
       Restart - GR Mode-Restarting, Helper - GR Mode-Helper,
       Disable - GR Mode-Disable.

Global BGP GR Mode :  Restart

BGP neighbor is 2001:db8::2
    Local GR Mode: Restart*
    Remote GR Mode: Restart
    R bit: False
    N bit: True
    Timers:
      Configured Restart Time(sec): 30
      Received Restart Time(sec): 30
      Configured LLGR Stale Path Time(sec): 0
    IPv4 Unicast:
      F bit: True
      End-of-RIB sent: Yes
      End-of-RIB sent after update: Yes
      End-of-RIB received: Yes
      Timers:
        Configured Stale Path Time(sec): 15
        Configured Selection Deferral Time(sec): 360
        LLGR Stale Path Time(sec): 0
    IPv6 Unicast:
      F bit: True
      End-of-RIB sent: Yes
      End-of-RIB sent after update: Yes
      End-of-RIB received: Yes
      Timers:
        Configured Stale Path Time(sec): 15
        Configured Selection Deferral Time(sec): 360
        LLGR Stale Path Time(sec): 0

dawkopagh added the triage Needs further investigation label Feb 24, 2025

ton31337 added the bgp label Feb 24, 2025

ton31337 self-assigned this Feb 24, 2025

ton31337 mentioned this issue Feb 24, 2025

bgpd: Fix GR when using R-bit with hard reset (adminstrative reset) #18241

Closed

ton31337 removed the triage Needs further investigation label Feb 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Graceful Restart procedures do not apply when hard-administrative-reset flag is disabled #18238

Graceful Restart procedures do not apply when hard-administrative-reset flag is disabled #18238

dawkopagh commented Feb 24, 2025

ton31337 commented Feb 24, 2025 •

edited

Loading

piotrsuchy commented Feb 25, 2025

ton31337 commented Feb 25, 2025 •

edited

Loading

piotrsuchy commented Mar 3, 2025

ton31337 commented Mar 3, 2025

piotrsuchy commented Mar 3, 2025

ton31337 commented Mar 3, 2025

piotrsuchy commented Mar 5, 2025 •

edited

Loading

ton31337 commented Mar 5, 2025

dawkopagh commented Mar 10, 2025

Graceful Restart procedures do not apply when hard-administrative-reset flag is disabled #18238

Graceful Restart procedures do not apply when hard-administrative-reset flag is disabled #18238

Comments

dawkopagh commented Feb 24, 2025

Description

Version

How to reproduce

Expected behavior

Actual behavior

Additional context

Checklist

ton31337 commented Feb 24, 2025 • edited Loading

piotrsuchy commented Feb 25, 2025

ton31337 commented Feb 25, 2025 • edited Loading

piotrsuchy commented Mar 3, 2025

ton31337 commented Mar 3, 2025

piotrsuchy commented Mar 3, 2025

ton31337 commented Mar 3, 2025

piotrsuchy commented Mar 5, 2025 • edited Loading

1-st scenario logs (for 10.1.1, exactly the same thing happening for master)

With logs:

Output of show bgp ipv4 detail after clear:

2nd scenario

ton31337 commented Mar 5, 2025

dawkopagh commented Mar 10, 2025

Tested on master:

show bgp ipv4 detail (before clear):

logs:

kernel withdrawals:

negotiated graceful-restart settings between peers:

ton31337 commented Feb 24, 2025 •

edited

Loading

ton31337 commented Feb 25, 2025 •

edited

Loading

piotrsuchy commented Mar 5, 2025 •

edited

Loading

Output of `show bgp ipv4 detail` after clear: