Branch v2.21.0 auto alloc zonca

[zonca]

No description provided.

[zonca]

Networking error:

Error: Error associating openstack_networking_floatingip_associate_v2 floating_ip a6281ddb-291a-4354-b63b-dba747fe28d4
 with port 94822361-f0c1-4a62-b014-42072fb8e515: 

Resource not found: [PUT https://js2.jetstream-cloud.org:9696/v2.0/fl
oatingips/a6281ddb-291a-4354-b63b-dba747fe28d4], error message: 

{"NeutronError": {"type": "ExternalGatewayForFloatingI
PNotFound", 

"message": "External network 3fe22c05-6206-4db2-9a13-44f04b6796e6 is not reachable from subnet fe34c9c1-64
33-4c0e-9975-a4b0ddc0b23f. 
Therefore, cannot associate Port 94822361-f0c1-4a62-b014-42072fb8e515 with a Floating IP."

, "detail": ""}}                                                                                                      
                                                                                                                      
  on ../../contrib/terraform/openstack/modules/compute/main.tf line 944, in resource "openstack_networking_floatingip_
associate_v2" "k8s_master":                                                                                           
 944: resource "openstack_networking_floatingip_associate_v2" "k8s_master" {

What is strange is that this was not happening when I had just the hardcoded network id.

[zonca]

o subnet show kubejetstream-internal-network
+----------------------+--------------------------------------+
| Field                | Value                                |
+----------------------+--------------------------------------+
| allocation_pools     | 10.1.128.2-10.1.128.254              |
| cidr                 | 10.1.128.0/24                        |
| created_at           | 2023-06-15T19:35:57Z                 |
| description          |                                      |
| dns_nameservers      |                                      |
| dns_publish_fixed_ip | None                                 |
| enable_dhcp          | True                                 |
| gateway_ip           | 10.1.128.1                           |
| host_routes          |                                      |
| id                   | fe34c9c1-6433-4c0e-9975-a4b0ddc0b23f |
| ip_version           | 4                                    |
| ipv6_address_mode    | None                                 |
| ipv6_ra_mode         | None                                 |
| name                 | kubejetstream-internal-network       |
| network_id           | bdd21138-b4d2-4e2d-8114-efaf248c3722 |
| project_id           | 83bd0e70b4ba4cd092c2fb82b1ce06fb     |
| revision_number      | 0                                    |
| segment_id           | None                                 |
| service_types        |                                      |
| subnetpool_id        | be988956-1bfb-4828-b511-a58229fbd4ac |
| tags                 |                                      |
| updated_at           | 2023-06-15T19:35:57Z                 |
+----------------------+--------------------------------------+

[zonca]

If I compare it to the auto allocated subnet, it seems fine:

o subnet show auto_allocated_subnet_v4
+----------------------+--------------------------------------+
| Field                | Value                                |
+----------------------+--------------------------------------+
| allocation_pools     | 10.1.90.2-10.1.90.254                |
| cidr                 | 10.1.90.0/24                         |
| created_at           | 2023-04-04T18:50:08Z                 |
| description          |                                      |
| dns_nameservers      |                                      |
| dns_publish_fixed_ip | None                                 |
| enable_dhcp          | True                                 |
| gateway_ip           | 10.1.90.1                            |
| host_routes          |                                      |
| id                   | 48970d8d-59a1-4a99-aae9-1bfa1ce4e338 |
| ip_version           | 4                                    |
| ipv6_address_mode    | None                                 |
| ipv6_ra_mode         | None                                 |
| name                 | auto_allocated_subnet_v4             |
| network_id           | bdd21138-b4d2-4e2d-8114-efaf248c3722 |
| project_id           | 83bd0e70b4ba4cd092c2fb82b1ce06fb     |
| revision_number      | 0                                    |
| segment_id           | None                                 |
| service_types        |                                      |
| subnetpool_id        | be988956-1bfb-4828-b511-a58229fbd4ac |
| tags                 |                                      |
| updated_at           | 2023-04-04T18:50:08Z                 |
+----------------------+--------------------------------------+

@zacharygraber would you be able to help me out with this? Understand if it is not possible to use too much of your time on this.

[zacharygraber]

Oops--this may be my/our fault! I would be surprised if networking was working on any of your instances at all. We removed the external gateway on your router to delete the subnet you had before, but it looks like this left you with completely no route to the outside world:

Go ahead and try again. I re-added the gateway with os router set --external-gateway 3fe22c05-6206-4db2-9a13-44f04b6796e6 15cc195f-02d4-4d9b-ae24-5b2e9e52a13a, and the network topology looks a lot more sane now.

[zonca]

Good I asked ;) working now, thanks!

[zonca]

currently the recipes deploys fine, I can then install kubernetes with ansible and deploy jupyterhub. Networking works fine.

Now testing deleting the resources.

[zonca]

ok, so now I have a similar problem trying to delete the resources.

Terraform cannot delete the subnet ba6a036f-4724-40be-912e-7222c8dc6515

o port list | grep ba6
| 4cfaae3a-79fc-4d9b-9fd5-93e996a66e77 |                                                      | fa:16:3e:4b:9f:6a | ip_address='10.1.128.1', subnet_id='ba6a036f-4724-40be-912e-7222c8dc6515'   | ACTIVE |
| 81bf94b4-09fe-489d-bfbe-9a7b43da100b | octavia-lb-45d0453f-d458-4a6d-be3b-84bc03527853      | fa:16:3e:6d:03:6a | ip_address='10.1.128.14', subnet_id='ba6a036f-4724-40be-912e-7222c8dc6515'  | DOWN   |
| 920139b5-47d8-4525-9ccb-f71e7e14bebe |                                                      | fa:16:3e:1e:7f:18 | ip_address='10.1.128.68', subnet_id='ba6a036f-4724-40be-912e-7222c8dc6515'  | ACTIVE |
| 987be2cb-1dc6-45ae-a20c-5e570401b497 | octavia-lb-vrrp-330414e8-ae93-4d1d-a3b3-ed28ab169d2f | fa:16:3e:54:ea:46 | ip_address='10.1.128.114', subnet_id='ba6a036f-4724-40be-912e-7222c8dc6515' | ACTIVE |
| c6147875-6b77-47f5-9e7d-cf2612ac9b0d |                                                      | fa:16:3e:b2:d7:82 | ip_address='10.1.128.2', subnet_id='ba6a036f-4724-40be-912e-7222c8dc6515'   | ACTIVE |

There are now 4 ports attached to the subnet.
The problem is that Terraform is tracking only 1:

  # module.network.openstack_networking_router_interface_v2.k8s[0] will be destroyed                                                   
  - resource "openstack_networking_router_interface_v2" "k8s" {    
      - id        = "4cfaae3a-79fc-4d9b-9fd5-93e996a66e77" -> null
      - port_id   = "4cfaae3a-79fc-4d9b-9fd5-93e996a66e77" -> null
      - region    = "IU" -> null
      - router_id = "15cc195f-02d4-4d9b-ae24-5b2e9e52a13a" -> null
      - subnet_id = "ba6a036f-4724-40be-912e-7222c8dc6515" -> null
    }

It also seem some of them are created by Octavia. However I don't see in the Terraform recipe anything about load balancing, is Jetstream automatically deploying some load balancing resources?

@zacharygraber

[zonca]

If I try to delete the port managed by Openstack I get:

o port delete 4cfaae3a-79fc-4d9b-9fd5-93e996a66e77
Failed to delete port with name or ID '4cfaae3a-79fc-4d9b-9fd5-93e996a66e77': 
ConflictException: 409: Client Error for url: https://js2.jetstream-cloud.org:9696/v2.0/ports/4cfaae3a-79fc-4d9b-9fd5-93e996a66e77, 
Port 4cfaae3a-79fc-4d9b-9fd5-93e996a66e77 cannot be deleted directly via the port API: has device owner network:router_interface_distributed.
1 of 1 ports failed to delete.

[zonca]

this is different compared to zonca/jupyterhub-deploy-kubernetes-jetstream#62, there I was in a inconsistent state due to testing. Now instead I just ran my recipe cleanly, so I thought I could just easily delete the resources that Terraform created.

These problems arose when I started trying to use the auto allocated network instead of having Terraform create a dedicated network just for Kubernetes, as suggested in jetstream-cloud/js2docs#46.

[zonca]

superseeded by #25