Skip to content

Commit

Permalink
OCPBUGS-42045: Add an Internal Warmup for IBMCloud DNS in E2E
Browse files Browse the repository at this point in the history
During testing, we found that IBMCloud's DNS resolution works well from
outside the cluster (e.g., the test runner cluster). However, internal
DNS queries within the test cluster trigger to an unchangeable
~30-minute negative caching TTL.

This E2E test fix introduces an internal warmup period for IBMCloud
clusters to mitigate the negative caching issue. Only one test,
TestUnmanagedDNSToManagedDNSInternalIngressController, requires this
workaround.
  • Loading branch information
gcs278 committed Jan 29, 2025
1 parent b5af04d commit 3bfdce3
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 0 deletions.
8 changes: 8 additions & 0 deletions test/e2e/operator_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,14 @@ var (
operandNamespace = operatorcontroller.DefaultOperandNamespace
defaultName = types.NamespacedName{Namespace: operatorNamespace, Name: manifests.DefaultIngressControllerName}
clusterConfigName = types.NamespacedName{Namespace: operatorNamespace, Name: manifests.ClusterIngressConfigName}

// Platforms that need a DNS "warmup" period for internal (inside the test cluster) DNS resolution.
// The warmup period is a period of delay before the first query is executed to avoid negative caching.
// This is not intended for external (i.e. the test runner cluster) DNS resolution.
platformsNeedInternalDNSWarmup = map[configv1.PlatformType]time.Duration{
// 7 minutes of warmup was required past testing for internal IBMCloud DNS queries.
configv1.IBMCloudPlatformType: 7 * time.Minute,
}
)

const (
Expand Down
14 changes: 14 additions & 0 deletions test/e2e/util_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -766,6 +766,10 @@ func verifyInternalIngressController(t *testing.T, name types.NamespacedName, ho
}
}()

// Since we've created a new IngressController, and we are about to use a curl pod to query the
// wildcard DNS record, we need to wait for platforms that require a warmup period for internal queries.
waitForInternalDNSWarmup(t)

extraArgs := []string{
"--header", "HOST:" + echoRoute.Spec.Host,
"-v",
Expand Down Expand Up @@ -849,6 +853,16 @@ func verifyInternalIngressController(t *testing.T, name types.NamespacedName, ho
}
}

// waitForInternalDNSWarmup waits for a designated "warmup" period
// to prevent negative caching on platforms that require it.
func waitForInternalDNSWarmup(t *testing.T) {
warmup, _ := platformsNeedInternalDNSWarmup[infraConfig.Status.PlatformStatus.Type]
if warmup > 0 {
t.Logf("this platform requires DNS warmup time to avoid negative caching...waiting for %s", warmup)
time.Sleep(warmup)
}
}

// assertDeleted tries to delete a cluster resource, and causes test failure if the delete fails.
func assertDeleted(t *testing.T, cl client.Client, thing client.Object) {
t.Helper()
Expand Down

0 comments on commit 3bfdce3

Please sign in to comment.