Migrate from Ingress NGINX to Gateway API on Managed Kubernetes with Cilium

The Ingress NGINX controller is being phased out. This guide explains how to migrate step by step to the Gateway API on a managed Kubernetes cluster with Cilium while keeping workloads available throughout the process. You will learn how to manage TLS certificates, perform a safe DNS cutover, and configure the cloud provider LoadBalancer for the new Gateway. The migration approach lets Ingress and Gateway API run in parallel, so you can validate production readiness before switching DNS traffic.

If Gateway API concepts or recommended practices are new to you, review tutorials about Gateway API with Cilium and HTTPS traffic routing first. These resources help explain the new API model, common configuration patterns, and the main differences compared with the traditional Ingress model. Careful planning and thorough testing are essential for a smooth migration.

Important: This guide is tested with managed Kubernetes version 1.33.x. If your cluster runs an older release, upgrade it before starting the migration.

Key Takeaways

  • Zero downtime is possible when moving from Ingress NGINX to Gateway API on managed Kubernetes by operating both controllers side by side and completing a controlled DNS cutover.
  • LoadBalancer endpoints will change. The Gateway API provisions a new LoadBalancer with a new IP address. Plan for a short phase with two LoadBalancers and only update DNS after production traffic has been validated.
  • Annotation migration is required. Ingress NGINX and Gateway API use different configuration models. Cloud LoadBalancer annotations belong in spec.infrastructure.annotations in Gateway resources, not in metadata.annotations.
  • Certificates must be defined explicitly. Instead of relying on Ingress annotations, Gateway API requires separate Certificate resources. The cert-manager solver must be configured for Gateway API, not for Ingress.
  • HTTP-to-HTTPS redirects move to filters. Gateway API does not process NGINX redirect annotations. Use a dedicated HTTPRoute with a RequestRedirect filter.
  • Test carefully before cutover. Use the Gateway LoadBalancer IP to confirm readiness before changing DNS. Remove Ingress only after stability has been verified.
  • Plan for temporary dual resources. During testing and cutover, two LoadBalancers will run at the same time, so include this in your migration planning.

Prerequisites

  • VPC-integrated managed Kubernetes cluster version 1.33+; verify with kubectl get gatewayclass cilium and confirm ACCEPTED: True
  • kubectl configured for your cluster
  • Existing Ingress NGINX deployment with cert-manager
  • Domain name with DNS access
  • Optional: ExternalDNS installed in the external-dns namespace
  • Budget for running two LoadBalancers during the migration

Key Migration Considerations

LoadBalancer IP Changes

The Gateway creates a new cloud LoadBalancer with a new IP address.

Solution: Run both LoadBalancers at the same time, move DNS to the new Gateway LoadBalancer, and remove the old Ingress LoadBalancer only after validation.

Annotation Mapping

Ingress NGINX Gateway API Location
kubernetes.io/ingress.class: nginx spec.gatewayClassName: cilium
cert-manager.io/cluster-issuer Explicit Certificate resource
external-dns.alpha.kubernetes.io/* HTTPRoute metadata.annotations
nginx.ingress.kubernetes.io/force-ssl-redirect Separate HTTPRoute with redirect filter
service.beta.kubernetes.io/cloud-loadbalancer-* Gateway spec.infrastructure.annotations

Cloud LoadBalancer annotations must be placed in spec.infrastructure.annotations, not in metadata.annotations. This is a frequent configuration error that can prevent the LoadBalancer from being created correctly. Review the Gateway API infrastructure documentation for details about infrastructure annotations.

Certificate Management

  • Create explicit Certificate resources instead of using an annotation-based method.
  • The ClusterIssuer must use the gatewayHTTPRoute solver instead of the Ingress solver.

HTTP to HTTPS Redirect

Gateway API requires a separate HTTPRoute resource with a RequestRedirect filter. This is configured directly in the route, not through an annotation.

Step-by-Step Migration Guide

Use a blue-green migration approach: deploy Gateway API next to Ingress, test through the Gateway IP address, perform the DNS cutover, monitor stability, and then clean up. Both systems run at the same time to maintain zero downtime. You will temporarily pay for two LoadBalancers, usually for 24 to 48 hours. This method ensures that the Gateway configuration can be validated before production traffic is moved.

Phase 1: Prepare the Gateway API Stack

Step 1: Enable Gateway API in cert-manager

To allow cert-manager to work with Gateway API, configure it to support certificate issuance for Gateway-managed routes. The following Helm upgrade command updates the cert-manager installation with the required flag:

helm upgrade cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --reuse-values \
  --set extraArgs="{--enable-gateway-api=true}"

What this does:

  • helm upgrade cert-manager jetstack/cert-manager: Upgrades or installs cert-manager with the Jetstack Helm chart.
  • --namespace cert-manager: Applies the change in the cert-manager namespace.
  • --reuse-values: Keeps the existing cert-manager configuration and only changes the new setting.
  • --set extraArgs="{--enable-gateway-api=true}": Adds an argument so cert-manager can detect and manage Gateway API resources in addition to traditional Kubernetes Ingress resources.

Include this flag together with any existing extraArgs values you already use. This step is required so cert-manager can issue certificates for HTTPS routes managed by Gateway API.

Step 2: Create the Gateway Resource

The following gateway.yaml example shows how to define a Kubernetes Gateway resource for Gateway API. Each section has a specific purpose:

  • apiVersion, kind, and metadata: Define the API group, the resource type, and the name of the Gateway resource.
  • spec.gatewayClassName: Tells Kubernetes which GatewayClass to use, in this example cilium.
  • spec.infrastructure.annotations: Adds cloud LoadBalancer annotations. These configure the resulting LoadBalancer name, size, and health check path in the cloud environment. These annotations should be migrated from the old NGINX Ingress resource.
  • spec.listeners: Defines how the Gateway receives network traffic.
  • Two listeners are configured: one for HTTP on port 80 and one for HTTPS on port 443, both for the hostname www.example.com.
  • The HTTPS listener includes TLS configuration with mode: Terminate. This tells the Gateway to decrypt incoming HTTPS traffic. It references a Kubernetes Secret named www-tls that contains the TLS certificate.

Update the hostname, annotations, and referenced TLS secret name, such as www-tls, so they match your application configuration.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-gateway
spec:
  gatewayClassName: cilium
  infrastructure:
    annotations:
      # Copy cloud LoadBalancer annotations from your Ingress
      service.beta.kubernetes.io/cloud-loadbalancer-name: "gateway-api-lb"
      service.beta.kubernetes.io/cloud-loadbalancer-size-unit: "2"
      service.beta.kubernetes.io/cloud-loadbalancer-healthcheck-path: "/"
  listeners:
  - name: http
    protocol: HTTP
    port: 80
    hostname: "www.example.com"
  - name: https
    protocol: HTTPS
    port: 443
    hostname: "www.example.com"
    tls:
      mode: Terminate
      certificateRefs:
      - kind: Secret
        name: www-tls

This YAML creates a new Gateway resource that provisions a cloud LoadBalancer capable of serving HTTP and HTTPS traffic. It uses custom annotations and references the correct TLS certificate for secure connections.

Apply and verify the resource:

kubectl apply -f gateway.yaml
kubectl get gateway my-gateway  # Should show PROGRAMMED: True, ADDRESS assigned

Step 3: Create a ClusterIssuer for Automated HTTPS Certificates with Gateway API

This step defines a ClusterIssuer for cert-manager that uses the Gateway API HTTPRoute solver. The ClusterIssuer specifies how cert-manager should request and manage Let’s Encrypt certificates for domains served through the Gateway API rather than an NGINX Ingress.

With the gatewayHTTPRoute solver, HTTP validation challenges are handled through the Gateway itself. This enables automatic certificate issuance and renewal for applications and routes managed by the Gateway API.

How to do it:

  • Create a file named cluster-issuer-gateway.yaml.
  • Configure a ClusterIssuer resource in this file as shown below.
  • Replace email with your real email address.
  • Make sure metadata.name is unique and does not conflict with existing ClusterIssuers.
  • The parentRefs must match the name and namespace of the Gateway created earlier.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod-gateway
spec:
  acme:
    email: your-email@example.com   # <- Replace with your real email address
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod-gateway-key
    solvers:
      - http01:
          gatewayHTTPRoute:
            parentRefs:
              - name: my-gateway       # The Gateway name you defined earlier
                namespace: default     # The namespace of your Gateway
                kind: Gateway

Apply the ClusterIssuer resource to the cluster:

kubectl apply -f cluster-issuer-gateway.yaml

After the resource is applied, cert-manager can issue and renew certificates using HTTP-01 challenges routed through the new Gateway. This automates HTTPS for workloads managed by Gateway API.

Step 4: Copy the Existing TLS Certificate

Copy the Ingress TLS secret so the Gateway can use it. At this point, DNS still points to Ingress, so a new certificate cannot be issued yet. This step lets the Gateway use the existing certificate temporarily.

First, identify the TLS secret name used by the Ingress:

kubectl get ingress <your-ingress-name> -o jsonpath='{.spec.tls[0].secretName}'

The Gateway TLS secret name must match the value specified in spec.listeners[].tls.certificateRefs[].name in the Gateway resource, such as www-tls from Step 2.

Copy the secret:

INGRESS_TLS_SECRET=<your-ingress-tls-secret>  # From the command above
GATEWAY_TLS_SECRET=www-tls  # Must match Gateway spec.listeners[].tls.certificateRefs[].name

kubectl get secret $INGRESS_TLS_SECRET -o yaml | \
  sed "s/name: $INGRESS_TLS_SECRET/name: $GATEWAY_TLS_SECRET/" | \
  sed '/uid:/d' | sed '/resourceVersion:/d' | sed '/creationTimestamp:/d' | \
  kubectl apply -f -

Verify the secret:

kubectl get secret $GATEWAY_TLS_SECRET  # Should show kubernetes.io/tls type

Create a Certificate resource in Phase 4 after the DNS cutover so cert-manager can manage and renew the certificate properly. Without this resource, the certificate will expire after 90 days and will not renew automatically.

Step 5: Create an HTTPRoute for HTTPS Traffic

The following httproute.yaml example shows how to define a Kubernetes HTTPRoute resource for Gateway API. Each part has a specific purpose:

  • apiVersion, kind, and metadata: Define the API group, resource type, and resource name www-https.
  • metadata.annotations: Adds ExternalDNS annotations for hostname and TTL. These are optional, but useful for tracking DNS propagation.
  • spec.parentRefs: References the Gateway resource my-gateway and uses the section name https to match the Gateway listener.
  • spec.hostnames: Lists the hostnames matched by this route, here www.example.com.
  • spec.rules: Defines the routing rules for the HTTPRoute.
  • matches: Defines the path prefix / for incoming requests.
  • backendRefs: References the backend service my-www-service and the port 80 where traffic should be sent.

Create httproute.yaml and adjust the hostname and backend service:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: www-https
  annotations:
    # Add if using ExternalDNS
    external-dns.alpha.kubernetes.io/hostname: www.example.com
    external-dns.alpha.kubernetes.io/ttl: "300"
spec:
  parentRefs:
  - name: my-gateway
    sectionName: https
  hostnames:
  - www.example.com
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: my-www-service
      port: 80


kubectl apply -f httproute.yaml

Step 6: Create the HTTP to HTTPS Redirect

The following code block defines a Kubernetes HTTPRoute resource that redirects HTTP traffic to HTTPS for the domain. Each section has a specific function:

  • apiVersion, kind, and metadata: Define an HTTPRoute named http-redirect.
  • spec.parentRefs: Connects this route to the my-gateway Gateway resource and associates it with the http listener, which usually listens on port 80.
  • spec.hostnames: Targets traffic for www.example.com.
  • spec.rules: Adds a rule with a RequestRedirect filter. This filter tells the Gateway to redirect matching HTTP requests to HTTPS by setting scheme: https. The statusCode: 301 value creates a permanent redirect.

This resource instructs the Gateway to catch all HTTP requests for the domain and send them to the secure HTTPS endpoint, improving security and ensuring consistent TLS-based access.

Here is the YAML manifest for the redirect route:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: http-redirect
spec:
  parentRefs:
  - name: my-gateway
    sectionName: http
  hostnames:
  - www.example.com
  rules:
  - filters:
    - type: RequestRedirect
      requestRedirect:
        scheme: https
        statusCode: 301

Apply the redirect with this command:

kubectl apply -f http-redirect.yaml

Step 7: Validate That HTTPRoutes Are Attached

kubectl get httproute  # Should show both www-https and http-redirect
kubectl get gateway my-gateway  # Should show PROGRAMMED: True
kubectl describe httproute www-https
kubectl describe httproute http-redirect

In the describe output, check Status.Parents.Conditions for these values:

  • Type: Accepted with Status: True
  • Type: ResolvedRefs with Status: True

Common issues:

  • ResolvedRefs shows Status: False: Check whether the backend Service name exists and whether the port is correct.
  • Accepted shows Status: False: Confirm that sectionName and hostnames match the Gateway listeners.

Phase 2: Validate the Gateway Stack

Test through the IP address before changing DNS:

GATEWAY_IP=$(kubectl get gateway my-gateway -o jsonpath='{.status.addresses[0].value}')

# Test HTTP redirect (expect 301 to HTTPS, server: envoy)
curl -I --resolve www.example.com:80:$GATEWAY_IP http://www.example.com

# Test HTTPS traffic (expect 200, server: envoy)
curl -k -I --resolve www.example.com:443:$GATEWAY_IP https://www.example.com

# Verify content
curl -k --resolve www.example.com:443:$GATEWAY_IP https://www.example.com

Confirm that all tests return the expected results before continuing.

Phase 3: Execute the DNS Cutover

Use the section that matches how DNS records are managed, either manually or with ExternalDNS.

Manual DNS Update

# Optional: Lower TTL for faster rollback
cloudctl compute domain records update example.com --record-id <a-record-id> --record-ttl 60
sleep 300  # Wait for old TTL to expire

# Update A record to Gateway IP
cloudctl compute domain records update example.com --record-id <a-record-id> --record-data "$GATEWAY_IP"

# Monitor DNS propagation (Ctrl+C to stop)
while true; do echo "$(date): $(dig +short www.example.com)"; sleep 5; done

ExternalDNS with TXT Ownership Transfer

Deploy a second ExternalDNS instance for Gateway API and adjust secretKeyRef as needed:

helm install external-dns-gateway external-dns/external-dns \
  --namespace external-dns \
  --set provider=cloud-provider \
  --set sources[0]=gateway-httproute \
  --set txtOwnerId=gateway \
  --set interval=1m \
  --set env[0].name=CLOUD_PROVIDER_TOKEN \
  --set env[0].valueFrom.secretKeyRef.name=external-dns \
  --set env[0].valueFrom.secretKeyRef.key=token

Transfer TXT ownership to trigger the DNS cutover. Replace the example domains in these commands with the domain hosted by your DNS provider.

# Find TXT record. Replace update pattern to find your record: a-<hostname>
TXT_RECORD_ID=$(cloudctl compute domain records list example.com --format ID,Type,Name --no-header | grep "TXT.*a-<hostname>" | awk '{print $1}')

# Validate TXT_RECORD_ID has a value
echo $TXT_RECORD_ID

# Transfer ownership from default to gateway
cloudctl compute domain records update example.com \
  --record-id $TXT_RECORD_ID \
  --record-data "heritage=external-dns,external-dns/owner=gateway,external-dns/resource=gateway/default/my-gateway"

Monitor the cutover and stop the command with Ctrl+C when appropriate:

while true; do echo "$(date): $(dig +short www.example.com)"; sleep 5; done

After the IP changes to the Gateway IP, verify the result:

curl -I https://www.example.com  # Should return 200, server: envoy

Phase 4: Post Migration

Step 1: Establish Certificate Management

Create a Certificate resource so cert-manager can manage and renew the certificate correctly:


apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: www-tls-gateway
spec:
  secretName: www-tls
  issuerRef:
    name: letsencrypt-prod-gateway
    kind: ClusterIssuer
  dnsNames:
  - www.example.com


kubectl apply -f certificate.yaml
kubectl get certificate www-tls-gateway -w  # Wait for READY: True

Without this Certificate resource, certificate renewal will fail after 90 days. The Certificate resource tells cert-manager to automatically renew the certificate before it expires by using the ClusterIssuer created in Phase 1, Step 3.

Step 2: Monitor Stability

Monitor the setup for 24 to 48 hours before removing Ingress. Watch traffic volume, Certificate READY status, error rates, and response times. This monitoring period confirms that the Gateway is handling production traffic correctly before the old Ingress setup is removed as a safety fallback.

Step 3: Cleanup

Confirm certificate management before cleanup:

kubectl get certificate www-tls-gateway  # Must show READY = True
# Must show letsencrypt-prod-gateway or the name of the ClusterIssuer you created in Phase 1 Step 3
kubectl get certificate www-tls-gateway -o jsonpath='{.spec.issuerRef.name}'

Perform the cleanup:

helm uninstall external-dns -n external-dns  # If using ExternalDNS
helm uninstall ingress-nginx -n ingress-nginx
kubectl delete namespace ingress-nginx

Verify the final state:

curl -I https://www.example.com  # Should return 200, server: envoy
cloudctl compute load-balancer list --format ID,Region,Name,IP  # Should show only Gateway LoadBalancer

Rollback Procedure

Rollback is appropriate if the Gateway becomes unreachable, certificate issues appear, error rates rise significantly, or critical feature gaps are found. The rollback is quick, usually 2 to 6 minutes, because the Ingress NGINX setup remains in place during the monitoring period.

ExternalDNS Rollback

This command updates the TXT record managed by ExternalDNS for the domain example.com to move DNS ownership and traffic routing back to the Ingress NGINX controller. It uses the cloud provider CLI to set the TXT record data so it references the original Ingress resource. This ensures ExternalDNS updates the domain A record back to the old Ingress LoadBalancer IP.

cloudctl compute domain records update example.com \
  --record-id $TXT_RECORD_ID \
  --record-data "heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/default/sample-nginx"
# Wait ~60s for DNS update

Manual DNS Rollback

INGRESS_IP=$(kubectl get svc -n ingress-nginx ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
cloudctl compute domain records update example.com --record-id <a-record-id> --record-data "$INGRESS_IP"
# Wait for DNS propagation (~60-300s)

Validate the rollback with dig +short www.example.com. The output should show the Ingress IP. Total rollback time is usually 2 to 6 minutes.

Frequently Asked Questions

Can Ingress NGINX and Gateway API run at the same time during migration?

Yes. This is the recommended method for a zero-downtime migration. Both systems can operate in parallel. Gateway API creates a new cloud LoadBalancer with a separate IP address, which lets you test the Gateway configuration before changing DNS. After the Gateway has been validated, DNS is updated to the new LoadBalancer IP. The old Ingress NGINX setup is removed only after the environment has been monitored for stability.

What happens to existing TLS certificates during the migration?

The existing TLS certificates used by Ingress NGINX can be copied to Gateway API at first. In Phase 1, Step 4, the TLS secret is copied from Ingress to Gateway. After the DNS cutover, however, an explicit Certificate resource must be created, as shown in Phase 4, Step 1, so cert-manager can manage and renew the certificate properly. Without this Certificate resource, the certificate will expire after 90 days and will not renew automatically.

Why is a separate ClusterIssuer needed for Gateway API?

Gateway API requires a different certificate challenge solver than Ingress. Ingress uses the Ingress solver, while Gateway API uses the gatewayHTTPRoute solver. Because of this, a new ClusterIssuer must be created with gatewayHTTPRoute configuration that points to the Gateway resource. cert-manager must also be started with the --enable-gateway-api=true flag so Gateway API support is enabled.

How long does the migration process take?

The technical migration can be completed in a few hours, but a 24 to 48 hour monitoring phase should be planned before removing the old Ingress NGINX setup. The actual cutover time depends on DNS propagation, which usually takes 60 to 300 seconds after DNS records are updated. The period with two LoadBalancers, where both are billed, commonly lasts 24 to 48 hours while stability is monitored.

What happens if rollback is needed after migration?

The rollback process is simple. When ExternalDNS is used, TXT ownership is transferred back to the original Ingress. With manual DNS management, the A record is changed back to the Ingress LoadBalancer IP. The rollback usually takes 2 to 6 minutes in total, depending on DNS propagation. Since the Ingress NGINX setup remains active during the monitoring phase, returning to the previous setup is straightforward if needed.

Conclusion

The migration from Ingress NGINX to Gateway API on managed Kubernetes can be completed without downtime. The Gateway uses Cilium’s built-in controller and provides modern, extensible traffic management with explicit configuration and advanced routing capabilities. Gateway API also offers clearer separation of responsibilities, role-based access control, and more expressive routing options than the traditional Ingress API.

Migrate from Ingress NGINX to Gateway API on Managed Kubernetes with Cilium

The Ingress NGINX controller is being phased out. This guide explains how to migrate step by step to the Gateway API on a managed Kubernetes cluster with Cilium while keeping workloads available throughout the process. You will learn how to manage TLS certificates, perform a safe DNS cutover, and configure the cloud provider LoadBalancer for the new Gateway. The migration approach lets Ingress and Gateway API run in parallel, so you can validate production readiness before switching DNS traffic.

If Gateway API concepts or recommended practices are new to you, review tutorials about Gateway API with Cilium and HTTPS traffic routing first. These resources help explain the new API model, common configuration patterns, and the main differences compared with the traditional Ingress model. Careful planning and thorough testing are essential for a smooth migration.

Important: This guide is tested with managed Kubernetes version 1.33.x. If your cluster runs an older release, upgrade it before starting the migration.

Key Takeaways

  • Zero downtime is possible when moving from Ingress NGINX to Gateway API on managed Kubernetes by operating both controllers side by side and completing a controlled DNS cutover.
  • LoadBalancer endpoints will change. The Gateway API provisions a new LoadBalancer with a new IP address. Plan for a short phase with two LoadBalancers and only update DNS after production traffic has been validated.
  • Annotation migration is required. Ingress NGINX and Gateway API use different configuration models. Cloud LoadBalancer annotations belong in spec.infrastructure.annotations in Gateway resources, not in metadata.annotations.
  • Certificates must be defined explicitly. Instead of relying on Ingress annotations, Gateway API requires separate Certificate resources. The cert-manager solver must be configured for Gateway API, not for Ingress.
  • HTTP-to-HTTPS redirects move to filters. Gateway API does not process NGINX redirect annotations. Use a dedicated HTTPRoute with a RequestRedirect filter.
  • Test carefully before cutover. Use the Gateway LoadBalancer IP to confirm readiness before changing DNS. Remove Ingress only after stability has been verified.
  • Plan for temporary dual resources. During testing and cutover, two LoadBalancers will run at the same time, so include this in your migration planning.

Prerequisites

  • VPC-integrated managed Kubernetes cluster version 1.33+; verify with kubectl get gatewayclass cilium and confirm ACCEPTED: True
  • kubectl configured for your cluster
  • Existing Ingress NGINX deployment with cert-manager
  • Domain name with DNS access
  • Optional: ExternalDNS installed in the external-dns namespace
  • Budget for running two LoadBalancers during the migration

Key Migration Considerations

LoadBalancer IP Changes

The Gateway creates a new cloud LoadBalancer with a new IP address.

Solution: Run both LoadBalancers at the same time, move DNS to the new Gateway LoadBalancer, and remove the old Ingress LoadBalancer only after validation.

Annotation Mapping

Ingress NGINX Gateway API Location
kubernetes.io/ingress.class: nginx spec.gatewayClassName: cilium
cert-manager.io/cluster-issuer Explicit Certificate resource
external-dns.alpha.kubernetes.io/* HTTPRoute metadata.annotations
nginx.ingress.kubernetes.io/force-ssl-redirect Separate HTTPRoute with redirect filter
service.beta.kubernetes.io/cloud-loadbalancer-* Gateway spec.infrastructure.annotations

Cloud LoadBalancer annotations must be placed in spec.infrastructure.annotations, not in metadata.annotations. This is a frequent configuration error that can prevent the LoadBalancer from being created correctly. Review the Gateway API infrastructure documentation for details about infrastructure annotations.

Certificate Management

  • Create explicit Certificate resources instead of using an annotation-based method.
  • The ClusterIssuer must use the gatewayHTTPRoute solver instead of the Ingress solver.

HTTP to HTTPS Redirect

Gateway API requires a separate HTTPRoute resource with a RequestRedirect filter. This is configured directly in the route, not through an annotation.

Step-by-Step Migration Guide

Use a blue-green migration approach: deploy Gateway API next to Ingress, test through the Gateway IP address, perform the DNS cutover, monitor stability, and then clean up. Both systems run at the same time to maintain zero downtime. You will temporarily pay for two LoadBalancers, usually for 24 to 48 hours. This method ensures that the Gateway configuration can be validated before production traffic is moved.

Phase 1: Prepare the Gateway API Stack

Step 1: Enable Gateway API in cert-manager

To allow cert-manager to work with Gateway API, configure it to support certificate issuance for Gateway-managed routes. The following Helm upgrade command updates the cert-manager installation with the required flag:

helm upgrade cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --reuse-values \
  --set extraArgs="{--enable-gateway-api=true}"

What this does:

  • helm upgrade cert-manager jetstack/cert-manager: Upgrades or installs cert-manager with the Jetstack Helm chart.
  • --namespace cert-manager: Applies the change in the cert-manager namespace.
  • --reuse-values: Keeps the existing cert-manager configuration and only changes the new setting.
  • --set extraArgs="{--enable-gateway-api=true}": Adds an argument so cert-manager can detect and manage Gateway API resources in addition to traditional Kubernetes Ingress resources.

Include this flag together with any existing extraArgs values you already use. This step is required so cert-manager can issue certificates for HTTPS routes managed by Gateway API.

Step 2: Create the Gateway Resource

The following gateway.yaml example shows how to define a Kubernetes Gateway resource for Gateway API. Each section has a specific purpose:

  • apiVersion, kind, and metadata: Define the API group, the resource type, and the name of the Gateway resource.
  • spec.gatewayClassName: Tells Kubernetes which GatewayClass to use, in this example cilium.
  • spec.infrastructure.annotations: Adds cloud LoadBalancer annotations. These configure the resulting LoadBalancer name, size, and health check path in the cloud environment. These annotations should be migrated from the old NGINX Ingress resource.
  • spec.listeners: Defines how the Gateway receives network traffic.
  • Two listeners are configured: one for HTTP on port 80 and one for HTTPS on port 443, both for the hostname www.example.com.
  • The HTTPS listener includes TLS configuration with mode: Terminate. This tells the Gateway to decrypt incoming HTTPS traffic. It references a Kubernetes Secret named www-tls that contains the TLS certificate.

Update the hostname, annotations, and referenced TLS secret name, such as www-tls, so they match your application configuration.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-gateway
spec:
  gatewayClassName: cilium
  infrastructure:
    annotations:
      # Copy cloud LoadBalancer annotations from your Ingress
      service.beta.kubernetes.io/cloud-loadbalancer-name: "gateway-api-lb"
      service.beta.kubernetes.io/cloud-loadbalancer-size-unit: "2"
      service.beta.kubernetes.io/cloud-loadbalancer-healthcheck-path: "/"
  listeners:
  - name: http
    protocol: HTTP
    port: 80
    hostname: "www.example.com"
  - name: https
    protocol: HTTPS
    port: 443
    hostname: "www.example.com"
    tls:
      mode: Terminate
      certificateRefs:
      - kind: Secret
        name: www-tls

This YAML creates a new Gateway resource that provisions a cloud LoadBalancer capable of serving HTTP and HTTPS traffic. It uses custom annotations and references the correct TLS certificate for secure connections.

Apply and verify the resource:

kubectl apply -f gateway.yaml
kubectl get gateway my-gateway  # Should show PROGRAMMED: True, ADDRESS assigned

Step 3: Create a ClusterIssuer for Automated HTTPS Certificates with Gateway API

This step defines a ClusterIssuer for cert-manager that uses the Gateway API HTTPRoute solver. The ClusterIssuer specifies how cert-manager should request and manage Let’s Encrypt certificates for domains served through the Gateway API rather than an NGINX Ingress.

With the gatewayHTTPRoute solver, HTTP validation challenges are handled through the Gateway itself. This enables automatic certificate issuance and renewal for applications and routes managed by the Gateway API.

How to do it:

  • Create a file named cluster-issuer-gateway.yaml.
  • Configure a ClusterIssuer resource in this file as shown below.
  • Replace email with your real email address.
  • Make sure metadata.name is unique and does not conflict with existing ClusterIssuers.
  • The parentRefs must match the name and namespace of the Gateway created earlier.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod-gateway
spec:
  acme:
    email: your-email@example.com   # <- Replace with your real email address
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod-gateway-key
    solvers:
      - http01:
          gatewayHTTPRoute:
            parentRefs:
              - name: my-gateway       # The Gateway name you defined earlier
                namespace: default     # The namespace of your Gateway
                kind: Gateway

Apply the ClusterIssuer resource to the cluster:

kubectl apply -f cluster-issuer-gateway.yaml

After the resource is applied, cert-manager can issue and renew certificates using HTTP-01 challenges routed through the new Gateway. This automates HTTPS for workloads managed by Gateway API.

Step 4: Copy the Existing TLS Certificate

Copy the Ingress TLS secret so the Gateway can use it. At this point, DNS still points to Ingress, so a new certificate cannot be issued yet. This step lets the Gateway use the existing certificate temporarily.

First, identify the TLS secret name used by the Ingress:

kubectl get ingress <your-ingress-name> -o jsonpath='{.spec.tls[0].secretName}'

The Gateway TLS secret name must match the value specified in spec.listeners[].tls.certificateRefs[].name in the Gateway resource, such as www-tls from Step 2.

Copy the secret:

INGRESS_TLS_SECRET=<your-ingress-tls-secret>  # From the command above
GATEWAY_TLS_SECRET=www-tls  # Must match Gateway spec.listeners[].tls.certificateRefs[].name

kubectl get secret $INGRESS_TLS_SECRET -o yaml | \
  sed "s/name: $INGRESS_TLS_SECRET/name: $GATEWAY_TLS_SECRET/" | \
  sed '/uid:/d' | sed '/resourceVersion:/d' | sed '/creationTimestamp:/d' | \
  kubectl apply -f -

Verify the secret:

kubectl get secret $GATEWAY_TLS_SECRET  # Should show kubernetes.io/tls type

Create a Certificate resource in Phase 4 after the DNS cutover so cert-manager can manage and renew the certificate properly. Without this resource, the certificate will expire after 90 days and will not renew automatically.

Step 5: Create an HTTPRoute for HTTPS Traffic

The following httproute.yaml example shows how to define a Kubernetes HTTPRoute resource for Gateway API. Each part has a specific purpose:

  • apiVersion, kind, and metadata: Define the API group, resource type, and resource name www-https.
  • metadata.annotations: Adds ExternalDNS annotations for hostname and TTL. These are optional, but useful for tracking DNS propagation.
  • spec.parentRefs: References the Gateway resource my-gateway and uses the section name https to match the Gateway listener.
  • spec.hostnames: Lists the hostnames matched by this route, here www.example.com.
  • spec.rules: Defines the routing rules for the HTTPRoute.
  • matches: Defines the path prefix / for incoming requests.
  • backendRefs: References the backend service my-www-service and the port 80 where traffic should be sent.

Create httproute.yaml and adjust the hostname and backend service:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: www-https
  annotations:
    # Add if using ExternalDNS
    external-dns.alpha.kubernetes.io/hostname: www.example.com
    external-dns.alpha.kubernetes.io/ttl: "300"
spec:
  parentRefs:
  - name: my-gateway
    sectionName: https
  hostnames:
  - www.example.com
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: my-www-service
      port: 80


kubectl apply -f httproute.yaml

Step 6: Create the HTTP to HTTPS Redirect

The following code block defines a Kubernetes HTTPRoute resource that redirects HTTP traffic to HTTPS for the domain. Each section has a specific function:

  • apiVersion, kind, and metadata: Define an HTTPRoute named http-redirect.
  • spec.parentRefs: Connects this route to the my-gateway Gateway resource and associates it with the http listener, which usually listens on port 80.
  • spec.hostnames: Targets traffic for www.example.com.
  • spec.rules: Adds a rule with a RequestRedirect filter. This filter tells the Gateway to redirect matching HTTP requests to HTTPS by setting scheme: https. The statusCode: 301 value creates a permanent redirect.

This resource instructs the Gateway to catch all HTTP requests for the domain and send them to the secure HTTPS endpoint, improving security and ensuring consistent TLS-based access.

Here is the YAML manifest for the redirect route:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: www-https
  annotations:
    # Add if using ExternalDNS
    external-dns.alpha.kubernetes.io/hostname: www.example.com
    external-dns.alpha.kubernetes.io/ttl: "300"
spec:
  parentRefs:
  - name: my-gateway
    sectionName: https
  hostnames:
  - www.example.com
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: my-www-service
      port: 80

Step 7: Validate That HTTPRoutes Are Attached

kubectl get httproute  # Should show both www-https and http-redirect
kubectl get gateway my-gateway  # Should show PROGRAMMED: True
kubectl describe httproute www-https
kubectl describe httproute http-redirect

In the describe output, check Status.Parents.Conditions for these values:

  • Type: Accepted with Status: True
  • Type: ResolvedRefs with Status: True

Common issues:

  • ResolvedRefs shows Status: False: Check whether the backend Service name exists and whether the port is correct.
  • Accepted shows Status: False: Confirm that sectionName and hostnames match the Gateway listeners.

Phase 2: Validate the Gateway Stack

Test through the IP address before changing DNS:

GATEWAY_IP=$(kubectl get gateway my-gateway -o jsonpath='{.status.addresses[0].value}')

# Test HTTP redirect (expect 301 to HTTPS, server: envoy)
curl -I --resolve www.example.com:80:$GATEWAY_IP http://www.example.com

# Test HTTPS traffic (expect 200, server: envoy)
curl -k -I --resolve www.example.com:443:$GATEWAY_IP https://www.example.com

# Verify content
curl -k --resolve www.example.com:443:$GATEWAY_IP https://www.example.com

Confirm that all tests return the expected results before continuing.

Phase 3: Execute the DNS Cutover

Use the section that matches how DNS records are managed, either manually or with ExternalDNS.

Manual DNS Update

# Optional: Lower TTL for faster rollback
cloudctl compute domain records update example.com --record-id <a-record-id> --record-ttl 60
sleep 300  # Wait for old TTL to expire

# Update A record to Gateway IP
cloudctl compute domain records update example.com --record-id <a-record-id> --record-data "$GATEWAY_IP"

# Monitor DNS propagation (Ctrl+C to stop)
while true; do echo "$(date): $(dig +short www.example.com)"; sleep 5; done

ExternalDNS with TXT Ownership Transfer

Deploy a second ExternalDNS instance for Gateway API and adjust secretKeyRef as needed:

helm install external-dns-gateway external-dns/external-dns \
  --namespace external-dns \
  --set provider=cloud-provider \
  --set sources[0]=gateway-httproute \
  --set txtOwnerId=gateway \
  --set interval=1m \
  --set env[0].name=CLOUD_PROVIDER_TOKEN \
  --set env[0].valueFrom.secretKeyRef.name=external-dns \
  --set env[0].valueFrom.secretKeyRef.key=token

Transfer TXT ownership to trigger the DNS cutover. Replace the example domains in these commands with the domain hosted by your DNS provider.

# Find TXT record. Replace update pattern to find your record: a-<hostname>
TXT_RECORD_ID=$(cloudctl compute domain records list example.com --format ID,Type,Name --no-header | grep "TXT.*a-<hostname>" | awk '{print $1}')

# Validate TXT_RECORD_ID has a value
echo $TXT_RECORD_ID

# Transfer ownership from default to gateway
cloudctl compute domain records update example.com \
  --record-id $TXT_RECORD_ID \
  --record-data "heritage=external-dns,external-dns/owner=gateway,external-dns/resource=gateway/default/my-gateway"

Monitor the cutover and stop the command with Ctrl+C when appropriate:

while true; do echo "$(date): $(dig +short www.example.com)"; sleep 5; done

After the IP changes to the Gateway IP, verify the result:

curl -I https://www.example.com  # Should return 200, server: envoy

Phase 4: Post Migration

Step 1: Establish Certificate Management

Create a Certificate resource so cert-manager can manage and renew the certificate correctly:


apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: www-tls-gateway
spec:
  secretName: www-tls
  issuerRef:
    name: letsencrypt-prod-gateway
    kind: ClusterIssuer
  dnsNames:
  - www.example.com


kubectl apply -f certificate.yaml
kubectl get certificate www-tls-gateway -w  # Wait for READY: True

Without this Certificate resource, certificate renewal will fail after 90 days. The Certificate resource tells cert-manager to automatically renew the certificate before it expires by using the ClusterIssuer created in Phase 1, Step 3.

Step 2: Monitor Stability

Monitor the setup for 24 to 48 hours before removing Ingress. Watch traffic volume, Certificate READY status, error rates, and response times. This monitoring period confirms that the Gateway is handling production traffic correctly before the old Ingress setup is removed as a safety fallback.

Step 3: Cleanup

Confirm certificate management before cleanup:

kubectl get certificate www-tls-gateway  # Must show READY = True
# Must show letsencrypt-prod-gateway or the name of the ClusterIssuer you created in Phase 1 Step 3
kubectl get certificate www-tls-gateway -o jsonpath='{.spec.issuerRef.name}'

Perform the cleanup:

helm uninstall external-dns -n external-dns  # If using ExternalDNS
helm uninstall ingress-nginx -n ingress-nginx
kubectl delete namespace ingress-nginx

Verify the final state:

curl -I https://www.example.com  # Should return 200, server: envoy
cloudctl compute load-balancer list --format ID,Region,Name,IP  # Should show only Gateway LoadBalancer

Rollback Procedure

Rollback is appropriate if the Gateway becomes unreachable, certificate issues appear, error rates rise significantly, or critical feature gaps are found. The rollback is quick, usually 2 to 6 minutes, because the Ingress NGINX setup remains in place during the monitoring period.

ExternalDNS Rollback

This command updates the TXT record managed by ExternalDNS for the domain example.com to move DNS ownership and traffic routing back to the Ingress NGINX controller. It uses the cloud provider CLI to set the TXT record data so it references the original Ingress resource. This ensures ExternalDNS updates the domain A record back to the old Ingress LoadBalancer IP.

cloudctl compute domain records update example.com \
  --record-id $TXT_RECORD_ID \
  --record-data "heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/default/sample-nginx"
# Wait ~60s for DNS update

Manual DNS Rollback

INGRESS_IP=$(kubectl get svc -n ingress-nginx ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
cloudctl compute domain records update example.com --record-id <a-record-id> --record-data "$INGRESS_IP"
# Wait for DNS propagation (~60-300s)

Validate the rollback with dig +short www.example.com. The output should show the Ingress IP. Total rollback time is usually 2 to 6 minutes.

Frequently Asked Questions

Can Ingress NGINX and Gateway API run at the same time during migration?

Yes. This is the recommended method for a zero-downtime migration. Both systems can operate in parallel. Gateway API creates a new cloud LoadBalancer with a separate IP address, which lets you test the Gateway configuration before changing DNS. After the Gateway has been validated, DNS is updated to the new LoadBalancer IP. The old Ingress NGINX setup is removed only after the environment has been monitored for stability.

What happens to existing TLS certificates during the migration?

The existing TLS certificates used by Ingress NGINX can be copied to Gateway API at first. In Phase 1, Step 4, the TLS secret is copied from Ingress to Gateway. After the DNS cutover, however, an explicit Certificate resource must be created, as shown in Phase 4, Step 1, so cert-manager can manage and renew the certificate properly. Without this Certificate resource, the certificate will expire after 90 days and will not renew automatically.

Why is a separate ClusterIssuer needed for Gateway API?

Gateway API requires a different certificate challenge solver than Ingress. Ingress uses the Ingress solver, while Gateway API uses the gatewayHTTPRoute solver. Because of this, a new ClusterIssuer must be created with gatewayHTTPRoute configuration that points to the Gateway resource. cert-manager must also be started with the --enable-gateway-api=true flag so Gateway API support is enabled.

How long does the migration process take?

The technical migration can be completed in a few hours, but a 24 to 48 hour monitoring phase should be planned before removing the old Ingress NGINX setup. The actual cutover time depends on DNS propagation, which usually takes 60 to 300 seconds after DNS records are updated. The period with two LoadBalancers, where both are billed, commonly lasts 24 to 48 hours while stability is monitored.

What happens if rollback is needed after migration?

The rollback process is simple. When ExternalDNS is used, TXT ownership is transferred back to the original Ingress. With manual DNS management, the A record is changed back to the Ingress LoadBalancer IP. The rollback usually takes 2 to 6 minutes in total, depending on DNS propagation. Since the Ingress NGINX setup remains active during the monitoring phase, returning to the previous setup is straightforward if needed.

Conclusion

The migration from Ingress NGINX to Gateway API on managed Kubernetes can be completed without downtime. The Gateway uses Cilium’s built-in controller and provides modern, extensible traffic management with explicit configuration and advanced routing capabilities. Gateway API also offers clearer separation of responsibilities, role-based access control, and more expressive routing options than the traditional Ingress API.

Source: digitalocean.com

Create a Free Account

Register now and get access to our Cloud Services.

Posts you might be interested in: