Port-forward to a GKE Service that survives pod restarts

Why kubectl's forward ends with the pod

A port-forward is a stream to one pod. Given a Service, kubectl looks up a pod behind it once, at start, and translates the Service port to that pod's target port:

kubectl -n acme port-forward svc/echo 18080:80

Forwarding from 127.0.0.1:18080 -> 8080
Forwarding from [::1]:18080 -> 8080

Port 80 of the Service became container port 8080 of one pod. From then on kubectl talks to that pod only. In kubectl's source the command stores the pod's name before it opens the stream, and the forwarder returns lost connection to pod as soon as the stream's connection closes, whatever closed it. kubectl then exits with status 1. It does not look for another pod.

Restart it in a loop

Each new run looks the Service up again, so a shell loop gets you a forward to whichever pod is ready next:

while true; do
  kubectl -n acme port-forward svc/echo 18080:80
  sleep 1
done

A connection that was open when the stream closed fails, and the local port is closed until kubectl has started again and found a pod, so a client that connects in that gap is refused. That is fine for a browser tab you reload and not for a database client or a test suite that holds connections.

Forwards through the Connect gateway

If your kubeconfig reaches the cluster through the Connect gateway (the server is connectgateway.googleapis.com), a forward also ends on a timer. Google's Connect gateway page says long streams end after about 20 to 30 minutes, “an expected limitation due to security reasons”, and that “There is a quota limit of 10 active streams per fleet host project”.

We measured it on 6 November 2026 with kubectl 1.35.0 against a cluster on GKE 1.36.4-gke.2046003 in europe-central2: one port-forward to a Service through the gateway and a request every 10 seconds. Every request got 200 up to 1798 seconds. At 1808 seconds the request failed and kubectl exited with status 1:

error: lost connection to pod

The same probe through a cluster's DNS endpoint ran the full 50 minutes we gave it, 3000 seconds, with all 298 requests answered. The message is the same one a pod restart gives, so a forward that dies after half an hour on a fleet cluster points at the gateway, not at the pod.

Port-forward through the gateway also needs gkehub.gateway.stream. With roles/gkehub.gatewayReader and the RBAC for the forward, kubectl get pods worked, and port-forward printed a Google Error 400 (Bad Request) page. With -v=4 or higher, the line before it shows the real refusal: websocket: bad handshake (403 Forbidden): Permission 'gkehub.gateway.stream' denied. A custom role with that permission made the same forward work. The gateway's roles and setup are in kubectl through the GKE Connect gateway.

How Ostgate does it

Ostgate's GKE area has a forward that keeps the local port open and picks a pod per connection:

1. In GKE, open the cluster, select the Service and choose Forward Port… in its menu or inspector.
2. Pick the port. Under Address, Local port listens on 127.0.0.1; Cluster name answers on the Service's own name, such as echo.acme.svc.cluster.local, at the Service's port numbers, through the Ostgate port helper.
3. Keep Access at My processes so your terminal and apps may connect, and turn on Start when Ostgate opens if the forward should come back with the app.

Each new connection to the local port asks the cluster for a ready pod of the Service and opens its own stream to it. When no pod is ready, the forward shows Waiting for a pod and holds the connection for up to 10 seconds before it gives up. One failed connection is shown under the forward's row and does not stop it.

On 6 November 2026 we forwarded a two-pod Service on a cluster reached through its DNS endpoint (GKE 1.36.4-gke.2046003, europe-central2) and ran curl against the local port every half second with a 3-second timeout. In each of three runs of 120 requests we deleted the pod serving the forward partway through. All 360 requests got 200, and the forward moved to the replacement pods on its own. A forward to a pod, not a Service, ends when that pod is deleted, as kubectl's does.

The same day a 6000-token streamed answer from a model server on a GKE GPU node took 77.6 seconds through a Cluster name forward, and a second request after 60 seconds of silence on the same forward was answered too.

Through the Connect gateway each connection is its own gateway stream, so Google's time limit ends one long connection, not the forward, which keeps listening for the next one. The quota on active streams per fleet project still counts every open connection. We have not timed an Ostgate connection through the gateway past 30 minutes. A cluster whose only path is its public IP endpoint can't be forwarded in Ostgate yet. The GKE docs cover the forward sheet and logs.

When the forward stops

Questions

Does kubectl port-forward to a Service follow the Service's pods?

No. kubectl resolves the Service to one pod when the command starts and forwards to that pod only. When the connection to it closes, kubectl exits with error: lost connection to pod, and the local port closes with it.

Why does kubectl port-forward stop after about 30 minutes on GKE?

If the kubeconfig goes through the Connect gateway, Google ends long streams after about 20 to 30 minutes. In our test the forward answered for 1798 seconds and kubectl exited with error: lost connection to pod at 1808 seconds. The same test through a DNS endpoint ran the full 50 minutes.

Why does port-forward through the Connect gateway fail with 400 Bad Request?

The account has a gateway role without gkehub.gateway.stream, such as roles/gkehub.gatewayReader. The WebSocket upgrade gets 403 for that permission, kubectl falls back to SPDY, and the gateway answers that with 400. kubectl -v=4 shows the 403 line.

Why does kubectl port-forward svc/echo 18080:80 print -> 8080?

kubectl forwards to the pod, not to the Service, so it translates the Service port to the pod's target port. The echo Service maps port 80 to container port 8080, so the forward goes to 8080 on the pod it picked.