30. 12. 2024 Damiano Chini APM, Development, NetEye

Supporting HTTP/2 and gRPC in nginx

Since its introduction the HTTP/2 protocol has been adopted more and more in servers and clients applications thanks to its improved performance compared to its ancestor HTTP/1.1.

This poses an issue to services exposed via nginx, since some specific configurations are needed on nginx in order to allow clients and servers to fully use the HTTP/2 protocol.

At the same time, the gRPC protocol, which uses the HTTP/2 protocol, also needs specific configurations on your nginx server in case some of the services exposed by nginx use the gRPC protocol. For example OTLP (the protocol used by OpenTelemetry clients to send data to the OpenTelemetry server) makes use of the gRPC protocol.

Scenario

Imagine a scenario where an nginx server serves as a load balancer for various services, including for example an OTLP server.

In such a scenario, it’s not enough to know which host name and port the service listens on, but you also need to understand whether the service supports HTTP/2 and whether the service uses the gRPC protocol.

If you do not explicitly tell nginx to handle such requests, it will refuse them. For example if our service my.internal.service uses gRPC, this simple proxy_pass configuration will not work for it:

stream {
    upstream my_internal_service {
        server my.internal.service:1234;
    }

    server {
        listen my.exposed.service:4321;
        proxy_pass my_internal_service;
    }
}

nginx Configuration

To support gRPC, we need to use the http directive, paying attention in particular to add the http2 flag to the listen directive, and use the grpc_pass directive (present since nginx 1.13.10) to proxy the gRPC traffic, as we see here:

http {
    upstream my_internal_service {
        server my.internal.service:1234;
    }

    server {
        listen my.exposed.service:4321 ssl http2;
        location / {
            grpc_pass grpcs://my_internal_service;
        }
    }
}

Moreover, in cases where your service accepts both HTTP and gRPC traffic, as in the case of an OTLP server, you’ll need to separately handle HTTP traffic and gRPC traffic, since the grpc_pass directive will not work for standard HTTP requests.

In this case, we can handle traffic separately by inspecting the Content-Type header of the HTTP request, and if we discover that its value is application/grpc, we then redirect the traffic to the grpc_pass directive, otherwise we’ll redirect it to the standard proxy_pass directive.

Let’s see this in action:

http {
    map $http_content_type $is_grpc {
        default                         0;
        "application/grpc"              1;
    }

    upstream my_internal_service {
        server my.internal.service:1234;
    }

    server {
        listen my.exposed.service:4321 ssl http2;
        location / {
            if ($is_grpc = 0) {
                proxy_pass https://my_internal_service;
            }

            if ($is_grpc = 1) {
                grpc_pass grpcs://my_internal_service;
            }
        }
    }
}

As you can see, we use the map directive to remember whether the request header Content-Type ($http_content_type) contains the value application/grpc. The variable is_grpc will evaluate to 1 if the request is gRPC, and 0 otherwise. At this point in the server directive we can check against the value of is_grpc to determine if the request should go through the grpc_pass directive or the proxy_pass directive.

These Solutions are Engineered by Humans

Did you read this article because you’re knowledgeable about networking? Do you have the skills necessary to manage networks? We’re currently hiring for roles like this as well as other roles here at Würth Phoenix.