The Network Is Not Free: Protocols and the Cost of Chattiness

The logistics platform has 6 backend services. The route optimizer calls the package service 50 times per route computation to fetch package dimensions and weight. Each call is an HTTP/1.1 GET request returning a JSON response. The route optimizer takes 2.3 seconds per computation. Profiling shows that 1.8 seconds, 78% of the total, is network overhead: connection setup, serialization, deserialization, and protocol framing.

Network overhead is not mysterious. It is measurable in bytes on the wire, round trips between services, and CPU cycles spent parsing. This chapter quantifies each component.

Serialization: JSON vs Protocol Buffers

The package service returns package metadata for a single package. The same data encoded in three formats:

JSON (142 bytes)

{"packageId":"PKG-40291","weight":2.4,"dimensions":{"length":30,"width":20,"height":15},"warehouse":"WH-042","status":"IN_TRANSIT","fragile":false}

142 bytes. Every field name is a string transmitted with every message. "packageId", "weight", "dimensions", "length", "width", "height" account for 68 bytes, 48% of the payload. The values account for 74 bytes.

Protocol Buffers (28 bytes)

// Concept: Protocol Buffers schema for package metadata
// Field names are NOT transmitted. Only field numbers (1 byte each).

syntax = "proto3";

message PackageInfo {
    string package_id = 1;
    float weight = 2;
    Dimensions dimensions = 3;
    string warehouse = 4;
    string status = 5;
    bool fragile = 6;
}

message Dimensions {
    int32 length = 1;
    int32 width = 2;
    int32 height = 3;
}

The same data in Protobuf:

08 0a 50 4b 47 2d 34 30 32 39 31  // field 1 (package_id): "PKG-40291"
15 9a 99 19 40                      // field 2 (weight): 2.4
1a 06 08 1e 10 14 18 0f            // field 3 (dimensions): {30, 20, 15}
22 06 57 48 2d 30 34 32            // field 4 (warehouse): "WH-042"
2a 0a 49 4e 5f 54 52 41 4e 53 49 54  // field 5 (status): "IN_TRANSIT"
                                    // field 6 (fragile): false (default, not transmitted)

28 bytes. Field names replaced by 1-byte field numbers. Integers encoded as varints (1 byte for values under 128). Boolean false is the default and is not transmitted at all.

The Cost Comparison

Format	Size (bytes)	Parse time (µs)	Schema required
JSON	142	8.2	No
Protobuf	28	0.9	Yes (.proto file)
Avro	24	1.1	Yes (.avsc file)

For 50 package lookups per route computation:

Metric	JSON	Protobuf
Response payload	7,100 bytes	1,400 bytes
Parse time (all 50)	410 µs	45 µs
Serialization time (server)	350 µs	38 µs

The serialization difference alone is 677 µs. The size difference affects network transfer time. On a 1 Gbps internal network, 7,100 bytes takes 57 µs. 1,400 bytes takes 11 µs. The difference is negligible on fast networks but compounds on congested links, high-latency connections, or when the response is much larger (batch queries returning thousands of packages).

gRPC and HTTP/2 Multiplexing

The route optimizer makes 50 HTTP/1.1 requests sequentially. Each request requires:

Connection establishment (TCP handshake: 1 round trip, ~0.5ms within a datacenter).
HTTP request/response framing (headers: ~300 bytes per request).
JSON parsing.

With HTTP/1.1 and no connection reuse: 50 connections, 50 TCP handshakes, 50 request/response cycles. Minimum overhead: $50 \times 0.5\text{ms} = 25\text{ms}$ just for TCP handshakes.

With HTTP/1.1 and keep-alive (connection reuse): 1 connection, but requests are serialized. Each request waits for the previous response. Total: $50 \times \text{round-trip-time} = 50 \times 0.5\text{ms} = 25\text{ms}$ in serial.

With gRPC over HTTP/2: 1 connection, all 50 requests multiplexed concurrently. Each request is an independent stream within the connection. The 50 responses arrive in parallel. Total: approximately 1 round trip for all 50 requests ($\approx 0.5\text{ms}$) plus server processing time.

// Concept: gRPC service definition for batch package lookup
// One RPC call replaces 50 HTTP requests

service PackageService {
    // Unary RPC: one request, one response
    rpc GetPackage(GetPackageRequest) returns (PackageInfo);

    // Server streaming: one request, multiple responses
    rpc GetPackages(GetPackagesRequest) returns (stream PackageInfo);
}

message GetPackagesRequest {
    repeated string package_ids = 1;  // Send all 50 IDs in one request
}

// Concept: gRPC batch call replacing 50 individual HTTP calls
// One request, 50 results, one round trip

ManagedChannel channel = ManagedChannelBuilder
    .forAddress("package-service", 9090)
    .usePlaintext()
    .build();

PackageServiceGrpc.PackageServiceBlockingStub stub =
    PackageServiceGrpc.newBlockingStub(channel);

GetPackagesRequest request = GetPackagesRequest.newBuilder()
    .addAllPackageIds(packageIds)  // 50 package IDs
    .build();

Iterator<PackageInfo> packages = stub.getPackages(request);
// Server streams 50 PackageInfo messages over a single HTTP/2 stream.
// Total: 1 round trip + server processing time.
// vs 50 HTTP/1.1 requests: 50 round trips + 50x parse overhead.

Protocol comparison showing bytes on the wire for JSON over HTTP/1.1 vs Protobuf over gRPC for the same 50-package lookup

The bar chart compares the total bytes transmitted and round trips required for the 50-package lookup scenario. JSON over HTTP/1.1 with individual requests transmits approximately 22KB (7.1KB payload + 15KB headers) across 50 round trips. Protobuf over gRPC with a batch request transmits approximately 1.6KB (1.4KB payload + 0.2KB framing) in 1 round trip. The difference is 14x in bytes and 50x in round trips.

The Decision Rule

Use JSON over HTTP for external APIs, browser clients, and services where human readability of payloads during debugging outweighs the performance cost. The parse overhead of JSON is acceptable when request volume is low (under 100 requests/second per client).

Use Protobuf over gRPC for internal service-to-service communication where the request volume is high (hundreds or thousands of RPCs per second). The schema requirement is a feature: it provides compile-time type safety and backward-compatible evolution. The binary encoding reduces payload size by 3-5x. HTTP/2 multiplexing reduces round trips.

When a single operation requires multiple lookups from the same service (the route optimizer’s 50 package lookups), batch the lookups into a single RPC call. The network overhead of 50 individual calls dominates the processing time. A single batch call eliminates 49 round trips and reduces serialization/deserialization from 50 invocations to 1.