> Sadly, capnproto's awesome RPC system doesn't appear to support streaming

Sure it does. You can implement "streaming" in Cap'n Proto by introducing a callback object, and making one RPC call for each item / chunk in the stream. In Cap'n Proto, "streaming" is just a design pattern, not something that needs to be explicitly built in, because Cap'n Proto is inherently far more expressive than gRPC.

That is, you can define a type like:

    interface Stream(T) {
      write @0 (item :T);
      end @1 ();  # signal successful end of stream
    }

Then you can define streaming methods like:

    streamUp @0 (...params...) -> (stream :Stream(T), ...results...)
    # Method with client->server stream.

    streamDown @0 (...params..., stream :Stream(T)) -> (...results...)
    # Method with server->client stream.

Admittedly, this technique has the problem that the application has to do its own flow control -- it has to keep multiple calls in-flight to saturate the connection, but needs to place a cap on the number of calls in order to avoid excess buffering. This is doable, but somewhat inconvenient.

So I am actually in the process of extending the implementation to make this logic built-in:

https://github.com/capnproto/capnproto/pull/825

Note that PR doesn't add anything new to the RPC protocol; it just provides helpers to tell the app how many concurrent calls to make.

Interesting. I do tend to favor protocols that are a bit lower level and more flexible for composing higher-level functionality. Though this does sound pretty complicated to implement using only what capnproto offers right now. Would there be a way to jury rig "request(n)" backpressure as described in reactive streams[0] (also implemented by RSocket[1]) on top of capnproto? That's what I'm using for omnistreams and it's proven very simple to reason implement and reason about.

[0] https://github.com/reactive-streams/reactive-streams-jvm

[1] http://rsocket.io/

[2] https://github.com/omnistreams/omnistreams-spec

Actually the more I think about it I don't think it would work, since request(n) assumes the producer can send multiple messages for each request.