A CourierEnvelope carries an epoch field that identifies the replica epoch whose envelope key the client used to encrypt the MKEM ciphertext. Conforming couriers and storage replicas MUST accept epoch ∈ {current − 1, current, current + 1} where current is the courier’s / replica’s own view of the current replica epoch.

The current − 1 tolerance handles the grace window immediately after a replica-epoch boundary, when a client with a slightly stale PKI view still encrypts to the previous envelope public key. Combined with current, this gives a two replica-epoch data TTL — roughly two weeks — because the future replica-epoch key is by definition a key that hasn’t started being used yet, so current and current − 1 are the epochs where actual data flows.

The current + 1 tolerance handles the same boundary seen from the other side: a client whose clock or PKI view is slightly ahead of the courier / replica.

Envelopes outside this three-epoch window MUST be rejected, because by definition no replica still holds the matching decapsulation key:

Couriers SHOULD reject with a dedicated envelope-level error code (EnvelopeErrorInvalidEpoch) so clients can distinguish “stale encryption” from other courier-side rejections.

The maximum plaintext BACAP payload a single box can carry is derived backwards from the Sphinx UserForwardPayloadLength by subtracting every layer of framing and cryptographic overhead that sits between the Sphinx payload and the BACAP plaintext. The authoritative calculation is performed by NewGeometryFromSphinx in pigeonhole/geo/geometry.go.

Informally:

MaxPlaintextPayloadLength
  = UserForwardPayloadLength
    − CourierQuery framing (1 byte: query_type discriminator)
    − CourierEnvelope header (intermediate_replicas[2] + Dek1 + Dek2
                              + reply_index + epoch + sender_pubkey_len
                              + sender_pubkey + ciphertext_len)
    − MKEM AEAD overhead (28 bytes)
    − ReplicaInnerMessage framing (1 byte: message_type discriminator)
    − ReplicaWrite header (box_id + signature + payload_len = 100 bytes)
    − BACAP AEAD overhead (16 bytes)
    − BACAP length prefix (4 bytes)  

With a hybrid CTIDH-1024 × X25519 NIKE, the fixed per-packet overhead between UserForwardPayloadLength and the BACAP plaintext is on the order of 560 bytes; the exact figure depends on the configured NIKE scheme and should always be obtained from geometry.NewGeometryFromSphinx() rather than computed by hand.

The courier, upon unwrapping the Sphinx payload of a client’s packet, sees a CourierQuery whose content is a CourierEnvelope with the following layout:

struct courier_envelope {
    u8  intermediate_replicas[2];          // replica indices in the PKI
    u8  dek1[MKEM_DEK_SIZE];               // DEK encapsulation for replica 0
    u8  dek2[MKEM_DEK_SIZE];               // DEK encapsulation for replica 1
    u8  reply_index;                       // which replica's reply to prefer
    u64 epoch;                             // replica-epoch under which the
                                           //   MKEM ciphertext was produced
    u16 sender_pubkey_len;
    u8  sender_pubkey[sender_pubkey_len];  // client's ephemeral hybrid NIKE pk
    u32 ciphertext_len;
    u8  ciphertext[ciphertext_len];        // MKEM-encrypted ReplicaInnerMessage
} 

Notable points:

Once an intermediate replica decrypts the MKEM envelope, it obtains a ReplicaInnerMessage — a discriminated union over message_type:

struct replica_inner_message {
    u8 message_type IN [0, 1];    // 0 = read, 1 = write
    union content[message_type] {
        0: struct replica_read  read_msg;   // { box_id[32] }
        1: struct replica_write write_msg;  // { box_id[32], signature[64],
                                            //   payload_len[u32], payload[] }
    };
} 

A ReplicaWrite with payload_len == 0 is a tombstone.

A CourierQuery is the top-level discriminated union that a client places into a Sphinx packet payload for the courier:

struct courier_query {
    u8 query_type IN [0, 1];
    union content[query_type] {
        0: struct courier_envelope envelope;      // read or write a single box
        1: struct copy_command     copy_command;  // AllOrNothing copy
    };
}  

The symmetric reply is a CourierQueryReply:

struct courier_query_reply {
    u8 reply_type IN [0, 1];
    union content[reply_type] {
        0: struct courier_envelope_reply envelope_reply;
        1: struct copy_command_reply     copy_command_reply;
    };
}  

The CourierEnvelope layout was given in the section called “The CourierEnvelope as seen by the courier”. The client constructs one CourierEnvelope per read or write. The courier performs the following actions.

ReplicaMessage is not a trunnel pigeonhole type; it is the core/wire/commands command sent from a courier to a replica. Its payload fields are copied verbatim from the matching CourierEnvelope:

// core/wire/commands
type ReplicaMessage struct {
    SenderEPubKey []byte     // copied from CourierEnvelope.sender_pubkey
    DEK           *[MKEM_DEK_SIZE]byte  // dek1 or dek2, depending on destination
    Ciphertext    []byte     // copied from CourierEnvelope.ciphertext
}

The recipient replica decapsulates the MKEM envelope using the replica-epoch key that corresponds to CourierEnvelope.epoch (trying each of the three keys in its tolerance window), yielding a padded ReplicaInnerMessage. The replica then dispatches on message_type to ReplicaRead or ReplicaWrite handling.

In response to a ReplicaMessage, the courier expects an asynchronous ReplicaMessageReply wire command from the replica:

// core/wire/commands
type ReplicaMessageReply struct {
    ErrorCode     uint8                 // see the replica error-code table below
    EnvelopeHash  *[HASH_SIZE]byte      // lets the courier demultiplex the reply
    EnvelopeReply []byte                // MKEM-encrypted ReplicaMessageReplyInnerMessage
}

The EnvelopeReply byte blob is produced by the replica via the MKEM scheme’s EnvelopeReply() method (see replica/handlers.go). It carries a ReplicaMessageReplyInnerMessage — a discriminated union over either a ReplicaReadReply or a ReplicaWriteReply — padded to ReplicaReplyInnerMessageReadSize() so that read replies and write replies are indistinguishable in size, and encrypted to the client’s ephemeral NIKE public key under the replica’s envelope keypair.

For each outstanding EnvelopeHash, the courier maintains an in-memory dedup entry. Its actual structure is:

// courier/server/plugin.go
type CourierBookKeeping struct {
    Epoch                uint64        // replica-epoch at cache insertion
    CreatedAt            time.Time     // for TTL eviction (~5 minutes)
    QueryType            uint8         // the query_type that produced this entry
    IntermediateReplicas [2]uint8
    EnvelopeReplies      [2]*commands.ReplicaMessageReply
}  

Note that the courier does not cache SURBs or SURB timestamps. A client’s SURB is consumed by the Sphinx-layer plugin infrastructure at the moment the courier emits a reply and is not retained by the courier’s Pigeonhole state. Should the client not receive that reply, it is expected to retransmit a fresh Sphinx packet carrying a fresh SURB but the identical CourierEnvelope body; the courier, recognising the EnvelopeHash, replies using the new SURB.

The dedup cache has a TTL of 5 minutes (DedupCacheTTL in courier/server/plugin.go).

The courier’s reply to a CourierEnvelope has the following trunnel layout:

struct courier_envelope_reply {
    u8  envelope_hash[HASH_SIZE];       // identifies the originating envelope
    u8  reply_index;                    // which intermediate replica's reply
                                        //   is being returned (may differ
                                        //   from the client's requested index)
    u8  reply_type IN [0, 1];           // 0 = ACK, 1 = PAYLOAD
    u32 payload_len;
    u8  payload[payload_len];           // the MKEM-encrypted EnvelopeReply,
                                        //   present iff reply_type == PAYLOAD
    u8  error_code;                     // see envelope error codes below
}

A reply_type of ACK (0) indicates the courier has received the envelope and dispatched it to the replicas but has not yet received a reply for the requested index. A reply_type of PAYLOAD (1) indicates the payload field carries the MKEM-encrypted EnvelopeReply produced by a replica.

Embedded inside the MKEM-encrypted ReplicaInnerMessage a client sends to a replica (via the courier) for a read operation.

struct replica_read {
    u8 box_id[BACAP_BOX_ID_SIZE];
}  

Embedded inside the MKEM-encrypted ReplicaMessageReplyInnerMessage a replica returns for a successful read. Padding is applied at the outer ReplicaMessageReplyInnerMessage level; this struct carries no padding of its own.

struct replica_read_reply {
    u8  error_code;
    u8  box_id[BACAP_BOX_ID_SIZE];
    u8  signature[BACAP_SIGNATURE_SIZE];
    u32 payload_len;
    u8  payload[payload_len];
}

Used both (a) embedded inside the MKEM-encrypted ReplicaInnerMessage for a client write, and (b) carried directly as a core/wire/commands command between replicas during replication.

struct replica_write {
    u8  box_id[BACAP_BOX_ID_SIZE];
    u8  signature[BACAP_SIGNATURE_SIZE];
    u32 payload_len;
    u8  payload[payload_len];
}

A ReplicaWrite with payload_len == 0 is a tombstone. Replicas treat tombstone writes as overwrites: an existing box at the same box_id is replaced by the tombstone, and subsequent reads return ReplicaErrorTombstone.

Embedded inside a ReplicaMessageReplyInnerMessage on the client reply path, and also used as the core/wire/commands reply to inter-replica replication.

struct replica_write_reply {
    u8 error_code;
}

Returned by a replica in ReplicaMessageReply.ErrorCode, ReplicaReadReply.error_code, and ReplicaWriteReply.error_code.

Codes 1, 10 and 11 are “expected outcomes” — they correspond to normal protocol states rather than faults. The thin-client helper thin.IsExpectedOutcome(err) treats these three codes as non-errors.

Returned by the courier in CourierEnvelopeReply.error_code.

Returned by the courier in CopyCommandReply.status (see below).

Returned by the courier in the status field of a CopyCommandReply. (See the section called “CopyCommandReply” for more information.)

The client uploads a temporary Pigeonhole stream.

The stream is sequence of CourierEnvelopes. Each CourierEnvelope is serialized and prefixed with a single 4-byte (u32) length field giving the size of that one envelope; the resulting length-prefixed blobs are concatenated back-to-back into one continuous byte stream.

That byte stream is then split across the BACAP Boxes of the temporary stream. A serialized CourierEnvelope is strictly larger than the maximum BACAP box payload (it wraps a full box payload plus its own metadata), so a single envelope does not fit in one box and the concatenated stream necessarily spans several boxes. Envelope boundaries therefore do not align with box boundaries. The precise framing is given in the section called “Temporary stream data format”.

The client sends a random courier the copy command which encapsulates the write capability to the temporary Pigeonhole stream written in Step 1 above. When the courier receives this copy command it extracts the ReadCap from the given WriteCap and uses it to read the stream of data. The courier then reads a box at a time and tries to extract 0 or 1 envelopes from each accumulation of stream segments.

After processing the command, the courier then overwrites the temporary stream with tombstones.

Each box in the temporary stream is a serialized CopyStreamElement. Defined in trunnel as:

// CopyStreamElement - wraps a CourierEnvelope chunk with stream position flags.
// Overhead: 1 byte (flags) + 4 bytes (envelope_len) = 5 bytes
struct copy_stream_element {
    // Flags: bit 0 = isStart, bit 1 = isFinal
    u8 flags;
  
    // The CourierEnvelope serialized bytes
    u32 envelope_len;
    u8 envelope_data[envelope_len];
}

Here it is in golang:

type CopyStreamElement struct {
    Flags        uint8
    EnvelopeLen  uint32
    EnvelopeData []uint8
}

The purpose of this specific format is to use the isStart and isFinal flags to tell the courier the first box and last box of the stream to process. The payload encapsulated within the EnvelopeData fields of many of these CopyStreamElements is itself a stream of data which contains CourierEnvelopes with four-byte prefixes.

A key property of this encoding is that envelope boundaries do not align with box boundaries. Each BACAP box payload has a maximum size of N bytes, but a serialized CourierEnvelope (which contains a full box payload plus metadata) exceeds N bytes. Therefore envelopes are serialized into a continuous byte stream and split across multiple boxes in the temporary copy stream:

TEMPORARY COPY STREAM BOXES (each holds N bytes):
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│    Box 0    │ │    Box 1    │ │    Box 2    │ │    Box 3    │ │    Box 4    │
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
SERIALIZED ENVELOPE DATA (envelope boundaries don't align with box boundaries):
┌─────────────────────────┐┌────────────────────┐┌──────────────────────────┐
│       Envelope 1        ││     Envelope 2     ││       Envelope 3         │
└─────────────────────────┘└────────────────────┘└──────────────────────────┘
|         |         |         |         |         |
     Box 0     Box 1     Box 2     Box 3     Box 4

The courier reads the stream box by box, accumulating data until it can extract complete envelopes. The isStart and isFinal flags on the CopyStreamElement wrappers tell the courier where the stream begins and ends.

Each embedded CourierEnvelope is processed as a normal write and results in a ReplicaMessage being dispatched to the intermediate replicas over the wire protocol.

The courier does NOT need to keep track of the EnvelopeHash for each contained CourierEnvelope for the purpose of replying to the client (the “client” of these envelopes being, in this case, the courier itself), but it does need to keep resending them until the intermediate replicas have ACKed them.

The courier MUST keep track of the hash of the CopyCommand (computed as BLAKE2b-256(write_cap)) and MUST NOT process a given command more than once. This dedup cache has a TTL of 30 minutes (CopyDedupCacheTTL in courier/server/plugin.go).

CopyCommand is sent by a client to its chosen courier after the client has successfully uploaded every box of the temporary stream. The trunnel layout is:

struct copy_command {
    u32 write_cap_len;
    u8  write_cap[write_cap_len];   // serialized BACAP BoxOwnerCap
}

On receipt, the courier:

Replica error handling during a CopyCommand is classified as temporary or permanent by courier/server/copy_errors.go: transient errors (storage full, database failure, internal error, replication failed, box-ID not found) trigger bounded retries against the same shard before failover; permanent errors (invalid box ID, invalid signature, invalid payload, invalid epoch, box already exists, tombstone) cause immediate failover or abort.

The courier’s reply to a CopyCommand has the following trunnel layout:

struct copy_command_reply {
    u8  status;                   // CopyStatus{Succeeded, InProgress, Failed}
    u8  error_code;               // replica error code (meaningful iff status == Failed)
    u64 failed_envelope_index;    // 1-based sequential position in the
                                  //   CourierEnvelope stream at which processing
                                  //   stopped (meaningful iff status == Failed)
}  

The status codes are enumerated in “Copy command status codes” above. failed_envelope_index counts envelopes within the stream, not boxes: the first envelope in the first box of the temporary stream is index 1.

A client receiving CopyStatusInProgress should continue polling — i.e. resend the same CopyCommand via a fresh SURB after a short interval (CopyPollInterval, currently 5 seconds) — until it receives either CopyStatusSucceeded or CopyStatusFailed. Because the courier’s copy dedup cache keys on BLAKE2b-256(write_cap), these repeated polls will not cause the CopyCommand to be processed more than once.

In no particular order: