Summary:
This commit converts IntervalSet to use CHECKs instead of throwing exceptions and provides safe tryInsert methods that return quic::Expected for error handling.
**Core Problem Solved:**
IntervalSet was throwing `std::invalid_argument` exceptions in two scenarios:
1. When constructing an Interval with `start > end`
2. When interval bounds exceed the maximum allowed value
This change eliminates exceptions in favor of CHECKs (for internal validation) and Expected-based error handling (for caller validation).
**Implementation Details:**
**1. IntervalSet Core Changes:**
- Replaced `throw std::invalid_argument` with `CHECK_LE` in Interval constructor
- Replaced `throw std::invalid_argument` with `CHECK_LE` in `insert(start, end)`
- Added `IntervalSetError` enum for error classification
- Added `folly::Expected` include
**2. Safe API Layer:**
- Added `tryInsert(interval)` method returning `Expected<Unit, IntervalSetError>`
- Added `tryInsert(start, end)` method with pre-validation
- Added `tryInsert(point)` method
- Added static `Interval::tryCreate()` method for safe interval construction
**3. Updated Code:**
- **QuicWriteCodec.cpp**: Updated `fillFrameWithPacketReceiveTimestamps` to use `tryInsert`
- Returns `QuicError` if interval validation fails
- Maintains existing error handling patterns
- **QuicTransportFunctions.cpp**: Updated `implicitAckCryptoStream` to use `tryInsert`
- Logs errors and continues processing other packets
- Robust error handling for crypto stream implicit acks
Reviewed By: kvtsoy
Differential Revision: D76792362
fbshipit-source-id: 5bd7c22e69a91d60cc41c603a1f2380893f4c8a0
Summary:
This is an API break, but it should mostly be a manageable one. We want to be able to compile mvfst internally without exceptions, and folly::Optional is one dependency that makes this challenging. Additionally, we already have an imported secondary optional type for performance/struct size reasons, tiny-optional.
This second optional interface is mostly compatible in an API sense (including the use of std::nullopt) with std::optional. Thus our approach is to remove the dependency on folly::Optional, and offer a quic::Optional instead.
The next diff will properly vendor tiny-optional so that quic::Optional is an independent version of it.
Reviewed By: sharmafb, kvtsoy
Differential Revision: D74133131
fbshipit-source-id: 715f8bb5043ba3bb876cacfe54236887e0686b30
Summary: I started with the QuicStreamManager, but it turns out that the path from the manager up to the close path touches a LOT, and so this is a big diff. The strategy is basically the same everywhere, add a folly::Expected and check it on every function and enforce that with [[nodiscard]]
Reviewed By: kvtsoy
Differential Revision: D72347215
fbshipit-source-id: 452868b541754d2ecab646d6c3cbd6aacf317d7f
Summary:
With normal resets, we transition from `StreamSendState::ResetSent` to `StreamSendState::Closed` when we get an ACK for a RESET_STREAM frame.
With reliable resets, this is going to be a little more complicated. We can't automatically transition from `StreamSendState::ResetSent` to `StreamSendState::Closed` when we get an ACK for a RESET_STREAM_AT frame because it's possible that the peer hasn't yet received all data until the reliable reset offset.
My idea is the following: Keep track of the `minReliableSizeAcked`, which is the lowest value of the reliable size in any RESET_STREAM_AT frame that was ACKed by the peer. We set it to 0 if a RESET_STREAM frame was ACKed by the peer.
Then, we can transition from `StreamSendState::ResetSent` to `StreamSendState::Closed` if any one of the following two events happen:
* We get an ACK for a RESET_STREAM_AT or a RESET_STREAM frame, and all data until the `minReliableSizeAcked` has been ACKed by the peer.
* We get an ACK for stream data, and this puts us in a state where all data until the `minReliableSizeAcked` has been ACKed by the peer.
Note: This diff doesn't have any functional change. The only change is that we're keeping track of the `minReliableSizeAcked`, but aren't using it anywhere.
Reviewed By: mjoras
Differential Revision: D66781199
fbshipit-source-id: 2aa5138a18f70e9801e59e747460558ba706939c
Summary: This is effectively an unused field encoding duplicated information, but it was widespread.
Reviewed By: kvtsoy
Differential Revision: D57289922
fbshipit-source-id: ca1499e2576e5ae28e3880b865a29c2b8d9a3d1b
Summary:
Keep track of ECN values in incoming packets and update the AckState that is used for echoing the counts to the peer in ACK_ECN frames.
Reading and writing the ACK_ECN frames is in the next change.
Reviewed By: kvtsoy
Differential Revision: D54927783
fbshipit-source-id: 44471c6224ee8578aaacc0d1a1c54370ef6d2ffe
Summary: Neither QUIC not TransportMonitor is using the `packetsInflight` field of the `OutstandingPacketMetadata`
Reviewed By: hanidamlaj
Differential Revision: D55926288
fbshipit-source-id: 32efd66add1e6374a8d3446ff635fe582b36e644
Summary: Neither QUIC not TransportMonitor is using the `totalBodyBytesSent` field of the `OutstandingPacketMetadata`
Reviewed By: hanidamlaj
Differential Revision: D55897240
fbshipit-source-id: 521f8a016f838dd1fe0593daa7a4e45c4fd222cf
Summary:
This diff changes `WriteAckFrameState::ReceivedPacket` so that it stores a complete `ReceivedUdpPacket::Timings` object, instead of just one field extracted from that object. This lets us have access to both user space and socket timestamps in our ACK RX timestamp handling code. See D48785086 for a similar change.
--
This diff is part of a larger stack focused on the following:
- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
- The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
- The client currently has three receive paths, and none of them are well tested.
- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
- `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
- Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
- By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.
- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps.
Reviewed By: silver23arrow
Differential Revision: D48795108
fbshipit-source-id: 70471d2654a09cbf25e711af583c18084eb90ca0
Summary:
This diff renames the `RecvdPacketInfo` structure to `ReceivedPacket` and nests it within the `WriteAckFrameState` object to make its use clear and to avoid name conflicts.
--
This diff is part of a larger stack focused on the following:
- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
- The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
- The client currently has three receive paths, and none of them are well tested.
- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
- `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
- Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
- By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.
- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps.
Differential Revision: D48787902
fbshipit-source-id: 2c3e96d4888345c139e0a6d60b27f9fc9aa1879e
Summary:
This diff renames `ReceivedPacket` to `ReceivedUdpPacket` to clarify that it maps to a UDP packet and not a QUIC packet. A single UDP packet can contain multiple QUIC packets due to coalescing.
--
This diff is part of a larger stack focused on the following:
- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
- The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
- The client currently has three receive paths, and none of them are well tested.
- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
- `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
- Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
- By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.
- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps.
Reviewed By: silver23arrow
Differential Revision: D48788809
fbshipit-source-id: 3793c30212d545e226f3e5337289bc2601dfa553
Summary:
This diff renames `updateLargestReceivedPacketNum` to `addPacketToAckState`, and changes the function signature so that it takes the structure of `ReceivedPacket::Timings` instead of just a single receive timestamp. This allows us to plumb through steady clock and socket timestamps.
The current caller to `updateLargestReceivedPacketNum` had a `ReceivedPacket::Timings` and just passed the`receiveTimePoint` field from that structure to to `updateLargestReceivedPacketNum`
```
uint64_t distanceFromExpectedPacketNum = updateLargestReceivedPacketNum(
conn, ackState, packetNum, readData.udpPacket.timings.receiveTimePoint);
```
Now, we just pass the entire `ReceivedPacket::Timings` structure and extract `receiveTimePoint` inside:
```
uint64_t distanceFromExpectedPacketNum = addPacketToAckState(
conn, ackState, packetNum, readData.udpPacket.timings);
```
--
This diff is part of a larger stack focused on the following:
- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
- The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
- The client currently has three receive paths, and none of them are well tested.
- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
- `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
- Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
- By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.
- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps.
Differential Revision: D48785086
fbshipit-source-id: 48a424e3e27918a8efe41918e0bcfa57337d9337
Summary:
Previously, the maximum stored rx timestamps was controlled by a constant `kMaxReceivedPktsTimestampsStored`. Make this programmable via MC so different values can be pushed based on server and client criteria (via GK groups).
(Note: this ignores all push blocking failures!)
Reviewed By: mjoras
Differential Revision: D45343231
fbshipit-source-id: aafa799925da2c11e14394d11fa4855f7107daf4
Summary:
This is essentially a further extension of the previous idead of having a sequence number per-DSR stream.
The rationale is the same -- to reduce spurious loss declared from the reordering threshold.
The primary case this works around is the following
```
<!DSR><!DSR><!DSR><DSR><DSR><DSR><!DSR>
```
Suppose we get an ACK for [0-1,3-5] leaving only packet number 2 remaining.
The naive reordering threshold will declare packet number 2 as lost. However, in principle this is arguably wrong since when considered on the timeline of !DSR packets, the gap does not exceed the reordering threshold.
To accomodate this we need to track a monotonically increasing sequence number for each non-DSR packet written, and store that in the packet metadata. This way we can use that for the reordering comparison rather than the packet number itself.
When there is no DSR packets ever written to a connection this should devolve to an identical result to using the packet number, as they will increment together.
Reviewed By: kvtsoy
Differential Revision: D46742386
fbshipit-source-id: 2983746081c7b6282358416e2bb1bcc80861be58
Summary:
Somehow we never implemented this stat despite having it for ages.
It's relatively easy to do, we just need to check whether an entry was inserted to the IntervalSet we are already using for tracking what to ACK.
Note that this has the limitation that when the ACK interval set is cleared out (on ACK of ACK), we will no longer be able to detect duplicates. This is something we can tune later.
Reviewed By: kvtsoy
Differential Revision: D45131856
fbshipit-source-id: aad4e07e1a9cd5b2dc5dec60424f7cee15906c7e
Summary:
We don't need to carry these states after the handshake is confirmed, so make them pointers instead. This will facilitate adding a structure to the AckState for tracking duplicate packets.
(Note: this ignores all push blocking failures!)
Reviewed By: hanidamlaj
Differential Revision: D41626895
fbshipit-source-id: d8ac960b3672b9bb9adaaececa53a1203ec801e0
Summary:
Store timestamps/packet numbers of recently received packets in AckState.
- The maximum number of packets stored is controlled by kMaxReceivedPktsTimestampsStored.
- The packet number of entries in the deque is guarenteed to increase
monotonically because an entry is only added for a received packet
if the packet number is greater than the packet number of the last
element in the deque (e.g., entries are not added for packets that
arrive out of order relative to previously received packets).
Reviewed By: bschlinker
Differential Revision: D37799023
fbshipit-source-id: 3b6bf2ba8ea15219a87bbdc2724fe23eebe66b70
Summary:
The current logic for handling the ACK_FREQUENCY frame mistakenly uses the frame's reorderThreshold for deciding losses, rather than triggering immediate acknowledgements.
This change fixes the logic by tracking the out-of-order distance and comparing that against the reorderThreshold.
Reviewed By: mjoras
Differential Revision: D39331920
fbshipit-source-id: 125fd99dce9b2725ea0d5b26236f48f72db53c48
Summary: Make `min(minRTT - ACK delay)` obsered so far over the life of the connection available in `TransportInfo`. In addition, expose the last RTT and last RTT's AckDelay value as well.
Reviewed By: mjoras
Differential Revision: D28385923
fbshipit-source-id: a58000ac8bd9fdc63caa0b636bdb83905cd6b448
Summary:
RFC9002 states the following in section 5.3
> 5.3. Estimating smoothed_rtt and rttvar
> Therefore, when adjusting an RTT sample using peer-reported acknowledgment delays, an endpoint [...] MUST NOT subtract the acknowledgment delay from the RTT sample if the resulting value is smaller than the min_rtt. This limits the underestimation of the smoothed_rtt due to a misreporting peer
However, today we subtract the ACK delay from an RTT sample used to estimate `smoothed_rtt` if it is the first RTT sample, as we will not yet have a minimum. This can lead to underestimation of `smoothed_rtt` if the peer is misreporting, and furthermore can create a situation during which `smoothed_rtt` is lower than `min_rtt`.
Resolving by not allowing `smoothed_rtt` to be adjusted by ACK delay for the first RTT sample, since doing so would lead to `smoothed_rtt` being lower than `min_rtt`.
Reviewed By: jbeshay
Differential Revision: D34792648
fbshipit-source-id: e0d6207ff73cdc13e0b40193abd27abce7142499
Summary: As titled. Reduces confusion, as we always expect this to be populated.
Differential Revision: D31887097
fbshipit-source-id: 153b05bae8abd559fe49d2c07c64d2ad0d92a809
Summary:
- Skip ACK-only response to initial crypto data from client
- Enabled through experimental transport settings
Reviewed By: mjoras
Differential Revision: D31863653
fbshipit-source-id: b290db0399dd7a3be41078c4a839b484da864f2f
Summary:
Introruced a new DetailsPerStream struct inside the outstanding packet metadata
so we can easily track all streams carrying app data within a packet and each
stream's bytes sent (both new and total).
With this, consumers can easily deduce the number of retransmitted bytes per
stream using the StreamDetails members:
newStreamBytesSent - streamBytesSent.
Reviewed By: bschlinker
Differential Revision: D28063916
fbshipit-source-id: d915514f85f24f4889cfac2f7a66bfd2d6b0a4af
Summary:
The goal of this change is to fix the bug that ```ackState.needsToSendActImmediately``` can be set to false even if it has been set to true. It leads to an issue when a batch of packets are consumed.
**Changes**
1. No longer set **false** for ```ackState.needsToSendAckImmediately```. If it is ever set to **true**, it continues to be **true**. Otherwise, it sticks to the default value of **false**.
1. Set ```ackState.scheduleAckTimeout``` to **true** only if ```ackState.needsToSendAckImmediately``` is **false**.
1. Always set ```ackState.numRxPacketsRecvd``` and ```ackState.numNonRxPacketsRecvd``` to **0** if ```ackState.needsToSendAckImmediately``` is **true**.
Reviewed By: yangchi
Differential Revision: D28241894
fbshipit-source-id: 988ac973beaa6b4348d99aa0bd6036318a6e1778
Summary:
Previously, we maintained state and counters to count both, header and body
bytes together. This commit introduces additional counters and state to keep
track of just the body bytes that were sent and acked etc. Body bytes received
will be implemented later.
Reviewed By: bschlinker
Differential Revision: D27312049
fbshipit-source-id: 33f169c9168dfda625e86de45df7c00d1897ba7e
Summary: As in title. This doesn't actually send any frames, but implements basic support for the transport parameter and responding to the frames.
Reviewed By: yangchi
Differential Revision: D26134787
fbshipit-source-id: 2c48e01084034317c8f36f89c69d172e3cb42278
Summary:
- Adds counter of the number of ack-eliciting packets sent; useful for understanding lost / retransmission numbers
- Adds the following to `OutstandingPacketMetadata`
- # of packets sent and # of ack-eliciting packets sent; this enables indexing of each packet when processed by an observer on loss / RTT event, including understanding its ordering in the flow of sent packets.
- # of packets in flight; useful during loss analysis to understand if self-induced congestion could be culprit
- Fixes the inflightBytes counter in `OutstandingPacketMetadata`; it was previously _not_ including the packets own bytes, despite saying that it was.
Right now we are passing multiple integers into the `OutstandingPacket` constructor. I've switched to passing in `LossState` to avoid passing in two more integers, although I will need to come back and clean up the existing ones.
Differential Revision: D25861702
fbshipit-source-id: e34c0edcb136bc1a2a6aeb898ecbb4cf11d0aa2c
Summary:
I think this should just work without the trailing `_E`. It was added
when we mixed up our own union based variant and boost::variant. Some compiler
flags didn't like that. Now we no longer have mixed up cases, this should be
fine
Reviewed By: lnicco
Differential Revision: D25589393
fbshipit-source-id: 6430dc20f8e81af0329d89e6990c16826da168b8
Summary:
Due to high number of RTT samples I refactored the OutstandingPacket to
split the packet data and packet metrics. We know have access to metrics
without the need of saving the packet data.
Pull Request resolved: https://github.com/facebookincubator/mvfst/pull/178
Reviewed By: mjoras
Differential Revision: D23711641
Pulled By: bschlinker
fbshipit-source-id: 53791f1f6f6e184f37afca991a873af05909fbd2
Summary:
This is following a similar pattern than what was done for the client side.
Pull Request resolved: https://github.com/facebookincubator/mvfst/pull/160
Reviewed By: yangchi
Differential Revision: D23560951
Pulled By: xttjsn
fbshipit-source-id: 351417cbfa3230112fff4c4de59b307f88389cf6
Summary:
This diff:
1) introduces `EnumArray` - effectively an `std::array` indexed by an enum
2) changes loss times and `lastRetransmittablePacketSentTime` inside `LossState` to be an `EnumArray` indexed by `PacketNumberSpace`
3) makes the method `isHandshakeDone()` available for both client and server handshakes.
4) uses all those inputs to determine PTO timers in `earliestTimeAndSpace()`
Reviewed By: yangchi
Differential Revision: D19650864
fbshipit-source-id: d72e4a0cf61d2dcb76f0a7f4037c36a7c8156942
Summary:
It is useful to be able to control the ACK generation frequency based on how many packets have been received. This adds 3 tunables: a threshold, and an ACK frequency below and above that thresold.
Note this keeps the default so there is no behavior change.
Reviewed By: yangchi
Differential Revision: D19455514
fbshipit-source-id: 53b765d4c872bc6b7c19d5ea859f8eee86d9b5ff
Summary:
Previously we track them since we thought we can get some additional
RTT samples. But these are bad RTT samples since peer can delays the acking of
pure acks. Now we no longer trust such RTT samples, there is no reason to keep
tracking pure ack packets.
Reviewed By: mjoras
Differential Revision: D18946081
fbshipit-source-id: 0a92d88e709edf8475d67791ba064c3e8b7f627a
