Summary:
This commit converts IntervalSet to use CHECKs instead of throwing exceptions and provides safe tryInsert methods that return quic::Expected for error handling.
**Core Problem Solved:**
IntervalSet was throwing `std::invalid_argument` exceptions in two scenarios:
1. When constructing an Interval with `start > end`
2. When interval bounds exceed the maximum allowed value
This change eliminates exceptions in favor of CHECKs (for internal validation) and Expected-based error handling (for caller validation).
**Implementation Details:**
**1. IntervalSet Core Changes:**
- Replaced `throw std::invalid_argument` with `CHECK_LE` in Interval constructor
- Replaced `throw std::invalid_argument` with `CHECK_LE` in `insert(start, end)`
- Added `IntervalSetError` enum for error classification
- Added `folly::Expected` include
**2. Safe API Layer:**
- Added `tryInsert(interval)` method returning `Expected<Unit, IntervalSetError>`
- Added `tryInsert(start, end)` method with pre-validation
- Added `tryInsert(point)` method
- Added static `Interval::tryCreate()` method for safe interval construction
**3. Updated Code:**
- **QuicWriteCodec.cpp**: Updated `fillFrameWithPacketReceiveTimestamps` to use `tryInsert`
- Returns `QuicError` if interval validation fails
- Maintains existing error handling patterns
- **QuicTransportFunctions.cpp**: Updated `implicitAckCryptoStream` to use `tryInsert`
- Logs errors and continues processing other packets
- Robust error handling for crypto stream implicit acks
Reviewed By: kvtsoy
Differential Revision: D76792362
fbshipit-source-id: 5bd7c22e69a91d60cc41c603a1f2380893f4c8a0
Summary:
This is an API break, but it should mostly be a manageable one. We want to be able to compile mvfst internally without exceptions, and folly::Optional is one dependency that makes this challenging. Additionally, we already have an imported secondary optional type for performance/struct size reasons, tiny-optional.
This second optional interface is mostly compatible in an API sense (including the use of std::nullopt) with std::optional. Thus our approach is to remove the dependency on folly::Optional, and offer a quic::Optional instead.
The next diff will properly vendor tiny-optional so that quic::Optional is an independent version of it.
Reviewed By: sharmafb, kvtsoy
Differential Revision: D74133131
fbshipit-source-id: 715f8bb5043ba3bb876cacfe54236887e0686b30
Summary: More in the theme of returning Expected instead of throwing. For the folly case, we keep the try/catches in there and translate to Expected. For Libev, we convert directly to Expected.
Reviewed By: kvtsoy
Differential Revision: D73217128
fbshipit-source-id: d00a978f24e3b29a77a8ac99a19765ae49f64df8
Summary: As in title, this is more of a theme on adding an Expected return.
Reviewed By: kvtsoy
Differential Revision: D72579218
fbshipit-source-id: 25735535368838f1a4315667cd7e9e9b5df1c485
Summary: I started with the QuicStreamManager, but it turns out that the path from the manager up to the close path touches a LOT, and so this is a big diff. The strategy is basically the same everywhere, add a folly::Expected and check it on every function and enforce that with [[nodiscard]]
Reviewed By: kvtsoy
Differential Revision: D72347215
fbshipit-source-id: 452868b541754d2ecab646d6c3cbd6aacf317d7f
Summary:
We should only transition the `recvState` to `Closed` once we receive all of the reliable data, otherwise we'll drop all incoming data for that stream. This can happen in one of the two cases:
* We get a reset stream frame, and the reliable size in that frame is such that we've received all of the reliable data.
* We get a read stream frame, and that puts us in a situation where we've received all of the reliable data.
Reviewed By: jbeshay
Differential Revision: D67658440
fbshipit-source-id: b37786c7720014f3598b3c8c0a88084b06c5261e
Summary:
We've made the following changes to the send stream state machine:
* We allow the sending of multiple resets
* We only transition from `ResetSent` to `Closed` once we've successfully delivered all data until the `reliableSize` of some reset that has been ACKed by the peer. This transition can happen either when we get an ACK for a reset or an ACK for stream data.
Reviewed By: kvtsoy
Differential Revision: D66885041
fbshipit-source-id: f2f2617a56ea5713b77c9b48d7b1fa94de8e79db
Summary:
With normal resets, we transition from `StreamSendState::ResetSent` to `StreamSendState::Closed` when we get an ACK for a RESET_STREAM frame.
With reliable resets, this is going to be a little more complicated. We can't automatically transition from `StreamSendState::ResetSent` to `StreamSendState::Closed` when we get an ACK for a RESET_STREAM_AT frame because it's possible that the peer hasn't yet received all data until the reliable reset offset.
My idea is the following: Keep track of the `minReliableSizeAcked`, which is the lowest value of the reliable size in any RESET_STREAM_AT frame that was ACKed by the peer. We set it to 0 if a RESET_STREAM frame was ACKed by the peer.
Then, we can transition from `StreamSendState::ResetSent` to `StreamSendState::Closed` if any one of the following two events happen:
* We get an ACK for a RESET_STREAM_AT or a RESET_STREAM frame, and all data until the `minReliableSizeAcked` has been ACKed by the peer.
* We get an ACK for stream data, and this puts us in a state where all data until the `minReliableSizeAcked` has been ACKed by the peer.
Note: This diff doesn't have any functional change. The only change is that we're keeping track of the `minReliableSizeAcked`, but aren't using it anywhere.
Reviewed By: mjoras
Differential Revision: D66781199
fbshipit-source-id: 2aa5138a18f70e9801e59e747460558ba706939c
Summary:
Concatenate adjacent namespaces + format
This is a codemod. It was automatically generated and will be landed once it is approved and tests are passing in sandcastle.
You have been added as a reviewer by Sentinel or Butterfly.
Autodiff project: nc
Autodiff partition: fbcode.quic
Autodiff bookmark: ad.nc.fbcode.quic
Reviewed By: hanidamlaj
Differential Revision: D65365244
fbshipit-source-id: 0bbaa7684d03caf8fc8eff3439a0865940398220
Summary:
The idea here is to make it so we can swap out the type we are using for optionality. In the near term we are going to try swapping towards one that more aggressively tries to save size.
For now there is no functional change and this is just a big aliasing diff.
Reviewed By: sharmafb
Differential Revision: D57633896
fbshipit-source-id: 6eae5953d47395b390016e59cf9d639f3b6c8cfe
Summary:
This is the major transition that updates mvfst code to use the new interfaces. The new Folly implementations of the interfaces maintain all the existing behavior of folly types so this should not introduce any functional change. The core changes are:
- Update the BatchWriters to use the new interfaces.
- Update the FunctionLooper to use the new interfaces.
- Change QuicServerTransport to take the folly types and wrap them in the new types for use in the QuicTransportBase.
The rest of the diff is for updating all the existing uses of the QuicTrasnport to initialize the necessary types and pass them to the QUIC transport instead of directly passing folly types.
Reviewed By: mjoras
Differential Revision: D51413481
fbshipit-source-id: 5ed607e12b9a52b96148ad9b4f8f43899655d936
Summary:
This diff changes `QuicAsyncUDPSocketWrapper` so that it is an abstraction layer that inherits from `QuicAsyncUDPSocketType`, instead of simply being a container with aliases.
- Key changes in `QuicAsyncUDPSocketWrapper.h`, the rest of the updates switch us from using `QuicAsyncUDPSocketType` to `QuicAsyncUDPSocketWrapper`.
- It's difficult to mock the UDP socket today given that we expose the entire `folly::AsyncUDPSocket` type to the higher layers of the QUIC stack. This complicates testing and emulation because any mock / fake has to implement low level primitives like `recvmmsg`, and because the `folly::AsyncUDPSocket` interface can change over time.
- Pure virtual functions will be defined in `QuicAsyncUDPSocketWrapper` in a follow up diff to start creating an interface between the higher layers of the mvfst QUIC stack and the UDP socket, and this interface will abstract away lower layer details such as `cmsgs` and `io_vec`, and instead focus on populating higher layer structures such as `NetworkData` and `ReceivedPacket` (D48714615). This will make it easier for us to mock or fake the UDP socket.
This diff relies on changes to `folly::MockAsyncUDPSocket` introduced in D48717389.
--
This diff is part of a larger stack focused on the following:
- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
- The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
- The client currently has three receive paths, and none of them are well tested.
- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
- `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
- Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
- By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.
- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps.
Reviewed By: jbeshay, hanidamlaj
Differential Revision: D48717388
fbshipit-source-id: 4f34182a69ab1e619e454da19e357a6a2ee2b9ab
Summary: Update flow control settings names to reflect that these are indeed flow control
Reviewed By: jbeshay
Differential Revision: D48137685
fbshipit-source-id: a48372e21cdd529480e25785a9bd5de456427ef3
Summary:
To get reliable packet destruction events, created a `OutstandingPacketWrapper` wrapper that wraps the current `OutstandingPacket` class, so callback functions can be added to the wrapper instead of the underlying object itself.
#### Why do we need this wrapper?
`std::deque::erase` does not guarantee that appropriate object destructors will be called (only that the number of destructions = number of objects erased). This is a real problem as packet destruction events are then no longer reliable (OutstandingPacket object is wrong!). The wrapper class handles this condition by detecting it in the move assignment constructor (called during erase) and calls the appropriate packet destruction callback before packets are moved. If we did the same fix in the old OutstandingPacket, we have to make sure the callback is called before all the fields of OutstandingPacket are moved - this is not scaleable. Hence a wrapper with the underlying object and a destruction callback function.
I also disabled copy construction for OutstandingPacket (otherwise we will get duplicate OnPacketDestroyed callbacks and cannot track packets reliably). Removing packet copies also improves performance. Some code changes (in tests mostly) are mostly in service of this particular change.
Reviewed By: bschlinker
Differential Revision: D43896148
fbshipit-source-id: c295d3c4dba2368aa66f06df5fc82b473a03fb4d
Summary:
We don't need to carry these states after the handshake is confirmed, so make them pointers instead. This will facilitate adding a structure to the AckState for tracking duplicate packets.
(Note: this ignores all push blocking failures!)
Reviewed By: hanidamlaj
Differential Revision: D41626895
fbshipit-source-id: d8ac960b3672b9bb9adaaececa53a1203ec801e0
Summary:
This is a longstanding off by one. When a stream frame is ACKed the largest ACKed offset contained is the stream frame offset + len - 1, unless there's an fin.
(Note: this ignores all push blocking failures!)
Reviewed By: kvtsoy
Differential Revision: D39085665
fbshipit-source-id: cf353c7e72466085a27537d2c95748781f70f79a
Summary: The bug here is that we weren't accounting for the DSR state when checking if the stream should be closed. This causes the stream state to live until the connection closes.
Reviewed By: yairgott
Differential Revision: D36269711
fbshipit-source-id: 403268db1940b2f00130709243aa6df1d2af39cd
Summary:
I think this should just work without the trailing `_E`. It was added
when we mixed up our own union based variant and boost::variant. Some compiler
flags didn't like that. Now we no longer have mixed up cases, this should be
fine
Reviewed By: lnicco
Differential Revision: D25589393
fbshipit-source-id: 6430dc20f8e81af0329d89e6990c16826da168b8
Summary:
This is following a similar pattern than what was done for the client side.
Pull Request resolved: https://github.com/facebookincubator/mvfst/pull/160
Reviewed By: yangchi
Differential Revision: D23560951
Pulled By: xttjsn
fbshipit-source-id: 351417cbfa3230112fff4c4de59b307f88389cf6
Summary: we don't use it in most of our code any more
Reviewed By: sharmafb
Differential Revision: D23378968
fbshipit-source-id: 876c27e8dea0e82f939c5f1b52a1cb16bb13195d
Summary:
As it turns out, the extra indirection from storing a unique_ptr is not worse than the gain from using an `F14ValueMap` versus an `F14VectorMap`.
This reduces the `find` cost measurably in profiles, and doesn't appear to have any real negative effects otherwise.
Reviewed By: yangchi
Differential Revision: D20923854
fbshipit-source-id: a75c4649ea3dbf0e6c89ebfe0d31d082bbdc31fd
Summary: no more surprises in upper layer
Reviewed By: mjoras
Differential Revision: D19976510
fbshipit-source-id: 3487e9aa2cb28d7bc748f13bc2bbc393216b4a8a
Summary:
The retransmission buffer tracks stream frame data we have sent that is currently unacked. We keep this as a sorted `deque`. This isn't so bad for performance, but we can do better if we break ourselves of the requirement that it be sorted (removing a binary search on ACK).
To do this we make the buffer a map of offset -> `StreamBuffer`.
There were two places that were dependent on the sorted nature of the list.
1. For partial reliablity we call `shrinkBuffers` to remove all unacked buffers less than an offset. For this we now have to do it with a full traversal of the retransmission buffer instead of only having to do an O(offset) search. In the future we could make this better by only lazily deleting from the retransmission buffer on ACK or packet loss.
2. We used the start of the retransmission buffer to determine if a delivery callback could be fired for a given offset. We need some new state to track this. Instead of tracking unacked buffers, we now track acked ranges using the existing `IntervalSet`. This set should be small for the typical case, as we think most ACKs will come in order and just cause existing ranges to merge.
Reviewed By: yangchi
Differential Revision: D18609467
fbshipit-source-id: 13cd2164352f1183362be9f675c1bdc686426698
Summary: The state machine logic is quite abstruse, this modifies it to make it more readable.
Reviewed By: siyengar
Differential Revision: D18488301
fbshipit-source-id: c6fd52973880931e34904713e8b147f56d0c4629
Summary:
No-op change; This is so that test cases of the same test classes are
together.
Reviewed By: sharma95
Differential Revision: D18000631
fbshipit-source-id: 84690509e66fa6e1279b978b8f665bdbf09492b1
Summary:
Use the custom variant type for write frames as well, now that
we use them for read frames.
Reviewed By: mjoras
Differential Revision: D17776862
fbshipit-source-id: 47093146d0f1565c22e5393ed012c70e2e23d279
Summary:
The retransmission buffer list is sorted by offset. New written
always has higher offset than existing ones. Thus the binary search isn't
necessary.
Reviewed By: mjoras
Differential Revision: D17477354
fbshipit-source-id: d413a5c84c3831b257d5c1e6375bec56a763926b
Summary:
Removes clientConnId completely from ServerConnectionIdParams.
This diff first fixes an incorrect assumption; it calls
`shortHeader.getConnectionId()` which is actually the destination id (server)
not the client connection id.
Next, this entire block is unnecessary, because this will be called after
the transport is created, so the clientConnectionId will always be set.
This also setsconn.serverConnectionId earlier (shouldn't depend on
connClientId).
Reviewed By: yangchi
Differential Revision: D16792866
fbshipit-source-id: 537ba12baa9939c9d5512e46eb914c1d3a7a9aa2
Summary:
With partial reliability we adjust retransmission queue on skips; we need to account for that when we match buffers in retransmission queue.
In fully reliable mode an ack frame must always match a single buffer from re-tranmission queue (offset, length and eom).
In partially reliable mode we can run into two more scenarios in addition to the above:
* re-transmisison buffer for which the ack frame arrived has been removed fully
* re-transmisison buffer for which the ack frame arrived has been removed partially
If full buffer is missing - ignore the ack.
If part of the buffer is missing, make sure that the adjusted offset and length match to that found in the ack frame.
Reviewed By: yangchi
Differential Revision: D15301304
fbshipit-source-id: f281fa30957c1c13c42c05684984f9a1acaa4e34
Summary: This is no longer part of the spec. It's up to the application how to handle reset
Reviewed By: lnicco
Differential Revision: D15107164
fbshipit-source-id: 2a1fe0c552bd7f054e84ef86a01a78c379b0a483
Summary: This is step 1 for removing reset on reset, since the send side may need to transition to waiting for a reset ack while the read side is an any state.
Reviewed By: lnicco
Differential Revision: D15075849
fbshipit-source-id: 1e094942a8a1ca9a01d4161cd6309b4136a9cfbf
Summary:
There is a race condition with min data frame (skip) and ack frame when it comes to retransmission buffer.
Sometimes, we receive both min data and ack frames, the min data frame gets processed first and it clears out retransmission buffer (expected), but then we get to process the ack frame and there is that CHECK that expects the retransmission buffer to have at least one unacked buffer in it - there is none and then the app crashes.
Reviewed By: siyengar
Differential Revision: D14908630
fbshipit-source-id: d8bcd70291e1fd3618065fdeb44bedc6ac66bcaa
