Summary:
This diff renames the `RecvdPacketInfo` structure to `ReceivedPacket` and nests it within the `WriteAckFrameState` object to make its use clear and to avoid name conflicts.
--
This diff is part of a larger stack focused on the following:
- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
- The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
- The client currently has three receive paths, and none of them are well tested.
- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
- `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
- Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
- By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.
- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps.
Differential Revision: D48787902
fbshipit-source-id: 2c3e96d4888345c139e0a6d60b27f9fc9aa1879e
Summary: As in title. This is probably old legacy predating varints.
Reviewed By: jbeshay, hanidamlaj
Differential Revision: D47521039
fbshipit-source-id: d52a09a5fda045d2d3101f9b77a64a75e656954c
Summary: We shouldn't have config in the codec types. Instead solve the plumbing problem more explicitly, and only define the config in one place.
Reviewed By: jbeshay
Differential Revision: D45881730
fbshipit-source-id: fab6c967a38172f16e57a8978b10460fd196902e
Summary:
Padding frames by definition have no semantics. There also can be quite a large number of them, which makes their storage expensive.
Optimize for this by effectively coalescing multiple contiguous padding frames into a single frame.
(Note: this ignores all push blocking failures!)
Reviewed By: kvtsoy
Differential Revision: D44636497
fbshipit-source-id: 4c1cd88123c4fdf40ec9c553885f46669f7326da
Summary:
Fixing dependency loop introduced by D37799050 (96abc8160d)
Running `autodeps` yields the following patch:
```
--- a/xplat/quic/state/TARGETS
+++ b/xplat/quic/state/TARGETS
@@ -43,8 +43,8 @@
exported_deps = [
"//folly:random",
"//quic:constants",
+ "//quic/codec:codec",
"//quic/codec:types",
- "//quic/common:circular_deque",
"//quic/common:interval_set",
],
)
```
If this patch is applied, there is a circular dependency loop between `//quic/codec:codec` and `//quic/state:ack_states` by way of `//quic/codec:types`; this loop was introduced by D37799050 (96abc8160d).
Fixed by separating out headers files and targets. In parallel, renamed structures used for writing ACK frames (which were the reason this loop occurred) to make their role clear.
Differential Revision: D42281359
fbshipit-source-id: 8514c99f3fe72ff1d942d7f303e4a209838c7623
Summary:
- Use varint reordering threshold (this is still being discussed, using varint until a decision is made)
- Track server's minAckDelay on client
- Track ack frequency frame sequence number
- Improve frame parsing error logs
- Add some unit tests
Reviewed By: hanidamlaj
Differential Revision: D38289108
fbshipit-source-id: 274e45115022ffd4e15b60dc57f77f1cce69bd82
Summary:
The problem with Ping being a simple frame:
(1) All SimpleFrames are in the same scheduler. So sending ping means we may
also send other frames which can be problematic if we send in Initial or
Handshake space
(2) Ping isn't retranmisttable. But other Simple frames are. So we are
certainly setting this wrong when we send pure Ping packet today.
That being said, there are cases where we need to treat Ping as retransmittable.
One is when it comes to update ack state: If peer sends us Ping, we may want to
Ack early rather than late. so it makes sense to treat Ping as retransmittable.
Another place is insertion into OutstandingPackets list. When our API user sends
Ping, then also add a Ping timeout. Without adding pure Ping packets into OP list,
we won't be able to track the acks to our Pings.
Reviewed By: mjoras
Differential Revision: D21763935
fbshipit-source-id: a04e97b50cf4dd4e3974320a4d2cc16eda48eef9
Summary:
Becuase when we clone an existing packet, the logic inside the current
writetStreamFrameHeader is no longer correct.
Reviewed By: mjoras
Differential Revision: D21383828
fbshipit-source-id: 8e6bbb048eefd97ca7cf17b89edc2f395f274a73
Summary:
Different Builders now want to have its own way of writing or
appending write buffer. So let builders handle them.
This also add new APIs in BufWriter to copy data from IOBuf/BufQueue directly
into a destination IOBuf without cloning inbetween.
Reviewed By: mjoras
Differential Revision: D20821789
fbshipit-source-id: c0a24eb12378f64cf26c27d4232f610ed80fba84
Summary:
As title.
For now the buildPacket() api will still build out separate IOBufs for header and body even though they are just two separate IOBuf wrapping continuous memory. This is so that API in other layers don't have to change for now, and I can reuse all the existing packet builder unit tests for the new builder.
Reviewed By: mjoras
Differential Revision: D20781977
fbshipit-source-id: 64e5ed9fbcff102ca20d3730511b02e6e7426b40
Summary:
The encoding logic is the same no matter what builder and what IOBuf
manipulater we use. Make these functions reusable.
This requires the QuicInteger encode function to support both BufAppender and
BufWriter. Note that the template is on a callable object. Because later on we
will also need to support wrapped memcpy that isn't a member function.
Reviewed By: mjoras
Differential Revision: D20781979
fbshipit-source-id: 1b8cbdfd7580132c113b12687f932d47207a7cce
Summary:
Previously we stored an `IntervalSet` in each `WriteAckFrame`. We don't need to do this, as by the time we are writing an `ACK` the `IntervalSet` is complete. Instead of bothering with the `IntervalSet` operations, we can simply serialize it to a reverse-sorted `vector.`
Additionally this has some micro-optimizations for filling the ACK frame, with a new function for getting varint size.
Reviewed By: yangchi
Differential Revision: D19397728
fbshipit-source-id: ba6958fb36a4681edaa8394b1bcbbec3472e177d
Summary: By modifying `IntervalSet` a bit we can make it so it takes a `folly::small_vector` as the container. We expect that for real traffic there will not generally be a lot of ACK blocks per frame, so optimize for that.
Reviewed By: siyengar
Differential Revision: D18919975
fbshipit-source-id: 199a2ea9ba5003382e2d7d99fc7a6de7e8aafdca
Summary:
Don't use IOBufQueue for most operations in mvfst and use BufQueue instead. Since BufQueue did not support a splitAtMost, added it in instead.
The only place that we still use IOBufQueue is in crypto because fizz still requires it
Reviewed By: mjoras
Differential Revision: D18846960
fbshipit-source-id: 4320b7f8614f8d2c75f6de0e6b786d33650e9656
Summary:
Don't clone during decoding a stream frame.
This uses a BufQueue to manage the data instead of passing around cursors.
Reviewed By: yangchi
Differential Revision: D18827718
fbshipit-source-id: 7874d308bb5f43b983b9965b2862e733a2107039
Summary: As a part of Draft 17, application close frame has been removed, we use connection close frame to represent both application close and connection close.
Reviewed By: mjoras
Differential Revision: D18580856
fbshipit-source-id: d274fa2d3dbc59b926bca5a2b8a20328ae582703
Summary:
Get rid of IObufQueue usage in the PacketBuilder and replace it with a vanilla IOBuf.
This requires replacing QueueAppender with something else as well, so this diff adds a new class call BufAppender which does exactly the same things as a QueueAppender.
Having a BufAppender will allow us to avoid cloning the stream buffer in the future and avoid a clone during the write path.
Reviewed By: mjoras
Differential Revision: D18673517
fbshipit-source-id: 31fd7758688686371d038111514eb62d6b21672c
Summary:
Use the custom variant type for write frames as well, now that
we use them for read frames.
Reviewed By: mjoras
Differential Revision: D17776862
fbshipit-source-id: 47093146d0f1565c22e5393ed012c70e2e23d279
Summary:
Create a custom type for read frame types. This allows us to reduce size of code.
We use a macro to generate new variant types whenever we need to.
Reviewed By: yangchi
Differential Revision: D17266468
fbshipit-source-id: 59a1183dce728e71f0924f39f95a7b78449642b0
Summary:
Make a custom variant type for PacketHeader. By not relying on boost::variant
this reduces the code size of the implementation.
This uses a combination of a union type as well as a enum type to emulate a variant
Reviewed By: yangchi
Differential Revision: D17187589
fbshipit-source-id: 00c2b9b8dd3f3e73af766d84888b13b9d867165a
Summary:
Implement sending stream limit updates in a windowed fashion, so that as a peer exhausts its streams we will grant it additional credit. This is implemented by having the stream manager check if an update is needed on removing streams, and the api layer potentially sending an update after it initiates the check for closed streams.
This also makes some driveby changes to use `std::lower_bound` instead of `std::find` for the sorted collections in the stream manager.
Reviewed By: yangchi
Differential Revision: D16808229
fbshipit-source-id: f6e3460d43e4d165e362164be00c0cec27cf1e79
Summary:
Prior to this diff we would clone out an entire flow control's worth of data from the writebuffer and then clone out a smaller portion of that to write into the packet builder. This is extremely wasteful when we have a large flow control window.
Additionally we would always write the stream data length field even when we are going to fill the remainder of the packet with the current stream frame. By first calculating the amount of data that needs to can be written and writing the header, we can now omit the data length field when we can fill the whole packet.
Reviewed By: yangchi
Differential Revision: D15769514
fbshipit-source-id: 95ac74eebcde87dd06de54405d7f69c42362e29c
Summary: These were changed to varints. To support this we need to do some extra horrible version plumbing. I don't want to keep this long term but it works for now.
Reviewed By: yangchi
Differential Revision: D16293568
fbshipit-source-id: a9ea9083be160aa3e6b338a7d70d7f00e44ec5ab
Summary: post draft-17 it is interpreted as count not the max stream id
Reviewed By: mjoras
Differential Revision: D15523021
fbshipit-source-id: 779a1c5269eb8dbcdcd169303caa7a65e467339c
Summary:
The plan is to not treat probing frame as simple frame, but that is not
going to happen soon.
Reviewed By: sharma95
Differential Revision: D15176635
fbshipit-source-id: 62ac13cdb82a09161e9148dfc437cf7377a01c96
