1
0
mirror of https://github.com/facebookincubator/mvfst.git synced 2025-11-09 10:00:57 +03:00
Commit Graph

367 Commits

Author SHA1 Message Date
Hani Damlaj
e9c9feea7f remove QuicStreamManager::swapStreams()
Summary: - `swapStreams` is likely mis-implemented and causing an unnecessary copy of the "dst" vector

Reviewed By: mjoras

Differential Revision: D53139321

fbshipit-source-id: adbf1417ee46f10ca4b32389eae7ddaca873c790
2024-01-31 15:13:07 -08:00
Hani Damlaj
d8952820d9 QuicTransportBase
Summary: - refactor some code for conciseness

Reviewed By: mjoras, sharmafb

Differential Revision: D48408121

fbshipit-source-id: 5f774328e1b28f8cbf130c1dc45b3ead0201a874
2024-01-31 15:13:07 -08:00
Hani Damlaj
6573a9f979 maybeSetGenericAppError
Summary: - move `maybeSetGenericAppError` to anonymous namespace

Reviewed By: mjoras, sharmafb

Differential Revision: D48408122

fbshipit-source-id: 1e4f77d2aa6d2440217dfed5b83b5643f8e090c6
2024-01-31 15:13:07 -08:00
Joseph Beshay
cf6f6b98ec Fix active connection counter in QuicStats, and make it work for the client too
Summary:
The current open connection counting is broken. To fix it this change:
- moves the counter to be for both the server and client transport
- calls onNewConnection() when the transport is ready.
- calls onConnectionClose() when the transport is closed after it was ready.

Reviewed By: lnicco, frankfeir

Differential Revision: D52638628

fbshipit-source-id: bcff7a8c671f8eede53c22e698e1b95332c56959
2024-01-12 08:59:15 -08:00
Joseph Beshay
e606de45db Add stream read/write errors to the StreamTransportInfo struct
Summary: As title.

Reviewed By: sharmafb

Differential Revision: D52678304

fbshipit-source-id: 3df2d588c40d40059141c72d0945d31b319b61f8
2024-01-11 19:35:07 -08:00
Joseph Beshay
96b65104dc Move QuicEventBase and QuicTimer callback operations back to the Callbacks themselves.
Summary:
This moves all the loop and timer callback operations back to the callbacks instead of relying the QuicEventBase to perform them. I.e. it's now Callback::cancelCallback() instead of QuicEventBase::cancelCallback(&callback).

To simplify the design, the lifetime of LoopCallbackWrapper and TimerCallbackWrapper has been changed. Previously these wrappers lasted for one iteration of the loop or the timer callback. In the new design the wrapper is created the first time the callback is scheduled and is only destroyed when the callback is destroyed. This significantly simplifies the lifetime management of the wrapper and the code overall.

While transitioning LibevQuicEventBase to this new design, this change also consolidates the QuicTimer functionality in the LibevQuicEventBase itself.

Reviewed By: hanidamlaj, mjoras

Differential Revision: D52217879

fbshipit-source-id: da7b95d592650ddc112813fdc3b9f5010d32e7fb
2024-01-03 12:38:20 -08:00
Matt Joras
3ba2147b67 Always cancel drain timeout in QuicTransportBase destructor
Summary: This is basically as a safeguard, to ensure that the drain timeout doesn't try to keep running after the destruction of the transport.

Reviewed By: jbeshay

Differential Revision: D52267494

fbshipit-source-id: d9ba99e499de7d01281b56bec40298f4e52e6fca
2023-12-18 20:29:12 -08:00
Matt Joras
cc927374d7 Optional writable bytes backpressure
Summary: Add backpressure based on CCA's writable bytes (cwnd), which can be useful for the app to use as backpressure. This also takes into account stream bytes currently buffered.

Differential Revision: D52098629

fbshipit-source-id: c672e73bab01eb301a67c9b8a95225780000cbd6
2023-12-15 21:58:10 -08:00
Joseph Beshay
ead139adef Move all mvfst use-cases to the new Eventbase, Timer, and Socket interfaces
Summary:
This is the major transition that updates mvfst code to use the new interfaces. The new Folly implementations of the interfaces maintain all the existing behavior of folly types so this should not introduce any functional change. The core changes are:
- Update the BatchWriters to use the new interfaces.
- Update the FunctionLooper to use the new interfaces.
- Change QuicServerTransport to take the folly types and wrap them in the new types for use in the QuicTransportBase.

The rest of the diff is for updating all the existing uses of the QuicTrasnport to initialize the necessary types and pass them to the QUIC transport instead of directly passing folly types.

Reviewed By: mjoras

Differential Revision: D51413481

fbshipit-source-id: 5ed607e12b9a52b96148ad9b4f8f43899655d936
2023-12-14 00:24:12 -08:00
Matt Joras
f7a59e8465 onStreamPreReaped
Summary: Move the callback to before the stream is destroyed.

Reviewed By: jbeshay

Differential Revision: D52127833

fbshipit-source-id: 1a9dbcf9ad92dbdb4e15409a418307dc6852b091
2023-12-13 11:19:53 -08:00
Matt Joras
9285422f2a Add onStreamPreReaped.
Summary: This is potentially useful to an application to know when the underlying stream state has been freed by the transport.

Reviewed By: jbeshay

Differential Revision: D52048888

fbshipit-source-id: e7b2d33c3702ce8aa348459a37094198d16af60f
2023-12-13 10:08:47 -08:00
Brandon Schlinker
83ad2ad99d Cleanup and modularize receive path, improve timestamp support [20/x]
Summary:
This diff renames `ReceivedPacket` to `ReceivedUdpPacket` to clarify that it maps to a UDP packet and not a QUIC packet. A single UDP packet can contain multiple QUIC packets due to coalescing.

--

This diff is part of a larger stack focused on the following:

- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
  - The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
  - The client currently has three receive paths, and none of them are well tested.

- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
  - `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
  - Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
  - By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.

- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps.

Reviewed By: silver23arrow

Differential Revision: D48788809

fbshipit-source-id: 3793c30212d545e226f3e5337289bc2601dfa553
2023-11-28 07:47:57 -08:00
Brandon Schlinker
04facac67d Cleanup and modularize receive path, improve timestamp support [15/x]
Summary:
This diff drops `NetworkDataSingle` in favor of `ReceivedPacket`. The latter contains a `ReceivedPacket::Timings` field that has the same `receiveTimePoint` currently in `NetworkDataSingle`, while also providing other useful signals.

--

This diff is part of a larger stack focused on the following:

- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
  - The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
  - The client currently has three receive paths, and none of them are well tested.

- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
  - `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
  - Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
  - By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.

- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps

Reviewed By: silver23arrow

Differential Revision: D48739219

fbshipit-source-id: fc2cdb7b425d68c729dd3bec00b6c6ff3c4bf8ec
2023-11-05 19:58:46 -08:00
Brandon Schlinker
05b98a99db Cleanup and modularize receive path, improve timestamp support [11/x]
Summary:
This diff:
1. Introduces a new `ReceivedPacket::Timings` structure, which will be expanded upon in subsequent diffs.
2. Adds a `ReceivedPacket::Timings` field to each `ReceivedPacket`
3. Uses the accessors added in the previous diff (D48724715) to populate the `ReceivedPacket::Timings` structure in each `ReceivedPacket` held by a `NetworkData` object. This is done by propagating the `NetworkData::receiveTimePoint` field to all `ReceivedPacket` held in a `NetworkData.` This propagation occurs each time a `ReceivedPacket` is added. The value is propagated again if the `NetworkData::receiveTimePoint` field is updated by looping over all previously added `ReceivedPacket`.

--

This diff is part of a larger stack focused on the following:

- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
  - The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
  - The client currently has three receive paths, and none of them are well tested.

- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
  - `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
  - Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
  - By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.

- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps.

Reviewed By: jbeshay

Differential Revision: D48725209

fbshipit-source-id: 580e7d7d1f3587f9947774b5ed19e9985df404c9
2023-11-05 19:58:46 -08:00
Brandon Schlinker
cb10ddf0ce Cleanup and modularize receive path, improve timestamp support [10/x]
Summary:
This diff makes the fields in `NetworkData` private so that they can only be manipulated via accessors.
- This will be used in a subsequent diff to make it possible to populate a new `ReceivedPacket::Timings` structure in each `ReceivedPacket` held by a `NetworkData` object with the timing information currently held in `NetworkData.`
- These accessors are temporary and being used to split up the stack, so relying on existing tests to provide coverage instead of adding new tests here.

--

This diff is part of a larger stack focused on the following:

- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
  - The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
  - The client currently has three receive paths, and none of them are well tested.

- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
  - `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
  - Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
  - By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.

- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps.

Reviewed By: jbeshay

Differential Revision: D48724715

fbshipit-source-id: 0230ea3feea525fa15b908161f444f4f6d9d4b39
2023-11-05 19:58:46 -08:00
Timm Böttger
5fca38ef00 Use MaybeManagedPtr for QuickSocket for optional lifetime management
Summary:
`connSetupCallback_` and `connCallback_` in `QuicTransportBase.h` are raw pointers, which delegates the responsibility to keep these callbacks alive to the caller.

There are use cases where it would be convenient to be able to tie the lifetime of the callback to the Quic transport, e.g,. as long as the Quic transport is alive, it keeps the callbacks alive as well.

This diff uses MaybeManagedPtr to achieve this lifetime tie if desired. A MaybeManagedPtr intialized with a shared pointer manages lifetime of the contained object, whereas a MaybeManagedPtr initialized with a raw pointer does not manage lifetime of the contained object. This way caller can decide to pass in a shared ptr or raw pointer and achieve the desired behavior.

Note that we cannot simply use a shared_ptr for that. Using a shared_ptr would potentially mean that callbacks passed are destroyed when the transport is destroyed. Callbacks would not be destroyed if they were managed by a shared_ptr already, but this is something we cannot assume for every case. This would thus be a change in semantics to the current implementation, where the callbacks can outlive the transport.

Reviewed By: mjoras

Differential Revision: D49502381

fbshipit-source-id: 771a9328b99dc4f94f8e9679f9caf98af9180428
2023-11-03 09:37:21 -07:00
Dan Melnic
e4f9973008 Try to fix a compile error
Summary: Try to fix a compile error

Reviewed By: avasylev

Differential Revision: D50066567

fbshipit-source-id: 4a774219f9badfd8c470809bec2ef44b84031828
2023-10-09 18:54:29 -07:00
Andrii Vasylevskyi
c0e008c266 Folly SocketOptionMap with strings - separate cmsgs
Summary: Splitting cmgs from socket options.

Reviewed By: mjoras

Differential Revision: D49557451

fbshipit-source-id: fa3005170d049557d1c845fd14b0ada73b58b377
2023-10-01 11:33:09 -07:00
Brandon Schlinker
a1445434b0 Cleanup and modularize receive path, improve timestamp support [5/x]
Summary:
This diff changes `QuicAsyncUDPSocketWrapper` so that it is an abstraction layer that inherits from `QuicAsyncUDPSocketType`, instead of simply being a container with aliases.
- Key changes in `QuicAsyncUDPSocketWrapper.h`, the rest of the updates switch us from using `QuicAsyncUDPSocketType` to `QuicAsyncUDPSocketWrapper`.
- It's difficult to mock the UDP socket today given that we expose the entire `folly::AsyncUDPSocket` type to the higher layers of the QUIC stack. This complicates testing and emulation because any mock / fake has to implement low level primitives like `recvmmsg`, and because the `folly::AsyncUDPSocket` interface can change over time.
- Pure virtual functions will be defined in `QuicAsyncUDPSocketWrapper` in a follow up diff to start creating an interface between the higher layers of the mvfst QUIC stack and the UDP socket, and this interface will abstract away lower layer details such as `cmsgs` and `io_vec`, and instead focus on populating higher layer structures such as `NetworkData` and `ReceivedPacket` (D48714615). This will make it easier for us to mock or fake the UDP socket.

This diff relies on changes to `folly::MockAsyncUDPSocket` introduced in D48717389.

--

This diff is part of a larger stack focused on the following:

- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
  - The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
  - The client currently has three receive paths, and none of them are well tested.

- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
  - `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
  - Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
  - By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.

- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps.

Reviewed By: jbeshay, hanidamlaj

Differential Revision: D48717388

fbshipit-source-id: 4f34182a69ab1e619e454da19e357a6a2ee2b9ab
2023-09-21 07:57:58 -07:00
Brandon Schlinker
086822ca76 Cleanup and modularize receive path, improve timestamp support [2/x]
Summary:
This diff:
- Changes `NetworkDataSingle` to have `ReceivedPacket` instead of `Buf`, in line with earlier change to `NetworkData` in D48714615

--

This diff is part of a larger stack focused on the following:

- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
  - The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
  - The client currently has three receive paths, and none of them are well tested.

- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
  - `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
  - Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
  - By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.

- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps.

Reviewed By: mjoras

Differential Revision: D48714796

fbshipit-source-id: d96c2abc81e7c27a01bcd0dd552f274a0c1ede26
2023-09-21 07:57:58 -07:00
Brandon Schlinker
a2c43bcf4c Cleanup and modularize receive path, improve timestamp support [1/x]
Summary:
This diff:
- Changes `NetworkData` to have `vector<ReceivedPacket>` instead of `vector<IOBuf>` to make it easier to associate metadata with individual UDP packets.

--

This diff is part of a larger stack focused on the following:

- **Cleaning up client and server UDP packet receive paths while improving testability.** We currently have multiple receive paths for client and server. Capabilities vary significantly and there are few tests. For instance:
  - The server receive path supports socket RX timestamps, abet incorrectly in that it does not store timestamp per packet. In comparison, the client receive path does not currently support socket RX timestamps, although the code in `QuicClientTransport::recvmsg` and `QuicClientTransport::recvmmsg` makes reference to socket RX timestamps, making it confusing to understand the capabilities available when tracing through the code. This complicates the tests in `QuicTypedTransportTests`, as we have to disable test logic that depends on socket RX timestamps for client tests.
  - The client currently has three receive paths, and none of them are well tested.

- **Modularize and abstract components in the receive path.** This will make it easier to mock/fake the UDP socket and network layers.
  - `QuicClientTransport` and `QuicServerTransport` currently contain UDP socket handling logic that operates over lower layer primitives such `cmsg` and `io_vec` (see `QuicClientTransport::recvmmsg` and `...::recvmsg` as examples).
  - Because this UDP socket handling logic is inside of the mvfst transport implementations, it is difficult to test this logic in isolation and mock/fake the underlying socket and network layers. For instance, injecting a user space network emulator that operates at the socket layer would require faking `folly::AsyncUDPSocket`, which is non-trivial given that `AsyncUDPSocket` does not abstract away intricacies arising from the aforementioned lower layer primitives.
  - By shifting this logic into an intermediate layer between the transport and the underlying UDP socket, it will be easier to mock out the UDP socket layer when testing functionality at higher layers, and inject fake components when we want to emulate the network between a mvfst client and server. It will also be easier for us to have unit tests focused on testing interactions between the UDP socket implementation and this intermediate layer.

- **Improving receive path timestamping.** We only record a single timestamp per `NetworkData` at the moment, but (1) it is possible for a `NetworkData` to have multiple packets, each with their own timestamps, and (2) we should be able to record both userspace and socket timestamps.

Reviewed By: mjoras

Differential Revision: D48714615

fbshipit-source-id: b21a88f14156ed6398308223f50b2334328586ee
2023-09-21 07:57:58 -07:00
Ilango Purushothaman
189d63ad2c Disable Ack Receive Timestamps based on hostname
Summary:
Added the capability to disable Ack receive timestamps based on hostname.
We want to disable timestamps on dynamic content (graph.facebook.com, i.instagram.com etc) to reduce network and resource overhead (thereby saving uplink BW). The hostnames can be specified via a MC. Timestamps should still be enabled for hostnames serving static content (like video, fbcdn...)

We have observed upto 40% more duplicate timestamps being sent on ACKs and ACK packet size has increased by more than 30% on average for such cases. This is a first step in reducing ACK wastage.

(Note: this ignores all push blocking failures!)

Reviewed By: jbeshay

Differential Revision: D48812624

fbshipit-source-id: aa72ec2dcc41ebe9d982364d8d45739e062daddb
2023-09-12 07:47:19 -07:00
Ashkan Nikravesh
4862982004 throttling-detector: create an interface to provide throttling signals to CCAs
Summary: An interface which will be extended by throttling detectors and will provide throttling signal to CCAs. This interface can be set through `QuicSocket` and can be accessed via `QuicConnectionStateBase`.

Reviewed By: silver23arrow, jbeshay

Differential Revision: D47027498

fbshipit-source-id: 6a2b9ee51ca54395e10500dd7c9787968cc281d4
2023-08-30 17:01:06 -07:00
Joseph Beshay
32e07e022c Ensure the pacing rtt factor is not applied for bbr2
Summary: As title.

Reviewed By: mjoras

Differential Revision: D48608150

fbshipit-source-id: 52bc0d6aa3938bfb7f217874ce70a609b2ca31fc
2023-08-24 17:05:28 -07:00
Joseph Beshay
6e3cddc5a4 Ignore incoming knob frames if we haven't advertised knob support
Summary: Only accept knob frames if support has been advertised.

Reviewed By: hanidamlaj

Differential Revision: D48329363

fbshipit-source-id: 3f06574415d95a2129e5d776b7a4f4808791ed45
2023-08-16 10:24:01 -07:00
Joseph Beshay
7059e2f13e Log the congestion controller type to qlog
Summary: With bbr, bbr_testing, and bbr2 there is no way to tell the difference between qlog traces especially during Startup. This explicitly logs whenever the congestion controller is created or changed.

Reviewed By: hanidamlaj, mjoras

Differential Revision: D48194475

fbshipit-source-id: 30ba5662e0fe4aa2ccb4b28696049d8aadab38db
2023-08-10 20:03:02 -07:00
Joseph Beshay
0ef984e147 Use BBRTesting to test experimental pacer settings
Summary: Change BBRTesting to test the existing BBR CCA when replacing the pace scaling factor by the experimental pacing improvements.

Reviewed By: kvtsoy

Differential Revision: D48120180

fbshipit-source-id: 6b99754ae0f8cb7a76596243a112ddd500f6fcc8
2023-08-10 20:03:02 -07:00
Joseph Beshay
502d26d19b Do not attempt to configure the pacer before it's allocated
Summary: When BBR2 is configured on the server side, the pacer may not be allocated yet when setCongestionControl is called.

Reviewed By: hanidamlaj

Differential Revision: D47881213

fbshipit-source-id: ddb695dbb5d5d8c07ccef5e7282babdb661482b3
2023-07-28 18:43:20 -07:00
Konstantin Tsoy
9823ea87e2 Add bufAccessor to the client
Summary: Add bufAccessor to the client

Reviewed By: jbeshay, mjoras

Differential Revision: D47649494

fbshipit-source-id: e14049d3556c01a3c61b9264d43e1304d781232d
2023-07-27 17:00:41 -07:00
Joseph Beshay
d85f726e36 Consolidate logic for setting the congestion controller in one function
Summary: Ensure all CCA-related settings are included in setCongestionControl(), and that it's called from the knob setting the congestion controller.

Reviewed By: kvtsoy

Differential Revision: D47741830

fbshipit-source-id: ff1d2347581c61a58f2caaff8189126930bf4e04
2023-07-25 19:48:01 -07:00
Matt Joras
80fd271483 Default max nexts to 5 for DSR.
Summary: This has empirically shown good results in experiments with no downside. Only set it if the transport setting hasn't been overridden already.

Reviewed By: jbeshay

Differential Revision: D47447380

fbshipit-source-id: 294582544b6e42d7f6c46d1d810228a0b5fad9e8
2023-07-17 12:12:45 -07:00
Konstantin Tsoy
401f72867a Use QuicBackingEventBase instead of folly::EventBase
Summary: Use QuicBackingEventBase instead of folly::EventBase

Reviewed By: mjoras

Differential Revision: D47135548

fbshipit-source-id: c779d66a044bddb61586e2603a7d4bb4e96ac1a0
2023-07-13 18:39:45 -07:00
Konstantin Tsoy
4b83c1477f Do not call to evb observer for mvfst mobile
Summary: Will not be doing evb observers for now for mvfst mobile

Reviewed By: hanidamlaj, mjoras

Differential Revision: D47135550

fbshipit-source-id: f18f50b91f1575b0d4f4874f52c72fb6a7d5c1e8
2023-07-13 18:39:45 -07:00
Konstantin Tsoy
fe12ab7241 Hide direct timer invocations on the event base into the wrapper
Summary: Needed for future libev-based event base impl.

Reviewed By: hanidamlaj

Differential Revision: D46670916

fbshipit-source-id: 55740ed05da9c1f54cc8639ece89d1f21aaff0d3
2023-07-12 17:26:15 -07:00
Konstantin Tsoy
4a0dd1e2a4 QuicAsyncUDPSocketWrapper
Reviewed By: jbeshay

Differential Revision: D46379200

fbshipit-source-id: f6a7c1cf68108872e05e6fd8adb7f00aae22b2ed
2023-07-11 15:21:15 -07:00
Joseph Beshay
5426bc08a8 Add BBRv2 Congestion Controller
Summary:
This implements the BBRv2 congestion controller for Mvfst. This implementation aims at staying as close as possible to the naming and organization used in the latest draft. Wherever there are gaps in the spec or the pseudocode, this attempts to fill them with reasonable assumptions.

https://datatracker.ietf.org/doc/html/draft-cardwell-iccrg-bbr-congestion-control-02

Besides the spec differences, this implementation differs from Mvfst's BBRv1 as follows. These differences may be revisited in the future:
- The pacing rate is exclusively controlled by the CCA. No further scaling of the pacer rate is enforced. This enables BBRv2 control loop to keep tight control over queueing delay.
- BBRv2 does not rely on an external bandwidth or RTT sampler. Mvfst's BBRv1 decouples these which makes tracking the state a bit harder.

Reviewed By: mjoras

Differential Revision: D44599226

fbshipit-source-id: 1e488bd936db5826b73ff6205f988dd40e908eba
2023-06-22 13:56:23 -07:00
Joseph Beshay
931abc64af Separate transport settings for pacing tick and the pacing timer resolution
Summary:
The pacingTimerTickInterval transport setting conflates two options: the minimum interval a pacer can use, and the resolution of the underlying timer. This means that a higher value leads to lower timer resolution and less pacing accuracy.

This change adds a separate parameter for the timer resolution to ensure that a larger pacing tick does not degrade the pacer accuracy.

Reviewed By: mjoras

Differential Revision: D46564066

fbshipit-source-id: 0d0e54077b80da03e6e6c9baaab49a4c969966b6
2023-06-20 15:18:38 -07:00
Konstantin Tsoy
dccfc706b5 QuicEventBase wrapper
Summary:
Create and use an actual wrapper around folly::EventBase.
Later an interface will be added that the wrapper will be implementing.

Reviewed By: jbeshay

Differential Revision: D45822401

fbshipit-source-id: 3b33f796c31043ec2881b753a9b60943bdf91f1d
2023-06-15 17:12:24 -07:00
Matt Joras
3c5a34ee11 Put opportunistic ACKing behind TransportSetting
Summary:
This has been a default behavior for us for a long time. Basically, we will always include ACK frames in a burst if there's new packets to ACK.

This is overall probably fine and is flexible to peer behavior such that the peer can get away with basically never sending anything ACK-eliciting.

However it is not without its issues. In particular, for DSR it means that we write an excessive amount of pure ACK packets since they cannot be coalesced with a DSR burst.

Make this behavior optional, such that we only include ACK frames in non-probing schedulers when we are only writing stream data.

Reviewed By: kvtsoy

Differential Revision: D39479149

fbshipit-source-id: 6e92d45311a3fb23750c93483b94e97dd3fdce26
2023-06-15 13:15:08 -07:00
Crystal Jin
0fd6ba7f78 Use finalWriteOffset for stream bytes sent
Reviewed By: JunqiWang

Differential Revision: D46663584

fbshipit-source-id: 1b22eb90501a9b904c17818991d01e5a0fbf2734
2023-06-13 08:16:16 -07:00
Joseph Beshay
004c656d3c Adjust minCwnd value for Cubic to respect the one from transport settings
Summary: The minCwnd should not be overwritten by the initCwnd. This affects Cubic minCwnd only. BBR uses its constants directly.

Reviewed By: mjoras

Differential Revision: D46076172

fbshipit-source-id: 63d0be96334e6ca294cdf89bc7545aaa4b960678
2023-05-22 19:25:55 -07:00
Joseph Beshay
a55b14d0f0 Rename QUIC lossbyPto to lossByTimeout to avoid confusion with PTO (probe timeout)
Summary: Classifying loss as `byPto` is confusing because it is not related to Probe Timeouts (PTO). This changes the occurrences of this name in the transport code to `byTimeout`.

Reviewed By: mjoras, kvtsoy

Differential Revision: D45788216

fbshipit-source-id: 9900b928d3f44572a21d6ed7e76be658e21451c7
2023-05-16 12:37:47 -07:00
Matt Joras
e41ea339f8 Don't close via exception on receiving a close.
Summary: This is a dumb shortcut. Just check whether it's happened after reading data and call close that way.

Reviewed By: kvtsoy

Differential Revision: D45205285

fbshipit-source-id: 57edafc50f5e5e9cdaf9ac7e46781b762acd30d8
2023-04-24 13:59:02 -07:00
Paul Farcasanu
e56d4bd7f6 refactor setStreamPriority to accept Priority struct
Summary:
## Context
Context: see summary for D43854603.

## In this diff
In this diff, make way for easily adding new elements to Priority struct.

Reviewed By: afrind

Differential Revision: D43882907

fbshipit-source-id: f74f6ec41162a970dcd666bfa5192676f968d740
2023-03-28 16:00:38 -07:00
Joseph Beshay
24f00aad4a Allow Packet Processors to specify prewrite requests that apply to each write loop
Summary:
Adds a prewrite function in packet processors that can be used to set PreWrite Reuqests that will apply apply to the next write loop in the QuicSocket.

Currently, the PrewriteRequest only contains a list of cmsgs but it can be expanded when needed.

This replaces the setAdditionalCmsgs callback that was previously passed down directly to AsyncUDPSocket, allowing multiple processors to affect the additionalCmsgs sent to the AsyncUDPSocket.

Reviewed By: bschlinker

Differential Revision: D44233700

fbshipit-source-id: 799357ee5df89b2a526611bd83d0b4a9a13b71d5
2023-03-24 16:14:59 -07:00
Hani Damlaj
01b439f2f2 remove QuicConnectionStateBase::logger references
Summary: - purging references to unused variable

Reviewed By: kvtsoy

Differential Revision: D43857376

fbshipit-source-id: bfd721efb21ba966496c0131fb12d2357f620ed4
2023-03-15 15:58:02 -07:00
Crystal Jin
b78833805a Expose stream bytes received in StreamTransportInfo
Reviewed By: hanidamlaj

Differential Revision:
D43921381

Privacy Context Container: L1127813

fbshipit-source-id: f2e659c602e792b27617b75e1eb1f4d9841138f9
2023-03-09 11:09:02 -08:00
Crystal Jin
9dab1046b1 Propagate stream bytes sent up stack for logging
Reviewed By: hanidamlaj

Differential Revision:
D43852121

Privacy Context Container: L1071327

fbshipit-source-id: 7cc94ac52667d1757cb1d0aa150ccbc6562eaf09
2023-03-08 15:39:11 -08:00
Joseph Beshay
58393be7e4 Expose per-write dynamic Cmsgs in QuicTransportBase
Summary: Allow setting the additionalCmsgsFunc in the underlying AsyncUDPSocket

Reviewed By: mjoras, kvtsoy

Differential Revision: D43681325

fbshipit-source-id: 5ebe6c7b81ea2f5f4aa9715deb590736ebd20544
2023-03-07 15:40:46 -08:00
Joseph Beshay
cd5aba8254 Add a transport setting to enable pacing before the 1-RTT keys are available
Summary: As title

Reviewed By: hanidamlaj

Differential Revision: D43773469

fbshipit-source-id: d57ab4de0bea6947e98797d35f84d086f3f900e8
2023-03-03 21:42:10 -08:00