Summary: - adding an API to the QuicSocket to enable users to synchronously query the socket for the maximum amount of data that should be written.
Reviewed By: mjoras
Differential Revision: D37390086
fbshipit-source-id: 5dd64e28b7f841ba3e28a4bc2a79c1b1e5a95047
Summary:
These functions are used to determine when the TX/delivery callbacks are delivered. If we don't consider the writeBufMeta lengths then these callbacks will never be delivered for DSR streams, causing the stream to never destruct.
(Note: this ignores all push blocking failures!)
Reviewed By: kvtsoy
Differential Revision: D36552031
fbshipit-source-id: fe6c0de1afc7c93d158294d965db7701639499fb
Summary: The code has changed since this was made an inlined vector. We now see higher cost from copy operations due to this rather than being able to move the data around.
Reviewed By: bschlinker
Differential Revision: D36151209
fbshipit-source-id: 0b10558c6bd8ebfea9bb960aac36a6c4044fc95f
Summary:
This diff is part of larger change to switch to using `folly::ObserverContainer` (introduced in D27062840).
This diff:
- Changes `LegacyObserver` to inherit from the new `Observer` class by adding a compatibility layer. This compatibility layer enables existing observers to continue to be supported.
- Changes generation of observer events so that they are routed through `ObserverContainer::invokeInterfaceMethod`
- Temporarily removes some of the reentrancy protection that previously existed, as it was not being applied consistently — some events had reentrancy protection, some did not. This will be reintroduced in the next diff.
- Improves some unit tests for observers during the transition process.
Differential Revision: D35268271
fbshipit-source-id: 5731c8a9aa8da8a2da1dd23d093e5f2e1a692653
Summary:
This diff is part of larger change to switch to using `folly::ObserverContainer` (introduced in D27062840).
This diff:
- Renames `quic::Observer` to `quic::LegacyObserver`.
- Switches to using `quic::SocketObserverInterface` instead of `quic::Observer` when referencing structures used for the observer interface. For instance, `quic::Observer::WriteEvent` changes to `quic::SocketObserverInterface::WriteEvent`.
Differential Revision: D35000152
fbshipit-source-id: f387231fd58e8e763c3d7ed0a9c6fcec3b2324e2
Summary: Extend `PacketsWrittenEvent` to include the CWND and number of writable bytes from the congestion controller after the write completed. If no congestion controller is installed, these values are left blank.
Differential Revision: D32588533
fbshipit-source-id: 4b8361937fcab36daa17e5f6dc4386762987d2b4
Summary:
QE result post: https://fb.workplace.com/groups/646964542156536/permalink/1989360141250296/
We are going ahead with shipping this, with a default value of 32 for paddingModulo.
Leaving the transport knob settings still in place in case vips want to selectively disable the feature (not sure if that is the proper way to config that but I think so it is?)
Reviewed By: mjoras
Differential Revision: D34596608
fbshipit-source-id: 5603bb391113c29830f43a67e73c0f50154dcca1
Summary: Report all bytes written in observer packets written events.
Reviewed By: jbeshay
Differential Revision: D33086404
fbshipit-source-id: 1456cf23a420abd025f047f190cb2b8a9868826e
Summary:
Maine change is `MockConnectionCallback` -> `MockConnectionSetupCallback` + `MockConnectionCallbackNew`.
Everything else is changing tests to use the two new classes.
Differential Revision: D33076321
fbshipit-source-id: a938b63ce59f07f549b3e725caad8785348db7ed
Summary:
This is a bug that could prevent us from writing data if we ran out of connection flow control while we had lost data.
The last attempt missed a mistake in the scheduling of sequential priority streams.
Reviewed By: kvtsoy
Differential Revision: D33030784
fbshipit-source-id: e1b82234346a604875a9ffe9ab7bc5fb398450ed
Summary:
Fixing four issues in open source build.
First, two missing changes to `CMakeLists.cpp` changes
- `handshake/TokenGenerator.cpp`, added in D31673160 (7233c55d29)
- `state/AckEvent.cpp`, added in D31221302 (7a916869ff)
The two two issues are caused by inconsistencies between our build env and open source build env, including gtest versions. We should investigate getting open source gtest on same version.
---
**[D31216724 (20c17a882b)] We cannot use lambdas with `testing::ResultOf` for older versions of gtest and/or older compilers.**
```
gmock-matchers.h:2310:29: error: no type named 'result_type' in '(lambda at /data/sandcastle/temp/fbcode_builder_getdeps/shipit/mvfst/quic/api/test/QuicTransportTest.cpp:711:19)'
```
In particular, the `decltype(f(arg))` logic in the following code from gtest is going to fail on the lambda, since a lambda has no type. I believe it would need to be a `declval(decltype(f(arg)))` for the lambda to work.
```
template <typename Functor>
struct CallableTraits {
typedef Functor StorageType;
static void CheckIsValid(Functor /* functor */) {}
template <typename T>
static auto Invoke(Functor f, const T& arg) -> decltype(f(arg)) {
return f(arg);
}
};
```
See this for details: https://stackoverflow.com/questions/4846540/c11-lambda-in-decltype
-----
**[D32772165 (e784fafb10)] `EXPECT_CALL` does not work with the formatting used for older versions of gtest and/or older compilers (again, unclear).**
Specifically
```
EXPECT_CALL(handshakeFinishedCallback, onHandshakeFinished);
```
must instead be
```
EXPECT_CALL(handshakeFinishedCallback, onHandshakeFinished());
```
Reviewed By: kvtsoy
Differential Revision: D33026131
fbshipit-source-id: 886a6165f2e217cadbe479320195abbd4895eb7c
Summary: This is causing empty write loops for some reason.
Reviewed By: jbeshay, lnicco
Differential Revision: D32990291
fbshipit-source-id: 2a183d591de54c7fe0ca54aea828a697a4cd9af0
Summary: This is an edge case we weren't handling properly. This can lead to situations where we run out of conn flow control, the peer hasn't received all data, and then we never write the data to trigger a flow control update.
Reviewed By: afrind
Differential Revision: D32887140
fbshipit-source-id: df5dfad8e0775ef43e5ca6ab98b8ca6b5987ce31
Summary:
Currently the `packetsWritten` observer event is only triggered when ACK-eliciting packets are written to the network. This change makes it so that the event is triggered even when non ACK-eliciting packets are written. In addition, this change increases the information provided in the observer event to reduce the processing that must be performed by observers.
More specifically, this diff makes the following changes:
- It creates two new structs — `WriteEvent` and `PacketsWrittenEvent` — and makes `PacketsWrittenEvent` and `AppLimitedEvent` inherit from `WriteEvent`, which contains the base set of fields that appear in both derived events. The base set of fields includes `writeCount` and the reference to the list of `OutstandingPackets`.
- A `PacketsWrittenEvent` is generated after each write loop during which packets are written.
- The `PacketsWrittenEvent` records the total number of packets written (the sum of ACK-eliciting and non ACK-eliciting packets written) and the total number of ACK-eliciting packets written during the last loop. By exposing this information, observers can determine whether there is value in inspecting the list of `OutstandingPackets` before doing so.
- In the future, I'll extend this event to also report the number of stream bytes written.
- It adopts a new builder pattern for observer events to make it easier to create instances of these structures:
- We use this approach instead of aggregate initialization (specifically designated initializers via aggregate initialization), because designated initializers (C++20 feature, although supported by some compilers long before) is not yet supported on all platforms for which we need this code to compile.
- I intend to change the other observer events to use this pattern as the number of arguments in some of their constructors makes them increasingly difficult to deal with.
Reviewed By: mjoras
Differential Revision: D31216724
fbshipit-source-id: 42ceb922d46ab0e0dc356f8726c2d0467a219587
Summary: Provide observers with visibility into stream events triggered by local and peer (e.g., new stream opened locally or by peer).
Reviewed By: mjoras
Differential Revision: D31886978
fbshipit-source-id: 7556fef0f336bd0f190b4474f1a7b0120aae6ef1
Summary: As titled. We currently allow `QuicSocket::Observer::close` to be called multiple times. Only call it once.
Reviewed By: lnicco
Differential Revision: D31886995
fbshipit-source-id: 61d959e8575b08c34ef896b87da0cf8ada482144
Summary: Do not call onAppRateLimited() before connection is ready
Reviewed By: bschlinker
Differential Revision: D30563194
fbshipit-source-id: dd8c2da656041f9219a645935b6c4c3c85b27816
Summary: This solves test flake that resulted from using EXPECT_NEAR with varying timing on devservers and CI situations
Reviewed By: yangchi
Differential Revision: D29574796
fbshipit-source-id: 804935e8e0148fcccec642a1894e8863bee4465b
Summary:
Add a new QuicSocket API that sets the maximum pacing rate in Bytes per second to be used if pacing is enabled. This setting is passed down to the Pacer which enforces it.
The change also includes:
- Refactoring the setPacingRate function of the Pacer to make it more consistent with the other refreshPacingRate function.
- A new flag for testing the MAX_PACING_RATE using tperf.
Reviewed By: mjoras
Differential Revision: D29276789
fbshipit-source-id: 818d86707084b2697f7417b4a47e62cbbce65c73
Summary: Remove a check that only allowed getting and setting the receive window on writable streams. This had prevented a receiver from getting/setting the receive window of a unidirectional stream initiated by the peer.
Reviewed By: mjoras
Differential Revision: D29366097
fbshipit-source-id: 3bc5cba826663b3499c19fe26bdd9a070682e533
Summary:
Another bug around offset when DSR is used. When frontend sends reset,
it should use writeBufMeta.offset for reset offset if some bytes have been sent
via the DSR path.
Reviewed By: mjoras
Differential Revision: D28791467
fbshipit-source-id: d563964504a5d55ba584f5c90a0883c6900084b4
Summary:
Previously we were using a single flag appDataSentEvents to enable 3 callbacks
(startWritingFromAppLimited, packetsWritten and appRateLimited). This commit
creates separate flags to enable each of these callbacks, we have use-cases
where Observers might want only packetsWritten events and not the others.
Also, this commit refactors the logic to deliver these 3 callbacks to all
observers into a separate method so they can be unit tested easily.
Reviewed By: bschlinker
Differential Revision: D27925011
fbshipit-source-id: 7f7436dfc3d50c3abcb8ec121b221d20e30b0c4b
Summary:
The CHECK is too harsh today. It blocks any write if BufferMeta has
been written to a Quic stream. Writing a pure EOM should be allowed. If caller
wants to write just an EOM, it should be free to choose either writeChain or
writeBufMeta API.
(Note: this ignores all push blocking failures!)
Reviewed By: mjoras
Differential Revision: D27684033
fbshipit-source-id: 475c65c7eac110ca6eb2a8c2abaf1eb00628a492
Summary:
This assigns a DSRPacketizationRequestSender to a QUIC stream, and let
it own it.
Reviewed By: mjoras
Differential Revision: D27668523
fbshipit-source-id: 4beba6ddf247801368c3e1c24a0a4956490d45cd
Summary:
As before we will now aggressively send probes on all spaces with probes available when the PTO timer fires.
This time with more unit tests and some bug fixes.
Reviewed By: yangchi
Differential Revision: D27338523
fbshipit-source-id: 8a9ccb90ed691e996fab4afa2f132c0f99044fbc
Summary:
This commit demonstrates the use of ByteEvents in maps. Intentionally didn't define the custom comparator and hash function inside the quic library because we want consumers of ByteEvents to define equality of 2 ByteEvents in their own way.
This will help recipients of ByteEventCallback keep track of pending,
received and cancelled ByteEvents easily.
In addition, the behavior of registerByteEventCallback was modified to NOT allow duplicate registrations. A pair of registrations is considered duplicate if they both have the same stream ID, stream offset, ByteEvent type and recipient callback pointer. In such cases, registerByteEventCallback returns an INVALID_OPERATION error.
This is critical to make sure that ByteEvents work well in maps, which may be used by recipients.
Reviewed By: bschlinker
Differential Revision: D27100623
fbshipit-source-id: 084a4765fa6c98fdc1b98414fbd30582cf1e5139
Summary: As in title. There's a bug here somewhere with empty write loops we need to find.
Reviewed By: yangchi
Differential Revision: D27279100
fbshipit-source-id: e1d26fbf8d6df1590d464a6504a8b940b46794e0
Summary:
Previously we would only send probes for the first space which had one available, i.e. Initial before Handshake before AppData. Since we only have one PTO timer this can lead to situations where we perpetually probe with only Initials, which can significantly delay the handshake if we should have probed with Handshakes.
With this diff we will keep the single PTO timer but aggressively write more probes from all spaces if they are available.
Additionally this refactors some counters into EnumArrays
Reviewed By: yangchi
Differential Revision: D27235199
fbshipit-source-id: ef3614a833bf0f02f5806846a1335fa7ac2a4dc8
Summary:
Add an onByteEventRegistered callback that will be invoked when the
registration for a ByteEvent callback is successful.
This makes it easy for recipients of the callback to track the successful
registration AND callback recipt in a single place (the recipient class). For
example, if all successfully registered ByteEvents have been received, the
recipient can destroy itself safely, knowing that no more ByteEvents will be
sent to it.
Reviewed By: bschlinker
Differential Revision: D27100624
fbshipit-source-id: dbfeff1f4cf2367587fdb73cbd334165b3b159de
Summary:
Previously, we only had support for notifying observers when a QUIC socket
became application rate limited.
This change adds a similar notification when a socket does not become
application rate limited, i.e when the application *starts* writing data to the
socket - either for the first time on a newly established socket or after a
previous block of writes were written and the app had no more to write.
In addition, we include the number of outstanding packets (only those that are
carrying app data in them) so that observers can use this data to timestamp the
start and end of periods where the socket performs app data writes.
Reviewed By: yangchi
Differential Revision: D26559598
fbshipit-source-id: 0a8df7082b83e2ffad9b5addceca29cc03897243
Summary: they need to be prioritized together
Reviewed By: mjoras
Differential Revision: D26918282
fbshipit-source-id: 061a6135fd7d31280dc4897b00a17371044cee60
Summary:
Instead of writing real data into the transport, we want to support a
use case where only its metadata is written to the transport. Sending of the
real data is delegated to another entity in such setup.
Reviewed By: mjoras
Differential Revision: D26131772
fbshipit-source-id: 4fcfa3a1626203f63c61898e6de089a3079d043d
Summary: As in title. This doesn't actually send any frames, but implements basic support for the transport parameter and responding to the frames.
Reviewed By: yangchi
Differential Revision: D26134787
fbshipit-source-id: 2c48e01084034317c8f36f89c69d172e3cb42278
Summary: this param is passed to transport then ignored
Reviewed By: avasylev
Differential Revision: D26133327
fbshipit-source-id: 459dd0132185513215ba034f213d4137d7b56ba1
Summary:
Given the large number of callbacks that are being triggered from the Observer
this change makes it possible to enable through a simple config, just the
subset of callbacks that a consumer is interested in receiving.
Observer and Socket Lifecycle callbacks are enabled by default, they are not
configurable.
Reviewed By: bschlinker
Differential Revision: D25879382
fbshipit-source-id: abe79ed92e958ddc96475c347f8ec7204400cdfa
Summary:
We were using the LifecycleObserver and InstrumentationObserver classes
separately, to generate and receive callbacks.
This change migrates both these to use the unified Observer callback class and
adjusts the unit tests.
Reviewed By: bschlinker
Differential Revision: D25845845
fbshipit-source-id: c489400f5d70bccadbcc1d957136c5ade36b65ff
Summary:
I think this should just work without the trailing `_E`. It was added
when we mixed up our own union based variant and boost::variant. Some compiler
flags didn't like that. Now we no longer have mixed up cases, this should be
fine
Reviewed By: lnicco
Differential Revision: D25589393
fbshipit-source-id: 6430dc20f8e81af0329d89e6990c16826da168b8
Summary: Adds a top level API to set stream priorities, mirroring what is currently proposed in httpbis. For now, default all streams to the highest urgency, round-robin, which mirrors the current behavior in mvfst.
Reviewed By: mjoras
Differential Revision: D20318260
fbshipit-source-id: eec625e2ab641f7fa6266517776a2ca9798e8f89
Summary: This was probably a premature optimization and introduces complexity for dubious gain. Additionally a sequence of losses could potentially cause multiple updates to be delayed.
Reviewed By: yangchi
Differential Revision: D23628058
fbshipit-source-id: d6cf70baec8c34f0209ea791dadc724795fe0c21
Summary:
There are situations where application needs to know how many packets were transmitted for a stream on certain byte range. This diff provides *some* level of that information, by adding the `cumulativeTxedPackets` field in `StreamLike`, which stores the number of egress packets that contain new STREAM frame(s) for a stream.
~~To prevent double-counting, I added a fast set of stream ids.~~
Application could rely on other callbacks (e.g. ByteEventCallback or DeliveryCallback) to get notified, and use `getStreamTransportInfo` to get `packetsTxed` for a stream.
Reviewed By: bschlinker, mjoras
Differential Revision: D23361789
fbshipit-source-id: 6624ddcbe9cf62c628f936eda2a39d0fc2781636
Summary: Implemented necessary changes to emit DATA BLOCKED frame when a QUIC connection is write blocked.
Reviewed By: mjoras
Differential Revision: D23067313
fbshipit-source-id: f80d7425c9a3c4e9b81405716bcd944c83b97ac2
