Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14807
The "counts" array in CompactionServiceResult (one compaction count per CompactionReason) is non-critical information. During cross-process remote compaction the primary and the remote worker can run different RocksDB versions and therefore disagree on kNumOfReasons, so the serialized counts array can have a different length than the reader expects. The plain OptionTypeInfo::Array parser rejects any element-count mismatch, which fails deserialization of the entire result over this non-critical field.
This makes deserialization of "counts" size-tolerant via SizeTolerantDeserializeArray, which overrides only the parse callback: extra trailing elements are dropped and missing trailing elements are zero-filled, so a length mismatch no longer fails deserialization. Serialization and comparison are inherited from Array<T, kSize> unchanged.
The serialized width is intentionally kept at kNumOfReasons - 1 here, so this change does not alter what is written -- only the read path becomes tolerant. Widening the serialized width to the full kNumOfReasons is done as a separate follow-up change, which must not land until this tolerant read has been deployed to every primary (including rollback targets), so that no reader still rejects a wider result.
== Test plan ==
**Legend**
CompactionReason is an enum whose last entry is kNumOfReasons (currently 21). CompactionStats::counts is an int[kNumOfReasons] holding one count per reason; it is serialized into the remote compaction result and read back by the primary.
- "narrow" = kNumOfReasons - 1 counts (omits the newest reason); what current code writes.
- "wide" = kNumOfReasons counts (includes the newest reason).
- strict reader: accepts only its exact expected element count; any other count fails deserialization (the plain OptionTypeInfo::Array behavior).
- tolerant reader: accepts any count -- more elements than expected are dropped, fewer are zero-filled; never fails on a count mismatch (this change, SizeTolerantDeserializeArray).
**Unit test**
CompactionServiceTest.CompatCheckCountsWidthTolerance -- verifies that deserializing "counts" tolerates a serialized element count equal to, one greater than, or one less than the reader's expected width (extras dropped, missing zero-filled), and that the binary serializes exactly the expected number of elements.
**Manual cross-version verification (ldb remote_compaction)**
Test 1: baseline (unchanged behavior) -- expect-narrow strict-read primary reads narrow counts written by a worker
```
$ strict_narrow remote_compaction_worker --db=$DB --job_dir=$JOB &
$ strict_narrow remote_compaction_primary --db=$DB --job_dir=$JOB
remote_compaction_worker: OK (4913 bytes result)
remote_compaction_primary: OK
$ grep 'remote compaction result' $DB/LOG
[default] [JOB 3] Received remote compaction result, output path: /tmp/cs_run/output, files: 000022.sst
```
=> PASS: the primary parsed and installed the remote result.
Test 2: expect-narrow tolerant-read primary reads the narrow counts a worker writes
```
$ strict_narrow remote_compaction_worker --db=$DB --job_dir=$JOB &
$ tolerant_narrow remote_compaction_primary --db=$DB --job_dir=$JOB
remote_compaction_worker: OK (4912 bytes result)
remote_compaction_primary: OK
$ grep 'remote compaction result' $DB/LOG
[default] [JOB 3] Received remote compaction result, output path: /tmp/cs_run/output, files: 000022.sst
```
=> PASS: the primary parsed and installed the remote result.
Test 3: expect-narrow tolerant-read primary reads the wide counts a worker writes (extra reason dropped)
```
$ tolerant_wide remote_compaction_worker --db=$DB --job_dir=$JOB &
$ tolerant_narrow remote_compaction_primary --db=$DB --job_dir=$JOB
remote_compaction_worker: OK (4915 bytes result)
remote_compaction_primary: OK
$ grep 'remote compaction result' $DB/LOG
[default] [JOB 3] Received remote compaction result, output path: /tmp/cs_run/output, files: 000022.sst
```
=> PASS: the primary parsed and installed the remote result.
Test 4: expect-wide, tolerant primary reads the narrow counts a worker writes (missing reason zero-filled)
```
$ strict_narrow remote_compaction_worker --db=$DB --job_dir=$JOB &
$ tolerant_wide remote_compaction_primary --db=$DB --job_dir=$JOB
remote_compaction_worker: OK (4911 bytes result)
remote_compaction_primary: OK
$ grep 'remote compaction result' $DB/LOG
[default] [JOB 3] Received remote compaction result, output path: /tmp/cs_run/output, files: 000022.sst
```
=> PASS: the primary parsed and installed the remote result.
Test 5: expect-wide, tolerant primary reads a wide counts a worker write
```
$ tolerant_wide remote_compaction_worker --db=$DB --job_dir=$JOB &
$ tolerant_wide remote_compaction_primary --db=$DB --job_dir=$JOB
remote_compaction_worker: OK (4917 bytes result)
remote_compaction_primary: OK
$ grep 'remote compaction result' $DB/LOG
[default] [JOB 3] Received remote compaction result, output path: /tmp/cs_run/output, files: 000022.sst
```
=> PASS: the primary parsed and installed the remote result.
Reviewed By: jaykorean
Differential Revision: D106321320
fbshipit-source-id: fc1bd66a4d11ac6bf38dff6aa214ebafab75f3d0
7c80a5ac85