1
0
mirror of https://github.com/sqlite/sqlite.git synced 2026-10-28 10:57:26 +03:00
Commit Graph

596 Commits

Author SHA1 Message Date
drh
a7751a871b Fix a buffer overread that could occur in fts5 within a memcmp() when doing an integrity-check on corrupted records.
FossilOrigin-Name: f73adcfc87803f373b7c55ec280e3bcae4b9f3a1f3edc2ee21bf12c696a46be4
2026-06-26 14:11:19 +00:00
drh
b2de7df934 Fix an fts5 problem causing attempts to create locale=1 tables with more than 115 columns to fail.
FossilOrigin-Name: 4d993e8f5ffe4d34d4b1c579f9d50b60be6a3c6738e1c06d298e43b1222f7a66
2026-06-19 11:18:51 +00:00
drh
051745a6c3 Avoid a possible buffer overrun in 32-bit builds of fts5 that could occur when processing corrupt records.
FossilOrigin-Name: 3a4667ce36a1548fffff8bbf946d84d6bbd21e7abb216d1f37717de74ce7e2a7
2026-06-15 13:53:44 +00:00
drh
633e08da82 Fix a problem that could cause fts5 to return incorrect answers when performing prefix queries on tokens with synonyms.
FossilOrigin-Name: 4f21004f9b5fc9766183a3c4ac44cfeb6cd3e5583fef9176a26fecc2910f8c08
2026-06-15 11:52:31 +00:00
drh
a89c9f34d8 Fix a signed-integer overflow in fts5 that might occur when dealing with strategicly corrupted records.
FossilOrigin-Name: a36fc58e55c100f0c26e4426100c15287dc5613e06d5d1fb43845e3c7cda09df
2026-06-11 23:28:49 +00:00
drh
78a50f9e74 Fix four obscure yet separate problems in FTS5.
FossilOrigin-Name: a6eb055c35f091a8e864c1de3387d389e2bd3025c32596c0b7a78ebdf091898b
2026-06-08 19:23:56 +00:00
drh
6c1aa834f6 Fix another buffer overread in fts5 that could occur when processing
corrupt records.

FossilOrigin-Name: 8427cac76455ecbef224ceaccbb37b79338015bcfe67209ae462f36ffefa8466
2026-06-03 15:02:29 +00:00
drh
e7d405200a Fix another potential buffer overrun that could occur in fts5 when
processing corrupt records.

FossilOrigin-Name: e583a723fa501d0c53ad124b571bcd6a42d76daba1ea983d3a547763c2c94c22
2026-06-03 14:24:19 +00:00
drh
48aa50931a Fix a potential buffer overwrite that could occur when in fts5 when handling corrupt records.
FossilOrigin-Name: 02733fc4bc27df64b50ca5027ac94fc643146efb24bb4fdbcc70c45e1c58c919
2026-05-25 17:23:58 +00:00
drh
f034bd10f0 Fix a potential buffer overread in fts5 that could occur when handling
corrupt records.  Improved detection of corrupt fts5 records.

FossilOrigin-Name: fad98805b7d73abf0cf17919397ac50757a2b888354fcf42b72640a12f32926c
2026-05-20 10:01:25 +00:00
drh
0d8e4e27b7 Silently ignore nested porter tokenizers in FTS5. Having nested porter
tokenizers is pointless, but it does use stack space unnecessarily.

FossilOrigin-Name: 8207ca671e1253135e5e36cd123fdcfb4d1f0fcf939d2c94238425fafd38c75a
2026-05-20 09:36:42 +00:00
drh
e0b995b2a6 Fix potential buffer overwrite that could occur in fts5 when processing corrupt records.
FossilOrigin-Name: 061febcf41ca4872a0f407951e1507209daca7895122b909a7806c60b6e200c4
2026-05-11 12:00:19 +00:00
dan
0cb7603d4c Update fts5 scan costs to take into account that a prefix query combined with a rowid equality constraint is still quite expensive.
FossilOrigin-Name: 539769b63e4044018be2993894b1962df6a6ae58636682b7fb7a2dd5aaf4075c
2026-03-19 17:56:10 +00:00
drh
f6c8adf3a2 Omit explain_i calls from TCL test scripts.
FossilOrigin-Name: 204f25c882832058c3291d4d6a0a6ac2711c081cb91b19feb2cf94a25ed61ecf
2026-03-02 16:40:16 +00:00
dan
4b9def1903 Fix further problems with deleting from corrupt fts5 databases in secure-delete mode.
FossilOrigin-Name: f8f89d2e10f06b54acf58a3b0543aef749c3d2b65670d4cc71530dea7997651e
2026-01-16 20:04:04 +00:00
dan
90942efd21 Fix an error in new test script fts5corrupt9.test.
FossilOrigin-Name: a5918835e172fd4eda6511d297b61368952d617eac33950b67368671e3bd40fe
2026-01-15 17:17:00 +00:00
dan
ce83fd80ad Fix a buffer overrun that could occur when deleting rows in secure-delete mode from a strategically corrupted fts5 database.
FossilOrigin-Name: 1e66d6e3276cc6aab4d002a1df13b044e61f3946322bf97cac06d98dbb68e433
2026-01-15 16:09:35 +00:00
dan
6e92494955 Avoid signed integer overflow when an fts5 'merge' command is passed -2147483648 as a parameter.
FossilOrigin-Name: 4c5d18d7f5bdcf95efdf96062c7434d5d10584a1fdfd65b18d331b8aae42e26a
2025-12-30 11:19:27 +00:00
dan
b4f6c09e60 In fts5, avoid holding an open blob handle after an INSERT/UPDATE/DELETE statement, as this can interfere with sqlite3_interrupt(). Forum post [forum:/forumpost/95413eb410 | 95413eb410].
FossilOrigin-Name: f88e1d03573f394fc7caf60704c96152717b3fe8be4eef0caeb64737054f0fd2
2025-12-09 13:41:01 +00:00
dan
8fa43479b3 Fix a problem in the xFilter() method of the fts5_vocab virtual table causing it to omit some result rows in some cases where a virtual cursor is used more than once by SQLite.
FossilOrigin-Name: 6c99e81e6c49c078f1b00952774a7007c60dc1b4ba8a8fc11357efe9c9233212
2025-11-28 11:52:40 +00:00
dan
d2c7e6925c Fix a buffer overread in fts5 that might occur while processing a corrupt db.
FossilOrigin-Name: 8b0cbc18be3c6f2501b102757af6be98c48044a296104cca7bce822ac2304515
2025-11-26 13:58:53 +00:00
dan
264bcad69a Avoid integer overflows (a) when dealing with zeroblob(N) calls when 2*N is larger than 2^31, and (b) in fts5 when a corrupt record is close to 2^31 bytes in size. Both of these require special builds with SQLITE_MAX_LENGTH set to a non-default value.
FossilOrigin-Name: 27927519696dcb78ff72e245158ea6d33a1b2c3d4da314d4d0283c47d847c3da
2025-10-13 16:07:28 +00:00
dan
40d299ae32 Avoid corrupting the fts5 index if a value that was inserted via sqlite3_bind_blob() in a non-utf8 db is deleted.
FossilOrigin-Name: 8bf26c956e199762d55c159392c2e3813a9e12b914d3ca33000bf332cd946cb0
2025-10-09 14:50:27 +00:00
dan
0247e15f69 Updates to where.c to handle vtabs in joins more similarly to regular tables.
FossilOrigin-Name: 991383262b0d07ebc79fdb6bed448013cea3a0ac13292d5397dce012b1e6a673
2025-09-13 19:23:12 +00:00
dan
81e9bbf8da Avoid evaluating special vtab operators (e.g. MATCH) that are part of ON clauses attached to left joins from being evaluated too early. Fix for [forum:/forumpost/428ef7c468 | forum post 428ef7c468].
FossilOrigin-Name: 9f184f8dfa5ef6d57e10376adc30e0060ceda07d283c23dfdfe3dbdd6608f839
2025-07-15 19:00:01 +00:00
dan
192d0ff8cc Optimize allocation of large tombstone arrays in fts5.
FossilOrigin-Name: 0fcc3cbdfa21adf97aed01fa76991cccf9380e2755b0182a9e2c94e3c8fb38d7
2025-07-15 14:06:15 +00:00
dan
bcfe38b96a Avoid evaluating special vtab operators (e.g. MATCH) that are part of ON clauses attached to left joins from being evaluated too early. Possible fix for [forum:/forumpost/428ef7c468 | forum post 428ef7c468].
FossilOrigin-Name: 18ba07c152294a96d510a7851a0860f982ff1e0725afeebed9a343f550f6925a
2025-07-12 18:14:41 +00:00
dan
9b91aac83b Fix a problem with the fix in [5cb8e342e1].
FossilOrigin-Name: 043ff54fb746c54bc6cfa6aa8c8a32c876c09d36163125916ad01024b98d447b
2025-07-08 19:02:26 +00:00
drh
925b79039c Fix test case results in FTS5 that were changed error message improvements
in [48044a6b57c0a16c] but which were omitted from that check-in.

FossilOrigin-Name: e1f8e0240b14f1338e947634e840af785e5042a09cd365975e922a13a562f4cd
2025-07-08 12:37:25 +00:00
dan
526399b0fd Avoid an assert() failure in fts5 that may occur when processing corrupt records.
FossilOrigin-Name: 8afd6ca85724a69970181042d6aac53742ea2b76ded5966b2541c3afe1121fb9
2025-07-01 16:21:47 +00:00
dan
fe182f74b2 Improve some of the error messages emitted by fts5 when it encounters corruption.
FossilOrigin-Name: 48044a6b57c0a16cb75139c103ad88ca4ab64d74f70a3dee0d8b817fbfbec3c6
2025-06-23 19:38:22 +00:00
dan
6a5e74cbf0 Fix a problem with UPDATEs on fts5 tables that contain blob values.
FossilOrigin-Name: badf3014bd1620fd3d4b8013f641fd820b249649fb93cc75b7b8df9dfd6f32a6
2025-06-11 15:03:53 +00:00
drh
37f22ff75e Fix a problem in the fts5matchinfo.test file that prevented it from
working on Mac.

FossilOrigin-Name: 5a411c609ba0ee17b1ce5bd0b40b9ded92d76f35933f92b2e02a58a3f169c715
2025-04-22 18:45:40 +00:00
stephan
da5f813878 Approximately 100 typo corrections spanning the whole tree, submitted via [forum:0db9827f0464bc33|forum post 0db9827f0464bc33] and individually audited and verified. Affects only code comments, innocuous test strings, error message text in tool (not library) code, and configure-level help text.
FossilOrigin-Name: f50c21484d3cac73589da0376c423de39ae8b842218105786c5aa3726e4dcaed
2025-02-27 21:17:55 +00:00
dan
eb3a069fc8 Have SQLITE_FTS5_ENABLE_TEST_MI builds avoid reading the database schema from
within sqlite3_open().

FossilOrigin-Name: 15dc524fd4113026cc542140c39c1c8f9e052d36946f0c599f282d9ac27efdab
2025-02-18 15:11:30 +00:00
dan
b80d01a182 Have fts5 better handle OOM errors from sqlite3_blob_close().
FossilOrigin-Name: f418350f3f83147bc5817a885be6e39ff9ff5722742a88d17600729c53c65010
2025-01-21 14:34:59 +00:00
dan
467e20ad1d Fix an assert() that could fail if a virtual table called sqlite3_step() from within the xSync() method while committing a "PRAGMA defer_foreign_keys=1" transaction.
FossilOrigin-Name: 39bdbb3f6dd1d30d180526d35c11f789f5e9d45b99ead72fd4a3b136afab66b1
2025-01-20 18:26:58 +00:00
dan
b1cbae8bd6 When removing an fts5 in secure-delete mode, defer setting the table version to SECUREDELETE until flushing data to disk. This prevents problems that can occur if there is a rollback or statement rollback operation.
FossilOrigin-Name: c359e555ceafcaab2ae38074bc4f57cccdc5bc6080d17f82290c09b9e5dd80c2
2025-01-09 14:10:25 +00:00
dan
c1242d60d9 Fix another assert() failure in fts5.
FossilOrigin-Name: 6da37893f5b5729ea5fd632e8d98789e867488a67501d4a4dad92f8e7cb6bda0
2025-01-08 20:43:03 +00:00
dan
ad460db7eb Fix a crash in fts5 that could occur if shadow tables are modified or removed.
FossilOrigin-Name: c0b691095ae72fc07530777ef6d23688fb4196ce2e0feff14fc3c597c572252d
2025-01-08 15:54:44 +00:00
dan
3e4ccc3208 Update fts5 so that the xInstToken() extension API works with prefix queries.
FossilOrigin-Name: 41b79bde4a4b6510b41cc9175ff2527fbd3ca9209812361783754f24cf67ef9e
2024-11-26 20:46:18 +00:00
dan
0cd2ffffb7 Fix the fts5 trigram tokenizer so that it handles non-nul-terminated strings.
FossilOrigin-Name: 84f4e37178a65e3128ac0240d37ac40df08b4050ab070d10707e35d11dcbeb10
2024-11-11 19:49:26 +00:00
dan
8acaa6d039 Add tests for xInstToken() and prefix queries with various fts5 configurations.
FossilOrigin-Name: 9cc04331a01760189d88697233009dbe8a60eda589792ad01b56300499e9f54d
2024-11-06 16:20:16 +00:00
dan
090b8649be Allow an fts5 table or query to be configured to collect xInstToken data for any prefix terms as part of the first parse of the main index, if any.
FossilOrigin-Name: 46929ae92b26f02bc70de9931b21a8a7cf9a2453d5fb07f68b712f62e28e9152
2024-11-02 19:10:50 +00:00
dan
b27a30c620 Merge latest trunk into this branch.
FossilOrigin-Name: 740a37c5d54b57befa86a6bb299ffa89ed4243d10db885a08ab5c63238460dad
2024-10-27 14:41:58 +00:00
dan
59b4f75e0f Add test case for fts5 trigram tokenizer.
FossilOrigin-Name: ba358d265b7ee360d62b5219faaa1010ea90dac4e20cc7adc3ebd46161a65f94
2024-10-26 18:09:13 +00:00
dan
44b8c37017 Fix a typo in fts5delete.test.
FossilOrigin-Name: b1f001435eff72c2119ecee973194385f70fd1b66ef0be8a66c0b0ad02ae43df
2024-10-02 17:43:06 +00:00
dan
c5c3cbc026 Add the contentless_unindexed=1 option to fts5. This causes the values of any UNINDEXED columns of a contentless fts5 table to be stored persistently in the database.
FossilOrigin-Name: 58313ac59e0bd164f601d68a1474f658c5d1c038638e00f3dc15eb58202e661c
2024-10-02 17:04:30 +00:00
dan
d2a88e961a Rationalize code further. And add tests.
FossilOrigin-Name: 0ca002a4ab88f3e7ae1e6e518038157eaa20759f57888c2ed7e50cb92bd96348
2024-10-01 20:38:08 +00:00
dan
d5838eaa42 In fts5, avoid starting a new merge of level L if there exists already an ongoing merge of a level less than L.
FossilOrigin-Name: 350c6e75ce3c1e81458d1baa73045df489284206e8b279ab3c2f5e3d011c262a
2024-09-30 17:28:45 +00:00