24730de8d1
Add the fts5 'delete-automerge' integer option. A level is eligible for auto-merging if it has a greater than or equal percentage of its entries deleted by tombstones than the 'delete-automerge' option. Default value is 10.
...
FossilOrigin-Name: b314be66b9ac0190b5373b3b6baec012382bc588c2d86c2edab796669a4303c3
2023-07-24 19:13:06 +00:00
2159292ce0
Integrate contentless delete with auto-merge.
...
FossilOrigin-Name: 85c1589ab1fc69d1eef4bbc1bdefa2b10af5f6b9c08e813130b93829b592f416
2023-07-22 19:47:46 +00:00
330e36c2c6
Fix fts5 incremental optimization so that it too can handle an index that consists of a single segment with one or more tombstone hash pages.
...
FossilOrigin-Name: e61c9b083f5e0b6b6ee18f9394581ad816f445dbfb72ed1fe954f4182755a576
2023-07-21 21:10:33 +00:00
4e0c157d02
Ensure the fts5 'optimize' command correctly rewrites any index that consists of a single segment and one or more tombstone hash pages.
...
FossilOrigin-Name: f4926006b371d9a1439a25384bd50a50c2f1c03f75a7c2c3134ae72abb971c91
2023-07-21 19:33:35 +00:00
3a51f8c307
Fix some divide-by-zero errors that could occur when handling corrupt tombstone hash records.
...
FossilOrigin-Name: 7567ca0676f0d45026f5cd4f3fbcd09119c2eaab8ec1711499609c16c452b5e4
2023-07-20 20:29:56 +00:00
d6f5aa824e
Avoid an infinite loop that could be entered when dealing with corrupt fts5 tombstone hash pages.
...
FossilOrigin-Name: 69ce2ce035279f2a00c2238187cf4d2a9092c3410f5900e4613fe4e46311169e
2023-07-20 20:09:26 +00:00
0b3791b537
Add tests and fixes for the new code on this branch.
...
FossilOrigin-Name: 5aac50e92e956b15367c75c20c17bc1c75e84e2752bfffe4ad0a266cb9bd3b8a
2023-07-20 16:07:04 +00:00
d1fbaa071b
Fix various issues with code added to this branch.
...
FossilOrigin-Name: 8d09011fa2c6ae9cc88e1766f9aad4578efbf9e0e311b8c6efdffe7a3f88f923
2023-07-19 18:47:02 +00:00
d05bf0fe61
Fix various problems with fts5 contentless_delete=1 tables.
...
FossilOrigin-Name: 0d005112b8aca9e9eca9d86d5fed9168f6a0218fd290b5489b9e7b05714610f4
2023-07-18 19:52:32 +00:00
2cccced140
Fix cases where a row is inserted into a contentless_delete=1 fts5 table and then deleted within the same transaction.
...
FossilOrigin-Name: d928856a226fb7f001e55ff7e8eb58a656b982f1efa811de46c382b8b7cd778c
2023-07-17 18:40:39 +00:00
55e0fd4a9d
Do not allow the 'delete' command to be used on contentless_delete=1 fts5 tables.
...
FossilOrigin-Name: cc694b83408ccb5d42204cb624145c76e95329cbe1d1fe8815c70a7a00af231a
2023-07-17 17:59:58 +00:00
0dac350f3c
Store large tombstone hash tables in multiple database records. Ensure the same hash tables handle rowid 0.
...
FossilOrigin-Name: 4410e60d0c76e057ee962124f9239c6e17fd5ccafdbb4d9b703448eabd7781e3
2023-07-15 18:57:57 +00:00
b184c91076
Have contentless_delete=1 tables support REPLACE statements.
...
FossilOrigin-Name: 2f553a660e00564e51bc4209c92bd3628fb1266f4a52832792fbf91e4234a0ba
2023-07-11 20:57:19 +00:00
b92669db73
Have contentless_delete=1 tables support regular DELETE statements, instead of just the special INSERT syntax.
...
FossilOrigin-Name: fffb8616905501669a94231d5d9f53446bf09553353f2cdab7c43ca54bbb7fa6
2023-07-11 20:19:46 +00:00
d23f210377
Use a hash-table instead of a flat list to store tombstone rowids.
...
FossilOrigin-Name: 948267b066d0dbe667881b3d26a007fa24576da6e57c112676fadeb846c13f0b
2023-07-11 18:55:19 +00:00
6788c7b7c0
Begin adding support for deleting rows from contentless fts5 tables.
...
FossilOrigin-Name: e513bea84dfaf2280f7429c9a528b3a1354a46c36e58ab178ca45478975634e0
2023-07-10 20:44:09 +00:00
d679f9e5d1
Fix a buffer over-read that could occur as part of an fts5 secure-delete operation on a database containing corrupt fts5 records.
...
FossilOrigin-Name: a35c50eca907d0cf373b7962c359aebfce8ac7c34ce81edeb99122d4fb5804f7
2023-06-12 11:06:57 +00:00
870fe69c72
Add extra tests for fts5, large rowids and "ORDER BY rowid DESC". No changes to code.
...
FossilOrigin-Name: b5b5eaeed3ef55415e27aa04561ab446f3be881b56b7cbf9b8ba3307f0b2bf69
2023-05-27 19:34:46 +00:00
b838a229be
Fix a buffer overrun that could occur in fts5 when processing corrupt records.
...
FossilOrigin-Name: 4891dbd938f4bfd345eaef01f2addea9512eaa98f860844c73abb907b6a8e0e8
2023-05-22 11:02:15 +00:00
4042d2b424
Limit the number of nested NOT nodes in an fts5 expression to 256.
...
FossilOrigin-Name: 0e5c1ee40a146ef8b2b3c5f53d0a45e092bc8d8e933f3819805c995819d31bae
2023-05-15 17:14:16 +00:00
cad6aae45e
Allow virtual tables to prepare writes to shadow tables in defensive mode from with xRelease() callbacks - even if the xRelease() is invoked from within sqlite3_finalize().
...
FossilOrigin-Name: a17106494a8a67bc92cfe27ee8ef9bf160517d6930d796980ae12287b98c1cfe
2023-05-05 11:47:31 +00:00
e01e3b5b94
Fix a crash that could occur in fts5 'secure-delete' mode when operating on corrupt records.
...
FossilOrigin-Name: 2e85b0e3dcae0915aa6472a3654c8ac72a6b2083c11747f3f657c79bbdaf530b
2023-04-29 18:31:44 +00:00
4d24eff8dc
Avoid a crash in fts5 when combining an empty expression with another.
...
FossilOrigin-Name: 33681ff3d29c9f096dbd8c5d885993f780e93974a7653adc00fa41a158356d7c
2023-04-22 20:47:08 +00:00
8cb4f0b1b2
Add tests for the progress-handler callback with the fts5 'secure-delete' feature.
...
FossilOrigin-Name: 48505ad950bc0902d58210be066d4672e6085eb27c525ba2bc663fde7e932ff7
2023-04-19 20:29:26 +00:00
a5eab9579a
Fix an assert() in fts5 that could fail following an OOM or IO error.
...
FossilOrigin-Name: 49595083ba58e13d404b6a11c28f53768a257c59f3f36be3591438127df7e875
2023-04-18 11:33:35 +00:00
68c0da6f66
Fix some problems with using fts5 options 'secure-delete' and detail=none together.
...
FossilOrigin-Name: 4d3f27ba90aa59837e49ceaae9f36cad426af3a33707d85d1bb848fc1bda6fed
2023-04-17 18:32:22 +00:00
48824f23cb
Fix problems handling corrupt records in new fts5 code.
...
FossilOrigin-Name: 2f63d4a8bac6dc72197df32efd84c148adbc4794943088dff08da57b15dd7900
2023-04-17 11:41:42 +00:00
ae6f6ef3b0
Change a memcpy() to memmove() in new fts5 code.
...
FossilOrigin-Name: 8de4542553ae20947787b4daf5e88bbe69f815679960e700e19f9c1148234b08
2023-04-15 21:00:55 +00:00
0e429cf934
Fix a use-after-free that could follow an OOM or IO error in fts5 when using 'secure-delete' mode.
...
FossilOrigin-Name: 904be83d9f3dd4b21a3aa47dde337b5c045b925ffdde4bccd1f8e8fdbef68c18
2023-04-15 19:58:58 +00:00
e45317a50a
Avoid dropping the error code following an OOM in fts5 when using 'secure-delete' mode.
...
FossilOrigin-Name: 9ee24a499c7b4a048f269f1c03c7c0b22b2898a3c23ea3e2511c2ba0342afa30
2023-04-15 19:13:51 +00:00
112500028e
Further tests for fts5 secure-delete mode.
...
FossilOrigin-Name: c5a47063b7971868ebbb3f0718434dd6bd22d01c43697f48941df9aae3885abc
2023-04-14 18:00:03 +00:00
3fc123b7bc
Add extra OOM test for the new code on this branch.
...
FossilOrigin-Name: 846ae7e099ce67dab6b5df0ad5648a01d7bf7f2acf8312d817ed8505d4ea5aec
2023-04-14 17:00:29 +00:00
d36106c954
Add tests for the new feature on this branch.
...
FossilOrigin-Name: 0268d1a59316d169927cbd4c562725e46a9023b7d7123ade911c12203e222adf
2023-04-14 16:11:05 +00:00
015020cd1a
Add the 'secure-delete' option to fts5. Used to configure fts5 to aggressively remove old full-text-index entries belonging to deleted or updated rows.
...
FossilOrigin-Name: 4240fd09b717dbc69dffe3b88ec9149777ca4c3efa12f282af65be3af6fa5bb0
2023-04-12 17:40:44 +00:00
ebec9c475e
A prepared statement that aborts due to SQLITE_SCHEMA should not invalidate
...
cursors in other prepared statements that are already running.
See [forum:/forumpost/cae4367d9b|forum post cae4367d9b] for the original
trouble report.
FossilOrigin-Name: 857d0f5e16ba69ac9e5ee581befca2f7ed933edfe6e36396b10dae7979b44a57
2023-04-06 13:35:42 +00:00
5f708cc832
Fix a problem with the fts5 snippet() function that shows up when snippets just 1 token in length are requested.
...
FossilOrigin-Name: 96d5116d17cbf3de6be38952881c6ddf6a226ddfd7649197bdb2df39c42b5d8e
2023-03-10 13:36:19 +00:00
a3e6192941
Fix a problem with the fts5 trigram tokenizer and LIKE or GLOB patterns for which contain runs of 2 or fewer non-wildcard characters that are 3 or more bytes when encoded as utf-8.
...
FossilOrigin-Name: 00714b39b39c51519edbc0194f98c7275fecf96763a06fd95db6e1d81bb9f1f1
2023-02-10 17:17:04 +00:00
0d12a79310
Better fix the problem where optimizing an fts5 table too often causes it to become unreadable (first attempt was [35bed981]).
...
FossilOrigin-Name: 459d986d38fc0ccbfd66801e0f22900cfed831268cf59ac8d1cd1e556f0d1441
2023-02-08 17:28:08 +00:00
d40a183506
Fix a problem where optimizing an fts5 table too often might cause it to become unreadable.
...
FossilOrigin-Name: 35bed981416269018a0d52171d9997c43d3b097f5db1910d10d882eabe2e137f
2023-02-03 19:28:50 +00:00
1dff401c42
Fix a broken assert() in fts5.
...
FossilOrigin-Name: ff5a9b054f4d5ad3d33787aee93b4181822c6dfc513626236d9867bb431f64da
2022-12-27 13:52:25 +00:00
73e460c932
Fix a faulty assert() in fts5.
...
FossilOrigin-Name: 569018170b928cadfa0745db631981084e595d90a8c3d1e236c42adae161018f
2022-12-19 20:14:22 +00:00
1532f9d155
Fix an assert() in fts5. Simplify memdb xLock/xUnlock some.
...
FossilOrigin-Name: 7fe158aa8071acadd959b2b4a4b66e8a7f7eecec207ba681abf516b0457c2921
2022-12-19 14:06:36 +00:00
6aafd74853
Avoid some cases of signed integer overflow in fts5 by casting to unsigned values.
...
FossilOrigin-Name: 46a78c8c0ed518c4521e6e0bdebeb065bab07076abc444775002e7f4361d2242
2022-08-08 19:29:53 +00:00
cf2ad7ae36
Remove a few unsuitable scripts from the "veryquick" test suite. Also have every second testrunner.tcl process favour running test scripts that contain text like "testrunner: slow" before any others.
...
FossilOrigin-Name: 22d280a5cd395abbedcfffbac3d3b3a614c327be25763ca380c1338a2a7bd33a
2022-07-18 19:32:30 +00:00
f703b42dc4
Update the expected error messages in some OOM test cases to account for [44d77a7f807].
...
FossilOrigin-Name: b3d6b3c3fc68dca7e20418eefa35ce3b583322b884b88a11c6773419f027a7a4
2022-07-11 21:40:34 +00:00
7ef4d75bd2
Fix a case in fts5 where a corrupt database could cause a crash.
...
FossilOrigin-Name: 5e95df261cce275adb5b2226e9584f4c389bb0d53c3b4feafd5a14be8d02ce63
2022-03-11 12:02:18 +00:00
7d44b22d45
Fix test cases so that they all still work even with -DSQLITE_DQS=0.
...
FossilOrigin-Name: 4883776669ee2f2310ea82b0d6df4d008eebaa7cb252102539cf21a635402ebb
2022-01-16 19:11:13 +00:00
c436b3056d
Fix an OOB read that could occur in fts5 when processing corrupt records.
...
FossilOrigin-Name: bb9b1a15f7e80483162049dfd981d059dc69d03348b521f7ac164a8cd3ae3cc4
2021-12-06 18:57:02 +00:00
76c3ae6848
Fix a broken assert() in fts5 that could fail if the snippet() function were used with a query phrase containing 0 tokens on a detail=none or detail=col table.
...
FossilOrigin-Name: 0a5b94b99cf45df759a59cb5fc62da111ca33f8c2614769a81930d72b51f093a
2021-11-02 07:32:13 +00:00
f7ff755651
Fix an assert() in fts5 that could fail with a corrupt database.
...
FossilOrigin-Name: e99979855de937ed5ee0994b180054501400bf8776fb70acd31786d2ba1ad49a
2021-10-16 13:59:08 +00:00
