7b0fd0c564
Fix a problem with amalgamation builds on this branch.
...
FossilOrigin-Name: 8f046c82c9cf51fc349674577c68d3d2499ee37009deacbf937d711d9930fd49
2023-11-02 18:10:22 +00:00
e186fe20f5
Add the "remove_diacritics" option to the fts5 trigram tokenizer.
...
FossilOrigin-Name: 83da80135b6105f47d1de560232449562ae8ac176c8011a6f75589f62bc9b1db
2023-11-02 17:31:06 +00:00
a238c57eb8
Fix problems with previous commit on Makefile.msc. Also some win32-specific test failures in fts5misc.test.
...
FossilOrigin-Name: 527f2237a6fb5e2ac97afddd7c65c0afa971efc616cb3b738e7245ee0b8d9570
2023-10-30 18:07:18 +00:00
9f20bde65a
Enhance the new xIntegrity method of the sqlite3_module object with new
...
parameters that provide the name of the table being checked and
a flag to indicate a "quick_check". Based on feedback in
[forum:/forumpost/965c0d02ea|forum post 965c0d02ea].
FossilOrigin-Name: bc8afa3f15954bb35f65dbf940bf069de5e14d333036676c24430cf17b658d05
2023-10-25 10:37:11 +00:00
d548f74024
Fix a problem with the fts5 highlight() and snippet() functions when used with tokenizers like "trigram" that output overlapping tokens. Forum post [forum:/forumpost/63735293ec|63735293ec].
...
FossilOrigin-Name: d570aa02f79b1d7d3889e33f9eebab1b7edcf5231b1357451eed9a538607de54
2023-10-24 15:53:02 +00:00
8c86f56e90
New test cases for PRAGMA integrity_check against FTS4 and FTS5 tables.
...
FossilOrigin-Name: 3950d20c3a4433543546ce4495ae07aab3d6cefb163b1f30813a9776c5b962e8
2023-10-23 15:23:10 +00:00
45b2e946cc
Improved error messages for the xIntegrity method in FTS3/4 and FTS5.
...
FossilOrigin-Name: df69abc0823f4dd9c051f8d5993d2f447ee0dadab5cef73d670107b1071db255
2023-10-23 13:46:22 +00:00
ef6bf1bbe5
Fix a problem allowing a COMMIT following an OOM to cause fts3/4 corruption.
...
FossilOrigin-Name: 7f41d7006db4225cf9b3d197d3a76842778669ac079e76361214a8023c9976e6
2023-10-21 18:12:07 +00:00
71bae9f19f
Fix a problem allowing a COMMIT following an OOM to cause fts5 corruption.
...
FossilOrigin-Name: fba3129d1362dd3ee04d61cb3d3c48518cd98dde3febc85298f411776d60073e
2023-10-20 19:59:12 +00:00
d7cdfe74f6
Fix an fts5 problem caused by a 'rebuild' followed by a DELETE in secure-delete mode.
...
FossilOrigin-Name: 18be505c628d9b13431ca6cfe822d4aeae119c53ae08aef26a67f310a8bd7bd3
2023-10-13 15:59:11 +00:00
3046039309
Fix a problem with an fts5 secure-delete on a rowid/term pair that follows a legacy delete of the same pair.
...
FossilOrigin-Name: 579aea0c28e01a79620ac758edc02db3a01baaa073e7773b8f0b6f610479520b
2023-10-12 19:46:58 +00:00
4b136ed70a
Enable SQLITE_VTAB_INNOCUOUS for FTS3, FTS5, and RTREE.
...
FossilOrigin-Name: f34c533b6c1f8ef3f69c75c1039406c12751cdde1fbdcb4d9776b24455facf8a
2023-10-12 18:46:27 +00:00
03204e9106
Add the tokendata=1 option to ignore trailing token-data when querying an fts5 table.
...
FossilOrigin-Name: 122935182ad5869ce3a4c6d796c38a0509f6f3384dd1b3e60a3f2f0f366cc5f5
2023-10-11 21:08:12 +00:00
1935887a68
Ensure that all fields of static sqlite3_module objects are explicitly
...
initialized, in order to hush-up nuisance compiler warnings.
FossilOrigin-Name: f3b3d712d6e58b1cb8fdebd2b6b3125080b6b3ac8c7c849a8cc1e5e778d62fe7
2023-10-06 12:51:05 +00:00
1846de49a4
Fixes for fts5 expression parser module to allow embedded 0x00 bytes in tokens.
...
FossilOrigin-Name: 342c8d0783f449817d3f565ff6b9f010a6c690beeea32f1861640810490a8b5f
2023-10-03 19:06:52 +00:00
eb28787b5f
Update fts5_decode() to allow for embedded 0x00 bytes in tokens.
...
FossilOrigin-Name: e051120067fd87f57b498e505e3960cf4d14e8e33bad940618cc0823253254f7
2023-10-03 17:07:54 +00:00
a35ae44150
Changes so that fts5 can handle tokens with embedded '\0' bytes.
...
FossilOrigin-Name: c027c092c4af53bd6ae3cc6e2b4439167d9eeb0f9de549b6a2c2a72a67ee886c
2023-09-30 18:13:35 +00:00
2d3c5385bf
Add extra tests for the fts5 xPhraseQuery API.
...
FossilOrigin-Name: b5edea16317ccc896394011f45442af44d5ea1041b6abfe152c90412c78bfc5b
2023-09-23 15:34:02 +00:00
2599705105
Enable "OR IGNORE" support via SQLITE_VTAB_CONSTRAINT_SUPPORT for internal-content (not contentless or external-content) fts5 tables.
...
FossilOrigin-Name: c362bde4f4b8489947f080154d7fddcfd6e8e21d742a483c496fb7fbe59969d2
2023-09-16 17:11:44 +00:00
fef1c11f92
Replace an if() condition in fts5 that is always true with an assert().
...
FossilOrigin-Name: 2170312c8d7f076cbb8319227de3fac981432dae186bc1928cd217e41119b580
2023-09-13 11:24:58 +00:00
12439f9c16
Fix a use-after-free error in fts5 that could occur when querying the "rank" column immediately after another connection changes its definition. [forum:a2dd636330|forum post a2dd636330].
...
FossilOrigin-Name: cb54c2da52d3175844127ead41cfed18793e91ea52e02af2d57c1d17b4312437
2023-09-12 18:36:46 +00:00
1b2d170a8f
Add extra tests for [4b60a1c3].
...
FossilOrigin-Name: 6357491bd1b97f7abddf1a83860f3763cd870033bddc74bd9994a6440fe8c762
2023-09-08 19:18:02 +00:00
410acf6944
Fix a problem with fts5 secure-delete mode causing fts5 to corrupt its records.
...
FossilOrigin-Name: 4b60a1c358c966f161601e749ccb0a7c5f4126e20350f4d6c83b6f35841fbe0a
2023-09-08 18:34:42 +00:00
d5ab4dd9e4
Change the xIntegrity virtual table method signature so that it returns
...
an integer error code and writes the error message into a parameter.
FossilOrigin-Name: f1d4024a8ca06cf954aaf1f612684d1a5d28492bde757695db3f22c50c649709
2023-09-06 14:00:01 +00:00
961c2a9f36
Add the xIntegrity method to the sqlite3_module object. Implement this
...
method in RTREE, FTS3/4, and FTS5 so that "PRAGMA integrity_check" also
verifies the correctness of shadow tables associated with those virtual
tables.
FossilOrigin-Name: 17bede8cdefd968210dd8a5a2617acbe12ba2c99fdd5e88c5def8665e7bec2d7
2023-09-06 12:52:00 +00:00
dd24b1c8a1
Add tests for, and source code comments to, fts5. No changes to code.
...
FossilOrigin-Name: b12afff4efe4de84388d102060e8c312abd2f9eca8270b3c0f01ac6d1f2e329a
2023-09-04 16:48:31 +00:00
b44b802378
Change a variable from "int" to "i64" to make it easier to prove that it cannot overflow.
...
FossilOrigin-Name: 00a8b3a263f3537588063ce42fad6e21fa343dad850b086d0929ed1617eb44fc
2023-08-29 10:50:11 +00:00
1f1169ad09
Updates to testrunner.tcl so that it runs "make fuzztest" using multiple jobs.
...
FossilOrigin-Name: 7596ea7074e0ac73312586ed3d28cdacf97f54b8af73f804cbc8066c94d4b4ef
2023-08-26 21:04:54 +00:00
e174dba739
Change a variable name in fts5FlushOneHash() to avoid shadowing another.
...
FossilOrigin-Name: 3f5bec9a684f6b3d88383617831862fc5595a51ad99ec12430ac6650b704ccbb
2023-08-24 12:01:30 +00:00
da7662a4b0
Fix an undefined integer overflow in fts5.
...
FossilOrigin-Name: f5c50349b1e4da233b554f371be49297b8b40ca5fe0188899ed906f682bf70d6
2023-08-21 16:14:58 +00:00
e15b35d5a6
Fix an error in documentation comments in fts5.h.
...
FossilOrigin-Name: 24d96bd65d8ccdd50a6f4fe217d196728a2f8658d128f9352c287f64e034104a
2023-08-17 16:59:35 +00:00
fab6882fd6
Add extra test to fts5synonym2.test.
...
FossilOrigin-Name: 39ea11be3b35c5ecde9bc0fe66af673b8f440dc523422887d05a29feef036b01
2023-08-17 16:49:06 +00:00
8c5611a169
Fix problems in fts5 test code.
...
FossilOrigin-Name: c16ed3b1cc7b4c21bfa96062bf9028ae071c89e63df3b444b59f1753594e4e94
2023-08-15 14:53:49 +00:00
be3ab76bfb
Changes so that sqlite3_vtab_nochange() works with "UPDATE ... FROM...". Use this to allow UPDATE on a contentless fts5 table if new values are supplied for all indexed columns.
...
FossilOrigin-Name: 16cd2161e312cf97129011fc829079db8f762b822b2f4fabf7ff6742c071302f
2023-08-07 16:15:56 +00:00
7c26811c2f
Rename fts5_api pContext parameters to pUserData, per /chat discussion. This is a cosmetic change made to reduce confusion between those parameters and the two other context-type parameters in that API.
...
FossilOrigin-Name: 2ca064d8eb37252e16b0fec9924e9ba9289d96a737346431c6ba9cb1c161e5de
2023-08-07 09:44:00 +00:00
b383744835
Change comment on Fts5ExtensionApi.iVersion from "always set to 3" to "always set to 2".
...
FossilOrigin-Name: b7de399c277394a6a3bbf253d4177aa82753271e5087b98b2c2268f629a7dfdd
2023-08-04 11:10:10 +00:00
c1a07d3bab
Fix an assert() in fts5_index.c that might fail when dealing with corrupt records.
...
FossilOrigin-Name: aa55c83f35c2ab134e0842201e46e021079283f9c65595c86664060b3aa8d715
2023-08-01 11:03:06 +00:00
808cf29b55
Fix a buffer overread in the fts5_structure virtual table (test code).
...
FossilOrigin-Name: b837aff79cd159061b46af59eaf96a1a1920eeece27e9e27931cf3387068d96a
2023-07-29 20:13:19 +00:00
ea6bccaa44
Fix a couple of compiler warnings in fts5_index.c.
...
FossilOrigin-Name: bf71faa2a1d29ea762c4d2485522d6f4f8a5a7166981a92d3ba9c96ccbbe1213
2023-07-27 20:08:44 +00:00
5326953e57
Add extra test for 'deletemerge'.
...
FossilOrigin-Name: bc33cff4203cef12518e0f43d380a06d53d67c725fb96cfe6e934b7dc97a7efd
2023-07-25 16:48:54 +00:00
3f874b58fb
Change the name of the fts5 'delete-automerge' option to 'deletemerge'. And add tests for it.
...
FossilOrigin-Name: 1079300db2a7d1fbc86a01c215c234a3af64889c5396e6da63ff4f3c7efae4c5
2023-07-25 15:48:58 +00:00
039d494d5f
Further tests for 'delete-automerge'.
...
FossilOrigin-Name: ca26c7a37a7e680be633f43be28f8877bdf9917448ea51c3bedc9b2352a00601
2023-07-25 13:53:42 +00:00
24730de8d1
Add the fts5 'delete-automerge' integer option. A level is eligible for auto-merging if it has a greater than or equal percentage of its entries deleted by tombstones than the 'delete-automerge' option. Default value is 10.
...
FossilOrigin-Name: b314be66b9ac0190b5373b3b6baec012382bc588c2d86c2edab796669a4303c3
2023-07-24 19:13:06 +00:00
2159292ce0
Integrate contentless delete with auto-merge.
...
FossilOrigin-Name: 85c1589ab1fc69d1eef4bbc1bdefa2b10af5f6b9c08e813130b93829b592f416
2023-07-22 19:47:46 +00:00
330e36c2c6
Fix fts5 incremental optimization so that it too can handle an index that consists of a single segment with one or more tombstone hash pages.
...
FossilOrigin-Name: e61c9b083f5e0b6b6ee18f9394581ad816f445dbfb72ed1fe954f4182755a576
2023-07-21 21:10:33 +00:00
4e0c157d02
Ensure the fts5 'optimize' command correctly rewrites any index that consists of a single segment and one or more tombstone hash pages.
...
FossilOrigin-Name: f4926006b371d9a1439a25384bd50a50c2f1c03f75a7c2c3134ae72abb971c91
2023-07-21 19:33:35 +00:00
3a51f8c307
Fix some divide-by-zero errors that could occur when handling corrupt tombstone hash records.
...
FossilOrigin-Name: 7567ca0676f0d45026f5cd4f3fbcd09119c2eaab8ec1711499609c16c452b5e4
2023-07-20 20:29:56 +00:00
d6f5aa824e
Avoid an infinite loop that could be entered when dealing with corrupt fts5 tombstone hash pages.
...
FossilOrigin-Name: 69ce2ce035279f2a00c2238187cf4d2a9092c3410f5900e4613fe4e46311169e
2023-07-20 20:09:26 +00:00
0b3791b537
Add tests and fixes for the new code on this branch.
...
FossilOrigin-Name: 5aac50e92e956b15367c75c20c17bc1c75e84e2752bfffe4ad0a266cb9bd3b8a
2023-07-20 16:07:04 +00:00
d1fbaa071b
Fix various issues with code added to this branch.
...
FossilOrigin-Name: 8d09011fa2c6ae9cc88e1766f9aad4578efbf9e0e311b8c6efdffe7a3f88f923
2023-07-19 18:47:02 +00:00
