ff6ab9dd2b
Update the fts5_tokenizer_v2 API so that the locale is passed as parameter to xTokenize(), instead of via a separate call to xSetLocale().
...
FossilOrigin-Name: 03e63ed24e7a08817341e59b075ae2d4e3f7a5e5b37e0e6138359d5fd29a5e9e
2024-08-10 18:59:36 +00:00
5af9fd53f4
Revision to check-in [d9f726ade6b258f8] so that OOM and other unrelated failures
...
are not overridden by a syntax error in the tokenizer spec.
FossilOrigin-Name: 3778b2a9ca1cc12a88ef6c32a1ee7c58a0a829ed9715a3d32a225d377d7527ef
2024-08-10 15:46:57 +00:00
5eba4281b0
Ensure tokenizers registered with xCreateTokenizer_v2() can be accessed using xFindTokenizer(), and that those registered with xCreateTokenizer() work with xFindTokenizer_v2().
...
FossilOrigin-Name: a3ef7d47b582a66a85ebe802ab40114f77289d6b44a4b375ea7858c60f59b27f
2024-08-09 20:59:50 +00:00
679f5d458e
Add assert() statements and reorganize code slightly in fts3 and fts5 to make it easier to follow.
...
FossilOrigin-Name: 797b0a13fd7a42b0a48ecbf0cd1961aa932da3e9c9ccffd903a3a4d963d0cc54
2024-08-08 15:07:27 +00:00
5cd01c30b4
Fix a broken assert in fts5_expr.c.
...
FossilOrigin-Name: 08cc5488404d068e59378b82988460793710df43ec21b4a83a794b497abd035f
2024-08-07 21:20:08 +00:00
eaa560f3fc
Remove unused test cases from FTS5, as they interfere with "grep".
...
FossilOrigin-Name: 048a71bae48aacdfc91e59330db4d4b3d663f00665f51a7110ea82e0508758ec
2024-08-06 23:09:37 +00:00
e9b919d550
Improved robustness of parsing of tokenize= arguments in FTS5.
...
[forum:/forumpost/171bcc2bcd|Forum post 171bcc2bcd].
FossilOrigin-Name: d9f726ade6b258f8723f90d0b04a4682e885e30939eb29773913e4dfc8e85503
2024-08-06 22:49:01 +00:00
e317e7f4b4
Change things so that locale=1 is required to write fts5_locale() values to an fts5 table, and so that blobs may not be stored in indexed (i.e. not UNINDEXED) columns of these tables.
...
FossilOrigin-Name: c98ccc12169419b8b27ead89ef0665de40320277c5daa748b80869337419e43e
2024-08-02 21:06:13 +00:00
f971c15b1f
Clarify the role of Fts5Storage.pSavedRow in the new feature on this branch.
...
FossilOrigin-Name: e8a61d5c48073fdd4d99d0b6fc70469b37af009f281336a44e3789e7eeed820d
2024-08-01 17:15:17 +00:00
5bd8cc7dd5
Fix various problems with the code on this branch.
...
FossilOrigin-Name: 8bd4ae7e95c7b6ce34db5ea705dc136e742a22f333d0e7370b485ebd736b5ec2
2024-07-31 20:49:00 +00:00
8e3042b565
Add missing documentation for new features to fts5.h.
...
FossilOrigin-Name: fa0da3b28e411affd45a918d6b7faba49f744ca8b4adf3cd5ce6609bb630499e
2024-07-31 15:46:41 +00:00
9015b8822d
Fix cases surrounding blob values being stored in locale=1 fts5 tables.
...
FossilOrigin-Name: ae435aff1785d5832821c19dc88ccf6c496a7ff55be80276b31e5c9abee723db
2024-07-30 20:39:58 +00:00
1c2ad465c6
Tcl_ChannelType implementations for Tcl9 apparently require that
...
wideSeekProc be implemented. Also adjust minor test script issues for
fts5 tests so that they can be run sequentially and so that they do not
depend on the specific floating point output formats generated by Tcl.
FossilOrigin-Name: 19fda979c5dc1a385ed3f8ab8df34388c1acfc7ff951fe1b183a79186bd20cdb
2024-07-30 18:15:59 +00:00
780187f090
Get rid of "-encoding binary" across all test cases, as that is no longer
...
supported in Tcl9. Use "-translation binary" instead.
FossilOrigin-Name: e0d9670dd98a22d2e22aac8ba8720787a05e280e18ec9b2d4323f947c83718f5
2024-07-30 17:09:51 +00:00
3302bf4dbd
Use the SQLite format() function to format floating point values for test
...
results, to avoid inconsistences in TCL versions.
FossilOrigin-Name: 6a63c472190dafb36582c116c925bbcbe48c3a2f3217fc1ffcb64a733b0151e1
2024-07-30 16:59:33 +00:00
1631c64f78
Fix the incrblob channel so that partial close works with Tcl9.
...
FossilOrigin-Name: d54ec2de78c2aed12be643e59a30b6827fe4728a0ab6504dd64ee9116e5f885a
2024-07-30 16:51:51 +00:00
8174305e17
Update the highlight() and snippet() functions to use locales when available.
...
FossilOrigin-Name: 569ae56a673a2e101fac7003a1cb41c7a02a515adf55bd1e1f2a03c19eb2b085
2024-07-30 15:55:51 +00:00
064b681e9b
First attempt at getting the build to work with Tcl 9.0.
...
FossilOrigin-Name: 6e5bb48a74d63fb8c30528f0005d1763cd2dbb882abf86baf1565721e6bfcf84
2024-07-30 15:49:02 +00:00
8f14c45197
Fix xColumnText() and xColumnSize() APIs. Add xColumnLocale() and xTokenizeSetLocale().
...
FossilOrigin-Name: 14006711d83d098e665c540b978b0e29aa8f458da1c2c8e9c2baf2ad5ebd502c
2024-07-29 20:31:17 +00:00
b4afecc626
Fix a couple of asan problems on this branch.
...
FossilOrigin-Name: 4e3e529313c86b42d5f1ba113d0714d1545bc6641e09e4537d622057f0352989
2024-07-29 15:00:11 +00:00
53bef48fbe
Ensure that an fts5 UPDATE does not strip the locale from columns that it does not write to.
...
FossilOrigin-Name: 69205264debd829573b1c777a5a493cfeb6083c4cdec106b1f819989f859ac75
2024-07-27 19:46:10 +00:00
2ec78c0e4b
Add the fts5_locale() function, and begin adding the related functionality to fts5.
...
FossilOrigin-Name: 8839ef7cfb49239e7f1c4812a53a93a672827c88d6921408b1d5062b352c87cc
2024-07-26 20:50:33 +00:00
d109de8f84
Fix a problem in FTS5 where it reports the wrong error code following an OOM.
...
FossilOrigin-Name: 0c1cc4071edcd3e465779bbb17ea3ee6ddaa7c93c4fde55e5f3e07aa4b45783b
2024-07-18 18:44:44 +00:00
a592883d87
Add extra assert() statements to the fixes on this branch.
...
FossilOrigin-Name: 81b6360050eea95e4367de1b41b5864a640b4d1c5c8fc3bea3b96ed770cb0325
2024-07-17 16:27:36 +00:00
7acf972c59
Fixes for platforms with 32-bit pointers that require 64-bit values to be aligned.
...
FossilOrigin-Name: 2212d7488ed4ec2839ffa45cb9567056b36519434834634e4ecc441c330694d7
2024-07-13 16:53:56 +00:00
bee10b987c
Improve the error message returned when fts5 finds a row is missing from its content table.
...
FossilOrigin-Name: 9b1afd4acb8cf9738154e921dcb074d895bbcc4742bf1016ef9f045a0da78bc8
2024-07-08 16:01:49 +00:00
3d24637325
Add assert() statements to FTS5 to hush-up warnings from scan-build.
...
FossilOrigin-Name: 77a76654e64c05f2c67be120f579fc60924ff51a9d0fa2cf9eb158f40171b5f5
2024-07-02 13:54:46 +00:00
75bdf00e89
Update fts5 comment/docs to include the order matches are visited in by the xPhraseFirst/xPhraseNext API.
...
FossilOrigin-Name: e41930ef5b883c0f2a6b7eb9606021d14a3df145bc94cb0d508af22b4db7f168
2024-06-28 18:21:30 +00:00
105c20648e
Fix an incorrect tcl comment that appeared in many fts5 test files.
...
FossilOrigin-Name: d07085e2035b52a7edd27980523225e59c5bf851fb4a6de975f03e653b937c9c
2024-06-24 18:06:15 +00:00
fdda1cbe0c
Fix some problems with calling fts5 api functions from within non-full-text queries.
...
FossilOrigin-Name: 56d265f956fe6433b625c6d732e55f387af3f643e705316f5a6f709d63731669
2024-06-24 16:08:01 +00:00
6593b340ff
Fix harmless compilation issues seen with MSVC.
...
FossilOrigin-Name: 816d4749384c7f398912c905a16c83b88f4c55632050b4c6117c61301d1a53e1
2024-06-05 20:50:39 +00:00
6d98c55b56
Fix an fts5 problem with secure-delete mode causing integrity-check to erroneously report a corrupt index.
...
FossilOrigin-Name: 80bef4d60ba9e3679ea66655ca36fcfaa888775a3d1598d50e9649ad84a95b63
2024-06-05 14:47:54 +00:00
9c3eef8c0d
Have fts5.xBestIndex return SQLITE_CONSTRAINT, instead of a large cost, when no usable plan can be found.
...
FossilOrigin-Name: 7c470945372dc98610f7c9840ce8cab18c19b655352f0187e4f31040cea77363
2024-06-01 17:56:58 +00:00
7da33383c7
Remove an unused parameter from fts5ConfigParseSpecial(). Compiler-warning
...
fix only - no functional changes.
FossilOrigin-Name: c08dd245f7706f2fd2269d700be480477619a722e27e6b439462ae543302c49f
2024-05-29 15:16:17 +00:00
2386943f19
Fix a problem with the error message reported by fts5 in some OOM cases.
...
FossilOrigin-Name: c7aaa28f786567e29ebee3848283b5c740dd3b8144dc14372e56260ae1b76922
2024-05-28 10:57:24 +00:00
c598fdd10c
Fix another fts5 problem related to unknown tokenizer modules.
...
FossilOrigin-Name: e78b7783040ef1ad34d4287af8a7a5998d360d50076e0de42a116d9a96fe74b4
2024-05-24 21:38:37 +00:00
0e7a7e57cc
Fix a crash in fts5 that could occur within "PRAGMA integrity_check" if the specified tokenizer was unknown.
...
FossilOrigin-Name: 061d7b69e6a43474cd7123a69f4ee65b5649a0b5e51474343defef5ae5262657
2024-05-24 21:23:36 +00:00
a0693f21fc
Optimize queries against fts5vocab(row) tables that do use the "cnt" column.
...
FossilOrigin-Name: 96a591c202cb4218999ece3de063bf466b6fd81bd23383893f3913ecb52dda36
2024-05-14 19:41:19 +00:00
433af7ab93
Avoid calculating the value of the "cnt" column for fts5vocab(row) queries that do not use it.
...
FossilOrigin-Name: c0c91e306c252ce1b398e92ab958e2a3de0fc43db05cfdffc548380654fc9f13
2024-05-14 19:09:54 +00:00
32ca0dbcdf
Have fts5 tables delay initializing the tokenizer until it is first used in all cases where the tokenizer is not "trigram".
...
FossilOrigin-Name: ca4fdcb8ae95d2a61236b949f852d2bf25ea2dbbff7eedafbd8eb84e8fd96687
2024-05-14 17:16:09 +00:00
12b205c637
Allow existing fts5 tables to be dropped even if the associated tokenizer is not available.
...
FossilOrigin-Name: 69ef47eeee8b53684c321393be34f03600694fbc86377f8720ff80307846aff6
2024-05-13 20:06:08 +00:00
6921a2d205
Update comments in fts5.h.
...
FossilOrigin-Name: 862945d5c432c27377e90d93d64c4655eefcc369d086eb51edef925fb3d80b57
2024-03-27 20:34:14 +00:00
5e5831a760
Handle an OOM case in fts5 code to query tokendata=1 tables.
...
FossilOrigin-Name: 01222d96b9ab24c7013a2587544304ba32c1f039b08c06a04d8d58ec9c790db0
2024-02-16 11:41:47 +00:00
3ab9c021ff
Fix harmless compiler warnings seen with MSVC.
...
FossilOrigin-Name: e52c87420b072fa68d921eda66069542d50accbfaf1110ac4cc1543a4162200d
2024-02-04 04:01:11 +00:00
09768559b6
Ensure the fts5 xIntegrity method correctly returns error codes unrelated to corruption or missing SQL elements.
...
FossilOrigin-Name: d18d9a05c2d6f290d7b16eafced0cde4eb8dd3f869778a49103812d35a0cee65
2024-02-01 15:42:22 +00:00
75f9ba32bd
Ensure that the xIntegrity methods of fts3 and fts5 work on read-only databases.
...
FossilOrigin-Name: b855886c4ccce0745af6957943e77be18949722f09821688725d546d3d79b4fb
2024-01-16 16:05:21 +00:00
c08a8a33d3
In fts5, flush the contents of the in-memory hash table whenever the secure-delete option is toggled. This prevents spurious corruption reports under some circumstances.
...
FossilOrigin-Name: ccf552319a62bfb329820a3bc1f490bacbaa6e90694a257fc65a568a605542c3
2024-01-11 14:03:08 +00:00
4eb626350a
Fix a problem in fts5 caused by a COMMIT involving fts5 data that immediately follows a ROLLBACK TO that does not.
...
FossilOrigin-Name: 55c61f6a8d6a1bc79497b05669beac5c5397b06382bf24b6bec54845962d219b
2024-01-02 21:08:25 +00:00
751c4e2152
Update fts5origintext4.test to work with SQLITE_DIRECT_OVERFLOW_READ.
...
FossilOrigin-Name: 15ed002aed12556aeb9bbe537c4ba839f0c95bac65a69d03401b37cc3fd11b92
2023-12-28 20:54:12 +00:00
42203e1b75
Fix harmless compiler warnings in FTS5.
...
FossilOrigin-Name: 3cd5ef44e40570c357f913a9483fa1cd72e7f2827a5ed5826bff99febae213b1
2023-12-28 16:25:48 +00:00
