database/sqlite
database/sqlite
1
0
Fork 0
mirror of https://github.com/sqlite/sqlite.git synced 2025-04-21 19:26:38 +03:00
Code
29,836 Commits 1,540 Branches 176 Tags
Commit Graph

185 Commits

Author SHA1 Message Date
drh
502b7a236e Turn Fts5Colset.aiCol into a flexible array. 
FossilOrigin-Name: 0c4d9c74741794468adc444908f6024f016738aa2852d3a646f2c28d079d9446
 2025-03-14 23:57:53 +00:00
dan
090b8649be Allow an fts5 table or query to be configured to collect xInstToken data for any prefix terms as part of the first parse of the main index, if any. 
FossilOrigin-Name: 46929ae92b26f02bc70de9931b21a8a7cf9a2453d5fb07f68b712f62e28e9152
 2024-11-02 19:10:50 +00:00
dan
b27a30c620 Merge latest trunk into this branch. 
FossilOrigin-Name: 740a37c5d54b57befa86a6bb299ffa89ed4243d10db885a08ab5c63238460dad
 2024-10-27 14:41:58 +00:00
dan
bcd6d5d393 Allow UPDATEs of unindexed columns in fts5 contentless_unindexed=1 tables. Testing to come. 
FossilOrigin-Name: cd36d66c88d7282eb0a3ccde5713253f72f5843e451b2693b71adfdae28b41fb
 2024-09-27 10:57:41 +00:00
dan
2eff8f2252 Change the way tokendata indexes are collected for prefix queries. 
FossilOrigin-Name: 204ddf4e726b695dd12ab4a945ec2461655aa0bcc38b74e970f07ed2ac43c6ff
 2024-09-25 18:55:11 +00:00
dan
0218424e5a Require that the contentless_unindexed=1 option be specified before storing the values of fts5 UNINDEXED column belonging to contentless tables. 
FossilOrigin-Name: c51dc2a5e75baacbd905cf314e7b1a58a81993ff05ca656739e028d7db25d5b2
 2024-09-13 16:30:18 +00:00
dan
6e8b3d3caa Merge latest trunk changes, including the changes to the fts5 locale=1 feature, into this branch. 
FossilOrigin-Name: d2f0d19936222911bc317efecc831007d3aba81f9b32877030ffb29d1728bbdc
 2024-09-13 15:37:31 +00:00
dan
7a41b48cf9 Alternative implementation of fts5 locale=1 feature that allows blobs to be stored in indexed columns of fts5 locale=1 tables. 
FossilOrigin-Name: 55c5c119a0a77fac2c9f46d718ef78c0f33ed3520e10c240cf5bf1801e0586ee
 2024-09-10 16:19:31 +00:00
dan
cf25c16a54 Fix a problem with fts5 locale=1 tables and UPDATE statements that may affect more than one row. 
FossilOrigin-Name: 70e42f941c0778a04b82655409c7caf4c1039589f7e43a8ec1e736ea8f931b26
 2024-09-07 16:22:22 +00:00
dan
de30fb5fc2 Store the values of any UNINDEXED columns of a contentless fts5 table persistently in the database. Warning: This currently creates a (technically) incompatible file-format for contentless fts5 tables that have UNINDEXED columns. 
FossilOrigin-Name: dcacb1a8ef359b4507b4733356d3150ba5dc105cc9867c103d16a0908a1a9f64
 2024-09-03 18:55:38 +00:00
dan
41ebf965cd Extra tests. 
FossilOrigin-Name: fb257540d32ced4a5fe0759d3ad9c1b36fd2b7d64d93cb0a0ecdb48355a68b7a
 2024-08-15 20:33:05 +00:00
dan
10e54e365a Fix further issues to do with fts5 locale support. 
FossilOrigin-Name: e626123580065986f7df50b6140f00048944becce179b9391fbf09f97ac55485
 2024-08-12 17:03:37 +00:00
dan
ff6ab9dd2b Update the fts5_tokenizer_v2 API so that the locale is passed as parameter to xTokenize(), instead of via a separate call to xSetLocale(). 
FossilOrigin-Name: 03e63ed24e7a08817341e59b075ae2d4e3f7a5e5b37e0e6138359d5fd29a5e9e
 2024-08-10 18:59:36 +00:00
dan
e317e7f4b4 Change things so that locale=1 is required to write fts5_locale() values to an fts5 table, and so that blobs may not be stored in indexed (i.e. not UNINDEXED) columns of these tables. 
FossilOrigin-Name: c98ccc12169419b8b27ead89ef0665de40320277c5daa748b80869337419e43e
 2024-08-02 21:06:13 +00:00
dan
5bd8cc7dd5 Fix various problems with the code on this branch. 
FossilOrigin-Name: 8bd4ae7e95c7b6ce34db5ea705dc136e742a22f333d0e7370b485ebd736b5ec2
 2024-07-31 20:49:00 +00:00
dan
9015b8822d Fix cases surrounding blob values being stored in locale=1 fts5 tables. 
FossilOrigin-Name: ae435aff1785d5832821c19dc88ccf6c496a7ff55be80276b31e5c9abee723db
 2024-07-30 20:39:58 +00:00
dan
53bef48fbe Ensure that an fts5 UPDATE does not strip the locale from columns that it does not write to. 
FossilOrigin-Name: 69205264debd829573b1c777a5a493cfeb6083c4cdec106b1f819989f859ac75
 2024-07-27 19:46:10 +00:00
dan
2ec78c0e4b Add the fts5_locale() function, and begin adding the related functionality to fts5. 
FossilOrigin-Name: 8839ef7cfb49239e7f1c4812a53a93a672827c88d6921408b1d5062b352c87cc
 2024-07-26 20:50:33 +00:00
dan
a592883d87 Add extra assert() statements to the fixes on this branch. 
FossilOrigin-Name: 81b6360050eea95e4367de1b41b5864a640b4d1c5c8fc3bea3b96ed770cb0325
 2024-07-17 16:27:36 +00:00
dan
32ca0dbcdf Have fts5 tables delay initializing the tokenizer until it is first used in all cases where the tokenizer is not "trigram". 
FossilOrigin-Name: ca4fdcb8ae95d2a61236b949f852d2bf25ea2dbbff7eedafbd8eb84e8fd96687
 2024-05-14 17:16:09 +00:00
dan
12b205c637 Allow existing fts5 tables to be dropped even if the associated tokenizer is not available. 
FossilOrigin-Name: 69ef47eeee8b53684c321393be34f03600694fbc86377f8720ff80307846aff6
 2024-05-13 20:06:08 +00:00
dan
3dfc063705 Fix a problem with the xInstCount() API and "ORDER BY rank" queries. 
FossilOrigin-Name: 317a50563d9e8586fda136e513727241b414e7267d50a06571c8ebd0eae710bc
 2023-12-04 17:45:33 +00:00
dan
c22d2b7b7f Fix various compiler warnings and other problems with the new code on this branch. 
FossilOrigin-Name: 3a623cfa173b4035c759cb84985d11d8727053beb383648503987d6ab15c0ef0
 2023-12-02 20:35:04 +00:00
dan
c44041e03b Ensure that tokendata=1 queries avoid loading large doclists for queries like "common AND uncommon", just as tokendata=0 queries do. 
FossilOrigin-Name: 7bda09ab404a110d57449e149a3281fca8dc4cacf7bd9832ea2a1356ad20fe8e
 2023-12-02 17:32:16 +00:00
dan
b5effc0605 Different approach to querying a tokendata=1 table. Saves cpu and memory. 
FossilOrigin-Name: c523f40895866e6fc979a26483dbea8206126b4bbdf4b73b77263c09e13c855e
 2023-12-01 20:09:59 +00:00
dan
af54826e4a Defer building xInstToken() hash-table until it is to be used. 
FossilOrigin-Name: 9b005085ff4a53cda0a1dff0c836630d6d3b95b9c40658ffd2a886f3e1b37faa
 2023-11-22 20:02:55 +00:00
dan
5c268bbf67 Fix tokendata=1 and xInstToken() APIs for detail=none and detail=column tables. 
FossilOrigin-Name: 37b271c19d772bd06524db816ded03377b426efed7a7783c8a96f6fb156ecd86
 2023-11-22 19:02:54 +00:00
dan
50b0e25a55 Add implementation of xInstToken() API. 
FossilOrigin-Name: a34b26fe7f60b74e7ae5cf64900920a3d352a20da2496401bcbc27041689cd07
 2023-11-15 11:45:19 +00:00
dan
e108029332 Add new fts5 API xQueryToken(). 
FossilOrigin-Name: 828566392b3ea8db603cb1ae5eccbc8ac035efaa284bc7c15ba89874f634aec9
 2023-11-13 14:29:12 +00:00
dan
03204e9106 Add the tokendata=1 option to ignore trailing token-data when querying an fts5 table. 
FossilOrigin-Name: 122935182ad5869ce3a4c6d796c38a0509f6f3384dd1b3e60a3f2f0f366cc5f5
 2023-10-11 21:08:12 +00:00
dan
a35ae44150 Changes so that fts5 can handle tokens with embedded '\0' bytes. 
FossilOrigin-Name: c027c092c4af53bd6ae3cc6e2b4439167d9eeb0f9de549b6a2c2a72a67ee886c
 2023-09-30 18:13:35 +00:00
dan
3f874b58fb Change the name of the fts5 'delete-automerge' option to 'deletemerge'. And add tests for it. 
FossilOrigin-Name: 1079300db2a7d1fbc86a01c215c234a3af64889c5396e6da63ff4f3c7efae4c5
 2023-07-25 15:48:58 +00:00
dan
24730de8d1 Add the fts5 'delete-automerge' integer option. A level is eligible for auto-merging if it has a greater than or equal percentage of its entries deleted by tombstones than the 'delete-automerge' option. Default value is 10. 
FossilOrigin-Name: b314be66b9ac0190b5373b3b6baec012382bc588c2d86c2edab796669a4303c3
 2023-07-24 19:13:06 +00:00
dan
2159292ce0 Integrate contentless delete with auto-merge. 
FossilOrigin-Name: 85c1589ab1fc69d1eef4bbc1bdefa2b10af5f6b9c08e813130b93829b592f416
 2023-07-22 19:47:46 +00:00
dan
55e0fd4a9d Do not allow the 'delete' command to be used on contentless_delete=1 fts5 tables. 
FossilOrigin-Name: cc694b83408ccb5d42204cb624145c76e95329cbe1d1fe8815c70a7a00af231a
 2023-07-17 17:59:58 +00:00
dan
6788c7b7c0 Begin adding support for deleting rows from contentless fts5 tables. 
FossilOrigin-Name: e513bea84dfaf2280f7429c9a528b3a1354a46c36e58ab178ca45478975634e0
 2023-07-10 20:44:09 +00:00
drh
3442306989 Protect a macro argument with parentheses in FTS5. 
FossilOrigin-Name: bc07fe51fe0c6bb50ca8ae1baefcc35c8f5395b2d0de641bf0b0cedc92d754d4
 2023-05-03 13:48:33 +00:00
dan
015020cd1a Add the 'secure-delete' option to fts5. Used to configure fts5 to aggressively remove old full-text-index entries belonging to deleted or updated rows. 
FossilOrigin-Name: 4240fd09b717dbc69dffe3b88ec9149777ca4c3efa12f282af65be3af6fa5bb0
 2023-04-12 17:40:44 +00:00
dan
6aafd74853 Avoid some cases of signed integer overflow in fts5 by casting to unsigned values. 
FossilOrigin-Name: 46a78c8c0ed518c4521e6e0bdebeb065bab07076abc444775002e7f4361d2242
 2022-08-08 19:29:53 +00:00
drh
134f544ab2 Fix a harmless scan-build warning in FTS5. 
FossilOrigin-Name: 0bf42bb5611dc3f672cb898b8be245fd25f7a3862c1e0734effd18d75e812f22
 2021-10-16 19:50:03 +00:00
drh
922c54206f Some #defines somehow failed to get set correctly in the previous check-in. 
Fixed here.

FossilOrigin-Name: 15bbdf9ac840a220f384411d3025ef22f949d310194b60bca8e6d6a759e6042e
 2021-10-04 18:57:42 +00:00
drh
11a9ad5669 Fix harmless static analyzer warnings in sessions, rtree, fts3 and fts5. 
Add the -DSQLITE_OMIT_AUXILIARY_SAFETY_CHECKS compile-time option to cause
ALWAYS() and NEVER() macros to be omitted from the build.

FossilOrigin-Name: 1c67f957fc77e37ce8f0d447c41ca975e8e79a35d332739c24a633649b5b0387
 2021-10-04 18:21:14 +00:00
dan
cc516af4cc Instead of disallowing writes to fts5 tables if there are fts5vocab cursors open on them (commit [c49a6ed7]), abort any fts5vocab queries if the on-disk structure of the fts5 table changes. 
FossilOrigin-Name: 9dbdc9001e3258e71ca995fbcdebf66ab95890ded87fa7125c6cb4bd43010aaf
 2021-07-07 11:51:03 +00:00
dan
b9324fea07 Do not allow writes to an fts5 table if there are any open fts5vocab cursors. 
FossilOrigin-Name: c49a6ed78a917d4972e048e2a9bbe4d400691f97ce7e022f0e4436ceaed7fb73
 2021-07-05 19:01:09 +00:00
dan
f46be6a1b9 Allow fts5 trigram tables created with detail=column or detail=none to optimize LIKE and GLOB queries. Allow case-insensitive tables to optimize GLOB as well as LIKE. 
FossilOrigin-Name: 64782463be62b72b5cd0bfaa7c9b69aa487d807c5fe0e65a272080b7739fd21b
 2020-10-05 16:41:56 +00:00
dan
33a99fad08 Add experimental unicode-aware trigram tokenizer to fts5. And support for LIKE and GLOB optimizations for fts5 tables that use said tokenizer. 
FossilOrigin-Name: 0d7810c1aea93c0a3da1ccc4911dbce8a1b6e1dbfe1ab7e800289a0c783b5985
 2020-09-30 20:35:37 +00:00
dan
7548ab20e6 In fts5 integrity checks, do not compare the contents of the index against an external content table unless specifically requested. 
FossilOrigin-Name: 782163693f37aeb65209bebbaeb6659a36881b8c4b4bec778b366658488bf966
 2020-09-21 14:53:21 +00:00
dan
52612bec3c Fix a resource leak in fts5 that could occur if an auxiliary function is called from within a query that does not use the full-text index. 
FossilOrigin-Name: b528bdcd45db1b783ecd9739c3d3c890f04de7003f079668970eafaf8e23b2f3
 2019-10-20 08:26:08 +00:00
dan
ae55737fbf Do not allow users to effectively disable fts5 crisismerge operations by setting the crisismerge threshold to higher than the maximum allowable segment b-trees on a single level. Fix for [d392017c]. 
FossilOrigin-Name: 86e497209217abb7bcb491a023cd353f3c7c9c103ebd9f58dd8661b12cf3694c
 2019-10-09 18:36:32 +00:00
dan
685b2ee0c3 Allow fts5 to filter on multiple MATCH clauses in a single scan. 
FossilOrigin-Name: 9d418a7a491761eeb38a70898677a493e2631e5d62e75ee88431f52d3dfd2344
 2019-09-12 19:38:40 +00:00