e8dee7d1ed
Modify the fts3tokenize table implementation so that it does not use the SQL function fts3_tokenizer. The user may have installed an authorizer callback that prohibits this.
...
FossilOrigin-Name: 0ba67b64de258883e4c43db09e131bb67083855e
2013-05-07 12:16:48 +00:00
acf9c2b555
Add an assert() to fts3_expr.c in order to silence a clang warning.
...
FossilOrigin-Name: d8dc2c5fb554eb5898ddc8d703c6a360af7fe023
2013-05-06 14:57:48 +00:00
3a01b600d3
Improve the error message issued when an FTS query exceeds the maximum allowable tree depth.
...
FossilOrigin-Name: f480b1fe6012f36c59cd0525efdc6df74143ccd0
2013-04-29 18:07:37 +00:00
181f4f789d
Fix an off-by-one in the code for limiting the depth of FTS expression trees.
...
FossilOrigin-Name: 72ac73189c3577740a77d2ea2fc7118391c0703f
2013-04-29 17:12:06 +00:00
6f77140f78
Fix harmless compiler warnings in the FTS expression parser.
...
FossilOrigin-Name: 3c78af8c535e16518f18733325f4cd1df7ec8282
2013-04-26 13:14:19 +00:00
aa29c86e49
Rebalance FTS expressions after parsing to limit recursion during evaluation. Avoid recursion when deleting FTS expression trees. Enforce a limit on the depth of an expression tree.
...
FossilOrigin-Name: f968d43f80cc2f236e7d09ba1e8278343e2b6976
2013-04-25 20:34:02 +00:00
da8caa0b2d
Fix harmless compiler warnings.
...
FossilOrigin-Name: 1a1cf5aa86734c832d845e07780262a178188d56
2013-04-22 23:38:50 +00:00
4d648c777b
Update MSCV makefile for the fts3tokenize virtual table. Clear some
...
harmless compiler warnings in the fts3tokenize code.
FossilOrigin-Name: 2b3fed09eb89f1e9d6a5f7487ef526b46eb51490
2013-04-22 17:07:56 +00:00
d7a959c694
Add virtual table module 'fts3tokenize' to fts3. fts3tokenize provides SQL access to FTS tokenizer implementations.
...
FossilOrigin-Name: c5a4b21a15bd8a184380817ff41657acbccb866d
2013-04-22 15:30:37 +00:00
a8f60381a9
Change the fts4aux virtual table module so that fts4aux tables created in the temp database may report on fts3/fts4 tables in any attached database.
...
FossilOrigin-Name: 546a187f1361dad340ed8f6c28dd89e6c19f9c33
2013-04-12 16:47:27 +00:00
1cb90c668c
In fts3, when filtering lists for hits in a specific column, edit the list in place in the same way as it is for NEAR filtering. Fix for [38b1ae018f].
...
FossilOrigin-Name: f85f9103cffa5c8ba6a63a68beb90817147ba080
2013-03-25 11:38:44 +00:00
48864df97d
Many spelling fixes in comments. No changes to code.
...
FossilOrigin-Name: 6f6e2d50941e444ebc83604daddcc034137a05b7
2013-03-21 21:20:32 +00:00
7b6be6f7bf
Fix an uninitialized variable detected by valgrind.
...
FossilOrigin-Name: ffd1e51490286abfaea68fd4b4b4cb967d87b04b
2012-12-04 00:53:08 +00:00
bba44d0d1b
Fix a problem causing the "number-of-documents" field maintained by FTS4 to be set incorrectly.
...
FossilOrigin-Name: e38fb02d5ea5daa6992df4dfbbeec92bf7b525f6
2012-11-27 15:56:38 +00:00
e62c0694de
Fix a bug in fts3 causing it to return SQLITE_NOMEM when NEAR, OR and some auxiliary functions were used together. Also a segfault in vtab.c that could follow an OOM condition.
...
FossilOrigin-Name: 33f4f9817e8b3cb97ce02a4c49bd586332da37b2
2012-10-25 19:25:39 +00:00
8749c1831a
Make sure substructure elements have proper alignment in the ICU tokenizers
...
of FTS2 and FTS3.
FossilOrigin-Name: aaa2d9b0db74d8452d9294de17cff786ab4ec7c8
2012-10-19 02:10:53 +00:00
1276cd0ba8
Add a comment to fts3_snippet.c.
...
FossilOrigin-Name: b5f5ba3103e6fcf117fb69d5379fa70f691a652d
2012-10-17 20:28:52 +00:00
6c42aa4a0e
Initialize variables used as outputs of custom tokenizer method calls in fts4.
...
FossilOrigin-Name: 514505a84bcc677518d60a42896598f85ed78265
2012-10-17 20:15:10 +00:00
d5578433ff
Fix all known instances of 'repeated the' style typos in comments. No changes to code.
...
FossilOrigin-Name: 9b19b847533f944f289d93dcdba29c0d67bf251c
2012-08-25 10:01:29 +00:00
4dc3d73d69
Add the SQLITE_DISABLE_FTS4_DEFERRED compile time option.
...
FossilOrigin-Name: e799222f3b8246e65657a758437914ece7069ba9
2012-08-20 17:24:48 +00:00
fe7b2bc9ff
Merge several compiler compatibility fixes to trunk.
...
FossilOrigin-Name: d3d491a5461e21d84c6425977b80deedc7fa8a3a
2012-06-21 14:05:15 +00:00
2ae26b759d
Fix a problem with identifying white-space characters outside of the ascii range in the ICU tokenizer.
...
FossilOrigin-Name: 892b74116a3b23268895b96433d18ef00c1433d8
2012-06-18 20:52:32 +00:00
c8bde37d19
Fix a few compilation issues that can occur with certain compilers (e.g. GCC 2.95.3, MSVC).
...
FossilOrigin-Name: f970a3de61fe2ebaf1778c4a3383cfdc52299162
2012-06-18 08:00:56 +00:00
1c30debd41
Fix an obscure memory leak in FTS3 that can come about when two memory
...
allocations are immediately adjacent to one another.
FossilOrigin-Name: 025227be5495f950c466dfabac140cba69e498be
2012-06-08 14:01:53 +00:00
25cdf46ae4
Add the "tokenchars=" and "separators=" options, for customizing the set of characters considered to be token separators, to the unicode61 tokenizer.
...
FossilOrigin-Name: e56fb462aa1f11bb23303ae0dc62815c21e26a52
2012-06-07 15:53:48 +00:00
2c897e3e5f
Disable FTS unicode61 by default. It is enabled by specifying compile time option SQLITE_ENABLE_FTS4_UNICODE61.
...
FossilOrigin-Name: eccd6b6580637084495b80e9232262188ba0cf8d
2012-06-06 19:51:27 +00:00
754d3adf7c
Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0".
...
FossilOrigin-Name: 790f76a5898dad1a955d40edddf11f7b0fec0ccd
2012-06-06 19:30:38 +00:00
a9cfaba95a
Omit the fts3 unicode character class routines from the build if fts3/4
...
is disabled.
FossilOrigin-Name: c00bb5d4601efc15933f222349e96a043b610a19
2012-05-28 12:22:00 +00:00
7946c53009
If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer.
...
FossilOrigin-Name: e71495a817b479bc23c5403d99255e3f098eb054
2012-05-26 18:28:14 +00:00
501c74d3e1
Change the format of the tables used by sqlite3FtsUnicodeTolower() to make them a little smaller.
...
FossilOrigin-Name: b89d3834f6690073fca0fc22c18afa1fb280ea7d
2012-05-26 17:57:02 +00:00
cf9f6f1375
Add fault-injection tests that use the unicode61 tokenizer. Fix a problem revealed by the same.
...
FossilOrigin-Name: ed28c48a3dd7e766e60db0d96ef5460bf9913e6b
2012-05-26 15:44:08 +00:00
ab322bd21e
Change the name of the "unicode" tokenizer to "unicode61" to emphasize that the case folding and separator-character identification routines are based on unicode version 6.1.
...
FossilOrigin-Name: 8f3e60aa2253f21bcee5d03982cfdd7f16c00060
2012-05-26 14:54:50 +00:00
1c7016c9a5
Add special fast paths to sqlite3FtsUnicodeTolower() and Isalnum() for codepoints in the ASCII range.
...
FossilOrigin-Name: cf7b25d47687635a04f4347d45f135c686b9d758
2012-05-25 19:50:12 +00:00
80ed5a56a5
Fix comments in generated file fts3_unicode2.c.
...
FossilOrigin-Name: 3dc567ef4702d9a63d78d11ff705cb7f7359f7a6
2012-05-25 18:48:48 +00:00
3d403c71a8
Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators.
...
FossilOrigin-Name: 0c13570ec78c6887103dc99b81b470829fa28385
2012-05-25 17:50:19 +00:00
5cfed52dd1
Have the FTS auxiliary functions correctly handle terms that appear in non-matching branches of the FTS expression. Fix for [bdc6bbbb38].
...
FossilOrigin-Name: 4d3e1673b309a3d6a6cac01d67dfd79a8007f991
2012-05-10 17:43:14 +00:00
83cc139231
Fix harmless compiler warnings on x64 MSVC, mostly in test code, but also in
...
tclsqlite.c and in the FTS4 module.
FossilOrigin-Name: 3281972eaa46cb57fd9f0387063f47430dc0a3b4
2012-04-19 18:04:28 +00:00
85e7243acf
Fix harmless static-analysis warnings, mosting having to do with memory
...
leaks in the command-line shell. Add a clang analysis of the command-line
shell to the "warnings-clang.sh" script. Other minor cleanups to the
command-line shell code.
FossilOrigin-Name: 93a0f452a7023898ad3d62ee81b39a80477c332f
2012-04-11 11:38:53 +00:00
3b06a2a056
Add #ifdefs to allow a test build to succeed even if SQLITE_ENABLE_FTS3 is
...
not defined.
FossilOrigin-Name: fb121980e48af368353431fd04924e414b65c852
2012-04-02 17:18:23 +00:00
a25a42cfe7
Change the ICU tokenizer so that it does not attempt to call strlen(NULL).
...
FossilOrigin-Name: 04298f1ac42c40cb2a48092b415acf96a08954b7
2012-03-31 11:58:23 +00:00
7da5fcb0b7
Fix MSVC compiler warnings in test code.
...
FossilOrigin-Name: cb7a850439c9a4a7887650d6b81d95ab8025de5b
2012-03-30 14:59:43 +00:00
4e245a4c35
Fix compiler warnings on GCC and MSVC and fix a C89-ism that
...
broke the build for MSVC.
FossilOrigin-Name: b451c0f97f0abe78ebe6c62ff489ec1ad8a1f767
2012-03-30 00:00:36 +00:00
cdb86dc437
Add output of PRAGMAs auto_vacuum and encoding to the "schema" command of the
...
fts3view utility program.
FossilOrigin-Name: e31076319363a46905836880765bae3bf204ed19
2012-03-27 18:00:05 +00:00
311ec02587
Remove the fts3merge.test script in favour of changing the fts4merge.test script so that it runs tests using both fts4 and fts3. Fix some problems with incr-merge and FTS3 tables.
...
FossilOrigin-Name: 5c447e226afca0d46b9ed994dea26a16a9ae168c
2012-03-27 15:00:06 +00:00
a1a9f0a0d8
In the fts3view utility, label the blank segments used to mark the end of a
...
segment sequence for a level/idx as "null". Improve the alignment of root
segment names.
FossilOrigin-Name: 04aea0245e4183fef3664609f5a6353b65d71a85
2012-03-27 14:54:44 +00:00
1ecb59a1e5
Enhance the fts3view tool with the big-segment command and fix a bug in the
...
display of doclists.
FossilOrigin-Name: e9436d8038e5a0d1ba992a77d1064d4a55595f57
2012-03-27 13:51:31 +00:00
5da0aa1603
Allow multiple incremental merges to proceed concurrently. This is required to prevent a large crisis-merge from occuring while an even larger incremental-merge is underway.
...
FossilOrigin-Name: 7ed9d2f24a650b424b97dfc19b8042c4cf09c82c
2012-03-27 11:48:02 +00:00
40aab4c56d
Minor correct errors in the file format description for FTS3/4 contained in
...
the fts3.c header comment.
FossilOrigin-Name: fb8aacdd8fbdc946cb271cc589f76b806387937d
2012-03-27 00:38:33 +00:00
280d5c33cd
Enhance fts3view to show decodes of segments and doclists.
...
FossilOrigin-Name: 6d09de231b68dd9520d99c65d133f26e90eb784f
2012-03-27 00:34:04 +00:00
36f6b891e6
Add the fts3view utility program.
...
FossilOrigin-Name: f936c8ea16d21345fd1622272dc7e9850acb2493
2012-03-26 21:57:53 +00:00
