modified fts2:
Modify handling of SQLITE_SCHEMA in fts2 code. An SQLITE_SCHEMA error
may cause SQLite to reload the internal schema, deleting and
recreating v-table objects. So the sqlite3_vtab structure can be
deleted out from under a v-table implementation. (CVS 4183)
FossilOrigin-Name: f9020cffda02923ef45979bb447ec2e232086ad5
is omitted. Add the SQLite blessing to the header comments on all FTS2
source files. (CVS 4120)
FossilOrigin-Name: c795e6fd8f01bcbc1967062632c13d4952abf4d8
docids are ascending if there was a prior docid set for the doclist,
ignore the initial docid of 0. (CVS 4026)
FossilOrigin-Name: ed3a131f1d3fe51d1e79bdfe1bfafa55f825afa9
character immediately after the end of a term is '*', that term is
marked for prefix matching. Modify term comparison in
snippetOffsetsOfColumn() to respect isPrefix. fts2n.test runs prefix
searching through some obvious test cases. (CVS 3893)
FossilOrigin-Name: 7c4c65924035d9f260f6b64eb92c5c6cf6c04b7b
The new function docListUnion() is used to accumulate a union of the
hits for the matching terms, which will be merged across segments
using docListMerge(). (CVS 3891)
FossilOrigin-Name: 72c796307338c2751a91c30f6fb16989afbf3816
Also implement correct prefix-handling for traversal of interior nodes
of segment tree. A given prefix can span multiple children of an
interior node, and from there the branches need to be followed in
parallel. (CVS 3889)
FossilOrigin-Name: cae844a01a1d87ffb00bba8b4e7b62a92e633aa9
search. Doclists from multiple prefix matches will need a union merge
function, which will have to logically happen across a segment before
doclists are merged between segments. (CVS 3887)
FossilOrigin-Name: 7ddb82668906e33e2d6a796f2da1795032e036d5
Previously, the code looped until the block was a leaf node as
indicated by a leading NUL. Now the code loops until it finds a block
in the range of leaf nodes for this segment, then reads it using
LeavesReader. This will make it easier to traverse a range of leaves
when doing a prefix search. (CVS 3884)
FossilOrigin-Name: 9466367d65f43d58020e709428268dc2ff98aa35
Prefix-searching will want to accumulate data across multiple leaves
in the segment, using LeavesReader instead of LeafReader is the first
step in that direction. (CVS 3881)
FossilOrigin-Name: 22ffdae4b6f3d0ea584dafa5268af7aa6fdcdc6e
the other, the code potentially tries to read past the end of the
doclist. http://www.sqlite.org/cvstrac/tktview?tn=2309 (CVS 3862)
FossilOrigin-Name: dfac6082e8ffc52a85c4906107a7fc0e1aa9df82
assumed that the row had values in all columns, sigh. Fixes bug
http://www.sqlite.org/cvstrac/tktview?tn=2289 . (CVS 3833)
FossilOrigin-Name: 81be7290a4db7b74a533aaf95c7389eb4bde6a88
updates happen within a single transaction, there was a lot of wasted
encode/decode overhead due to segment merges. This code buffers
updates in memory and writes out larger level-0 segments. It only
works when documents are presented in ascending order by docid.
Comparing a test set running 100 documents per transaction, the total
runtime is cut almost in half. (CVS 3751)
FossilOrigin-Name: 0229cba69698ab4b44f8583ef50a87c49422f8ec
assertions when this occurs, and it's almost certainly not the right
thing to do in the first place. (CVS 3746)
FossilOrigin-Name: f6c3abdc6c5e916e5366ba28fb1cd06ca3554303
Collector) now handles the case where PLWriter (Position List Writer)
needed a local buffer. Change to using the associated DLWriter
(Document List Writer) buffer, which reduces the number of memory
copies needed in doclist processing, and brings PLWriter operation in
line with DLWriter operation. (CVS 3707)
FossilOrigin-Name: d04fa3a13a84f49074c673b8ee2fb6541da061b5
Currently, PLWriter (Position List Writer) creates a locally-owned
DataBuffer to write into. This is necessary to support doclist
collection during tokenization, where there is no obvious buffer to
write output to, but is not necessary for the other users of PLWriter.
This change adds a DLCollector (Doc List Collector) structure to
handle the tokenization case.
Also fix a potential memory leak in writeZeroSegment(). In case of
error from leafWriterStep(), the DataBuffer dl was being leaked. (CVS 3706)
FossilOrigin-Name: 1b9918e20767aebc9c1e7523027139e5fbc12688
malloc/calloc/realloc appropriately, and use sizeof(var) instead of
sizeof(type) to make certain that we don't get a mismatch between
them as the code rots. (CVS 3693)
FossilOrigin-Name: fbc53da8c645935c74e49af2ab2cf447dc72ba4e
When creating fts tables in an attached database, the backing tables
are created in database 'main'. This change propagates the
appropriate database name to the routines which build sql statements.
Note that I propagate the database name and table name separately. I
briefly considered just making the table name be "db.table", but it
didn't fit so well in the model used to store the table name and other
information, and having the db name passed separately seemed a bit
more transparent. (CVS 3631)
FossilOrigin-Name: 283385d20724f0144f38de89bd179715ee5e738b
Calling UPDATE against an fts table in a UTF-16 database inserts
corrupted data into the database. The UTF-8 data is being inserted
directly. This appears to happen because sqlite3_ value_text()
destructively coerces a value to UTF-8, and it's never converted back
when updating the table. This works around the problem by rearranging
things so that the update happens before the coercion. (CVS 3596)
FossilOrigin-Name: 4f2ab4b6320ffc621900049b41f50bc30d76d7f5
The virtual table interface allows for a cursor to field multiple
xFilter() calls. For instance, if a join is done with a virtual
table, there could be a call for each row which potentially matches.
Unfortunately, fulltextFilter() assumes that it has a fresh cursor,
and overwrites a prepared statement and a malloc'ed pointer, resulting
in unfinalized statements and a memory leak.
This change hacks the code to manually clean up offending items in
fulltextFilter(), emphasis on "hacks", since it's a fragile fix
insofar as future additions to fulltext_cursor could continue to have
the problem. (CVS 3521)
FossilOrigin-Name: 18142fdb6d1f5bfdbb1155274502b9a602885fcb
that this is of marginal utility when encoding terms resulting from
regular English text, it turns out to be very useful when encoding
inputs with very large terms. (CVS 3520)
FossilOrigin-Name: c8151a998ec2423b417566823dc9957c7d5d782c
between leaf nodes, instead of storing the entire leftmost term of the
rightmost child, store only that portion of the leftmost term
necessary to distinguish it from the rightmost term of the leftmost
child. (CVS 3513)
FossilOrigin-Name: f6e0b080dcfaf554b2c05df5e7d4db69d012fba3
LeafWriter to use empty data buffer (instead of empty term) to detect
an empty block. Code to validate interior nodes. Moderate revisions
to leaf-node and doclist validation. Recast leafWriterStep() in terms
of LeafWriterStepMerge(). (CVS 3512)
FossilOrigin-Name: f30771d5c7ef2b502af95d81a18796b75271ada4
where excessively large terms keep the tree from finding a single
root. A downside is that this could result in large interior nodes in
the presence of large terms, which may be prone to fragmentation,
though if the nodes were smaller that would translate into more levels
in the tree, which would also have that problem. (CVS 3510)
FossilOrigin-Name: 64b7e3406134ac4891113b9bb432ad97504268bb
( http://www.sqlite.org/cvstrac/chngview?cn=3486 ) broke test fts2a-5.3.
This change should make the expected result more obvious. (CVS 3489)
FossilOrigin-Name: cde383eb467de0d752e94a22cd2f890c2dc599cc
