1
0
mirror of https://github.com/sqlite/sqlite.git synced 2025-11-16 23:02:26 +03:00
Commit Graph

1577 Commits

Author SHA1 Message Date
drh
991d1085e3 Add requirements marks and tuning.
FossilOrigin-Name: ac951490fd7d5864fe422a80ee8557478e823e79461bec2ee538f57b6733eb5a
2022-01-21 00:38:49 +00:00
drh
82801a5b72 Initial implementation of the sqlite3_vtab_rhs_value() interface and the
qpvtab extension used for testing the virtual table interface.

FossilOrigin-Name: 0873c76b9b96b66fa9d13ddc8bca126d575ea3352349c7fd648f0c2f75d770f5
2022-01-20 17:10:59 +00:00
drh
e1961c55c3 When constructing the sqlite3_index_info object for the xBestIndex method
of a virtual table, omit constant trims from the ORDER BY clause, as they
will always be in the correct order.

FossilOrigin-Name: 524c2b87d74c8cad6fb377aed7275788d61beafe61e675480de732519987102c
2021-12-30 17:36:54 +00:00
drh
b6592f6563 Improved documentation for the sqlite3_vtab_collation() interface.
FossilOrigin-Name: 826f84a92d3b1f760560bff72c911a2c664737820af0c9114eb3494df7b8e878
2021-12-17 23:56:43 +00:00
drh
b71a485386 Fix harmless compiler warning in the new Bloom filter logic.
FossilOrigin-Name: 9406d95d3fbaf0d8e65623adb790845f78a7456fb32aafd7fa4f74eda2f2dacc
2021-12-16 14:36:36 +00:00
drh
8a95d3d433 Simplify the generation of the sqlite3_index_info object during query planning
for virtual tables.

FossilOrigin-Name: 241dc0428a6e0238c57e2449e98ea60047e777e29c83a4ebe6da16f7cba40e19
2021-12-15 20:48:15 +00:00
drh
52576b78f6 Enhance the virtual table query planner so that it is able to deal with
ORDER BY terms that contain COLLATE clauses as long as the specified
collation matches the virtual table.  This is especially important for
UNION ALL since a "COLLATE binary" is added to ORDER BY clauses if no
COLLATE clause exists in the original SQL.

FossilOrigin-Name: 5c3d398d20b86a1558720e995eddf11403aec2d160590571fa9525fe8f6efff9
2021-12-14 20:13:28 +00:00
drh
4f2006ddec Add NEVER() on an unreachable branch in the Bloom filter pull-down logic.
FossilOrigin-Name: 471070462593faba865d8d05ee89161b340145351e2fa0bfbaa766d974e932d6
2021-12-13 18:53:10 +00:00
drh
c5860af006 Fix an off-by-one error in the Bloom filter pulldown logic, found by OSSFuzz.
Also fix over-length source code lines in the immediate vicinity.

FossilOrigin-Name: 027626521c02be06ef61e8229bde49d20cb3f1cb600c4cb127c5f139b9de8858
2021-12-13 18:43:46 +00:00
drh
dc56dc93a5 Do not allow early evaluation of Bloom filters that use the IN operator as
the machinery to deal with the IN operator is not available.
dbsqlfuzz 5b51c247518278f79a45cea978702e86e86cd4f9.

FossilOrigin-Name: 799db7cb2e0d73031182d26a0e5919368f9f9823df81cb2863bfe79eca344f5c
2021-12-11 17:10:58 +00:00
drh
27a9e1f615 Rename the internal routine constructBloomFilter() to
sqlite3ConstructBloomFilter(). OSSFuzz is reporting a crash with a garbled
stack that we cannot reproduce. Perhaps the original "constructBloomFilter()"
name is colliding with some internal name used by OSSFuzz. We'll see if this
rename clears the problem.

FossilOrigin-Name: 403e7312dd9a3fe493a21aceb82e387d6f152622d66c1b403c881597713e8cc3
2021-12-10 17:36:16 +00:00
drh
5a4ac1cc30 Remove unused code and fix comments. Final cleanup before merging.
FossilOrigin-Name: ce42039f5647b1f276acf5d9911528ecb47df1544a587def72c8cd6b2f664289
2021-12-09 19:42:52 +00:00
drh
a11c5e22b5 Enable bloom filters for the right table of a LEFT JOIN. Fix unreachable
branches.

FossilOrigin-Name: d342ab722de1fc3f34219c9755253db9d88eb50a6fa5cc257207008b00e7fc03
2021-12-09 18:44:03 +00:00
drh
7e910f6422 Add SQLITE_TESTCTRL_LOGEST and enhance the LogEst utility program.
Improvements to testability of bloom filters.

FossilOrigin-Name: 88b43d798cc5aa59855e92d3e658aee9f0a5def6ffbc5db77af048d75ecdf8cc
2021-12-09 01:28:15 +00:00
drh
fb82caf0be Improvements on the decision of whether or not to use a Bloom filter.
FossilOrigin-Name: 0fb2a4e08f518cb38ea3edc6a084d1e4874fd622ba3cf9101b49b3e7dc1a3f2b
2021-12-08 19:50:45 +00:00
drh
c491a25631 Merge the WhereClause fix from trunk
FossilOrigin-Name: d3250256772e3348abe887c0ca3550a6647cce3804c9456a9d0112aea7ee1c46
2021-12-08 16:15:41 +00:00
drh
132f96fc75 In the WhereClause object, do not assume that all TERM_VIRTUAL terms appear
at the end of the list, because that is no longer true.  Instead, keep a
separate nBase count that is the size of the list excluding the tail of
virtual terms.  Use nBase instead of nTerm when scanning terms that are not
virtual.  Add assert()s to validate correctness of WhereClause.

FossilOrigin-Name: 6024682ca467fa4fe49608772b0bbfa2f8a419b32cebfa715941073c8b29da49
2021-12-08 16:07:22 +00:00
drh
761d64b73b Do not generate a Bloom filter if it cannot be used prior to the next seek,
as that leads to a misleading EXPLAIN QUERY PLAN.

FossilOrigin-Name: 2739ed5192058fbcc816ecbc252be687efc606e038bfcd6cf71194a3f4f5684e
2021-12-07 22:37:50 +00:00
drh
50fb7e09b4 Omit the OP_FilterInit opcode. Use OP_Blob to initialize each Bloom filter
instead.  Size the Bloom filter based on sqlite_stat1 size estimates rather
than a run-time measurement for improved testability.

FossilOrigin-Name: 8a9036ee617a6ad93bfe827b0789773c49d3d45b085cb76fa4b9b20a41b79b97
2021-12-06 20:16:53 +00:00
drh
5baaf40af1 Attempt to vary the size of Bloom filters based on an estimate of how many
keys the filter will hold.

FossilOrigin-Name: a7adcf69088cba4b86cc5731a45c9a5263af4355bc0a38f5225cab421c915f7f
2021-12-06 13:07:28 +00:00
drh
6ae49e67cc Run as many Bloom filters as possible before index lookups.
FossilOrigin-Name: 06f6fefd67086896bc49272c6319545ff6c6792f18babe23aced27b60b032119
2021-12-05 20:19:47 +00:00
drh
35685d3e5e Try to run all Bloom filters before any Seeks. This gives a small performance
gain on the Star-Schema Benchmark.

FossilOrigin-Name: 5be2470f8755ef454f813c880e659bdbf82f2396be9320cf3079cd4ca8e81a19
2021-12-05 00:45:55 +00:00
drh
fecbf0a179 Miscellaneous cleanup of the new Bloom-filter code.
FossilOrigin-Name: 201b6dd875b0ae2bbc9969b098e88abfc09e37b59e857decd41f2dcbeeb13e01
2021-12-04 21:11:18 +00:00
drh
067c60cfc9 Add VdbeCoverage() macros. Adjust the Bloom-filter hash function so that it
correctly deals with zero-blobs.

FossilOrigin-Name: 629ee2e3e3125bfd2af435c6713d49e46691213ad15db0a5e93a63a77f1130c2
2021-12-04 18:45:08 +00:00
drh
770dade262 Apply the Bloom filter only on those terms of an index that have equality
constraints.

FossilOrigin-Name: a70429596a3c6a413b03118b0d800521b3526d99dcf88a48acc3189b51518d82
2021-12-04 14:24:30 +00:00
drh
fa35f5c5a7 First attempt to use Bloom filters to optimize star-schema queries.
FossilOrigin-Name: 28161fba9bcde5ae4b36b22d766c881b795af111a3a323c90f6149d0fea9297d
2021-12-04 13:43:57 +00:00
drh
7439d3580c Merge the sqlite3WhereBegin() simplification from trunk.
FossilOrigin-Name: 41ba2dfdd3a18671fc78d60935a16fa50f36af3d6481eff2ca9fba88e7093997
2021-12-03 19:10:17 +00:00
drh
70b403b667 Factor the noop-join-elimination optimization out of the sqlite3WhereBegin()
routine and into a separate "no-inline" subroutine, in order to reduce the
complexity of sqlite3WhereBegin() and thereby encourage C compilers to
inline sqlite3WhereCodeOneLoopStart().

FossilOrigin-Name: 6225e9abcb0261fefca4a26530524ffc449f937f8ae1ece718af2c3c3d73d78d
2021-12-03 18:53:53 +00:00
drh
2b05905521 Merge recent trunk enhancements into the bloom-filter branch.
FossilOrigin-Name: 11d97fb8be6b5155f38df130d7e828edd0f381e32f651458939b1cb9cb973fff
2021-12-03 16:02:05 +00:00
drh
844a89b518 More small performance optimizations for sqlite3WhereGetMask().
FossilOrigin-Name: 3de2c557d915d1b4a9b0e540d5e80ffb243e7123d0ec65c5052f4d6f1cce56b2
2021-12-02 12:34:05 +00:00
drh
dae2a10934 Small performance optimization in sqlite3WhereGetMask().
FossilOrigin-Name: 7edec54ab045c0493fc5d27d47d7bf0e08d3b0b298f428646fd3f0ee24dfcade
2021-12-02 04:00:45 +00:00
drh
79ab384121 Small performance enhancement for whereScanInit().
FossilOrigin-Name: ed84e124c6097b756ca687ac64df7ee801d14b16c835f1a9e8b336f785157d85
2021-12-02 02:22:35 +00:00
drh
9b3bfa00cd Performance improvement in query planning.
FossilOrigin-Name: ca59533bbb7f21d7b39dee7999d3f36ac516f8c75b2203039beeb2852ada42d4
2021-12-02 01:30:16 +00:00
drh
2db144c33b Add a Bloom filter to the automatic-index mechanism.
FossilOrigin-Name: 50ac4de1d7cbb586ea7969e1ae80ea8b021e194edc2fa7db19374b4ee9369bee
2021-12-01 16:31:02 +00:00
drh
c1085ea412 In the automatic index generator logic, be more precise about when a
partial automatic index is allowed in order to capture more cases where it
is legal to use a partial automatic index.

FossilOrigin-Name: 664b461bb5063d98047fc2e51a3827235cd9f55ca2e23cb66e719eac53fb5437
2021-11-30 14:07:58 +00:00
drh
6d64b4a027 Ensure that the window function rewrite does not leave the parse tree
in an invalid state that might cause problems downstream before the
error is recognized and unwinds the stack.  Also take steps such that
an invalid parse tree does not cause problems even if it goes 
unrecognized.  
[forum:/forumpost/398e9d5aa9|Forum post 398e9d5aa9].

FossilOrigin-Name: 0f9fc6b6073365d5159cd71e7fe08f8dadbc7b42abd324361e809502f4359155
2021-11-07 23:33:01 +00:00
drh
220f0d6f8e Protect the WhereTerm.u union using nearby assert()s and/or branches.
FossilOrigin-Name: 8a56de5b9c6f4522000f8d991373490b67b9e9d97f03c1ca2cf32816d84789ef
2021-10-15 17:06:16 +00:00
drh
0475629d47 Protect access to the WhereLoop.u union using nearby assert()s and
branches.

FossilOrigin-Name: bdd840216cc4c5293c112f182a189f7562b3cc0f6270e3c4af5eb2e8bd61ccc9
2021-10-14 19:28:28 +00:00
drh
dbfbb5a0c5 New assert() statements to protect the u1 and u2 unions of SrcList.
FossilOrigin-Name: 9b91fbcfcc14048f7d0755d47d9b7f9212fa2eaa6f3c04f417fa16c3a47943bf
2021-10-07 23:04:50 +00:00
drh
a4eeccdfdf Protect all accesses to the Expr.x union using nearby assert()s and branches.
FossilOrigin-Name: 8eaa1d4a98b24adf245bbd2fe9212aa6a924a0f09c445906d7f87574f36a7423
2021-10-07 17:43:30 +00:00
drh
3768f175cc Once corner case for the fix at [8b24c177061c3836] was not quite right.
Caught overnight by both OSSFuzz and dbsqlfuzz and repaired here.

FossilOrigin-Name: 79e90a8225b90593e384eab49a8d1d3c93a3deaf2739dcc30f6c88852df80d04
2021-10-06 10:04:04 +00:00
drh
c59ffa8c1e Fix harmless static analyzer warnings.
FossilOrigin-Name: 1ebcde72e2046dffaa408d2a5a4a5bbf35fdf4b096e0e088dcffd9360effdaa7
2021-10-04 15:08:49 +00:00
drh
36db90d3dd Fix query plans created by whereShortCut() so that they always check
transitive constraints that drive an index.  The is analogous to the
[f1f9b5de3c59489b] check-in, just for whereShortCut() rather than the
full query planner. Fix for the issue described by
[forum:/forumpost/a65cacbf5e1c41ba|forum post a65cacbf5e1c41ba].

FossilOrigin-Name: 8b24c177061c38361588f419eda9b7943b72a0c6b2855b6f39272451b8a1b813
2021-10-04 11:10:15 +00:00
dan
748d8b9cdd Have the planner ensure that if one scan uses a subset of the WHERE clause of another, that scan is estimated to cost less and return fewer rows.
FossilOrigin-Name: c7b34930e27597e7f634ad76be55fc436dcb84ea48d5b41b5d7f3596285dd672
2021-08-31 15:53:58 +00:00
drh
cf9d36d1b3 Refactor field names in the Column object, zCnName and zCnColl, to make them
unique and thus easier to find amid all the other code.

FossilOrigin-Name: 8b781dcaf68e0cf12a844708c82eee00193e340195cbca915d077e4846983bf3
2021-08-02 18:03:43 +00:00
drh
f38524d20d Refactor the Table object to reduce its memory footprint.
FossilOrigin-Name: bbb6759bcf6e01d36dfc787a82a610d359f50aaeac8104b73883a84906d54e1f
2021-08-02 16:41:57 +00:00
drh
9045e7d696 Reduce the scope of a variable in order to fix a harmless compiler warning
that occurs under -DSQLITE_OMIT_AUTOMATIC_INDEX.

FossilOrigin-Name: 36d5dbbe13562f4ea6435e620d102f8515bd0e6b8e66fd1841062a4e4621e330
2021-07-28 01:22:23 +00:00
drh
415ac68ae8 When an index is used by all branches of the WHERE_MULTI_OR optimization and
becomes a covering index, make sure the index has been created prior to
NULLing it in the OP_NullRow opcode of a LEFT JOIN.  See
[forum:/forumpost/0575376e07|forum post 0575376e07]. The covering-index for
WHERE_MULTI_OR optimization was added by [62678be3df35cdcb].  Test cases are
in the orindex01.test module of TH3.

FossilOrigin-Name: 787c76a865dc51dbca4f30432d64783199c2efb1df4619e4bbaf32182977321e
2021-06-22 23:24:58 +00:00
drh
9c32c914b8 Add an ALWAYS on a branch in STAT4-only logic that is now always
true due to check-in [c21bc5a2353e660f].

FossilOrigin-Name: 7bd55eee1ac63cf6d5699ce85bc5a29bf51afdf7a80bce44937fa833947a40f4
2021-06-16 19:23:24 +00:00
drh
3074faabed Add an optimization control bit to disable the OP_SeekScan optimization.
FossilOrigin-Name: f674c41e5cbbb08ffc3cc00ab449229b672054c872e209337f12e7eacb531fe5
2021-06-02 19:28:07 +00:00