1
0
mirror of https://github.com/sqlite/sqlite.git synced 2025-11-18 10:21:03 +03:00
Commit Graph

296 Commits

Author SHA1 Message Date
drh
a25bbaf74a Further improvements to codeEqualityTerm() for cases when an IN operator with
a right-hand side subquery is used as a constraint that needs to be processed
by the RIGHT JOIN non-matched logic.

FossilOrigin-Name: bb2798be3fb5737827eec505db2d4c8cb510d227d98fc26ce748bf2da7e8b382
2022-04-30 00:06:52 +00:00
drh
3d5665366f Enhance the codeEqualityTerm() routine inside the code generator so that it is
able to reuse an IN operator that has an invariant subquery on its right-hand
side while coding the non-matched loop of a RIGHT JOIN.
dbsqlfuzz 19f1102a70cf966ab249de56d944fc20dbebcfcf

FossilOrigin-Name: 56dd3065469e49320b13af039471b3f0b85e6f7368cfd97b1cdc0cf8fa8e1956
2022-04-29 19:00:11 +00:00
drh
b77c312956 For debug builds, if the RIGHT JOIN body subroutine contains a jump that
escapes the subroutine, then abort the prepared statement with a descriptive
error and SQLITE_INTERNAL.  This extra sanity check causes many tests to
fail.

FossilOrigin-Name: 2c5bb2bff26cc70d8cac78ddd12d5ac37ab1472f5f88afbd975950a18ac2804d
2022-04-23 18:04:31 +00:00
drh
503ad9c72d Make sure the code generator knows to invoke row-value subroutines which
running the right-join post-processing loop.

FossilOrigin-Name: fd328e52aee1dace12b1c2f44b6f7e9d15d8f77c8e9d9e3d85840a129a4b1808
2022-04-21 14:48:40 +00:00
drh
7348ca4e0b Ensure that left-hand side WITHOUT ROWID tables are set to OP_NullRow prior
to running the right-join loop.  This fixes a problem introduced by the
previous check-in.

FossilOrigin-Name: aab7665ce6c004df701a72aef1a5135f2c78f16c6ff728a00076afb66ba9d3a0
2022-04-18 16:20:59 +00:00
drh
2e1bcc9dd0 Fix problems with WHERE clauses that follow multiple FULL JOINs. This
introduces some new faults in the tests, probably due to something
unrelated.

FossilOrigin-Name: 95b242d4c2fed7c713299565ac1010f8a7534a5695589e87d5a0204c7bf5f3dc
2022-04-18 16:10:07 +00:00
drh
d875c7eefe Do not allow an empty table bypass to jump outside of a right-join body
subroutine.

FossilOrigin-Name: 1549dcd2353903b70abadd428eeef971ab940df04fb05a6b83b04ee30932db6d
2022-04-18 10:26:50 +00:00
drh
2c31c00bfa Rerun the subroutines that compute row-values if necessary from within
the RIGHT JOIN body subroutine.

FossilOrigin-Name: 9b9038bcd0ab5c4f01661456635526cef764f854ff24018a5e6e43825d07eb59
2022-04-14 16:34:07 +00:00
drh
f7ecd956ce Move the explain comment for the RIGHT-JOIN post-processing loop to the
verify beginning of the loop - to make the start of the loop clearer to
human bytecode readers.

FossilOrigin-Name: 7ed2a271e6fcbb5e69a7f3a88d3f45fe6318819c0cc6a0dcc06c3dae5aa1503f
2022-04-13 16:46:35 +00:00
drh
146e64d2e4 Add missing VdbeCoverage() macros on new branch byte-code opcodes.
FossilOrigin-Name: 218c7167e562f5c327124f02a92de85079315320a221fb0508310d927596b14c
2022-04-13 01:52:32 +00:00
drh
949e2ab49a Factor out the RIGHT JOIN non-matched row loop from sqlite3WhereEnd(). This
reduces the register pressure on that routine and helps it to run faster
in the common case where there is no RIGHT JOIN.

FossilOrigin-Name: beeecf1604d4fb11e45058f48cb2289c6542e0bc218d63a245198113d8d5476b
2022-04-12 18:04:29 +00:00
drh
c583719b65 Show LEFT and RIGHT JOIN processing in the EXPLAIN QUERY PLAN output.
FossilOrigin-Name: d91faeffea5cf0585fb71e5311fdcc6b8be85c7e9c732050b4448e617c970101
2022-04-11 14:26:37 +00:00
drh
189c065942 Merge the Expr.w.iJoin name change from trunk into the right-join branch.
FossilOrigin-Name: 29927926eb32acd963e2c496ad67d55177615ec4150fd218afaf2f9a730cabec
2022-04-11 11:48:17 +00:00
drh
d198526289 Rename the Expr.w.iRightJoinTable to just Expr.w.iJoin, so that the words
"RightJoin" in the former name do not lead readers to believe that this has
something to do with RIGHT JOINs in particular.

FossilOrigin-Name: e8c00442d2daedec079748d13147bf73b0ec3c3cf432bce2cdccb706bdff2853
2022-04-11 11:25:28 +00:00
drh
7c1734b09e The interior of the RIGHT JOIN loop is now a subroutine.
FossilOrigin-Name: 549f5a7ee639de80f049445002f58e93c805f9a3d3db1987ec9d139ccef4805e
2022-04-09 12:27:20 +00:00
drh
c2308ad2a0 A few bits and bobs of code generation toward getting RIGHT JOIN to work.
Much more remains to do.

FossilOrigin-Name: 55b4543122646997d928598343bc467c993f971e86e9037c85430cc948750576
2022-04-09 03:16:26 +00:00
drh
c187163090 Merge the sqlite3WhereMalloc() routine from trunk into the right-join branch.
FossilOrigin-Name: b1b3845aa38f469a27f07c8f6e8a98334f7967b19661ee4cf155d6997afd9d4c
2022-04-09 03:12:52 +00:00
drh
f8bdcfa3f1 The sqlite3WhereMalloc() routine allocates memory that is automatically
deleted when the corresponding WhereInfo object is destroyed.

FossilOrigin-Name: f237e1d8cc41b937f34288daebfacf5f7b0990a807a805e0cb6b45bc730192d6
2022-04-09 03:06:01 +00:00
drh
a76ac88af8 Preliminary code to support RIGHT JOIN. Everything seems to work, except that
the code to compute the unmatched rows for the RIGHT JOIN has not yet been
added, so the result of a RIGHT JOIN is currently the same as an INNER JOIN.

FossilOrigin-Name: 415abd6731b8e8a605adabfa6066c8a852a8531c300df41325d5f7e75cae5a70
2022-04-08 19:20:12 +00:00
drh
e603ab00ee Enhance the EXPLAIN output formatting in the CLI so that the subroutine that
implements the loop body for the multi-index OR optimization is indented.

FossilOrigin-Name: c2965fea9df7076b235d3eadaf84f0a36242476d0329030b0e57557b66e2540c
2022-04-07 19:06:31 +00:00
drh
6d6ea42a25 Fix the code generated for vector IN operator constraints on virtual tables
so that they work even if the "omit" field in the sqlite3_index_info object
is off.  This has apparently never worked correctly before.  Presumably, nobody
has ever before written a virtual table that can use vector IN operator
constraints and that relies on bytecode to double-check the constraints.
Test cases in TH3.  Problem discovered by
dbsqlfuzz cab8e26194a40147627094f3c6849c0a7b1e0310.

FossilOrigin-Name: 21b656572d066b640ff5774205a4f0db13e1b08a35d0fd484da9130e759b0c26
2022-03-07 14:51:29 +00:00
drh
07559b2716 Restore the ability to push-down OR subterms of the WHERE clause when
processing a multi-index OR.

FossilOrigin-Name: d71fb6fdc32d2fce73059b6ab86d7d7bbc812c637469755dd45b6794aadc3a80
2022-03-03 19:40:21 +00:00
drh
c9bcc5aab7 Fix for the problem identified in
[forum:/forumpost/0cd8e058bf|forum post 0cd8e058bf]:
When evaluating an multi-index OR, do not push down auxiliary WHERE clause
terms that involve subqueries into the OR-subqueries.  Otherwise, the
covering-index optimizer might convert table-references into index-references
for the particular OR index that is active for the branch in which the
subquery subroutine is coded, and those index-references
will not work if the subquery subroutine is invoked from a different OR branch
that uses a different index.

FossilOrigin-Name: 61a1c6dbd089979cbeb8b0c0c5ee1ab1abcb466be1d21a3a851be73c27e67a6c
2022-03-03 15:59:22 +00:00
drh
5694569591 Bloom filter pull-down optimization is incompatible with Skip-Scan.
Make sure the query planner does not try to to both.
[forum:/info/50a1bbe08ce4c29c|Forum post 50a1bbe08ce4c29c].

FossilOrigin-Name: ad3ffa1a75a5a032ebb64d8e014ee0a85c5e44b732e4b11bd67f31a59e729b94
2022-03-02 21:04:10 +00:00
drh
7fd6a7761f Improved comment on the codeDeferredSeek() routine. No code changes.
FossilOrigin-Name: 54f49f65ac943263a1622b1efe519c8a61f30f23694fd4fec89ad2bf0e17b473
2022-02-25 13:29:56 +00:00
drh
796588ae07 Record the position of many identifiers in the parse using the new
Expr.w.iOfst field.  This is done with the idea of providing offset
results for sqlite3_error_offset() for a more kinds of errors, though that
part is not yet implemented.

FossilOrigin-Name: 32a3a53b62ab09ac2212905e7f42fe6f1b09d891e015203e5caf4cb16262f18e
2022-02-05 21:49:47 +00:00
drh
02e3e04134 For the MULTI-INDEX-OR optimization, when pushing down WHERE clause terms from
the main query into the various OR-term subqueries, do not push down slices
of a vector comparison, since the right-hand operand of the comparison might
have only been initialized in a different OR branch that was not taken.
dbsqlfuzz 80a9fade844b4fb43564efc972bcb2c68270f5d1.

FossilOrigin-Name: 9f67ad00cd38b7c5ec6d14b379e1a611777bbdf6901d843a80712ba7d94d6d33
2022-02-04 13:05:29 +00:00
drh
0fe7e7d924 Add new interfaces to enable virtual table to process IN operator constraints
all at once, rather than one element at a time.

FossilOrigin-Name: eb84b80e1f6d8c32bf0c9e1731f0233de0160a13f714f766779ae01fdf504e7b
2022-02-01 14:58:29 +00:00
drh
8f2c0b59da Query planner always honors the aConstraintUsage[].omit flag for the
SQLITE_INDEX_CONSTRAINT_OFFSET constraint.

FossilOrigin-Name: 38c5151eb8cc789ad082a1b40d16b2ddb55454219bd685b6e420995af6a15f55
2022-01-27 21:18:14 +00:00
drh
895bab3383 Give the virtual table xBestIndex method access to (some) OFFSET and LIMIT
clauses.

FossilOrigin-Name: 74fa5757ee0a8499bcd6546dac1a3ecc6048ba9cff9e3e574e28e6e82e894a3d
2022-01-27 16:14:50 +00:00
drh
0c7d3d399d Remove many redundant checks for sqlite3.mallocFailed now that any OOM should
cause Parse.nErr to be non-zero.

FossilOrigin-Name: 1f7fa46126ea33ed30e93186aff3df51068aeb4be6f79a102bfe8c4e44941d71
2022-01-24 16:47:12 +00:00
drh
e482fde6ee When computing a vector to be used as a key for an index lookup, do not
check for NULL values and abort until after all key values have been
computed, in case one of the later key values involves some initialization
that is needed by a LEFT JOIN.  Fix for the problem identified by
[forum:/forumpost/ab95010d410a0a55|Forum post ab95010d410a0a55].

FossilOrigin-Name: 4db5217a28ce767fa14ddfe51cf3ca25eceb72079d46a2fc00f7d6b8ae9abe0b
2022-01-21 16:41:11 +00:00
drh
dc56dc93a5 Do not allow early evaluation of Bloom filters that use the IN operator as
the machinery to deal with the IN operator is not available.
dbsqlfuzz 5b51c247518278f79a45cea978702e86e86cd4f9.

FossilOrigin-Name: 799db7cb2e0d73031182d26a0e5919368f9f9823df81cb2863bfe79eca344f5c
2021-12-11 17:10:58 +00:00
drh
27a9e1f615 Rename the internal routine constructBloomFilter() to
sqlite3ConstructBloomFilter(). OSSFuzz is reporting a crash with a garbled
stack that we cannot reproduce. Perhaps the original "constructBloomFilter()"
name is colliding with some internal name used by OSSFuzz. We'll see if this
rename clears the problem.

FossilOrigin-Name: 403e7312dd9a3fe493a21aceb82e387d6f152622d66c1b403c881597713e8cc3
2021-12-10 17:36:16 +00:00
drh
5a4ac1cc30 Remove unused code and fix comments. Final cleanup before merging.
FossilOrigin-Name: ce42039f5647b1f276acf5d9911528ecb47df1544a587def72c8cd6b2f664289
2021-12-09 19:42:52 +00:00
drh
a11c5e22b5 Enable bloom filters for the right table of a LEFT JOIN. Fix unreachable
branches.

FossilOrigin-Name: d342ab722de1fc3f34219c9755253db9d88eb50a6fa5cc257207008b00e7fc03
2021-12-09 18:44:03 +00:00
drh
7e910f6422 Add SQLITE_TESTCTRL_LOGEST and enhance the LogEst utility program.
Improvements to testability of bloom filters.

FossilOrigin-Name: 88b43d798cc5aa59855e92d3e658aee9f0a5def6ffbc5db77af048d75ecdf8cc
2021-12-09 01:28:15 +00:00
drh
c491a25631 Merge the WhereClause fix from trunk
FossilOrigin-Name: d3250256772e3348abe887c0ca3550a6647cce3804c9456a9d0112aea7ee1c46
2021-12-08 16:15:41 +00:00
drh
132f96fc75 In the WhereClause object, do not assume that all TERM_VIRTUAL terms appear
at the end of the list, because that is no longer true.  Instead, keep a
separate nBase count that is the size of the list excluding the tail of
virtual terms.  Use nBase instead of nTerm when scanning terms that are not
virtual.  Add assert()s to validate correctness of WhereClause.

FossilOrigin-Name: 6024682ca467fa4fe49608772b0bbfa2f8a419b32cebfa715941073c8b29da49
2021-12-08 16:07:22 +00:00
drh
3bd7cd736d Improved EXPLAIN QUERY PLAN output for Bloom filters.
FossilOrigin-Name: 00070e1fff6aec3d7c7b121f2b02bbca38a1664aca9afc3fb7e293f07fd1704f
2021-12-06 23:07:59 +00:00
drh
6ae49e67cc Run as many Bloom filters as possible before index lookups.
FossilOrigin-Name: 06f6fefd67086896bc49272c6319545ff6c6792f18babe23aced27b60b032119
2021-12-05 20:19:47 +00:00
drh
35685d3e5e Try to run all Bloom filters before any Seeks. This gives a small performance
gain on the Star-Schema Benchmark.

FossilOrigin-Name: 5be2470f8755ef454f813c880e659bdbf82f2396be9320cf3079cd4ca8e81a19
2021-12-05 00:45:55 +00:00
drh
067c60cfc9 Add VdbeCoverage() macros. Adjust the Bloom-filter hash function so that it
correctly deals with zero-blobs.

FossilOrigin-Name: 629ee2e3e3125bfd2af435c6713d49e46691213ad15db0a5e93a63a77f1130c2
2021-12-04 18:45:08 +00:00
drh
770dade262 Apply the Bloom filter only on those terms of an index that have equality
constraints.

FossilOrigin-Name: a70429596a3c6a413b03118b0d800521b3526d99dcf88a48acc3189b51518d82
2021-12-04 14:24:30 +00:00
drh
19ce9aafdb Add the "WITH BLOOM FILTER" clause to the EXPLAIN QUERY PLAN output for
cases were a Bloom filter is used.

FossilOrigin-Name: 8e078c0e404fe4b3661dd4a11a992a8914c99e2a144cecc417421cbd68fa08cc
2021-12-04 13:52:08 +00:00
drh
2db144c33b Add a Bloom filter to the automatic-index mechanism.
FossilOrigin-Name: 50ac4de1d7cbb586ea7969e1ae80ea8b021e194edc2fa7db19374b4ee9369bee
2021-12-01 16:31:02 +00:00
drh
5d742e39d9 Proposed fix for a problem associated with OP_SeekScan that results in an
incorrect answer.  See ticket [5981a8c041a3c2f3].

FossilOrigin-Name: 266e9cbada531716c908c21d60a6038109722e3dd040237679ac0240fb5020ea
2021-11-02 20:52:20 +00:00
drh
220f0d6f8e Protect the WhereTerm.u union using nearby assert()s and/or branches.
FossilOrigin-Name: 8a56de5b9c6f4522000f8d991373490b67b9e9d97f03c1ca2cf32816d84789ef
2021-10-15 17:06:16 +00:00
drh
0475629d47 Protect access to the WhereLoop.u union using nearby assert()s and
branches.

FossilOrigin-Name: bdd840216cc4c5293c112f182a189f7562b3cc0f6270e3c4af5eb2e8bd61ccc9
2021-10-14 19:28:28 +00:00
drh
477572b9f1 Protect access to the Expr.y union using nearby assert()s and branches.
FossilOrigin-Name: 87e2f5eb436fc448427b0e24fb70f29688796b513b8c7b12f1a21540dae1e56d
2021-10-07 20:46:29 +00:00