1
0
mirror of https://github.com/sqlite/sqlite.git synced 2025-11-11 01:42:22 +03:00
Commit Graph

1569 Commits

Author SHA1 Message Date
drh
7960da0346 Improved comments. Add assert()s to verify that the AggInfo structure
is unchanged after registers have been assigned.

FossilOrigin-Name: 5200b84195ee1ccaa387f7032eae3d463724c48cb53ba0251bbc79e927dd9752
2022-11-25 13:08:20 +00:00
drh
8683c09281 Correctly deal with IF-NULL-ROW operators when dealing with
indexed expressions in aggregates.

FossilOrigin-Name: 939cb47025354e2df047de7654c0b06f791957cfe4e904abe8892207cea90215
2022-11-24 23:35:27 +00:00
drh
c25f5ea6e8 Add NEVER() and ALWAYS() macros on branches that are believed to be
unreachable.

FossilOrigin-Name: 3a901e88c87fc76c7fe42e47a976a5706830f0dbd6027605663e4d55f4f33590
2022-11-24 15:04:23 +00:00
drh
9b80cb5f39 Add explanatory comment to the new optimization. And add a test case.
FossilOrigin-Name: e6c20f61de7d048eee65c8e74a3eb36760ab9747ebd1ab50e49642b777c10306
2022-11-24 01:40:20 +00:00
drh
2e30d95fb6 Aggregates with GROUP BY now make use of expressions on indexes. This code
works and gets the correct answer for the test case in the ticket.  Lots more
testing and documentation is needed, however.

FossilOrigin-Name: 8dcf9f2031c16f296d187fe876d4204c71fc96fec120984ff11b6d8b03d58a5f
2022-11-23 18:51:04 +00:00
drh
590f013a87 This attempt at modifying AggInfo to make use of indexed expressions does not
work.  It gets an incorrect answer for the test case shown in the ticket.

FossilOrigin-Name: 84c06023f4a1606664fdb9811312603b31f7c94a43d0e443ba7dde7fdba029e3
2022-11-23 17:56:00 +00:00
drh
c6138e970e Further foundation prep work prior to starting to flesh-out the
optimizeAggregateUseOfIndexedExpr() routine.

FossilOrigin-Name: 23145fe999ff74d787a3999baedd4ffe755c5f1f1275082ed0338ba637ecc56e
2022-11-23 14:13:39 +00:00
drh
58c0652923 Merge enhancements and fixes from trunk.
FossilOrigin-Name: f8932e04d4d18eb9d71edda15aa08af2eb139ff14d77ca147ea6e9b173e0f5e0
2022-11-22 20:58:18 +00:00
drh
b23f1572ab Add the stub function: optimizeAggregateUsingIndexedExpr(). The hope is that
we can fill this in with a routine that does useful optimizations.

FossilOrigin-Name: d85bb724fdd6fbad2b88ed7f60e4174e3f65182356f404d04620c5cf6b17f77e
2022-11-22 20:37:41 +00:00
drh
cbef156d93 Merge trunk fixes and enhancements into the agg-with-indexed-expr branch.
FossilOrigin-Name: 070634781a5eb41f96b001e48b064e3cd8c82314f576335eb1fcd43792179291
2022-11-22 19:51:16 +00:00
drh
5d7aef16bf Rename the SELECTTRACE macro to TREETRACE, so that is corresponds to the new
CLI command.  Renumber all of the bits in the bitmask used to enable
various kinds of tracing, and add a trace bitmap decoder in sqliteInt.h.
Changes to debugging logic only.  No (intentional) changes to production code.

FossilOrigin-Name: 8036445a36d9d982c1305935e7e37367bdf9e466b923eb6286b52524802e3ccd
2022-11-22 19:49:16 +00:00
drh
3c8e438583 Since the memory registers used by the columns and functions of an
AggInfo object are sequential, it is not neecessary to remember each register
separately.  We can simply remember the first one and do the math when
others are needed.

FossilOrigin-Name: dc5bd34963b761c819c565653156d0befbf65cc2cc5dc4113b0ce952450f0352
2022-11-22 15:43:16 +00:00
drh
89e5dfac01 Omit the unnecessary AggInfo.mnReg field.
FossilOrigin-Name: d79c58ef08b917bacc0f24d210d8eb23f659f955c219b4757af42eee8f17099b
2022-11-22 14:31:13 +00:00
drh
42b78237cd Factor out the allocation of registers for aggregates into a separate
subroutine.

FossilOrigin-Name: 4475799d5b41e93eabc32fac502ac2de240642b3b64739216e32e9af92ee191d
2022-11-22 14:10:22 +00:00
drh
ff5e4ecced This is the first in what is anticipated to be a long sequence of incremental
changes aimed at improving aggregate query processing, and in particular
helping aggregate queries take better advantage of indexes on expression.
The end goal is to resolve ticket [99378177930f87bd], though it remains to
be seen whether or not I can get there with this approach.

FossilOrigin-Name: cba837eae93f6b842d4e78ef00661a4f09deb99c53f12b3e8f46763749602597
2022-11-22 13:33:58 +00:00
drh
da217c958e Split out the debugging code that dumps an AggInfo object into a separate
subroutine called printAggInfo() so that it can be invoked interactively
during debugging.  No changes to production code.

FossilOrigin-Name: dc33cfbedef4b444adeadea17f8183b7c4ce5b87432d0c712f986b34c7374ff1
2022-11-21 17:40:23 +00:00
drh
54422235e9 Fix an obsolete comment.
FossilOrigin-Name: e377c0a1ef030395293c5f24d7cb8e5b36ce972e9fac31b99c8425075486a46a
2022-11-07 15:23:51 +00:00
drh
b6d9167945 Do not flatten a UNION ALL subquery where the column affinities do not match
on all arms.  Force that subquery to be materialized.

FossilOrigin-Name: 1b8a128832c1633cbb86999d347fef64ac45fe134dac61132112fc97f24676a0
2022-11-01 01:07:29 +00:00
drh
5fdb9a352c Enforce column affinity on the materialization of a subquery or view.
FossilOrigin-Name: b6692de374d7f489d8cf8877bdfef19501bc5ecb0781d42750807eeb58f2e7cd
2022-11-01 00:52:22 +00:00
dan
195687f1bf Disable the push-down optimization for sub-queries that are INTERSECT, UNION or EXCEPT compounds. dbsqlfuzz a34f455c91ad75a0cf8cd9476841903f42930a7a.
FossilOrigin-Name: 346a3b12b861ce7ba369e98cd336f79a1d4f7a7bb9acd7a4f63f37b391755bf5
2022-10-26 21:14:21 +00:00
drh
fb643592fe More frequently send the complete SELECT statement into sqlite3WhereBegin()
so that it can more accurately determine index coverage.

FossilOrigin-Name: f6d09c5a0df940dbaa716df70df28218985b83540b34c468989729a3421ef38c
2022-10-22 23:09:29 +00:00
drh
3b88065d35 Fix references to "SrcList_item" in comments as that object has since
[bfd5bf2c73110fcb] (2021-02-21) been called "SrcItem".  Comment changes
only.  No changes to code.

FossilOrigin-Name: e3648a07f5607dbd4d6ad5b6e5a62a4a1d0b173d3f22aa36c74b768281fc86d8
2022-10-19 11:58:24 +00:00
dan
879164ed74 Fix a problem with using the push-down optimization on compound SELECTs where component SELECTs use different collation sequences. dbsqlfuzz 11516f050100243e5a845f5a2b48a90ed2efaf2e.
FossilOrigin-Name: ed14863dd72e35fa3a23320c3d5a8166515faea39a555c28a27b2d35e701eac4
2022-10-14 19:30:34 +00:00
drh
5a0771a1f4 Remove code that has been previously commented out.
FossilOrigin-Name: 852b385a5de622aa32026824210d4bd23db52a4a8a697b83d22d6000edeba487
2022-09-20 18:18:00 +00:00
drh
40c9becc81 Is the query flattener restriction 29 (do not allow both EP_InnerON and
EP_OuterON constraints on the same FROM clause term) really necessary?   This
branch explores what happens if we omit that restriction.  This is an effort
to address the performance regression reported by
[forum:/forumpost/96b9e5709cf47cda|forum post 96b9e5709cf47cda] that is caused
by flattener restriction 29.

FossilOrigin-Name: 19270577ad1d08c9e6e5f0600396b8a1ccda9df1e67a0df845aee6b60437f379
2022-09-20 16:57:49 +00:00
drh
b22b493338 Clarify comments that describe query flattener restrictions associated with
outer joins.  No code changes.

FossilOrigin-Name: 2647d73bb1876d01851f52cb3fd6af08a962ab336b6d51c160d27b1baf94f75f
2022-09-19 19:25:15 +00:00
drh
316cd40122 Fix a minor comment typo. No changes to code or documentation.
FossilOrigin-Name: 8032db4ade66f70a102abb9d3a49e20620e204c45ad7a954a49d20403985bf07
2022-09-19 17:41:59 +00:00
drh
44132244fa Defer deleting a transient SELECT statement associated with a flattening
of one arm of a compound SELECT until after the parse has completed.

FossilOrigin-Name: 1c4157c71cd1e062a9c2c79787d17e34e340f28ce1e40573851dfe174f5da7d7
2022-09-01 13:51:09 +00:00
drh
41ce47c4f4 Add the new internal interface sqlite3DbNNFreeNN(db,ptr) where both the
db and ptr parameters are guaranteed to be non-NULL.  Use this where
appropriate to save more than 2 million CPU cycles on the standard
performance test.

FossilOrigin-Name: e5eaa80e81fdf86f2875a912b880272b8d099b82b08e945a7988c5dd0fe9d6b5
2022-08-22 02:00:26 +00:00
drh
bffd5c1ece Fix a problem with the query optimizer for LIMIT/OFFSET queries when
underlying query is a UNION ALL and both arms of the UNION ALL are
subqueries with an ORDER BY clause.  This bug was reported at
[forum:/forumpost/6b5e9188f0657616|forum post 6b5e9188f0657616].  The
problem was introduced in 2015 (SQLite version 3.9.0) by check-in
[4b631364354068af].  See also ticket [b65cb2c8d91f6685].

FossilOrigin-Name: 6c806f64bbc9e98891bad0868575ee2ec5d0951ceb0c71d3ed417b45d5f27561
2022-08-04 17:15:00 +00:00
drh
4b1b65ca2e Make sure IF_NULL_ROW expressions receive a separate slot in the sorter used
to implement GROUP BY.

FossilOrigin-Name: 2bda4fca06ab6be5ad02377a7d1fd9fb9586e3181f1052e4b4937958bdd45efe
2022-07-26 15:32:02 +00:00
drh
f4c291275b Improved AggInfo.aCol debugging output, intended to debug a problem with
LEFT JOIN flattening into an aggregate query with GROUP BY.

FossilOrigin-Name: e717e029bde4ee68b6ea77a68721c02ddb6e296f1d310a368137ea3c4164f68c
2022-07-26 14:37:04 +00:00
drh
3c8fb6fb84 Performance optimization: avoid unnecessary calls to computeLimitRegisters().
FossilOrigin-Name: f48bd8f85d86fd93329dda7ba57f468854a732b89e4126e9c5beec3495d24b31
2022-07-26 01:20:18 +00:00
drh
ee37302095 Allow subqueries on the right-hand side of a LEFT JOIN to be flattened even
if they contain a GROUP BY clause.

FossilOrigin-Name: 816da9a893ae97a21463562479edb419a8b511ae731d86eccee3fa6e3e7dc96e
2022-07-25 15:54:23 +00:00
drh
e834484d89 When applying the omit-ORDER-BY optimization, defer deleting the AST of
the deleted ORDER BY clause until after code generation ends.

FossilOrigin-Name: f22f95b838873f1d2a320afe3d0f4e4847948fcd343097b93a9f684a6f66d6ba
2022-07-14 01:48:27 +00:00
drh
b88bf865c3 The query flattener should not run if the subquery is a compound that contains
a RIGHT JOIN in any arm and the subquery is not the first element of the
outer query.  Otherwise, prior elements of the outer query will not have
the JT_LTORJ flag set.  Fix for the problem reported in
[forum:/forumpost/174afeae5734d42d|forum post 174afeae5734d42d].

FossilOrigin-Name: 274e244c85935084b2f0f85176283f018bf9b74e7703f985bd5a2f6f8bdcff5d
2022-07-13 15:52:15 +00:00
drh
0e400f4e7d Remove an obsolete statement from a comment. No changes to code.
FossilOrigin-Name: 99afb3f41b923f06211d655d46c902da13c8f7aadf2421d6e93c048f5b49e660
2022-07-08 11:55:54 +00:00
mistachkin
07fae32db0 Fix harmless compiler warnings seen with MSVC.
FossilOrigin-Name: 61e2094afbbcbd5fdf5c3ec06b96134fafb7b854dc9bfa7d0619bed6d35efbe4
2022-07-06 23:50:01 +00:00
dan
2c1b1ddc55 Do not attempt the OP_Count optimization on queries with HAVING clauses.
FossilOrigin-Name: 566b7842ee286ed8620a87b06de65bc173ad0d208df4157292238fcb059cb484
2022-07-01 21:03:19 +00:00
drh
fe9833482f Allow flattening of a subquery that is the right operand of a LEFT JOIN
in an aggregate query as long as there is no GROUP BY clause.  (The GROUP BY
clause will interfere with the operation of the TK_IF_NULL_ROW expression
nodes.)

FossilOrigin-Name: 2cf373b10c9bc4cbc5fe63d0a6948011df7bbc2f40dc025c9349f875da782b88
2022-06-25 19:43:44 +00:00
drh
3a6e0c3f04 Reintroduce flattener constraint (3c), but this time make it apply only if
the outer query holds a GROUP BY, not if the outer query is an aggregate.

FossilOrigin-Name: 641dfb9182a6cbadb3c452f5420f896791b7844b794f693443bcd38dca14da35
2022-06-23 20:56:45 +00:00
drh
d198183465 Add back the ability to flatten a LEFT JOIN subquery - previously removed
due to ticket [cad1ab4cb7b0fc344].

FossilOrigin-Name: f8fe936ad4f7678f9b26ba6fab41c5df9f4938634cdb3286b0f2dcd1357f919a
2022-06-23 15:15:03 +00:00
drh
9efc618607 Minor correction to a comment. No code changes.
FossilOrigin-Name: 5fa00959ebc3a46fe400564d5a30e8c44857cd4f0473fb25e84154019eff3f39
2022-06-23 12:36:56 +00:00
drh
cf5cab01cf More comment fixes. No changes to code.
FossilOrigin-Name: 5c9133ef431a031249760267ca851239d0857ffba633cac27d3865cf179c3274
2022-06-22 15:55:28 +00:00
drh
d7480403bc Do not allow an ON clause to references tables to its right if there is a
RIGHT or LEFT join anywhere in the query.  Other RDBMSes prohibit this always,
but SQLite must allow ON clauses to reference tables to their right for legacy
compatibility, unless there is a RIGHT or LEFT join someplace in the query,
in which case there is no legacy to support.

FossilOrigin-Name: e615dbe02ca949252d1526ed5c48f8ce08159773ea2008ce666484379d0d9854
2022-06-20 17:04:44 +00:00
drh
ae8776e092 The fix at [cab9b4cccd13bf0a] was incomplete, as demonstrated by
[forum:/forumpost/57bdf2217d|forum post 57bdf2217d].  This check-in
should complete the fix.

FossilOrigin-Name: fb0a23b6789da8e934562ce9ebd9d58ea13a10fd10dee5cbfc7ac8f394e1aeec
2022-06-20 12:42:28 +00:00
drh
92d1afba9b Do not remove the EP_CanBeNull flag from expressions during a LEFT JOIN
strength reduction if the query also contains a RIGHT JOIN. Fix for
the problem identified by
[forum/forumpost/b40696f50145d21c|forum post b40696f50145d21c].

FossilOrigin-Name: b1be2259e2e08ec22a88bc9a18b3ab4d83246ad4c635c05cdf80d3eff84df06a
2022-06-13 12:42:24 +00:00
drh
958fcd417c Do not allow constant propagation between WHERE-clause terms and ON-clause terms
as this can confuse RIGHT JOIN.  Fix for the problem reported by
[forum:/forumpost/8e4c352937e82929|forum post 8e4c352937e82929].

FossilOrigin-Name: cab9b4cccd13bf0ab2bc38dc9a9c04ddd34e29c65ab6aef07b6bb3c31a43bece
2022-06-10 11:28:52 +00:00
drh
8a5469b6f5 Improved comments on the new query flattener restriction of the previous
check-in.  Also a NEVER() macro on an unreachable branch.

FossilOrigin-Name: 8c9e2d6315fde014bd6e25a7fa58ba054f5328c8a56e9d3d1c2d069c025a1b03
2022-06-08 17:48:14 +00:00
drh
5c118e3946 Add restriction (29) to the query flattener - do not allow flattening that
would leave both EP_InnerON and EP_OuterON constraints on the same join term.

FossilOrigin-Name: c585d6a4678b04f4cedc08852d01c44cdf52ae2c8ccd1174c3d5a395088bf528
2022-06-08 15:30:39 +00:00