1
0
mirror of https://github.com/sqlite/sqlite.git synced 2025-11-11 01:42:22 +03:00
Commit Graph

1569 Commits

Author SHA1 Message Date
drh
89b3cbed67 Fix to check-in [b9190d3da70c4171] - the agg-with-indexed-expr optimization
requested by ticket [99378177930f87bd] - that can cause an incorrect answer
if an aggregate subquery has a GROUP BY clause, and that GROUP BY contains a
term that is not in the result set, and the outer query makes use of expression
indexes.  Problem reported by
[forum:/forumpost/a68313d054|forum post a68313d054].

FossilOrigin-Name: e06973876993926fd56181281d04b8dd504c689abf883fa21a5721cc1d478ea8
2023-03-04 15:36:51 +00:00
drh
bf90114867 When flattening a view that is the right operand of a LEFT JOIN, using
the optimization of check-in [41c27bc0ff1d3135], always insert the
TK_IF_NULL_ROW expression nodes, even for TK_COLUMN expressions, as
the TK_COLUMN might be a column from an outer query and hence still need
to be NULLed out. This fixes the problem described by
[forum:/forumpost/26387ea7ef|forum post 26387ea7ef].

FossilOrigin-Name: 198b3e33dcfd74c7ba6abcf789ee81dfed464a50ebf15c8edeff349d36789fca
2023-03-01 20:23:46 +00:00
drh
46960e4677 Do not attempt to apply the count-of-view optimization to a CTE.
dbsqlfuzz ef8623915d843b150c159166ee4548c78cc6895a

FossilOrigin-Name: abc3a383636c0346053b5d09d96585f56c64cacb5751673ea3bf339e4955d1cd
2023-03-01 13:54:07 +00:00
drh
4edbcdd391 In the [/info/7c2d3406000dc8ac|omit-unused-subquery-columns optimization], be
sure to remove the EP_Skip and EP_Unlikely flags from the result set expressions
that get nulled-out.  dbsqlfuzz bf1d3ed6e0e0dd8766027797d43db40c776d2b15.

FossilOrigin-Name: 21aec65e5e2a01e58dd0bb8c8b9b29b8414373b53353fc7ca80a152fdd27566b
2023-02-26 11:36:35 +00:00
drh
648de49c03 Fix harmless compiler warnings and a code indentation error.
FossilOrigin-Name: de6c5c6bb49a0beca793cdc0462da2c17bb05f3292e08dd815be29b45b59a8fb
2023-02-23 14:43:15 +00:00
drh
af03eb3241 Enable the count-of-view optimization by default.
Enhancement request [eaed8e36ce888f1e].

FossilOrigin-Name: a4aacdd323a854d771c8cb1e2e4cfc4fb66b0020cfed23525733603605f5c63b
2023-02-22 21:47:02 +00:00
drh
95dee6d2e3 Avoid computing the values for unused result-set columns in subqueries.
Performance optimization request [baa5bb76c35a124c].

FossilOrigin-Name: 7c2d3406000dc8ac5a99cc205b036356b67e4b0b94738592ffc5680749696904
2023-02-22 21:11:34 +00:00
drh
bcb3215630 Remove the push-down optimization restriction that was added by
check-in [1ad41840c5e0fa70] because it is no longer needed after
the enhancements to compound query processing in check-in [27655c9353620aa5]
This resolves the performance regression reported by
[forum:/forumpost/bcc4375032|forum post bcc4375032].

FossilOrigin-Name: aa6bd6dff751223e302575c60f335707e4bb820bff716cab3706b564e18ed7ed
2023-02-22 20:42:15 +00:00
drh
61f0a03492 Yet another comment typo fix. No changes to working code.
FossilOrigin-Name: cdbdc8013fcfa8ce1bd4f5521c8d98afc53b9c577d43e5e6d0d1e15c789b910f
2023-02-21 12:42:25 +00:00
drh
ab31a5df93 Do not allow the COUNTOFVIEW optimization to run if the count() contains
a FILTER clause.  dbsqlfuzz 4f8e0de6e272bbbb3e1b41cb5aea31e0b47297e3

FossilOrigin-Name: 29fc06465efb948f98d2733bb25ffa1e0662a0189304006b3d0be9fec7dd28c5
2023-02-17 18:27:48 +00:00
drh
7defd20cb0 Provide an optimization-disable mask for this optimization. Do not do the
optimization if the subquery is an aggregate or is distinct, but allow it to
be an ephemeral subquery.  Do not omit columns that are used in the ORDER BY
of the subquery.

FossilOrigin-Name: 6b1a1f374d1372f11f5420d99645b218867100bf070bd3a8885bf5f00c189dff
2023-02-16 18:04:49 +00:00
drh
90b7af7715 Do not perform the omit-unused-subquery-columns optimizations on a
subquery that is DISTINCT, as that can lead to incorrect results.

FossilOrigin-Name: cc148503db8ef180bce984328da7e84959afadd6a9613c2d03bc1eafeb95dfad
2023-02-16 15:54:55 +00:00
drh
6945ba787f Fix the subquery result column NULL-ifier so that it correctly handles
subquery columns past the 63rd column.

FossilOrigin-Name: 77b220a7240425fa83e142ceef78505208e6e38d797070e146b9f7d255f753c7
2023-02-15 19:53:08 +00:00
drh
e3ec00ccb8 Do not compute result columns of subqueries that are never used. Make those
columns NULL instead.  This optimization potentially resolves the enhancement
request described by [/tktview/baa5bb76c35a124c|ticket baa5bb76c35a124c].

FossilOrigin-Name: 5dec3cc0225296a043d17f73126d477d90a604f82b3180628176d8f950adbce8
2023-02-15 17:53:17 +00:00
drh
54725efd80 Just because a CTE is used more than once, does not mean it should be
tagged with M10d_Yes and thereby prohibited from participating in the
query flattening optimization.  See
[forum:/forumpost/1d571c02963355ed|forum thread 1d571c02963].

FossilOrigin-Name: 66f29c403d28630bfaea9124bd63ee4a047b1fe4a7e27dc5d10d67d1601b15e0
2023-02-01 15:41:07 +00:00
drh
2fc9dc9330 Fix sqlite3_prepare() so that it only invokes the progress handler on every
N-th call to sqlite3ProgressCheck(), where N is the progress handler step
count.  Also fix faulty asserts exposed by the ability to interrupt in the
middle of sqlite3_prepare().

FossilOrigin-Name: 05461651599bb490ac6cfd893645dabab9cccedc6adcce15aee2487b2ea6027a
2023-01-12 19:51:49 +00:00
drh
f84cbd1676 Attempt to provide a mechanism to do early termination of long-running
statement preparation by invoking the progress handler at strategic points
during sqlite3_parpare().  This experiment shows that sqlite3_prepare() might
leave the resulting prepared statement uninitialized following an interrupt.

FossilOrigin-Name: 79636f2d80aee70832913a78933da2a7e30cc037810b93903ebbc1925ea93fef
2023-01-12 13:25:48 +00:00
drh
706631de32 When computing the datatypes for columns in a view, use the same datatype name
as the underlying table if such is available and is consistent with
the computed affinity of the column.
[forum:/forumpost/7fb1fe9dcf310ef5|Forum thread 7fb1fe9dcf310ef5].

FossilOrigin-Name: 497a98363fd1ed079544620ec5d0883f987ed03013131542741c93d7568e8568
2023-01-10 19:57:26 +00:00
drh
98164c3452 Do not use indexed expressions to replace the expression being indexed on a
RIGHT JOIN as that leads to problems, as shown by
dbsqlfuzz a644e70d7683a7ca59c71861a153c1dccf8850b9.

FossilOrigin-Name: f113eebdbe68246fd38d640b0cdf0fd73ab5d2abad9226274dfa2e41fe286bf0
2022-12-20 01:48:43 +00:00
drh
a4569bd5d0 Fix a minor problem with the printAggInfo() routine (used on during
debugging).

FossilOrigin-Name: c8329967f32efedfc8244631a5682f7f7d21c1b3fcd7d5cc67716555716d4926
2022-12-19 12:25:41 +00:00
drh
1c70429569 Add NEVER/ALWAYS to branches that have become unreachable due to recent
enhancements.

FossilOrigin-Name: b4281db4e4762773684d27b809ef911da08b81cb953b3666e535b990d4d5f701
2022-12-17 12:49:04 +00:00
drh
00d6b2755f Create a new affinity called FLEXNUM that works like NUMERIC except that it
never tries to convert integer to real or real to integer.  The affinity is
only used internally - it is not possible to create a table column with this
affinity.  This affinity is used on subqueries and views that are built off
of a compound SELECT and where the datatype is controlled by a CAST expression.
dbsqlfuzz c9ee6f9a0a8b8fefb02cf69de2a8b67ca39525c8

FossilOrigin-Name: 44135d6ea84f7ba6b36549954b38a8bc048d5ffea5a9779e35950afa4eb2dfb2
2022-12-15 20:03:08 +00:00
drh
f000c9b638 Omit the columnTypeImpl() function from the build when SQLITE_OMIT_DECLTYPE
is defined, because it is no longer used in that case.

FossilOrigin-Name: 751e344f4cd2045caf97920cc9f4571caf0de1ba83b94ded902a03b36c10a389
2022-12-15 15:37:52 +00:00
drh
569aef04b0 Fix harmless compiler warnings.
FossilOrigin-Name: ddb260cc52efce6472b9423cbc6b788d8c38746f1842c9f1c184c08c651b3387
2022-12-15 13:45:49 +00:00
drh
da69802e92 Fix minor problems in the new sqlite3ExprDataType() function.
FossilOrigin-Name: c1d5261b222bbf94c20e558089f3d2eae6a88b6d739225ee4f7d0338e0e59994
2022-12-14 23:53:29 +00:00
drh
604314d94e Merge all the latest trunk enhancements into the coroutines-exp2 enhancement
branch.

FossilOrigin-Name: ea278438b72e85d217e72c836cbefd68bd8b336baf33507b2d8d12ef436424cd
2022-12-14 17:52:15 +00:00
drh
ed07d0ea76 If the SELECT that implements a subquery or a VIEW or a CREATE TABLE AS is
a compound with different result datatypes on two or more arms of the compound,
then the overall column type becomes BLOB (ANY).

FossilOrigin-Name: 6ebb178c6b688ebd632e91a5ec4c748567833037c679ab3d72fb770deb230fe1
2022-12-14 14:41:35 +00:00
drh
a8b5c8739c Back out the part of the change in [88a05141c28e5ff1] that adds affinity to
the materialization of a view, as the affinity can be undefined for a compound
query.  This passes all TCL tests, but shows failures in the TH3 tests derived
from [forum:/forumpost/6f842bc5b2dadcb2|forum post 6f842bc5b2dadcb2], presumably
because the WHERE clause of the query uses constraints of the form
"source_crs_code='8675'" instead of "source_crs_code=8675".  Perhaps further
changes on this branch should reimplement affinity on joins in cases where
the affinity is unambiguous.

FossilOrigin-Name: fe5a77bcc4de8f49cc4fe6bd2e2e1f31da8d3bc84120daaa99eb853b06291d15
2022-12-14 09:06:45 +00:00
drh
c799217cc8 Remove an unused variable from sqlite3SubqueryColumnTypes().
FossilOrigin-Name: c6e3ff396947e6954e77195419b35de8b39bd540d3f4bff456bab25b512e1792
2022-12-13 16:59:09 +00:00
drh
177b8c9c28 Use standard datatype names for the column datatypes in views and subqueries.
FossilOrigin-Name: baf2c2413346feb72ecb0bac800be9b945ebb4e01b42212132434d8ddef9cad0
2022-12-13 16:55:23 +00:00
drh
9e66087a49 Refactor the sqlite3SelectAddColumnTypeAndCollation() routine. Improved
comments.  Now called sqlite3SubqueryColumnTypes().

FossilOrigin-Name: 4dfb1b450b87e2c6207c83c102b785781d1dbdeadbeeab6a83f96d58bb96cfbf
2022-12-13 15:54:43 +00:00
drh
393021938d Merge recent trunk changes into the coroutine-exp2 branch.
FossilOrigin-Name: c43f433bcab29db0f1f8afd3948f5a4149e1f277c853c66f99c31f226d82bc0f
2022-12-13 00:51:34 +00:00
drh
89e160a96a If a subquery has a result column of the form "CAST(... AS NUMERIC)" then
give that column no affinity rather than NUMERIC affinity.  This is because
casting to numeric preserves real values that could be integers but numeric
affinity does not.  By using no affinity on the column, we make the behavior
consistent if the subquery is implemented as a co-routine or is materialized.

FossilOrigin-Name: ece07d091c2ef3367a914187e0b6512c1f2390b8c34844536ad50e88c7e8c2f2
2022-12-12 18:58:53 +00:00
dan
6fce561905 Ensure that the types and collation sequences of a view are taken from the leftmost component of compound SELECTs, even when column names are explicitly provided for the view. Possible fix for [679ed6a2].
FossilOrigin-Name: b0e49aa81a09c183b8baf8346b8a9b593c2b13073958617b6594500958d5bd46
2022-12-10 17:54:55 +00:00
drh
93f41e22d4 Remove the SQLITE_PREPARE_SAFEOPT flag. The name is obsolete and it is at the
wrong level.  Instead use the SF_UpdateFrom flags on the Select object.

FossilOrigin-Name: 78723a9a7e72b42d28fc5645661da17f20cedcf864819b861800ad9340007be1
2022-12-09 18:26:15 +00:00
drh
a60d61bf14 Improved comments on the new co-routine logic. Fix a C++-ism in the code.
Update test cases to accommodate the more aggressive use of co-routines.

FossilOrigin-Name: 9243e850ae656d16adc8f0e5c4dcf3dcf476312cee192c39c38685fc437ccbbd
2022-12-08 16:47:16 +00:00
drh
05cb9d881e Do not use a co-routine on a subquery that is participating in a self-join.
FossilOrigin-Name: ea4306a03cd8dd706100dac3f11653068a25c50c5d06c34ad08534aec0a580bc
2022-12-08 13:56:06 +00:00
drh
ad9ff1d5f2 Simplified experimental changes to promote the use of co-routines. Less
cruft than the coroutines-exp1 branch, but still does not work.  Checked in
as a work-in-progress.

FossilOrigin-Name: e2318a30bf6ad2aeb4c4cb29661bc24ff78eb51bf07decc4b8da1ecac0015ef0
2022-12-06 15:24:05 +00:00
dan
109f173b6f Support an SQLITE_SCANSTAT_NCYCLE statistic for "CO-ROUTINE" elements.
FossilOrigin-Name: d3f6a207fd1b3f53688fa1ff35eb71e1c5bcc4115c0c35bb187d22ccf7100b71
2022-12-06 11:41:05 +00:00
dan
a3d0c158a0 Add loops and rows counters to "USE TEMP B-TREE FOR ORDER BY" records. Also fix the sqliteHwtime() function so that it returns a 64-bit value.
FossilOrigin-Name: 41a0e05e8c0fca3b803fe4bd017a157c172b2ca518356a2a4d4ed4f12d01a1e3
2022-12-05 18:19:56 +00:00
dan
f6f01f15dd Add CYCLES scanstat measurement to "USE TEMP B-TREE FOR ORDER BY" lines.
FossilOrigin-Name: 365011ae8b7e3fcaa2c4ea9601231a3ef2223e60d7a53ec33013109dca22ad58
2022-12-03 18:16:25 +00:00
dan
231ff4b027 Enhance the sqlite3_stmt_scanstatus() API and add sqlite3_stmt_scanstatus_v2(). For creation of easier to read query performance reports.
FossilOrigin-Name: 55800833645739efeddcacef464c623931cb6aeb43f4219b4e4faf473c25c8bb
2022-12-02 20:32:22 +00:00
drh
5b046dadfc Fix harmless compiler warnings.
FossilOrigin-Name: 8d5b76593d82b3a57bc904ced33c24cd49369868b5dd2fe7f2c7b114f5aee2f6
2022-11-28 20:08:15 +00:00
drh
743c9ec4a5 Fix an incorrect assertion-fault in the "TREETRACE" debugging logic on
select.c.  The problem does not affect production builds.

FossilOrigin-Name: f11bb42292d94e7b7687bd2134f72afe1353182238cb4837fc0a6d78a021dd7e
2022-11-26 20:12:02 +00:00
drh
902ea83925 Relax restriction (8) on the push-down optimization so that it only applies
if one or more columns uses a collating sequence other than BINARY.
See [forum:/forumpost/3824ced748baa808|forum post 3824ced748baa808] and
check-in [346a3b12b861ce7b].

FossilOrigin-Name: adbca3448e2099f0d6149a073978f230ed9a92a2f384779879ef89e672231bcf
2022-11-25 17:05:55 +00:00
drh
49d400b091 Fix #ifdefs so that restrictions (8) and (9) of the push-down optimization
are still enforced even if compiled with SQLITE_OMIT_WINDOWFUNC.  This
fixes a bug introduced by check-in [346a3b12b861ce7b].

FossilOrigin-Name: 09e1e42e0ff26f9a71cbd128169f060a66425828d637bf8f781490ca38d99103
2022-11-25 16:32:59 +00:00
drh
ae5cfda656 Immprove the query planner such that it is able to make use of indexed
expressions within an aggregate query with GROUP BY.  This implements
enhancement request [99378177930f87bd].

FossilOrigin-Name: b9190d3da70c41717eb188474fd225ee43d0b46646e1b03de5967bd332553870
2022-11-25 16:10:48 +00:00
drh
23f61a4ba8 Add restriction (9) to the push-down optimization: If the subquery is
a compound then all arms of the compound must have the same affinity.
dbsqlfuzz 3a548de406a50e896c1bf7142692d35d339d697f.

FossilOrigin-Name: 1ad41840c5e0fa702ba2c0df77a3ea126bd695b910b5d1271fa3129c38c58f5f
2022-11-25 15:52:00 +00:00
drh
bedbb1bfab Remove a redundant assert() statement.
FossilOrigin-Name: e3e1b453dc47884ddc9e51c4302fa2f4d40bca3d69ac7c13d8e2ae1adc81ac56
2022-11-25 13:26:51 +00:00
drh
f5d0656f16 Improve the robustness of the updateAccumulator() routine against OOM.
FossilOrigin-Name: d2844f577b69fdc16a0a2568c0958fc3d8aff33e9a0ef80e0f58e92f01097432
2022-11-25 13:15:48 +00:00