1
0
mirror of https://github.com/sqlite/sqlite.git synced 2025-08-08 14:02:16 +03:00

289 Commits

Author SHA1 Message Date
drh
dce57658bf Merge the latest trunk enhancements into the bedrock branch through the
wal2 intermediary.

FossilOrigin-Name: e7d6e993df772ca390186188c4b844a08402111cbc06bae9c731ebd3283da1c1
2025-05-19 16:58:50 +00:00
drh
691b44a1b3 Back out the "low-quality index" query planner hack of check-in
[bcac937526d9a6ef].  Subsequent query planner enhancements for dealing
with star-queries make that change unnecessary and the change was recently
found to cause a performance regression in an unrelated query.
Also fix a typo in a debugging message.

FossilOrigin-Name: e7dcf25efae364b7cdf9eb8265803c816c8b8557e4a7684da428badc6ffb3875
2025-05-14 16:40:05 +00:00
drh
1b493be98d Merge the latest trunk enhancements into the bedrock branch through the wal2
intermediary.

FossilOrigin-Name: 203376300781167f3e96da0197a9c8e8da6fed15c56c8980a4c283ef34478a07
2025-03-03 16:01:11 +00:00
drh
c071c47b51 Tamp down various harmless compiler warnings. Use "int" in places instead
of "u16" or "i16" since the compiler complains less and generates faster
code.

FossilOrigin-Name: 742827f049768c4f69ccdfaadfad339aaad3bc126d3a68b90cfea01d825bf7ce
2025-02-22 16:44:14 +00:00
drh
cc42fddfd9 Merge the latest trunk enhancements into the bedrock branch.
FossilOrigin-Name: c09656c62155a6e82d0fa6c27bf8ec40a14854845adfa53db3cb42a0b1b29101
2025-02-18 14:48:14 +00:00
drh
ef86b942b9 Code changes that make it easier to prove that no 32-bit integer overflows
happen during memory allocation.  No problems fixed; this change is just
to make future maintenance easier.

FossilOrigin-Name: 215650a5a1d55bdbca9c92524804a1a54456a17f42a17e53747b21a6507506f5
2025-02-17 17:33:14 +00:00
drh
7fd936e5ed Further reduction in the amount of memset() needed to initialize the Parse
object.

FossilOrigin-Name: 45e462c0060e51c3375a226d636148e3415ee6020e544ecc84861c7aef4ecf7b
2025-02-07 15:49:21 +00:00
dan
bbda3d68da Consolidate memory allocations made while loading stat4 data in a way that may be more efficient on systems under load.
FossilOrigin-Name: af65a902d10e50d827ff31f9ded7d05bc7ab0956a767366e1321b28fcb60b0bd
2025-01-29 17:26:44 +00:00
drh
fe15ed4342 Fixes from code inspection:
(1) Extra paranoid defense against integer overflow.
(2) Remove a stray blank line in an FTS5 comment.

FossilOrigin-Name: 8563728deeb1d3ead9ff99a3ad2c7819b2223ead44e7374ecc7bbeda2d955ade
2024-10-17 18:12:03 +00:00
drh
b8123415b2 Ensure that memory allocations in ANALYZE are always 8-byte aligned in order
to satisfy a new assert() added by [539e4f661767ef90].

FossilOrigin-Name: a1915cbdeccac2ed49fb8a9733abf2c6239396e3174fed2ff275f66b606c48b6
2024-08-16 15:35:20 +00:00
dan
a592883d87 Add extra assert() statements to the fixes on this branch.
FossilOrigin-Name: 81b6360050eea95e4367de1b41b5864a640b4d1c5c8fc3bea3b96ed770cb0325
2024-07-17 16:27:36 +00:00
drh
09a71d8d44 Add in a VdbeCoverage() macro needed by STAT4 that should have been part of
check-in [63ef234e88857a65].

FossilOrigin-Name: d51c699ae413215d534f954c129691ef5d95d540dc5d4304cbba576c976d931c
2024-02-28 15:32:32 +00:00
drh
e7bdb2172c If a table has one or more rows and it has a partial index has zero rows,
still make an entry in the sqlite_stat1 table for the partial index, so that
we know that "PRAGMA optimize" does not need to redo the whole table.

FossilOrigin-Name: e147b18991dd462fff367442acb0504fdf193a31843ed34ec8c1ced30747bf8a
2024-02-19 16:22:58 +00:00
drh
4cfc19f34c Remove some unnecessary computations from ANALYZE so that ANALYZE runs with
fewer CPU cycles.  These changes were spotted while working on the nearby
enhanced-stat1 branch.  So even if enhanced-stat1 is abandoned, that effort
put into it will not have been in vain.

FossilOrigin-Name: 5527e8c4abb904b1a438ec1c353d4a960bf82faaf3a2c742af1df7c613850441
2024-01-01 17:58:57 +00:00
drh
4ccf707c6e Extra steps taken to avoid using low-quality indexes in a query plan.
This branch accomplishes the same end as the nearby enhanced-stat1 branch,
but with much less change and hence less risk.

FossilOrigin-Name: c030e646262fee43a59b45fdc1630d972f8bf88ac3c142b6bdaf4cbb36695a4f
2024-01-01 17:48:02 +00:00
drh
04b92471f7 Extra space to prevent a buffer overread on corrupt STAT4 records.
dbsqlfuzz 7128d1b41ce9df2c007f9c24c1e89e2f1b2590ca.

FossilOrigin-Name: b99135288b157044e2319833e8632c89483778f876aa45ee66e46ffb6ae42ab2
2023-06-10 17:05:05 +00:00
drh
790adfd8ec Do not overflow the Index.aSample[] array if the same index appears in
the sqlite_stat4 table under multiple names because it is a WITHOUT ROWID
primary key index.  [forum:/info/537d8ab118df7edd|Forum post 537d8ab118df7edd]

FossilOrigin-Name: 9350a25ac0b55a6b901bc50e4db6d4e883c2617e1d2a8fdc90effabe52bb0012
2023-05-03 05:00:10 +00:00
drh
8777bdb78a Do not even attempt to load the sqlite_stat4 table if the use of STAT4 is
disabled using sqlite3_test_control().

FossilOrigin-Name: 0bf94c77d97582be5368bcfd149f3db7b9f928b4684aaa2626d98a2bdee8f96f
2023-05-03 04:21:31 +00:00
drh
223c6b48a9 The sqlite_stat4.idx field is case insensitive. It should work even if
some entries use a different case than others.  Fix for
[forum:/info/6c118daad0f1f5ef|forum post 6c118daad0f1f5ef].

FossilOrigin-Name: f097ca70b5b967d1aadebd74ac5020813e00b7c30cc543814dbf5f359d1328f1
2023-04-22 22:32:19 +00:00
drh
a9fd5f224c New #ifdefs to omit code that is unused except under STAT4.
FossilOrigin-Name: 09a9b30ba7828a487a33a3ebf8028dfaa147dff67d2724584123b90a88d9814b
2023-03-29 14:42:11 +00:00
drh
aa9192e6aa Improvements to register allocation, especially in the ANALYZE command.
New assert() statements added to help verify that memory allocation is
correct, and to help fuzzer find lingering errors.

FossilOrigin-Name: 6f8b97f31a4c8552312b4c98432ea356ae54c06d9cc929969f50c3c88360cd7b
2023-03-26 16:36:27 +00:00
drh
418f947b98 Disable factoring of constant values during ANALYZE. This is a temporary
fix for [forum:/forumpost/07de5f6216|forum post 07de5f6216].  The register
allocation logic in ANALYZE needs to be completely refactored, but that will
take longer.  This check-in will serve to resolve the issue until a better
fix can be devised.

FossilOrigin-Name: c3967d1259f1df969d303394986960bd098e174dcd337e374c9c3c39e0efa466
2023-03-26 11:54:51 +00:00
drh
52786ec185 Add the tag-20230325-1 comment that was omitted from the prior check-in.
FossilOrigin-Name: a13c01d076d23f0de500e8e6283e803dfc96f0da7509c0d97d598d6b3e7b930b
2023-03-25 18:41:42 +00:00
drh
4924e82863 The fix at [2bf5413dc2c19d5f] was incomplete in that it failed to clear
the reusable register cache that might contain registers in the STAT4
buffer region.  This additional change corrects the problem.
[forum:/forumpost/83cb4a95a0|Forum post 83cb4a95a0].  Test case in TH3.

FossilOrigin-Name: 5d554e4d0f59a4309fed40e4fb26c7be42f1d4d55ccdcaaf7b4d445aa3122955
2023-03-25 18:31:24 +00:00
drh
28fe02cd51 Fix byte-code register allocation in ANALYZE for STAT4 when there multiple
indexes with differing numbers of columns.
[forum:/forumpost/bc39e531e5|forum post bc39e531e5].

FossilOrigin-Name: 2bf5413dc2c19d5feb32e5b01aa9b990ec2f74f45f5ca0dca15215f8c9dbc9b9
2023-03-24 16:57:21 +00:00
drh
45ae9d6ba6 New assert() statements to verify that sqlite3DeleteIndexSamples() is always
called with non-NULL parameters.

FossilOrigin-Name: 92c71fdd7167a7db055ef0aec522bbba8c7b4b37658efda10dac44699921b57f
2023-02-01 20:45:12 +00:00
drh
4bc1cc1847 This experimental branch attempts to use columns for an index-on-expression
in place of the expression that is being indexed.  This particular check-in
mostly works, but there are still issues.

FossilOrigin-Name: 2e8d4fd4cfd9e82f33c707ba246fe2bb3ca01762cf5ac5905058fbc7adf0abe7
2022-10-13 21:08:34 +00:00
drh
609959285b When computing STAT1 values using ANALYZE, if a ratio comes out to be between
1.0 and 1.1, then round it down to 1 rather than the using the default rounding
rule of changing it to 2.  The reduces the estimation error for the
case where a column value is very nearly, but not quite unique.

FossilOrigin-Name: eb59c46a5aed69bc6fd096997bf24c082e533c1085439f6ec1fbe5ff78e8b374
2022-04-27 16:41:56 +00:00
drh
c45924ebc2 Remove unused P4 types on the Opcode object. Saves a few bytes of code
space and simplifies the code.

FossilOrigin-Name: aca538435939e6146f9a42e72e3a65030bbc5f646a0bbc7684e43688a0339888
2022-02-25 01:10:57 +00:00
drh
9966621da1 Use the IsOrdinaryTable() macro, not a test of tnum, to distinguish tables
from views and virtual tables.

FossilOrigin-Name: 6189b7809d774bf697d14e27a4bd961df5c318d261dce97db7801ce83f12c59a
2021-10-11 15:21:42 +00:00
drh
5bf4715e01 Add the sqlite3ResultStrAccum() internal interface to simplify the
the implementation of functions that return strings.

FossilOrigin-Name: e548e9299d3fd6ce5b647cf0dd93ff8e917a5eda43076c6a02389c52640e2e50
2021-10-03 00:12:43 +00:00
drh
e85e1da07b Fix harmless compiler warnings.
FossilOrigin-Name: 94b59691ee50a4666b25e36d1529fc52f714bbe94c3e8ccb35bf0a4ea11050db
2021-10-01 21:01:07 +00:00
drh
ebd1ff62c5 Ensure that sqlite_stat1 and sqlite_stat4 are ordinary tables (not views or
virtual tables) before trying to load them
(dbsqlfuzz bc02a0cde82dee801a8d6f653d2831680f87dca1).  This prevents
sqlite3_declare_vtab() from running with db->init.busy turned on.  Even so,
enhance sqlite3_declare_vtab() to be able to deal with db->init.busy being on,
in case there are undiscovered paths to that state.
Each of these two changes are independently sufficient to prevent the problem
fixed by the previous check-in [c7560c1329965ab5] but there
is no harm in keeping that third layer of protection in place.

FossilOrigin-Name: eb94f4a8174436b1f0deed0a43618a20018387bb815be658314ca6b454c446fb
2021-09-24 12:59:33 +00:00
drh
cf9d36d1b3 Refactor field names in the Column object, zCnName and zCnColl, to make them
unique and thus easier to find amid all the other code.

FossilOrigin-Name: 8b781dcaf68e0cf12a844708c82eee00193e340195cbca915d077e4846983bf3
2021-08-02 18:03:43 +00:00
drh
403869680b Sometimes it makes sense to do a full table scan rather than try to use
an index when most of the rows will be selected.  This branch is trying to
tune the query planner to make that happen more often.

FossilOrigin-Name: 0f42099ad65855c94af8472f3a6fddac7fc2a82e8fdfcc06a298eb6683a28688
2020-10-22 15:47:48 +00:00
drh
abc3815860 Continuing work toward supporting unsigned 32-bit page numbers.
FossilOrigin-Name: 9ce1710aad43cebe5ad50859c7685fb83e40cdd4a60913bd2b7e659bc59942fd
2020-07-22 13:38:04 +00:00
drh
e557b0155b Improved VDBE comments on the ANALYZE code generator. This change
also fixes a harmless use of an uninitialized integer variable as an input
to the %d format on a VDBE comment.

FossilOrigin-Name: 1cb248a3fc4c35c5bc3993b554edcccaa5c5e91570af67ebb99643a15221ae33
2020-05-30 00:30:08 +00:00
drh
3d42fb788a Fix harmless compiler warnings.
FossilOrigin-Name: 8eee591d3cb9fadfd5cac5543bd66ef9cb371a72d3ad3241fb3bfd67fb216eda
2020-05-04 19:52:00 +00:00
drh
f72981f2d9 Add a missing VdbeCoverage() macro.
FossilOrigin-Name: 77a55c394d2c313a5710229bee9262457dcfc6620e6500f9f526c5f6acf87cef
2020-05-02 03:29:21 +00:00
drh
31e3744ecb Simple fixes to PRAGMA analysis_limit.
FossilOrigin-Name: c20d4fdee21409ebc9c65c9540af8ac48d1f4425499a6674ef9319655c192612
2020-04-01 01:15:16 +00:00
drh
49a76a8fe5 Add "PRAGMA analysis_limit=N;" to limit the number of rows visited by
ANALYZE when N is positive.  Positive N also disables collecting stat4.

FossilOrigin-Name: a279b151c1623807774daf4975175c62ea252eefb71f9820ced6773769b392c5
2020-03-31 20:57:06 +00:00
drh
78a50d7280 Fix the build for when SQLITE_ENABLE_STAT4 is defined.
FossilOrigin-Name: 8f0a8c2aa45f7cf7339094d83893aeb046b010b5b97bb4dae99ac07a8ebf2fa6
2020-03-18 15:58:13 +00:00
drh
59a8cb7931 Add the SQLITE_ANALYZE_LIMIT compile-time option (expected to be temporary)
that sets a threshold at which ANALYZE starts to use approximations during
the analysis process.

FossilOrigin-Name: a773fd4698d474fda5e57bc77ed66a79cf74efee2706f43f6def6f450bfd1fc0
2020-03-18 14:43:05 +00:00
drh
9f27463684 Provide an estimated row count to stat_init() for STAT1 analysis.
FossilOrigin-Name: 714419fe85cfdad22979183a94e4569c87740652758ab76b646753cf2b013b54
2020-03-17 17:11:23 +00:00
drh
e50478d727 Remove the SQLITE_OMIT_BTREECOUNT option. Btree count is required.
FossilOrigin-Name: a9bfa47aeea27e91611ba913d33e6635d2016e2c2ab78f9b0657f1bd8933e1a8
2020-03-17 13:41:51 +00:00
drh
dc4a1687b8 Improved bytecode comments for the ANALYZE command.
FossilOrigin-Name: c38ea4139d87535b789f43eb1e38b2dc4b8312352a721035283e67cf0f3d5067
2020-03-17 13:07:04 +00:00
drh
9d33d9e7ac Background work for experiments trying to enhance ANALYZE so that it runs
off of samples of the entire index and does not need to read the entire
index.

FossilOrigin-Name: 29d1cc5c3619a88229f18c3c8131228f8a2d151ac3d9203f0c7fc538a996ecec
2020-03-10 01:24:13 +00:00
drh
afa1ecac9b Cleaner separation of the STAT4-specific logic in the implementation of
ANALYZE.

FossilOrigin-Name: 3df07e5a9a3781a4cf866fc6ee0e5c6f9cd7ca35ce0a6eb3aa7f5f3502e0ffae
2020-03-09 18:26:11 +00:00
drh
920cf596e6 Simplify the bytecode generation for SQL function calls such that the
OP_Function or OP_PureFunc opcodes are coded directly, rather than using
the intermediate OP_Function0 or OP_PureFunc0 - opcodes that are now removed.

FossilOrigin-Name: 84e02d773d60cffe619104991d21d7f0c68616c0f6bb99686bf54f5306c756d0
2019-10-30 16:29:02 +00:00
drh
b9bcf7ca60 Refactor names of column index transformation functions, for clarity.
Get generated columns working with ALTER TABLE RENAME COLUMN.

FossilOrigin-Name: 27ab41c9102e7801ff829488fc123a8040da008bef373d6704efbe2f93e1da90
2019-10-19 13:29:10 +00:00