Problem:
The following functions-
- `Item_func_last_day::get_date`
- `Item_date_typecast::get_date`
- `Item_datetime_typecast::get_date` forwarded the TIME_INTERVAL_DAY /
TIME_INTERVAL_hhmmssff flags, set by an enclosing EXTRACT, into the
DATE/DATETIME conversion of their argument.This made, for ex:
LAST_DAY(200012010000) and CAST(200012010000 AS DATETIME) parse number
as TIME interval instead of a DATETIME. Under ZERO_DATE_TIME_CAST
that hour became an out-of-range month, overflowing days_in_month in
`check_date` and invalidating the Datetime.
Fix:
A DATE / DATETIME can never be an interval, so suppress the
TIME_INTERVAL_hhmmssff / TIME_INTERVAL_DAY flags when `LAST_DAY` and
`CAST` converts its argument. Also add a debug assert in `check_date`
to catch any future caller that violates its "month is assumed valid"
contract.
Before this fix, TO_DATE() returned NULL when the date string
contained extra non-alphanumeric separators that were not present
in the format string. For example:
TO_DATE('2002 - AD', 'YYYY AD') -- returned NULL
TO_DATE('2024---01---01', 'YYYYMMDD') -- returned NULL
Oracle handles these cases flexibly by ignoring extra separators.
Fix: skip non-alphanumeric separators in the date string before
matching each format token. A '-' or '+' sign is preserved when
the format is SYYYY, to allow signed year parsing.
Reported-by: Elena Stepanova <elenastepanova@mariadb.com>
Signed-off-by: Alexandru Diaconu <alexandrudiaconu93@gmail.com>
Remove the unscoped #pragma GCC diagnostic ignored
"-Wdeprecated-declarations" from include/violite.h and replace all
sprintf/vsprintf calls with snprintf/vsnprintf across the codebase.
Where possible, pass actual buffer sizes as function parameters
instead of using hardcoded constants or recomputing strlen().
Notable API changes:
- get_date(): added to_len parameter for buffer size
- nice_time(), end_timer() in client/mysql.cc: added buff_size
- generate_new_name(): added name_size parameter
- calc_md5(): added buffer_size parameter (fixes off-by-one)
- MD5_HASH_TO_STRING macro: added _size parameter
- Added MYSQL_UDF_MAX_RESULT_LENGTH constant in mysql_com.h
Also fixed a pre-existing bug in sql/rpl_mi.cc where the ellipsis
"..." was written to the wrong buffer (dbuff instead of buff).
Vendored code is excluded: extra/readline, extra/wolfssl,
wsrep-lib, libmariadb, storage/mroonga/vendor, zlib.
The Oracle-mode TO_DATE() function incorrectly parsed strings containing
conflicting format specifiers within the Day-name family (e.g., 'DAY DY')
without throwing an ER_STD_INVALID_ARGUMENT error.
This patch completes the existing validation framework for family-based
conflict checks in item_timefunc.cc during format string parsing.
Specifically, it adds formats_used() checks for FMT_DAY, ensuring that
specifiers within the Day-name family strictly block each other. This
aligns the Day-name handling with the already existing checks for the
Year, Month, Hour, and Meridian families.
Compatible specifiers, such as YYYY and DDD (Year and Day of Year),
remain unaffected and parse successfully. A comprehensive MTR test
suite has been added to func_to_date.test to cover standard and
edge-case conflicts.
Reported-by: Elena Stepanova <elenastepanova@github>
Signed-off-by: Alexandru Diaconu <alexandrudiaconu93@gmail.com>
Update STR_TO_DATE() to accept an optional third argument for the locale.
DATE_FORMAT() and STR_TO_DATE() return NULL when the locale argument is NULL;
issue a warning and fall back to the session @@lc_time_names for invalid locale names.
STR_TO_DATE() can now be used in indexed virtual columns while having all 3 arguments,
but not allowed when 2 as it depends then on the session @@lc_time_names
Months names were taken from https://unicode.org/cldr/charts/48/summary/root.html,
which has two different categories for full month names: "Months - wide - Formatting"
and "Months - wide - Standalone".
Formatting is normaly used when the month is a part of a date (e.g., "12 January 2026").
In many languages (like Slavic or Baltic), the month's ending changes in this context.
Standalone is used when the month name appears alone.
%B prints "formatting" months names, %M - "standalone" names
Syntax:
TO_DATE(string_expression [DEFAULT string_expression ON CONVERSION ERROR],
format_string [,NLS_FORMAT_STRING])
The format_string has the same format elements as TO_CHAR(), except a
few elements that are not supported/usable for TO_DATE().
TO_DATE() returns a datetime or date value, depending on if the format
element FF is used.
Allowed separators, same as TO_CHAR():
space, tab and any of !#%'()*+,-./:;<=>
'&' can also be used if next character is not a character a-z or A-Z
"text' indicates a text string that is verbatim in the format. One cannot
use " as a separator.
Format elements supported by TO_DATE():
AD Anno Domini ("in the year of the Lord")
AD_DOT Anno Domini ("in the year of the Lord")
AM Meridian indicator (Before midday)
AM_DOT Meridian indicator (Before midday)
DAY Name of day
DD Day (1-31)
DDD Day of year (1-336)
DY Abbreviated name of day
FF[1-6] Fractional seconds
HH Hour (1-12)
HH12 Hour (1-12)
HH24 Hour (0-23)
MI Minutes (0-59)
MM Month (1-12)
MON Abbreviated name of month
MONTH Name of Month
PM Meridian indicator (After midday)
PM_DOT Meridian indicator (After midday)
RR 20th century dates in the 21st century. 2 digits
50-99 is assumed from 2000, 0-49 is assumed from 1900.
RRRR 20th century dates in the 21st century. 4 digits
SS Seconds
SYYYY Signed 4 digit year; MariaDB only supports positive years
Y 1 digit year
YY 2 digits year
YYY 3 digits year
YYYY 4 digits year
Note that if there is a missing part of the date, the current date is used!
For example if 'MM-DD HH-MM-SS' then the current year will be used.
(Oracle behaviour)
Not supported options:
- BC, D, DL, DS, E, EE, FM, FX, RM, SSSSS, TS, TZD, TZH, TZR, X,SY
BC is not supported by MariaDB datetime.
- Most of the other are exotic formats does not make sence in MariaDB as
we return datetime or datetime with fractions, not string.
- D (day-of-week) is not supported as it is not clear exactly how it would
map to MariaDB. This element depends on the NLS territory of the session.
- RR only works with 2 digit years (In Oracle RR can also work with 4
digit years in some context but the rules are not clear).
Extensions / differences compared to Oracle;
- MariaDB supports FF (fractional seconds). If FF[#] is used,
then TO_DATE will return a datetime with # of subseconds.
If FF is not used a datetime will be returned.
There is warning (no error) if string contains more digts than what
is specified with F(#]
- Names can be shortened to it's unique prefix. For example January and Ja
works fine.
- No error if the date string is shorter format_string and the next
not used character is not a number.. This is useful to get a date
from a mixed set of strings in date or datetime format.
Oracle gives an error if date string is too short.
- MariaDB supports short locales as language names
- NLS_DATE_FORMAT can use both " and ' for quoting.
- NLS_DATE_FORMAT must be a constant string.
- This is to ensure that the server knows which locale to use
when executing the function.
New formats handled by TO_CHAR():
FF[1-6] Fractional seconds
DDD Daynumber 1-366
IW Week 1-53 according to ISO 8601
I 1 digit year according to ISO 8601
IY 2 digit year according to ISO 8601
IYY 3 digit year according to ISO 8601
IYYY 4 digit year according to ISO 8601
SYYY 4 digit year according to ISO 8601 (Oracle can do signed)
Supported NLS_FORMAT_STRING options are:
NLS_CALENDAR=GREGORIAN
NLS_DATE_LANGUAGE=language
Support languages are:
- All MariaDB short locales, like en_AU.
- The following Oracle language names:
ALBANIAN, AMERICAN, ARABIC, BASQUE, BELARUSIAN, BRAZILIAN PORTUGUESE
BULGARIAN, CANADIAN FRENCH, CATALAN, CROATIAN, CYRILLIC SERBIAN CZECH,
DANISH, DUTCH, ENGLISH, ESTONIAN, FINNISH, FRENCH, GERMAN,
GREEK, HEBREW, HINDI, HUNGARIAN, ICELANDIC, INDONESIAN ITALIAN,
JAPANESE, KANNADA, KOREAN, LATIN AMERICAN SPANISH, LATVIAN,
LITHUANIAN, MACEDONIAN, MALAY, MEXICAN SPANISH, NORWEGIAN, POLISH,
PORTUGUESE, ROMANIAN, RUSSIAN, SIMPLIFIED CHINESE, SLOVAK, SLOVENIAN,
SPANISH, SWAHILI, SWEDISH, TAMIL, THAI, TRADITIONAL CHINESE, TURKISH,
UKRAINIAN, VIETNAMESE
Development bugs fixed:
MDEV-38403 Server crashes in Item_func_to_date::fix_length_and_dec upon
using an invalid argument
MDEV-38400 compat/oracle.func_to_date fails with PS protocol and cursor
protocol (Fixed by Serg)
MDEV-38404 TO_DATE: MTR coverage omissions, round 1
MDEV-38509 TO_DATE: AD_DOT does not appear to be supported
MDEV-38513 TO_DATE: NULL value for format string causes assertion failure
MDEV-38521 TO_DATE: Date strings with non-ASCII symbols cause warnings
and wrong results
MDEV-38578 TO_DATE: Possibly unexpected results upon wrong input
MDEV-38582 TO_DATE: NLS_DATE_LANGUAGE=JAPANESE does not parse values
which work in Oracle
MDEV-38584 TO_DATE: NLS_DATE_LANGUAGE=VIETNAMESE does not parse values
which work in Oracle
MDEV-38703 TO_DATE: Quotation for multi-word NLS_DATE_LANGUAGE leads
to syntax error in view definition
MDEV-38675 TO_DATE: MSAN/Valgrind/UBSAN errors in
extract_oracle_date_time
MDEV-38635 TO_DATE: UBSAN errors in item_timefunc.h upon comparison with
a view column
MDEV-38719 TO_DATE: Assertion `&my_charset_bin != charset()' failed in
String::append_for_single_quote_using_mb_wc
MDEV-38756 TO_DATE: MSAN/Valgrind errors in
Item_func_to_date::fix_length_and_dec upon PREPARE with
parameters
Known issues:
- Format string character matches inside quotes are done
one-letter-to-one-letter, like in LIKE predicate. That means things
like expansions and contractions do not work.
For example 'ss' does not match 'ß' in collations which treat them
as equal for the comparison operator.
Match is done taking into account case and accent sensitivity
of the subject argument collation, so for example this now works:
MariaDB [test]> SELECT TO_DATE('1920á12','YYYY"a"MM') AS c;
+---------------------+
| c |
+---------------------+
| 1920-12-17 00:00:00 |
+---------------------+
Co-author and reviewer: Alexander Barkov <bar@mariadb.com>
If one compiles with EXTENDED_MONTHS_BETWEEN MariaDB would take hours,
minutes, seconds and fractional seconds into account when comparing
dates when computing the fractional months.
For example
MONTHS_BETWEEN("1995-02-15 12:00:00", "1995-01-17 12:00:01") would
return "one day less" than if time part would not have been given.
It returns the DATETIME data type (which is the closest to Oracle's DATE).
The following Oracle formats are supported:
Truncate to day: DD, DDD,J
Truncate to month: MM,MON,MONTH,RM
Truncate to Year: SYEAR,SYYYY,Y,YEAR,YY,YYY,YYYY
TRUNC(date) is same as TRUNC(date, "DD")
This patch incorporates a fix for:
MDEV-37414 SIGSEGV in Binary_string::c_ptr | Item_func_trunc::get_date
Fixing the problem that the code did not take into
account that args[1] can return SQL NULL.
- Removed duplicate words, like "the the" and "to to"
- Removed duplicate lines (one double sort line found in mysql.cc)
- Fixed some typos found while searching for duplicate words.
Command used to find duplicate words:
egrep -rI "\s([a-zA-Z]+)\s+\1\s" | grep -v param
Thanks to Artjoms Rimdjonoks for the command and pointing out the
spelling errors.
STR_TO_DATE function in MariaDB does date to string conversion for invalid
dates strings, e.g.: 1949-02-30, and 1949-06-31, but null is return by
other invalid dates such as 1949-01-32 because the checking function
returns null for any date whose day is larger than 31, which is too naive
and needs to be changed.
This commit changed the rule of check_date in extract_date_time function,
which is executed after extracting date and time and check if the result
is valid. Now the conversion will consider the number of days in a month,
including skip years.
All new code of the whole pull request, including one or several files that
are either new files or modified ones, are contributed under the BSD-new
license. I am contributing on behalf of my employer Amazon Web Services, Inc.
This patch adds support for SYS_REFCURSOR (a weakly typed cursor)
for both sql_mode=ORACLE and sql_mode=DEFAULT.
Works as a regular stored routine variable, parameter and return value:
- can be passed as an IN parameter to stored functions and procedures
- can be passed as an INOUT and OUT parameter to stored procedures
- can be returned from a stored function
Note, strongly typed REF CURSOR will be added separately.
Note, to maintain dependencies easier, some parts of sql_class.h
and item.h were moved to new header files:
- select_results.h:
class select_result_sink
class select_result
class select_result_interceptor
- sp_cursor.h:
class sp_cursor_statistics
class sp_cursor
- sp_rcontext_handler.h
class Sp_rcontext_handler and its descendants
The implementation consists of the following parts:
- A new class sp_cursor_array deriving from Dynamic_array
- A new class Statement_rcontext which contains data shared
between sub-statements of a compound statement.
It has a member m_statement_cursors of the sp_cursor_array data type,
as well as open cursor counter. THD inherits from Statement_rcontext.
- A new data type handler Type_handler_sys_refcursor in plugins/type_cursor/
It is designed to store uint16 references -
positions of the cursor in THD::m_statement_cursors.
- Type_handler_sys_refcursor suppresses some derived numeric features.
When a SYS_REFCURSOR variable is used as an integer an error is raised.
- A new abstract class sp_instr_fetch_cursor. It's needed to share
the common code between "OPEN cur" (for static cursors) and
"OPER cur FOR stmt" (for SYS_REFCURSORs).
- New sp_instr classes:
* sp_instr_copen_by_ref - OPEN sys_ref_curor FOR stmt;
* sp_instr_cfetch_by_ref - FETCH sys_ref_cursor INTO targets;
* sp_instr_cclose_by_ref - CLOSE sys_ref_cursor;
* sp_instr_destruct_variable - to destruct SYS_REFCURSOR variables when
the execution goes out of the BEGIN..END block
where SYS_REFCURSOR variables are declared.
- New methods in LEX:
* sp_open_cursor_for_stmt - handles "OPEN sys_ref_cursor FOR stmt".
* sp_add_instr_fetch_cursor - "FETCH cur INTO targets" for both
static cursors and SYS_REFCURSORs.
* sp_close - handles "CLOSE cur" both for static cursors and SYS_REFCURSORs.
- Changes in cursor functions to handle both static cursors and SYS_REFCURSORs:
* Item_func_cursor_isopen
* Item_func_cursor_found
* Item_func_cursor_notfound
* Item_func_cursor_rowcount
- A new system variable @@max_open_cursors - to limit the number
of cursors (static and SYS_REFCURSORs) opened at the same time.
Its allowed range is [0-65536], with 50 by default.
- A new virtual method Type_handler::can_return_bool() telling
if calling item->val_bool() is allowed for Items of this data type,
or if otherwise the "Illegal parameter for operation" error should be raised
at fix_fields() time.
- New methods in Sp_rcontext_handler:
* get_cursor()
* get_cursor_by_ref()
- A new class Sp_rcontext_handler_statement to handle top level statement
wide cursors which are shared by all substatements.
- A new virtual method expr_event_handler() in classes Item and Field.
It's needed to close (and make available for a new OPEN)
unused THD::m_statement_cursors elements which do not have any references
any more. It can happen in various moments in time, e.g.
* after evaluation parameters of an SQL routine
* after assigning a cursor expression into a SYS_REFCURSOR variable
* when leaving a BEGIN..END block with SYS_REFCURSOR variables
* after setting OUT/INOUT routine actual parameters from formal
parameters.
Adding support for the "FM" format in function TO_CHAR(date_time, fmt).
"FM" in the format string disables padding of all components following it.
So now TO_CHAR() works as follows:
- By default string format components DAY (weekday name) and
MONTH (month name) are right-padded with spaces to the maximum
possible DAY and MONTH name lengths respectively,
according to the current locale specified in @@lc_time_names.
So for example, with lc_time_names='en_US' all month names are
right-padded with spaces up to 9 characters ('September' is the longest).
SET lc_time_names='en_US';
SELECT TO_CHAR('0001-02-03', 'MONTH'); -> 'February ' (padded to 9 chars)
NEW: When typed after FM, DAY and MONTH names are not right-padded
with trailing spaces any more:
SET lc_time_names='en_US';
SELECT TO_CHAR('0001-02-03', 'FMMONTH'); -> 'February' (not padded)
- By default numeric components YYYY, YYY, YY, Y, DD, H12, H24, MI, SS
are left-padded with leading digits '0' up to the maximum possible
number of digits in the component (e.g. 4 for YYYY):
SELECT TO_CHAR('0001-02-03', 'YYYY'); -> '0001' (padded to 4 chars)
NEW: When typed after FM, these numeric components are not left-padded
with leading zeros any more:
SELECT TO_CHAR('0001-02-03', 'FMYYYY'); -> '1' (not padded)
- If FM is specified multiple times in a format string,
every FM negates the previous padding state:
* an odd FM disables padding
* an even FM enables padding
Implementation details:
- Adding a helper class Date_time_format_oracle.
- Adding a helper method Date_time_format_oracle::append_lex_cstring()
- Moving the function append_val() to Date_time_format_oracle as a method.
- Moving the function make_date_time_oracle() to Date_time_format_oracle
as a method format().
- Adding helper methods month_name() and day_name() in class MY_LOCALE,
to return the corresponding components as LEX_CSTRINGs.
Item_{date|datetime}_typecase::get_date() erroneously passed the
TIME_INTERVAL_DAY flag from the caller to args[0] which made
CAST('100000:00:00' AS DATETIME) parse '100000:00:00' as TIME
rather that DATETIME.
Suppressing this flag.
Changing the return type of the following functions:
- CURRENT_TIMESTAMP, CURRENT_TIMESTAMP(), NOW()
- SYSDATE()
- FROM_UNIXTIME()
from DATETIME to TIMESTAMP.
Note, the old function NOW() returning DATETIME is still available
as LOCALTIMESTAMP or LOCALTIMESTAMP(), e.g.:
SELECT
LOCALTIMESTAMP, -- DATETIME
CURRENT_TIMESTAMP; -- TIMESTAMP
The change in the functions return data type fixes some problems
that occurred near a DST change:
- Problem #1
INSERT INTO t1 (timestamp_field) VALUES (CURRENT_TIMESTAMP);
INSERT INTO t1 (timestamp_field) VALUES (COALESCE(CURRENT_TIMESTAMP));
could result into two different values inserted.
- Problem #2
INSERT INTO t1 (timestamp_field) VALUES (FROM_UNIXTIME(1288477526));
INSERT INTO t1 (timestamp_field) VALUES (FROM_UNIXTIME(1288477526+3600));
could result into two equal TIMESTAMP values near a DST change.
Additional changes:
- FROM_UNIXTIME(0) now returns SQL NULL instead of '1970-01-01 00:00:00'
(assuming time_zone='+00:00')
- UNIX_TIMESTAMP('1970-01-01 00:00:00') now returns SQL NULL instead of 0
(assuming time_zone='+00:00'
These additional changes are needed for consistency with TIMESTAMP fields,
which cannot store '1970-01-01 00:00:00 +00:00'
