MDEV-24854: Change innodb_flush_method=O_DIRECT by default

We have innodb_use_native_aio=ON by default since the introduction of
that parameter in commit 2f9fb41b05
(MySQL 5.5 and MariaDB 5.5).

However, to really benefit from the setting, the files should be
opened in O_DIRECT mode, to bypass the file system cache.
In this way, the reads and writes can be submitted with DMA, using
the InnoDB buffer pool directly, and no processor cycles need to be
used for copying data. The use of O_DIRECT benefits not only the
current libaio implementation, but also liburing.

os_file_set_nocache(): Test innodb_flush_method in the function,
not in the callers.

mysql-test/suite/sys_vars/r/innodb_flush_method_func.result

@@ -1,7 +1,7 @@
 call mtr.add_suppression("InnoDB: Failed to set .*DIRECT");
 select @@innodb_flush_method;
 @@innodb_flush_method
-fsync
+O_DIRECT
 create table t(a serial) engine=innodb;
 FLUSH TABLES;
 # restart: --innodb-flush-method=5