The issue was a bad merge of MDEV-12253 from 10.1 to 10.2
in commit f9cc391863ab962e8c8a8a8a676d730498eb8d6f.
In that merge, I wrongly assumed that all test file conflicts
for mysql-test/suite/encryption had been properly resolved in
bb-10.2-MDEV-12253 (commit 76aa6be77635c7017459ce33b41c837c9acb606d)
while in fact, some files there had been copied from the 10.1 branch.
This commit is based on a manually done conflict resolution of
the mysql-test/suite/encryption on the same merge, applied to
the current 10.2 branch.
As part of this commit, the test encryption.innodb-bad-key-change4
which was shortly disabled due to MDEV-11336 will be re-enabled again.
(While the test enables innodb_defragment, it does not fail even though
enabling innodb_defragment currently has no effect.)
Folloup: Made encryption rules too strict (and incorrect). Allow creating
table with ENCRYPTED=OFF with all values of ENCRYPTION_KEY_ID but create
warning that nondefault values are ignored. Allow creating table with
ENCRYPTED=DEFAULT if used key_id is found from key file (there was
bug on this) and give error if key_id is not found.
Analysis: Problem sees to be the fact that we allow creating or altering
table to use encryption_key_id that does not exists in case where
original table is not encrypted currently. Secondly we should not
do key rotation to tables that are not encrypted or tablespaces
that can't be found from tablespace cache.
Fix: Do not allow creating unencrypted table with nondefault encryption key
and do not rotate tablespaces that are not encrypted (FIL_SPACE_ENCRYPTION_OFF)
or can't be found from tablespace cache.
