c83d20a520
* Migrate docker package to the new test framework Move the docker package from the legacy Arquillian testsuite to tests/base and migrate DockerClientTest to KeycloakIntegrationTest with injected realm/events and server feature config. Closes #50760 Signed-off-by: Erik Jan de Wit <erikjan.dewit@gmail.com> * Include docker package in Base2TestSuite Add the migrated docker tests package to the base suite selector so check-base-suites recognizes it and Build CI no longer fails. Signed-off-by: Erik Jan de Wit <erikjan.dewit@gmail.com> * Remove docker package from legacy base-suite Drop the migrated docker package from the Arquillian base-suite list so base-suite.sh no longer fails on a missing org.keycloak.testsuite.docker directory. Signed-off-by: Erik Jan de Wit <erikjan.dewit@gmail.com> * Skip body staleness wait for HtmlUnit page reloads HtmlUnit form submission is synchronous, so waiting for the body element to become stale after login submit times out and makes SessionRestServiceTest flaky in Base2TestSuite. Closes #52283 Signed-off-by: Erik Jan de Wit <erikjan.dewit@gmail.com> * Address PR review feedback for Docker test migration Revert the unrelated WaitUtils change, remove the unused DockerHostIpSupplier, declare the Testcontainers dependency directly, and update HOW-TO-RUN.md for the new test framework. Signed-off-by: Erik Jan de Wit <erikjan.dewit@gmail.com> * Move Docker test docs to tests/docs Relocate DockerClientTest setup instructions from the Arquillian HOW-TO-RUN guide to tests/docs, per PR review feedback. Signed-off-by: Erik Jan de Wit <erikjan.dewit@gmail.com> * Fix Docker test doc for test framework defaults Document distribution mode as the default, use ./mvnw, and describe the Quarkus distribution build plus KC_TEST_SERVER=embedded opt-in. Signed-off-by: Erik Jan de Wit <erikjan.dewit@gmail.com> --------- Signed-off-by: Erik Jan de Wit <erikjan.dewit@gmail.com>
New Tests
The new tests parent module incorporate:
- tests written in Keycloak Test Framework: base, clustering, webauthn
- support modules for utilities and extensions: utils, utils-shared, custom-providers, custom-scripts
- migration utility (from Arquillian to Keycloak Test Framework)
Tests modules
Keep attention where a particular test belong, most of the tests are placed in the base module, but some require very specific configuration, like clusterting or webauthn.
Support for test suites:
- Based on JUnit Platform Suite
- All currently available suites are in the suites package
- Part of the GitHub Actions CI worklow
Useful guides:
- Keycloak Test Framework
- FIPS tests execution - how to set up a local machine to successfully execute the FIPS tests.
- Docker Authentication test - how to run
DockerClientTestlocally.
Utilities
utils
Some test classes need complex executions and use a logic, which can be reused over and over. To remove a boilerplate code, this is the place - the utils module, where some static utilities can be placed.
Are you thinking of adding a new util class?
- Always check if this functionality is part of the test framework.
- Check the existing util classes.
- If none of them are sufficient, then create a new one, but:
- Be cautious, since it creates a new complexity.
- Be as specific with the names as possible => clear package and class name.
utils-shared
The utils-shared module is for migration purpose only and will be removed once the migration is over! Meaningful utilities will be refactored and moved to utils.
It prevents the new tests to be dependent on the old Arquillian testsuite and in the same time, it simplifies the overall migration. Is a util class (in the old testsuite) shared by multiple test cases?
- Migrate util class into this module first.
- Then migrate your test.
Extensions
custom-providers
The custom-providers module is used within tests to specify a custom extension, built on top of the Keycloak SPI. Every custom extension with particular factory must be stored in this module!
custom-scripts
Same applies for the custom-scripts module, it deploys custom authenticators (feature scripts) written in JavaScript. Every such script must be stored in this separate module!
All of them can be deployed as a dependency within the KeycloakServerConfig implementation, please check the Keycloak Test Framework docs.
Migration
The migration-util is a tool, written in Java, to simplify the overall migration. It can be executed via migrate.sh script. Its purpose is to copy/paste selected test case, and parse and replace patterns to follow the Keycloak Test Framework.
Pay attention! It doesn't support every pattern just speed up the initial phase, before digging into a manual migration.
Example of usage: ./migrate.sh AttributesTest
Please, consider using this script for making a commit, because it allows to preserve a history tree for a specific test case:
commit-migration.sh
Available guides, which cover the required knowledge to successfully migrate a single test case:
- Migrating tests - Quick start for a single test migration (includes the migration utility best practice)
- Manual migration - Guide with recommendations on how to fully migrate a test
Once the migration is over the migration-util module will be removed!