Mirrors/fish-shell
2
0
Fork 0
mirror of https://github.com/fish-shell/fish-shell synced 2024-09-20 14:32:04 +00:00
Code Issues Projects Releases Packages Wiki Activity
fish-shell/cmake/Tests.cmake

151 lines
6 KiB
CMake
Raw Normal View History

Support for fish_tests This adds a basic Tests.cmake that can build and run fish_tests. It also adds a 'test' target.
2017-09-01 07:31:51 +00:00
# Define fish_tests.
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
add_executable(fish_tests EXCLUDE_FROM_ALL
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
src/fish_tests.cpp)
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
fish_link_deps_and_sign(fish_tests)
Support for fish_tests This adds a basic Tests.cmake that can build and run fish_tests. It also adds a 'test' target.
2017-09-01 07:31:51 +00:00
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
# The "test" directory.
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
set(TEST_DIR ${CMAKE_CURRENT_BINARY_DIR}/test)
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
# The directory into which fish is installed.
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
set(TEST_INSTALL_DIR ${TEST_DIR}/buildroot)
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
# The directory where the tests expect to find the fish root (./bin, etc)
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
set(TEST_ROOT_DIR ${TEST_DIR}/root)
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
# Copy tests files.
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
file(GLOB TESTS_FILES tests/*)
add_custom_target(tests_dir DEPENDS tests)
copy test files with ADD_CUSTOM_COMMAND This will copy the files every time "make test" is called, so the files are never out of sync. Fixes issue #4633
2018-01-06 12:07:12 +00:00
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
if(NOT FISH_IN_TREE_BUILD)
add_custom_command(TARGET tests_dir
[cmake] only copy tests for out-of-tree builds Makes in-tree builds work again.
2018-01-23 09:35:43 +00:00
COMMAND ${CMAKE_COMMAND} -E copy_directory
${CMAKE_SOURCE_DIR}/tests/ ${CMAKE_BINARY_DIR}/tests/
COMMENT "Copying test files to binary dir"
VERBATIM)
copy test files with ADD_CUSTOM_COMMAND This will copy the files every time "make test" is called, so the files are never out of sync. Fixes issue #4633
2018-01-06 12:07:12 +00:00
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
add_dependencies(fish_tests tests_dir)
endif()
Support for fish_tests This adds a basic Tests.cmake that can build and run fish_tests. It also adds a 'test' target.
2017-09-01 07:31:51 +00:00
Port printf tests to littlecheck and teach the tester how to run it This adds support for .check files inside the tests directory. .check files are tests designed to be run with littlecheck. Port printf test to littlecheck and remove the printf.in test.
2019-06-09 18:13:31 +00:00
# Copy littlecheck.py
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
configure_file(build_tools/littlecheck.py littlecheck.py COPYONLY)
Port printf tests to littlecheck and teach the tester how to run it This adds support for .check files inside the tests directory. .check files are tests designed to be run with littlecheck. Port printf test to littlecheck and remove the printf.in test.
2019-06-09 18:13:31 +00:00
Add pexpect-based interactive testing framework This adds a new interactive test framework based on Python's pexpect. This is intended to supplant the TCL expect-based tests. New tests go in `tests/pexpects/`. As a proof-of-concept, the pipeline.expect test and the (gnarly) bind.expect test are ported to the new framework.
2020-03-02 23:20:29 +00:00
# Copy pexpect_helper.py
configure_file(build_tools/pexpect_helper.py pexpect_helper.py COPYONLY)
Support for fish_tests This adds a basic Tests.cmake that can build and run fish_tests. It also adds a 'test' target.
2017-09-01 07:31:51 +00:00
# Make the directory in which to run tests.
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
# Also symlink fish to where the tests expect it to be.
Add fish_test_helper executable In tests we would like to arrange for an executable to invoke certain system calls, e.g. to claim or relinquish control of the terminal. This is annoying to do portably via e.g. perl. fish_test_helper is a little program where we can add custom commands to make it act in certain ways.
2019-04-07 05:22:11 +00:00
# Lastly put fish_test_helper there too.
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
add_custom_target(tests_buildroot_target
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
COMMAND ${CMAKE_COMMAND} -E make_directory ${TEST_INSTALL_DIR}
COMMAND DESTDIR=${TEST_INSTALL_DIR} ${CMAKE_COMMAND}
--build ${CMAKE_CURRENT_BINARY_DIR} --target install
Add fish_test_helper executable In tests we would like to arrange for an executable to invoke certain system calls, e.g. to claim or relinquish control of the terminal. This is annoying to do portably via e.g. perl. fish_test_helper is a little program where we can add custom commands to make it act in certain ways.
2019-04-07 05:22:11 +00:00
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_BINARY_DIR}/fish_test_helper
${TEST_INSTALL_DIR}/${CMAKE_INSTALL_PREFIX}/bin
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
COMMAND ${CMAKE_COMMAND} -E create_symlink
${TEST_INSTALL_DIR}/${CMAKE_INSTALL_PREFIX}
${TEST_ROOT_DIR}
Add fish_test_helper executable In tests we would like to arrange for an executable to invoke certain system calls, e.g. to claim or relinquish control of the terminal. This is annoying to do portably via e.g. perl. fish_test_helper is a little program where we can add custom commands to make it act in certain ways.
2019-04-07 05:22:11 +00:00
DEPENDS fish fish_test_helper)
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
if(NOT FISH_IN_TREE_BUILD)
[cmake] only copy tests for out-of-tree builds Makes in-tree builds work again.
2018-01-23 09:35:43 +00:00
# We need to symlink share/functions for the tests.
# This should be simplified.
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
add_custom_target(symlink_functions
[cmake] only copy tests for out-of-tree builds Makes in-tree builds work again.
2018-01-23 09:35:43 +00:00
COMMAND ${CMAKE_COMMAND} -E create_symlink
${CMAKE_CURRENT_SOURCE_DIR}/share/functions
${CMAKE_CURRENT_BINARY_DIR}/share/functions)
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
add_dependencies(tests_buildroot_target symlink_functions)
else()
add_custom_target(symlink_functions)
endif()
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
# Prep the environment for running the unit tests.
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
add_custom_target(test_prep
Override all XDG paths when running tests This is far preferable to the per-test XDG overrides that we may or may not remember to add the next time around. Also rename all the directories so it is clear via which variable a file made its way into that path.
2020-10-26 02:57:45 +00:00
# Add directories hard-coded into the tests
COMMAND ${CMAKE_COMMAND} -E remove_directory ${TEST_DIR}/data
COMMAND ${CMAKE_COMMAND} -E make_directory ${TEST_DIR}/data
COMMAND ${CMAKE_COMMAND} -E remove_directory ${TEST_DIR}/temp
COMMAND ${CMAKE_COMMAND} -E make_directory ${TEST_DIR}/temp
# Add the XDG_* directories
COMMAND ${CMAKE_COMMAND} -E remove_directory ${TEST_DIR}/xdg_data
COMMAND ${CMAKE_COMMAND} -E make_directory ${TEST_DIR}/xdg_data
COMMAND ${CMAKE_COMMAND} -E remove_directory ${TEST_DIR}/xdg_config
COMMAND ${CMAKE_COMMAND} -E make_directory ${TEST_DIR}/xdg_config
COMMAND ${CMAKE_COMMAND} -E remove_directory ${TEST_DIR}/xdg_runtime
COMMAND ${CMAKE_COMMAND} -E make_directory ${TEST_DIR}/xdg_runtime
DEPENDS tests_buildroot_target tests_dir
USES_TERMINAL)
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
Allow running test suites independently This makes test_low_level, test_interactive, test_invocation, and test_fishscript independent. This allows running a smaller subset of tests. To prevent all tests running in parallel, we also have new targets serial_test_low_level, serial_test_interactive, etc. which have the dependency chain that enforces serial execution.
2019-06-22 21:17:10 +00:00
# Define our individual tests.
# Each test is conceptually independent.
# However when running all tests, we want to run them serially for sanity's sake.
# So define both a normal target, and a serial variant which enforces ordering.
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
foreach(TESTTYPE test serial_test)
add_custom_target(${TESTTYPE}_low_level
Tests: Set $XDG_DATA_DIRS to empty instead of unsetting NetBSD's `env` doesn't have `-u` and this should be harmless.
2021-05-18 10:24:24 +00:00
COMMAND env XDG_DATA_DIRS=
Correct test environment setup XDG_RUNTIME_DIR was misspelled. XDG_DATA_DIRS could affect tests. Remove it since it's used read-only.
2021-04-20 22:43:28 +00:00
XDG_DATA_HOME=${CMAKE_CURRENT_BINARY_DIR}/test/xdg_data
Override all XDG paths when running tests This is far preferable to the per-test XDG overrides that we may or may not remember to add the next time around. Also rename all the directories so it is clear via which variable a file made its way into that path.
2020-10-26 02:57:45 +00:00
XDG_CONFIG_HOME=${CMAKE_CURRENT_BINARY_DIR}/test/xdg_config
Correct test environment setup XDG_RUNTIME_DIR was misspelled. XDG_DATA_DIRS could affect tests. Remove it since it's used read-only.
2021-04-20 22:43:28 +00:00
XDG_RUNTIME_DIR=${CMAKE_CURRENT_BINARY_DIR}/test/xdg_runtime
Override all XDG paths when running tests This is far preferable to the per-test XDG overrides that we may or may not remember to add the next time around. Also rename all the directories so it is clear via which variable a file made its way into that path.
2020-10-26 02:57:45 +00:00
./fish_tests
Allow running test suites independently This makes test_low_level, test_interactive, test_invocation, and test_fishscript independent. This allows running a smaller subset of tests. To prevent all tests running in parallel, we also have new targets serial_test_low_level, serial_test_interactive, etc. which have the dependency chain that enforces serial execution.
2019-06-22 21:17:10 +00:00
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
DEPENDS fish_tests
USES_TERMINAL)
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
add_custom_target(${TESTTYPE}_fishscript
Override all XDG paths when running tests This is far preferable to the per-test XDG overrides that we may or may not remember to add the next time around. Also rename all the directories so it is clear via which variable a file made its way into that path.
2020-10-26 02:57:45 +00:00
COMMAND
cd tests &&
Tests: Set $XDG_DATA_DIRS to empty instead of unsetting NetBSD's `env` doesn't have `-u` and this should be harmless.
2021-05-18 10:24:24 +00:00
env XDG_DATA_DIRS=
Correct test environment setup XDG_RUNTIME_DIR was misspelled. XDG_DATA_DIRS could affect tests. Remove it since it's used read-only.
2021-04-20 22:43:28 +00:00
XDG_DATA_HOME=${CMAKE_CURRENT_BINARY_DIR}/test/xdg_data
Override all XDG paths when running tests This is far preferable to the per-test XDG overrides that we may or may not remember to add the next time around. Also rename all the directories so it is clear via which variable a file made its way into that path.
2020-10-26 02:57:45 +00:00
XDG_CONFIG_HOME=${CMAKE_CURRENT_BINARY_DIR}/test/xdg_config
Correct test environment setup XDG_RUNTIME_DIR was misspelled. XDG_DATA_DIRS could affect tests. Remove it since it's used read-only.
2021-04-20 22:43:28 +00:00
XDG_RUNTIME_DIR=${CMAKE_CURRENT_BINARY_DIR}/test/xdg_runtime
Override all XDG paths when running tests This is far preferable to the per-test XDG overrides that we may or may not remember to add the next time around. Also rename all the directories so it is clear via which variable a file made its way into that path.
2020-10-26 02:57:45 +00:00
${TEST_ROOT_DIR}/bin/fish test.fish
Allow running test suites independently This makes test_low_level, test_interactive, test_invocation, and test_fishscript independent. This allows running a smaller subset of tests. To prevent all tests running in parallel, we also have new targets serial_test_low_level, serial_test_interactive, etc. which have the dependency chain that enforces serial execution.
2019-06-22 21:17:10 +00:00
DEPENDS test_prep
USES_TERMINAL)
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
add_custom_target(${TESTTYPE}_interactive
Override all XDG paths when running tests This is far preferable to the per-test XDG overrides that we may or may not remember to add the next time around. Also rename all the directories so it is clear via which variable a file made its way into that path.
2020-10-26 02:57:45 +00:00
COMMAND cd tests &&
Tests: Set $XDG_DATA_DIRS to empty instead of unsetting NetBSD's `env` doesn't have `-u` and this should be harmless.
2021-05-18 10:24:24 +00:00
env XDG_DATA_DIRS=
Correct test environment setup XDG_RUNTIME_DIR was misspelled. XDG_DATA_DIRS could affect tests. Remove it since it's used read-only.
2021-04-20 22:43:28 +00:00
XDG_DATA_HOME=${CMAKE_CURRENT_BINARY_DIR}/test/xdg_data
Override all XDG paths when running tests This is far preferable to the per-test XDG overrides that we may or may not remember to add the next time around. Also rename all the directories so it is clear via which variable a file made its way into that path.
2020-10-26 02:57:45 +00:00
XDG_CONFIG_HOME=${CMAKE_CURRENT_BINARY_DIR}/test/xdg_config
Correct test environment setup XDG_RUNTIME_DIR was misspelled. XDG_DATA_DIRS could affect tests. Remove it since it's used read-only.
2021-04-20 22:43:28 +00:00
XDG_RUNTIME_DIR=${CMAKE_CURRENT_BINARY_DIR}/test/xdg_runtime
Override all XDG paths when running tests This is far preferable to the per-test XDG overrides that we may or may not remember to add the next time around. Also rename all the directories so it is clear via which variable a file made its way into that path.
2020-10-26 02:57:45 +00:00
${TEST_ROOT_DIR}/bin/fish interactive.fish
Allow running test suites independently This makes test_low_level, test_interactive, test_invocation, and test_fishscript independent. This allows running a smaller subset of tests. To prevent all tests running in parallel, we also have new targets serial_test_low_level, serial_test_interactive, etc. which have the dependency chain that enforces serial execution.
2019-06-22 21:17:10 +00:00
DEPENDS test_prep
USES_TERMINAL)
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
endforeach(TESTTYPE)
Allow running test suites independently This makes test_low_level, test_interactive, test_invocation, and test_fishscript independent. This allows running a smaller subset of tests. To prevent all tests running in parallel, we also have new targets serial_test_low_level, serial_test_interactive, etc. which have the dependency chain that enforces serial execution.
2019-06-22 21:17:10 +00:00
# Now add a dependency chain between the serial versions.
# This ensures they run in order.
Fix parallel build race condition for test targets When executing “make test -jX” (with X > 1) to build and run tests in a build directory, there is a race condition between the serial_test_low_level target and the test_prep target (a dependency of serial_test_fishscript and serial_test_interactive). As far as I can tell, these events happen in a serial build scenario (“make test” with the “Unix Makefiles” CMake generator): 1. The fish_tests binary is built and executed. 2. The test_prep target (a dependency of serial_test_fishscript) cleans up test directories. 3. Tests in test.fish are executed. In a parallel build scenario, this often happens: 1. Build of the fish_tests binary is started. 2. The test_prep target cleans up test directories. 3. Build of the fish_tests binary is finished. 4. Execution of the fish_tests binary starts. 5. Execution of the fish_tests binary finishes. 6. Tests in test.fish are executed. However, if building the fish_tests binary is fast enough but not instant (e.g. when using ccache), this can happen: 1. Build of the fish_tests binary is started. 2. Build of the fish_tests binary is finished. 3. Execution of the fish_tests binary starts. 4. The test_prep target cleans up test directories. 5. fish_tests tests that depend on said test directories may, depending on timing, fail because they are wiped by test_prep. Fix this by making test_prep a dependency of serial_test_low_level so that test_prep can’t interfere with fish_tests execution.
2021-03-08 21:11:47 +00:00
add_dependencies(serial_test_low_level test_prep)
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
add_dependencies(serial_test_fishscript serial_test_low_level)
add_dependencies(serial_test_interactive serial_test_fishscript)
Add support for running high-level tests under CMake This adds support in Tests.cmake for running the script and other "high level" tests, in addition to the unit tests fish_tests.
2017-10-05 04:34:48 +00:00
Teach cmake to run expect tests Fixes #4794
2018-07-21 22:43:31 +00:00
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
add_custom_target(serial_test_high_level
Remove invocation tests These are now all performed by littlecheck, so there's no need for the entire target anymore.
2019-06-25 20:03:49 +00:00
DEPENDS serial_test_interactive serial_test_fishscript)
Allow running test suites independently This makes test_low_level, test_interactive, test_invocation, and test_fishscript independent. This allows running a smaller subset of tests. To prevent all tests running in parallel, we also have new targets serial_test_low_level, serial_test_interactive, etc. which have the dependency chain that enforces serial execution.
2019-06-22 21:17:10 +00:00
# Create the 'test' target.
# Set a policy so CMake stops complaining about the name 'test'.
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
cmake_policy(PUSH)
cmake: only define test policy on old versions CMP0037 only reserves the test name if CTest is included on newer versions of CMake. This commit fixes a build warning.
2019-11-01 12:58:35 +00:00
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
if(${CMAKE_VERSION} VERSION_LESS 3.11.0 AND POLICY CMP0037)
cmake_policy(SET CMP0037 OLD)
endif()
add_custom_target(test)
cmake_policy(POP)
add_dependencies(test serial_test_high_level)
Improve IDE grouping in the CMake build This adds intelligent groups and hides unused files when generating IDE projects (Xcode, CLion, etc) in the CMake build.
2017-10-05 04:40:58 +00:00
# Group test targets into a TestTargets folder
Use lowercase CMake function names This is a best practice to distinguish them from variables.
2020-03-14 23:11:35 +00:00
set_property(TARGET test tests_dir
Allow running test suites independently This makes test_low_level, test_interactive, test_invocation, and test_fishscript independent. This allows running a smaller subset of tests. To prevent all tests running in parallel, we also have new targets serial_test_low_level, serial_test_interactive, etc. which have the dependency chain that enforces serial execution.
2019-06-22 21:17:10 +00:00
test_low_level
test_fishscript
test_interactive
Remove invocation tests These are now all performed by littlecheck, so there's no need for the entire target anymore.
2019-06-25 20:03:49 +00:00
test_fishscript test_prep
[cmake] make build_lexicon_filter part of DocTargets only
2018-01-14 13:14:04 +00:00
tests_buildroot_target
Allow running test suites independently This makes test_low_level, test_interactive, test_invocation, and test_fishscript independent. This allows running a smaller subset of tests. To prevent all tests running in parallel, we also have new targets serial_test_low_level, serial_test_interactive, etc. which have the dependency chain that enforces serial execution.
2019-06-22 21:17:10 +00:00
serial_test_high_level
serial_test_low_level
serial_test_fishscript
serial_test_interactive
Improve IDE grouping in the CMake build This adds intelligent groups and hides unused files when generating IDE projects (Xcode, CLion, etc) in the CMake build.
2017-10-05 04:40:58 +00:00
symlink_functions
PROPERTY FOLDER cmake/TestTargets)
Reference in a new issue Copy permalink