Changelog

[1.1.24] - 2021-03-04

• Improving UTF documentation(DSTR-120)
  • Updated "Actions outside of tests" section of user manual. Added flow charts to clarify execution order for SetUp/TearDown, TestActions, and complete flow (DSTR-121).
  • Fixed accepted values for scriptingBackend argument to be string literals instead of int values (DSTR-122).
  • Fixed possible values of ResultState to be Passed, Failed, Skipped, Inconclusive, plus labels instead of Success and Failure (DSTR-125).
  • Added NUNit version information (DSTR-130).
  • Added namespace information for LogAsset in user manual (DSTR-124).
  • Added instructions for creating additional sets of tests (DSTR-129).
  • Added information on testResults XML output format and exit codes (DSTR-131).
  • Updated description of testPlatform command line argument to clarify accepted values and their meaning (DSTR-123).
• Reduce time taken by filtering operations when only a subset of tests is run.
• Reduced the time taken to rebuild the test tree and to scan for assets a test created but did not delete.
• Reduce the per-test overhead of running tests in the editor.
• Added profiler markers around test setup, teardown, and execution.
• Fixed unstable timeout bug (DSTR-21).

[1.1.23] - 2021-01-21

• Improving UTF documentation(DSTR-120)
  • Updated "Actions outside of tests" section of user manual. Added flow charts to clarify execution order for SetUp/TearDown, TestActions, and complete flow (DSTR-121).
  • Fixed accepted values for scriptingBackend argument to be string literals instead of int values (DSTR-122).
  • Fixed possible values of ResultState to be Passed, Failed, Skipped, Inconclusive, plus labels instead of Success and Failure (DSTR-125).
  • Added NUNit version information (DSTR-130).
  • Added namespace information for LogAsset in user manual (DSTR-124).
  • Added instructions for creating additional sets of tests (DSTR-129).
  • Added information on testResults XML output format and exit codes (DSTR-131).
  • Updated description of testPlatform command line argument to clarify accepted values and their meaning (DSTR-123).

[1.1.22] - 2021-01-21

• Fixed issue where test result of an explicit test was set to skipped in case it was passing and running from command line with testfilter set to the explicit test (DS-1236).
• Fixed an issue where tests located in assemblies that did not directly reference any test assemblies were not included (DSTR-30).
• Fixed an issue where UnitySetup methods were incorrectly being rerun when entering playmode, rather than being skipped (DSTR-68).
• Internal: Remove ##utp message AssemblyCompilationErrors (DS-1277)
• Fixed issue where if the timeout was exceeded in SetUp the timeout exception was not thrown(DSTR-21).
• Removed ability to Enable playmode tests for all assemblies from the TestRunner UI, since it is a deprecated behavior. It enforces to use of assembly definition files (DSTR-45).
• Fixed typo in LogAssert.cs documentation.

[1.1.21] - 2020-12-04

• Fixed issue where test result of an explicit test was set to skipped in case it was passing and running from command line with testfilter set to the explicit test (DS-1236).
• Fixed an issue where tests located in assemblies that did not directly reference any test assemblies were not included (DSTR-30).
• Fixed an issue where UnitySetup methods were incorrectly being rerun when entering playmode, rather than being skipped (DSTR-68).
• Internal: Remove ##utp message AssemblyCompilationErrors (ds-1277)
• Fixed issue where if the timeout was exceeded in SetUp the timeout exception was not thrown(DSTR-21).
• Removed ability to Enable playmode tests for all assemblies from the TestRunner UI, since it is a deprecated behavior. It enforces to use of assembly definition files (DSTR-45).

[1.1.20] - 2020-12-04

• The logscope is now available in OneTimeTearDown.
• Fixed an issue where failing tests would not result in the correct exit code if a domain reload happens after the test has run (DS-1304).
• If a player build fails, the test specific build settings should be cleaned up and the original values restored as intended (DS-1001).
• Added better error message when using TestRunCallbackAttribute and the implementation is stripped away (DS-454).
• Fixed an issue where the test results xml would have a zero end-time for tests executed before a domain reload (DSTR-63).
• Fixed OpenSource in case of a Test in a nested class (DSTR-6)
• UnityTests with a domain reload now works correctly in combination with Retry and Repeat attributes (DS-428).
• Fixed OpenSource in case of Tests located inside a package (DS-432)

[1.1.19] - 2020-11-17

• Command line runs with an inconclusive test result now exit with exit code 2 (case DS-951).
• Fixed timeout during UnitySetUp which caoused test to pass instead of failing due to wrong time format.
• Timeout exeption thrown when timeout time is exeded in the UnitySetup when using WaitForSeconds(n).
• Updating com.unity.ext.nunit version
• Method marked with UnityTest that are not returning IEnumerator is now giving a proper error (DS-1059).

[1.1.18] - 2020-10-07

• Fixed issue of timeout during UnitySetUp which wasn't detected and allowed the test to pass instead of failing (case DSTR-21)

[1.1.17] - 2020-10-05

• Fixed an issue where the WaitForDomainReload yield instruction would sometimes let the test continue for one frame before the domain reload.
• Added support for negation in filters using !. E.g. !CategoryToExclude.
• Fixed an issue where if the first test enters PlayMode from UnitySetup then the test body will not run on consecutive runs (case 1260901).
• Clear Results button clears the test results in the GUI (DSTR-16)
• Improved UI in Test Runner window, added new options:
  • Run Selected Tests in player
  • Build/Export project with all tests in player
  • Build/Export project with selected tests in player
• Fixed issue on loading EditMode or Playmode test tree in the wrong tab when switching between tabs when TestRunner is loading (DS-865)

[1.1.16] - 2020-07-09

• Follow up on fix when UTF picks up on outdated compilation errors

[1.1.15] - 2020-07-02

• Fixed an issue where an exception is thrown on getting the enumerator of a UnityTest would result in stopping the test run instead of failing it (case 1212000).
• Including a trailing semi-colon in a testName filter no longer results in all tests being run (case 1171200).
• Fixed and issue when Unity Test Framework exits editor on an outdated script compilation error (during api updates)

[1.1.14] - 2020-04-03

• Added the 'assemblyNames' command line argument for filtering on the assembly level.
• The dll and project level of the tree view should now correctly show the results when running tests in a player (case 1197026).
• Optimize usage of player connection when transfering test results (case 1229200).
• Ignore internal test framework tests assertions (case 1206961).

[1.1.13] - 2020-03-16

• Fixed an issue where a combination of Entering / Exiting playmode and recompiling scripts would result in the test run repeating (case 1213958).
• Fixed a regression from 1.1.12 where prefabs left in the scene would be cleaned up to aggressively.
• Fixed Test execution timed out. No activity received from the player in 600 seconds error when player is not supposed to start (case 1225147)

[1.1.12] - 2020-03-02

• Now 'Open error line' for a failed UTF test does not throw exceptions for corrupted testable pdb in Editor release mode (case 1118259)
• Fixed an issue where running a test fixture would also run other fixtures with the same full name (namespace plus classname) in other assemblies (case 1197385).
• Running tests with the same full name, with a domain reload inbetween, will no longer fail to initialize the fixture of the second class (case 1205240).
• Running a playmode tests with "Maximize on Play" will now correctly show the result of the tests in the test runner window (case 1014908).
• Fixed an issue where leaving a game object in a scene with a DontSaveInEditor hideFlags would result in an error on cleanup (case 1136883).
• Now ITestPlayerBuildModifier.ModifyOptions is called as expected when running tests on a device (case 1213845)

[1.1.11] - 2020-01-16

• Fixed test runner dlls got included into player build (case 1211624)
• Passing a non-full-path of XML file for -testResults in Unity Batchmode issue resolved, now passing "result.xml" creates the result file in the project file directory (case 959078)
• Respect Script Debugging build setting when running tests

[1.1.10] - 2019-12-19

• Introduced PostSuccessfulLaunchAction callback
• Fixed an issue where canceling a UnityTest while it was running would incorrectly mark it as passed instead of canceled.
• Added command line argument for running tests synchronously.
• The test search bar now handles null values correctly.
• The test output pane now retains its size on domain reloads.

[1.1.9] - 2019-12-12

• Rolled back refactoring to the test run system, as it caused issues in some corner cases.

[1.1.8] - 2019-11-15

• Ensured that a resumed test run is continued instantly.

[1.1.7] - 2019-11-14

• Fixed an issue with test runs after domain reload.

[1.1.6] - 2019-11-12

• Building a player for test will no longer look in unrelated assemblies for relevant attributes.

[1.1.5] - 2019-10-23

• Fixed a regression to synchronous runs introduced in 1.1.4.

[1.1.4] - 2019-10-15

• Running tests in batch mode now correctly returns error code 3 (RunError) when a timeout or a build error occurs.
• Fixed an issue where a test run in a player would time out, if the player takes longer than 10 minutes to run.
• Added command line argument and api setting for specifying custom heartbeat timeout for running on players.

[1.1.3] - 2019-09-23

• Fixed a regression where tests in a player would report a timeout after a test run is finished.
• Made it possible for the ui to change its test items when the test tree changes without script compilation.
• Added synchronous runs as an option to the TestRunnerApi.

[1.1.2] - 2019-09-11

• Fixed an issue where Run Selected would run all tests in the category, if a category filter was selected, regardless of what tests were selected.
• Unsupported attributes used in UnityTests now give an explicit error.
• Added support for the Repeat and Retry attributes in UnityTests (case 1131940).
• Tests with a explicit timeout higher than 10 minutes, no longer times out after running longer than 10 minutes when running from command line (case 1125991).
• Fixed a performance regression in the test runner api result reporting, introduced in 2018.3 (case 1109865).
• Fixed an issue where parameterized test fixtures would not run if selected in the test tree (case 1092244).
• Pressing Clear Results now also correctly clears the counters on the test list (case 1181763).
• Prebuild setup now handles errors logged with Debug.LogError and stops the run if any is logged (case 1115240). It now also supports LogAssert.Expect.

[1.1.1] - 2019-08-07

• Tests retrieved as a test list with the test runner api incorrectly showed both mode as their TestMode.
• Fixed a compatibility issue with running tests from rider.

[1.1.0] - 2019-07-30

• Introduced the TestRunnerApi for running tests programmatically from elsewhere inside the Editor.
• Introduced yield instructions for recompiling scripts and awaiting a domain reload in Edit Mode tests.
• Added a button to the Test Runner UI for clearing the results.

[1.0.18] - 2019-07-15

• Included new full documentation of the test framework.

[1.0.17] - 2019-07-11

• Fixed an issue where the Test Runner window wouldn’t frame selected items after search filter is cleared.
• Fixed a regression where playmode test application on the IOS platform would not quit after the tests are done.

[1.0.16] - 2019-06-20

• Fixed an issue where the Test Runner window popped out if it was docked, or if something else was docked next to it, when re-opened (case 1158961)
• Fixed a regression where the running standalone playmode tests from the ui would result in an error.

[1.0.15] - 2019-06-18

• Added new [TestMustExpectAllLogs] attribute, which automatically does LogAssert.NoUnexpectedReceived() at the end of affected tests. See docs for this attribute for more info on usage.
• Fixed a regression where no tests would be run if multiple filters are specified. E.g. selecting both a whole assembly and an individual test in the ui.
• Fixed an issue where performing Run Selected on a selected assembly would run all assemblies.
• Introduced the capability to do a split build and run, when running playmode tests on standalone devices.
• Fixed an error in ConditionalIgnore, if the condition were not set.

[1.0.14] - 2019-05-27

• Fixed issue preventing scene creation in IPrebuildSetup.Setup callback when running standalone playmode tests.
• Fixed an issue where test assemblies would sometimes not be ordered alphabetically.
• Added module references to the package for the required modules: imgui and jsonserialize.
• Added a ConditionalIgnore attribute to help ignoring tests only under specific conditions.
• Fixed a typo in the player test window (case 1148671).

[1.0.13] - 2019-05-07

• Fixed a regression where results from the player would no longer update correctly in the UI (case 1151147).

[1.0.12] - 2019-04-16

• Added specific unity release to the package information.

[1.0.11] - 2019-04-10

• Fixed a regression from 1.0.10 where test-started events were triggered multiple times after a domain reload.

[1.0.10] - 2019-04-08

• Fixed an issue where test-started events would not be fired correctly after a test performing a domain reload (case 1141530).
• The UI should correctly run tests inside a nested class, when that class is selected.
• All actions should now correctly display a prefix when reporting test result. E.g. "TearDown :".
• Errors logged with Debug.LogError in TearDowns now append the error, rather than overwriting the existing result (case 1114306).
• Incorrect implementations of IWrapTestMethod and IWrapSetUpTearDown now gives a meaningful error.
• Fixed a regression where the Test Framework would run TearDown in a base class before the inheriting class (case 1142553).
• Fixed a regression introduced in 1.0.9 where tests with the Explicit attribute could no longer be executed.

[1.0.9] - 2019-03-27

• Fixed an issue where a corrupt instance of the test runner window would block for a new being opened.
• Added the required modules to the list of package requirements.
• Fixed an issue where errors would happen if the test filter ui was clicked before the ui is done loading.
• Fix selecting items with duplicate names in test hierarchy of Test Runner window (case 987587).
• Fixed RecompileScripts instruction which we use in tests (case 1128994).
• Fixed an issue where using multiple filters on tests would sometimes give an incorrect result.

[1.0.7] - 2019-03-12

This is the first release of Unity Package com.unity.test-framework.

• Migrated the test-framework from the current extension in unity.