About a year ago Eleftherios Kokkinakis, the owner of RedSoup Studios which was developing Westwood…
Creating Unit Tests with the Automation System
In this post I’m going to show you how to use Unreal’s Automation System in order to create unit tests for your code. Unit tests are tests written by developers in order to test various aspects of their code and identify bugs or other unintended behavior before it happens.
In order to test out the Automation System, open up the .Build.cs file of your project and add “UnrealEd” and “Engine” on the dependecy modules list:
|
1 |
PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "UnrealEd", "Engine", "InputCore", "HeadMountedDisplay" }); |
Implementing a dummy class
For the sake of this post, I’m also adding a dummy class to test its behavior:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
struct FMathStruct { static int32 Add(int32 A, int32 B); static float Add(float A, float B); }; int32 FMathStruct::Add(int32 A, int32 B) { return A+B; } float FMathStruct::Add(float A, float B) { return A+B; } |
Creating the first unit test
We have created a class that we can test so go ahead and create a new class. On its source file add the following code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
#include "Tests/MathTests.h" #include "AutomationPost/TestClass.h" //Contains FMathStruct #include "Misc/AutomationTest.h" #if WITH_DEV_AUTOMATION_TESTS IMPLEMENT_SIMPLE_AUTOMATION_TEST(FMathStructTest, "MathTests",EAutomationTestFlags::ApplicationContextMask | EAutomationTestFlags::SmokeFilter) bool FMathStructTest::RunTest(const FString& Parameters) { //Addition tests { int32 ResultA = FMathStruct::Add(5,15); int32 ExpectedResultA = 20; float ResultB = FMathStruct::Add(3.5f,1.5f); float ExpectedResultB = 5.f; TestEqual(TEXT("Testing sum in integers"), ResultA, ExpectedResultA); TestEqual(TEXT("Testing sum in floats"),ResultB, ExpectedResultB); } return true; } #endif //WITH_DEV_AUTOMATION_TESTS |
The IMPLEMENT_SIMPLE_AUTOMATION_TEST macro defines the class which contains our test. The macro requires:
- The name of the class we want to define (in this case FMathStructTest)
- A string which defines tha category that this test will be assigned in the Editor
- Some flags regarding our test
For more information about the different flags you can use I would suggest to check out the AutomationTest.h file in the Engine’s source code. In this case, I have added the ApplicationContextMask flag which makes this test availabe in the Editor and the SmokeFilter which means that we expect this test to be completed quickly. In the provided link you can find out more about those flags so I would highly suggest to check it out!
Since we’re happy with the first unit test we have created, you can compile your code and open up the “Session Frontend” window from Window->Developer Tools. When you open up the Session Frontend you can click the Automation tab and you can locate the test we have created above:
If the test doesn’t appear on that window try clicking “Refresh Tests” or restarting your editor. Once you select the MathTests test you can click the Start Tests button and the editor will run your tests:
As we can see the results from the previous test passed. In case the result would fail, the editor would inform us about what went wrong. For example, if we change the expected result in the first addition on the code above from 20 to 19 we’re going to get the following result:
In this case I changed the expected value of the test from 20 to 19 and we can see that the editor informs us about why the test failed. Moreover, if you click on the red mathtests button in the SessionFrontend window the editor will display the error message of the output log to your window.
Creating another unit test
The previous unit test was pretty straightforward, however in real applications we may need to check various stuff inside our levels. For demonstration purposes, I created an automated process that verifies that we have a valid character placed inside the level we have currently open in the Editor:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
#include "Tests/LevelTests.h" #include "Misc/AutomationTest.h" #include "Tests/AutomationEditorCommon.h" #include "Editor.h" #include "GameFramework/Character.h" #include "Kismet/GameplayStatics.h" #include "AutomationPost/AutomationPostCharacter.h" #if WITH_DEV_AUTOMATION_TESTS IMPLEMENT_SIMPLE_AUTOMATION_TEST(FValidPlayerLevelTest, "TestCategory.LevelRelated.Check for Valid Players", EAutomationTestFlags::ApplicationContextMask | EAutomationTestFlags::EngineFilter) bool FValidPlayerLevelTest::RunTest(const FString& Parameters) { //Checking if there is at least one player placed in the world UWorld* World = GEditor->GetWorld(); { if (World) { ACharacter* PlayerChar = UGameplayStatics::GetPlayerCharacter(World,0); //For more tests check AutomationTest.h (lines 1347 - 1639) TestNotNull<ACharacter>(TEXT("Testing if there is a player actor in level"),PlayerChar); } } return true; } #endif //WITH_DEV_AUTOMATION_TESTS |
Once you compile your code the Automation Tab should detect the new test and place it in the corresponding category:
Creating Complex Tests
Sometimes we want to test complex stuff and we don’t know exactly the duration of tests. Fortunately, Unreal provides an easy to use way of creating latent tests that can run for some time, allowing us to test things out. To implement a complex test we need to:
- Define a new complex test
- Override the GetTests function
- Add the various tests in our complex test
Create a new class and in the header file add the following code in order to create a new test:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
#include "CoreMinimal.h" #include "Misc/AutomationTest.h" //For IAutomationLatentCommand /** * A command that gets completed after a random delay */ class FRandomDelayCommand : public IAutomationLatentCommand { private: /* Min delay in seconds */ float MinDelay; /* Max delay in seconds */ float MaxDelay; /* False by default, true when we have started the test */ bool bTestStarted; /* The actual time in seconds we're going to delay */ float Delay; /* Stores the time we have started our test */ FDateTime StartedTime; public: FRandomDelayCommand(float Min, float Max) : MinDelay(Min), MaxDelay(Max), bTestStarted(false), Delay(0.f) {} /** * Will execute each frame and will stop once we return true */ virtual bool Update() override; }; |
On the source file of our class, add the following code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 |
#include "Tests/ComplexTest.h" #include "Tests/AutomationEditorCommon.h" //for EngineAutomationTestUtilities #include "Tests/AutomationCommon.h" //for FWaitLatentCommand or other latent commands #include "FileHelpers.h" #if WITH_DEV_AUTOMATION_TESTS //Defining a complex test class named FComplexAutomationTest IMPLEMENT_COMPLEX_AUTOMATION_TEST(FComplexAutomationTest,"TestCategory.LevelRelated.A Complex Test",EAutomationTestFlags::ApplicationContextMask | EAutomationTestFlags::EngineFilter) //Once we define a complex test, we need to override the GetTests function in order to build various parameters for our Tests. //The OutTestCommands will be contained in the Parameters parameter of the RunTest function //The OutBeautifiedName will be used from the editor in the Automation tab as a name for each test void FComplexAutomationTest::GetTests(TArray<FString>& OutBeautifiedNames, TArray<FString>& OutTestCommands) const { TArray<FString> FileList; #if WITH_EDITOR FEditorFileUtils::FindAllPackageFiles(FileList); #else // Look directly on disk. Very slow! FPackageName::FindPackagesInDirectory(FileList, *FPaths::ProjectContentDir()); #endif // Iterate over all files, adding the ones with the map extension for (int32 FileIndex = 0; FileIndex < FileList.Num(); FileIndex++) { const FString& Filename = FileList[FileIndex]; //Get all files that have a .umap extension if (FPaths::GetExtension(Filename, true) == FPackageName::GetMapPackageExtension()) { //OutBeautifiedName contain names such as: ThirdPersonDefaultMap //OutTestCommands contain names such as: <CompletePathToYourProject>/ThirdPersonDefaultMap.umap OutBeautifiedNames.Add(FPaths::GetBaseFilename(Filename)); OutTestCommands.Add(Filename); } } } bool FComplexAutomationTest::RunTest(const FString& Parameters) { //Retrieve the map name FString MapName = Parameters; //Loads given map name FAutomationEditorCommonUtils::LoadMap(MapName); //Add a new command to our queue ADD_LATENT_AUTOMATION_COMMAND(FRandomDelayCommand(1.f,3.f)); return true; } #endif //WITH_DEV_AUTOMATION_TESTS bool FRandomDelayCommand::Update() { if (!bTestStarted) { Delay = FMath::RandRange(MinDelay, MaxDelay); GLog->Log("Started update!"); GLog->Log("Delay value:"+FString::SanitizeFloat(Delay)); StartedTime=FDateTime::Now(); bTestStarted = true; } else if((FDateTime::Now().GetSecond() - StartedTime.GetSecond())>=Delay) { GLog->Log("finished waiting latent task!"); GLog->Log("Elapsed time:"+FString::FromInt(FDateTime::Now().GetSecond() - StartedTime.GetSecond())); return true; } return false; } |
Once you compile your code and refresh your tests you will notice a new category and new tests in your Automation Tab:
To sum up, by defining a complex test we’re able to run several different tests on different maps. Once we have defined our tests, we can select the required maps we want to test out and run the tests in each one of them! This post was written using 4.26 version of the engine so depending on your version things may be slightly different.
In the next post, I’m going to show you how to run Functional Tests which simulate a user’s behavior when running our application so stay tuned! Thanks for reading and have a nice day!
Related Posts
This Post Has 0 Comments