In this tutorial, I’ll show how to modify engine to be able to compile shaders manually. Users without high-end PC (like me) can find this mod really useful. Tested on UE4.11.2 and UE4.12.5.
Goals are:

  • Give user an option to compile automatically or by hand
  • In manual option: Inform user that shaders are waiting to be compiled
  • Add editor toolbar icon to start it manually
  • Bind keyboard shortcut to it

tl;dr version

GithubFork: UE4.12.5
PullRequest: Here
To access this fork you have to ba a member of EpicGames organization.

Introduction

Let’s begin with some valuable context. Out starting point is FShaderCompilingManager class (ShaderCompiler.h)┬áThis class is responsible for asynchronous and parallel shader compilation.

It’s worth to mention that instance of this is owned globally:

extern ENGINE_API FShaderCompilingManager* GShaderCompilingManager;

And to access it you have to:

#include "ShaderCompiler.h"

It’s lifecycle starts in FEngineLoop::PreInit method (LaunchEngineLoop.cpp). Global instance is created there.

check(!GShaderCompilingManager);
GShaderCompilingManager = new FShaderCompilingManager();

It’s lifecycle ends in FEngineLoop::AppPreExit method.

if ( GShaderCompilingManager )
{
    GShaderCompilingManager->Shutdown();

    delete GShaderCompilingManager;
    GShaderCompilingManager = nullptr;
}

We don’t want to mess with that since we’ll use this class the same way as it’s used now.

Ok, now let’s see how shader compiler works inside. There are 2 important parts:

  • Jobs queue – TArray<FShaderCommonCompileJob*> CompileQueue;
  • Runnable TScopedPointer Thread; that holds FRunnableThread

Their workflow looks like this:

  • External systems push jobs into CompileQueue array, this array is shared between threads.
  • Runnable thread takes jobs from queue and starts new processes ShaderCompileWorker.exe

This loop is turned on all the time (from FEngineLoop::PreInit to FEngineLoop::AppPreExit). Each time user change something in material or open new map, jobs are pushed to queue and external processes are started. At this point I found that it’s bad idea to:

  • Block queue from receiving jobs. We’ll miss information that something has to be rebuild.
  • Change shader compiler manager lifecycle. A lot of engine subsystems assume that it always exist (nullptr exception).
  • Disable runnable from automatic start. Not only editor uses this logic (commandlets, commandline, automation, cooking etc.)

Shader Compiler Manager

That’s why I decided to modify lifecycle of the runnable thread. In this situation compiler manager can accept new jobs, but has to wait until runnable thread will be initiated (always on engine start). Next cycle can be initiated manually.

FShaderCompilingManager (ShaderCompiler.h)

Add public member variables:

bool bThreadStarted = false;
bool bShutdownOnFinish = false;

Add public member function:

ENGINE_API void Initiate();

Change ShouldDisplayCompilingNotification function from:

return NumOutstandingJobs > 80;

to:

return NumOutstandingJobs > 0;

Change IsCompiling function from:

return NumOutstandingJobs > 0 || PendingFinalizeShaderMaps.Num() > 0;

to:

return bThreadStarted && (NumOutstandingJobs > 0 || PendingFinalizeShaderMaps.Num() > 0);

bThreadStarted – Informs if runnable thread is doing his work.
bShutdownOnFinish – Controls if runnable thread has to stop when all jobs are done.

FShaderCompilingManager (ShaderCompiler.cpp)
Modify methods BlockOnShaderMapCompletion and BlockOnAllShaderMapCompletion by adding line at the beginning:

if (!bThreadStarted) { return; }

These functions are blocking editor until all jobs are done. Now it can freeze application forever if thread is not started.

Add line at the end of Shutdown method:

bThreadStarted = false;

Add member function :

void FShaderCompilingManager::Initiate()
{
    Thread->StartThread();
    bThreadStarted = true;
}

Modify ProcessAsyncResults and put this:

if (bShutdownOnFinish && bThreadStarted && NumOutstandingJobs == 0 && PendingFinalizeShaderMaps.Num() == 0)
{
    Shutdown();
}

Right below:

if (bBlockOnGlobalShaderCompletion)
{ ... }
else if (NumPendingShaderMaps - PendingFinalizeShaderMaps.Num() > 0)
{ ... }

This condition will turn worker off when all jobs are finished.

In constructor, replace:

Thread->StartThread();

With:

Initiate();

We still want to start thread fully operational.

At this point:

  • Initiate() – Starts thread
  • Shutdown() – Stops thread

Editor Extension

Let’s begin with extending editor toolbar. 2 new controls are required:

  • Button – Compile shaders – That will initiate thread after it was previously shut down.
  • Checkbox – Only manual mode – That will shutdown thread after all jobs are done.

2016-07-26_18h00_58
FLevelEditorCommands (LevelEditorActions.h)
Add public member variables:

TSharedPtr< FUICommandInfo > CompileShaders;
TSharedPtr< FUICommandInfo > CompileShadersManualOnly;

Each menu button is separate object instance.

FLevelEditorActionCallbacks (LevelEditorActions.h)
Add public member function definitions:

static void CompileShaders_Execute();
static void CompileShadersManualOnly_Toggle();
static bool CompileShadersManualOnly_IsChecked();

Each menu button have callbacks bound.

LevelEditorActions.cpp
Include:

#include "ShaderCompiler.h"

To have access to global GShaderCompilingManager.

FLevelEditorActionCallbacks (LevelEditorActions.cpp)
Add callback bodies:

void FLevelEditorActionCallbacks::CompileShaders_Execute()
{
    if (GShaderCompilingManager && !GShaderCompilingManager->bThreadStarted)
    {
        GShaderCompilingManager->Initiate();
    }
}

void FLevelEditorActionCallbacks::CompileShadersManualOnly_Toggle()
{
    if (GShaderCompilingManager)
    {
        GShaderCompilingManager->bShutdownOnFinish = !GShaderCompilingManager->bShutdownOnFinish;
    }
}

bool FLevelEditorActionCallbacks::CompileShadersManualOnly_IsChecked()
{
    return GShaderCompilingManager ? GShaderCompilingManager->bShutdownOnFinish : false;
}

Add code RegisterCommands function:

UI_COMMAND( CompileShaders, "Compile Shaders", "Automatic shaders compilation", EUserInterfaceActionType::Button, FInputChord());
UI_COMMAND( CompileShadersManualOnly, "Disable auto compilation on que empty", "Disables automatic shaders compilation on finish", EUserInterfaceActionType::ToggleButton, FInputChord());

To instantiate action objects.

FLevelEditorModule (LevelEditor.cpp)
Add to BindGlobalLevelEditorCommands function:

ActionList.MapAction(
    Commands.CompileShaders,
    FExecuteAction::CreateStatic(&FLevelEditorActionCallbacks::CompileShaders_Execute));

ActionList.MapAction(
    Commands.CompileShadersManualOnly,
    FExecuteAction::CreateStatic(&FLevelEditorActionCallbacks::CompileShadersManualOnly_Toggle),
    FCanExecuteAction(),
    FIsActionChecked::CreateStatic(&FLevelEditorActionCallbacks::CompileShadersManualOnly_IsChecked));

To bind action objects with callbacks.

FLevelEditorToolBar (LevelEditorToolBar.cpp)

Add at the end of GenerateBuildMenuContent method:

MenuBuilder.BeginSection("LevelEditorShaders", LOCTEXT("ShadersHeading", "Shaders"));
{
    MenuBuilder.AddMenuEntry(FLevelEditorCommands::Get().CompileShaders, NAME_None, LOCTEXT("CompileShaders", "Compile shaders"));
    MenuBuilder.AddMenuEntry(FLevelEditorCommands::Get().CompileShadersManualOnly, NAME_None, LOCTEXT("CompileShadersManualOnly", "Manual compilation only"));
}
MenuBuilder.EndSection();

To add actions to toolbar menu.

FSlateEditorStyle::FStyle (SlateEditorStyle.cpp)
Add to SetupLevelEditorStyle method:

Set( "LevelEditor.CompileShaders", new IMAGE_BRUSH("Icons/icon_build_40x", Icon40x40) );

To define an icon of “Compile Shaders” button.

FShaderCompilingNotificationImpl (ShaderCompilingNotification.cpp)
Set SetNotificationText function body to:

FFormatNamedArguments Args;
Args.Add(TEXT("ShaderJobs"), FText::AsNumber(GShaderCompilingManager->GetNumRemainingJobs()));
FText ProgressMessage;

if (GShaderCompilingManager->bThreadStarted)
{
    ProgressMessage = FText::Format(NSLOCTEXT("ShaderCompile", "ShaderCompileInProgressFormat", "Compiling Shaders ({ShaderJobs})"), Args);
}
else
{
    ProgressMessage = FText::Format(NSLOCTEXT("ShaderCompileWait", "ShaderCompileInWaitFormat", "Shaders in queue ({ShaderJobs})"), Args);
}

InNotificationItem->SetText(ProgressMessage);

To define behavior of notification popup.
2016-07-26_18h18_51
2016-07-26_18h18_29

Enjoy ! :)