Author: https://twitter.com/ldl19691031

This is a brief analysis of the rendering pipeline of unreal engine 5. This article is based on the capture result by RenderDoc. As I'm not a developer of Epic, this article may contain mistakes. And the contents are NOT the true ideas of Epic but just my own understanding. If I mislead someone, please forgive me. And if you have a better explanation or want to point out some mistakes, please make comments.

(I'm not a English native speaker so I may make some language mistakes)

Prepare

In this part, I will explain how to capture the debug frame. If you are familiar with the usage of Render Doc in UE4, you can skip this part.

In order to capture one frame of UE5 with debug info, you need to:

Download Render Doc : https://renderdoc.org/
Enable Render Doc plugin in UE5's plugin settings After you enable the plugin, you need to restart the engine.

Modify the ConsoleVariables.ini under the Engine/Config folder. In my computer, the path is C:\Program Files\Epic Games\UE_5.0EA\Engine\Config. You need to remove the ';' before those two lines:

; Uncomment when running with a graphical debugger (but not when profiling)
r.Shaders.Optimize=0
; When this is enabled, shaders will have extra debugging info. This could change patch sizes, uniqueness, etc and will recompile the shaders
r.Shaders.KeepDebugInfo=1

Create a short cut or a bat file to start the engine. I create this short cut:
```
"C:\\Program Files\\Epic Games\\UE_5.0EA\\Engine\\Binaries\\Win64\\UnrealEditor.exe" "C:\\Users\\LUO\\Documents\\Unreal Projects\\UE5TestProject\\UE5TestProject.uproject" -d3ddebug
```
You need to add two parameters: one is the path to your test project, another one is '-d3ddebug'.

Then, use this shortcut or bat file to start your engine. This may take a long time if this is your first time enabling shader to debug flags.
Capture one frame
1. You can change some items in the default scene. I recommend to add at least one Nanite mesh.
2. Press this button to capture one frame
3. The Render Doc will be automatically opened, and you can load that capture data.

Render Pipeline Minimap

https://kroki.io/plantuml/svg/eNpdT80KwjAMvu8pctSDL7DdpjgEkeLYA4Q124qzLU2L-PaG_cjYLfl-E44YYpY3XmMkqFQDdUuWiiw_p3GEm-WItiWGwzUkjukNreDG9scig86FVwb5hXwcQAXyyLzAgD0aK-QDrZHknYasnsyLt0xdRwGehPor1XfTD3EWBzllVpbIBGpybwrW_A37z652sfWA2n1ktZqCvLA2zaNyHE8-OHmWJ0RkPyIKYRc=

Here is a brief map of the whole pipeline.

Logically the Nanite rendering is not related to the traditional deferred rendering passes.

So, as a summary, the new UE5 pipeline contains those new things:

A new GPU-driven culling system
Nanite
Lumen lighting.

I will provide my guess about those three points in this article.

GPU-Driven Culling

Scene Instance Data Structure

In order to achieve GPU-Driven Culling, UE5 needs to save the scene's representation on the GPU side. The core data structure we are dealing with is 'GPUScene_InstanceData', which contains data in this structure:

struct FInstanceSceneData
{
	float4x4 LocalToWorld;
	float4x4 PrevLocalToWorld;
	float4x4 WorldToLocal;
	float4   NonUniformScale;
	float4   InvNonUniformScaleAndDeterminantSign;
	float3   LocalBoundsCenter;
	uint     PrimitiveId;
	float3   LocalBoundsExtent;
	uint     LastUpdateSceneFrameNumber;
	uint     NaniteRuntimeResourceID;
	uint     NaniteHierarchyOffset;
	bool     NaniteHasImposter;
	// TODO: bool     CastShadows;
	float    PerInstanceRandom;
	float4   LightMapAndShadowMapUVBias;
	bool     ValidInstance;
};

Prepare

Render Pipeline Minimap

GPU-Driven Culling

Scene Instance Data Structure

Data Updating