Shaders

Overview The big picture

Shaders are the small programs the GPU runs for you: a vertex shader once per vertex, to work out where it lands on screen, and a fragment shader once per covered pixel, to work out its colour. RustyVulkan writes them in GLSL, the same C-like language OpenGL used.

If you last used OpenGL's fixed-function pipeline, think of the vertex shader as replacing the built-in matrix transform, and the fragment shader as replacing the built-in lighting and texture combination. You write the rules now. The vertex shader produces clip-space coordinates; the hardware clips triangles and turns them into fragments, interpolating values such as texture coordinates across each triangle. The fragment shader chooses each fragment's colour, which still has to pass depth testing and blending before reaching the image. GLSL looks much like C, with vector and matrix types such as vec3 and mat4 built in.

Compiled before the program runs

OpenGL took shader source text at run time and handed it to the driver's built-in compiler. Every vendor's compiler was different, so a shader that worked on one GPU could fail to compile, or behave differently, on another, and compiling cost time while the program was running.

Vulkan drivers don't accept GLSL at all. They take SPIR-V, a compact binary intermediate form defined by Khronos, much as a Java virtual machine takes bytecode rather than Java. Turning GLSL into SPIR-V is done ahead of time by a separate compiler. In this project that happens during cargo build.

How a shader gets its data

A shader can't call back into your program. Everything it needs has to be wired to it, and Vulkan offers three routes, each labelled with a number in the GLSL that the Rust side must match:

Data passed from the vertex shader to the fragment shader is matched by location too, and the fragment shader's output at location = 0 is the colour written to the image being rendered.

Skip to next overview →

The build script

The script starts with an explicit list of shader sources:

// This explicit list is also the shader manifest: adding a shader here makes Cargo watch it.
// Generated SPIR-V is deliberately not watched, otherwise writing it retriggers this script.
const SHADERS: &[&str] = &[
    "grass.vert",
    "grass.frag",
// ...
    "vertex_color.frag",
];

As the comment says, the list doubles as a manifest. For each entry, the script tells Cargo to rerun it if that file changes, works out the shader stage from the extension, and compiles:

let mut manifest = String::from("const EMBEDDED_SHADERS: &[(&str, &[u8])] = &[\n");
for shader_name in SHADERS {
    let path = shader_dir.join(shader_name);
    println!("cargo:rerun-if-changed={}", path.display());

    let shader_kind = match path.extension().and_then(|s| s.to_str()) {
        Some("vert") => Some(shaderc::ShaderKind::Vertex),
        Some("frag") => Some(shaderc::ShaderKind::Fragment),
        Some("geom") => Some(shaderc::ShaderKind::Geometry),
        Some("comp") => Some(shaderc::ShaderKind::Compute),
        Some("tesc") => Some(shaderc::ShaderKind::TessControl),
        Some("tese") => Some(shaderc::ShaderKind::TessEvaluation),
        _ => None,
    };

    let Some(shader_kind) = shader_kind else {
        continue;
    };
// ...
    let mut options = shaderc::CompileOptions::new().unwrap();
    options.set_target_env(shaderc::TargetEnv::Vulkan, 0);
    // This is the one shared interface include; reject other names rather than
    // silently allowing an untracked include to bypass Cargo's dependency list.
    options.set_include_callback(|name, _, _, _| {
        if name != "global.glsl" {
            return Err(format!("unknown shader include: {name}"));
        }
        let path = shader_dir.join(name);
        let content = fs::read_to_string(&path).map_err(|error| error.to_string())?;
        Ok(shaderc::ResolvedInclude {
            resolved_name: path.to_string_lossy().into_owned(),
            content,
        })
    });

    let binary_result = compiler
        .compile_into_spirv(
            &source,
            shader_kind,
            path.file_name().unwrap().to_str().unwrap(),
            "main",
            Some(&options),
        )
        .expect("📝 Shader compilation failed");

    fs::write(&spv_path, binary_result.as_binary_u8()).unwrap();

set_target_env(Vulkan, 0) selects Vulkan's flavour of GLSL, which adds things OpenGL GLSL lacks, such as descriptor set numbers and push constants. The 0 means the default target, Vulkan 1.0's SPIR-V, which is all these shaders need. (shaderc is Google's wrapper around glslang, Khronos's reference GLSL compiler.) The output goes to shaders/ under Cargo's OUT_DIR, inside target/. Failures use expect and panic!, which abort the build with the compiler's message. That's exactly right for a build script: a shader typo shows up as a build error, not a run-time surprise.

The script also generates a Rust table using include_bytes! for each compiled shader. ShaderStageInfo includes that table with include!(concat!(env!("OUT_DIR"), "/shader_assets.rs")). Both macros run during Rust compilation: the finished executable contains the SPIR-V and needs no shader files at run time. Add a shader to SHADERS to compile and embed it; after editing GLSL or the shared include, rebuild to update the executable.

Models and textures still come from external assets, so launch from the repository root or the packaged application as described in Chapter 1.

The same script also finds the Vulkan loader on macOS so the linked build of ash (Chapter 1) can link against it:

let library_dir = candidates
    .into_iter()
    .find(|directory| directory.join("libvulkan.dylib").is_file())
    .expect(
        "Vulkan loader not found. Install the macOS Vulkan SDK with System Global Installation enabled, or set VULKAN_SDK.",
    );

println!("cargo:rustc-link-search=native={}", library_dir.display());
println!("cargo:rustc-link-arg=-Wl,-rpath,{}", library_dir.display());

It tells the linker where libvulkan.dylib is, and adds an rpath so the finished program can find it again at run time.

The six lit fragment shaders include assets/shaders/global.glsl, which declares their common lights, cascade counts and global uniform block. The build script resolves this include and tells Cargo to watch it, so an edit recompiles the shaders. Each material keeps its own lighting and shadow-filtering code; only the shared data interface is factored out.

Loading SPIR-V

At run time, ShaderStageInfo::load looks up embedded SPIR-V by name and records which stage (vertex or fragment) it's for:

/// One programmable stage (vertex or fragment) and its SPIR-V.
pub struct ShaderStageInfo {
    pub stage: vk::ShaderStageFlags,
    code: Vec<u32>,
}
// ...
    /// * `Result<Self, Box<dyn Error>>` - The loaded stage, or an unknown-name or SPIR-V decoding error.
    pub fn load(stage: vk::ShaderStageFlags, name: &str) -> Result<Self, Box<dyn Error>> {
        let bytes = EMBEDDED_SHADERS
            .iter()
            .find(|(shader_name, _)| *shader_name == name)
            .map(|(_, bytes)| *bytes)
            .ok_or_else(|| format!("unknown embedded shader: {name}"))?;
        let mut file = Cursor::new(bytes);
        // A byte buffer need not be u32-aligned. Ash also validates the magic
        // number and handles byte-swapped SPIR-V, returning an I/O error for
        // malformed input instead of panicking during a slice cast.
        let code = ash::util::read_spv(&mut file)?;
        Ok(Self { stage, code })
    }

SPIR-V is a sequence of 32-bit words, and Vulkan wants it as a pointer to u32s. ash::util::read_spv decodes the embedded bytes into a properly aligned Vec<u32> and checks the SPIR-V header, which is safer than reinterpreting raw bytes. Nothing has been given to Vulkan yet: the words just sit in memory.

They reach the driver when a pipeline is created (Chapter 7). Each pipeline stage is described by a PipelineShaderStageCreateInfo, and with the VK_KHR_maintenance5 extension (Chapter 3) the SPIR-V can be chained straight onto it as a ShaderModuleCreateInfo, using push_next from Chapter 2:

/// Describes this stage's SPIR-V, ready to chain into its pipeline stage info.
pub fn module_create_info(&self) -> vk::ShaderModuleCreateInfo<'_> {
    vk::ShaderModuleCreateInfo::default().code(&self.code)
}

/// Builds the pipeline stage info, chaining the SPIR-V from [`Self::module_create_info`].
pub fn to_create_info<'a>(
    &self,
    module_info: &'a mut vk::ShaderModuleCreateInfo<'_>,
) -> vk::PipelineShaderStageCreateInfo<'a> {
    vk::PipelineShaderStageCreateInfo::default()
        .stage(self.stage)
        .name(ENTRY_POINT)
        .push_next(module_info)
}

The driver compiles the stage as part of building the pipeline, so there's no separate shader object to create first and destroy afterwards. Both structs only borrow: the stage info points at the module info, which points at the words, so a pipeline builder keeps all three alive until create_graphics_pipelines returns.

Materials load their vertex/fragment pair together:

pub fn load(
    vertex_shader: String,
    fragment_shader: String,
) -> Result<Self, Box<dyn std::error::Error>> {
    Ok(LoadedShaders {
        vertex: ShaderStageInfo::load(vk::ShaderStageFlags::VERTEX, &vertex_shader)?,
        fragment: ShaderStageInfo::load(vk::ShaderStageFlags::FRAGMENT, &fragment_shader)?,
    })
}

Reading the main vertex shader

main.vert and main.frag draw the textured, normal-mapped objects: the test cubes, and imported models that have textures, such as the marble sphere in the screenshot:

layout(push_constant) uniform Push { mat4 mvp; mat4 mv; vec4 uv_tiling; } pc;

layout(location = 0) in vec3 inPos;
layout(location = 1) in vec3 inNormal;
layout(location = 2) in vec3 inColor;
layout(location = 3) in vec2 inUV;
layout(location = 4) in vec3 inTangent;
layout(location = 5) in vec3 inBitangent;

layout(location = 0) out vec2 UV;
layout(location = 1) out vec3 vT;
layout(location = 2) out vec3 vB;
layout(location = 3) out vec3 vN;
layout(location = 4) out vec3 vFragPosView;

void main() {
    // View-space position of the fragment (for per-fragment light vectors)
    vFragPosView = (pc.mv * vec4(inPos, 1.0)).xyz;

    // Build T, B, N in view space for correct normal mapping.  mat3(mv) is a valid
    // normal matrix only because scene transforms use uniform scale (see `Transform`);
    // non-uniform scale would need the inverse-transpose for the normal.
    mat3 mv3 = mat3(pc.mv);
    vT = normalize(mv3 * inTangent);
    vB = normalize(mv3 * inBitangent);
    vN = normalize(mv3 * inNormal);
    // Apply UV tiling (xy), keep z,w unused; also flip Y for Vulkan
    vec2 uvScaled = inUV * pc.uv_tiling.xy;
    UV    = vec2(uvScaled.x, 1.0 - uvScaled.y);
    gl_Position = pc.mvp * vec4(inPos, 1.0);
}

• Push constants (the first line): two matrices and a UV tiling factor, sent with each draw. mvp takes a vertex all the way to the screen; mv takes it into view space, where the camera sits at the origin, which is where lighting is computed. The fragment shader declares the same block without uv_tiling; that's legal, because each stage may declare just the leading part of the push-constant range it uses.
• Inputs at locations 0–5: exactly the six fields of the Rust Vertex struct from Chapter 5, in the same order.
• Outputs at locations 0–4, received by the fragment shader at the same locations: UV, the tangent frame for normal mapping, and the view-space position.
• Normals are transformed with mat3(mv). Strictly, normals need the inverse-transpose of that matrix, but the two agree (after normalize) whenever the scale is uniform, and the scene's Transform type only allows uniform scale. The comment records that assumption.
• gl_Position is the built-in output for the clip-space position, as in OpenGL.

The UV flip (1.0 - y) is commented "for Vulkan", but whether a flip is right depends on the incoming UVs. Vulkan, like image files, puts texture coordinate (0, 0) at the top-left. The flip suits UVs written with OpenGL's bottom-left origin, the convention Assimp's loaders usually deliver. Vulkan's coordinate conventions differ from OpenGL's in several ways, covered with the camera in Chapter 12.

The fragment shader's inputs

layout(set = 0, binding = 1) uniform sampler2DArray shadowMap;
layout(set = 1, binding = 0) uniform sampler2D diffuseMap;
layout(set = 1, binding = 1) uniform sampler2D normalMap;

layout(std140, set = 0, binding = 0) uniform GlobalUBO {
    DirLight sun;
    mat4 light_vp[SHADOW_CASCADE_COUNT];
    vec4 cascade_splits;
    Light lights[MAX_LIGHTS];
    uint light_count;
    uint _pad0, _pad1, _pad2;
} ubo;

This is the descriptor side of the table above. Set 0 is global, shared by every object in a frame: a uniform buffer of lighting data (binding 0) and the shadow map (binding 1). Set 1 is per-material: a colour texture and a normal map. Grouping resources into sets by how often they change lets the renderer swap materials without touching the global data (Chapter 8).

std140 fixes the buffer's memory layout by strict rules: for instance, a vec3 is aligned like a vec4, to 16 bytes. The Rust struct that's copied into this buffer each frame (Chapter 5) has to reproduce that layout:

/// Global (per-frame/per-image) uniform buffer object shared across all pipelines via **descriptor set 0, binding 0**.
/// There is **one buffer per swapchain image** so the CPU can update the UBO while another image is still in-flight.
/// Keep fields 16-byte aligned for std140. Field order must match every GLSL `GlobalUBO`.
#[repr(C, align(16))]
#[derive(Clone, Copy, Default, Pod, Zeroable)]
pub struct GlobalUbo {
    pub dir_light: GpuDirLight,
    /// One view-to-light-clip matrix for each camera-depth slice.
    pub light_vp: [[[f32; 4]; 4]; crate::graphics::shadow_math::SHADOW_CASCADE_COUNT],
    /// Positive view-space far depth for each cascade.
    pub cascade_splits: [f32; crate::graphics::shadow_math::SHADOW_CASCADE_COUNT],
    // Then the array of point lights (std140 array of structs)
    pub lights: [GpuLight; crate::app::app::MAX_LIGHTS], // array of point lights
    pub light_count: u32,                                // number of active point lights
    pub _pad0: [u32; 3],                                 // three GLSL uints, never a uvec3 (std140)
}

std140 is a contract, and the Rust side reproduces it by hand with explicit _pad fields and align(16). Compile-time assertions in gpu_data.rs check the Rust sizes and offsets the GLSL block expects. When changing this interface, update both declarations together: otherwise the shader silently reads shifted values. cascade_splits shows the care needed: it's a vec4 in GLSL because a GLSL float[4] would give each element 16 bytes under std140 and no longer match Rust's 16-byte [f32; 4].

The tail shows the same trap. The GLSL declares the padding after light_count as three separate uints. A single uvec3 would be aligned to 16 bytes, leaving a gap and making the shader's block 16 bytes longer than the Rust struct's [u32; 3]. This vec3-aligns-to-16 rule is one of the most common sources of garbled uniform data, in OpenGL as well as Vulkan.

The fragment shader's work

The body of main.frag is classic Blinn-Phong lighting with normal mapping, done in tangent space. In outline:

vec3 albedo = texture(diffuseMap, UV).rgb;

// Tangent-space normal from normal map
vec3 normalTangent = normalize(texture(normalMap, UV).rgb * 2.0 - 1.0);
// ...
vec3 Ldir = normalize(TBN * (-ubo.sun.direction));
vec3 Hdir = normalize(Ldir + V);
float diff_dir = max(dot(normalTangent, Ldir), 0.0);
float spec_dir = pow(max(dot(normalTangent, Hdir), 0.0), shininess);
float sun_visibility = shadow_factor(vFragPosView);
lighting += sun_visibility * ubo.sun.intensity * ubo.sun.color * (diff_dir * albedo + spec_dir * vec3(1.0));
// ...
outColor = vec4(clamp(lighting, 0.0, 1.0), 1.0);

It samples the colour and normal maps, lights the surface with the sun (scaled by shadow_factor, which reads the cascaded shadow map and is the subject of Chapter 11), adds up to eight point lights in a loop, and writes the clamped result. Note the last line. Because the swapchain is UNORM and nothing here converts to sRGB, that value goes to the screen as-is, closing the loop on Chapter 4's note about gamma.