Part 5 · Chapter 15 · ~8 min prose skim

The ImGui overlay

How the menus, sliders and debug windows in the screenshot are described each frame by Dear ImGui, and how the renderer turns that description into Vulkan draws on top of the scene.

Overview The big picture

Immediate-mode UI

The panels in the screenshot (the menu bar with its CPU and GPU timings, the "Sun Light" sliders, the "Shadow Cascades" window) come from Dear ImGui, a popular C++ UI library used here through the imgui Rust crate. It's an immediate-mode UI: there are no button or window objects kept around. Every frame, the code simply runs through the interface again, "a window called Sun Light, containing a slider for X direction…", and ImGui works out layout, input and state as it goes.

ImGui never draws anything itself. At the end of the frame it hands back draw data: lists of plain 2-D triangles (positions, texture coordinates and colours), grouped into commands that each say which texture to use and which rectangle to clip to. Turning that into pixels is the application's job, and that's what ImGuiRenderer does.

The application describes the UI; ImGui turns that into triangles; the renderer draws them in the UI pass, after the scene.

Drawing it with Vulkan

The UI is drawn last, in its own pass, recorded into the same command buffer straight onto the finished scene in the swapchain image (Chapter 10). The renderer needs a handful of things:

a texture holding every letter of the font, built by ImGui once and uploaded at start-up;
vertex and index buffers that are refilled every frame, because the UI's triangles change every frame;
a simple pipeline: 2-D, no depth test, standard alpha blending;
a scissor rectangle per command, so text in a window can't spill outside it.

The same machinery can show any GPU image in a window. That's how the four shadow-map cascades (Chapter 11) appear as greyscale pictures in the "Shadow Cascades" panel.

Skip to next overview →

Deeper dive

Describing the UI

UiPass::execute starts an ImGui frame (the winit support crate passes in mouse, keyboard and window size) and builds the interface:

ui_ctx
    .platform
    .prepare_frame(ui_ctx.imgui.io_mut(), ui_ctx.window)
    .expect("Failed to prepare imgui frame");

let ui = ui_ctx.imgui.frame();
if ui_ctx.show_ui
    && let Some(_menu_bar) = ui.begin_main_menu_bar()
{
    if let Some(_main_menu) = ui.begin_menu("Main") {
        if ui.menu_item("Exit") {
            *ui_ctx.exit_flag = true;
        }
        _main_menu.end();
    }

    if let Some(_world_menu) = ui.begin_menu("World Controls") {
        if ui.menu_item("Point Lights") {
            self.show_point_lights_window = true;
        }
        if ui.menu_item("Sun Light") {
            self.show_sun_window = true;
        }
        if ui.menu_item("Ground Surface") {
            self.show_ground_window = true;
        }
        _world_menu.end();
    }

src/vulkan/ui_pass.rs:51-78

menu_item returns true on the frame it's clicked, so the UI code reacts in the same place it's declared. The values handed to sliders are fields of the app's world controls, which the render graph copies into the uniform buffer each frame (Chapter 9), so dragging the "Sun Light" sliders moves the sun on the very next frame.

The pipeline

ImGui's vertex is 20 bytes: a 2-D position, a texture coordinate, and a colour packed as four bytes. The colour uses R8G8B8A8_UNORM, so the GPU unpacks it to four floats for the shader, as the grass did with 16-bit values (Chapter 14):

let binding_desc = vk::VertexInputBindingDescription {
    binding: 0,
    stride: std::mem::size_of::<DrawVert>() as u32,
    input_rate: vk::VertexInputRate::VERTEX,
};
let attribute_descs = [
    vk::VertexInputAttributeDescription {
        location: 0,
        binding: 0,
        format: vk::Format::R32G32_SFLOAT,
        offset: offset_of!(DrawVert, pos) as u32,
    },
    vk::VertexInputAttributeDescription {
        location: 1,
        binding: 0,
        format: vk::Format::R32G32_SFLOAT,
        offset: offset_of!(DrawVert, uv) as u32,
    },
    vk::VertexInputAttributeDescription {
        location: 2,
        binding: 0,
        format: vk::Format::R8G8B8A8_UNORM,
        offset: offset_of!(DrawVert, col) as u32,
    },
];

src/vulkan/imgui_renderer/pipeline.rs:31-55

The rest is the simplest pipeline in the project: no culling, no depth test, a single sample (the UI draws after the anti-aliasing resolve), no depth attachment format at all, and "over" blending:

let color_blend_attachment = vk::PipelineColorBlendAttachmentState {
    blend_enable: vk::TRUE,
    src_color_blend_factor: vk::BlendFactor::SRC_ALPHA,
    dst_color_blend_factor: vk::BlendFactor::ONE_MINUS_SRC_ALPHA,
    color_blend_op: vk::BlendOp::ADD,
    src_alpha_blend_factor: vk::BlendFactor::ONE,
    dst_alpha_blend_factor: vk::BlendFactor::ONE_MINUS_SRC_ALPHA,
    alpha_blend_op: vk::BlendOp::ADD,
    color_write_mask: vk::ColorComponentFlags::R
        | vk::ColorComponentFlags::G
        | vk::ColorComponentFlags::B
        | vk::ColorComponentFlags::A,
};

src/vulkan/imgui_renderer/pipeline.rs:123-135

The only per-draw data is a projection matrix, sent as a 64-byte push constant to the vertex shader. ImGui works in window coordinates (pixels from the top-left corner), and this matrix maps them to Vulkan's −1..1 clip space. Because Vulkan's Y already points down, like screen pixels, no flip is needed:

// Compute orthographic projection matrix for ImGui (matching Vulkan NDC and winit coords)
let (l, t) = (draw_data.display_pos[0], draw_data.display_pos[1]);
let (w, h) = (draw_data.display_size[0], draw_data.display_size[1]);
let r = l + w;
let b = t + h;
// clang-format off
let proj = [
    [2.0 / (r - l), 0.0, 0.0, 0.0],
    [0.0, 2.0 / (b - t), 0.0, 0.0],
    [0.0, 0.0, 1.0, 0.0],
    [(r + l) / (l - r), (t + b) / (t - b), 0.0, 1.0],
];
// clang-format on
unsafe {
    device.cmd_push_constants(
        cmd_buf,
        self.pipeline_layout,
        vk::ShaderStageFlags::VERTEX,
        0,
        bytemuck::cast_slice(&proj),
    );
}

src/vulkan/imgui_renderer.rs:297-318

bytemuck::cast_slice views the 4 × 4 array of floats as bytes without copying, the safe reinterpretation that Pod types allow (Chapter 5). The shaders are tiny: the vertex shader, imgui.vert, applies the matrix, and the fragment shader multiplies the vertex colour by the texture.

void main() {
    vec4 tex = texture(Texture, frag_uv);
    // Replicate red channel to RGB so single-channel textures (e.g., depth) show as grayscale
    vec4 gray = vec4(tex.rrr, tex.a);
    out_color = frag_color * gray;
}

assets/shaders/imgui.frag:14-19

Copying the red channel into all three colour channels shows a one-channel image, like a depth map, as grey. That's harmless for the font, whose texture is white with the shapes in its alpha channel.

Uploading the font

ImGuiRenderer::new, in imgui_renderer/resources.rs, asks ImGui for its font atlas as RGBA pixels and uploads it exactly as textures were uploaded in Chapter 5: staging buffer, device-local image, a barrier-copy-barrier sequence, an image view and a sampler. The font's view and sampler become entry 0 of the renderer's texture list, which ImGui refers to as texture id 0.

Per-frame geometry

Each frame, all of ImGui's vertex and index lists are copied into a host-visible buffer pair that is persistently mapped (mapped once and never unmapped). There's one pair per swapchain image:

#[derive(Default)]
struct UiFrameBuffers {
    vertex_buffer: vk::Buffer,
    vertex_allocation: Option<Allocation>,
    vertex_buffer_size: vk::DeviceSize,
    index_buffer: vk::Buffer,
    index_allocation: Option<Allocation>,
    index_buffer_size: vk::DeviceSize,
}
// ...
    /// One buffer pair per swapchain image: rewriting a single pair each frame would race
    /// with the previous frame, which may still be drawing from it.
    frame_buffers: Vec<UiFrameBuffers>,

src/vulkan/imgui_renderer.rs:37-45,56-58

let index_size =
    (draw_data.total_idx_count as usize * std::mem::size_of::<DrawIdx>()) as vk::DeviceSize;
if self.frame_buffers.len() <= image_index {
    self.frame_buffers
        .resize_with(image_index + 1, UiFrameBuffers::default);
}
// `begin_frame` has already waited on this image's previous submit, so its buffers
// are idle and may be replaced or rewritten. Capacity grows geometrically to avoid
// reallocating for small UI changes.
let buffers = &mut self.frame_buffers[image_index];
let grow_vertex_buffer = vertex_size > buffers.vertex_buffer_size;

src/vulkan/imgui_renderer.rs:81-91

Why one pair per image? With two frames in flight (Chapter 9), the GPU may still be drawing the previous frame's UI while the CPU prepares this one. A single shared buffer would be overwritten while it's being read. This is a classic mistake when coming from OpenGL, and no validation layer reports it, because the write happens on the CPU. begin_frame has already waited for this image's previous use, so its own pair is free to rewrite, or even to replace with a larger one. The grass's indirect buffers follow the same rule.

Recording the draws

With the buffers filled, render binds them, sets the viewport and pushes the matrix, then walks ImGui's command lists:

for draw_list in draw_data.draw_lists() {
    for cmd in draw_list.commands() {
        if let imgui::DrawCmd::Elements { count, cmd_params } = cmd {
            let image_info = self
                .textures
                .get(cmd_params.texture_id.id())
                .unwrap_or(&self.textures[0]);
            if image_info.image_view != bound_view {
                let write = vk::WriteDescriptorSet::default()
                    .descriptor_type(vk::DescriptorType::COMBINED_IMAGE_SAMPLER)
                    .image_info(std::slice::from_ref(image_info));
                self.push_descriptor.cmd_push_descriptor_set(
                    cmd_buf,
                    vk::PipelineBindPoint::GRAPHICS,
                    self.pipeline_layout,
                    0,
                    &[write],
                );
                bound_view = image_info.image_view;
            }
            // Set scissor rectangle from ImGui clip rect
            let clip = cmd_params.clip_rect;
            // Convert logical clip rect to physical pixels
            let scissor = vk::Rect2D {
                offset: vk::Offset2D {
                    x: (clip[0] * fb_scale[0]).max(0.0) as i32,
                    y: (clip[1] * fb_scale[1]).max(0.0) as i32,
                },
                extent: vk::Extent2D {
                    width: ((clip[2] - clip[0]) * fb_scale[0]).max(0.0) as u32,
                    height: ((clip[3] - clip[1]) * fb_scale[1]).max(0.0) as u32,
                },
            };
            device.cmd_set_scissor(cmd_buf, 0, &[scissor]);
            // Draw indexed
            device.cmd_draw_indexed(
                cmd_buf,
                count as u32,
                1,
                index_offset + cmd_params.idx_offset as u32,
                vertex_offset + cmd_params.vtx_offset as i32,
                0,
            );
        }
    }
    index_offset += draw_list.idx_buffer().len() as u32;
    vertex_offset += draw_list.vtx_buffer().len() as i32;
}

src/vulkan/imgui_renderer.rs:324-371

Three things happen per command:

Texture. Id 0 is the font; ids 1 and up are extra images registered with the renderer. The texture is pushed as set 0 (Chapter 8) only when its image differs from the last one pushed.
Scissor. ImGui's clip rectangle is in logical points, so it's scaled by the framebuffer scale (2 on a Retina display) into pixels.
Draw. All lists share one vertex and one index buffer, so each draw adds the list's starting offsets to the command's own. ImGui uses 16-bit indices, hence UINT16 when the index buffer is bound.

Showing the shadow cascades

To show a GPU image in a window, it needs an entry in the texture list like the font's, registered under a texture id. The cascade panel does this for each layer of the shadow map:

let shadow_tex_ids: Option<[imgui::TextureId; SHADOW_CASCADE_COUNT]> =
    ctx.attachments.get(&AttachmentKind::Shadow).map(|handle| {
        std::array::from_fn(|cascade| {
            ui_ctx.renderer.shadow_texture_id(
                cascade,
                ctx.vulkan_base.shadow_sampler,
                handle.layer_views[cascade],
            )
        })
    });

let mut open = true;
ui.window("Shadow Cascades")
    .opened(&mut open)
    .always_auto_resize(true)
    .build(|| {
        if let Some(texture_ids) = &shadow_tex_ids {
            for (cascade, &texture_id) in texture_ids.iter().enumerate() {
                ui.group(|| {
                    ui.text(format!("Cascade {}", cascade + 1));
                    imgui::Image::new(texture_id, [224.0, 224.0])
                        .uv0([0.0, 1.0])
                        .uv1([1.0, 0.0])
                        .build(ui);

src/vulkan/ui_pass.rs:228-251

Each cascade uses its own single-layer view of the shadow array (Chapter 10), sampled with the shadow sampler. The shadow map is a per-swapchain-image attachment, so shadow_texture_id points each cascade's texture id at this frame's view. Pushed descriptors are copied into the command buffer as it's recorded, so changing the entry can't disturb a frame in flight. By the UI pass the shadow map is already in its shader-readable layout, with fragment-shader read access established by the main pass. The UI pass declares that same read, so the scheduler sees the dependency and needs no additional barrier. The flipped UVs (uv0 at the bottom) turn the light's-eye view the right way up: the shadow projection has no Y flip (Chapter 11), so the light's "up" is stored at the bottom of the image.

Recap

Dear ImGui is immediate-mode: the UI is re-declared every frame, and ImGui returns plain 2-D triangle lists with per-command texture and clip rectangle.
The renderer draws them in the UI pass with a minimal pipeline: 20-byte vertices, a projection push constant, alpha blending, no depth.
Geometry goes into persistently mapped buffers, one pair per swapchain image, so the CPU never overwrites triangles a frame in flight is still drawing.
Any GPU image can be shown by registering it under a texture id, and each draw pushes the matching descriptor; the shadow cascades are displayed this way.