The frame loop

Overview The big picture

Two processors, working at the same time

The CPU and GPU are separate processors. When the renderer submits work to the GPU's queue, the call returns almost immediately; the GPU picks it up later and works through it on its own. That's what makes high frame rates possible: while the GPU draws frame N, the CPU can already be preparing frame N+1. But it creates three hazards:

1. The CPU must not overwrite data the GPU is still reading, such as this frame's lighting values or the list of commands itself.
2. The GPU must not draw into a swapchain image that's still being shown.
3. An image must not be shown before the GPU has finished drawing it.

OpenGL handled all of this invisibly, stalling or copying when it had to. Vulkan gives you two kinds of signal and expects you to use them:

• A fence is a signal from the GPU to the CPU: "the work you submitted with this fence has finished". The CPU can wait on it.
• A semaphore is a signal from the GPU to the GPU: one piece of queued work waits for another to finish, without involving the CPU at all.

Each frame therefore runs: wait for a fence, acquire a swapchain image, record commands, submit them, present the image. That's the loop from Chapter 1, now with the signals that make it safe.

Command buffers

Every drawing operation goes into a command buffer, a recorded list that's submitted in one go. Recording is just the CPU writing a list; nothing runs until submission. This renderer has one command buffer per swapchain image, and re-records it from scratch each frame.

Frames in flight

RustyVulkan lets the CPU get at most two frames ahead of the GPU. Each of those two frame slots has its own fence and its own "image available" semaphore. At the start of a frame, the CPU waits on that slot's fence, which only blocks if the GPU is still busy with the frame from two frames ago.

A subtlety: there are typically three swapchain images but only two slots, and the swapchain doesn't promise to hand images out in a fixed rotation. So resources that belong to a specific image (its command buffer, its uniform buffer and descriptor set, its "render finished" semaphore) are tracked separately from resources that belong to a slot. The renderer also remembers which slot's fence last used each image, and waits on that too if needed.

Skip to next overview →

Where a frame starts

winit sends a RedrawRequested event whenever the app asks for one, which it does continuously (Chapter 16). The handler skips drawing for a minimised window, moves the camera, then runs the render graph:

WindowEvent::RedrawRequested => {
    let size = win.inner_size();
    if size.width == 0 || size.height == 0 {
        app.last_movement_update = Instant::now();
        return;
    }
// ...
    let mut graph = std::mem::take(&mut app.render_graph);
    if let Err(e) = graph.execute(app) {
        app.fatal_error = Some(e);
        // A failure after acquisition may leave a reset fence
        // unsignalled. Retrying could hang the next frame.
        event_loop.exit();
    }
    app.render_graph = graph;

Note what happens on error: the app exits. DEVELOPMENT.md explains that a render failure is fatal on purpose. A frame interrupted after its fence was reset can leave synchronisation objects in a state where the next frame would wait forever.

The synchronisation objects

Created in VulkanBase::new:

let mut image_available_semaphores = Vec::with_capacity(INFLIGHT_FRAMES);
let mut in_flight_fences = Vec::with_capacity(INFLIGHT_FRAMES);
let semaphore_info = vk::SemaphoreCreateInfo::default();
let fence_info = vk::FenceCreateInfo {
    flags: vk::FenceCreateFlags::SIGNALED,
    ..Default::default()
};
for _ in 0..INFLIGHT_FRAMES {
    unsafe {
        image_available_semaphores.push(device.create_semaphore(&semaphore_info, None)?);
        in_flight_fences.push(device.create_fence(&fence_info, None)?);
    }
}

let mut render_finished_semaphores = Vec::with_capacity(image_count);
for _ in 0..image_count {
    unsafe {
        render_finished_semaphores.push(device.create_semaphore(&semaphore_info, None)?);
    }
}

// Per-swapchain-image tracker: which fence currently owns each image (or null)
let image_owner_fence = vec![vk::Fence::null(); image_count];

Two of each per-slot object (INFLIGHT_FRAMES is 2), but one "render finished" semaphore per swapchain image, and an owner-fence entry per image, initially empty. The fences start SIGNALED: the first frame on each slot waits on its fence, and with nothing yet submitted, an unsignalled fence would never fire.

begin_frame

pub fn begin_frame(&mut self) -> Result<Option<FrameCtx>, Box<dyn Error>> {
    unsafe {
        let slot = self.frame_slot;

        // Wait for the previous work submitted on this CPU slot.
        self.device
            .wait_for_fences(&[self.in_flight_fences[slot]], true, u64::MAX)?;
// ...
        // Acquire an image; signal when it's ready via the per-slot image-available semaphore.
        let (image_index, is_suboptimal) = match self.swapchain_loader.acquire_next_image(
            self.swapchain.handle,
            u64::MAX,
            self.image_available_semaphores[slot],
            vk::Fence::null(),
        ) {
            Ok(result) => result,
            Err(vk::Result::ERROR_OUT_OF_DATE_KHR) => {
                self.swapchain_recreation_needed = true;
                return Ok(None);
            }
            Err(e) => return Err(e.into()),
        };
        self.swapchain_recreation_needed |= is_suboptimal;

        let idx = image_index as usize;
        self.current_image_index = idx;

        // If this image is still tied to an older in-flight slot, wait for that slot to finish first.
        if self.image_owner_fence[idx] != vk::Fence::null() {
            self.device
                .wait_for_fences(&[self.image_owner_fence[idx]], true, u64::MAX)?;
        }

        // Reuse this slot's fence for the new submit and associate it with this image.
        self.device.reset_fences(&[self.in_flight_fences[slot]])?;
        self.image_owner_fence[idx] = self.in_flight_fences[slot];

        // Begin recording; passes are responsible for starting rendering.
        let cmd_buf = self.command_buffers[idx];

        let begin_info = vk::CommandBufferBeginInfo::default();
        self.device.begin_command_buffer(cmd_buf, &begin_info)?;

In order:

1. Wait for this slot's fence. u64::MAX means no timeout. If the GPU has already finished this slot's last frame, this returns immediately. (Elided here: with the fence passed, that frame's GPU timings can now be read back; see below.)
2. Acquire an image. acquire_next_image returns an image index right away, but the image may still be on screen. So it also takes a semaphore to signal once the image is truly free, and the GPU will wait on that before writing to it. "Out of date" means the window changed: return None and let the caller rebuild (Chapter 4).
3. Wait for the image's previous owner. Because images don't come back in strict rotation, the image just acquired might have been last used by the other slot, whose work may still be running. Its command buffer and uniform buffer are about to be overwritten, so wait for that slot's fence too.
4. Claim the fence and the image. Reset this slot's fence to unsignalled, ready for the coming submit, and record it as the image's new owner. This happens only after acquisition succeeded, so an early "out of date" return never leaves a reset fence that nothing will signal.
5. Begin recording into the image's command buffer. Beginning implicitly resets it, which the command pool permits because it was created with RESET_COMMAND_BUFFER.

Everything indexed by image_index from here on (command buffer, uniform buffer, set 0, timestamp queries) is now safe for the CPU to touch. That's the guarantee Chapter 8 relied on when updating descriptors.

Recording the frame

Between begin_frame and end_frame, RenderGraph::execute does the frame's real work, in this order:

1. write a start timestamp;
2. compute the lighting and shadow-cascade data and copy it into this image's uniform buffer;
3. gather the images each pass needs, and point set 0 at the shadow map;
4. for each pass (shadow, main, UI), insert any needed image barriers, then let the pass record its commands;
5. transition the swapchain image to its presentation layout, and write an end timestamp.

Steps 3–5 are the subject of Chapter 10.

end_frame: submit and present

self.device.end_command_buffer(frame.cmd_buf)?;

let wait = [vk::SemaphoreSubmitInfo::default()
    .semaphore(self.image_available_semaphores[frame.frame_slot])
    .stage_mask(vk::PipelineStageFlags2::COLOR_ATTACHMENT_OUTPUT)];
let command_buffers =
    [vk::CommandBufferSubmitInfo::default().command_buffer(frame.cmd_buf)];
let signal = [vk::SemaphoreSubmitInfo::default()
    .semaphore(self.render_finished_semaphores[frame.image_index as usize])
    .stage_mask(vk::PipelineStageFlags2::ALL_COMMANDS)];
let submit_info = vk::SubmitInfo2::default()
    .wait_semaphore_infos(&wait)
    .command_buffer_infos(&command_buffers)
    .signal_semaphore_infos(&signal);
self.device.queue_submit2(
    self.graphics_queue,
    &[submit_info],
    self.in_flight_fences[frame.frame_slot],
)?;
// ...
match self
    .swapchain_loader
    .queue_present(self.graphics_queue, &present_info)
{
    Ok(true) | Err(vk::Result::ERROR_OUT_OF_DATE_KHR) => {
        self.swapchain_recreation_needed = true;
    }
    Err(e) => return Err(e.into()),
    _ => {}
}

// Advance to next CPU slot
self.frame_slot = (frame.frame_slot + 1) % self.image_available_semaphores.len();

The submit ties the three hazards together:

• Wait on the slot's image-available semaphore, but only at the COLOR_ATTACHMENT_OUTPUT stage. A pipeline stage is a point in the GPU's processing; this is the point where fragment colours are written to an image. GPU work that doesn't write colour (the shadow pass, vertex shading) can start before the image is free; only writing pixels into it has to wait.
• Signal the image's render-finished semaphore when all the commands complete.
• Signal the slot's fence at the same moment, so a later begin_frame knows the slot is free.

queue_submit2, from synchronization2, describes each semaphore with its own stage, which keeps the wait and signal points next to the semaphores they belong to.

Presentation then waits on that render-finished semaphore, so the image is shown only after drawing completes. All of this ordering happens on the GPU, without the CPU waiting. (One queue does both drawing and presenting, which is why this simple pair of semaphores is enough.) Present reports Ok(true) for "suboptimal" and an error for "out of date"; both only request a rebuild before the next frame. Finally the slot advances, alternating between 0 and 1.

Measuring GPU time without stalling

The "GPU: … ms" figure in the screenshot's menu bar comes from timestamp queries: commands recorded into the command buffer that make the GPU write its clock into a query pool at that point. Five are written per frame: start, end of shadows, start and end of vegetation, and end. Reading results straight after submitting would mean waiting for the GPU, so instead they're read in a later begin_frame, just after that slot's fence wait has guaranteed they're complete:

if let Some(prev_image_index) = self.last_image_per_slot[slot] {
    let base = self.image_query_base(prev_image_index);
    let mut data = [0_u64; TIMESTAMPS_PER_IMAGE as usize];
    let res = self.device.get_query_pool_results(
        self.timestamp_query_pool,
        base,
        &mut data,
        vk::QueryResultFlags::TYPE_64,
    );
    if res.is_ok() {
        self.last_gpu_timings = GpuPassTimings {
            total_ms: self.timestamp_delta_ms(data[0], data[4]),
            shadow_ms: self.timestamp_delta_ms(data[0], data[1]),
            scene_ms: self.timestamp_delta_ms(data[1], data[2]),
            vegetation_ms: self.timestamp_delta_ms(data[2], data[3]),
            vegetation_in_scene: data[2] == data[3],
        };
    }

Results are read without asking Vulkan to wait for them; that's safe only because the fence has already been waited on. Raw ticks are converted to milliseconds using the device's timestamp_period (nanoseconds per tick). The reported numbers therefore describe the frame from two frames ago, which is perfectly good for a profiler and costs nothing.

Not every GPU can honour a timestamp anywhere. Apple GPUs, under KosmicKrisp, record them only at the edges of a render pass, so both vegetation timestamps, which sit inside the main pass, land at its end with the same value. Equal timestamps can also mean there was no measurable vegetation work, so they don't prove a GPU capability limit. vegetation_in_scene notes equal timestamps, and the profiler labels the scene figure as including vegetation and reports grass as "not separately measurable this frame".

When the swapchain changes

Recreation (Chapter 4) happens only at frame boundaries: before begin_frame, after a failed acquire, or after end_frame. It's safe at any of those points because Swapchain::recreate starts by waiting for the whole device to go idle. On top of the swapchain itself, VulkanBase::recreate_swapchain rebuilds everything sized per image: command buffers, render-finished semaphores, owner-fence tracking, uniform buffers and set 0, the per-image attachments (Chapter 10) and the timestamp pool. The per-slot fences and image-available semaphores are kept, because the number of slots doesn't change.