feat: Add glyph atlas cache with LRU eviction for vello spare strips renderers

[grebmeg]

Introduces a glyph atlas cache that rasterizes glyphs once and reuses the bitmaps on subsequent frames, reducing redundant work for text-heavy scenes. The cache uses LRU age-based eviction, deterministic hashing for reproducible atlas packing, subpixel-quantized cache keys (supporting variable fonts and COLR glyphs), and a multi-page atlas backed by vello_common::ImageCache.

[grebmeg]

Some tests are failing due to minor differences, I skimmed through the Kompari HTML report, and they’re very likely caused by recent changes in Vello that slightly affect the rendered output.

[LaurenzV]

Still a lot missing, but that's as far as I got today. :( The overall approach looks great to me though!

[LaurenzV]

Am I understanding correctly that we have a number of different frames which exercise different paths (different layouts, different caching behavior, etc.), and then we are summarizing the number of all frames across the different stages? If yes, isn't that a bit meaningless since each frame tests something different? So the numbers will just always vary wildly.

[grebmeg]

The aggregated stats are intentional, they give an amortized view across the full cache lifecycle (cold, warm, partial overlap, etc.), which is useful for comparing overall cached vs uncached performance. Each phase also prints its own per-phase stats. This is a development/benchmarking aid and the mixed summary is helpful for getting a quick read on the overall picture. Happy to revisit the output format later if needed.

[LaurenzV]

Wouldn't it be possible to clear all regions in one pass? Is this just for simplicity for now?

[grebmeg]

Yes, this is just for simplicity for now. The proper approach would be to use vello_hybrid's GPU clear shader to clear all rects in a single render pass, but that requires exposing a batched clear API on vello_hybrid's Renderer. I'll do that as a follow-up in Vello. For now I've at least reused a single zeroed buffer across rects to avoid the per-rect allocation.

[taj-p]

Looking good! Just flushing my comments before I go into meetings

[taj-p]

take_pending_atlas_commands uses mem::take, which transfers ownership of the SmallVec and every inner Vec<AtlasCommand> to the caller. After replay the caller drops everything, so the heap allocations are lost every frame and re-allocated from scratch next frame.

Since consumers only need to iterate and drain the commands, we could avoid this by keeping the recorders in place and letting consumers borrow them. Something like:

pub fn replay_pending_atlas_commands(
    &mut self,
    mut f: impl FnMut(&mut AtlasCommandRecorder),
) {
    for slot in &mut self.pending_atlas_commands {
        if let Some(recorder) = slot.as_mut() {
            if !recorder.commands.is_empty() {
                f(recorder);
                recorder.commands.clear();
            }
        }
    }
}

Consumers change from:

for mut recorder in glyph_caches.glyph_atlas.take_pending_atlas_commands() {
    glyph_renderer.reset();
    replay_atlas_commands(&mut recorder.commands, glyph_renderer);
    // ...
}

to

glyph_caches.glyph_atlas.replay_pending_atlas_commands(|recorder| {
    glyph_renderer.reset();
    replay_atlas_commands(&recorder.commands, glyph_renderer);
    // ...
});

The benefit of this approach is that the command list for each AtlasCommandRecorder isn't freed each frame. I think that's beneficial for operations like zoom, where we need to insert into the cache every frame (since resolved font size will presumably change each frame).

[grebmeg]

Good idea! Switched to a replay_pending_atlas_commands callback API that keeps recorders in place and only clears their command Vecs after replay, preserving capacity across frames. For the CPU backend, added a replay_pending_atlas_commands_with_pixmaps helper that splits the borrow to give the closure access to both the recorder and the page pixmaps.

[taj-p]

Since we renamed the other similar structs to use the a "Glyph" prefix, should we do the same here?

[grebmeg]

Oh yeah, of course! Renamed ColrGlyph → GlyphColr for consistency with the other structs.

[taj-p]

Is the plan to (sometime in the future) move these vello_common dependencies (and the shelf allocator) into a separate crate so that Glifo doesn't depend on vello_common?

[grebmeg]

Yes, the long-term plan is to extract shared types (color, basic geometry) and the allocator into their own crate(s) so glifo doesn't depend on vello_common directly. For now the coupling is pragmatic: those types are stable and we'd just be re-exporting the same things. Happy to track this as a follow-up issue if that would be helpful?

[taj-p]

Happy to track this as a follow-up issue if that would be helpful?
However you like 🙏 !

[taj-p]

Is it possible to configure minimum font size (after which we use renderer sampling to interpolate smaller sizes) and a maximum size (after which we always directly render)?

[grebmeg]

Added max_cached_font_size to GlyphCacheConfig (default 128 ppem) — glyphs above this threshold now bypass the atlas and render directly each frame.

For min_cached_font_size: this would involve caching at the clamped size and applying a downscale transform at composite time, plus adjustments to hinting, metrics, and subpixel offsets across all three glyph branches. I think it's a good idea but a bit too involved for this PR. I'll track it as a follow-up. Happy to hear what you think!

[taj-p]

I think it's a good idea but a bit too involved for this PR. I'll track it as a follow-up. Happy to hear what you think!

SGTM!

[taj-p]

It's unfortunate that this form of API necessitates a redundant hash map lookup on cache miss. I.e., on cache miss, we perform another hash map operation to insert (being unable to re-use the RawEntry or similar).

We could precompute the hash once and use something like HashMap::raw_entry_mut().from_key_hashed_nocheck(...) but let's maybe leave that as a follow up

[grebmeg]

Agreed, the double lookup on miss is a known trade-off of the current get/insert split. I'll keep raw_entry_mut with precomputed hashes in mind as a follow-up optimisation. Thanks for flagging it.

[LaurenzV]

Isn't this the same code as for fill_glyphs? Where is the stroking happening here? 🤔

[grebmeg]

Good catch, thanks! This is actually already like that on main — I just didn't notice. Split render_outline_directly into fill_outline_directly and stroke_outline_directly, with the stroke variant now correctly calling stroke_path.

[LaurenzV]

Now that we encode the fractional translation in the subpixel offset, do we not need to update the transform to discard the fraction from there? Also, why are y fractions not handled here?

[taj-p]

I think the fractional removal is handled in render_outline_glyph_from_atlas

[grebmeg]

Good question! The fractional removal at render time is handled by render_outline_glyph_from_atlas which uses tx.floor()/ty.floor() — so the full transform is passed through but only the integer part is used for placement.

For y fractions: when hinting is enabled, the y translation is already rounded in calculate_outline_transform. When hinting is off, if I'm not missing something, vertical sub-pixel shifts are less perceptually impactful for horizontal text, and adding a y dimension would multiply cache entries. Added a comment to clarify this.

[LaurenzV]

Not sure I understand this, I thought vello_cpu and vello_hybrid work the same, where we have one (or multiple) huge atlas pages and glyphs are drawn into that at specific offsets. Why is there different behavior here?

[grebmeg]

Both backends do use atlas pages with glyphs at specific offsets. The difference is in how the paint transform is set up when reading back from the atlas:

• vello_cpu: The ImageSource references the entire page pixmap, so we need to shift the paint origin to the glyph's (x, y) position within the page.

• vello_hybrid: Vello's image cache resolves each allocation to its own origin, so we only need to compensate for the GLYPH_PADDING inset.

The underlying atlas structure is the same — it's just the coordinate system for the paint source that differs between the two rendering paths.

[LaurenzV]

strip = "none" might also be worth adding!

[grebmeg]

Added strip = "none" to make sure symbols are preserved for profiling.

[LaurenzV]

Do we not need glyph padding for vello_cpu?

[grebmeg]

The vello_cpu doesn't need an explicit GLYPH_PADDING offset in paint_transform because atlas_slot.x / atlas_slot.y are already inset by GLYPH_PADDING from the allocation origin (see GlyphAtlas::insert_entry). So the translate to (-slot.x, -slot.y) already moves past the padding. The hybrid backend's image cache resolves to the allocation origin (before padding), so it needs the explicit (-padding, -padding) shift.

[LaurenzV]

Why do Colr and Outline have different behavior here? Because we don't use subpixel offsets here, since they are not as critical for emojis?

[grebmeg]

Colr glyphs use fractional scaled_bbox dimensions for their area (computed from the glyph's actual bounding box), while outline glyphs use integer atlas_slot.width/height since they snap to pixel boundaries. The Rect carries these fractional bounds so the rendered quad matches the original glyph dimensions rather than the padded integer atlas allocation, which avoids subtle scaling artifacts.

[taj-p]

Is there a way to configure a maximum value for this?

[grebmeg]

Yes, the underlying AtlasConfig (from vello_common) already has a max_atlases field that caps the number of atlas pages. When the limit is reached, allocate() fails and the glyph is drawn directly without caching. This is configured on the ImageCache side rather than GlyphCacheConfig.

[taj-p]

Nice! This looks really good! Some comments

[taj-p]

Depending on var_coords for the run, this could allocate every time. I think we either should consider:

1. removing var_coords from the key and rely on the 2 level map structure (which encodes the var_coords anyway); or
2. we should use the Equivalent trait to create borrowed keys (shown below) and convert it to an owned key on cache misses only - see parley/src/lru_cache.rs for an example implementation

#[derive(Clone, Copy)]
pub struct GlyphCacheKeyRef<'a> {
    pub font_id: u64,
    pub font_index: u32,
    pub glyph_id: u32,
    pub size_bits: u32,
    pub hinted: bool,
    pub subpixel_x: u8,
    pub context_color_packed: u32,
    pub var_coords: &'a [NormalizedCoord],
}

[LaurenzV]

I only took a very coarse second look because it would take ages to review again carefully, but I don't think I would have much to add in addition to my first review. So LGTM if LGTM to @taj-p 😄 Awesome work!

[taj-p]

LGTM 🎉 ! Let's goooo!!! I didn't re-read all the code line-by-line again, but skimmed through the comments and their resolutions. All seems absolutely BRILLIANT!