Caching & Memory Management

There's 970 filters in Aesthetikk, each one is already coverted to smaller dimensions , however, we're still looking at ~300MB of static assets. Storing these in the binary is probably not the best idea. I instead store these in a object storage, then serve it on demand and through CDN. But, if user keep downloading everytime they refresh the app, first, they would give up right after the first use, second, I don't have infinite bandwidth budget.

I want to walk you through two things in this post:

1. How we cache the filters on the client without storing user download history in a database
2. How we enforce memory limits so editing stays fast and the browser does not crash

The goal of this is to discuss about the architecture & design choices that I made, If you want to make something similar, I hope this would be helpful. If you have not read it yet, start with How-2-WebGL2 , I will give you a better picture of how the filters are applied to the photo.

Bottleneck of needing to store a large amount of static assets

A preset library is an inventory problem, think about loading a couple hundred of static assets at once will significantly slow down the app, the other features will be catching strays for no reason.

In reality, users never need all 900 files at once. They tend to stick to a few categories, so assumming this will lead us to a clearer road to design the system to serves it.

I also made a deliberate product decision: Don't save users downloaded filters in our database. There is no "saved cache" column in Supabase, no sync of downloaded categories across devices probably a trade-off that I had to make.

Why?

• Preset files are large. Storing user's cache state server side adds complexity with little payoff .
• Downloads are idempotent. If a user clears browser data, they can redownload a category in one batch call.
• Faster textures load time, less codes we have to maintain.

Instead, IndexedDB was implemented to cache the downloaded filters ( browser level ). The server holds the catalog and serves files on demand. The browser decides what to keep locally, within hard caps.

Part 1: Three tiers of caching

This diagram below will visualize the storage layers that the LUTs are stored and cached.

Even

Part 2: Tier 0 — Origin and batch download

Each preset is a 512×512 RGBA PNG, roughly 200KB to 1MB ( already coverted from 3D .cube textures). Files live in private object storage. Browsers cannot fetch them directly, so the server generates presigned URLs: temporary download links that expire.

When a user chooses to download a category, the client makes one batch request for every signed URL in that category. Then it fetches the PNGs in parallel straight from the CDN. We just use simple REST here. Bandwidth goes origin to browser, not through your app server for every file.

The server also caches presigned URLs in memory for 15 minutes, so repeat requests for the same file do not hammer object storage:

const urlCache = new Map<string, { url: string; expiresAt: number }>();
const CACHE_TTL = 15 * 60 * 1000; // 15 minutes

Design choice: Batch download the whole category, 1 single API call, since each categories vary between 10 - 30 filters.

Part 3: Tier 1 — IndexedDB (persistent disk cache)

This is where the filters live after being downloaded, straight up in their browser.

What gets stored

Each cached preset record holds metadata plus the raw PNG bytes:

interface CachedLut {
  id: string;
  name: string;
  filename: string;
  categoryId: string;
  categoryName: string;
  blob: Blob;           // the actual PNG bytes
  downloadUrl: string;
  cachedAt: number;     // used for LRU eviction
  fileSize: number;
  expiresAt?: number;
}

The cache manager opens a dedicated IndexedDB database with indexes on category, download time, and filename. That lets us query by category and walk oldest-first when evicting.

A counterintuitive decision: ignore signed URL expiry

Presigned URLs expire in 15 minutes. Cached blobs do not.

When saving to IndexedDB, we intentionally skip tying cache expiry to URL expiry:

await downloadAndCacheLut(
  lut.id, lut.name, lut.filename,
  category.id, category.name,
  download.signedUrl,
  undefined,  // do not use signed URL expiration for cache expiry
);

Why? The blob is already on disk. The URL was only the delivery mechanism. If we expired cache entries when URLs expired, users would re-download the same category every session. That wastes bandwidth and feels broken.

Tradeoff: if a preset file changes on the server, stale blobs could persist until LRU evicts them or the user clears site data. For a preset catalog that rarely changes, that is acceptable.

LRU eviction at 500 MB

After every write, a deferred cleanup checks total size. If it exceeds 500 MB, the oldest entries (by cachedAt) are deleted until under budget.

If the browser throws QuotaExceededError, we force cleanup to 80% of max and retry once. Multi-tab PWA usage is handled with connection deduplication and stale connection detection so reads do not fail silently.

Design choice: IndexedDB over the Cache API. We need structured queries (by category, by age), large blobs, and persistence across sessions. The Cache API is URL-keyed and harder to manage programmatically.

Part 4: Tier 2 — GPU texture cache

IndexedDB is fine for storage. It is kinda slow for switching presets in real time. The path: disk → JavaScript heap → GPU upload takes milliseconds that add up when someone clicks through ten presets quickly.

A cache bridge keeps recently used presets as WebGL textures in VRAM:

class LutCacheBridge {
  private textureCache = new Map<string, CachedWebGLTexture>();
  private maxCachedTextures = 10;
  private maxGpuMemory = 50 * 1024 * 1024; // 50MB
}

Load flow

When the color engine needs a preset texture:

async loadLutTexture(lutId, colorEngine) {
  const cachedTexture = this.textureCache.get(lutId);
  if (cachedTexture && gl.isTexture(cachedTexture.texture)) {
    cachedTexture.lastUsed = Date.now();
    return cachedTexture.texture;
  }

  const cachedLut = await getCachedLut(lutId);
  if (!cachedLut) return null;

  const arrayBuffer = await cachedLut.blob.arrayBuffer();
  const texture = await colorEngine.loadLutTexture(arrayBuffer);

  this.cacheTexture(lutId, texture, estimatedMemory);
  return texture;
}

Before inserting, dual-limit eviction runs until both count is under 10 and memory plus the new texture is under 50 MB. The least recently used texture goes first.

Design choice: GPU textures load on selection only, not when browsing the category grid. A preload helper exists but nothing calls it today. Browsing should not spike VRAM.

When a texture is evicted from GPU, the PNG still sits in IndexedDB. Next selection re-uploads from disk. No network required.

Part 5: App state stores metadata only, never blobs

Global state tracks which categories are cached, preset names and IDs, and the currently selected preset. Binary data never enters React state.

This prevents re-renders from retaining megabytes of PNG data in the JavaScript heap. The sidebar that display presets only updates selection state. It does not know about WebGL. The effect hook picks up the new selection and loads the texture.

On app open, the client fetches category metadata from the API, then scans IndexedDB to see which categories already have local copies. That scan rehydrates the UI without a server round trip per user cache preference.

Part 6: E2E logic from what user experiences.

App opens. Category names load from the API. IndexedDB is scanned. Categories with local data show as ready.

User opens an uncached/undownloaded category. A confirmation modal appears. One button calls the batch API call to returns signed URLs. Parallel downloads write PNGs to IndexedDB. The category is marked cached in app state only if every download succeeds. Partial failure means do not mark cached. The user can retry.

User clicks a preset. Selection updates in state. The color engine hook loads the texture through the bridge (IndexedDB → GPU), runs the render pipeline, canvas updates.

User returns next week. IndexedDB still has the category. No database lookup. No re-download unless LRU evicted it or they cleared site data.

Part 7 : Collect the garbage.

We also need a garbage collector since photo editor also creates mad temporary memory everywhere:

• Uploaded images as blob URLs
• A fresh GPU texture for every preview frame
• ML model weights for background removal
• Full-resolution buffers at export time

Without cleanup, the tab slows down, crashes and you know,... It's not gon be well 🙏🏼. Especially on mobile Safari.

Memory work happens at three boundaries:

Part 8: Dual resolution

Even tho the allowedimageSize is capped at 20MB max , run the pipeline + applying all effects on the original image quality will feels like you're using software from 2001. So a dual resolution technique was implemented, it simply means that we will have a function to compressed the image to a smaller , display quality version of the original image.

Assume a max 20MB will be somewhere around over 80MB raw RGBA on the GPU. At 2048px ( compressed ) on the long edge, preview drops to roughly 12MB. Sliders stay responsive. Export still outputs full quality using the original file and a scale factor for text sizing.

Design choice: never run the live editing pipeline at full camera resolution. Memory cost scales with pixel count. Preview quality at 2048px is enough for editing decisions.

Part 9: Layer 1 — JavaScript heap

Blob URLs must be revoked

URL.createObjectURL(blob) allocates memory until you call revokeObjectURL. Every create needs a matching revoke in a finally block or in the store setter that replaces the old URL:

setRemovedBgImage: (url) => {
  const { removedBgImageUrl } = get();
  if (removedBgImageUrl && removedBgImageUrl !== url) {
    URL.revokeObjectURL(removedBgImageUrl);
  }
  set({ removedBgImageUrl: url });
},

export const blobToCanvas = async (blob: Blob) => {
  const url = URL.createObjectURL(blob);
  try {
    const img = await loadImage(url);
    // draw to canvas…
    return canvas;
  } finally {
    URL.revokeObjectURL(url);
  }
};

Cancel stale async work

When a user clicks preset A then quickly preset B, the load for A is aborted:

if (abortControllerRef.current) {
  abortControllerRef.current.abort();
}
abortControllerRef.current = new AbortController();

// after async load:
if (abortController.signal.aborted) return;

Same pattern for background removal and color extraction when the user moves on before work finishes.

Ignore stale results

For image loads, a request ID increments on each new upload. If photo B starts loading while photo A is still in flight, A's result is discarded when it arrives.

Guard state updates after unmount

Long category downloads check a mounted ref before updating state. Prevents warnings and wasted work when the user navigates away mid-download.

Part 10: Layer 2 — GPU and WebGL

Delete the photo texture every frame

Each preview frame creates a new image texture, runs the pipeline, then deletes it immediately:

const imageTexture = colorEngine.createTextureFromImageData(imageData, { flipY: true });
const result = colorEngine.applyEffects({ inputTexture: imageTexture, ... });
colorEngine.renderTextureToCanvas(result.outputTexture);
colorEngine.deleteTexture(imageTexture);

Preset textures stay in the GPU LRU cache. The photo texture is ephemeral. Hundreds of slider drags would leak VRAM otherwise. (See How-2-WebGL2 for the full frame lifecycle.)

Full teardown on unmount

Leaving the editor runs a full color engine cleanup: pipeline, shaders, framebuffers, textures, WebGL context. Pending animation frames are cancelled so no GPU work runs after unmount.

Context loss

When the WebGL context changes or is lost (tab backgrounded, GPU driver reset), the entire GPU preset cache clears. Textures reload from IndexedDB on next selection. Network not required.

Export reclamation

Full-resolution export allocates large off-screen buffers. After export completes, the bridge runs aggressive GPU cleanup to free textures unused for several minutes. Preview mode gets its headroom back.

Part 11: Layer 3 — Keeping persistent storage bounded

Tier 1 already covered IndexedDB LRU at 500 MB. Worth stating again: IndexedDB blobs sit on disk until explicitly read. They do not load into the JavaScript heap until a preset is selected for GPU upload. Browsing a list of fifty cached preset names costs almost nothing in RAM.

Eviction is automatic. Users never manage cache size manually.

Part 12: Keeping memory out of React

State stores hold primitives, IDs, URLs, and small objects. They do not hold:

• Image blobs
• ArrayBuffers
• WebGL textures
• Base64 strings of full photos

Undo/redo stores up to 30 deep cloned JSON snapshots of slider values and text layout. Old entries drop off automatically. References to live objects never enter the history stack.

When a user removes their photo, cleanup runs in order: history cleared, text-behind state reset (revoking blob URLs), color engine torn down, preset selection cleared, effects reset to defaults. URLs are revoked only after nothing still references them.

Part 13: Platform constraints

iOS Safari export: large PNG exports can out-of-memory on iOS. Export falls back to JPEG at 0.92 quality on that platform.

iOS PWA fonts: document.fonts.load() can crash standalone Safari. Font loading is wrapped with a timeout so export does not hang or crash waiting for a font that never resolves.

If you're building kind of a same thing, consider these optimization tricks.

Part 14: Failure handling

The theme: fail safely, never corrupt cache state, never mark success when work is incomplete.

System design choices (summary)

Here are the final summary of the system design choices:

Closing thought

If you're building something similar, It's worth to consider these architectures. Especially if you're want to optimize your app's performance in the browser . Thanks you for your time !