Monday, February 10, 2025

Memory cost of image conversion

In my web app, users can upload images as PNG, JPEG etc. I convert them to WebP because it uses much less space on disk and loads faster in web page requests. However, conversion uses raw pixels, so the memory (RAM) required is almost irrelevant to the original file size because file size depends on the compression of raw pixels. The memory needed is primarily determined by: Width * Height * Bytes per pixel. 

A PNG needs 3 bytes for RGB and 1 byte for alpha = 4 bytes per pixel. For a 5637x5637 PNG with RGBA colors, we need 5637 * 5637 * 4 bytes per pixel = ~127MB just for the uncompressed pixel data alone. The file size of that PNG is 6MB.

JPEG files typically don't have an alpha channel, so they need 3 bytes per pixel. For a 5637x5637 JPEG, memory needed would be 5637 * 5637 * 3 = ~95MB. The actual file size of the JPEG could range from:

  • High quality (90%): 2-8MB
  • Medium quality (70%): 1-4MB
  • Low quality (50%): 500KB-2MB
When setting upload limits for image processing, you should primarily consider the image dimensions (megapixels) rather than the file size (megabytes). Here's why:

  1. A user could upload a highly compressed 2MB PNG that's 10000x10000 pixels - this would need ~400MB RAM to process.
  2. Another user could upload a poorly compressed 10MB JPEG that's 1000x1000 pixels - this would only need ~3MB RAM to process.

Considering that current mobile cameras can typically capture 16MP images, the decompressed image would require approximately 16e6 pixels × 4 bytes/pixel / 1024 / 1024 = 61MB of memory. Additional couple of MB memory may be needed as a buffer for libraries like PHP GD, which might use it for copying bytes. 

Here are details from my web app of processing a 16MP JPEG with GD functions (total 172MB, calculated with memory_get_peak_usage):

  • imagecreatefromjpeg: +68MB
  • imagewebp: 130MB (+62MB)
  • resizeWebP:
    • imagecreatefromwebp: 140MB (+10MB)
    • imagecreatetruecolor: 156MB (+16MB)
    • imagewebp: 172MB (+16MB)

To be safe, allocating 200MB of memory would be prudent. If your server has 3GB total RAM, it means that your web app can handle at most 3GB/200MB ≈ 15 concurrent image conversions.

Converting images to WebP saves disk space and speeds up the web app in the long run, but it consumes a lot of RAM in the short run.

No comments:

Post a Comment