libvpx Memory Allocation for vpx_image_t

This article explains how the libvpx library manages memory allocation and deallocation for the vpx_image_t structure, which is used to represent raw video frames in VP8 and VP9 codecs. We will cover the differences between library-managed and developer-managed memory, look at the core functions involved in this process, and detail how memory cleanup is handled to prevent leaks.

The vpx_image_t Structure

The vpx_image_t structure acts as a descriptor for a video frame, holding metadata such as width, height, color format, stride, and pointers to the individual color planes (Y, U, V). However, the structure itself does not automatically contain the buffer space for the actual pixel data. The library handles the allocation of this descriptor and its underlying pixel buffers using two primary approaches: internal allocation and external wrapping.

Internal Allocation with vpx_img_alloc

When you need libvpx to handle the memory allocation for both the image descriptor and the raw pixel buffers, you use the vpx_img_alloc function:

vpx_image_t *vpx_img_alloc(vpx_image_t *img, vpx_img_fmt_t fmt, unsigned int d_w, unsigned int d_h, unsigned int align);

Descriptor Allocation: If you pass NULL as the first argument (img), libvpx dynamically allocates a vpx_image_t struct on the heap and sets an internal flag (self_allocd = 1). If you pass a pointer to an existing, pre-allocated struct, it uses that instead and leaves self_allocd as 0.
Buffer Calculation and Allocation: The function calculates the required memory size based on the specified color format (fmt), width (d_w), height (d_h), and memory alignment boundaries (align). It then allocates a single contiguous block of memory to store the pixel data for all planes.
Plane Setup: The pointer to this contiguous block is stored in img->img_data, and the internal flag img_data_owner is set to 1, indicating the structure owns this memory buffer. libvpx then calculates the correct offsets for each color plane (Y, U, and V) and populates the img->planes array and img->stride array accordingly.

External Memory Wrapping with vpx_img_wrap

If your application already manages its own frame buffers (for example, inside a custom media pipeline or GPU texture wrapper), you can use vpx_img_wrap to map a vpx_image_t descriptor to existing memory:

vpx_image_t *vpx_img_wrap(vpx_image_t *img, vpx_img_fmt_t fmt, unsigned int d_w, unsigned int d_h, unsigned int align, unsigned char *img_data);

This function behaves similarly to vpx_img_alloc but does not allocate a new pixel buffer. Instead, it assigns the passed img_data pointer to img->img_data and sets img_data_owner to 0. This tells libvpx that it does not own the pixel data and should not attempt to free it.

Memory Deallocation and Cleanup

To free resources allocated by either of these methods, you must call vpx_img_free:

void vpx_img_free(vpx_image_t *img);

When this function is called, it performs cleanup based on the ownership flags established during allocation:

Pixel Data Cleanup: It checks the img_data_owner flag. If it is non-zero (meaning the buffer was allocated via vpx_img_alloc), the function releases the contiguous pixel buffer pointed to by img->img_data using standard system deallocation. If img_data_owner is zero, the pixel buffer is left untouched.
Descriptor Cleanup: It checks the self_allocd flag. If the vpx_image_t struct itself was dynamically allocated by the library, vpx_img_free deallocates the struct. If you passed a pre-allocated struct to the setup functions, the struct memory remains intact.