A new optimization in Chrome improves how IndexedDB data is stored on disk. This enhancement focuses on how large values are managed within Chrome's storage system, specifically by introducing compression to certain files stored on disk. The article summarizes the key points of this update.
Chrome's storage mechanism before version 129
Chrome uses LevelDB to store IndexedDB data on disk. LevelDB is a fast key-value storage library, but it struggles when individual values exceed the size of a database page. To address this, since 2017, Chrome has stored values larger than a single page as plain files on the disk alongside the database file.
Storage from version 129
Compression of large values
Unlike the contents stored directly within the LevelDB database, which are compressed before being written to disk and decompressed after being read, the large files stored as plain files were not compressed. With the new update, Chrome will now compress these large files using the Snappy real-time compression library, resulting in significant space savings. This is especially effective for structured data, such as large arrays of JavaScript values, XML, or JSON. It's not effective for media files which are large but are already compressed, or, less common, if the site already does its own zipping and unzipping.
Impact on storage efficiency
If the compression reduces the file size to below the size of a LevelDB page, the data is moved back into LevelDB. This change not only saves space but also reduces disk I/O operations, improving overall performance.
Snappy compression algorithm
The Chrome team chose Snappy for compression because it optimizes for speed rather than maximum compression. It's fast enough that it introduces no measurable performance penalty during compression or decompression.
Performance improvements
The compression and decompression are handled within the renderer process, part of Chrome's multiprocess architecture. This reduces the size of messages sent to the browser process, leading to further performance improvements. Synthetic benchmarks have shown that this update can make some operations two to three times faster than before due to reduced IPC (Inter-Process Communication) and disk I/O. In practice, in the Chrome team's measurements, 1 MB structured data payloads are delivered from disk to page in about a quarter the time (with high variability based on device hardware and system activity).
Developer and user impact
These changes are entirely transparent to both developers and users. However, users may notice improvements in performance and reductions in storage usage.
- Space savings: The compression is only applied to new data stored after this update. The space savings can be indirectly observed using web APIs like
navigator.storage.estimate()or in the Storage section in Chrome DevTools under the Application panel. - Testing the feature: Developers can test this behavior in pre-release versions of Chrome (pre-129) by enabling the feature with the flag:
--enable-features="IndexedDBCompressValuesWithSnappy".
This update enhances Chrome's efficiency in managing large IndexedDB values, offering both space and time savings without compromising performance, marking a significant improvement in how data is stored and accessed in the browser. By understanding these changes, developers can profit from the ongoing efforts to optimize Chrome's performance and storage mechanisms, ensuring a smoother and more efficient user experience.
Acknowledgements
This document was reviewed by Evan Stade and Rachel Andrew.