Notion engineers sped up Notion's browser speed with WASM SQLite

🌈 Abstract

The article discusses how Notion used WebAssembly (WASM) and the SQLite database to improve the performance of their web application. It covers the technical challenges they faced, such as cross-origin isolation requirements, concurrency issues with the Origin Private File System (OPFS), and the need to balance performance gains with initial page load times. The article also describes the final architecture they implemented using a SharedWorker to manage SQLite access across tabs, and the mitigation strategies they used to address regressions.

🙋 Q&A

[01] Core technologies: OPFS and Web Workers

1. What are the core technologies used by Notion to improve performance in the browser?

• Notion used the WebAssembly (WASM) implementation of SQLite and the Origin Private File System (OPFS) to cache data on the client-side.
• They also used Web Workers to run the SQLite code in a separate thread from the main browser thread.

2. Why did Notion need to use Web Workers and OPFS?

• To persist data across sessions, the WASM SQLite library uses the OPFS, which is only available in Web Workers.
• Web Workers run in a separate thread from the main browser thread, allowing Notion to offload the SQLite processing and avoid blocking the main thread.

3. How did Notion set up the Web Worker to work with SQLite and OPFS?

• Notion's Web Worker was set up to either create a SQLite database file using OPFS or load an existing file.
• Notion then ran their existing caching code on this Web Worker, using the Comlink library to manage message passing between the main thread and the Worker.

[02] Notion's SharedWorker-powered approach

1. What was the issue with a simpler approach of having a dedicated Web Worker per tab?

• Notion initially tried a more straightforward approach of having a dedicated Web Worker per tab, with each Worker writing to the SQLite database.
• However, they encountered issues with cross-origin isolation requirements and database corruption due to concurrency problems.

2. How did Notion's final architecture using a SharedWorker work?

• Notion's final architecture used a SharedWorker to manage which tab is the "active tab" that can access the SQLite database.
• Each tab has its own dedicated Web Worker that can write to SQLite, but only the active tab's Worker is permitted to use the database.
• The SharedWorker detects when tabs are closed and selects a new active tab.
• Tabs send SQLite queries to the SharedWorker, which then redirects them to the active tab's dedicated Worker.

3. What were the benefits of Notion's SharedWorker-powered approach?

• It allowed Notion to use either the OPFS via sqlite3_vfs or the OPFS SyncAccessHandle Pool VFS variant of SQLite, without the concurrency issues they encountered with a simpler approach.
• The OPFS SyncAccessHandle Pool VFS variant, which Notion ultimately chose, did not require cross-origin isolation, making it easier to roll out to more browsers.

[03] Mitigating regressions

1. What performance regressions did Notion initially observe, and how did they address them?

• Notion observed that the initial page load was slower due to the time required to download and process the WASM SQLite library.
• To fix this, they loaded the WASM SQLite library asynchronously, so it didn't block the page load process.
• Notion also found that slower devices didn't benefit from the caching, and in fact experienced worse performance. To address this, they implemented a "race" between the SQLite and API requests, equalizing the 95th percentile of navigation time.

2. What were the key lessons Notion learned from delivering the performance improvements?

• OPFS doesn't come with graceful handling of concurrency, so developers need to design around this.
• Combining different web technologies like Web Workers, SharedWorkers, and Service Workers can be useful to overcome their individual limitations.
• Fully implementing cross-origin isolation on a sophisticated web application is challenging, especially when using third-party scripts.