Use db like you would normally, except now it efficiently reads and persists writes to IndexedDB! See the README for more information, and look at the sql.js docs for the database API. The APIs we need are only available there, but that’s fine because you shouldn’t be blocking the main thread. Lastly, this code must run in a web worker. The reason for register_for_idb is because Emscripten doesn’t properly hook up the file locking API, so we need to manually forward SQLite’s lock requests. Import initSqlJs from import ) A few notes: this currently requires my fork of sql.js but hopefully these changes will be merged in… This API is likely to change. Now, all you have to do is install absurd-sql and add some lines of code to hook it up. We use sql.js because it already has a large community and is by far the most common way to use SQL on the web. What this means is it never loads the database into memory because it only loads whatever SQLite asks for, and writes always persist. I wrote a whole filesystem layer that is aware of how SQLite reads and writes blocks, and it efficiently performs the operations correctly. To build any kind of app with it, we need the ability to write and persist.Ībsurd-sql solves this, and it works by intercepting read/write requests from SQLite and fetching and persisting them into IndexedDB (or any other persistent backend). While in-memory databases have their uses, it kneecaps SQLite into something far less useful. Once you refresh the page, all your changes are lost. It loads the entire db into memory, and only changes the in-memory data. The major problem is that you can’t persist any writes. It compiles SQLite to WebAssembly, and lets you read databases and run queries. Sql.js is already a great project for using SQLite on the web. It’s a very clever technique and it gave me the idea to research this. Anyway…Ī huge thanks to phiresky writing the article Hosting SQLite databases on Github Pages which inspired me to do this. Why? In all browsers except Chrome, IndexedDB is implemented using SQLite. This whole situation, and how well this project ended up working out, really is absurd. I ported my app to use and you can try it here. absurd-sql is a filesystem backend for sql.js that allows SQLite to read/write from IndexedDB in small blocks, just like it would a disk. I’m excited to announce absurd-sql which makes this possible. Key/value stores may have their place in large distributed systems, but wow wouldn’t it be great if we could use SQLite on the web? And it’s going to massively improve your life as a developer. I have something for you that you should reach for next time instead of all those other libraries. Every library I looked at was messy and made performance even worse (one of the most popular “fast” ones I looked at took ~45ms to just get one item!?). Maybe IDB was supposed to be low-level and you’re supposed to reach for a library to help provide better support for these features. You can only do it when opening the database, and doing so forces all other tabs to kill their database connection! Heck, you can’t even add a new “object store”, which is sort of like a table, at any point in time. You’ll have to construct your own query functionality by wiring up indexes and structuring your data in specific ways. The only function it provides is count, and the rest of the APIs just return a range of items. You’re on your own if you want to query data in IndexedDB. This makes a massive difference for what kinds of apps you can write against it. Simple operations against the database take ~10ms (and I’m being charitable!), while it’s common for me to be profiling SQLite at ~.01ms. What’s even worse, in my testing Chrome, which is by the far the major browser, has the slowest implementation. But if we really want webapps to be impressive (I do), we need a more powerful way to work with data. Sure, it might be fine for small bits of functionality. It’s the only option for something database-like that works across all browsers.Īs you attempt to build a local app, you will quickly find that it’s a poor database to build an entire app around. If you are writing a web app today, you’ll probably choose IndexedDB to store data. In this post I will explain the absurdities of the web’s storage APIs (mainly IndexedDB), show how SQLite provides a 10x perf improvement, explain all the cool tricks that make it work, and explain the locking/transactional semantics that make it robust. That means it doesn’t have to load the whole db into memory, and writes persist. The end result is absurd-sql, and it’s a persistent backend for SQLite on the web. I discovered something absurd recently and I’m very excited to tell you about it.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |