-
Introducing NULevelDB for iOS
November 14, 2011 by nulayer-brent comments
NULevelDB is an Objective-C API for Google’s leveldb key-value database. NULevelDB hides leveldb’s C++ interface while maintaining its functionality, using semantics and patterns which are more suited to Objective-C and more standard in Cocoa and iOS software.
We created a new Objective-C database API in response to performance issues with Core Data. Core Data is a powerful framework with a lot of incredible features. But those features do not always come for free, and they can create bottlenecks. And we didn’t need them. We needed a key-value database.
Alternatives
After evaluating a number of embeddable database solutions, leveldb looked like the best alternative. Other solutions that we considered include:
-
CouchDB is a document-oriented database sever. It’s not a pure key-value store. It lacks an Objective-C binding. CouchDB embedded still runs like a server: the database exposes a socket-based protocol interface on a dedicated thread, not a direct API. In addition, it’s quite large.
-
Berkeley DB, or “db” for short, has been around for a long time. A version of it even ships as part of iOS, but the API is inscrutable to those of us since spoiled by object-oriented design. Documentation is sorely lacking.
-
Vanilla SQLite is easy to use on its own, but, as with Core Data, it is designed to solve different kinds of problems.
We looked at other options, too, but either they weren’t mature enough, lacked Objective-C or at least C-friendly bindings, or had incompatible licenses.
leveldb
Although very new, leveldb is quite mature. It is reliable, fast, and powerful, and the API is clean and intuitive. It features concurrency support, built-in compression, and tight memory use. The fact that it’s written in C++ was only a minor impediment. Writing an Objective-C wrapper felt quite natural, even for someone with very little C++ experience.
The documentation and headers are your best guide, but here’s a quick overview.
You open a database with one function call, DB::Open(), passing a file path and options. This returns a status by value, and a new, opaque database object by reference.
Once you have a database, read, write and delete to your heart’s content. Put() take a key and a value, and adds an entry. Get()) take a key and returns a value. Delete() take a key and removes an entry. Keys and values are represented by a Slices: simple wrappers around data buffers. For efficiency, slices don’t copy data. Don’t delete their referenced data while the slice is still alive.
Besides basic operations, you can explore the database with the Iterator class. Ask the target database to create an Iterator on your behalf. Then use the Iterator to seek or step through keys sequentially. The sequence follows the sort order.
Sort order is determined by the Comparator class. You can make your own comparator, or use the default. Use it to find your boundary condition while iterating. Once you specify a comparator to go with a database, don’t change to another one—correct sorting and search of entries in the database requires that the comparison algorithm be consistent.
When you are finished with your database, delete it normally. But first, make sure you free any Iterators you have created, or it will crash.
leveldb has a a few more features, but they aren’t used by NULevelDB yet.
Getting NULevelDB
NULevelDB is available as source code from github.
The source includes an Xcode 3 project which builds an iOS universal static framework called NULevelDB. (For more about universal static frameworks for iOS, please see the iOS Universal Framework Template on github.)
The framework target depends on another target that builds leveldb as a static library. There is a third target for running a small set of unit tests.
To make the framework available to your application, build it and copy the built product to your application project directory, adding it to your application target at the same time. IN addition, you must add the libstdc++ library to your project. Also, add the -all_load option to your linker flags for all build configurations, or you will get countless C++ linking errors. If you wish, you can include NULevelDB in your project by building the leveldb library and adding the NULDBDB class files to your project directly. (You still need to make the same linking-related modifications to ensure your application builds and runs.)
Using NULevelDB
To create a new database from your application, instantiate a new NULDBDB object using the designated initializer:
or just use -init to use the default location and write buffer size.
The interface for NULevelDB mimics that of leveldb, with some simplifications and some additions. For our uses, defining separate classes for Iterators, Slices, Comparators and the rest has not yet been necessary. (These are areas of future work.) Instead, NULDBDB provides support for a variety of different kinds of keys and values, as well as convenience enumeration methods.
The simplest approach is to use keys and values that adopt the NSCoding protocol. Internally, NULDBDB converts objects to and from keyed archives and stores them as NSData. Then you only have to work with objects. By adopting NSCoding on your custom classes, they are immediately compatible with the database.
To add or update an entry:
To retrieve an entry:
To delete an entry:
In addition to reading and writing, you can check whether an entry already exists. In the implementation, this uses an Iterator to search for the provided key. If an Iterator is valid after a seek, it has found the key; otherwise the key does not exist.
In all cases, it’s up to the application to make sure that it knows what kind of object is being stored under any key, or to inspect the value to find out what it is. As is normal with NSCoding, if you use collections, objects held by them must also adopt the NSCoding protocol.
leveldb will store as much data as you ask it to, quickly and efficiently.
Faster Access for Common Cases
For better performance, use one of the streamlined interfaces. These reduce the number of times data needs to be copied. For example, there are methods customized for dealing with string keys and data values.
There are various enumeration methods, one for each of the API access styles. Most enumeration methods take two keys, for the start and end points, and a block that is called for each key found in the range. If the block returns NO, the method terminates early.
For example, you may wish to find the biggest object held by a group of keys in a certain range, all of which are prefixed by “MY_KEY:”. But you may wish to end the search early if you find a value bigger than some high water mark.
(Currently, the retain/release semantics are optional, but the enumeration APIs will probably be changed to avoid creating autoreleased objects, to reduce memory spikes while iterating.)
More Examples
The distribution includes some test projects for performance measurement. Along with the unit tests, these show the most common ways to use the database.
I have a fork of SDURLCache on github which uses NULevelDB. SDURLCache is a replacement for NSURLCache, which does not cache on disk in iOS.
There are some good opportunities for improving and enhancing the code with new functionality. Please fork us! We look forward to your pull requests.
blog comments powered by Disqus
-
Back
