The locking in Berkeley DB is fundamentally broken.

This isn’t something that’s bitten me recently, but talk about it did come up recently, and many BDB users are unaware of the problem. It’s especially surprising when you first encounter it because BDB is so solidly done in the rest of its implementation and the problem isn’t advertised. Of course locking will work — everything else does!

The problem is that BDB locks are bare mutexs, and no forcible release mechanism is provided. There is no way to rollback to a state before a lock was acquired. There is no way to retry. You take the lock, do your modifications, and release. Or you take the lock, do some of your modifications, and the circuit breaker trips, power is quickly restored, the server reboots, and now your BDB is hosed. You can’t take the lock because BDB thinks your old instance has it. You can’t release the lock because you don’t have it. If you could release the lock, your database is inconsistent because you can’t rollback. The only way you can do anything is to blow away the BDB environment and make a new one (suffering collateral damage when you just want to destroy a mere lock) and hope that the partial transaction you’re unleashing as a result isn’t a problem (and if it isn’t, why did you need a lock?).

So, what can you do? You can try to build your own transaction system on top of BDB, but that’s a lot of work. A lot of work, when there’s better featured databases out there already. If you were using BDB and need more, MySQL is probably good enough for you. It has partial transaction support (probably more than you want to write!), and does have a library implementation (libmysqld) so you can avoid running a server and just use data files, like in BDB. If you need full transaction support (good luck implementing that on your own), there’s always ye olde enterprise-grade PostgreSQL. Or maybe you don’t need transactions after all? Then Berkeley DB is solid as a brick.