memstore

Simple memory database for Erlang application using MVCC to store the data.

Design

Data are stored in an ETS table, writes are collected in a simple ERLANG process to allows atomic batch. Reads are none blocking and directly access to the table

Active version sand running databases snapshots are maintained via another Erlang OTP gen_server. Snapshots allows the user to maintain a stable view of the database during the queries. Until a snapshot or an iterator is released deleted keys/values associated to this database version won't be deleted.

Garbage collection (the removal of old versions) is done in background.

Basic Operations

Open a simple database

Db = test,
Options = [],


%% open the database
ok = memstore:open(Db, Options),

%% try to read an write a key/value

Key = a,
Value = b,

%% value is not yet created, fetching it return not_found
not_found = memstore:get(Db, Key),

%% create the key
ok = memstore:put(Db, Key, Value),

%% fetch the value for this key
{ok, b} = memstore:get(Db, Key),

%% delete the Key
ok = memstore:delete(Db, Key),
not_found = memstore:get(Db, Key),

%% close the database
ok = memstore:close(Db).

Atomic updates

ok = memstore:open(test, []),
ok = memstore:write_batch(test,
[
  {put, a, b},
  {put, c, d}
]
),
{ok, b} = memstore:get(test, a),
{ok, d} = memstore:get(test, c),
ok = memstore:write_batch(test,
[
  {delete, a},
  {delete, c}
]
),
not_found = memstore:get(test, a),
not_found = memstore:get(test, c),
ok = memstore:close(test).

Snapshots

Snapshots provide consistent read-only views over the entire state of the key-value store.

ok = memstore:open(test, []),
not_found = memstore:get(test, a),
ok = memstore:put(test, a, b),
{ok, b} = memstore:get(test, a),
Snapshot = memstore:new_snapshot(test),
1 = memstore:get_snapshot_sequence(Snapshot),
ok = memstore:put(test, c, d),
{ok, b} = memstore:get(test, a, [{snapshot, Snapshot}]),
not_found = memstore:get(test, c, [{snapshot, Snapshot}]),
{ok, d} = memstore:get(test, c),
ok = memstore:close(test).

Note that when a snapshot is no longer needed, it should be released using the memstore:release_snapshot/1 function. This allows the implementation to get rid of state that was being maintained just to support reading as of that snapshot.

Iteration

The following example shows how to iterate all the keys in the database

%% create some keys
ok = memstore:open(test, []),
ok = memstore:put(test, <<"a">>, <<"x">>),
ok = memstore:put(test, <<"b">>, <<"y">>),
{ok, <<"x">>} = memstore:get(test, <<"a">>),

%% create an iterator
{ok, I} = memstore:iterator(test, []),

%% move to an empty key 

{ok, <<"a">>, <<"x">>} =  memstore:iterator_move(I, {seek, <<>>}),

%% move forward
{ok, <<"b">>, <<"y">>} = memstore:iterator_move(I, next),

%% move backward
{ok, <<"a">>, <<"x">>} = memstore:iterator_move(I, prev),

%% close the iterator
ok = memstore:iterator_close(I),


ok = memstore:close(test).

Build

$ rebar3 compile