Maintenance

Online Cache Maintenance

Because MemDb is log-structured, writes accumulate stale and deleted records in the cache over time. In a long-running instance this memory is never automatically reclaimed — PurgeCache does that without closing the database:

(int stale, int deleted) purged = db.PurgeCache();

PurgeCache is thread-safe — it can be called while reads and writes are in progress; concurrent operations are briefly blocked until the purge completes.

How often to call it depends on write volume. A reasonable starting point is a periodic background task (e.g., hourly, or after a threshold number of writes). Use ResolveStatistics with Stats.StaleCount | Stats.DeletedCount to measure accumulation.

PurgeCache is a no-op in ReadOnly and AppendOnly modes — neither mode produces stale or deleted records.

Defragmentation

Defragmentation compacts the data files by removing stale and deleted records. It is an offline operation — the target database must be closed before calling it. Other open databases are unaffected.

(int stale, int deleted) result = MemDb.Defrag("people");

When ArchiveOnDefrag is configured, all stale and deleted records removed during defrag are packed into a zip archive in the configured directory rather than discarded. The zip file is named htl.{datasetName}.zip. Each defrag run adds a timestamped file set to it:

• htl.{yyyyMMdd.HHmm.ss.fff}.{datasetName}.db.bak
• htl.{yyyyMMdd.HHmm.ss.fff}.{datasetName}.map.bak

Repeated defrag runs accumulate file sets in the same zip, building a complete archive history.

MemDb.ConfigureFor<Person>("people", _dbDirectory)
    .SerializeWith(() => new PersonSerializer())
    .EncryptWithPassword(() => "SuperSecureEncryptionPasswordGoesHere.")
    .ArchiveOnDefrag(Path.Combine(_dbDirectory, "archives"))
    .Register();

Point-in-Time Restore

With ArchiveOnDefrag configured, the archive zip and the live .db and .map files together account for every version of every record, enabling a restore to any historical timestamp.

The archive is written using the same serializer and encryption as the original database — the restored database must be opened with the same configuration (serializer, encryption key, and any other settings must match).

If no defrag has ever run, no archive zip exists. MemDb.Restore handles this transparently — the restore runs against the live .db and .map files, scoped to whatever history they still contain.

Restore preconditions

MemDb.Restore enforces several preconditions and throws if any are violated:

In the restored database, every retained record is written with RecordState.Fresh. The historical state transitions (which versions were stale or deleted at the restore point) are used to select which records to include but are not preserved on the restored pointers.

MemDb.ConfigureFor<Person>("people", _dbDirectory)
    .SerializeWith(() => new PersonSerializer())
    .CloneWith(() => new PersonCloner())
    .EncryptWithPassword(() => "SuperSecureEncryptionPasswordGoesHere.")
    .ArchiveOnDefrag(Path.Combine(_dbDirectory, "archives"))
    .Register();

// 1. Capture a restore-point timestamp
//    MemDb.Restore takes a long timestamp produced by DateTime.ToBinary() — always UTC.
long timestamp;
using (var db = MemDb.Open<Person>("people"))
{
    // ... operations up to the desired restore point ...
    db.Flush();
    timestamp = DateTime.UtcNow.ToBinary();
    // ... operations after the desired restore point ...
}

// 2. (Optional) Defrag to roll the live files into the archive zip.
//    Skip this step if you want to restore using only the live files.
MemDb.Defrag("people");

// 3. Restore to a new directory at the captured timestamp
MemDb.Restore("people", timestamp, Path.Combine(_dbDirectory, "restore"));

// 4. Re-configure and open the restored database
MemDb.RemoveConfigurationFor("people");
MemDb.ConfigureFor<Person>("people", Path.Combine(_dbDirectory, "restore"))
    .SerializeWith(() => new PersonSerializer())
    .CloneWith(() => new PersonCloner())
    .EncryptWithPassword(() => "SuperSecureEncryptionPasswordGoesHere.")
    .ArchiveOnDefrag(Path.Combine(_dbDirectory, "archives"))
    .Register();

// this opened db will be in the exact of the time of the timestamp.
using (var db = MemDb.Open<Person>("people"))
{
    var records = db.FindAll(p => p.LastName == "Marsh");
}

MemDb.RemoveConfigurationFor(datasetName) is required here because the restored database lives at a new directory — the configuration must be re-registered to point at the restored files. The dataset must be disposed before its configuration can be removed.

Reading the Archive Directly

MemDb.ReadArchive<T> is the lower-level cousin of Restore. Rather than producing a new database at a chosen timestamp, it yields every archived record version as an enumerable stream — for audit trails, change history queries, or any case where you want to inspect archived state without materializing a full restored dataset.

foreach (MemDbArchivedRecord<Person> rec in MemDb.ReadArchive<Person>("people"))
{
    Console.WriteLine($"{rec.Id} {rec.State} at {DateTime.FromBinary(rec.StateSetAt)}");
    // rec.Value is the deserialized record at the version captured in the archive.
}

Each yielded MemDbArchivedRecord<T> exposes:

ReadArchive requires ArchiveOnDefrag to be configured on the dataset (throws InvalidOperationException otherwise) and throws ArgumentException if no configuration is registered for the name. The stream is yield-returned — records are deserialized one at a time as the enumerator advances, so memory usage stays bounded even for very large archives.