macula-neuroevolution-esdb

Bridge library between macula-neuroevolution lineage tracking and erl-esdb-gater.

Overview

This library provides an implementation of the neuroevolution_lineage_events behaviour using erl-esdb-gater as the event store backend. It enables persistent genealogy tracking for evolved neural networks.

Performance Design

Lineage tracking NEVER blocks the evolution loop:

persist_event/persist_batch return immediately (fire-and-forget)
I/O happens asynchronously in spawned processes
Errors are logged, not propagated
Under extreme load, events may be lost (acceptable trade-off)

Architecture

macula-neuroevolution          macula-neuroevolution-esdb         erl-esdb-gater
+---------------------+       +------------------------+       +-----------------+
| neuroevolution_     |       | esdb_lineage_backend   |       | esdb_gater_api  |
| lineage_events      |<------| (fire-and-forget)      |------>| append_events   |
| (behaviour)         |       |     spawn -> write     |       | (async)         |
+---------------------+       +------------------------+       +-----------------+

Installation

Add to your rebar.config:

{deps, [
    {macula_neuroevolution_esdb, "~> 0.1.0"}
]}.

Usage

Initialize the Backend

Config = #{store_id => my_lineage_store},
{ok, State} = esdb_lineage_backend:init(Config).

Persist Lineage Events (Fire-and-Forget)

%% Single event - returns immediately
Event = #{
    event_type => offspring_born,
    individual_id => <<"ind-001">>,
    parent_ids => [<<"ind-000">>],
    generation => 1
},
ok = esdb_lineage_backend:persist_event(Event, State).

%% Batch of events - returns immediately
Events = [
    #{event_type => fitness_evaluated, individual_id => <<"ind-001">>, fitness => 0.85},
    #{event_type => mutation_applied, individual_id => <<"ind-001">>, mutation_type => weight_perturb}
],
ok = esdb_lineage_backend:persist_batch(Events, State).

Read Events (For Recovery Only)

%% WARNING: This blocks! Only use for recovery/replay, not during evolution.
StreamId = <<"individual-ind-001">>,
Opts = #{from => 0, limit => 100, direction => forward},
{ok, Events} = esdb_lineage_backend:read_stream(StreamId, Opts, State).

Subscribe to Events

%% Subscribe to individual&#39;s events (for projections)
StreamId = <<"individual-ind-001">>,
ok = esdb_lineage_backend:subscribe(StreamId, self(), State).

%% Receive events
receive
    {lineage_event, StreamId, Event} ->
        handle_event(Event)
end.

Stream Routing

Events are automatically routed to streams based on entity type:

Event Types	Stream Pattern
Birth, death, fitness, mutations	`individual-{id}`
Speciation, lineage divergence	`species-{id}`
Generation, capacity events	`population-{id}`
Coalition lifecycle	`coalition-{id}`

Supported Events

Individual Events

offspring_born, pioneer_spawned, clone_produced, immigrant_arrived
individual_culled, lifespan_expired, individual_perished
individual_matured, fertility_waned
fitness_evaluated, fitness_improved, fitness_declined, champion_crowned
mutation_applied, neuron_added, neuron_removed
connection_added, connection_removed, weight_perturbed
knowledge_transferred, skill_imitated, behavior_cloned
weights_grafted, structure_seeded, mentor_assigned, mentorship_concluded
mark_acquired, mark_inherited, mark_decayed

Species Events

lineage_diverged, species_emerged, lineage_ended, lineage_merged

Population Events

generation_completed, population_initialized, population_terminated
stagnation_detected, breakthrough_achieved
carrying_capacity_reached, catastrophe_occurred

Coalition Events

coalition_formed, coalition_dissolved, coalition_joined

Dependencies

macula_neuroevolution ~> 0.18.2
erl_esdb_gater ~> 0.6.4

License

Apache License 2.0