Ecto Schema Store

This library is used to create customizable data stores for individual ecto schemas.

Getting Started

With the following schema:

defmodule Person do
  use Ecto.Schema, :model

  schema "people" do
    field :name, :string
    field :email, :string

    timestamps
  end

  def changeset(model, params) do
    model
    |> cast(params, [:name, :email])
  end
end

You can create a store with the following:

defmodule PersonStore do
  use EctoSchemaStore, schema: Person, repo: MyApp.Repo
end

Querying

The following functions are provided in a store for retrieving data.

Sample Queries:

# Get all records in a table.
PersonStore.all

# Get all records fields that match the provided value.
PersonStore.all %{name: "Bob"}
PersonStore.all %{name: "Bob", email: "bob@nowhere.test"}
PersonStore.all name: "Bob", email: "bob@nowhere.test"

# Return a single record.
PersonStore.one %{name: "Bob"}
PersonStore.one name: "Bob"

# Return a specific record by id.
PersonStore.one 12

# Refresh
record = PersonStore.one 12
record = PersonStore.refresh record

# Preload after query
PersonStore.preload_assocs record, :field_name
PersonStore.preload_assocs record, :all
PersonStore.preload_assocs record, [:field_name_1, :field_name_2]

# To Map
record = PersonStore.to_map PersonStore.one 12

Options:

# Get all records in a table.
PersonStore.all %{}, preload: :field_name

# Get all records fields that match the provided value.
PersonStore.all %{name: "Bob"}, preload: [:field_name_1, :field_name_2]

# Return a single record.
PersonStore.one %{name: "Bob"}, preload: :all

# Return a specific record by id.
PersonStore.one 12, preload: :all, to_map: true

# Order by
PersonStore.all %{}, order_by: :email
PersonStore.all %{}, order_by: [:name, :email]
PersonStore.all %{}, order_by: [asc: :name, desc: :email]

Filter Operators

Stores support a special syntax for changing the comparison operator in the passed filter map or keyword list.

Operators:

PersonStore.all %{name: nil}
PersonStore.all %{name: {:==, nil}}
PersonStore.all name: {:!=, nil}
PersonStore.all name: {:!=, "Bob"}
PersonStore.all name: {:in, ["Bob"]}, email: "bob@nowhere.test"

Editing

The following functions are provided in a store for editing data.

Options:

Sample Usage:

# Using Map
bob = PersonStore.insert! %{name: "Bob", email: "bob@nowhere.test"}
bob = PersonStore.update! bob, %{email: "bob2@nowhere.test"}
PersonStore.delete bob

# Using Keyword List
bob = PersonStore.insert! name: "Bob", email: "bob@nowhere.test"
bob = PersonStore.update! bob, email: "bob2@nowhere.test"

# Updates/deletes can also occur by id.
PersonStore.update! 12, %{email: "bob2@nowhere.test"}
PersonStore.delete 12

# Update a single record based upon a query.
# This will only update the first record retrieved by the database, it is not meant
# to be an update_all style function. Instead it is useful if you use another id to reference
# the record that is not the primary id.
PersonStore.update %{name: "bob"}, email: "otheremail@nowhere.test"

# Update or create
attributes_to_update =
query = %{name: "Bob"}
PersonStore.update_or_create attributes_to_update, query
PersonStore.update_or_create! %{email: "new@nowhere.test"}, name: "Bob"

Changesets

The insert and update functions by default use a changeset on the provided schema name :changeset for inserting and updating. This can be overridden and a specific changeset name provided.¬

bob = PersonStore.insert! %{name: "Bob", email: "bob@nowhere.test"}, changeset: :insert_changeset
bob = PersonStore.insert! [name: "Bob", email: "bob@nowhere.test"], changeset: :insert_changeset
bob = PersonStore.update! bob, %{email: "bob2@nowhere.test"}, changeset: :update_changeset
bob = PersonStore.update! bob, [email: "bob2@nowhere.test"], changeset: :update_changeset
bob = PersonStore.update! bob, %{email: "bob2@nowhere.test"}, changeset: :my_other_custom_changeset

It is also possible to pass a function reference in as the changeset.

def my_changeset(model, params) do
  model
  |> cast(params, [:name, :email])
  |> validate_required([:name])
end

insert [name: "Bob"], changeset: &my_changeset/2

Validate Insert/Update Params Only

Sometimes it is convienent to check if a changeset will pass before actually attempting to insert or update a record. There are two validation functions that can be used to check the changesets the same way they would be checked on an insert or update action.

Options:

# Checking an insert

with :ok <- PersonStore.validate_insert(%{name: "Bob"}, changeset: :my_changeset, errors_to_map: :person) do
  # Perform some action on validation, the :error tuple is return directly from the with statement.
end

# Chacking an update

existing_record = Person.Store.insert_fields! name: "Will"

with :ok <- PersonStore.validate_update(existing_record, %{name: "Bob"}, changeset: :my_changeset, errors_to_map: :person) do
  # Perform some action on validation, the :error tuple is return directly from the with statement.
end

Preconfigured Options

Each of the insert and update functions takes a set of options. It may be inconvienent to set these options every time one of theses functions is used. A duplicate set of these funcitons can be generated with a predefined set of options.

The preconfigure function will generate a customized variation of the following store functions:

defmodule PersonStore do
  use EctoSchemaStore, schema: Person, repo: MyApp.Repo

  # (custom_name, options)
  preconfigure :for_api, changeset: :my_changeset, errors_to_map: true
end

PersonStore.insert_for_api! name: "Bob", email: "bob@nowhere.test"
PersonStore.insert_for_api! name: "Will", email: "will@nowhere.test"
PersonStore.insert_for_api! name: "Carl", email: "carl@nowhere.test"
PersonStore.insert_for_api! name: "Mike", email: "mike@nowhere.test"

# Similar statements without preconfigure would have looked like this.
PersonStore.insert! [name: "Bob", email: "bob@nowhere.test"], changeset: :my_changeset, errors_to_map: true
PersonStore.insert! [name: "Will", email: "will@nowhere.test"], changeset: :my_changeset, errors_to_map: true
PersonStore.insert! [name: "Carl", email: "carl@nowhere.test"], changeset: :my_changeset, errors_to_map: true
PersonStore.insert! [name: "Mike", email: "mike@nowhere.test"], changeset: :my_changeset, errors_to_map: true

Update/Delete All

Ecto allows batch updates and deletes by passing changes directly to the database to effect multiple records.

# Update records by query
values_to_set = [name: "Generic Name"]
query_params = [email: {:like, "%@test.dev"}]

PersonStore.update_all values_to_set, query_params

# Update all records
PersonStore.update_all name: "Generic Name"

# Delete records by query
query_params = [email: {:like, "%@test.dev"}]

PersonStore.delete_all query_params

# Delete All Records on a Table
PersonStore.delete_all

References

The internal references to the schema and the provided Ecto Repo are provided as convience functions.

Custom Actions

Since a store is just an ordinary module, you can add your actions and build off private APIs to the store. For convience Ecto.Query is already fully imported into the module.

A store is provided the following custom internal API:

defmodule PersonStore do
  use EctoSchemaStore, schema: Person, repo: MyApp.Repo

  def get_all_ordered_by_name do
    build_query!
    |> order_by([:name])
    |> all
  end

  def find_by_email(email) do
    %{email: email}
    |> build_query!
    |> order_by([:name])
    |> all
  end

  def get_all_ordered_by_name_using_ecto_directly do
    query = from p in schema,
            order_by: [p.name]

    repo.all query
  end
end

Schema Field Aliases

Sometimes field names get changed or the developer wishes to have an alias that represents another field. These work for both querying and editing schema models.

defmodule PersonStore do
  use EctoSchemaStore, schema: Person, repo: MyApp.Repo

  alias_fields email_address: :email
end

PersonStore.all %{email_address: "bob@nowhere.test"}
PersonStore.update! 12, %{email_address: "bob@nowhere.test"}

Filter or Params Map/Keyword List

Many of the API calls used by a store take a map of fields as input. Normal Ecto requires param maps to be either all atom or string keyed but not mixed. A schema store will convert every map provided into atom keys before aliasing and passing on to Ecto. This means you can provide a mixture of both. This will allow a developer to combine multiple maps together and not worry about what kind of keys were used.

However, if you provide the same value twice as both an atom and string key then only one will be used.

PersonStore.insert! %{"name" => "Bob", email: "bob2@nowhere.test"}

Transaction

Under normal circumstances, the regular Ecto Repo transaction function can be used normally; if you would like to use the ! functions in a store you will need to use the EctoSchemaStore.transaction or the store specific transaction function to return a friendlier error tuple instead of passing the throw up the chain.

{:error, changeset_or_map} =
  PersonStore.transaction fn ->
    PersonStore.insert! age: "bad value"
  end

# Manual rollback
{:error, message} =
  PersonStore.transaction fn ->
    PersonStore.repo.rollback(message)
  end

# Using directly
{:error, changeset_or_map} =
  EctoSchemaStore.transaction MyApp.Repo, fn ->
    PersonStore.insert! name: "Bob"
    OtherStore.update! field: "bad value"
  end

Although the transaction function can be called on a store module, it only proxies to the EctoSchemaStore module passing in the repo associated with the store. The call can be used with any combination of stores or the Ecto Repo itself.

Generator Factories

Sometimes, such as in unit testing, a developer would like to create a common predefined data structure to work against. Generator Factories provide a composable method to create such a data structure. There two types of factory methods that can be used. One to create a common factory base for the store and another to append upon that base if it is used. The base factory is optional and not required.

If a default factory does not exist then the defaults will be used as defined in the Struct ecto will generate for the schema.

Macros:

Functions:

defmodule PersonStore do
  use EctoSchemaStore, schema: Person, repo: MyApp.Repo

  # Create the common base for starting factory generated record.
  factory do
    %{
      name: "Test Person",
      email: "test@nowhere.test"
    }
  end

  # Create a factory segment to override the base factory values.
  factory bob do
    %{
      name: "Bob"
    }
  end

  factory karen do
    %{
      name: "Karen"
    }
  end
end

# Sample Usage

# Using just the base
%Person{name: "Test Person", email: "test@nowhere.test"} = PersonStore.generate!
{:ok, %Person{name: "Test Person", email: "test@nowhere.test"}} = PersonStore.generate

# Using bob Factory
%Person{name: "Bob", email: "test@nowhere.test"} = PersonStore.generate! :bob

# Using karen factory
%Person{name: "Karen", email: "test@nowhere.test"} = PersonStore.generate! :karen

# Using multiple factories, each is overlayed in order.
%Person{name: "Karen", email: "test@nowhere.test"} = PersonStore.generate! [:bob, :karen]
%Person{name: "Bob", email: "test@nowhere.test"} = PersonStore.generate! [:karen, :bob]

# Manually setting values
%Person{name: "Test Person", email: "ignore@nowhere.test"} = PersonStore.generate_default! email: "ignore@nowhere.test"
%Person{name: "Bob", email: "bob@nowhere.test"} = PersonStore.generate! :bob, email: "bob@nowhere.test"
%Person{name: "Karen", email: "karen@nowhere.test"} = PersonStore.generate! [:bob, :karen], email: "karen@nowhere.test"

Event Announcements

A store supports the concept of an event through the Event Queues library on Hex. Event Queues must be included in your application and each queue and handler added to your application supervisor. Visit the instructions at (https://hexdocs.pm/event_queues) for more details.

Events:

Macros:

defmodule PersonStore do
  use EctoSchemaStore, schema: Person, repo: MyApp.Repo

  create_queue

  announces events: [:after_delete, :after_update]
end

defmodule PersonEventHandler do
  use EventQueues, type: :handler, subscribe: PersonStore.Queue

   def handle(%{category: Person, name: :after_delete, data: data}) do
    IO.inspect "Delete #{data.schema} id: #{data.id}"
   end
   def handle(%{category: Person, name: :after_update, data: data}) do
    IO.inspect "Changed #{data.schema} id: #{data.id}"
   end
   def handle(_), do: nil
end

A store can also use existing queues (1 or more):

defmodule Queue1 do
  use EventQueues, type: :queue
end

defmodule Queue2 do
  use EventQueues, type: :queue
end

defmodule PersonStore do
  use EctoSchemaStore, schema: Person, repo: MyApp.Repo

  announces events: [:after_delete, :after_update],
           queues: [Queue1, Queue2]
end

defmodule PersonDeleteEventHandler do
  use EventQueues, type: :handler, subscribe: Queue1

   def handle(%{category: Person, name: :after_delete, data: data}) do
    IO.inspect "Delete #{data.schema} id: #{data.id}"
   end
   def handle(_), do: nil
end

defmodule PersonUpdateEventHandler do
  use EventQueues, type: :handler, subscribe: Queue2

   def handle(%{category: Person, name: :after_update, data: data}) do
    IO.inspect "Changed #{data.schema} id: #{data.id}"
   end
   def handle(_), do: nil
end

For certain events, actions can be taken before the action is to take place. In order to continue, an event must be resubmitted after handling the initial event to tell the Store to continue or cancel the action originally submitted.

defmodule PersonStore do
  use EctoSchemaStore, schema: Person, repo: MyApp.Repo

  create_queue

  announces events: [:before_update]
end

defmodule PersonEventHandler do
  use EventQueues, type: :handler, subscribe: PersonStore.Queue

   def handle(%{category: Person, name: :before_update} = event) do
    # Perform some action

    if success do
      event.data.originator.continue event
    else
      event.data.originator.cancel event, "Something failed"
    end

    # The event must be continued or canceled, otherwise an error will be returned back to
    # the original function calling the store. Even if the operation was successful.
   end
   def handle(_), do: nil
end

Proxy Store Functions Through Schema

If you happen to come from other programming environments, you may have used an ORM that places store style functions directly on the entity or what in the case of Ecto is called a schema. EctoSchemaStore provides a modules that will allow you to include some of the store functions directly into the schema module.

defmodule Person do
  use Ecto.Schema, :model
  use EctoSchemaStore.Proxy, store: PersonStore

  schema "people" do
    field :name, :string
    field :email, :string

    timestamps
  end

  def changeset(model, params) do
    model
    |> cast(params, [:name, :email])
  end
end

defmodule PersonStore do
  use EctoSchemaStore, schema: Person, repo: MyApp.Repo
end

# Get all records in a table.
Person.all

# Get all records fields that match the provided value.
Person.all %{name: "Bob"}
Person.all %{name: "Bob", email: "bob@nowhere.test"}
Person.all name: "Bob", email: "bob@nowhere.test"

# Return a single record.
Person.one %{name: "Bob"}
Person.one name: "Bob"

# Return a specific record by id.
Person.one 12

# Refresh
record = Person.one 12
record = Person.refresh record

# Preload after query
Person.preload_assocs record, :field_name
Person.preload_assocs record, :all
Person.preload_assocs record, [:field_name_1, :field_name_2]

# To Map
record = Person.to_map Person.one 12

If a store is not provided, the proxy take the current schema module name and append .Store to the end of it. So the default for Person would be Person.Store.

To figure out what store is proxied into a module you can call the store function on the schema module.