Domo

Section

Build StatusMethod TDDhex.pm version

⚠️ This library generates code for structures that can bring suboptimal compilation times increased to approximately 20%

🔗 JSON parsing and validation example is in https://github.com/IvanRublev/contentful-elixir-parse-example-nestru-domo repo.

🔗 Commanded + Domo combo used in Event Sourcing and CQRS example app is in https://github.com/IvanRublev/bank-commanded-domo repo.

A library to ensure the consistency of structs modelling a business domain via their t() types and associated precondition functions.

Used in a struct's module, the library adds constructor, validation, and reflection functions. Constructor and validation functions guarantee the following at call time:

If the conditions described above are not met, the constructor and validation functions return an error.

Because precondition function associates with type the validation can be shared across all structs referencing the type.

In terms of Domain Driven Design the invariants relating structs to each other can be defined with types and associated precondition functions.

Run in Livebook

Mix.install([:domo], force: true)

Let's say that we have a LineItem and PurchaseOrder structs with relating invariant that is the sum of line item amounts should be less then order's approved limit. That can be expressed like the following:

defmodule LineItem do
  use Domo

  defstruct amount: 0

  @type t :: %__MODULE__{amount: non_neg_integer()}
end

defmodule PurchaseOrder do
  use Domo

  defstruct id: 1000, approved_limit: 200, items: []

  @type id :: non_neg_integer()
  precond(id: &(1000 <= &1 and &1 <= 5000))

  @type t :: %__MODULE__{
          id: id(),
          approved_limit: pos_integer(),
          items: [LineItem.t()]
        }
  precond(t: &validate_invariants/1)

  defp validate_invariants(po) do
    amounts = po.items |> Enum.map(& &1.amount) |> Enum.sum()
    
    if amounts <= po.approved_limit do
      :ok
    else
      {:error, "Sum of line item amounts (#{amounts}) should be <= to approved limit (#{po.approved_limit})."}
    end
  end
end

Then PurchaseOrder struct can be constructed consistently with functions generated by Domo like the following:

PurchaseOrder.new()
{:ok, %PurchaseOrder{approved_limit: 200, id: 1000, items: []}}

The constructor function takes any Enumerable as the input value:

{:ok, po} = PurchaseOrder.new(%{approved_limit: 250})
{:ok, %PurchaseOrder{approved_limit: 250, id: 1000, items: []}}

It returns the descriptive keyword list if there is an error in input arguments. And it validates nested structs automatically:

PurchaseOrder.new(id: 500, items: [%LineItem{amount: -5}])
{:error,
 [
   items: "Invalid value [%LineItem{amount: -5}] for field :items of %PurchaseOrder{}. 
    Expected the value matching the [%LineItem{}] type.
    Underlying errors:
       - The element at index 0 has value %LineItem{amount: -5} that is invalid.
       - Value of field :amount is invalid due to Invalid value -5 for field :amount 
         of %LineItem{}. Expected the value matching the non_neg_integer() type.",
   id: "Invalid value 500 for field :id of %PurchaseOrder{}. Expected the 
    value matching the non_neg_integer() type. And a true value from 
    the precondition function \"&(1000 <= &1 and &1 <= 5000)\" 
    defined for PurchaseOrder.id() type."
 ]}

The returned errors are verbose and are intended for debugging purposes. See the User facing error messages section below for more options.

And manually updated struct can be validated like the following:

po 
|> Map.put(:items, [LineItem.new!(amount: 150)]) 
|> PurchaseOrder.ensure_type()
{:ok, %PurchaseOrder{approved_limit: 200, id: 1000, items: [%LineItem{amount: 150}]}}

Domo returns the error if the precondition function attached to the t() type that validates invariants for the struct as a whole fails:

updated_po = %{po | items: [LineItem.new!(amount: 180), LineItem.new!(amount: 100)]}
PurchaseOrder.ensure_type(updated_po)
{:error, [t: "Sum of line item amounts should be <= to approved limit"]}

Getting the list of the required fields of the struct that have type other then nil or any is like that:

PurchaseOrder.required_fields()
[:approved_limit, :id, :items]

See the Callbacks section for more details about functions added to the struct.

User facing error messages

It's possible to attach error messages to types with the precond macro to display them later to the user. To filter such kinds of messages, pass the maybe_filter_precond_errors: true option to Domo generated functions like that:

defmodule Book do
  use Domo

  defstruct [:title, :pages]

  @type title :: String.t()
  precond title: &(if String.length(&1) > 1, do: :ok, else: {:error, "Book title is required."})

  @type pages :: pos_integer()
  precond pages: &(if &1 > 2, do: :ok, else: {:error, "Book should have more then 3 pages. Given (#{&1})."})
  
  @type t :: %__MODULE__{title: nil | title(), pages: nil | pages()}
end

defmodule Shelf do
  use Domo
  
  defstruct books: []
  
  @type t :: %__MODULE__{books: [Book.t()]}
end

defmodule PublicLibrary do
  use Domo
  
  defstruct shelves: []
  
  @type t :: %__MODULE__{shelves: [Shelf.t()]}
end

library = struct!(PublicLibrary, %{shelves: [struct!(Shelf, %{books: [struct!(Book, %{title: "", pages: 1})]})]})

PublicLibrary.ensure_type(library, maybe_filter_precond_errors: true)
{:error,
 [
   shelves: [
     "Book title is required.",
     "Book should have more then 3 pages. Given (1)."
   ]
]}

That output contains only a flattened list of precondition error messages from the deeply nested structure.

Integration with Ecto

Ecto schema changeset can be automatically validated to conform to t() type and associated preconditions. Then the changeset function can be like the following:

defmodule Customer do
  use Ecto.Schema
  use Domo, ensure_struct_defaults: false

  import Ecto.Changeset
  import Domo.Changeset

  schema "customers" do
    field :first_name, :string
    field :last_name, :string
    field :birth_date, :date

    timestamps()
  end

  @type t :: %__MODULE__{
          first_name: String.t(),
          last_name: String.t(),
          birth_date: Date.t()
        }

  def changeset(changeset, attrs) do
    changeset
    |> cast(attrs, typed_fields())
    |> validate_required(required_fields())
    |> validate_type()
  end

  # Domo adds typed_fields/0, required_fields/0 funcitons to the schema.
  # Domo.Changeset defines validate_type/1 function.
end

See typed_fields/0, required_fields/0, and Domo.Changeset module documentation for details.

See detailed example is in the ./example_avialia project.

Integration with libraries generating t() type for a struct

Domo is compatible with most libraries that generate t() type for a struct or an Ecto schema. Just use Domo in the module, and that's it.

An advanced example is in the ./example_typed_integrations project.

Compile-time and Run-time validations

At the project's compile-time, Domo can perform the following checks:

Domo validates struct type conformance with appropriate TypeEnsurer modules built during the project's compilation at the application's run-time. These modules rely on guards and pattern matchings. See __using__/1 for more details.

Depending types tracking

Suppose the given structure field's type depends on a type defined in another module. When the latter type or its precondition changes, Domo recompiles the former module automatically to update its TypeEnsurer to keep type validation in current state.

That works similarly for any number of intermediate modules between module defining the struct's field and module defining the field's final type.

Setup

To use Domo in a project, add the following line to mix.exs dependencies:

{:domo, "~> 1.2.0"}

And the following line to the compilers:

compilers: Mix.compilers() ++ [:domo_compiler]

To avoid mix format putting extra parentheses around precond/1 macro call, add the following import to the .formatter.exs:

[
  import_deps: [:domo]
]

Setup for Phoenix hot reload

To enable hot reload for type changes in structs using Domo, add the following line to the endpoint's configuration in the config.exs file:

config :my_app, MyApp.Endpoint,
  reloadable_compilers: [:phoenix] ++ Mix.compilers() ++ [:domo_compiler]

Callbacks

Constructor, validation, and reflection functions added to the struct module using Domo.

new!/1/0

Creates a struct validating type conformance and preconditions.

The argument is any Enumerable that emits two-element tuples (key-value pairs) during enumeration.

Returns the instance of the struct built from the given enumerable. Does so only if struct's field values conform to its t() type and all field's type and struct's type precondition functions return ok.

Raises an ArgumentError if conditions described above are not fulfilled.

This function will check if every given key-value belongs to the struct and raise KeyError otherwise.

new/2/1/0

Creates a struct validating type conformance and preconditions.

The argument is any Enumerable that emits two-element tuples (key-value pairs) during enumeration.

Returns the instance of the struct built from the given enumerable in the shape of {:ok, struct_value}. Does so only if struct's field values conform to its t() type and all field's type and struct's type precondition functions return ok.

If conditions described above are not fulfilled, the function returns an appropriate error in the shape of {:error, message_by_field}. message_by_field is a keyword list where the key is the name of the field and value is the string with the error message.

Keys in the enumerable that don't exist in the struct are automatically discarded.

Options

ensure_type!/1

Ensures that struct conforms to its t() type and all preconditions are fulfilled.

Returns struct when it's valid. Raises an ArgumentError otherwise.

Useful for struct validation when its fields changed with map syntax or with Map module functions.

ensure_type/2/1

Ensures that struct conforms to its t() type and all preconditions are fulfilled.

Returns struct when it's valid in the shape of {:ok, struct}. Otherwise returns the error in the shape of {:error, message_by_field}.

Useful for struct validation when its fields changed with map syntax or with Map module functions.

typed_fields/1/0

Returns the list of struct's fields defined with explicit types in its t() type spec.

Does not return meta fields with __underscored__ names and fields having any() type by default.

Includes fields that have nil type into the return list.

Options

required_fields/1/0

Returns the list of struct's fields having type others then nil or any().

Does not return meta fields with __underscored__ names.

Useful for validation of the required fields for emptiness. F.e. with validate_required/2 call in the Ecto changeset.

Options

Limitations

The recursive types like @type t :: :end | {integer, t()} are not supported. Because of that types like Macro.t() or Path.t() are not supported.

Parametrized types are not supported. Library returns {:type_not_found, :key} error for @type dict(key, value) :: [{key, value}] type definition. Domo returns error for type referencing parametrized type like @type field :: container(integer()).

Generated submodule with TypedStruct's :module option is not supported.

Migration

To complete the migration to a new version of Domo, please, clean and recompile the project with mix clean --deps && mix compile command.

Adoption

It's possible to adopt Domo library in the project having user-defined constructor functions as the following:

  1. Add :domo dependency to the project, configure compilers as described in the setup section
  2. Set the name of the Domo generated constructor function by adding config :domo, :name_of_new_function, :constructor_name option into the confix.exs file, to prevent conflict with original constructor function names if any
  3. Add use Domo to existing struct
  4. Change the calls to build the struct for Domo generated constructor function with name set on step 3 and remove original constructor function
  5. Repeat for each struct in the project

Performance 🐢

On the average, the current version of the library makes struct operations about 20% sower what may seem plodding. And it may look like non-performant to run in production.

It's not that. The library ensures the correctness of data types at runtime and it comes with the price of computation. As the result users get the application with correct states at every update that is valid in many business contexts.

The output of mix benchmark is following.

Generate 10000 inputs, may take a while.
=========================================

Construction of a struct
=========================================
Operating System: macOS
CPU Information: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
Number of Available Cores: 8
Available memory: 16 GB
Elixir 1.13.0
Erlang 24.1.5

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 0 ns
parallel: 1
inputs: none specified
Estimated total run time: 14 s

Benchmarking __MODULE__.new!(arg)...
Benchmarking struct!(__MODULE__, arg)...

Name                               ips        average  deviation         median         99th %
struct!(__MODULE__, arg)       13.72 K       72.88 μs    ±64.17%          73 μs         168 μs
__MODULE__.new!(arg)           11.33 K       88.24 μs    ±50.65%          91 μs         177 μs

Comparison: 
struct!(__MODULE__, arg)       13.72 K
__MODULE__.new!(arg)           11.33 K - 1.21x slower +15.36 μs

A struct's field modification
=========================================
Operating System: macOS
CPU Information: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
Number of Available Cores: 8
Available memory: 16 GB
Elixir 1.13.0
Erlang 24.1.5

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 0 ns
parallel: 1
inputs: none specified
Estimated total run time: 14 s

Benchmarking %{tweet | user: arg} |> __MODULE__.ensure_type!()...
Benchmarking struct!(tweet, user: arg)...

Name                                                        ips        average  deviation         median         99th %
struct!(tweet, user: arg)                               15.16 K       65.96 μs    ±65.51%          66 μs         148 μs
%{tweet | user: arg} |> __MODULE__.ensure_type!()       13.26 K       75.44 μs    ±59.84%          76 μs         163 μs

Comparison: 
struct!(tweet, user: arg)                               15.16 K
%{tweet | user: arg} |> __MODULE__.ensure_type!()       13.26 K - 1.14x slower +9.48 μs

Contributing

  1. Fork the repository and make a feature branch

  2. After implementing of the feature format the code with: 

    mix format

    run linter and tests to ensure that all works as expected with: 

    mix check || mix check --failed

  3. Make a PR to this repository

Changelog

1.5.1

1.5.0

Breaking change:

1.4.1

1.4.0

Breaking changes:

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

0.0.x - 1.0.x

Roadmap

Roadmap

License

Copyright © 2021 Ivan Rublev

This project is licensed under the MIT license.