ExSift

MongoDB-style query filtering for Elixir collections.

ExSift is an Elixir library inspired by sift.js that brings MongoDB's powerful query syntax to Elixir. Filter lists, maps, and any enumerable with an expressive, familiar query language.

Features

MongoDB-compatible query syntax - Use the same operators you know from MongoDB
Type-safe - Full Elixir typespecs and pattern matching
Comprehensive operators - Support for comparison, logical, array, and special operators
Nested property access - Query deeply nested maps with dot notation
Regex support - Pattern matching with Elixir's Regex module
Date/Time support - Compare DateTime, NaiveDateTime, and Date types
Well-tested - 40+ tests covering all operators and edge cases

Installation

Add ex_sift to your list of dependencies in mix.exs:

def deps do
  [
    {:ex_sift, "~> 0.1.0"}
  ]
end

Quick Start

data = [
  %{name: "Alice", age: 30, city: "NYC"},
  %{name: "Bob", age: 25, city: "SF"},
  %{name: "Charlie", age: 35, city: "NYC"}
]

# Simple equality
ExSift.filter(data, %{city: "NYC"})
# => [%{name: "Alice", ...}, %{name: "Charlie", ...}]

# Comparison operators
ExSift.filter(data, %{age: %{"$gt" => 28}})
# => [%{name: "Alice", age: 30, ...}, %{name: "Charlie", age: 35, ...}]

# Multiple conditions
ExSift.filter(data, %{city: "NYC", age: %{"$gte" => 30}})
# => [%{name: "Alice", ...}, %{name: "Charlie", ...}]

Supported Operators

Comparison Operators

$eq - Equals (same as direct value)

ExSift.filter(data, %{age: %{"$eq" => 30}})

$ne - Not equals

ExSift.filter(data, %{city: %{"$ne" => "NYC"}})

$gt - Greater than

ExSift.filter(data, %{age: %{"$gt" => 25}})

$gte - Greater than or equal

ExSift.filter(data, %{age: %{"$gte" => 30}})

$lt - Less than

ExSift.filter(data, %{age: %{"$lt" => 30}})

$lte - Less than or equal

ExSift.filter(data, %{age: %{"$lte" => 25}})

Logical Operators

$and - All conditions must match

ExSift.filter(data, %{
  "$and" => [
    %{age: %{"$gte" => 25}},
    %{city: "NYC"}
  ]
})

$or - At least one condition must match

ExSift.filter(data, %{
  "$or" => [
    %{age: %{"$lt" => 26}},
    %{city: "LA"}
  ]
})

$nor - No conditions match

ExSift.filter(data, %{
  "$nor" => [
    %{city: "NYC"},
    %{city: "SF"}
  ]
})

$not - Negation

ExSift.filter(data, %{age: %{"$not" => %{"$lt" => 30}}})

Array Operators

$in - Value in array

ExSift.filter(data, %{city: %{"$in" => ["NYC", "LA", "SF"]}})

$nin - Value not in array

ExSift.filter(data, %{city: %{"$nin" => ["NYC"]}})

$all - Array contains all values

ExSift.filter(data, %{tags: %{"$all" => ["admin", "user"]}})

$elemMatch - Array element matches query

data = [%{items: [%{id: 1, active: true}]}]
ExSift.filter(data, %{items: %{"$elemMatch" => %{id: 1, active: true}}})

$size - Array has specific length

ExSift.filter(data, %{tags: %{"$size" => 3}})

Other Operators

$exists - Field exists (not nil)

ExSift.filter(data, %{email: %{"$exists" => true}})
ExSift.filter(data, %{phone: %{"$exists" => false}})

$type - Type checking

ExSift.filter(data, %{age: %{"$type" => "number"}})
ExSift.filter(data, %{name: %{"$type" => "string"}})
# Supported types: "string", "number", "integer", "float",
#                  "boolean", "map", "list", "atom", "date", "datetime", "nil"

$mod - Modulus operation

ExSift.filter(data, %{count: %{"$mod" => [5, 0]}})  # divisible by 5

$regex - Regular expression matching

# Using Elixir regex
ExSift.filter(data, %{name: ~r/^[AB]/})

# Using $regex operator
ExSift.filter(data, %{email: %{"$regex" => "@gmail\\.com$"}})

Advanced Usage

Nested Properties

Use dot notation to query nested maps:

data = [
  %{user: %{profile: %{age: 30, verified: true}}},
  %{user: %{profile: %{age: 25, verified: false}}}
]

ExSift.filter(data, %{"user.profile.age" => %{"$gt" => 28}})
# => [%{user: %{profile: %{age: 30, ...}}}]

Complex Queries

Combine multiple operators for powerful filtering:

data = [
  %{name: "Alice", age: 30, status: "active", tags: ["admin"]},
  %{name: "Bob", age: 25, status: "inactive", tags: ["user"]},
  %{name: "Charlie", age: 35, status: "active", tags: ["admin", "moderator"]}
]

ExSift.filter(data, %{
  "$and" => [
    %{age: %{"$gte" => 25, "$lte" => 35}},
    %{status: "active"},
    %{tags: %{"$in" => ["admin"]}}
  ]
})
# => [%{name: "Alice", ...}, %{name: "Charlie", ...}]

Utility Functions

ExSift provides several utility functions beyond filter/2:

data = [%{a: 1}, %{a: 2}, %{a: 3}]

# Test a single item
ExSift.test(%{a: 1}, %{a: 1})  # => true

# Find first matching item
ExSift.find(data, %{a: 2})  # => %{a: 2}

# Check if any match
ExSift.any?(data, %{a: %{"$gt" => 2}})  # => true

# Check if all match
ExSift.all?(data, %{a: %{"$gte" => 1}})  # => true

# Count matches
ExSift.count(data, %{a: %{"$lt" => 3}})  # => 2

# Compile query for reuse
tester = ExSift.compile(%{a: %{"$gt" => 1}})
tester.(%{a: 2})  # => true
tester.(%{a: 1})  # => false

Architecture

ExSift is built with three main modules:

ExSift - Main API and utility functions
ExSift.Query - Query parsing and matching logic
ExSift.Operators - Operator implementations

The library leverages Elixir's pattern matching and protocol system for extensible, type-safe query operations.

Comparison with sift.js

ExSift is inspired by sift.js but adapted for Elixir's functional programming paradigm:

Feature	sift.js	ExSift
Language	JavaScript/TypeScript	Elixir
Architecture	Operation classes with state	Pattern matching + pure functions
Extensibility	Custom operations via options	Protocol-based (future)
Type Safety	TypeScript generics	Dialyzer typespecs
Immutability	Depends on usage	Built-in (Elixir default)

Performance

ExSift uses single-pass filtering with early termination where possible. All operations are implemented as pure functions without side effects.

For large datasets, consider:

Using ExSift.compile/1 to create reusable query functions
Leveraging ExSift.find/2 or ExSift.any?/2 for early termination
Pre-filtering with simpler queries before complex ones

Testing

Run the test suite:

mix test

ExSift includes 40+ tests covering:

All operators
Nested property access
Complex query combinations
Edge cases and error handling

License

MIT License - See LICENSE file for details

Acknowledgments

Inspired by sift.js by Craig Condon.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.