Eliot - IoT Data Ingestion System

"Hello, friend. Welcome to the world of connected devices."

Eliot is a production-ready IoT data ingestion system built with Elixir/OTP, designed for high-throughput sensor data processing and real-time fleet management. Born from the need to handle massive scale device communication with fault-tolerance and observability at its core.

🤖 Features

🏗️ Architecture

Eliot follows the "let it crash" philosophy - embrace failure, isolate it, and recover gracefully. The system is built around an OTP application with isolated supervision trees for maximum resilience.

┌─────────────────────────────────────────────────────────────┐
│                    Eliot Supervisor                         │
├─────────────────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────┐  ┌──────────────┐ │
│  │ Error Handler   │  │ Message Parser  │  │ Logger       │ │
│  │ (Retry Logic)   │  │ (JSON Processing)│  │ (Telemetry)  │ │
│  └─────────────────┘  └─────────────────┘  └──────────────┘ │
└─────────────────────────────────────────────────────────────┘

The architecture demonstrates understanding of:

🚀 Quick Start

Prerequisites

Installation

# Clone the repository
git clone https://github.com/christimahu/eliot.git
cd eliot

# Install dependencies
mix deps.get

# Run tests to verify everything works
mix test

# Start the application
mix run --no-halt

Development

# Run with interactive shell
iex -S mix

# Run linting (if configured)
mix credo

# Format code
mix format

# Build production release
MIX_ENV=prod mix release

Learning Resources

🔧 Configuration

Eliot supports environment-specific configuration:

# config/prod.exs
config :eliot,
  mqtt: [
    broker_host: System.get_env("MQTT_BROKER_HOST"),
    broker_port: 8883,
    ssl: true,
    keepalive: 300
  ]

Environment Variables

📊 Message Format

All device messages follow a standardized JSON schema:

{
  "device_id": "robot_001",
  "timestamp": "2025-06-05T14:30:00Z", 
  "sensor_type": "gps",
  "data": {
    "latitude": 37.7749,
    "longitude": -122.4194,
    "accuracy": 3.2
  }
}

🛡️ Error Handling

Eliot implements sophisticated error handling patterns:

# Example: Device connection timeout handling
{:retry, %{device_id: "robot_001", backoff_ms: 2000}}

# Example: Authentication failure handling  
{:circuit_break, %{device_id: "robot_002", reason: :auth_failure}}

📈 Monitoring & Observability

Every operation generates structured logs for production monitoring:

# Device connection events
Eliot.Logger.log_device_event("robot_001", :connected)

# Message processing metrics
Eliot.Logger.log_processing_event("message_123", 150, :ok)

# Security events
Eliot.Logger.log_error("Authentication failure", %{device_id: "unknown", reason: :auth_failure})

🧪 Testing

# Run all tests
mix test

# Run with coverage
mix test --cover

# Run integration tests only
mix test test/eliot/integration/

# Run tests excluding integration tests
mix test --exclude integration

The test suite covers:

See TESTING.md for detailed testing documentation.

🏭 Production Deployment

Building a Release

# Create production release
MIX_ENV=prod mix release

# Run the release
_build/prod/rel/eliot/bin/eliot start

Docker Deployment

FROM elixir:1.14-alpine
WORKDIR /app
COPY mix.exs mix.lock ./
RUN mix deps.get --only prod
COPY . .
RUN MIX_ENV=prod mix release
CMD ["_build/prod/rel/eliot/bin/eliot", "start"]

🤝 Contributing

We welcome contributions! Please read our Contributing Guide and Code of Conduct:

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature/new-capability
  3. Write tests for your changes
  4. Ensure all tests pass: mix test
  5. Run linting: mix credo (if configured)
  6. Submit a pull request

📋 Requirements Checklist

🏆 Quality Metrics

Tests:     Comprehensive unit and integration test coverage
Coverage:  High coverage of critical paths
Credo:     Clean code following Elixir best practices
Format:    All code properly formatted
Docs:      Comprehensive with examples

🔮 Roadmap

📚 Documentation

📄 License

This project is licensed under the GNU General Public License v3.0 - see the LICENSE file for details.

"The world is a dangerous place, not because of those who do evil, but because of those who look on and do nothing. Eliot watches, processes, and acts."

Built with ❤️ and lots of ☕ for the IoT revolution.