Xbase

A comprehensive, production-ready Elixir library for reading, writing, and manipulating dBase database files (DBF) with full support for memo fields (DBT) and indexes (CDX).

Built for performance, reliability, and ease of use, Xbase provides a complete solution for working with legacy dBase files in modern Elixir applications.

✨ Features

🗃️ Complete dBase Support

• File Formats: dBase III, IV, 5, FoxPro, and Visual FoxPro compatibility
• Field Types: Character (C), Numeric (N), Date (D), Logical (L), and Memo (M)
• File Operations: Create, read, update, delete, and pack operations
• Binary Parsing: High-performance binary pattern matching

📝 Advanced Memo Fields

• DBT Integration: Seamless variable-length text storage
• Smart Caching: Built-in memo block caching for performance
• Transaction Safety: ACID-compliant memo operations
• Content Management: Automatic block allocation and compaction

🚀 Performance & Scalability

• Memory Efficient: Stream-based processing for large files
• Lazy Evaluation: On-demand data loading
• Batch Operations: Optimized bulk operations
• Index Support: B-tree CDX indexes for fast lookups

🔐 Enterprise Ready

• Transaction Safety: Full ACID compliance with rollback support
• Error Handling: Comprehensive error reporting and recovery
• Resource Management: Proper file handle lifecycle management
• Production Tested: Built for reliability and performance

📦 Installation

Add xbase to your list of dependencies in mix.exs:

def deps do
  [
    {:xbase, "~> 0.1.0"}
  ]
end

Then run:

mix deps.get

🚀 Quick Start

Reading DBF Files

# Open a DBF file
{:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")

# Read a single record
{:ok, record} = Xbase.Parser.read_record(dbf, 0)
# => %Xbase.Types.Record{data: %{"NAME" => "John Doe", "AGE" => 30}, deleted: false}

# Read all records
{:ok, records} = Xbase.Parser.read_records(dbf)

# Stream records for memory efficiency
dbf
|> Xbase.Parser.stream_records()
|> Stream.filter(fn record -> record.data["AGE"] > 25 end)
|> Enum.to_list()

# Don't forget to close
Xbase.Parser.close_dbf(dbf)

Writing DBF Files

# Define field structure
fields = [
  %Xbase.Types.FieldDescriptor{name: "NAME", type: "C", length: 30},
  %Xbase.Types.FieldDescriptor{name: "AGE", type: "N", length: 3, decimal_count: 0},
  %Xbase.Types.FieldDescriptor{name: "BIRTHDATE", type: "D", length: 8},
  %Xbase.Types.FieldDescriptor{name: "ACTIVE", type: "L", length: 1}
]

# Create new DBF file
{:ok, dbf} = Xbase.Parser.create_dbf("output.dbf", fields)

# Append records
{:ok, dbf} = Xbase.Parser.append_record(dbf, %{
  "NAME" => "Jane Smith",
  "AGE" => 28,
  "BIRTHDATE" => ~D[1995-03-15],
  "ACTIVE" => true
})

# Update existing record
{:ok, dbf} = Xbase.Parser.update_record(dbf, 0, %{"AGE" => 29})

# Mark record as deleted
{:ok, dbf} = Xbase.Parser.mark_deleted(dbf, 0)

# Pack file to remove deleted records
{:ok, dbf} = Xbase.Parser.pack(dbf, "packed.dbf")

Xbase.Parser.close_dbf(dbf)

Working with Memo Fields

# Using MemoHandler for seamless memo support
{:ok, handler} = Xbase.MemoHandler.open_dbf_with_memo("data.dbf", [:read, :write])

# Append record with memo content
{:ok, handler} = Xbase.MemoHandler.append_record_with_memo(handler, %{
  "NAME" => "John Doe",
  "NOTES" => "This is a long memo that will be stored in the DBT file automatically"
})

# Read record with resolved memo content
{:ok, record} = Xbase.MemoHandler.read_record_with_memo(handler, 0)
# => %{"NAME" => "John Doe", "NOTES" => "This is a long memo..."}

# Update memo content
{:ok, handler} = Xbase.MemoHandler.update_record_with_memo(handler, 0, %{
  "NOTES" => "Updated memo content"
})

Xbase.MemoHandler.close_memo_files(handler)

Using Indexes

# Open DBF with index
{:ok, dbf} = Xbase.Parser.open_dbf("data.dbf")
{:ok, cdx} = Xbase.CdxParser.open_cdx("data.cdx")

# Search using index
{:ok, key_info} = Xbase.CdxParser.search_key(cdx, "SMITH")
# => Returns record number for fast access

# Range queries
keys = Xbase.CdxParser.search_range(cdx, "A", "M")

Xbase.CdxParser.close_cdx(cdx)

Transactions

# Wrap operations in a transaction
{:ok, result} = Xbase.Parser.with_transaction(dbf, fn dbf ->
  {:ok, dbf} = Xbase.Parser.append_record(dbf, record1)
  {:ok, dbf} = Xbase.Parser.append_record(dbf, record2)
  {:ok, dbf} = Xbase.Parser.update_record(dbf, 0, updates)
  {:ok, :success}
end)
# Automatically rolls back on error

Batch Operations

# Batch append for performance
records = [record1, record2, record3, ...]
{:ok, dbf} = Xbase.Parser.batch_append_records(dbf, records)

# Batch update
updates = [{0, %{"STATUS" => "ACTIVE"}}, {1, %{"STATUS" => "INACTIVE"}}]
{:ok, dbf} = Xbase.Parser.batch_update_records(dbf, updates)

# Batch delete
{:ok, dbf} = Xbase.Parser.batch_delete(dbf, [5, 10, 15])

📚 Documentation

Complete Guides

• Getting Started Guide - Your first steps with Xbase
• Working with Memo Fields - Complete memo field guide
• API Reference - Full API documentation

Core Modules

⚡ Performance

Benchmarks

Optimization Tips

• Large Files: Use stream_records/1 for memory-efficient processing
• Bulk Operations: Leverage batch functions (batch_append_records, batch_update_records)
• Fast Lookups: Create CDX indexes for frequently searched fields
• Memo Performance: Built-in caching optimizes repeated memo access
• Transactions: Group related operations for better performance and safety

🛡️ Error Handling

Xbase follows Elixir conventions with comprehensive error handling:

case Xbase.Parser.open_dbf("data.dbf") do
  {:ok, dbf} ->
    # Work with the file safely
    process_records(dbf)
  {:error, :enoent} ->
    {:error, "File not found: please check the file path"}
  {:error, :invalid_dbf_header} ->
    {:error, "Invalid DBF file format"}
  {:error, reason} ->
    {:error, "Unexpected error: #{inspect(reason)}"}
end

Error Categories

• File Errors: Missing files, permission issues, corruption
• Format Errors: Invalid DBF structure, unsupported versions
• Data Errors: Invalid field values, type mismatches
• Transaction Errors: Rollback scenarios, concurrent access issues

🤝 Contributing

We welcome contributions! Here's how you can help:

Development Setup

git clone https://github.com/your-org/xbase.git
cd xbase
mix deps.get
mix test

Contributing Guidelines

• Issues: Report bugs or request features via GitHub Issues
• Pull Requests: Fork, create a feature branch, and submit a PR
• Tests: Ensure all tests pass and add tests for new features
• Documentation: Update docs for any API changes

Running Tests

mix test                    # Run all tests
mix test --cover           # Run with coverage
mix dialyzer               # Type checking
mix credo                  # Code quality

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Built with ❤️ for the Elixir community
• Inspired by the need for modern dBase file handling
• Thanks to all contributors and users

Need help? Check out our documentation or open an issue.