JsonldEx

🚀 36x faster than pure Elixir JSON-LD implementations

High-performance JSON-LD processing library for Elixir with Rust NIF backend.

Documentation

• HexDocs: https://hexdocs.pm/jsonld_ex
• Changelog: ./CHANGELOG.md

Quick API

• Canonicalize: JSONLD.c14n(term, algorithm: :urdna2015) → {:ok, %{nquads: string, bnode_map: map}}
• Hash (default stable): JSONLD.hash(term, form: :stable_json | :urdna2015_nquads) → {:ok, %{algorithm: :sha256, form: atom, hash: hex, quad_count: non_neg_integer}}
• Equality: JSONLD.equal?(a, b, form: :stable_json | :urdna2015_nquads) → boolean

Notes

• When the Rust NIF is unavailable, canonicalization falls back to a simplified Elixir path. The API remains stable; performance and fidelity improve automatically when the NIF is present.
• Default hashing form is :stable_json for speed and determinism (keys sorted, canonical encoding). Use :urdna2015_nquads when you need RDF dataset canonicalization.

Provider selection (canonicalization)

• ENV: JSONLD_CANON_PROVIDER=none|ssi|vendor (explicit override)
• Mix config: config :jsonld_ex, canon_provider: :none | :ssi | :vendor
• Implicit default: if JSONLD_NIF_FEATURES includes ssi_urdna2015, provider defaults to :ssi; otherwise :none.
• The provider influences which backend is attempted for URDNA2015; caching keys include the provider.

Performance

JsonldEx delivers exceptional performance through its Rust-based NIF implementation:

Features

• 🚀 36x faster than pure Elixir implementations
• 📋 Full JSON-LD 1.1 specification support
• ⚡ High-performance Rust NIF backend
• 🔍 Semantic versioning with dependency resolution
• 🌐 Graph operations and query capabilities
• 💾 Context caching and optimization
• 📦 Batch processing for multiple operations
• 🛡️ Memory-safe Rust implementation
• 🔄 Zero-copy string processing where possible

Installation

Add jsonld_ex to your list of dependencies in mix.exs:

def deps do
  [
    {:jsonld_ex, "~> 0.1.0"}
  ]
end

Quick Start

# Expand a JSON-LD document
doc = %{
  "@context" => "https://schema.org/",
  "@type" => "Person",
  "name" => "Jane Doe",
  "age" => 30
}

json_string = Jason.encode!(doc)
{:ok, expanded} = JsonldEx.Native.expand(json_string, [])

# Compact with a context
context = %{"name" => "https://schema.org/name"}
context_string = Jason.encode!(context)
{:ok, compacted} = JsonldEx.Native.compact(expanded, context_string, [])

# Other operations
{:ok, flattened} = JsonldEx.Native.flatten(json_string, nil, [])
{:ok, rdf_data} = JsonldEx.Native.to_rdf(json_string, [])

API Reference

Core Operations

Utility Operations

• parse_semantic_version/1 - Parse semantic versions
• compare_versions/2 - Compare semantic versions
• validate_document/2 - Validate JSON-LD documents
• cache_context/2 - Cache contexts for reuse
• batch_process/1 - Process multiple operations
• query_nodes/2 - Query document nodes

Spec workflow helpers

• mix spec.hash --id <id> — compute and store hashes.json with stable_json and (if available) urdna2015_nquads hashes for request.json.

Why Choose JsonldEx?

• Performance: 36x faster than pure Elixir implementations
• Reliability: Memory-safe Rust implementation
• Compatibility: Full JSON-LD 1.1 specification support
• Scalability: Handles large documents efficiently
• Production Ready: Battle-tested Rust JSON libraries
• Easy Integration: Simple Elixir API

Build Notes

• Rust NIF is optional; Elixir fallbacks work when NIF is unavailable.
• Requires a recent Rust toolchain (Cargo.lock v4 compatible) to build native code.

macOS Build Issues

If you encounter LTO-related compiler errors on macOS (e.g., options -C embed-bitcode=no and -C lto are incompatible), use:

JSONLD_NIF_FORCE_BUILD=1 mix compile
# or
make macos

This forces local compilation with optimized settings for macOS compatibility. The project has been configured to handle recent macOS/Xcode toolchain changes automatically.

GitHub Releases & Precompiled NIFs

If you're experiencing rustler_precompiled download failures, you can trigger GitHub workflows to generate missing precompiled artifacts:

Quick Fix (Recommended)

# Setup GitHub CLI (one-time)
make gh-setup

# Check current status
make gh-status

# Auto-fix missing artifacts for current version
make gh-fix-missing

Manual Control

# Create new release with precompiled NIFs
make gh-release

# Rebuild precompiled NIFs for existing release
make gh-precompiled

# Check workflow status
make gh-check-releases

What This Does

• gh-status: Checks if your current version has precompiled macOS artifacts
• gh-fix-missing: Automatically triggers GitHub Actions to build missing artifacts
• gh-release: Creates a new release with full precompiled NIF matrix
• gh-setup: Installs and configures GitHub CLI

Requirements

• GitHub CLI (gh) - installed automatically by make gh-setup
• Repository write access (for maintainers)
• GitHub Actions enabled on the repository

This solves the rustler_precompiled issue by ensuring all target platforms have precompiled artifacts available for download.

URDNA2015 via ssi (optional)

• The NIF supports an optional integration with SpruceID’s ssi crate for URDNA2015 canonicalization.
• Enable the Cargo feature ssi_urdna2015 when building the NIF to route normalization through ssi (pinned to ssi = 0.11.0).
• By default this feature is off; Elixir fallbacks remain active.
• Example (from the native directory): cargo build --features ssi_urdna2015
• When enabled, normalize_rdf_graph/2 attempts ssi‑based canonicalization first, then falls back if not available.

Precompiled NIFs (rustler_precompiled)

• This library uses rustler_precompiled to download precompiled NIFs from GitHub releases matching the library version.
• Targets: x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-unknown-linux-musl, aarch64-unknown-linux-musl, aarch64-apple-darwin.
  • Note: macOS Intel (x86_64-apple-darwin) support has been removed as 99%+ of macOS developers are now on Apple Silicon. Missing targets fall back to local build.
• If a precompiled artifact is not available, it falls back to local build.
• Default features: none. The ssi_urdna2015 feature is opt-in and only used when explicitly enabled (see env toggles below). Artifact selection matches the chosen feature set.
  • Feature variants append -features-<features> to the tarball name, e.g. -features-ssi_urdna2015.
• Env toggles:
  • JSONLD_NIF_FORCE_BUILD=1 forces building from source (skips download).
  • JSONLD_NIF_FEATURES=ssi_urdna2015 enables optional Cargo features (e.g., ssi integration) for local builds.
  • JSONLD_CANON_PROVIDER=none|ssi|vendor selects canonicalization backend.
• Release artifacts are expected under: https://github.com/nocsi/jsonld/releases/download/v<version>/.

Publishing guide

• Bump version in mix.exs and ensure the tag matches (v<version>).
• Create a GitHub Release with tag v<version>; this triggers the build-precompiled-nifs workflow to build and upload assets for all supported targets and NIF versions.
• Verify uploaded assets follow the expected naming, for example:
  • libjsonld_nif-v<version>-nif-2.16-aarch64-apple-darwin.tar.gz
  • libjsonld_nif-v<version>-nif-2.16-x86_64-unknown-linux-gnu.tar.gz
  • libjsonld_nif-v<version>-nif-2.16-x86_64-unknown-linux-musl.tar.gz
  • feature variant example: libjsonld_nif-v<version>-nif-2.16-x86_64-unknown-linux-musl-features-ssi_urdna2015.tar.gz
  • macOS Apple Silicon example: libjsonld_nif-v<version>-nif-2.16-aarch64-apple-darwin-features-ssi_urdna2015.tar.gz
  • corresponding .sha256 files and aggregated checksums.txt.
• After assets are present, publish the Hex package:
  • mix hex.build
  • mix hex.publish

Prerelase workflow (dry run)

• To validate the pipeline without publishing to Hex, create a prerelease tag:
  • Example: v<version>-rc1 (any hyphen in ref_name is treated as prerelease).
• The release workflow marks artifacts as prerelease automatically and uploads all tarballs and checksums.txt.
• The docs workflow is gated to non-prerelease tags, so HexDocs are not published for -rc tags.

RC tag validation checklist

• Confirm a prerelease exists on GitHub for your tag, with all target matrices present (gnu, musl; macOS/Linux; base and ssi_urdna2015).
• Spot check a few tarballs:
  • Filenames match libjsonld_nif-v<version>-nif-<nif>-<target>[ -features-<features>].tar.gz.
  • .sha256 files exist and checksums.txt includes every artifact.
• Test install from a sample project by pinning the prerelease version and ensuring rustler_precompiled downloads the correct artifact (or falls back when forced).

If assets are temporarily missing or you need to build locally, either:

• Set JSONLD_NIF_FORCE_BUILD=1 when compiling, or
• Add to config/config.exs: config :rustler_precompiled, :force_build, jsonld_ex: true

Note: For local builds, ensure your Rust toolchain supports Cargo.lock v4 (cargo --version ≥ 1.79 recommended).

Continuous Integration

• CI builds and tests two configurations:
  • Base build (no ssi features): ci.yml job build-test-base.
  • ssi-enabled build: ci.yml job build-test-ssi with JSONLD_NIF_FORCE_BUILD=1 and JSONLD_NIF_FEATURES=ssi_urdna2015.
• Release publishing (release-precompiled.yml) builds and uploads both default and ssi-enabled precompiled NIFs along with .sha256 and an aggregate checksums.txt.

Local Preflight (Linux artifacts)

• The preflight uses cross Docker images for both GNU and MUSL targets. Docker must be available (Colima is fine).
  • Run base preflight: make preflight
  • Run ssi variant: make preflight-ssi
  • On first run, the script pulls images ghcr.io/cross-rs/<target>:latest, which can take a few minutes.
• Outputs tarballs to work/precompiled/ with the expected naming:
  • libjsonld_nif-v<version>-nif-<nif>-<target>.tar.gz
  • Feature variant: ...-features-ssi_urdna2015.tar.gz
• Fallback when Docker is not available:
  • MUSL builds can use cargo-zigbuild with Zig (install cargo-zigbuild and zig). GNU builds are skipped.
• macOS/Apple Silicon tip:
  • Colima with Rosetta can run amd64 containers; the script defaults to linux/amd64 images. You can override with CROSS_IMAGE_PLATFORM.
  • The Makefile auto-sets DOCKER_DEFAULT_PLATFORM=linux/amd64 on arm64/aarch64 hosts; override if needed.
• AArch64 host tip: To avoid x86_64 locally, skip it:
  • make preflight-aarch64 or SKIP_X86_64=1 make preflight
  • make preflight-ssi-aarch64 or SKIP_X86_64=1 make preflight-ssi
  • The script auto-skips x86_64 by default on aarch64/arm64 hosts (override by setting SKIP_X86_64=0).
• Subsets for faster iteration:
  • GNU-only: make preflight-gnu-only or with ssi make preflight-gnu-ssi
  • MUSL-only: make preflight-musl-only or with ssi make preflight-musl-ssi
• Check images only (fail-fast, no build): make preflight-check

Fail-fast checks

• The preflight verifies that the appropriate ghcr.io/cross-rs/<target> image can be pulled for your platform before invoking cross, to avoid falling back to host toolchains.
• If the image can’t be pulled, it exits with a clear message. Use:
  • DOCKER_DEFAULT_PLATFORM=linux/amd64 (or CROSS_IMAGE_PLATFORM=linux/amd64) on Apple Silicon
  • CROSS_IMAGE_TAG or CROSS_IMAGE_TAG_<TARGET> to pin a specific image tag

Setup helper

• Install cross and verify Docker in one step:
  • make install-cross (installs cross and fails early if Docker/Colima is not available)
• Install zigbuild fallback and verify zig:
  • make install-zigbuild

Preflight environment overrides

• CROSS_IMAGE_PLATFORM: Platform passed to docker run for cross images (default: linux/amd64).
• CROSS_IMAGE_TAG: Tag to pull for ghcr.io/cross-rs/<target>:<tag> (tried first, then falls back to latest, then main).
• CROSS_IMAGE_TAG_<TARGET>: Per-target tag override; format the <TARGET> by replacing dashes with underscores.
  • Examples:
    • CROSS_IMAGE_TAG_x86_64_unknown_linux_gnu=v0.2.5
    • CROSS_IMAGE_TAG_aarch64_unknown_linux_gnu=main

Suggested cross image tags

• Default recommendation: use latest for all targets unless you need to pin.
• If you prefer pinning, use the per-target overrides:
  • GNU (glibc):
    • x86_64-unknown-linux-gnu: CROSS_IMAGE_TAG_x86_64_unknown_linux_gnu=latest
    • aarch64-unknown-linux-gnu: CROSS_IMAGE_TAG_aarch64_unknown_linux_gnu=latest
  • MUSL:
    • x86_64-unknown-linux-musl: CROSS_IMAGE_TAG_x86_64_unknown_linux_musl=latest
    • aarch64-unknown-linux-musl: CROSS_IMAGE_TAG_aarch64_unknown_linux_musl=latest
• Notes:
  • On Apple Silicon/Colima, set DOCKER_DEFAULT_PLATFORM=linux/amd64 (or CROSS_IMAGE_PLATFORM=linux/amd64) so amd64 images run under Rosetta.
  • To diagnose image issues, try pulling manually:
    • docker pull --platform linux/amd64 ghcr.io/cross-rs/<target>:latest

Troubleshooting

Common Issues

rustler_precompiled Download Failures

Error: couldn't fetch NIF from https://github.com/.../releases/download/...

Solutions:

1. Quick fix: make gh-fix-missing (auto-triggers missing artifact builds)
2. Local build: make macos or JSONLD_NIF_FORCE_BUILD=1 mix compile
3. Manual trigger: make gh-release to create new release with all artifacts

macOS LTO Compiler Errors

Error: options -C embed-bitcode=no and -C lto are incompatible

Solution: Use local build with make macos - the project is pre-configured for latest macOS/Xcode compatibility.

Missing GitHub CLI

Error: GitHub CLI (gh) not found

Solution: Run make gh-setup to automatically install and configure GitHub CLI.

Authentication Issues

Error: Failed to trigger workflow. Make sure you're authenticated

Solution:

gh auth login
# or
make gh-setup

Workflow Permission Issues

Error: API calls fail with permission errors

Cause: Repository requires write access for release workflows.

Solution: Contact repository maintainers or fork the repository.

Getting Help

1. Check artifact status: make gh-status
2. View recent releases: make gh-check-releases
3. Force local build: make macos
4. Reset and retry: mix clean && make macos

For persistent issues, please file a GitHub issue with:

• Your OS and architecture (uname -a)
• Elixir/OTP versions (elixir --version)
• Error output from make gh-status

License

MIT

spec.apply flags (advanced)

• --dry-run: Simulate changes without writing files. Prints a summary; combine with --diff to preview.
• --diff: Show a unified diff of JSON before/after (uses git --no-index). Snapshots written under work/.tmp/<id>/.
• --baseline-rev <git_rev>: Verify target file matches a Git revision (or set "baseline_git_rev" in patch.json). Fails unless --force.
• --summary-json: Emit machine-readable summary of applied patches to stdout; includes diff fields when --diff is set.
• --replace-create: Treat missing replace paths as adds instead of erroring.
• --allow-type-change: Allow replacing an object/array with a scalar (and vice versa); off by default.
• --format pretty|compact: Control output formatting (default: compact).

Patch formats supported

• RFC6902 JSON Patch (ops: add/replace/remove/copy/move; supports /- for array append).
• RFC7396 JSON Merge Patch via top-level "merge" object.

Example

mix spec.apply --id <request_id> --dry-run --diff --summary-json \
  --baseline-rev main --format pretty