cimg_ex

Light-weight image processing module in Elixir with CImg. This module aims to create auxiliary routines for Deep Learning.

Note: It still has a few image processing functions currentrly.

Design detail

The entity of the image handled by CImg is on the NIF side. On Elixir, the reference to the image generated by NIF is stored in the %CImg{} structure. You cannot read out the pixels of the image and process it directly, instead you can use the image processing functions provided in CImg module.

The image will be assigned to Erlang Resource by NIF, so the image will automatically be subject to garbage collection when it is no longer in use.

This is a very important point. Some of the functions in CImg module mutably rewrite the original image. I recommend you to make a duplicate of the image before performing the image processing.

Platform

It has been confirmed to work in the following OS environment.

Requirements

The following libraries are required to display images on the PC screen.

Installation

Add following dependency to your mix.exs.

def deps do
  [
    {:cimg, "~> 0.1.4"}
  ]
end

and install dependencies:

$ mix deps.get
$ mix deps.compile

Demo

There is a simple program in demo directory. You can do it by following the steps below.

$ cd demo
$ mix deps.get
$ mix run -e "CImgDemo.demo1"

Close the appaired window, and stop the demo program.

Another demo:

$ mix run -e "CImgDemo.demo2"
$ mix run -e "CImgDemo.demo3"

Example: Deep Labelling for Semantic Image Segmentation

The below code is an excerpt of the inference part from the DeepLab app implemented with TflInterp. CImg preprocesses the input image and provides a DeepLab model. The DeepLab result tensor is then converted by CImg into a colormaedp image.

defmodule TflDemo.DeepLab3 do
  alias TflDemo.DeepLab3.Prediction

  def apply_deeplab3(img_file) do
    img = CImg.load(img_file)

    segments = Prediction.apply(img)
  end
end


defmodule TflDemo.DeepLab3.Prediction do
  use TflInterp, model: "priv/lite-model_deeplabv3_1_metadata_2.tflite"
  
  @deeplab3_shape {257, 257}

  def apply(img) do
    # preprocess
    bin =
      CImg.dup(img)
      |> CImg.get_resize(@deeplab3_shape)
      |> CImg.to_flat(range: {-1.0, 1.0})

    # prediction
    outputs =
      __MODULE__
      |> TflInterp.set_input_tensor(0, bin.data)
      |> TflInterp.invoke()
      |> TflInterp.get_output_tensor(0)
      |> Nx.from_binary({:f, 32}) |> Nx.reshape({257, 257, :auto})
      
    # postprocess
    outputs
    |> Nx.argmax(axis: 2)
    |> Nx.as_type({:u, 8})
    |> Nx.to_binary()
    |> CImg.create_from_bin(257, 257, 1, 1, "<u1")
    |> CImg.map("lines")    # convert into a colormaped image
  end
end

License

cimg_ex is licensed under the Apache License Version 2.0.

-- license overview of included 3rd party libraries --