Getting Rusty with Elixir and NIFs

Rust is one of the most loved languages alongside with Elixir, and it keeps getting some cool new libraries.

There are many times when we would want to port a library or use a replacement one.

Or maybe we want to do something that uses GPU acceleration and runs closer to bare metal.

For times like this, Erlang has NIFs, which means Elixir does get it too.

NIFs are:

a simpler and more efficient way of calling C-code than using port drivers

Which may not tell you much, but it boils down to having a way to run your Rust code from Elixir efficiently.

Tooling for NIFs

Rustler makes it easy to create NIFs in Rust and include them in your Elixir project.

It provides a mix rustler.new command, which sets up almost everything for you.

And that's it!

No need to invent your own glue when it's already available.

Example Phoenix project

There's no better way to learn code than diving right into it.

That's why I've prepared an example project that I will go over in detail(hopefully).

The goal of this project is to create a Rust NIF that crops an image and makes it grayscale.

On the frontend side, we will have a way to upload and select a cropping area for an image.

Creating your project

Pretty much standard from Phoenix's docs

mix phx.new snipping_crab
cd snipping_crab
mix ecto.create

If everything went well, you will be able to run:

mix phx.server

and go to localhost:4000 to see the default Phoenix welcome page.

Getting Rusty

Setup Rust on your system

https://www.rust-lang.org/tools/install

Add Rustler to your project

diff --git a/mix.exs b/mix.exs
index 7d1dec3..78f6993 100644
--- a/mix.exs
+++ b/mix.exs
@@ -48,7 +48,8 @@ defmodule SnippingCrab.MixProject do
       {:telemetry_poller, "~> 1.0"},
       {:gettext, "~> 0.18"},
       {:jason, "~> 1.2"},
-      {:plug_cowboy, "~> 2.5"}
+      {:plug_cowboy, "~> 2.5"},
+      {:rustler, "~> 0.25.0"}
     ]
   end

Generate your NIF

mix rustler.new

This is the name of the Elixir module the NIF module will be registered to.

Module name > SnippingCrab.SnippyCrab

This is the name used for the generated Rust crate. The default is most likely fine.

Library name (snippingcrab_snippycrab) > snippy_crab

Create a basic function

Rustler should've created a basic add Rust function:

// ./native/snippy_crab/src/lib.rs

#[rustler::nif]
fn add(a: i64, b: i64) -> i64 {
    a + b
}

rustler::init!("Elixir.SnippingCrab.SnippyCrab", [add]);

We won't be able to use it yet!

We need to create an additional Elixir module SnippingCrab.SnippyCrab same as in the Rust file except the Elixir. prefix.

# ./lib/snipping_crab/snippy_crab.ex

defmodule SnippingCrab.SnippyCrab do
  use Rustler, otp_app: :snipping_crab, crate: "snippy_crab"

  # When your NIF is loaded, it will override this function.
  @spec add(integer(), integer()) :: integer()
  def add(a, b), do: :erlang.nif_error(:nif_not_loaded)
end

Let's test the result!

Jump into the interactive elixir shell:

iex -S mix

iex(1)> alias SnippingCrab.SnippyCrab
iex(2)> SnippyCrab.add(1, 10)
11

It works!

You can also modify the Rust code while being inside iex:

// ./native/snippy_crab/src/lib.rs
#[rustler::nif]
fn add(a: i64, b: i64) -> i64 {
    a * b
}

rustler::init!("Elixir.SnippingCrab.SnippyCrab", [add]);

After making changes run recompile inside iex

iex(3)> recompile
iex(4)> SnippyCrab.add(10, 10)
100

Let's get more advanced

Setting up the frontend

Create new controllers/views/schema:

mix phx.gen.html Graphics Image images file x:integer y:integer width:integer height:integer

Remove the migrations & context, as we will not be hitting the database for anything.

rm priv/repo/migrations/*
rm lib/snipping_crab/graphics.ex

Change the controller

# ./lib/snipping_crab_web/controllers/image_controller.ex
defmodule SnippingCrabWeb.ImageController do
  use SnippingCrabWeb, :controller

  alias SnippingCrab.Graphics.Image

  def index(conn, _params) do
    changeset = Image.changeset(%Image{}, %{})
    render(conn, "new.html", changeset: changeset)
  end

  def create(conn, %{"image" => image_params}) do
    changeset = Image.changeset(%Image{}, image_params)
    render(conn, "new.html", changeset: changeset)
  end
end

Change the template

# ./lib/snipping_crab_web/templates/image/form.html.heex

<img id="imagePreview" src="#" hidden>

<.form let={f} for={@changeset} action={@action} multipart={true} id="image" >
  <%= if @changeset.action do %>
    <div class="alert alert-danger">
      <p>Oops, something went wrong! Please check the errors below.</p>
    </div>
  <% end %>

  <%= file_input f, :file %>
  <%= error_tag f, :file %>

  <%= hidden_input f, :x %>
  <%= error_tag f, :x %>

  <%= hidden_input f, :y %>
  <%= error_tag f, :y %>

  <%= hidden_input f, :width %>
  <%= error_tag f, :width %>

  <%= hidden_input f, :height %>
  <%= error_tag f, :height %>

  <div>
    <%= submit "Upload" %>
  </div>
</.form>

Add a controller to '/' path

diff --git a/lib/snipping_crab_web/router.ex b/lib/snipping_crab_web/router.ex
index e924506..96eba14 100644
--- a/lib/snipping_crab_web/router.ex
+++ b/lib/snipping_crab_web/router.ex
@@ -17,7 +17,7 @@ defmodule SnippingCrabWeb.Router do
   scope "/", SnippingCrabWeb do
     pipe_through :browser

-    get "/", PageController, :index
+    resources "/", ImageController
   end

   # Other scopes may use custom stacks.

Add Croppr.js to the project

npm add --prefix assets croppr

Setup JS code

// ./assets/js/app.js

import "../css/app.css";
import "phoenix_html";
import { Socket } from "phoenix";
import { LiveSocket } from "phoenix_live_view";
import topbar from "../vendor/topbar";

let csrfToken = document
  .querySelector("meta[name='csrf-token']")
  .getAttribute("content");
let liveSocket = new LiveSocket("/live", Socket, {
  params: { _csrf_token: csrfToken },
});

topbar.config({ barColors: { 0: "#29d" }, shadowColor: "rgba(0, 0, 0, .3)" });
window.addEventListener("phx:page-loading-start", (info) => topbar.show());
window.addEventListener("phx:page-loading-stop", (info) => topbar.hide());

liveSocket.connect();

window.liveSocket = liveSocket;

// Import Croppr.js
import Croppr from "croppr";
import "../node_modules/croppr/src/css/croppr.css";

// Load up all the needed elems
let formElem = document.getElementById("image");
let imagePreviewElem = document.getElementById("imagePreview");
let imageFileElem = document.getElementById("image_file");
let imageXElem = document.getElementById("image_x");
let imageYElem = document.getElementById("image_y");
let imageWidthElem = document.getElementById("image_width");
let imageHeightElem = document.getElementById("image_height");

imageFileElem.addEventListener("change", () => {
  // Assume single file selected only
  const [file] = imageFileElem.files;

  // Required for croppr to work
  imagePreviewElem.src = URL.createObjectURL(file);

  let croppr = new Croppr(imagePreviewElem);

  // Collect crop params before submit
  formElem.addEventListener("submit", () => {
    const { x, y, width, height } = croppr.getValue();
    imageXElem.value = x;
    imageYElem.value = y;
    imageWidthElem.value = width;
    imageHeightElem.value = height;
  });
});

Now we should have a cool submit form that looks like this:

But it doesn't do anything on submit yet!

Let's dive into Rust code for a change.

Image manipulation with Rust

Add image to Cargo.toml

diff --git a/native/snippy_crab/Cargo.toml b/native/snippy_crab/Cargo.toml
index 45241f2..d265e57 100644
--- a/native/snippy_crab/Cargo.toml
+++ b/native/snippy_crab/Cargo.toml
@@ -11,3 +11,4 @@ crate-type = ["cdylib"]

 [dependencies]
 rustler = "0.25.0"
+image = "0.24.2"

Here is where we go more advanced with the code.

We'll be creating a function with this signature:

fn crop_and_grayscale<'a>(env: rustler::env::Env<'a>, image_buffer: rustler::types::Binary<'a>, x: u32, y: u32, width: u32, height: u32) -> rustler::types::Binary<'a>

Env is required to do operations that require communication with BEAM.

Without it, you won't be able to access a Binary nor allocate a new one.

It's passed to your Rust code automatically by Rustler so you don't have to worry about managing that from Elixir.

Reading image data from a Binary :

let reader = image::io::Reader::new(std::io::Cursor::new(&*image_buffer))
    .with_guessed_format()
    .unwrap();
let mut image = reader.decode().unwrap();

&* coerces Binary into &[u8]

Now we can begin transforming our image:

image = image.grayscale().crop(x, y, width, height);

Time to return the image to Elixir:

let mut out = rustler::types::NewBinary::new(env, image.as_bytes().len());

Here we allocate a new Binary with the same bytesize as our transformed image.

image.write_to(
    &mut std::io::BufWriter::new(std::io::Cursor::new(out.as_mut_slice())),
    image::ImageOutputFormat::Png,
).unwrap();

Here we write the image to the Binary in the PNG format.

And finally we can just return our NewBinary as a Binary using the Into trait.

out.into()

Your lib.rs file should look like this:

// ./native/snippy_crab/src/lib.rs

#[rustler::nif]
fn crop_and_grayscale<'a>(
    env: rustler::env::Env<'a>,
    image_buffer: rustler::types::Binary<'a>,
    x: u32,
    y: u32,
    width: u32,
    height: u32,
) -> rustler::types::Binary<'a> {
    let reader = image::io::Reader::new(std::io::Cursor::new(&*image_buffer))
        .with_guessed_format()
        .unwrap();

    let mut image = reader.decode().unwrap();

    image = image.grayscale().crop(x, y, width, height);

    let mut out = rustler::types::NewBinary::new(env, image.as_bytes().len());

    image
        .write_to(
            &mut std::io::BufWriter::new(std::io::Cursor::new(out.as_mut_slice())),
            image::ImageOutputFormat::Png,
        )
        .unwrap();

    out.into()
}

rustler::init!("Elixir.SnippingCrab.SnippyCrab", [crop_and_grayscale]);

Now let's add it to Elixir:

# ./lib/snipping_crab/snippy_crab.ex

defmodule SnippingCrab.SnippyCrab do
  use Rustler, otp_app: :snipping_crab, crate: "snippy_crab"

  # When your NIF is loaded, it will override this function.
  @spec crop_and_grayscale(
          binary(),
          non_neg_integer(),
          non_neg_integer(),
          non_neg_integer(),
          non_neg_integer()
        ) :: binary()
  def crop_and_grayscale(image, x, y, width, height), do: :erlang.nif_error(:nif_not_loaded)
end

Time to test it in iex!

Copy your ferris.png to the Elixir project root. You can get it from here

iex(1)> alias SnippingCrab.SnippyCrab
SnippingCrab.SnippyCrab
iex(2)> image_buffer = File.read!("ferris.png")
<<137, 80, 78, 71, 13, 10, 26, 10, 0, 0, 0, 13, 73, 72, 68, 82, 0, 0, 2, 0, 0,
  0, 2, 0, 8, 6, 0, 0, 0, 244, 120, 212, 250, 0, 0, 1, 132, 105, 67, 67, 80, 73,
  67, 67, 32, 112, 114, 111, 102, 105, ...>>
iex(3)> result = SnippyCrab.crop_and_grayscale(image_buffer, 50, 50, 200, 200)
<<137, 80, 78, 71, 13, 10, 26, 10, 0, 0, 0, 13, 73, 72, 68, 82, 0, 0, 0, 200, 0,
  0, 0, 200, 8, 4, 0, 0, 0, 7, 81, 102, 21, 0, 0, 15, 19, 73, 68, 65, 84, 120,
  156, 237, 221, 11, 140, 84, 213, 25, ...>>
iex(4)> File.write("ferris-cropped.png", result)
:ok

And here's the result:

Gluing it together

Fix the schema type:

diff --git a/lib/snipping_crab/graphics/image.ex b/lib/snipping_crab/graphics/image.ex
index 31e10eb..4d81d7e 100644
--- a/lib/snipping_crab/graphics/image.ex
+++ b/lib/snipping_crab/graphics/image.ex
@@ -3,7 +3,7 @@ defmodule SnippingCrab.Graphics.Image do
   import Ecto.Changeset

   schema "images" do
-    field :file, :string
+    field :file, :map
     field :height, :integer
     field :width, :integer
     field :x, :integer

Change the show template:

# ./lib/snipping_crab_web/templates/image/show.html.heex

<h1>Show Image</h1>

<image src={@image_src} />
<br>

<span><%= link "Back", to: Routes.image_path(@conn, :index) %></span>

Change the controller:

# ./lib/snipping_crab_web/controllers/image_controller.ex
defmodule SnippingCrabWeb.ImageController do
  use SnippingCrabWeb, :controller

  alias SnippingCrab.Graphics.Image
  alias SnippingCrab.SnippyCrab

  def index(conn, _params) do
    changeset = Image.changeset(%Image{}, %{})
    render(conn, "new.html", changeset: changeset)
  end

  def create(conn, %{"image" => image_params}) do
    changeset = Image.changeset(%Image{}, image_params)

    if changeset.valid? do
      %{file: %{path: path}} = changeset.changes
      %{x: x, y: y, width: width, height: height} = changeset.changes

      image_buffer = File.read!(path)

      image_b64 =
        image_buffer
        |> SnippyCrab.crop_and_grayscale(x, y, width, height)
        |> Base.encode64()

      render(conn, "show.html", image_src: "data:image/png;base64,#{image_b64}")
    else
      render(conn, "new.html", changeset: changeset)
    end
  end
end

And now it should be working:

Safety

You've probably noticed the use of .unwrap()

That's not safe and can/will crash your server when something goes wrong.

But don't worry, Rustler has your back.

It supports Rust's native Result and is able to convert it to a {:ok, result} or {:error, "Error message"} tuple with a little bit of help.

Adding error handling

Create a generalized error type (or you can use your favorite library that will do that for you):

enum Error {
    ImageError(image::ImageError),
    IoError(std::io::Error)
}

impl From<image::ImageError> for Error {
    fn from(error: image::ImageError) -> Self {
        Error::ImageError(error)
    }
}

impl From<std::io::Error> for Error {
    fn from(error: std::io::Error) -> Self {
        Error::IoError(error)
    }
}

Implement Encoder for your Error :

impl rustler::Encoder for Error {
    fn encode<'a>(&self, env: rustler::env::Env<'a>) -> rustler::Term<'a> {
        let msg = match &self {
            Error::ImageError(error) => match error {
                image::ImageError::Decoding(_) => "Decoding error",
                image::ImageError::Encoding(_) => "Encoding error",
                image::ImageError::Parameter(_) => "Parameter error",
                image::ImageError::Limits(_) => "Limits error",
                image::ImageError::Unsupported(_) => "Unsupported format error",
                image::ImageError::IoError(_) => "Image IO error",
            },
            Error::IoError(_) => "Error reading the buffer"
        };

        let mut msg_binary = rustler::NewBinary::new(env, msg.len());
        msg_binary
            .as_mut_slice()
            .clone_from_slice(msg.as_bytes());

        msg_binary.into()
    }
}

Return a Result and remove unwraps:

#[rustler::nif]
fn crop_and_grayscale<'a>(
    env: rustler::env::Env<'a>,
    image_buffer: rustler::types::Binary<'a>,
    x: u32,
    y: u32,
    width: u32,
    height: u32,
) -> std::result::Result<rustler::types::Binary<'a>, Error> {
    let reader =
        image::io::Reader::new(std::io::Cursor::new(&*image_buffer)).with_guessed_format()?;

    let mut image = reader.decode()?;

    image = image.grayscale().crop(x, y, width, height);

    let mut out = rustler::types::NewBinary::new(env, image.as_bytes().len());

    image.write_to(
        &mut std::io::BufWriter::new(std::io::Cursor::new(out.as_mut_slice())),
        image::ImageOutputFormat::Png,
    )?;

    Ok(out.into())
}

Finally, handle the tuple in Elixir:

# ./lib/snipping_crab_web/controllers/image_controller.ex
defmodule SnippingCrabWeb.ImageController do
  use SnippingCrabWeb, :controller

  alias SnippingCrab.Graphics.Image
  alias SnippingCrab.SnippyCrab

  def index(conn, _params) do
    changeset = Image.changeset(%Image{}, %{})
    render(conn, "new.html", changeset: changeset)
  end

  def create(conn, %{"image" => image_params}) do
    changeset = Image.changeset(%Image{}, image_params)

    if changeset.valid? do
      %{file: %{path: path}} = changeset.changes
      %{x: x, y: y, width: width, height: height} = changeset.changes

      image_buffer = File.read!(path)
      {:ok, image} = SnippyCrab.crop_and_grayscale(image_buffer, x, y, width, height)
      image_b64 = Base.encode64(image)

      render(conn, "show.html", image_src: "data:image/png;base64,#{image_b64}")
    else
      render(conn, "new.html", changeset: changeset)
    end
  end
end

Now your server won't crash but 500 at worst!

Closing words

Thanks for reading!

Here's a link to this entire project's repo: https://github.com/ravensiris/snipping_crab