Build a simple, production-ready REST API with MongoDB in Elixir without Phoenix.

Elixir already has a popular web framework called Phoenix, like Ruby on Rails and Django. But as part of this course, we will build a simple REST API without using Phoenix. By the end of this course, you would have built your very own simple web framework, which should be production-ready.

This post is aimed at individuals with some background in building APIs and basic knowledge of Elixir syntax. The Elixir's functional paradigm is quite different from anything you would have done before, so we will go slowly through each file and understand the caveats at every level. As I am most familiar with Node.js, I will try to draw parallels with Javascript and Node.js whenever possible.

💡

The intention of this course is to get you familiar with the underlying concepts of a web framework so that a large opinionated web framework like Phoenix doesn't seem like magic.

The source code is also available on Github.

Let's get started

I assume that you have Elixir already set up on your system; if not, you can follow the official Installing Elixir guide.

Once that is done, the first thing that we need to do is create our project. Open your terminal, go to the directory where you keep your personal project and type in the following command:

$ mix new rest_api --sup

Let's deconstruct this command to understand what each of the tokens means.

mix is the built-in build tool that comes packaged with Elixir, which can be used to create, compile, test and manage dependencies for elixir projects. mix it is similar to npm in Node.js universe but much more powerful.
mix new command used to create a new project in the current directory.
rest_api is the name of our application.
--sup flag creates our application with a supervision tree. We will learn more about why we need a supervision tree later in this chapter.

On successful execution, you will see that the following files have been created in a directory named rest_api:

.
├── README.md
├── .formatter.exs  (Used by mix to format your elixir files)
├── .gitignore  (A default gitignore file with all temporary files ignored)
├── mix.exs  (Mix project definition file. Similar to package.json in Node.js)
├── lib/
│   ├── rest_api.ex  (File defining the root Module for our application)
│   └── rest_api/application.ex (File defining a OTP Application with supervisor)
└── test/
    ├── test_helper.exs  (File to write our test helpers)
    └── rest_api_test.exs (File to test our application)

Change your directory to rest_api and open this directory in your favourite text editor. If you are using VSCode, you can simply execute $ code rest_api.

mix.exs

The mix.exs file is similar to package.json in the node.js world. This file defines a special module called RestApi.MixProject that contains the information about our project, application and lists all our dependencies.

The first thing you will notice that is different about the mix file is the extension. It is exs instead of ex. The exs are used for scripting and are not intended to be compiled with the application.

defmodule RestApi.MixProject do
  use Mix.Project

  def project do
    [
      app: :rest_api,
      version: "0.1.0",
      elixir: "~> 1.13",
      start_permanent: Mix.env() == :prod,
      deps: deps()
    ]
  end

  def application do
    [
      extra_applications: [:logger],
      mod: {RestApi.Application, []}
    ]
  end

  defp deps do
    []
  end
end

The MixProject has two public functions:

project: It returns the project configuration like the project name, version, elixir version to use, etc.
application: It is the entry point to our application.

and one private function:

deps: It returns a list of dependencies of this project.

💡

The app name is :rest_api which is defined as an atom (a constant whose value is its own name, also called Symbol in other languages). Atoms are globally recognized within the system so they can be used to other applications to refer to this application.

The other interesting configuration is the start_permanent which is only enabled in the production environment.

When an application starts in Elixir, it can start in one of the following modes:

permanent If the app terminates, all other dependent applications are also terminated.
transient - If the app terminates with :normal reason, it is reported, but no other application is terminated. In abnormal conditions, the behaviour is the same as permanent.
temporary (default)- If the app terminates, it is reported, but no other application is terminated.

💡

In production, once our application crashes beyond repair, we would want the whole VM-node to terminate; otherwise, our monitoring systems would not recognize any errors.

Add dependencies

The first thing we need to do is add some dependencies. To start off, we need two dependencies:

plug_cowboy - This module is composed of two other different modules Plug and Cowboy. Plug gives us tools to work with HTTP requests, like building routers and endpoints, setting status code, body parsing, etc., but it does not know how to handle connections. This is where Cowboy comes into the picture. It is a web server, written in Erlang (not Elixir), that handles all the connections and processes any incoming and outgoing request. Both Plug & Cowboy together provide a simple framework like express.js.
jason- It is a super-fast JSON parser and generator which we will use to handle JSON. This is similar to the body-parser middleware in express.

defp deps do
  [
    {:plug_cowboy, "~> 2.5"},
    {:jason, "~> 1.3"}
  ]
end

mix.exs - update deps function

Next, run this command in the terminal so that mix can install all the required dependencies:

$ mix deps.get

Install all dependencies

Handle Routes

Before we can start testing our application, we need to build the router and configure our application to start listening on port 8080.

Create a new file router.ex in lib/rest_api directory with the following code:

defmodule RestApi.Router do
  # Bring Plug.Router module into scope
  use Plug.Router

  # Attach the Logger to log incoming requests 
  plug(Plug.Logger)
  
  # Tell Plug to match the incoming request with the defined endpoints
  plug(:match)

  # Once there is a match, parse the response body if the content-type
  # is application/json. The order is important here, as we only want to
  # parse the body if there is a matching route.(Using the Jayson parser)
  plug(Plug.Parsers,
    parsers: [:json],
    pass: ["application/json"],
    json_decoder: Jason
  )

  # Dispatch the connection to the matched handler
  plug(:dispatch)

  # Handler for GET request with "/" path
  get "/" do
    send_resp(conn, 200, "OK")
  end

  # Fallback handler when there was no match
  match _ do
    send_resp(conn, 404, "Not Found")
  end
end

lib/rest_api/router.ex - create new file

We have defined a new module called Router in our RestApi project where we will use the Plug.Router to handle our incoming connections. For this tutorial, we are just going to define the empty root path (/) handler that sends a 200 OK status response back.

At this point, our application server is still not in a runnable state. But we can, of course, unit test our router. So that's what we are going to do next. Replace file rest_api_test.exs in the test directory with the following:

defmodule RestApiTest.Router do
  # Bringing ExUnit's case module to scope and configure it to run
  # tests in this module concurrently with tests in other modules
  # https://hexdocs.pm/ex_unit/ExUnit.Case.html
  use ExUnit.Case, async: true
  
  # This makes the conn object avaiable in the scope of the tests,
  # which can be used to make the HTTP request
  # https://hexdocs.pm/plug/Plug.Test.html
  use Plug.Test
  
  # We call the Plug init/1 function with the options then store
  # returned options in a Module attribute opts.
  # Note: @ is module attribute unary operator
  # https://hexdocs.pm/elixir/main/Kernel.html#@/1
  # https://hexdocs.pm/plug/Plug.html#c:init/1
  @opts RestApi.Router.init([])
 
 
  # Create a test with the name "return ok"
  test "return ok" do
  	# Build a connection which is GET request on / url
    conn = conn(:get, "/")

    # Then call Plug.call/2 with the connection and options
    # https://hexdocs.pm/plug/Plug.html#c:call/2
    conn = RestApi.Router.call(conn, @opts)

    # Finally we are using the assert/2 function to check for the
    # correctness of the response
    # https://hexdocs.pm/ex_unit/ExUnit.Assertions.html#assert/2
    assert conn.state == :sent
    assert conn.status == 200
    assert conn.resp_body == "OK"
  end
end

test/rest_api_test.exs - replace test file

We are using the Plug.Test module here, which comes built-in the plug framework to test our router. First, we initialize our router and then call the router with the connection with GET / request. Finally, we are asserting that that status is 200 and the response body is OK. To run the test, you need to execute the following command in the terminal.

$ mix new rest_api --sup

Build and run tests

The output would look something like this.

Compiling 1 file (.ex)

10:03:30.570 [info]  GET /

10:03:30.577 [info]  Sent 200 in 5ms
.

Finished in 0.04 seconds (0.04s async, 0.00s sync)
1 test, 0 failures

Randomized with seed 518174

Running the Server

We have the router coded up and a test that checks the correctness of the endpoint. The final step of this process is to wire up our Router to Cowboy to start an HTTP server and handle some real requests.

If you don't have Postman installed, now it would be good to get it up and running!

Next, you need to update lib/rest_api/application.ex with this magical line. First, we will test out everything is working fine and then understand what happened here.

defmodule RestApi.Application do
  use Application

  # The @impl true here denotes that the start function is implementing a
  # callback that was defined in the Application module
  # https://hexdocs.pm/elixir/main/Module.html#module-impl
  # This will aid the compiler to warn you when a implementaion is incorrect
  @impl true
  def start(_type, _args) do
    children = [
      {Plug.Cowboy, scheme: :http, plug: RestApi.Router, options: [port: 8080]}
    ]

    opts = [strategy: :one_for_one, name: RestApi.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

lib/rest_api/application.ex - add line to file

After this change is done, we are ready to start our server and test for real if the endpoint is working. Run the following command in the terminal and once the build is complete, head over to Postman and send a GET request to http://localhost:8080.

$ iex -S mix run

Run the application

Yipee!! We have a server running and serving our requests. This concludes the implementation of Part 1.

Let's understand what happened in application.ex! If you don't really care about the internals, feel free to skip the next section.

GenServer & Supervisor

Unlike languages like Javascript, Ruby or Python, Elixir is a purely functional language, so a trivial act of storing a reference to the socket and using it later to respond becomes very non-trivial. To make it easier for programmers to actually be able to use Elixir, a lot of behavioural modules have been provided in the core.

GenServer

GenServer — Elixir v1.13.3

Elixir implements a behavioural module called GenServer to create the server of a client-server relation for programmers to quickly build servers. It abstracts away all of the low-level implementations and provides a simple API that can be used to keep state, execute asynchronous code, perform tracing, etc. Cowboy is one such implementation of GenServer.

Let's look at how we can implement a simple GenServer

defmodule MyServer do
	use GenServer
	
    # Invoked when the server is started.
    @impl true
    def init(config) do
    	{:ok, config}
    end
    
    # Invoked to handle synchronous messages
    @impl true
    def handle_call(:name, _from, data) do 
    	{:reply, data}
    end
    
    # Invoked to handle asynchronous
    @impl true
    def handle_cast({:name, data}, state) do
    	{:noreply, data}
    end
end

Simple GenServer Implementation

A module that wants to implement a GenServer can use the use GenServer to implement the necessary callbacks.

To interact with this GenServer, one has to start the server.

{:ok, pid} = GenServer.start_link(MyServer, [])

It starts a new GenServer process with MyServer implementation. Once the GenServer has started, we can interact with it using its process id (PID) and like everything else in Elixir, by passing messages. There are two types of messages that we can send:

call (synchronous) messages expect a reply from the server.
cast (asynchronous) messages do not wait for a reply from the server.

GenServer.call(pid, :name, :hello)
#=> :hello

GenServer.cast(pid, {:name, :hello})
#=> :ok

What happens if, for some reason, the process crashes?

Elixir does have a try-catch semantics, but they are confined to catching errors within a process but they cannot detect if an entire gen-server crashes. Then what do we do if there was an exception and the gen-server itself crashed? This is where the Supervisor comes to the rescue.

Supervisor

Supervisor — Elixir v1.13.2

The Supervisor is another behaviour module that can be used to implement supervisors. A supervisor is a process that supervises other child processes and restarts them when necessary. So if our process dies, then the supervisors will be able to resurrect it from the dead.

To use a Supervisor, we will no longer call GenServer.start_link function but pass the GenServer as a child to Supervisor and call Supervisor.start_link instead. Let's look at an example:

children = [
  {MyServer, []}
]

Supervisor.start_link(children, strategy: :one_for_all)

What this does is start the Server under the supervision of this Supervisor. If, for some reason, the process crashes, then the Supervisor will automatically restart it. This also means that we can no longer call GenServer.call/cast with the pid. When using Supervisor, we can directly call the call/cast method with the atom of the module name.

GenServer.call(MyServer, :name, :hello)
#=> :hello

GenServer.cast(MyServer, {:name, :hello})
#=> :ok

Application

Application — Elixir v1.13.2

An application is another module provided by Elixir to work with applications and define application callbacks for the application lifecycle. We want the HTTP server to start when the application starts, so we use the Application.start/2 callback to start the Supervisor.

defmodule RestApi.Application do
  use Application

  # Start callback called when the application starts
  @impl true
  def start(_type, _args) do
    children = [
      {Plug.Cowboy, scheme: :http, plug: RestApi.Router, options: [port: 8080]}
    ]

	# Starting the supervior with then name RestApi.Supervisor
    opts = [strategy: :one_for_one, name: RestApi.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Simple Application Implementation

For our use case, we use the one_for_one strategy, but there are a few others as well.

one_for_one - Whenever a process crashes, the Supervisor spins up a new process to replace it.
one_for_all - Whenever one process crashes, then all processes are killed and restarted.
rest_for_one When a process crashes, the process and all processes started after it will be restarted.

In the next post, we will look at how we can use the Config module to centrally define configurations, implement an echo and health-check endpoint and add the MongoDB database to our application.

Build a simple REST API with Elixir | Part 2

Build a simple, production-ready REST API with MongoDB in Elixir without Phoenix.

saswat.devSaswat Das

If you liked this tutorial then please support me by sharing this post with your friends and, if you like, consider becoming a member (it's free). 🎉

Disclaimer: The ideas and opinions in this post are my own and not the views of my employer.

Build a simple REST API with Elixir | Part 1