In the previous, we created all of our required endpoints, understood how to work with MongoDB and did some manual testing. If you reached this post directly, you might want to read the last post, as this is a continuation of that post.
Now that we have something working, it's the right time to add some tests and introduce some security to our endpoints. Let's head over to test/rest_api_test.exs and get started.
saswatdsLet's get started
Testing your code is simply asserting what you expect from the program and what it does. This simple idea is what software testing is all about. As software complexity increases, the number and type of assertions increase considerably and making sense of all the tests becomes pretty hard. Therefore, we build some software testing frameworks that help us organize and manage our test cases. I have been using the words assertions and test-cases inter-changeably, but before start writing tests, let's get out lingo right.
- Assertion - It is the most atomic check which verifies if the expected value is the same as the actual value.
- Test Case - A test case is an assumption of some behaviour we expect the program to exhibit. The test cases would contain one or more assertions to prove that the assumption holds. For example, a test case could be
GET / should respond with 200 OKand the assertion could beconn.status == 200 - Test Suite - A suite is an organizational strategy used to co-locate together. Let's say you were testing all
GETrequest, you could create a suite calledGET requestswhich would contain all test cases that test get endpoints.
If you are coming from a Javascript or Typescript world then you might be familiar with frameworks like Jest and Mocha. That provides a bunch of helper functions and scaffolding to manage your tests. In the elixir world, the equivalent framework is called ExUnit.
You have already been introduced to this in Part 1 of this course, but here is a refresher.
defmodule RestApiTest.Router do
# Bringing ExUnit's case module to scope and configure it to run
# tests in this module concurrently with tests in other modules
# https://hexdocs.pm/ex_unit/ExUnit.Case.html
use ExUnit.Case, async: true
# This makes the conn object avaiable in the scope of the tests,
# which can be used to make the HTTP request
# https://hexdocs.pm/plug/Plug.Test.html
use Plug.Test
# We call the Plug init/1 function with the options then store
# returned options in a Module attribute opts.
# Note: @ is module attribute unary operator
# https://hexdocs.pm/elixir/main/Kernel.html#@/1
# https://hexdocs.pm/plug/Plug.html#c:init/1
@opts RestApi.Router.init([])
# Create a test with the name "return ok"
test "return ok" do
# Build a connection which is GET request on / url
conn = conn(:get, "/")
# Then call Plug.call/2 with the connection and options
# https://hexdocs.pm/plug/Plug.html#c:call/2
conn = RestApi.Router.call(conn, @opts)
# Finally we are using the assert/2 function to check for the
# correctness of the response
# https://hexdocs.pm/ex_unit/ExUnit.Assertions.html#assert/2
assert conn.state == :sent
assert conn.status == 200
assert conn.resp_body == "OK"
end
end
Now you should be able to make the right connections in your head:
- Assertions are performed by the
assertfunctions provided by ExUnit.Assertions module. - Test Cases and Test Suites are defined by the
testanddescribemacro respectively, provided by ExUnit.Case module.
Working with database
The database is one of the most crucial dependencies and it is kind of a folklore among developers and testers who believe we should mock the dependencies and write unit tests. I feel that writing unit tests for application servers waste a bunch of time, but if we write tests which call the API and observe the database, we can cover more realistic situations with high confidence.
If we are not mocking out the database, then every test case could inadvertently affect other test cases. To prevent this from happening, we will do two simple things
- Prevent running test cases in parallel
- Clean up our database after every test
The known downside of this approach is that the tests would run slow, but hey, our tests are super reliable.
defmodule RestApiTest.Router do
use ExUnit.Case # removed async to prevent running cases in parallel
use Plug.Test
@opts RestApi.Router.init([])
test "GET / should return ok" do ... end
describe "Posts" do
# The setup callback is called before each test executes and the on_exit after each test is complete
# We will use this hook to list all the mongo db collections and for each of
# the collection to clear out the entire collection. This way for every test
# case we will start from a clean slate
setup do
on_exit fn ->
Mongo.show_collections(:mongo)
|> Enum.each(
fn col -> Mongo.delete_many!(:mongo, col, %{})
end)
end
end
end
Find out which other ExUnit callback you could use to clean the DB once before any test case is executed and implement the same.
Automate Testing
Creating a Post
Let's call our endpoint through the test and create a Post and then check if the response we obtain is correct and the database is updated accordingly
defmodule RestApiTest.Router do
...
describe "Posts" do
...
setup do ... end
test "POST /post should return ok" do
# Assert that there are no elements in the db
assert Mongo.find(:mongo, "Posts", %{}) |> Enum.count == 0
# Make an API to call create a post
conn = conn(:post, "/post", %{name: "Post 1", content: "Content of post"})
conn = RestApi.Router.call(conn, @opts)
# Checking that request was actually sent
assert conn.state == :sent
# Checking that response code was 200
assert conn.status == 200
# Asserting that response body was what we expected
# Note: We are using pattern matching here to perform the assertion
# The id is autogenerated by mongodb so we have not way of predicting it
# therefore we just use _ to match to anything but expect it to exist
assert %{
"id" => _,
"content" => "Content of post",
"name" => "Post 1"
} = Jason.decode!(conn.resp_body)
# Assert that there is something in the db
assert Mongo.find(:mongo, "Posts", %{}) |> Enum.count == 1
end
end
end
Fetching all Posts
Next, we test our GET all-post endpoint to see if it returns all the posts. Here's the catch: our database will be empty but we need some posts to exist for us to be able to make this test. In the previous test, we used the API to create the post and you might be tempted to use the same code to create two posts, but I strongly recommend we use the DB to directly create the post instead of calling the API. This way if there was ever a bug in the post-creating endpoint it would not cause the get endpoint to fail.
To help us create two posts we define a function createPosts which contains our logic to create two posts and it also returns the ids of the created posts as a list of strings. The rest of the code is pretty self-explanatory.
defmodule RestApiTest.Router do
...
describe "Posts" do
...
test "POST /post should return ok" do ... end
# A simple helper function that will help up quickly create
# two posts in the databse
def createPosts() do
result = Mongo.insert_many!(:mongo, "Posts",[
%{name: "Post 1", content: "Content 1"},
%{name: "Post 2", content: "Content 2"},
])
# The ids are BSON ObjectId which we are encoding to string for easier consumption
result.inserted_ids |> Enum.map(fn id -> BSON.ObjectId.encode!(id) end)
end
test "GET /posts should fetch all the posts" do
createPosts()
conn = conn(:get, "/posts")
conn = RestApi.Router.call(conn, @opts)
assert conn.state == :sent
assert conn.status == 200
resp = Jason.decode!(conn.resp_body);
assert Enum.count(resp) == 2
assert %{
"id" => _,
"content" => "Content 1",
"name" => "Post 1"
} = Enum.at(resp, 0)
assert %{
"id" => _,
"content" => "Content 2",
"name" => "Post 2"
} = Enum.at(resp, 1)
end
end
end
Fetching a single post
defmodule RestApiTest.Router do
...
describe "Posts" do
...
test "GET /posts should fetch all the posts" do ... end
test "GET /post/:id should fetch a single post" do
[id | _] = createPosts() # using pattern matching to get the first id
conn = conn(:get, "/post/#{id}") # string interpolation
conn = RestApi.Router.call(conn, @opts)
assert conn.state == :sent
assert conn.status == 200
assert %{
"id" => _,
"content" => "Content 1",
"name" => "Post 1"
} = Jason.decode!(conn.resp_body)
end
end
end
Updating a post
defmodule RestApiTest.Router do
...
describe "Posts" do
...
test "GET /post/:id should fetch a single post" do ... end
test "PUT /post/:id should update a post" do
[id | _] = createPosts()
conn = conn(:put, "/post/#{id}", %{content: "Content 3"})
conn = RestApi.Router.call(conn, @opts)
assert conn.state == :sent
assert conn.status == 200
assert %{
"id" => _,
"content" => "Content 3",
"name" => "Post 1"
} = Jason.decode!(conn.resp_body)
end
end
end
Deleting a post
defmodule RestApiTest.Router do
...
describe "Posts" do
...
test "PUT /post/:id should update a post" do ... end
test "DELETE /post/:id should delete a post" do
[id | _] = createPosts()
assert Mongo.find(:mongo, "Posts", %{}) |> Enum.count == 2
conn = conn(:delete, "/post/#{id}", %{content: "Content 3"})
conn = RestApi.Router.call(conn, @opts)
assert conn.state == :sent
assert conn.status == 200
assert Mongo.find(:mongo, "Posts", %{}) |> Enum.count == 1
end
end
end
Basic Authentication
Our server is running, it's interacting well with the database and has written some automation tests to prove the correctness of the application. In the next part, we want to explore how we can deploy our application and serve some real traffic. Before we do that we need some way to prevent nefarious actors from accessing our endpoints and causing havoc. For this purpose, in the short term, we can utilize the Basic Authentication Scheme and restrict access to all our endpoints with a username and password.
Basic auth is not the best way to secure servers. It is easily susceptible to timing attacks and other vulnerabilities. In future posts, we will explore and implement other authentication and authorization strategies like OAuth2, etc, but for now, this should suffice.
The basic auth scheme is very simple and all we need to do is set the Authorization header. The values start with the word Basic then whitespace followed by Base64 encoding of the username and password joined by :.
:? Excellent Question! You have now started to thinking like a good developer. Reading RFC-7617 will give you the answer and explain in details the whole scheme.Oh, now that's easy. I should implement this on my own. Now hold one and remember this golden rule. Never start by implementing your own version of a security library. Always find one which is open-source and maintained by the community. I don't want to discourage you to build it yourself, but take some time to read the implementation first to understand all the security aspects that need to be taken off.
Plug.BasicAuth
In our case, we will use the Plug.BasicAuth module that comes as part of Plug. First order of business, let's add these little notorious usernames and passwords somewhere. We could have them in the environment variables, but also the config should also be fine.
<username> and <password> with the strings that you decide as your username and password.import Config
config :rest_api, port: 8080
config :rest_api, database: "rest_api_db"
config :rest_api, pool_size: 3
config :rest_api, :basic_auth, username: "<username>", password: "<passwors>"
import Config
config :rest_api, port: 80
config :rest_api, database: "rest_api_db"
config :rest_api, pool_size: 3
config :rest_api, :basic_auth, username: "<username>", password: "<passwors>"
import Config
config :rest_api, port: 8081
config :rest_api, database: "rest_api_db"
config :rest_api, pool_size: 3
config :rest_api, :basic_auth, username: "<username>", password: "<passwors>"
The next step is to configure the application to use the basic auth module for our router.
defmodule RestApi.Router do
alias RestApi.JSONUtils, as: JSON
use Plug.Router
import Plug.BasicAuth
plug(Plug.Logger)
plug(:match)
plug(:basic_auth, Application.get_env(:rest_api, :basic_auth))
plug(Plug.Parsers,
parsers: [:json],
pass: ["application/json"],
json_decoder: Jason
)
...
end
There is a curious observation to be made here. We are used the use keyword to get the Plug.Router but when using Plug.BasicAuth do it using import?
There is very good official documentation about this here: alias, require, and import. But the TLDR; is
- use <module> - This brings in all the functions from the module to the current module and after that, it calls the
__using__macro on the module. If you are coming from an object-oriented programming world you can imagine it to be an inheritance, or using theextendskeyword. - import <module> - This just brings in functions to be used in the current module. There is no additional code executed in the parents' module lifecycle.
Then add the :basic_auth plug between the :match and :dispatch pipeline.
Tests
Now if you run all your tests, they should error out something like this.
1) test GET / should return ok (RestApiTest.Router)
test/rest_api_test.exs:8
Assertion with == failed
code: assert conn.status == 200
left: 401
right: 200
stacktrace:
test/rest_api_test.exs:15: (test)
As you can see our tests rightly detect that the server rejected our request with a 401 Unauthorized status code.
To fix this we simply have to add the following lines to all our tests. We are going to use the put_req_header API to set the Authorization header. Note that header keys are almost always case-insensitive so we don't need to
defmodule RestApiTest.Router do
# ...
test "GET / should return ok" do
conn = conn(:get, "/")
conn = put_req_header(conn, "authorization", encode_basic_auth("<username>", "<password>"))
conn = RestApi.Router.call(conn, @opts)
# ...
end
describe "Posts" do
# ...
test "POST /post should create a post" do
# ...
# Make an API to call create a post
conn = conn(:post, "/post", %{name: "Post 1", content: "Content of post"})
conn = put_req_header(conn, "authorization", encode_basic_auth("<username>", "<password>"))
conn = RestApi.Router.call(conn, @opts)
# ...
end
# ...
test "GET /posts should fetch all the posts" do
createPosts()
conn = conn(:get, "/posts")
conn = put_req_header(conn, "authorization", encode_basic_auth("<username>", "<password>"))
conn = RestApi.Router.call(conn, @opts)
# ...
end
test "GET /post/:id should fetch a single post" do
[id | _] = createPosts()
conn = conn(:get, "/post/#{id}")
conn = put_req_header(conn, "authorization", encode_basic_auth("<username>", "<password>"))
conn = RestApi.Router.call(conn, @opts)
# ...
end
test "PUT /post/:id should update a post" do
[id | _] = createPosts()
conn = conn(:put, "/post/#{id}", %{content: "Content 3"})
conn = put_req_header(conn, "authorization", encode_basic_auth("<username>", "<password>"))
conn = RestApi.Router.call(conn, @opts)
# ...
end
test "DELETE /post/:id should delete a post" do
# ...
conn = conn(:delete, "/post/#{id}", %{content: "Content 3"})
conn = put_req_header(conn, "authorization", encode_basic_auth("<username>", "<password>"))
conn = RestApi.Router.call(conn, @opts)
# ...
end
end
end
Next
That concludes part 4. To summarize,
- You now have automated tests for your application.
- You have added basic auth to protect your endpoint.
In the next post, we shall see how we can set up CI/CD for our application using GitHub Actions and Deploy it to some cloud hosting.
If you liked this tutorial then please support me by sharing this post with your friends. If you have questions consider becoming a member and adding your feedback as comments
Disclaimer: The ideas and opinions in this post are my own and not the views of my employer.
