In the previous, we created all of our required endpoints, understood how to work with MongoDB and did some manual testing. If you reached this post directly, you might want to read the last post, as this is a continuation of that post.
Now that we have something working, it's the right time to add some tests and introduce some security to our endpoints. Let's head over to test/rest_api_test.exs and get started.
Let's get started
Testing your code is simply asserting what you expect from the program and what it does. This simple idea is what software testing is all about. As software complexity increases, the number and type of assertions increase considerably and making sense of all the tests becomes pretty hard. Therefore, we build some software testing frameworks that help us organize and manage our test cases. I have been using the words assertions and test-cases inter-changeably, but before start writing tests, let's get out lingo right.
Assertion - It is the most atomic check which verifies if the expected value is the same as the actual value.
Test Case - A test case is an assumption of some behaviour we expect the program to exhibit. The test cases would contain one or more assertions to prove that the assumption holds. For example, a test case could be GET / should respond with 200 OK and the assertion could be conn.status == 200
Test Suite - A suite is an organizational strategy used to co-locate together. Let's say you were testing all GET request, you could create a suite called GET requests which would contain all test cases that test get endpoints.
You have already been introduced to this in Part 1 of this course, but here is a refresher.
Now you should be able to make the right connections in your head:
Assertions are performed by the assert functions provided by ExUnit.Assertions module.
Test Cases and Test Suites are defined by the test and describe macro respectively, provided by ExUnit.Case module.
Writing test cases is as much art as it is a science and beyond our scope here. Learning how to write good tests will be an entire course in itself, so if you want me to build that course, leave me a comment below. Meanwhile, browse around the internet for any good resources.
Working with database
The database is one of the most crucial dependencies and it is kind of a folklore among developers and testers who believe we should mock the dependencies and write unit tests. I feel that writing unit tests for application servers waste a bunch of time, but if we write tests which call the API and observe the database, we can cover more realistic situations with high confidence.
If we are not mocking out the database, then every test case could inadvertently affect other test cases. To prevent this from happening, we will do two simple things
Prevent running test cases in parallel
Clean up our database after every test
The known downside of this approach is that the tests would run slow, but hey, our tests are super reliable.
Assignment 1: What happens if the tests start with data already present in the DB? The very first test could potentially fail, making our tests flaky! Find out which other ExUnit callback you could use to clean the DB once before any test case is executed and implement the same.
Creating a Post
Let's call our endpoint through the test and create a Post and then check if the response we obtain is correct and the database is updated accordingly
Assignment 2: The last assertion we made here is very naive as it just checks if the post count has increased to one. Ideally, it should also check that the data stored in DB is correct. Implement your own logic to prevent MongoDB store the correct values.
Fetching all Posts
Next, we test our GET all-post endpoint to see if it returns all the posts. Here's the catch: our database will be empty but we need some posts to exist for us to be able to make this test. In the previous test, we used the API to create the post and you might be tempted to use the same code to create two posts, but I strongly recommend we use the DB to directly create the post instead of calling the API. This way if there was ever a bug in the post-creating endpoint it would not cause the get endpoint to fail.
To help us create two posts we define a function createPosts which contains our logic to create two posts and it also returns the ids of the created posts as a list of strings. The rest of the code is pretty self-explanatory.
Fetching a single post
Updating a post
Deleting a post
Our server is running, it's interacting well with the database and has written some automation tests to prove the correctness of the application. In the next part, we want to explore how we can deploy our application and serve some real traffic. Before we do that we need some way to prevent nefarious actors from accessing our endpoints and causing havoc. For this purpose, in the short term, we can utilize the Basic Authentication Scheme and restrict access to all our endpoints with a username and password.
Basic auth is not the best way to secure servers. It is easily susceptible to timing attacks and other vulnerabilities. In future posts, we will explore and implement other authentication and authorization strategies like OAuth2, etc, but for now, this should suffice.
The basic auth scheme is very simple and all we need to do is set the Authorization header. The values start with the word Basic then whitespace followed by Base64 encoding of the username and password joined by :.
If you are thinking what happens if the username contains the character :? Excellent Question! You have now started to thinking like a good developer. Reading RFC-7617 will give you the answer and explain in details the whole scheme.
Oh, now that's easy. I should implement this on my own. Now hold one and remember this golden rule. Never start by implementing your own version of a security library. Always find one which is open-source and maintained by the community. I don't want to discourage you to build it yourself, but take some time to read the implementation first to understand all the security aspects that need to be taken off.
In our case, we will use the Plug.BasicAuth module that comes as part of Plug. First order of business, let's add these little notorious usernames and passwords somewhere. We could have them in the environment variables, but also the config should also be fine.
Please replace <username> and <password> with the strings that you decide as your username and password.
The next step is to configure the application to use the basic auth module for our router.
There is a curious observation to be made here. We are used the use keyword to get the Plug.Router but when using Plug.BasicAuth do it using import?
use <module> - This brings in all the functions from the module to the current module and after that, it calls the __using__ macro on the module. If you are coming from an object-oriented programming world you can imagine it to be an inheritance, or using the extends keyword.
import <module> - This just brings in functions to be used in the current module. There is no additional code executed in the parents' module lifecycle.
Then add the :basic_auth plug between the :match and :dispatch pipeline.
Now if you run all your tests, they should error out something like this.
1) test GET / should return ok (RestApiTest.Router)
Assertion with == failed
code: assert conn.status == 200
As you can see our tests rightly detect that the server rejected our request with a 401 Unauthorized status code.
To fix this we simply have to add the following lines to all our tests. We are going to use the put_req_header API to set the Authorization header. Note that header keys are almost always case-insensitive so we don't need to
That concludes part 4. To summarize,
You now have automated tests for your application.
You have added basic auth to protect your endpoint.
In the next post, we shall see how we can set up CI/CD for our application using GitHub Actions and Deploy it to some cloud hosting.
If you liked this tutorial then please support me by sharing this post with your friends.If you have questions consider becoming a member and adding your feedback as comments
Disclaimer: The ideas and opinions in this post are my own and not the views of my employer.