Building a Scalable Promise Based NodeJS Library

How to tackle large async / await Promise based libraries

This article will discuss and demonstrate how to build a scalable promise-based NodeJS library for your apps. With the goal of supporting medium to large scale applications, libraries need to be coherent and manageable, yet be flexible and usable in your promise based execution.

Let’s break down our requirements:

• Scalable: We want to be able to create as many exports as our app requires.
• Economic: We also want to be economic in regards to resources — e.g. We only want one connection to our database client, which can then be passed into our exports when needed. This will be demonstrated with a MongoDB client.
• Promise-based: Our exports need to return a promise, therefore utilise async and await functionality. At the same time, we do not want to litter our scripts with verbose markup.
• Clean syntax: async and await declarations quickly clutter your syntax. We will discuss how to create a helper function, named my_async
  , which sets up an asynchronous function call for our promises whilst keeping syntax minimal.

As a brief example, our my_async helper will shrink this:

var handlePromise1 = (async () => (await (myPromise1(client))));

handlePromise1().then(data => { 

   data.success      ? res.json({'success': true})      : res.json({'success': false})});

into this:

my_async(myPromise1, [client]).then(data => {      data.success      ? res.json({'success': true})      : res.json({'success': false})});

This is a major improvement to your syntax. We will be visiting how this function is declared further down this article. (think the … operator to expand arrays).

Pre-reading

This is a continuation of my previous articles on promises, so if you are yet to explore the Javascript promise, jump into these before reading on:

Before exploring a fully-fledged promise example, let’s briefly visit a MongoDB exports file that will resolve a client for us to use throughout our library.

Importing MongoDB Promises as Database Client

In the previous article we created a MongoDB exports file to handle client connections. This was great for handling queries on a per-request basis, but the approach quickly becomes limited if you wish to connect separate query promises together, resulting in duplicate connection clients and a lack of flexibility for combining functions.

The solution is to connect to your database and pass the client or connection handler into your promises. Let’s take a look at a slightly updated dbMongo.js file for use in this article:

This file acts as a means to connect to your database client in an asynchronous fashion.

Next, let’s see what our proposed project structure looks like and how this file fits into it.

Structuring Your Exports

Let’s say I am using Express to handle my app backend. What I wish to do is have my promises as export functions that are readily available to import into my routes. For this example, let’s create a promises folder inside my routes folder:

public/routes/    db/      dbMongo.js    promises/      prmAuth.jsprmUser.js

   auth.js   user.js   ...

views/...

As you can see, the promise files adhere to my route file names, with a prm prefix before the file — this makes it easier to search for these specific files within your IDE.

The same approach is taken with my database clients, which I have opted to keep inside db/. dbMongo.js could be accompanied with dbMysql.js whereby db/ houses all my database connections and simply resolves the connection handler.

Now — our routes directory is beginning to get cluttered, and there are by no means any limitations to where you house your exports. However, the routes folder is the only folder here that utilises db/ and promises/ exports, therefore it makes little sense at this time to move them. Of course, this could change in the future as your project evolves! (think use cases outside of API calls: private backend services not accessible via HTTP).

Next up, let’s explore how we will utilise a promise to check a valid authentication token, within our auth.js file.

Promise Example: Auth Token Valid?

Let’s jump straight into an API call which is housed within my routes/auth.js file. Notice the execution flow here and how readable it is (I will add more promises to this call further down the article to demonstrate that readability is maintained as more promises are added).

Pretty cool implementation. As you can see, the mystery function my_async is lurking within this API call, which is effectively handling my promises by returning an asynchronous function that awaits the promise to resolve.

my_async will become clear soon. Let’s run down this API call before exploring how my_async is declared.

• My db/dbMongo.js exports we visited earlier are being imported via the mongo object. From here, I can access every export with mongo.exportName().
• My authentication related promises are being imported from promises/prmAuth.js and accessed via prmAuth. my_async is being imported from my utils/utils.js file.
• mongo.connect() is called which returns our client object. From here, it can be passed as an argument to any promise that requires it.
• Funnily enough, prmAuth.getAuthToken does require our Mongo client — it is passed into the promise, as well as the authToken.
• We wait for prmAuth.getAuthToken to resolve (courtesy of my_async), and then close the client and resolve the API call in the following then() block.

We are also using some nifty Javascript syntax to check whether prmAuth.getAuthToken resolves a valid data object. Essentially boiling down to:

(expression)   ? handle true result   : handle false result

Of course, if expression is a boolean you could also omit the braces surrounding it, further minimising the syntax.

What Does prmAuth.getAuthToken Look Like?

An important part of our implementation is our promise library. Now, promises can obviously process anything you code them to, but database queries are a very common use case, therefore I am showcasing them here.

Let’s see what the getAuthToken export looks like:

Very simple. Promise libraries do not have to be complicated. We are passing in our client and authToken objects to run a Mongo query to see if we can return a user with the authToken provided. This will determine whether we have a valid authentication token on our hands. We resolve either a successful user record, or an error message via JSON.

The dot syntax of Mongo really compliments Javascript syntax we are adopting here. Take a look at this screenshot to see how clean these exports look — they are not even taking up half of my screen width. Very tidy code.