2. 程式人生 > >An Overview of JavaScript Testing in 2018

An Overview of JavaScript Testing in 2018

阿新 • • 發佈：2018-12-23

Intro

Look at the logo of Jest, a testing framework by Facebook:

As you can see, their slogan promises a “painless” JavaScript Testing, but as “some guy from the comments” pointed out:

And indeed, Facebook have an excellent reason to use this slogan. In general, JS developers are not too happy with website testing

. JS tests tend to be limited, hard to implement, slow and sometimes expensive.

Nevertheless, with the right strategy and the right combination of tools a nearly full coverage can be achieved and tests can be very organized, simple, and relatively fast.

Test Types

You can read about different test types in more depth

here and here and here.
In general, the most important test types for a website are:

  • Unit Tests- Testing of individual functions or classes by supplying input and making sure the output is as expected.
  • Integration Tests- Testing processes or components to behave as expected, including the side effects.
  • UI Tests- (A.K.A Functional Tests) Testing scenarios on the product itself, by controlling the browser or the website, regardless of the internal structure to ensure expected behavior.

Test Tools Types

Test tools can be divided into the following functionalities. Some provide us with only one functionality, and some provide us with a combination.

In order to get a more flexible functionality, it’s common to use a combination of tools even if one can achieve relatively the same.

  1. Provide a testing structure (Mocha, Jasmine, Jest, Cucumber)
  2. Provide assertions functions (Chai, Jasmine, Jest, Unexpected)
  3. Generate, display, and watch test results (Mocha, Jasmine, Jest, Karma)
  4. Generate and compare snapshots of component and data structures to make sure changes from previous runs are intended (Jest, Ava)
  5. Provide mocks, spies, and stubs (Sinon, Jasmine, enzyme, Jest, testdouble)
  6. Generate code coverage reports (Istanbul, Jest, Blanket)
  7. Provide a browser or browser-like environment with a control on their scenarios execution (Protractor, Nightwatch, Phantom, Casper)

Let’s explain some of the terms mentioned above:

Testing structure refers to the organization of your tests. Nowdays, tests are usually organized in a BDD structure that supports behavior-driven development (BDD). It often looks like this:

Assertion functions are functions that make sure that tested variables contain the expected value. They usually looks like this, where the most popular are the first two:

TIP: Here is a nice article about advanced Jasmine assertions.

Spiesprovide us with information about functions- How many times were they called, in what cases, and by whom?

They are used in integration tests to make sure that the side effects of a process are as expected. For example, how many times was a calculation function called during some process?

Stubbing or dubbing (like doubles in movies) replaces selected functions with selected functions to ensure an expected behavior on selected modules.

If we want to ensure user.isValid() will always return trueduring a test, where you test a different component, for example, you can do this:

It also works with promises:

Mocks or Fakesare faking certain modules or behaviors to test different parts of a processes.

Sinon can, for example, fake a server to ensure offline, fast and expected responses when testing a process.

Snapshot Testing is when you compare a data structure to an expected one.

It won’t actually render and take a picture of the component, but it would save its internal data in a separate file like this:

When the test runs, and the new snapshot is different from the last one, the developer is prompted to agree that the change is intended.

Notice: Snapshots are usually made to compare component representation data but they can also compare other types of data like redux stores and inner structure of different units in the application.

Browser or browser-like environment can be one of the three:

  • jsdom—A Pure JavaScript environment that simulates a real browser. It has no UI and it renders nothing- It provides you with window, document, body, location, cookies, selectors and whatever you get when you run your JS inside the browser.
  • Headless Browser Environment— A browser that runs without a UI for the purpose of making the browser respond faster.
  • Real Browser Environment — An actual browser that opens and runs your tests.

Putting it All Together

We suggest using the same tools for all the test types if possible:
The same testing structure and syntax (1), assertion functions (2), result reporting, and watching mechanism (4).

We also suggest creating two different processes. One for running unit and integration tests and another for UI tests. This is because UI tests takes a longer time, especially if tested on different browsers and usually use an external service to provide you with different devices and browsers (this will be discussed later) and can cost money so you would probably prefer to run it much less than the first process. For example: only before a merge of a feature branch.

Unit Tests

Should cover all small pure units of the application- utils, services and helpers. Provide all these units with simple and edge case inputs and make sure their outputs are as expected using the assertion functions (3). Also make sure to use a coverage reporting tool (6) to know what units are covered.

Unit tests are one of the reasons to use functional programming and pure functions as much as possible-
The purer your application is, the easier you can test it.

Integration Tests

Old school tests were focused on unit testing and resulted in applications where many small parts are working but the processes as a whole keeps on failing.
Integration tests (including snapshots), on the other hand, can detect many unexpected failures where you fix one thing and break the other.
It is also important to remember that in the real world, for the reasons of imperfect design and the widespread use of black boxes, not all units are pure and not all units are testable- Some units can be tested only as part of a bigger process.

Integration tests should cover important cross-module processes. Comparing to unit tests, you would probably use spies (5) to ensure expected side effects instead of just asserting the output and stubs (5) to mock and modify the parts of the process that are not in the specific test.

Also, as opposed to unit tests, a browser or browser-like environment (7) could help with processes that are dependent on the windowand when part of the process is to render certain components or interact with them.

Component snapshot tests (4) fall into this category as well. They provide us with a way to test how processes affect selected components without actually render them or using a browser or browser-like environment.

UI Tests

Sometimes the quick and effective unit and integration tests are not enough.

UI tests are always running inside a browser or browser-like environment (7) that were discussed earlier.

They simulate user behavior on these environments (clicking, typing, scrolling etc…) and make sure these scenarios actually work from the point of view of an end user.

It is important to remember that these tests are the hardest to set up. Imagine yourself creating an environment to run a test on different machines, devices, browser types and versions. This is why there are many services that provide this service for you. And even more can be found here.

List of General Prominent Testing Tools

jsdom

jsdom is a JavaScript implementation of the WHATWG DOM and HTML standards. In other words, jsdom simulates a browser’s environment without running anything but plain JS.

As mentioned before, in this simulated browser environment, tests can run really fast. The drawback of jsdom is that not everything can be simulated outside a real browser (you can’t take a screenshot for example) so using it will limit your test’s reach.

It is worth mentioning that the JS community rapidly improves it and the current version is very close to a real browser.

Istanbul

Istanbul will tell you how much of your code is covered with unit tests. It will report on statement, line, function and branch coverage in percentages so you will understand better what is left to cover.

Karma

Karma lets you run tests in browser and browser like environments including jsdom.

Karma hosts a test server with a special web page to run your tests in the page’s environment. This page can be run across many browsers.

This also means tests can be run remotely using services like BrowserStack.

Chai

Chai is the most popular assertion library.

Unexpected

Unexpected is an assertion library with a slightly different syntax from Chai. It is also extensible so assertions can be more advanced with libraries that are based on it like unexpected-react that you can read about more in depth here.

Sinon.JS

Sinon is a very powerful standalone test spies, stubs and mocks for JavaScript that works with any unit testing framework.

testdouble.js

testdouble is a less popular library that does what Sinon does, and claims to do it better. With a few differences in design, philosophy, and features that could make it useful in many cases. you can read about it here, here and here.

Wallaby

Wallaby is another tool worth mentioning. It is not free, but many users recommend buying it. It runs on your IDE (it supports all major ones) and runs relevant to your code changes tests and indicates if anything fails in real time just alongside your code.

Cucumber

Cucumber help with writing tests in BDD by dividing them between the acceptance criteria files using the Gherkin syntax and the tests that are correspondent to them.

Tests can be written in a variety of languages that are supported by the framework, including JS that we are focusing on:

Many teams will find this syntax more convenient than TDD.

Choose Your Unit and Integration Tests Framework

The first choice you should probably make is what framework do you want to use and libraries to support it. It is recommended to use the tools your framework provides until a need for unique tools arises.

* In short, if you want to “just get started” or looking for a fast framework for large projects, go with Jest.
* If you want a very flexible and extendable configuration, go with Mocha.
* If you are looking for simplicity go with Ava.
* If you want to be really low-level, go with tape.

Here is a list of the most prominent tools with some of their characteristics:

mocha

Mocha is currently the most used library. Unlike Jasmine, it is used with third party assertion, mocking, and spying tools (usually Enzyme and Chai).

This means Mocha is a little harder to set up and divided into more libraries but it is more flexible and open to extensions.

For example, if you want special assertion logic, you can fork Chai and replace only Chai with your own assertion library. This can also be done in Jasmine but in Mocka this change will be more obvious.

  • Community- Has many plugins and extension to test unique scenarios.
  • Extensibility- Plugins, extensions and libraries such as Sinon includes features Jasmine does not have.
  • Globals- Creates test structure globals by default, but obviously not assertions, spies and mocks like Jasmine- some people are surprised by this seemingly inconsistency of globals.

Jest

Jest is the testing framework recommended by Facebook. It is based on Jasmine that we will discuss later. By today Facebook replaced most of its functionality and a added a lot of features on top of it.

After reading a huge amount of articles and blog posts, it’s incredible how people are impressed by Jest’s speed and convenience.
  • Performance- First of all Jest is considered to be faster for big projects with many test files by implementing aclever parallel testing mechanism (For example by us from our experience and in these blog posts: here, here, here, here).
  • UI- Clear and convenient.
  • Ready-To-Go- Comes with assertions, spies, mocks that are equivalent to libraries that do the same like Sinon. Libraries still can easily be used in case you need some unique features.
  • Globals- Like in Jasmine, it creates test globals by default so there is no need to require them. This can also be considered bad since it makes your tests less flexible and less controllable, but in most cases it just makes your life easier:
  • Snapshot testing- jest-snapshot is developed and maintained by Facebook, although it can be used in almost any other framework as part of the framework’s integration of the tool or by using the right plugins.
  • Improved modules mocking- Jest provides you with an easy way to mock heavy modules to improve testing speed. For example a service can be mocked to resolve a promise instead of making a network request.
  • Code coverage- Includes a powerful and fast built-in code coverage tool that is based on Istanbul.
  • Reliability- Although this is a relatively young library, throughout 2017 Jest stabilized and is now considered reliable. It is currently supported by all the major IDEs and tools.
  • Development- jest only updates the files updated so tests are running very fast in watch mode.

jasmine

Jasmine is the testing framework that Jest is based on. Why would you still use Jasmine? It has been around for a longer time and has a huge amount of articles and tools that were created by the community.

Also, Angular still suggests using it over Jest, although Jest is perfectly suitable to run Angular tests as well, and many people do it.

  • Ready-To-Go- Comes with everything you need to start testing.
  • Globals- Comes with all the important testing features in the global scope as well.
  • Community- It has been on the market since 2009 and gathered a vast amount of articles, suggestions and tools that are based on it.
  • Angular- Has widespread Angular support for all its versions and it is what recommended in the official Angular documentation.

AVA

Ava is a minimalistic testing library that runs tests in parallel.

  • Ready-To-Go- Comes with everything you need to start testing (besides spying and dubbing that you can add in no-time). Using the following syntax for test structure and assertions, and runs in Node.js:
  • Globals- As seen above, it does not create any test globals thus you have more control over your tests.
  • Simplicity- Simple structure and assertions without a complex API while supporting many advanced features.
  • Development- Ava only updates the files updated so tests are running fast in watch mode.
  • Speed- Runs tests in parallel as separate Node.js processes.

tape

Tape is the simple of them all. It’s just a JS file you run with node with a very short and “to-the-point” API.

  • Simplicity- Minimalistic structure and assertions without a complex API. Even more than Ava.
  • Globals- Does not create any test globals thus you have more control over your tests.
  • No Shared State between tests- Tape discourages the use of functions like “beforeEach” to ensure test modularity and maximum user control over the tests cycle.
  • No CLI is needed- Tape is simply run anywhere JS can be run.

UI Testing Tools

First of all, as mentioned above, here and here you can find great articles about service providers that would host the machines where tests would run and help you run these tests on different devices and browsers.

The number of permanent tools for the purpose of UI testing differs very much from each other both in their implementation, philosophy and API, so it is strongly suggested to invest time in understanding the different solutions and testing them on your product.

* In short, if you want to “just get started” with a reliable and simple to set-up cross-browser all-in one tool, go with TestCafe.
* If you want to go with the flow and have maximum community support, WebdriverIO is the way to go.
* If you don’t care about cross-browser support, use Puppeteer.
* If your application has no complex user interactions and graphics, like a system full of forms and navigations, use cross browser headless tools like Casper.

selenium

Selenium, automates the browser to simulate user behavior. It is not written specifically for tests and can control a browser for many purposes by exposing a server that simulates user behavior on a browser using an API.

Selenium can be controlled in many ways and using a variety programming languages, and with some tools even without any real programming.

To our needs, however, Selenium server is controlled by a Selenium WebDriverthat serves as a communication layer between our NodeJS and the server that operates the browser.

Node.js <=> WebDriver <=> Selenium Server <=> FF/Chrome/IE/Safari

The WebDriver can be imported into your testing framework and tests can be written as part of it:

The WebDriver itself might be sufficient for you and indeed some people suggest using it as it is, but various libraries were created to extend it ether by forking and altering it or by wrapping it.

And indeed wrapping the WebDriver might add redundant code and could make debugging harder, whereas forking it might diverge it from the very active (as for 2018) ongoing development of WebDriver.

Still, some people prefer to not use it directly. Let’s look at some of libraries for selenium operating:

Appium

Apium provides an API similar to Selenium for testing websites on a mobile device using the following tools:

So if you use Selenium or Selenium based tools, you can also use Apium to test on mobile devices.

Protractor

Protractor is a library that wraps Selenium to add it an improved syntax and special in-built hooks for Angular.

  • Angular- Has special hooks, although can be successfully used with other JS frameworks too. Angular official documentation suggests using this tool.
  • Error reporting- Good mechanism.
  • Support- TypeScript support is available and the library is operated and maintained by the huge Angular team.

WebdriverIO

WebdriverIO has its own implementation of the selenium WebDriver.

  • Syntax- very easy and readable.
  • Flexible- A very simple and agnostic from even being used for tests, flexible and extensible library.
  • Community- It has good support and enthusiastic developer community that makes it reach with plugins and extensions.

Nightwatch

Nightwatch has its own implementation of the selenium WebDriver. And provides its own testing framework with a test server, assertions, and tools.

  • Framework- Can be used with other frameworks too, but can be especially useful in case you want to run functional tests not as part of other framework.
  • Syntax- looks the easiest and the most readable.
  • Support- No typescript support and in general, this library seems to be slightly less supported than the others.

TestCafe

TestCafe is a great alternative to Selenium-Based tools. It was rewritten and open-sourced in the end of 2016.

There is still a paid version that offers non programming testing tools like a test recorder and a customer support, this is important because many outdated articles mistakenly state that its code is closed and count it like the libraries disadvantage.

It injects itself into the website as JavaScript scripts instead of controlling the browsers themselves like Selenium does. This allows TestCafe to run on any browser, including on mobile devices, and have a full control over the JavaScript execution loop.

TestCafe is JavaScript and test oriented. It is under massive development right now, although it is already considered stable and full of features.