Automation via generation

Once we had fully fledged out the architecture of the system, we needed to guarantee that both ends of the app were typed. This would mean that any changes to our .proto files or any files containing gql tags would result in compile time errors as opposed to runtime errors.

Generating TypeScript types from a GraphQL schema

One of the most powerful features offered by GraphQL is introspection. The ability to understand the structure of an entire API through a single query. The kind people at Apollo provide a library called graphql-tools which, given an endpoint, converts a schema into an equivalent JSON interpretation.

Once you have a JSON file, you can use GraphQL Code Generator, alongside its TypeScript plugin, to generate a TypeScript definition (d.ts) file from the schema.json file.

The downside of the tools above is that they rely on a running instance of the server. It’s annoying to have to remember to start the server everytime we want to regenerate the types; so we created a (slightly primitive)bash

1. Spin up a server in a background processs.
2. Introspect the schema and generate the schema.json file.
3. Kill the server.
4. Generate TypeScript definitions from the schema.json file.

Using the simple script above, TypeScript types are generated for us. This means that if we change the following schema.ts:

After defining our query, running tools/generate-gql.sh will generate the following schema.d.ts file for us:

The above types mean that we can even type our resolvers:

Using schema.json on the front end

The schema.json file goes beyond generating back end types — it can be used by any clients that consume the API to generate their own types. This means that when we change our API, our client (a front end app which is also written in TypeScript) is able to download the schema, generate types and have any potential errors show up at compile time — a win for both API and front end development.

Comparison of GraphQL TypeScript generation tools

We found two great tools that allowed for typed, generated code:

We decided to use GraphQL Code Generator as it generates resolver types, as well as Query and Mutation structures — meaning there’s no way that you can change an existing query/mutation without having to modify its accompanying resolver as can be seen by QueryResolvers.FetchChickenResolver, above.

Generating gRPC clients and types from proto files

Whilst we considered and tested using dynamically generated code at run time for our gRPC clients, statically generated code provided typing and all the advantages of compile time checking.

When you have protobuf API definition files that are shared across different services, it’s hard to share these files and keep your API consumption and implementation up to date. This problem is compounded when your consumers and producers are in separated across multiple git repos. To fix this, we have a proto repo that’s installed as a git submodule (proto directory in directory overview) in the consumers and producers, allowing for the API definition files to be shared across more easily. Here Statically generated JavaScript and TypeScript GRfilesis an example of the structure:

. ├── proto/ │   ├── chicken_service/ │   │   ├── model/ │   │   |   └── chicken.proto │   │   └── chicken_service.proto │   └── ... └── …

Models are separated from service and request definitions to allow for re-usability, and each service has its own directory so we can target code generation on a per-service basis.

In order to generate our TypeScript types, we chose to use grpc_tools_node_protoc. A one-man-created library that would generate the necessary JavaScript code, and TypeScript definitions to instantiate and use our clients — which was what we needed.

Quick note: We’re always wary of depending on libraries with limited support, but agreatfool has done an amazing job — we’ve had very few issues using this library.

Using this, we created a small bash script that generates JavaScript files and corresponding d.ts files:

We run this script whenever changes are made to our proto files — doing so will generate the following files:

Which, in turn, allows us to communicate to clients as follows:

Comparison of gRPC TypeScript generation tools

We found two tools that allowed for typed, generated code:

• Improbable’s grpc-web: A tool built to communicate to services implementing gRPC over the Web from a front-end client by creating a gateway proxy between the client and server.

We evaluated grpc-web first, as it’s built by a well know company and has multiple contributors, even though it is meant for the Web. There was one issue with this:

This package supports Node.js, but requires that the server has the gRPC-Web compatibility layer.

Due to the large amount of work needed for this, as well as forcing a dependency on each of our producers, we weren’t willing to implement this compatibility layer on each of our producers.

Setback: Error handling in gRPC

gRPC offers providers a number of different errors to choose from, that a producer can throw. Errors such as GRPC_STATUS_NOT_FOUND, however, are too vague to get any concrete value out of — we need to know what is not found and why it’s not found.

On our producers, we set this using the details property — think metadata for errors — and add something more explicit, e.g. CHICKEN_NOT_FOUND_IN_COUP.

At the time this article was written, if both code and details were provided, when using the Node.js library, one was overwritten by the other — meaning we were unable to get explicit error types without a few hacks.

Another quirk is that the TypeScript typing generated sets error type to any, meaning we need to explicitly typecast the error.