Introducing the express-typeorm-postgres Starter Kit

At This Dot, we’ve been working with ExpressJS APIs for a while, and we’ve created a starter kit that you can use to scaffold your next ExpressJS backend project. The starter kit uses many well-known npm packages, such as TypeOrm or BullMQ and integrates with databases such as PostgreSQL and Redis .

Kit contents

The express-typeorm-postgres starter kit provides you with infrastructure for development, and integrations with these infrastructures. It comes with a working Redis instance for caching and a second Redis instance for queues. It also starts up a Postgres instance for you, which you can seed with TypeOrm.

The generated project comes with prettier and eslint set up, so you only need to spend time on configuration if you want to tweak or change the existing rules.

Unit testing is set up using Jest, and there are some example tests provided with the example controllers.

How to initialise

API development usually requires more infrastructure than front-end development. Before you start, please make sure you have docker and docker-compose installed on your machine.

To initialize a project with the express-typeorm-postgres kit, run the following:

1. Run npx @this-dot/create-starter to run the scaffolding tool
2. Select the Express.js, TypeOrm, and PostgreSQL kit from the CLI library options
3. Name your project
4. cd into your project directory, install dependencies using the tool of your choice (npm, yarn or pnpm)
5. copy the contents of the .env.example file into a .env file

With this setup you have a working starter kit that you can modify to your needs.

TypeOrm and Database

When we started developing the kit, we decided to use PostgreSQL as the database, because it is a powerful, open-source object-relational database system that is widely used for storing and manipulating data. It has a strong reputation for reliability, performance, and feature richness, making it a great choice for a wide range of applications. It can also handle high levels of concurrency and large amounts of data, and supports complex queries and data types. Postgres is also highly extensible because it allows developers to add custom functions and data types to the database. It has a large and active community of developers and users who contribute to its ongoing development and support.

The kit uses TypeORM to connect to the database instance. We chose TypeORM because it makes it easy to manage database connections and perform common database operations, such as querying, inserting, updating and deleting data. It supports TypeScript and a wide range of databases, such as PostgreSQL, MySQL, SQLite and MongoDB, therefore if you want to be able to switch between databases, it makes it easier.

TypeORM also includes features such as database migrations, which help manage changes to database schema over time, and an entity model that allows you to define your database schema using classes and decorators. Overall, TypeORM is a useful tool for improving the efficiency and reliability of database-related code, and it can be a valuable addition to any TypeScript or JavaScript project that needs to interact with a database.

To seed an initial set of data into your database, run the following commands:

1. npm run infrastructure:start - this starts up the database instance
2. npm run db:seed - this leverages TypeOrm to seed the database.

The seed command runs the src/db/run-seeders.ts file, where you can introduce your own seeders for your own needs. The kit uses typeorm-extension for seeding. Please refer to the src/db/seeding/technology-seeder.ts file for an example.

Caching

Storing response data in caches allows subsequent requests for the same data to be served more quickly. This can improve the performance and user experience of an API by reducing the amount of time it takes to retrieve data from the server. It can reduce the load on the database or mitigate rate limiting on third-party APIs called from your back-end. It also improves the reliability of an application by providing a fallback mechanism in case the database or the server is unavailable or slow to respond.

There is a Redis instance set up to be used for caching data. Under the hood we use the cachified library to store cached data in Redis. The kit has a useCache method exported from src/cache/cache.ts, which requires a key and a callback function to be called to fetch the data.

const technologyId: number = parseInt(req.params.technologyId);
const technologyResult = await useCache<TechnologyResult>(req.originalUrl, () =>
  findTechnology(technologyId)
);

When you need to invalidate cache entries, you can use the clearCacheEntry method by supplying a key string to it. It will remove the cached data from Redis, and the next request that fetches from the database will cache the new values.

const technologyId: number = parseInt(req.params.technologyId);
const updateResult = await updateTechnologyEntry(technologyId, {
  displayName: req.body.name,
  description: req.body.description,
});

// ...

clearCacheEntry(req.baseUrl);
clearCacheEntry(req.originalUrl);

Under the src/modules/technology folder, you can see a complete example of a basic CRUD REST endpoint with caching enabled. Feel free to use those handlers as examples for your development needs.

Queue

A message queue allows different parts of an application, or different applications, to communicate with each other asynchronously by sending and receiving messages. This can be useful in a variety of situations, such as when one part of the application needs to perform a task that could take a long time, or when different parts of the application need to be decoupled from each other for flexibility and scalability.

We chose BullMQ because it is a fast, reliable, and feature-rich message queue system. It is built on top of the popular Redis in-memory data store, which makes it very performant and scalable. It has support for parallel processing, rate limiting, retries, and a variety of other features that make it well-suited for a wide range of use cases. BullMQ has a straightforward API and good documentation.

The kit has a second Redis instance set up to be used with BullMQ, and there is a queue set up out of the box, so resource-intensive tasks can be offloaded to a background process. The src/queue/ folder contains all the configuration and setup steps for the queue.

Both the queue and its worker is set up in the queue.ts file. The job-processor.ts file contains the function that will process the data. To run int in a separate thread, we must pass the path to this file into the worker:

const processorPath = path.join(__dirname, 'job-processor.js');
export const defaultWorker = new Worker(queueName, processorPath, {
	connection: {
		host: REDIS_QUEUE_HOST,
		port: REDIS_QUEUE_PORT,
	},
	autorun: true,
});

When to use this kit

This kit is most optimal when you:

• want to build back-end services that can be consumed by other applications and services using ExpressJS
• need a flexible and scalable way to build server-side applications
• need to deal with CPU-intense operations on the server and you need a messaging queue
• need to build an API with relational data
• would like to just jump right into API development with ExpressJS using TypeORM and Postgres

Conclusion

The express-typeorm-postgres starter kit can help you kickstart your development by providing you with a working preset. It has testing configured, and it comes with a complete infrastructure orchestrated by docker-compose.

This Dot Labs is a development consultancy that is trusted by top industry companies, including Stripe, Xero, Wikimedia, Docusign, and Twilio. This Dot takes a hands-on approach by providing tailored development strategies to help you approach your most pressing challenges with clarity and confidence. Whether it’s bridging the gap between business and technology or modernizing legacy systems, you’ll find a breadth of experience and knowledge you need. Check out how This Dot Labs can empower your tech journey.