Introduction

Hey there! Ever wanted to build an app that can handle lots of users and requests seamlessly? That's our mission today. We're going to build an app using three cool tools: NodeJS, NestJS, RabbitMQ

By the end of this article, we'll have a working app that's built to handle lots of action. Whether you're new to these tools or just looking for some hands-on practice, stick around. Let's build something cool together.

Getting to Know Our Tools: NodeJS, NestJS, and RabbitMQ

Before we jump into building the app, let's get to know the three main tools we'll use, a bit better. Imagine building a rocket (I love rockets by the way 🚀 ): you need an engine to power it, a design to follow, and someone to make sure all the tiny parts are assembled at the right time throughout the assembly line. That's pretty much how our app-building process will go with these tools!

NodeJS: The Engine

• What is it? NodeJS lets our computer understand and run JavaScript. So, we're not just using JavaScript for web stuff; now we can use it to build our entire app!

• Why is it cool? It's fast, think of it as a super-efficient waiter in a restaurant that is always moving, ensuring everything happens at the right speed, not waiting around.

NestJS: The Design or Blueprint

• What is it? NestJS helps us organize our app. NestJS gives us a neat blueprint to follow, which is great for scalability.

• Why is it cool? It keeps things neat. As our app grows, it's super handy to know where everything goes. It's like having a map of a big city!

RabbitMQ: The Assembly Helper

• What is it? RabbitMQ helps when our app has many tasks or messages to handle. It's like a traffic-policeman (roulage), making sure everything flows smoothly.

• Why is it cool? It prevents chaos. Imagine sending hundreds of letters at once. RabbitMQ ensures each one gets its turn at the right time, so there's no big mess and the system is not overwhelmed.

There we have it! With our engine (NodeJS), our map (NestJS), and our helper (RabbitMQ), we're ready to start building. Each tool has its job, and together, they'll help us create an app that's both powerful and organized.

Building Our Scalable Application

Now that we're familiar with the tools (NodeJS, NestJS, and RabbitMQ), let's start building!

We're going to be building a user management system. Here’s the plan:

• Users will sign up.

• Admins manage these users.

• Every action (like a user signing up or an admin making a change) will be logged.

• These logs? They'll be saved into a file.

• And on top, we’ll use RabbitMQ to handle all these logs efficiently.

Sounds fun, right? Let’s get started!

Step1: Setting Up Our Workspace

Installing the NestJS CLI: This command line tool makes it super easy to create and manage our NestJS projects.

# First, we install the NestJS CLI tool:
npm i -g @nestjs/cli

Creating Our NestJS Project: Now, let’s create our user management system project using the previously installed NestJS CLI:

# Then, we create our project:
nest new user-management-system

Follow the prompts, and in a few minutes, you'll have a brand-new NestJS project ready to go!

Installing dependencies: Now, let’s install the dependencies of our project:

# Navigate to our project's directory
cd user-management-system
# Install TypeORM, we'll use to connect to the database:
npm install --save @nestjs/typeorm typeorm mysql

Installing RabbitMQ: It will manage our logs, ensuring everything flows smoothly.

• Windows & macOS: Download the installer from the RabbitMQ website.

• Linux (Ubuntu example):

    # We install the rabbitmq server:
    sudo apt-get install -y rabbitmq-server
  

  After installation, we can start RabbitMQ:

    # We run this command to start the rabbitmq server:
    sudo service rabbitmq-server start
  

With these tools installed and set up, we've got a solid foundation. We are now ready to start building our app.

Step2: Building the Core of Our User Management System

Now that our tools are set up, let's start putting pieces together.

Setting up the User Module

Every user action, like signing up or updating info, will happen here.
Using the NestJS CLI, we can generate both the module and service for users.

nest generate module user
nest generate service user
nest generate controller user

Setting up the Admin Module

Let's use the CLI again to set up the Admins' module and service for admin actions.

nest generate module admin
nest generate service admin
nest generate controller admin

After setting up the User and Admin modules using the NestJS CLI, the directory structure of our project should typically look like this:

user-management-system/
│
├── src/
│   ├── admin/
│   │   ├── admin.module.ts
│   │   ├── admin.service.ts
│   │   └── admin.controller.ts
│   │
│   ├── user/
│   │   ├── user.module.ts
│   │   ├── user.service.ts
│   │   └── user.controller.ts
│   │
│   ├── app.controller.ts
│   ├── app.module.ts
│   ├── app.service.ts
│   └── main.ts
│
├── node_modules/
│
├── test/
│
├── nest-cli.json
├── package.json
├── tsconfig.build.json
├── tsconfig.json

This structure keeps everything neat and organized, making it easier for us (and any other developer) to navigate and understand the project.

Focusing the User Module

The User module is where all the magic around user management happens. From sign-up to profile updates, this module takes care of it all. Here's how we’ll develop it:

User Entity & Database Integration

We need to define what a 'user' looks like in our system.

In the user/user.entity.ts:

import { Entity, PrimaryGeneratedColumn, Column } from 'typeorm';

@Entity('users') // This tells TypeORM to link this entity with the 'users' table in our database
export class User {
  @PrimaryGeneratedColumn()
  id: number;

  @Column()
  username: string;

  @Column()
  password: string;  // In a real app, remember never to store passwords in plain text!

  @Column()
  email: string;

  // ... We can add any other fields you deem necessary here, like dateJoined, profileImage, etc.
}

User Service

Let's set up the user service to handle all the app's logic related to users.

In the user/user.service.ts:

import { Injectable, ConflictException } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { User } from './entities/user.entity';

@Injectable()
export class UserService {
  constructor(
    @InjectRepository(User)
    private readonly userRepository: Repository<User>,
  ) {}

  async signUp(userData: User): Promise<string> {
    // First, we'll check if a user with the same email already exists.
    const existingUser = await this.userRepository.findOne({ email: userData.email });

    if (existingUser) {
      throw new ConflictException('Email already exists');
    }

    // In a real-world app, we'd want to hash the password here. 
    // For simplicity's sake, we're skipping that step. Never store passwords in plain text!

    const newUser = this.userRepository.create(userData); // This creates a new User instance but doesn't save to the database yet
    await this.userRepository.save(newUser); // This saves the User instance to the database

    return 'User has been successfully signed up!';
  }

  // More methods related to user functionalities can be added here in the future.
  // For example: userProfile(), updateUser(), deleteUser(), etc.
}

User Controller

Let's set up the user controller to handle the API endpoints related to users.

In the user/user.controller.ts:

import { Controller, Post, Body } from '@nestjs/common';
import { UserService } from './user.service';
import { User } from './entities/user.entity';

@Controller('users')
export class UserController {
  constructor(private userService: UserService) {}

  @Post('signup')
  async signUp(@Body() userData: User): Promise<string> {
    return this.userService.signUp(userData);
  }

  // More endpoints related to user functionalities can be added here in the future.
}

User Module

Let's set up the user module to tie everything together: providers, controllers and entities.

In the user/user.module.ts:

import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { UserController } from './user.controller';
import { UserService } from './user.service';
import { User } from './entities/user.entity';

@Module({
  imports: [TypeOrmModule.forFeature([User])],  // This allows us to inject the UserRepository into our UserService
  controllers: [UserController],
  providers: [UserService],
  exports: [UserService],  // In case we want to use the UserService in another module
})
export class UserModule {}

With the user module set up like this, it ensures that everything related to user management is encapsulated in one module, making it easier to manage, update, and debug.

Step3: Integrating Logging with RabbitMQ

Now that our basic user module is set up, let's move on to the logging service with RabbitMQ for our project.

Setting Up RabbitMQ in NestJS

We’ve already installed the necessary packages for RabbitMQ. Let’s now set it up within our app.

Update the main app's module for our user management system, taking into account the modules and services we have discussed, including setting up RabbitMQ and its configurations.

In the app.module.ts:

import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { RabbitMQModule } from '@nestjs-plus/rabbitmq';

// Modules
import { UserModule } from './user/user.module';

// Entities
import { User } from './user/entities/user.entity';

// Logger
import { LoggerService } from './logger/logger.service';

@Module({
  imports: [
    // TypeORM Setup
    TypeOrmModule.forRoot({
      type: 'mysql',
      host: 'localhost',
      port: 3306,
      username: 'test',
      password: 'test',
      database: 'test',
      entities: [User], // We can add other entities here as we develop them
      synchronize: true,
    }),

    // RabbitMQ Setup
    RabbitMQModule.forRoot(RabbitMQModule, {
      exchanges: [
        {
          name: 'log-exchange',
          type: 'topic'
        }
      ],
      uri: 'amqp://localhost', // as rabbitMQ is running locally, in production we wil use production variables
    }),

    // User Module
    UserModule,
  ],
  controllers: [],
  providers: [],
})
export class AppModule {}

With this setup, our app.module.ts provides a clear overview of the major pieces of our user management system. This structure ensures that new modules and services can be easily integrated as the application grows.

Creating the Logger Service for RabbitMQ

Let's create a dedicated LoggerService to interact with RabbitMQ and manage our logging activities.

We first create a logger.service.ts

nest generate service logger

Inside logger.service.ts we implement LoggerService with RabbitMQ Integration:

import { Injectable } from '@nestjs/common';
import { RabbitMQService } from '@nestjs-plus/rabbitmq';

@Injectable()
export class LoggerService {
  constructor(private readonly rabbitMQService: RabbitMQService) {}

  log(message: string) {
    // Publish the log message to RabbitMQ
    this.rabbitMQService.publish('log-exchange', 'log-routing-key', message);
  }
}

Creating a Logger Middleware

Let's create a logging middleware that will intercept all incoming requests and log the request data using the LoggerService.

We first create a Logger Middleware

nest generate middleware logger

Inside logger.middleware.ts we implement the middleware to log request data:

import { Injectable, NestMiddleware } from '@nestjs/common';
import { Request, Response, NextFunction } from 'express';
import { LoggerService } from './logger/logger.service';

@Injectable()
export class LoggerMiddleware implements NestMiddleware {

  constructor(private readonly loggerService: LoggerService) {}

  use(req: Request, res: Response, next: NextFunction) {
    const { method, originalUrl, body } = req;

    // Create a log message
    const logMessage = `Method: ${method}, URL: ${originalUrl}, Body: ${JSON.stringify(body)}`;

    // Send this log message to the LoggerService
    this.loggerService.log(logMessage);

    // Continue with request processing
    next();
  }
}

Now, we need to apply this middleware to our application. Typically, we do this in the main module, app.module.ts.

Let's add the following to our app.module.ts:

import { Module, NestModule, MiddlewareConsumer } from '@nestjs/common';
import { LoggerMiddleware } from './logger/logger.middleware';
import { LoggerService } from './logger/logger.service';

// ... other imports ...

@Module({
  // ... our module setup as we already did

    providers: [LoggerService],  // We add "LoggerService" as it is provided at the app-level so it can be used throughout the application, we could also make Logger its own module if we want and import it in the modules
})
export class AppModule implements NestModule {
  configure(consumer: MiddlewareConsumer) {
    consumer
      .apply(LoggerMiddleware)
      .forRoutes('*');  // This applies the middleware to all routes
  }
}

With the middleware in place, every incoming request to the application will now be logged using the LoggerService. This setup ensures that we have a consistent logging mechanism capturing every interaction with our application.

Step4: Building the logs-consumer Application

For the purpose of this article, let's treat the logs consumer as a separate NestJS application, it will help us understand much better the concept of message queuing.

Setting Up a New NestJS App (logs consumer):

We now create a new NestJS App to consume our logs from RabbitMQ:

nest new logs-consumer

Installing RabbitMQ Modules

We navigate to the logs-consumer directory and install the necessary packages:

cd logs-consumer
npm install @nestjs-plus/rabbitmq

Implementing the Logs Consumer Service

We first create a service called logsConsumer using the NestJS CLI just as we have been doing so far:

nest generate service logsConsumer

Inside the generated logs-consumer.service.ts:

import { Injectable } from '@nestjs/common';
import { RabbitSubscribe } from '@nestjs-plus/rabbitmq';
import * as fs from 'fs';

@Injectable()
export class LogsConsumerService {

  @RabbitSubscribe({
    exchange: 'log-exchange',
    routingKey: 'log-routing-key',
    queue: 'log-queue'
  })
  public async handleLogMessage(msg: any) {
    // Here, we can handle the log message as we wish.

    // For this example (our project), we'll simply write it to a file.

    const logFilePath = './logs.txt'; 
    fs.appendFileSync(logFilePath, `${new Date().toISOString()}: ${msg.content.toString()}\n`);
  }
}

Configuring RabbitMQ in the logs-consumer App

In the main app module of the new app (logs-consumer) app.module.ts, we configure RabbitMQ:

import { Module } from '@nestjs/common';
import { RabbitMQModule } from '@nestjs-plus/rabbitmq';
import { LogsConsumerService } from './logs-consumer.service';

@Module({
  imports: [
    RabbitMQModule.forRoot(RabbitMQModule, {
      exchanges: [
        {
          name: 'log-exchange',
          type: 'topic'
        }
      ],
      uri: 'amqp://localhost',
    }),
  ],
  providers: [LogsConsumerService],
})
export class AppModule {}

With this setup, the logs-consumer application will listen for messages on RabbitMQ. When a log message is queued up by your main application, the logs-consumer will retrieve the message and append it to a file. In real-world scenarios, the consumer could process these logs in various ways, including forwarding them to centralized logging systems, triggering alerts, analyzing the logs for insights and much more.

Conclusion

Throughout this article, we embarked on a journey to create a scalable application using the powerful trio of Node.js, NestJS and RabbitMQ to empower a dedicated logging system.

Key Takeaways:

• Modularity with NestJS: We've seen how NestJS's modular structure simplifies building scalable and maintainable applications. Each module, like the User module, encapsulates specific functionalities, making it easier to develop, debug, and expand.

• Scalability with RabbitMQ: Modern web applications need to manage a bunch of tasks efficiently, especially under high-traffic conditions. RabbitMQ acts as a communicator, ensuring that the system processes tasks smoothly, balancing loads, and maintaining high performance even during peak times.

• Decoupling via Dedicated Consumers: Splitting responsibilities enhances clarity. By designating separate applications or services to handle specific tasks, the primary application remains neat and efficient. It delegates tasks to dedicated consumers, allowing for specialized processing and ensuring each component of the system does what it does best.

Looking Ahead:

While we've built a solid foundation, the world of backend development is vast, and there's always more to explore:

• Security: Our application can benefit from additional security layers, like JWT authentication for users and secure channels for RabbitMQ.

• Optimizing RabbitMQ: As our application scales, diving deeper into RabbitMQ’s offerings will be crucial. Implementing features like dead-letter exchanges and message prioritization can make our logging system even more robust.

• Monitoring and Alerts: With the logging in place, setting up monitoring and alerting systems would be the next logical step, ensuring system health and in-time interventions if something goes wrong.

Building this application was both a challenge and a revelation. It underlined the importance of choosing the right tools and designing with scalability in mind. As developers, our choices early in the development lifecycle can significantly impact the application's performance and maintainability down the road.

Thank you for joining me on this journey. Whether you're building a startup MVP or an enterprise system, I hope this guide empowers you to craft efficient, scalable, and resilient backend systems. Until next time, keep coding, keep learning!