In this post, we’ll walk through setting up an efficient email queue system in Node.js using Redis to process emails in the background. For a real-world example, we’ll build an OTP (One-Time Password) verification system to show how you can handle time-sensitive emails asynchronously without blocking your application.
Source Code on GitHub
Why Use an Email Queue?
Processing emails in real time during requests can significantly slow down your application. When a server is busy waiting for email processes to complete, it affects response times for users. Using a queue allows us to manage email processes in the background. This not only speeds up requests but also makes the app more responsive and scalable.
Project Overview
Our main goal is to set up:
- A Node.js/Express server.
- A MongoDB-based user model with OTP data storage.
- An email queue to handle OTP emails for account verification.
This architecture can be easily extended to other email-based notifications, such as alerts and promotional messages.
Prerequisites
To follow along, you’ll need:
- Basic knowledge of Node.js and Express
- Redis installed (we’ll use WSL in this example)
- MongoDB Atlas for cloud storage
Key Technologies
- Express: Our server framework
- Mongoose: ODM for MongoDB to handle data schemas
- Redis and Bull: For queue management
- Nodemailer: For sending email notifications
- Nunjucks: Our templating engine to render HTML pages
Project Structure
We’ll structure our files like this:
/models: For our User schema/lib: Email configuration and queue handler/views: HTML templates for signup, OTP input, and dashboard.env: Configuration for secrets like email credentials, Redis, and MongoDB
Step-by-Step Walkthrough
1. Setting up the Express Server
We start by creating a basic Express server (server.js) to handle our routes. Our signup and OTP verification routes will be written here for simplicity.
2. Creating Views
We’ll use Bootstrap for minimal styling to keep the focus on the backend. Our views include:
- signup.html: User signup page
- otp.html: OTP verification page
- dashboard.html: User dashboard after verification
3. Database Configuration
We’ll set up MongoDB with Mongoose and create a User schema that includes fields for the email, password, OTP, and verification status.
4. Email Configuration
In lib/emailService.js, we configure Nodemailer to send OTPs using Gmail’s SMTP settings. This lets us easily manage email templates for future expansion.
5. Building the Email Queue
The queue is configured in lib/emailQueue.js using Bull. Redis serves as our backend for the queue, making it easy to scale and manage background jobs.
6. Signup and Verification Routes
Our signup route creates a new user, generates an OTP, saves it, and queues an email to be sent. In the verification route, users submit their OTP, which is validated against the stored value to confirm the email.
7. Testing and Debugging
Finally, we test the application by signing up with a test email, submitting the OTP, and verifying that the queue processes the email independently of the main application flow.
Conclusion
By setting up an email queue, you’re not only optimizing your application for speed and scalability but also learning a robust pattern for managing any background job. This tutorial showcases an OTP verification example, but the queue setup can be adapted for a wide range of email and notification services.
For the complete code, visit the GitHub repository, and don’t forget to watch the full tutorial for an in-depth, hands-on guide.
0 Comments