Discover the critical role of Message Queues (MQ) in distributed systems. From asynchronous processing to service decoupling, traffic shaping, and beyond, this article explores 10 practical and classic use cases for MQ with detailed examples and code snippets to enhance your system's stability, scalability, and efficiency.

Message Queues (MQ): An Indispensable Technology in Distributed Systems

For many newcomers, MQ might initially seem like just a "messaging tool." However, as you use it, you'll realize it’s more like the "lubricant" of your system.

Whether for decoupling, traffic shaping, or handling asynchronous tasks, MQ plays a vital role.

Here, I'll combine practical scenarios and systematically break down 10 classic uses of MQ, from simple to complex. I hope this proves helpful to you.

1. Asynchronous Processing: Lighten the System Load

Scenario
You may often encounter situations like this: a user submits an operation, such as placing an order, which requires sending an SMS notification.
If the SMS API is called directly in the main flow, a slow SMS service could drag down the entire process.
This leads to impatient users and a negative experience.

Solution
Use MQ to offload non-critical processes asynchronously. When an order is placed, delegate the SMS task to MQ. The order service can immediately respond to the user, while MQ and the consumer handle the SMS task.

Code Example

// Order Service: Producer
Order order = createOrder(); // Order generation logic
rabbitTemplate.convertAndSend("order_exchange", "order_key", order);
System.out.println("Order created, SMS task delegated to MQ");

// SMS Service: Consumer
@RabbitListener(queues = "sms_queue")
public void sendSms(Order order) {
    System.out.println("Sending SMS for Order ID: " + order.getId());
    // Call SMS service API
}

Deep Dive
This approach decouples the main flow from slow services. The order service focuses on its task, and even if the SMS service fails, MQ can temporarily store the message, ensuring it’s processed once the SMS service recovers.

2. Traffic Shaping: Stabilize the System

Scenario
During events like "Singles' Day" e-commerce sales, a surge of users floods the system to grab deals.
Such high concurrent requests can not only overwhelm application services but also bring the database to a standstill.

Solution
First, queue seckill (flash sale) requests into MQ. The backend service then consumes these messages at a steady rate to process orders.
This prevents the system from being overwhelmed by instantaneous traffic while improving processing stability.

Code Example

// User submits seckill request: Producer
rabbitTemplate.convertAndSend("seckill_exchange", "seckill_key", userRequest);
System.out.println("User seckill request queued");

// Seckill Service: Consumer
@RabbitListener(queues = "seckill_queue")
public void processSeckill(UserRequest request) {
    System.out.println("Processing seckill request for User ID: " + request.getUserId());
    // Execute seckill logic
}

Deep Dive
MQ acts as a buffer pool here, distributing peak traffic over a period of time for steady processing. This improves system stability and user experience.

3. Service Decoupling: Reduce Mutual Dependency

Scenario
An order system needs to notify the inventory system to deduct stock and the payment system to process payment.
Direct synchronous API calls create strong dependencies between services. If one service fails, the entire chain collapses.

Solution
The order service only needs to send messages to MQ. The inventory and payment services consume these messages independently. This way, the order service doesn't directly depend on them.

Code Example

// Order Service: Producer
rabbitTemplate.convertAndSend("order_exchange", "order_key", order);
System.out.println("Order message sent");

// Inventory Service: Consumer
@RabbitListener(queues = "stock_queue")
public void updateStock(Order order) {
    System.out.println("Deducting stock for Order ID: " + order.getId());
}

// Payment Service: Consumer
@RabbitListener(queues = "payment_queue")
public void processPayment(Order order) {
    System.out.println("Processing payment for Order ID: " + order.getId());
}

Deep Dive
Using MQ achieves loose coupling between services. Even if the inventory service fails, the order generation process remains unaffected, significantly improving system fault tolerance.

4. Distributed Transactions: Ensure Data Consistency

Scenario
The order service needs to create an order and deduct inventory, which involves operations on two separate databases.
If one succeeds while the other fails, it leads to data inconsistency.

Solution
Achieve distributed transactions through MQ. After creating an order, the order service delegates the inventory deduction task to MQ, ensuring eventual consistency.

Code Example

// Order Service: Producer
rabbitTemplate.convertAndSend("order_exchange", "order_key", order);
System.out.println("Order creation message sent");

// Inventory Service: Consumer
@RabbitListener(queues = "stock_queue")
public void updateStock(Order order) {
    System.out.println("Updating inventory for Order ID: " + order.getId());
    // Execute inventory deduction logic
}

Deep Dive
The “eventual consistency” approach solves the distributed transaction challenge. While there may be temporary inconsistencies, the final state will always be correct.

5. Broadcast Notifications: A Single Message Notifying Multiple Services

Scenario
For example, when the price of a product changes, services like inventory, search, and recommendations all need to be updated.
If every service is notified individually, it would require significant effort.

Solution
The broadcast mode (Fanout) of an MQ allows multiple consumers to subscribe to the same message, achieving "one-to-many" message delivery.

Example Code

// Producer: Broadcast message
rabbitTemplate.convertAndSend("price_update_exchange", "", priceUpdate);
System.out.println("Product price update message broadcasted");

// Consumer 1: Inventory Service
@RabbitListener(queues = "stock_queue")
public void updateStockPrice(PriceUpdate priceUpdate) {
    System.out.println("Inventory price updated: " + priceUpdate.getProductId());
}

// Consumer 2: Search Service
@RabbitListener(queues = "search_queue")
public void updateSearchPrice(PriceUpdate priceUpdate) {
    System.out.println("Search price updated: " + priceUpdate.getProductId());
}

Analysis
This model allows multiple services to receive the same message, offering excellent scalability.

6. Log Collection: Centralizing Distributed Logs

Scenario
Logs generated by multiple services need to be stored and analyzed in a centralized way.
Directly writing to a database might create performance bottlenecks.

Solution
Each service writes logs into an MQ, and the log analysis system consumes the messages from MQ for unified processing.

Example Code

// Service Side: Producer
rabbitTemplate.convertAndSend("log_exchange", "log_key", logEntry);
System.out.println("Log sent");

// Log Analysis Service: Consumer
@RabbitListener(queues = "log_queue")
public void processLog(LogEntry log) {
    System.out.println("Log processed: " + log.getMessage());
    // Storage or analysis logic
}

7. Delayed Tasks: Scheduled Operations

Scenario
For instance, if a user places an order and doesn’t pay within 30 minutes, the order needs to be automatically canceled.

Solution
Use the delayed queue feature of MQ to set the message's delayed consumption time.

Example Code

// Producer: Send delayed message
rabbitTemplate.convertAndSend("delay_exchange", "delay_key", order, message -> {
    message.getMessageProperties().setDelay(30 * 60 * 1000); // Delay of 30 minutes
    return message;
});
System.out.println("Order cancellation task scheduled");

// Consumer: Handle delayed messages
@RabbitListener(queues = "delay_queue")
public void cancelOrder(Order order) {
    System.out.println("Order canceled: " + order.getId());
    // Order cancellation logic
}

8. Data Synchronization: Maintaining Consistency Across Systems

Scenario
In a distributed system, multiple services depend on the same data source.
For example, when the order status is updated on an e-commerce platform, the changes need to be synchronized with the cache and recommendation systems.

Solution
Use MQ for data synchronization. After the order service updates the order status, it sends an update message to MQ. The cache and recommendation services consume the message from MQ to synchronize data.

Example Code
Order Service: Producer

Order order = updateOrderStatus(orderId, "PAID"); // Update order status to "PAID"
rabbitTemplate.convertAndSend("order_exchange", "order_status_key", order);
System.out.println("Order status update message sent: " + order.getId());

Cache Service: Consumer

@RabbitListener(queues = "cache_update_queue")
public void updateCache(Order order) {
    System.out.println("Cache updated, Order ID: " + order.getId() + ", Status: " + order.getStatus());
    cacheService.update(order.getId(), order.getStatus());
}

Recommendation Service: Consumer

@RabbitListener(queues = "recommendation_queue")public void updateRecommendation(Order order) {
    System.out.println("Recommendation system updated, Order ID: " + order.getId() + ", Status: " + order.getStatus());
    recommendationService.updateOrderStatus(order);
}

Analysis
Using MQ for data synchronization offers the following benefits:

1. Reduced Database Load: Avoids multiple services querying the database simultaneously.

2. Eventual Consistency: Even if some services are delayed, MQ ensures no message loss, maintaining consistent data across services.

9. Distributed Task Scheduling

Scenario
Certain tasks need to be executed on a schedule, such as clearing expired orders at midnight.
If every service runs its own scheduled tasks, there might be duplicate processing or missed tasks.

Solution
Use MQ to distribute scheduling tasks uniformly. Each service consumes tasks from MQ and executes them based on its own business requirements.

Example Code
Task Scheduling Service: Producer

@Scheduled(cron = "0 0 0 * * ?") // Triggered at midnight daily
public void generateTasks() {
    List<Task> expiredTasks = taskService.getExpiredTasks();
    for (Task task : expiredTasks) {
        rabbitTemplate.convertAndSend("task_exchange", "task_routing_key", task);
        System.out.println("Task sent: " + task.getId());
    }
}

Order Service: Consumer

@RabbitListener(queues = "order_task_queue")
public void processOrderTask(Task task) {
    System.out.println("Processing order task: " + task.getId());
    orderService.cleanExpiredOrder(task);
}

Inventory Service: Consumer

@RabbitListener(queues = "stock_task_queue")
public void processStockTask(Task task) {
    System.out.println("Processing stock task: " + task.getId());
    stockService.releaseStock(task);
}

10. File Processing: Asynchronous Large File Tasks

Scenario
After a user uploads a large file, tasks like format conversion or compression need to be performed.
Synchronously executing these tasks might cause prolonged front-end loading times, impacting user experience.

Solution
Write tasks into MQ after file upload, and asynchronously process files in the background. Notify users or update statuses after completion.

Example Code
Upload Service: Producer

FileTask fileTask = new FileTask();
fileTask.setFileId(fileId);
fileTask.setOperation("COMPRESS");
rabbitTemplate.convertAndSend("file_task_exchange", "file_task_key", fileTask);
System.out.println("File processing task sent, File ID: " + fileId);

File Processing Service: Consumer

@RabbitListener(queues = "file_task_queue")
public void processFileTask(FileTask fileTask) {
    System.out.println("Processing file task: " + fileTask.getFileId() + ", Operation: " + fileTask.getOperation());
    if ("COMPRESS".equals(fileTask.getOperation())) {
        fileService.compressFile(fileTask.getFileId());
    } else if ("CONVERT".equals(fileTask.getOperation())) {
        fileService.convertFileFormat(fileTask.getFileId());
    }
    taskService.updateTaskStatus(fileTask.getFileId(), "COMPLETED");
}

Front-End Polling or Callback Notification

setInterval(() => {
    fetch(`/file/status?fileId=${fileId}`)
        .then(response => response.json())
        .then(status => {
            if (status === "COMPLETED") {
                alert("File processing completed!");
            }
        });
}, 5000);

Summary
Message queues are not just tools for message delivery but also critical for system decoupling, enhancing stability, and scalability.
These 10 classic use cases demonstrate their ability to solve specific business challenges effectively.

If this article is helpful or inspiring to you, please share it.