How do I write a queue for Adobe Commerce (Magento)?

To write a queue for Adobe Commerce or Magento Open Source, you need five things: four XML files in your module's etc directory (communication.xml, queue_consumer.xml, queue_topology.xml, queue_publisher.xml), a PHP consumer class with a process() method, a publish() call to put messages on the queue, and a running consumer process. The message queue framework is identical on Adobe Commerce, Magento Open Source, and Mage-OS, and this guide applies to versions 2.4.4 through 2.4.9.

We build queue-based integrations for B2B stores constantly: ERP syncs, inventory feeds, order exports. The pattern below is the one we ship, including the production parts most tutorials skip.

Should I use a queue or a cron job?

The rule of thumb: queues are for event-driven work that needs to be asynchronous, scaled, or prioritized. They decouple task execution from user actions, which is exactly what you want when a customer clicks "place order" and your ERP is having a slow day. Cron is fine for simple, scheduled, low-frequency tasks. For a deeper look at why inline integration calls hurt, see our post on ERP integration ROI.

What XML files do I need to create a message queue?

In your module's etc directory, add four files:

• communication.xml: defines the topic and handler
• queue_consumer.xml: links the queue to its consumer and handler method
• queue_topology.xml: declares the queue, exchange, and routing rules
• queue_publisher.xml: specifies the publisher and the exchange

communication.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Communication/etc/communication.xsd">
    <topic name="vendor.module.topic" request="string">
        <handler name="processHandler" type="Vendor\Module\Model\Queue\Consumer" method="process"/>
    </topic>
</config>

The topic name is the unique identifier for your queue topic. The handler is the class and method that will process each message.

queue_consumer.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/consumer.xsd">
    <consumer name="vendor.module.consumer"
              queue="vendor.module.queue"
              connection="db"
              maxMessages="1000"
              consumerInstance="Magento\Framework\MessageQueue\Consumer"
              handler="Vendor\Module\Model\Queue\Consumer::process"/>
</config>

queue is the queue to listen to. connection is db for MySQL or amqp for RabbitMQ. maxMessages caps how many messages one process handles before it exits, which matters for memory (more on that below).

queue_topology.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/topology.xsd">
    <exchange name="magento-db" type="topic" connection="db">
        <binding id="processHandlerBinding"
                 topic="vendor.module.topic"
                 destinationType="queue"
                 destination="vendor.module.queue"/>
    </exchange>
</config>

This binds the topic to the queue through the exchange. On RabbitMQ, use your amqp exchange name and connection="amqp".

queue_publisher.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/publisher.xsd">
    <publisher topic="vendor.module.topic">
        <connection name="db" exchange="magento-db"/>
    </publisher>
</config>

This links the topic to the exchange for publishing.

How do I write the consumer class?

Create a PHP class matching the handler you declared:

<?php
namespace Vendor\Module\Model\Queue;

use Psr\Log\LoggerInterface;

class Consumer
{
    public function __construct(
        private readonly LoggerInterface $logger
    ) {
    }

    public function process(string $message): void
    {
        // Handle the message (e.g., process order, sync inventory, update status)
        $this->logger->info('Processing queue message', ['message' => $message]);
    }
}

The process() method receives the message payload. Two habits that pay off in production:

• Keep the handler idempotent. Messages can be delivered again after a failure, so processing the same message twice must be safe.
• Throw an exception when processing genuinely fails. On the database driver the framework marks the message for retry instead of silently losing it. Catching and swallowing every error means failures vanish.

How do I publish messages to the queue?

Inject Magento\Framework\MessageQueue\PublisherInterface into your service or controller:

<?php
namespace Vendor\Module\Service;

use Magento\Framework\MessageQueue\PublisherInterface;

class OrderExporter
{
    public function __construct(
        private readonly PublisherInterface $publisher
    ) {
    }

    public function queueExport(string $data): void
    {
        $this->publisher->publish('vendor.module.topic', $data);
    }
}

Replace vendor.module.topic with your topic name and $data with your payload. Publishing is fast; the whole point is that the slow work happens later, in the consumer.

How do I start the queue consumer?

Run it from the Magento CLI:

bin/magento queue:consumers:start vendor.module.consumer

List every consumer the system knows about, including core ones:

bin/magento queue:consumers:list

Two useful flags: --max-messages=1000 makes the process exit after that many messages, and --single-thread uses a lock to prevent two copies of the same consumer running at once.

That works on your machine. Production needs the consumer running all the time, which brings us to the part most guides skip.

How do I run queue consumers in production?

By default, Adobe Commerce and Magento Open Source run consumers through cron. The consumers_runner job (cron group consumers) starts your consumers every minute. Each process consumes up to 10,000 messages by default and then terminates, and the next cron run starts a fresh one. That exit-and-restart cycle is deliberate: it keeps long-running PHP processes from accumulating memory.

You control this in app/etc/env.php:

'cron_consumers_runner' => [
    'cron_run' => true,
    'max_messages' => 10000,
    'consumers' => [
        'vendor.module.consumer'
    ]
],

An empty consumers array means "run all of them". Listing consumers explicitly is better on busy stores: you choose what runs on which box.

For high-volume queues, a process manager like Supervisor beats cron because the consumer restarts the moment it exits, not up to a minute later:

[program:magento-consumer-vendor-module]
command=/usr/bin/php /var/www/html/bin/magento queue:consumers:start vendor.module.consumer --max-messages=1000 --single-thread
directory=/var/www/html
autostart=true
autorestart=true
user=www-data
numprocs=1
stopasgroup=true
killasgroup=true
stdout_logfile=/var/log/supervisor/magento-consumer.log

If Supervisor manages a consumer, set cron_run to false for it in env.php so cron and Supervisor do not fight over the same queue.

Two production rules worth underlining:

• Never set max_messages to 0 in production. Zero means the process never exits, and PHP memory grows until something falls over. Finite batches plus automatic restart is the healthy pattern.
• Restart consumers on every deploy. A consumer is a long-running PHP process holding the old code in memory. Until it restarts, it processes new messages with pre-deploy code, which produces the strangest bugs you will ever chase.

Should I use the database or RabbitMQ?

The default db connection stores messages in MySQL. It needs no extra infrastructure and is fine for small to medium workloads.

For high-throughput or distributed systems, use RabbitMQ: set connection="amqp" in your XML and configure the broker in env.php:

'queue' => [
    'amqp' => [
        'host' => 'rabbitmq.internal',
        'port' => '5672',
        'user' => 'magento',
        'password' => '********',
        'virtualhost' => '/'
    ],
    'consumers_wait_for_messages' => 1
],

You can also set this at install time with bin/magento setup:config:set --amqp-host=... --amqp-port=5672 --amqp-user=... --amqp-password=....

Current releases keep this stack modern: Adobe Commerce and Magento Open Source 2.4.8 and 2.4.9 support RabbitMQ 4.x and run on PHP 8.3 and 8.4. If you are planning a version move anyway, we covered what 2.4.8 means for your business separately.

Why is my queue consumer not processing messages?

The classic failure modes, in the order we check them on client stores:

1. The consumer is not running at all. Check with ps aux | grep queue:consumers on the server. If nothing is there, cron is not starting it: verify Magento cron itself runs, and that cron_run is not set to false for that consumer in env.php.
2. The consumer runs but sits idle while messages wait. Look at consumers_wait_for_messages in env.php. Set to 1, a consumer holds its connection and waits for new messages up to its limits; set to 0, it processes what exists and exits so the next cron tick picks up fresh work. On cron-managed database queues, 0 is often the more predictable behavior.
3. Messages stuck "in progress". A consumer that was killed mid-message can leave rows in the queue_message_status table marked in progress, and they block reprocessing. Inspect the queue tables before touching them, back up, and clear the stuck rows; then restart consumers. On the amqp driver, unacknowledged messages return to the queue on their own when the connection drops.
4. A poison message. One malformed payload that makes the handler throw on every retry can wedge a queue. Log the payload on failure so you can find and remove it, and validate messages before publishing.
5. Everything worked until the last deploy. Stale code in a long-running consumer process. Restart the consumers; add that restart to your deploy script so it never happens again.
6. Admin mass actions or async API calls hang. Those features run on the same framework through consumers like async.operations.all. If bulk operations sit at "pending" forever, apply this same checklist; the platform's own queues fail the same way custom ones do.

Summary table

Which versions does this apply to?

Everything here works on Magento Open Source, Adobe Commerce, and Mage-OS, versions 2.4.4 through 2.4.9. The message queue framework has been stable across that whole range; what changed is the surrounding stack (RabbitMQ 4.x and PHP 8.4 support arriving with 2.4.8). One planning note: regular support for 2.4.5 and 2.4.6 ended on August 11, 2026, so if your queues run on either, fold the upgrade into the same work.

Queues are the backbone of every serious integration we build, and they are usually the difference between a storefront that survives a slow ERP and one that inherits its outages. If you are wiring Adobe Commerce to the rest of your stack, our integration guide covers the API side, and our integrations team does this daily. Stuck on a queue that will not drain right now? Tell us what you are seeing at x2y.dev/contact; this is a fun one for us.