Migration Manual

# Preface

This document is the migration guide for Kingdee Apusic Distributed Message Queue for RabbitMQ (Apusic Distributed Message Queue, referred to as: ADMQ for RabbitMQ), detailing how to migrate from RabbitMQ to ADMQ for RabbitMQ.

# Target Audience

This document is intended for IT information business leaders, R&D managers, software project managers, software architects, and operations engineers.

For more information about ADMQ for RabbitMQ products, please refer to the following ADMQ for RabbitMQ product documentation set:

No. Document Description
1 Kingdee Apusic Distributed Message Queue for RabbitMQ Quick Start Guide Briefly introduces how to quickly get started with ADMQ for RabbitMQ.
2 Kingdee Apusic Distributed Message Queue for RabbitMQ Installation Guide Detailed instructions on installing ADMQ for RabbitMQ on various operating systems, as well as ADMQ for RabbitMQ service start/stop operations.
3 Kingdee Apusic Distributed Message Queue for RabbitMQ Message Engine User Guide Detailed description of the usage, configuration, management, and supporting tools of the ADMQ for RabbitMQ message engine.
4 Kingdee Apusic Distributed Message Queue for RabbitMQ Console User Guide Detailed description of the usage and operation of the ADMQ for RabbitMQ management console.
5 Kingdee Apusic Distributed Message Queue for RabbitMQ Development Guide Detailed instructions on developing ADMQ for RabbitMQ client applications using various programming languages.
6 Kingdee Apusic Distributed Message Queue for RabbitMQ Migration Guide Detailed instructions on migrating from RabbitMQ to ADMQ for RabbitMQ.
7 Kingdee Apusic Distributed Message Queue for RabbitMQ Operations Guide Detailed description of ADMQ for RabbitMQ monitoring, operations, security hardening, and other maintenance tasks.
8 Kingdee Apusic Distributed Message Queue for RabbitMQ Performance Optimization Guide Detailed description of ADMQ for RabbitMQ performance tuning.

# Technical Support

ADMQ for RabbitMQ provides comprehensive technical support services. You can obtain technical support through the following channels:

  • Website: www.apusic.com
  • Phone: 400-855-5800
  • Email: support@apusic.com
  • Kingdee Cloud Community: https://vip.kingdee.com/?productId=73&productLineId=14&lang=zh-CN

When seeking technical support, please provide the following information:

  1. Your name

  2. Company information and contact details

  3. Operating system and version

  4. Product version number

  5. Detailed information such as logs and screenshots of exceptions and errors

# Migration Overview

# Migration Principles

ADMQ for RabbitMQ is fully compatible with the RabbitMQ AMQP 0-9-1 protocol. The migration process follows these principles:

  1. Zero code changes: Client code does not need modification, only the connection address needs to be changed
  2. Configuration compatibility: Supports native RabbitMQ configuration parameters
  3. Data migratable: Supports migration of queue data and configuration information
  4. Smooth switchover: Supports dual-write and grayscale switchover to reduce migration risk

# Migration Scope

Migration Item Supported Description
Application code Yes No modification needed, only connection address change required
Client configuration Yes Supports native RabbitMQ clients
Exchange definition Yes Supports all exchange types
Queue definition Yes Supports Classic/Quorum/Stream queues
Binding relationships Yes Supports all binding relationships
User permissions Yes Supports user and Vhost permission migration
Message data Yes Supports historical message migration
Policy Yes Supports mirroring policies, etc.
Plugins Partial Some plugins may require adaptation

# Pre-Migration Checks

# Version Compatibility Check

Original RabbitMQ Version Compatibility Description
3.8.x Fully compatible Recommended migration version
3.9.x Fully compatible Recommended migration version
3.10.x Fully compatible Recommended migration version
3.11.x Fully compatible Recommended migration version
3.12.x Fully compatible Recommended migration version

# Client Compatibility Check

Client Type Compatible Version Description
Java (amqp-client) 5.x Fully compatible
Python (pika) 1.x Fully compatible
Go (amqp091-go) 1.x Fully compatible
Node.js (amqplib) 0.10.x Fully compatible
.NET (RabbitMQ.Client) 6.x Fully compatible

# Migration Plans

Suitable for business scenarios that cannot tolerate downtime, ensuring data consistency through dual-write.

Steps:

  1. Deploy ADMQ for RabbitMQ cluster

    • Deploy the new cluster according to the installation guide
    • Configure network and firewall settings

  2. Configure dual-write

    • Connect to both the original RabbitMQ and ADMQ for RabbitMQ in the application simultaneously
    • Send messages to both clusters when publishing

  3. Sync historical data

    • Use Shovel or Federation plugins to synchronize queue data
    • Verify data consistency

  4. Switch consumers

    • First switch some consumers to ADMQ for RabbitMQ
    • Observe whether the business operates normally

  5. Full switchover

    • After confirmation, switch all consumers
    • Stop sending messages to the original RabbitMQ

  6. Decommission original cluster

    • Keep it for a period for data queries
    • Decommission after confirming no issues

# Plan 2: Downtime Migration

Suitable for business scenarios that can tolerate brief downtime, with faster migration speed.

Steps:

  1. Stop producers

    • Pause the business system from sending messages to RabbitMQ

  2. Export configuration

    • Export Exchange, Queue, Binding, User and other configurations using bin/admq rabbitmq admin ctl

  3. Migrate configuration to ADMQ for RabbitMQ

    • Create corresponding resources via the console or command line

  4. Migrate message data

    • Use shovel to export original queue messages
    • Import into ADMQ for RabbitMQ

  5. Switch connection address

    • Modify application configuration to point to ADMQ for RabbitMQ

  6. Start producers

    • Resume business system message sending

# Plan 3: Grayscale Migration

Suitable for large-scale systems, gradually migrating different business modules.

Steps:

  1. Divide migration batches by business module
  2. Execute each batch according to the dual-write migration plan
  3. Proceed to the next batch after verification
  4. Decommission the original cluster after all batches are complete

# Configuration Migration

# Export Original RabbitMQ Configuration

# 导出用户和权限
bin/admq rabbitmq admin ctl list_users > users.txt
bin/admq rabbitmq admin ctl list_vhosts > vhosts.txt

# 导出 Exchange
bin/admq rabbitmq admin ctl list_exchanges name type durable auto_delete internal arguments > exchanges.txt

# 导出 Queue
bin/admq rabbitmq admin ctl list_queues name durable auto_delete arguments pid slave_pids > queues.txt

# 导出 Binding
bin/admq rabbitmq admin ctl list_bindings source_name source_kind destination_name destination_kind routing_key arguments > bindings.txt

# 导出策略
bin/admq rabbitmq admin ctl list_policies > policies.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# Create Configuration in ADMQ for RabbitMQ

# Using the Console

  1. Log in to the console
  2. Navigate to the "Resource Management" page
  3. Create in order:
    • Vhost
    • MQ users and permissions
    • Exchange
    • Queue
    • Binding

# Using the Command Line

# 创建 Vhost
bin/admq rabbitmq admin ctl add_vhost /test

# 创建用户
bin/admq rabbitmq admin ctl add_user testuser testpass
bin/admq rabbitmq admin ctl set_user_tags testuser management
bin/admq rabbitmq admin ctl set_permissions -p /test testuser ".*" ".*" ".*"

# 创建 Exchange
bin/admq rabbitmq admin ctl eval 'rabbit_exchange:declare(<<"/test">>, <<"test.exchange">>, <<"direct">>, true, false, false, [], <<"acting_user">>).'

# 创建 Queue
bin/admq rabbitmq admin ctl eval 'rabbit_amqqueue:declare(<<"/test">>, <<"test.queue">>, true, false, false, [], none, <<"acting_user">>).'
1
2
3
4
5
6
7
8
9
10
11
12
13

# Data Migration

# Migrating Data Using Shovel

Shovel is a built-in data migration tool in RabbitMQ.

# Configuring Shovel

# 启用 shovel 插件
rabbitmq-plugins enable rabbitmq_shovel
rabbitmq-plugins enable rabbitmq_shovel_management
1
2
3

# Creating Shovel Configuration

{
  "shovels": [
    {
      "name": "migrate-queue1",
      "source": {
        "uri": "amqp://guest:guest@old-rabbitmq:5672/%2F",
        "queue": "queue1"
      },
      "destination": {
        "uri": "amqp://guest:guest@admq-rabbitmq:5672/%2F",
        "queue": "queue1"
      },
      "ack-mode": "on-confirm"
    }
  ]
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# Migrating Data Using Federation

Federation is suitable for cross-cluster data synchronization.

# 启用 federation 插件
rabbitmq-plugins enable rabbitmq_federation
rabbitmq-plugins enable rabbitmq_federation_management
1
2
3

# Migrating Data Using Client Scripts

For small-scale data, you can write client scripts for migration:

import pika

# 连接原 RabbitMQ
src_conn = pika.BlockingConnection(pika.ConnectionParameters('old-rabbitmq'))
src_ch = src_conn.channel()

# 连接 ADMQ for RabbitMQ
dst_conn = pika.BlockingConnection(pika.ConnectionParameters('admq-rabbitmq'))
dst_ch = dst_conn.channel()

# 获取消息并转发
while True:
    method, properties, body = src_ch.basic_get('queue1', auto_ack=False)
    if method is None:
        break
    
    dst_ch.basic_publish('', 'queue1', body, properties)
    src_ch.basic_ack(method.delivery_tag)

src_conn.close()
dst_conn.close()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

# Client Migration

# Connection Address Change

Only the connection address in the application configuration file needs to be modified:

# 原配置
rabbitmq.host=old-rabbitmq
rabbitmq.port=5672

# 新配置
rabbitmq.host=admq-rabbitmq
rabbitmq.port=5672
1
2
3
4
5
6
7

# SSL/TLS Configuration Changes

If SSL is enabled, certificates need to be updated:

# 原配置
rabbitmq.ssl.enabled=true
rabbitmq.ssl.ca-file=/path/to/old-ca.pem

# 新配置
rabbitmq.ssl.enabled=true
rabbitmq.ssl.ca-file=/path/to/admq-ca.pem
rabbitmq.ssl.cert-file=/path/to/client-cert.pem
rabbitmq.ssl.key-file=/path/to/client-key.pem
1
2
3
4
5
6
7
8
9

# Common Client Configurations

# Java (Spring Boot)

spring:
  rabbitmq:
    host: admq-rabbitmq
    port: 5672
    username: guest
    password: guest
    virtual-host: /
    ssl:
      enabled: false
1
2
3
4
5
6
7
8
9

# Python (Django)

BROKER_URL = 'amqp://guest:guest@admq-rabbitmq:5672//'
1

# Go

conn, err := amqp091.Dial("amqp://guest:guest@admq-rabbitmq:5672/")
1

# Migration Verification

# Functional Verification

Verification Item Verification Method
Connection test Use a client to connect to ADMQ for RabbitMQ and confirm the connection is established successfully
Message sending Send test messages and confirm they reach the Exchange successfully
Message routing Verify messages are correctly routed to the Queue
Message consumption Start a consumer and confirm messages can be received and processed normally
Message acknowledgment Test manual and automatic acknowledgment functions
Persistence Verify messages are retained after service restart
Cluster functionality Verify automatic failover when a node fails

# Performance Verification

Verification Item Verification Method
Throughput Compare message send rates between the original cluster and the new cluster
Latency Measure the delay from message send to consumption
Concurrency Test multi-producer, multi-consumer concurrent scenarios
Stability Run long-duration tests and observe for any anomalies

# Data Consistency Verification

# 对比原集群和新集群的队列消息数
bin/admq rabbitmq admin ctl list_queues -p /test name messages

# 对比消费者数量
bin/admq rabbitmq admin ctl list_queues -p /test name consumers

# 对比连接数
bin/admq rabbitmq admin ctl list_connections -p /test peer_host state
1
2
3
4
5
6
7
8

# Rollback Plan

If issues arise during the migration process, roll back to the original RabbitMQ cluster:

  1. Stop sending messages to ADMQ for RabbitMQ
  2. Switch the connection address back to the original RabbitMQ
  3. Sync new messages from ADMQ for RabbitMQ back to the original cluster (if needed)
  4. Restore the business system

# Rollback Checklist

  • [ ] Original RabbitMQ cluster is still running
  • [ ] Original cluster data has not been deleted
  • [ ] Application configuration can be switched quickly
  • [ ] Business system supports fast rollback

# Migration Best Practices

  1. Thorough testing: Complete the full migration process validation in a test environment
  2. Backup data: Back up the original cluster configuration and data before migration
  3. Monitor the migration process: Monitor various metrics in real-time during migration
  4. Prepare a rollback plan: Ensure quick rollback in case of issues
  5. Choose off-peak hours: Perform migration during business off-peak hours
  6. Gradual switchover: Switch non-core business first, then core business after verification
  7. Retain original cluster: Keep the original cluster for a period after migration for reference
编辑页面 (opens new window)
#迁移手册

← Operations Manual→