Product Introduction
# Preface
This document is the product introduction for ADMQ for Kafka (Apusic Distributed Message Queue, abbreviated as ADMQ for Kafka), helping users quickly understand the features, technical characteristics, and typical application scenarios of ADMQ for Kafka.
# Target Audience
This document is intended for IT informatization business leaders, R&D managers, software project managers, software architects, and operations engineers.
# Related Documents
For more information about ADMQ for Kafka, please refer to the following product manual documentation suite:
| No. | Manual Document | Description |
|---|---|---|
| 1 | ADMQ for Kafka Quick Start Guide | A brief introduction on how to quickly get started with ADMQ for Kafka. |
| 2 | ADMQ for Kafka Installation Guide | Detailed instructions on installing ADMQ for Kafka on various operating systems and starting/stopping ADMQ for Kafka services. |
| 3 | ADMQ for Kafka Message Engine User Guide | Detailed instructions on using, configuring, and managing ADMQ for Kafka message engine features and supporting tools. |
| 4 | ADMQ for Kafka Console User Guide | Detailed instructions on using and operating ADMQ for Kafka console features. |
| 5 | ADMQ for Kafka Development Guide | Detailed instructions on developing ADMQ for Kafka client applications in various programming languages. |
| 6 | ADMQ for Kafka Migration Guide | Detailed instructions on migrating from Kafka to ADMQ for Kafka. |
| 7 | ADMQ for Kafka Operations Guide | Detailed instructions on monitoring, operations, and security hardening for ADMQ for Kafka. |
| 8 | ADMQ for Kafka Performance Tuning Guide | Detailed instructions on performance tuning for ADMQ for Kafka. |
# Technical Support
ADMQ for Kafka provides comprehensive technical support services. You can obtain technical support through the following channels:
- Website: www.apusic.com
- Phone: 400-855-5800
- Email: support@apusic.com
- Kingdee Cloud Community: https://vip.kingdee.com/?productId=73&productLineId=14&lang=zh-CN
When seeking technical support, please provide the following information:
Your name
Company information and contact details
Operating system and version
Product version number
Detailed information such as logs and screenshots of anomalies or errors
# Product Overview
# Product Positioning
ADMQ for Kafka (Apusic Distributed Message Queue, abbreviated as ADMQ for Kafka) is a domestic-replacement cloud-native message middleware built specifically for Kafka users. It fully adheres to the Kafka core architecture design and feature specifications, achieving full compatibility with protocols, clients, ecosystem tools, and business scenarios. It can replace the existing Kafka cluster without modification, meeting the implementation requirements of domestic replacement projects.
# Core Value
It follows Kafka's core characteristics of distribution, high throughput, and low latency, while adapting to domestic replacement software and hardware environments. It provides more stable operational guarantees, more convenient operations support, and more comprehensive security capabilities, making it the preferred solution for Kafka domestic replacement.
# Core Capabilities
| Core Capability | Description |
|---|---|
| Fully Kafka Protocol Compatible | Fully implements Kafka's core protocols including produce/consume protocol, partition assignment protocol, Offset management protocol, and cluster coordination protocol, with behavior identical to native Kafka. |
| Full-Version Client Adaptation | Supports Java, Go, Python, C++ and other official Kafka clients (2.x-3.x versions) as well as third-party open-source clients, accessible without code modification. |
| Seamless Ecosystem Tool Reuse | Compatible with native tools such as Kafka-console-producer/consumer, kafka-topics.sh, kafka-configs.sh, and supports data synchronization tools like Kafka Connect and MirrorMaker. |
| Deep Domestic Environment Adaptation | Adapts to mainstream domestic operating systems (Kylin, UOS, etc.), domestic chips (ARM, RISC-V, etc.), and is compatible with domestic databases and middleware ecosystem. |
| Kafka Core Feature Inheritance | Supports Kafka native core features such as topic partitions, replica mechanism, consumer groups, message compression, and Offset commit (automatic/manual). |
# Product Architecture
# Architecture Overview
ADMQ for Kafka adopts the classic Kafka distributed architecture, fully adhering to the Kafka communication protocol and storage model. The overall product architecture is divided into three major modules: Console, Core Engine, and Distributed Coordination, featuring high reliability, high availability, and scalability.
# Console
- The Console is a visual operation interface for operations personnel and administrators, providing full lifecycle management capabilities:
- Cluster Management: Connect and monitor the status of message cluster nodes.
- User Permissions: Role-Based Access Control (RBAC) to ensure operational security.
- Topic Management: Create, delete, and configure message topics, manage partitions and replicas.
- Subscription Management: Manage consumer groups, subscription relationships, and consumption offsets.
- Message Search: Query historical messages by time, key, content, and other conditions.
# Core Engine
The Core Engine is the heart of message processing, compatible with the Kafka protocol, ensuring reliable message transmission and storage:
Kafka Protocol Access & Security Access Control
Provides a standard Kafka protocol compatibility layer, supporting seamless migration of existing Kafka clients.
Integrates authentication (e.g., SASL), authorization, and transport encryption (SSL/TLS) to ensure access security.
Message Processing Services
Message Produce Service: Receives messages from producers, performs validation and routing.
Message Consume Service: Pushes or pulls messages to consumers, supporting sequential consumption and batch consumption.
Message Persistent Storage: Writes messages to disk, supporting high-throughput writes and efficient retrieval, ensuring no data loss.
Metadata Service: Manages metadata such as topics, partitions, replicas, and consumer groups, ensuring cluster state consistency.
High Availability & Load Management
Multi-Replica Synchronization: Ensures data redundancy through multi-replica mechanism (e.g., ISR), preventing single points of failure.
Failure Recovery & Replica Migration: Automatic failover on node failure, supports replica migration within the cluster for load balancing.
Load Balancing: Evenly distributes produce and consume requests across cluster nodes to improve overall throughput.
Status Monitoring: Real-time monitoring of node, partition, and replica health status, providing basis for failure recovery.
# Distributed Coordination
The Distributed Coordination module (based on Kingdee Apusic Distributed Configuration Center) is responsible for maintaining cluster consistency and state:
Cluster Member Management: Maintains the online/offline status of all Broker nodes in the cluster.
Metadata Persistent Storage: Stores and synchronizes core metadata such as topics, partitions, and replica assignments.
Controller Election: Elects the Controller node in the cluster, responsible for partition management and fault handling.
Partition Leader Election: Automatically elects a new Leader from Followers when the partition Leader node fails.
Cluster State Change Notification: When cluster state changes (e.g., node online/offline, Leader change), sends notifications to relevant components to trigger corresponding actions.
# Product Advantages
Zero-Modification Domestic Replacement: Directly replace the existing Kafka cluster without modifying business code or adjusting client configurations (only need to change the connection address), with migration costs approaching zero.
Full Ecosystem Compatibility: Kafka native tools, monitoring systems (Prometheus+Grafana), data synchronization tools, etc., can all be directly reused without additional adaptation development.
Domestic Environment Adaptation: Fully compatible with the domestic software and hardware ecosystem, meeting government and enterprise domestic replacement project requirements, providing end-to-end domestic replacement solutions.
Authentic Feature Set: Fully adheres to Kafka core feature specifications, supporting all familiar business scenarios (e.g., high-throughput data transmission, streaming data processing), with zero business perception during switchover.
Enhanced Security Compliance: Adds new security features while retaining Kafka's original security capabilities, meeting domestic replacement project security compliance standards.
# Application Scenarios
- Domestic Replacement Scenario: In government and enterprise domestic replacement projects, directly replace the existing Kafka cluster without refactoring business systems, quickly meeting domestic replacement requirements.
- High-Throughput Data Transmission Scenario: Such as log collection, business data synchronization, etc., inheriting Kafka's high-throughput characteristics to support real-time transmission of massive data.
- Streaming Data Processing Scenario: Compatible with streaming computing engines such as Flink and Spark Streaming, seamlessly connecting to existing streaming data processing pipelines.
- Distributed System Decoupling Scenario: Based on Kafka's native publish-subscribe model, enabling asynchronous communication between distributed systems and reducing system coupling.