TDMQ for Apache Pulsar
2025-12-12 16:58TDMQ for Apache Pulsar (TDMQ Pulsar Edition) is a self-developed messaging middleware based on Apache Pulsar, offering excellent cloud-native and serverless characteristics. It is compatible with the various components and concepts of Pulsar and leverages the underlying advantages of Compute-Storage Separation and flexible scaling. The product is built on a compute-storage separation architecture, which not only enables flexible scaling but also supports millions of message topics without significant performance degradation as the number of topics increases, making it perfectly suited for large-scale business scenarios. At the functional level, the product provides a rich variety of message types. Among them, Scheduled Message can meet business needs for timed triggering, while Distributed Transaction Message ensures data consistency across systems. Coupled with strong data consistency (based on the BookKeeper consistency protocol), it guarantees the security and reliability of message data. In terms of business support, beyond classic scenarios like asynchronous decoupling and peak shaving, its Data Synchronization capability is particularly outstanding. It enables seamless message consumption and data synchronization between multiple data centers, facilitating global business expansion. The compatibility with Apache Pulsar reduces migration and usage costs, while compute-storage separation lays the foundation for high performance. The deep integration of scheduled messages, distributed transaction messages, and data synchronization makes this product a reliable choice for enterprises dealing with complex business requirements.
Frequently Asked Questions
Q: Built on Apache Pulsar, what advantages does the compute-storage separation architecture of Tencent Cloud TDMQ Pulsar Edition offer, and how does it support Data Synchronization scenarios?
A: As a cloud-native product within the Apache Pulsar ecosystem, Tencent Cloud TDMQ Pulsar Edition's compute-storage separation architecture offers significant advantages: On one hand, it allows flexible scaling of clusters based on business needs, with compute and storage resources adjusted independently to avoid resource wastage. On the other hand, this architecture supports stable operation with millions of topics, ensuring cluster performance does not degrade sharply even as the number of topics increases, meeting the message distribution demands of large-scale businesses. This architecture is crucial for Data Synchronization scenarios: Data synchronization often involves cross-regional transmission of massive messages across multiple data centers. Compute-storage separation enables independent scaling of storage resources, easily accommodating the massive message storage demands generated during data synchronization. Additionally, the elastic scaling capability of compute nodes can match sudden traffic spikes in data synchronization, ensuring efficient and stable data synchronization between multiple data centers. Combined with Apache Pulsar's native cross-regional collaboration capabilities, compute-storage separation further reduces latency and enhances the reliability of data synchronization.
Q: What characteristics do Tencent Cloud TDMQ Pulsar Edition's Scheduled Message and Distributed Transaction Message features offer, and how do they work together within the Apache Pulsar ecosystem to meet complex business needs?
A: Tencent Cloud TDMQ Pulsar Edition's Scheduled Message allows businesses to trigger message consumption at preset times, accurately adapting to scenarios like scheduled tasks and periodic business processing. Leveraging Apache Pulsar's reliable delivery mechanism, it ensures scheduled messages are neither lost nor delayed. Distributed Transaction Message, on the other hand, guarantees transactional consistency across systems and business processes, preventing data inconsistencies caused by message delivery failures, making it ideal for core business scenarios such as order payments and data synchronization. Within the Apache Pulsar ecosystem, these two features demonstrate significant synergy: For example, in an e-commerce order scenario, Distributed Transaction Message can ensure atomicity between order creation and inventory deduction after a user places an order, preventing issues like "over-selling" or "missing deductions." Simultaneously, Scheduled Message can set rules to automatically cancel orders if payment is not completed within a specified timeframe, triggering timed tasks after the transaction is complete. The unified messaging architecture provided by Apache Pulsar allows Scheduled Message and Distributed Transaction Message to share high-availability, high-throughput underlying capabilities. Their collaboration not only ensures consistency in business logic but also meets flexible scheduling requirements, fully leveraging the comprehensive advantages of the Apache Pulsar ecosystem.
Q: In multi-data center Data Synchronization scenarios, how do Tencent Cloud TDMQ Pulsar Edition's compute-storage separation architecture and Distributed Transaction Message work together, and what auxiliary role does Scheduled Message play in this context?
A: In multi-data center Data Synchronization scenarios, the collaboration between compute-storage separation architecture and Distributed Transaction Message serves as a core safeguard: Compute-storage separation allows compute nodes in each data center to independently handle data synchronization requests, while storage nodes centrally manage message data, avoiding uneven storage pressure caused by multi-center data transmission. Distributed Transaction Message ensures transactional consistency in data synchronization across multiple data centers. For example, when business data changes in one data center, it is synchronized to other centers via distributed transaction messages, ensuring data consistency across all centers and preventing data discrepancies. Scheduled Message plays a key auxiliary role in this scenario: It can be used to set up periodic data verification tasks, regularly checking synchronized data across data centers to promptly identify and correct synchronization anomalies. Additionally, for non-real-time synchronization requirements, scheduled messages can be used to batch-trigger data synchronization, reducing network and computational pressure during peak periods. Combined with Apache Pulsar's compatibility features, these three elements make multi-data center data synchronization both efficient and reliable, fully meeting the deployment needs of enterprises with globalized operations.