Part One: Overview of Order System

1.1 Business Scope

Service Business Lines: Express, fast logistics, small and medium-sized parcels, large items, cold chain, international, B2B contract logistics, CLPS, Jingxi, three imports and three exports (procurement import, return import, transfer import, sales export, return supply export, transfer export) and more

1.2 Value of Order Center

1. Decoupling (Improve System Stability

Original System:When trading and production are coupled together, new business requirements involve multiple systems in both upstream and downstream. This includes ECLP, external orders, waybills, terminal systems, etc. The logic of multiple business lines is coupled together, and the iterative update of a single business line's requirements involves the correlation transformation of other business lines in the original system.

New System:Decoupling of Trading and Production Operations: Requirements related to trading are resolved within the order domain; production-side requirements are resolved within the production domain, reducing mutual influence between upstream and downstream.

Business Line Decoupling: Different business lines have different business processes. The iterative update of a single business line's requirements is only done in the specific process, without affecting other business lines. This improves the stability of the entire process and business.

2. Improve the Speed of New Business Access

The order center provides reusable standard capabilities to the front-end, improving the speed of new business introduction.

The order center decomposes and abstracts the large applications in the original system into multiple small application combinations, and supports the on-demand orchestration of business processes in different scenarios. New businesses can quickly access the order center through the reuse of the public standard capabilities of the platform, avoiding the redundant construction of similar functions.

3. Provide Globalized Unified Data Model

Original System:Orders belong to multiple systems, such as external orders, ECLP, and large-item systems, with multiple databases, inconsistent business semantics, and not conducive to data construction.

New System:The order center defines a unified standard data model for orders, allowing data from different businesses to accumulate in the same system,Reducing the redundancy of related functions in the order domain, avoiding resource waste, and breaking down departmental barriers.Enabling the centralized management and optimization of data and processes, for group business analysis,Predicting the future innovation space of JD.comProvide standard data for the order domain.

Part Two: Introduction to Architecture

2.1 Overall Architecture Design

Through the upgrade project of the technical platform architecture, the transaction system has been restructured into a new four-layer architecture of access, transaction, performance, and execution. The transaction order is responsible for the intake of document flow between logistics and customers, and also bears the responsibility of distributing to the downstream OFC (Order Fulfillment Layer).

2.2 Design of Real-time Data Layer Architecture

2.2.1 System Interaction Diagram

The system interaction is as follows:

The standard interface of the order center has been unified at the upper level, and we have also done a unified intake at the data level.

WillBusiness Architecture and Data DecouplingDistributed database, caching, consistency, and other high availability, high-performance designs are stripped from the business architecture category, focusing the business architecture on the business itself.

Persistence System：Used to support data persistence such as order acceptance, order modification, order cancellation, order deletion, etc.

Search System：Provide order detail query, order list query, order status log query, judgment whether Bai Chu order, and other services.

Relay System：Data hub, through consuming message queues, write order data into Elasticsearch, HBase, MySQL.

Data Reconciliation System：Used to compare whether the data of multiple sets of storage middleware is consistent to ensure the final consistency of data.

Data Synchronization System：Synchronize the query conditions and list display fields required for querying order lists from the old system to the order center, used to solve the problem of pagination difficulty due to the existence of order data in both the old and new systems during the slicing process.

2.2.2 Technical Architecture Diagram

•【Read-write separation architecture】Adopts the read-write separation architecture pattern (CQRS), separating order read and write traffic to improve query performance and scalability, and achieve read-write decoupling at the same time.

•【Caching】Use distributed caching Redis to cache hot order data and related information to improve concurrency and response speedReduce access to HBaseAt the same time, improve the system's disaster recovery capability through three sets of high-performance caches (master, standby, temporary).

•【Message Queue】Use message queue JMQ to implement asynchronous processing of orders to improve system throughput, and at the same time, reduce the pressure on direct requests to ES, HBase, and databases by traffic shaping. Isolate different business scenarios (such as order placement, feedback) with different Topicscan better manage and maintain; isolate different business with different Topics to achieve parallel processing and horizontal expansion of messages, improving system throughput and performance.

•【Complex Queries】Use Elasticsearch, a search engine, to solve complex order queries, first obtain the order number through Elasticsearch, and then query the distributed cache Redis + columnar database HBase according to the order number.

•【Low-cost persistent storage】Adopts HBase columnar database to support massive data scale storage and strong scalability.

•【Data Consistency】Achieved through strong transactions, eventual consistency, idempotence, compensation, distributed locks, version numbers, etc.

•【Multi-tenant architecture】The system adopts a multi-tenant data model to separate the storage of tenant data to ensure data isolation and security. According to the needs of different tenants, dynamically expand the system's capacity and resources, which can support horizontal expansion of the system. By sharing infrastructure and resources, the multi-tenant architecture achieves higher resource utilization and cost reduction.

2.3 Design Advantages

2.3.1 High Availability

•Application servers, MySQL, Redis, HBase, JMQ, etc., are all deployed across machine rooms;ES single-machine room deployment, building an ES master-slave dual-machine room cluster

•Isolation, Throttling, Breaker, Peak Shaving, Monitoring

2.3.2 High Performance

•High PerformanceCachingStorage

•Asynchronous

2.3.3 Massive Data Processing

•Sharding and Partitioning

•Cold and Hot Separation

•Columnar Storage(HBase)

2.3.4 Data Security

Sensitive information is encrypted and stored, and Log, Redis, ES, MySQL, HBase, etc. all adopt encrypted storage, 'who stores who encrypts, who uses who decrypts'.

Third, Order Data Model

3.1 PDM Model

In the design of the order model, based on the principles of unified business attributes, abstracting general models, and summarizing common entities, the order model is mainly divided into several categories, such as the main information of the order, the product information of the order, the logistics service information of the order, the marketing information of the order, the financial information of the order, the customer channel information of the order, the receipt and delivery information of the order, the operation information of the order, and the extended information of the order.

3.2 Model Extensibility

3.2.1 Design of Standard Model Extensibility

There are dozens or even hundreds of identification fields in the order. If a new field is added each time, the order business attributes and data models will expand significantly, erode the model, and at the same time, the development efficiency is low. Therefore, the KV form is adopted to handle and store the identification. The identification is divided into various business domains, such as order identification, product identification, marketing identification, etc.

3.2.2 Extensibility of Personalized Business Model

For personalized business, a configurable database field management solution has been provided. Through some ready-to-use settings, when the order is submitted, modified, or queried, different data models and data expansion codes can be found according to business identity + business type + business field, that is, where to store the data in which table and field. N expansion attributes are reserved in each table, and the same expansion attribute has different meanings for different business identities + business types, thereby realizing extended storage.

Four, the Future and Challenges

4.1 Order Personalized Query

There is an increasing number of personalized query requirements, such as fuzzy queries, real-time aggregation based on query conditions, etc. If all ES indexes are placed in the same cluster, it will affect the overall stability of the cluster, but after splitting, the business data cannot be queried and displayed together with other business data.

4.2 Unitized Architecture

The current TP99 of order persistence is 47ms, and the TP99 under non-cross-data center conditions is 20ms. From the data, it can be seen that cross-data center has a significant impact on performance.

Unitization canLet the relevant requests of the same user complete all business 'closed loop' within a single data center, and no longer appear 'cross-data center' accessThe unitized deployment method allows each data center to be deployed in any region and expand new data centers at any time.Through unitization, continuously strengthen the stability of the base of the order platform.

4.3 Hardware Cost Control

The average daily order volume of orders is continuously rising, and the amount of data is increasing, which is followed by the increase in hardware costs. How to control the increase in hardware costs is a challenge in the present and future. We plan to do this through data archiving, cold and hot data stratification, and other ways toReduce data storage costs.