Pattern Sections
Pattern: Master Data Holder
a.k.a. Master Data Resource, Primary Data Access and Modification
Context
A domain model, an entity-relationship diagram, a glossary, or a similar dictionary of key concepts and their interconnections have been specified; it has been decided to expose some of these data entities in an API by way of Information Holder Resources.
The data specification unveils that the lifetimes and update cycles of these Information Holder Resource endpoints differ significantly (for instance, from seconds, minutes and hours to months, years and decades). Long-living data typically has many incoming relationships, whereas shorter-living data often references long-living data (outgoing relationships). In many application scenarios, data that is referenced in multiple places and lives long has high data quality and data protection needs. The data access profiles of these two types of data differ substantially.1
Problem
How can I design an API that provides access to master data that lives for a long time, does not change frequently, and will be referenced from many clients?2
Forces
The top-level forces that have to be resolved when dealing with any Information Holder Resource are discussed in the Information Holder Resource pattern. Additional concerns specific to master data are:
- Master data quality
- Master data protection
- Data under external control, for instance master data management systems
Pattern forces are explained in depth in the book.
Solution
Mark an Information Holder Resource to be a dedicated Master Data Holder endpoint that bundles master data access and manipulation operations in such a way that the data consistency is preserved and references are managed adequately. Treat delete operations as special forms of updates.
Optionally, offer other life cycle events or state transitions in this Master Data Holder endpoint. Also optionally, expose additional operations to give the Master Data Holder domain-specific responsibilities. For instance, an archive might offer time-oriented retrieval, bulk creations and purge operations.
Sketch
A solution sketch for this pattern from pre-book times is:
Example
Lakeside Mutual, our sample application from the insurance domain, features master data such as customers and contracts that are exposed as Web services and REST resources (Figure 2), thus applying the Master Data Holder pattern.
Are you missing implementation hints? Our papers publications provide them (for selected patterns).
Consequences
The resolution of pattern forces and other consequences are discussed in our book.
Known Uses
Known uses of Master Data Holders are many, appearing in sample applications, public Web APIs, and in real-world enterprise and government information systems:
- The Cargo Tracking system that serves as Domain-Driven Design Sample Application holds master data such as ports and possible routes and exposes this data to the application frontend via its (rather thin) application and presentation layer. See the root entity of the Cargo Aggregate and its contained entities.
- In the e-commerce application that serves as example for Chris Richardson’s microservices patterns the Inventory Service and the Account Service use this pattern.
- Several of the “method families” in the Slack Web API are data-oriented; the “users” family, for instance, deals with master data.
- In the order management SOA described in Zimmermann et al. (2005), customers with their billing plans and the managed telephony network qualify as master data. Specific business services are dedicated to updating it.
- The UID service offered by the Swiss government exposes company information that qualifies as master data. The public part of the API supports company searches and tax number validation.
- A major German car manufacturer offers a REST level 2 user profile management service for all its clients; this service features known uses of several responsibility patterns. For instance, it validates data strictly, converts addresses and phone numbers to country-specific standards, and offers create, read, update (but no search) operations in its Swagger/Open API contracts exposed in an API Description website. One objective of this elaborate design is to comply with the EU General Data Protection Regulation (GDPR).
- The Terravis system Berli, Lübke, and Möckli (2014) offers a Web service to query federated master data concerning parcels, rights, and persons in Swiss land registers. Terravis also offers a service which can be used to query rich master data including name, address and contact information for all banks, notaries, and land registries participating in the platform.
More Information
Related Patterns
The Master Data Holder pattern has two alternative patterns Reference Data Holder (immutable) and Operational Data Holder (shorter lived, less incoming references). Each operation exposed by an endpoint with Master Data Holder semantics requires a request and response message structure that can be expressed with patterns such as Atomic Parameter List.
Domain-Driven Design (DDD) does not distinguish between master data and operational data in its tactic patterns Evans (2003); both operational data and master data may be part of the Published Language and appear in dedicated Bounded Contexts and Aggregates as Entities (see Vernon (2013)).
Other Sources
The notion of master data vs. operational/transaction(al) data comes from literature in the database community (more specifically, information integration) and in business informatics (“Wirtschaftsinformatik” in German, Ferstl and Sinz (2006)). It also plays an important role in Online Analytical Processing (OLAP), Data Warehouses, and Business Intelligence (BI) efforts Kimball and Ross (2002). Such efforts are predecessors of the current big data analytics trend.
“Data on the Outside versus Data on the Inside” Helland (2005) does not distinguish between master data and operational data, but still is a good read when it comes to designing Mater Data Holders.
References
The context of this pattern is the similar to that of its alternative pattern Operational Data Holder, but emphasizes that the lifetimes and relationship structure of these two types of data differs.↩︎
Such data is often called master data and is contrasted to operational data a.k.a. transaction data (in German: “Stammdaten” vs. “Bewegungsdaten”, see Ferstl and Sinz (2006), White et al. (2006)).↩︎