Note: this blog is part 3 in a series of 3. If you want, do check out the previous blogs where we discussed the need for systematic and centralized data management, as well as went through master data management in great detail.
We’re here – the series finale. And this brings us to the most important discussion of the series: Which approach to choose between data quality and master data management. Although, now that we have seen both approaches in great detail, it is quite obvious that both these disciplines are highly integrated in one another. In fact, data quality is often believed to be the main driver and by-product of MDM initiatives. But still, when these solutions are packaged and sold as tools, it becomes imperative to know the overlapping areas and capabilities, so that you can choose the right solution for your business.
DQM and MDM: Complements and not opposites
One of the most important concepts to start off with is that data quality and master data management are not opposites of each other; rather, they are complements. MDM solutions contain some extra capabilities in addition to data quality management features.
This definitely makes MDM a more complex and resource-intensive solution to implement – something to consider while choosing between the two approaches.
Comparing capabilities of MDM and DQM
There are about 20 features or capabilities that data management tools usually do or don’t have. We are going to compare a typical MDM and DQM solution against these capabilities.
Keep in mind that these may not be true for all MDM/DQM solutions out there, and exceptions may exist. This comparison highlights the most commonly understood concepts and differences between the two approaches.
|1.||Data usage and sharing control||Yes||No|
|2.||Data access control||Yes||No|
|3.||Data verification workflows||Yes||No|
|4.||Data history and log management||Yes||No|
|5.||Data object modelling||Yes||No|
|7.||Reference data management||Depends||No||Not all MDM solutions facilitate reference data management, and the absence of this feature can undermine your MDM efforts.|
|8.||Data storage||Depends||No||DQM tools usually create copies of your data which you can then overwrite/export to other apps. On the other hand, MDM tools may have data storage for storing master data records, or in some cases, the MDM tool maintains a registry that references to the master data records residing at other apps/sources.|
|9.||Data ingestion or intake||Yes||Yes||This is the ability to connect or integrate with a multitude of sources and applications.|
|10.||Custom data queries and filter||Yes||Yes|
|11.||Data record creation||Depends||No||DQM tools are more focused towards cleaning data that was collected/created through other data apps. On the other hand, MDM tools having their own data storage allow data record creation, while other registry-based MDM tools do not allow data record creation.|
|12.||Data manipulation||Yes||No||In DQM tools, you cannot directly manipulate/update data – allowing you to maintain data integrity; while in MDMs, you can update master data records that were created by consolidating data from other sources.|
|14.||Data profiling||Yes||Yes||Read more about data profiling in a DQM solution.|
|15.||Data cleansing and standardization||Yes||Yes||Read more about data cleansing and standardization in a DQM solution.|
|16.||Data matching||Yes||Yes||Read more about data matching in a DQM solution.|
|17.||Data merge and survivorship||Yes||Yes||Read more about data merge and survivorship in a DQM solution.|
|18.||Data synchronization||Depends||No||For MDM, data synchronization depends on the architectural style of MDM.|
|19.||Runtime data analysis||Yes||Depends||This relates to flagging incoming data entries/manipulations as invalid or possible duplicates. For DQM tools, this may depend on whether the vendor offers data quality API, which can be integrated with your custom solution to analyze data in runtime.|
How are these capabilities packaged in software tools?
Now that we have understood the exact scope of these disciplines and tools, let’s look at another important aspect before making the final decision. And that is: understanding how vendors commonly package these capabilities in their product and service offerings.
1. Stand-alone data quality tools
These tools have more or less the same features as mentioned above for DQM. They do not connect to other data sources in real-time, and so these tools are mostly used for batch processing (including data profiling, cleaning, standardizing, matching, and merging), and then loading the consolidated records back to the destination source.
Some additional benefits include:
- Quickest and safest way of consolidating data records.
- Easiest to fine-tune matching algorithms and merging rules depending on the current nature of data (contrary to MDMs where the same configurations apply to all records generated over time).
- Some of these tools come with specialized word dictionaries that allow finding exact words (for example, first, middle, and last names), and replacing misspelled or missing fields.
- Some tools also support scheduling DQM tasks, and generating consolidated records at specified times.
- Especially helpful for consolidating email marketing lists, contacts, and customer records.
2. Integrating data quality functions in real-time
Some vendors expose necessary data quality functions through APIs or SDKs. This helps you to integrate all DQM features in your existing applications in real-time or runtime.
This may require some additional efforts, but some benefits include:
- Useful while implementing custom flows (especially for data governance) that are important to your business requirements.
- Can potentially act as a data quality firewall for your data warehouse, where incoming data is tested for quality before entering.
3. Operational MDM solution
Vendors package MDM solutions in different ways, depending on their purpose or use in an organization. One of them is an operational MDM. This MDM type is normally used in routine data operations and is heavily focused on providing a consolidated view of core data assets to everyone who handles data in an organization.
Operational MDMs usually have coexistence or centralized architectural style – which suggests that they have their own data storage.
4. Analytical MDM solution
This MDM type is normally used for analytics or business intelligence purposes in an organization. Since they are not used operationally, they don’t really require their own data storage. This is why architectural styles like registry or consolidated are more common with analytical MDM solutions, because they either just point towards master data records, or send them to BI apps that need it for analysis.
5. Custom in-house solutions
Despite various data quality and master data management solutions present in the market, many businesses invest in developing an in-house solution for their custom data needs. Although this may sound very promising, businesses often end up wasting a great number of resources – time and money – in this process. The development of such a solution may be easier to implement, but it is almost impossible to sustain over time.
To know more on this topic, you can read our whitepaper: Why in-house data quality projects fail.
Which one do you need: MDM or DQM?
This brings us to our last and concluding topic: how to choose between an MDM or DQM solution. And the answer is: it depends. Although, until now it may be clear to you which solution suits your requirements better – especially after going through the capabilities of each – but you may need to keep following aspects in mind as well while making the decision.
1. Primary business objectives
This is probably the most important thing to consider as it suggests that this decision is highly dependent on your business situation. For you to make the right decision, you need to know the primary objectives and goals of your business, and the role data plays in achieving them. In addition to that, you may want to note down how data operations are handled in your business and where exactly a data management solution is needed.
2. Framework for measuring data quality
Another helpful step is identifying your core data quality key performance indicators (KPIs). Data quality can mean something different for different organizations. Once you realize and identify your own definition of “data quality”, it will be easier to know which solution will best facilitate it and help you to introduce, maintain, and sustain data quality in your core data assets.
3. Budget and time your business is willing to invest
From our previous blog in the series, you might have figured out that implementing an MDM is quite a complex process, as it requires in-depth pre-planning and analysis, as well as involvement of key stakeholders.
Do keep in mind the budget and time your business can afford and is willing to invest in this process – as compared to return on investment. For example, you may not need a complete and independent entity for master data management, and a simple stand-alone data quality tool can fulfill your data management needs.
4. Who handles/uses/requires master data at your company?
This can possibly act as a decision-making point. Many people at your company might generate or handle master data assets, but not everyone uses it in their day-to-day operations.
Consider this scenario as an example: customer data records are generated or updated by sales reps, accountants, or other operational staff, but only your team of data analysts utilizes it to generate quarterly reports. In such a case, maintaining a full-fledged MDM over time may be too much to handle as compared to its usage requirements. You can simply provide your data analysts with their own personalized data quality tool which they can use for quick and efficient data quality assessment and management, and then using the data as needed.
But on the other hand, if multiple departments operationally require master data records almost every working day, a complete MDM solution might be the answer.
Here, we come to an end. We started off by looking at the rising need for systematic and centralized data management solutions, then went on to conceptually and technologically study MDM solutions, and finally ended with a comprehensive comparison between MDM and DQM approaches. This journey in itself has given you enough information to make the right decision for your business.
But if you want, our solution experts can definitely help you answer any question that you may still have. Don’t hesitate to download our free trial or book a demo today for a personal consultation session.