Artificial Intelligence in the Data Jungle: How Intelligent Technologies Revolutionize Master Data Management
January 18, 2024 | Reading time: Approx. 6 minutes
Business Development and Innovation Manager at FIS
Data is the new gold of companies and the basis for functioning business processes and decisions. However, correct data maintenance may be very time-consuming. In this blog article, you will learn how work steps can be accelerated and automated using AI and what application options are already available today.
Master data management – today
In the past, master data management was very often considered to be a purely technical issue, which – as many thought – would ideally be embedded in the IT department.
However, this falsified impression has been relativized over time and the user departments became more and more the center of focus with regard to this issue.
For they know best the content-related challenges relative to correct data and have detailed knowledge of the respective business processes of the company.
By now, one can say they are the driving force behind master data management projects. The IT department only provides supporting activities, such as software installation or the necessary infrastructure administration.
The run on good master data has begun. It is the basis for efficient and functioning business processes. The efforts are great to achieve good master data quality. Nowadays, companies try to master the data jungle using sophisticated business rules. However, this can often only be ensured through great domain knowledge. Complicated maintenance activities with innumerable rule variants are the reason for losing the overview and increased error potentials.
Master data management – in the future
Intelligent and self-learning mappings
Master data is often enriched in a system by using external data from other systems, which is transferred via files or Web services. Data of business partners from various systems, for instance, need to be referred to not only for transition projects but also in daily life. This requires the transfer of this data into the right structures and tables.
A predefined mapping (of incoming data) into the corresponding target tables or table fields is unavoidable. Recurring files usually do not have the same structure and often have different names. Therefore, it is difficult to integrate an automated process for inbound master data.
Here, the solution is an intelligent mapping which automatically recognizes file contents arranged in tabular form and maps them with the correct table fields. Exact mapping rules are no longer necessary. The mapping is structured dynamically and independently via machine learning.
Harmonization of master data through AI
The harmonization of master data is a Herculean task, in particular if data comes from heterogeneous systems and duplicates (also with domain knowledge) are difficult to recognize. Questions (such as “What are my golden records?”, “What would be the ideal procedure?” or “What procedures and algorithms are effective?”) are difficult to answer until comprehensive and intensive analysis and research activities for field tests or prototypes have been carried out.
What are the means of choice for this purpose? Is it intelligent search helps, sophisticated machine learning algorithms of the second or third generation or mathematical character string functions (e.g. Hamming or Levenshtein distance that are used for fuzzy search)?
Some frameworks in the data science environment already offer good options to use different algorithms and procedures for the harmonization of data. There are many similar procedures having the same objectives but being based on different algorithms and procedures.
Usually, one procedure is not sufficient to achieve an acceptable end result. Probably, a combination of different procedures that are causally linked with one another is more effective.
A structured preparation of data (i.e. which fields from which tables are relevant for data harmonization and how important is this data) is a significant step towards a promising result. Even for experts it is not always easy to correctly assess their priority as many fields are sometimes more or less important under specific conditions only. An intelligent process that could correctly assess the priority of the fields (according to context and occurrence) would undoubtedly be the ideal solution.
Discrepancies often occur in the course of the tests. Specific procedures may have problems with saving data or missing associations relative to duplicates due to large data records. The combination of suitable procedures results in duplicates with a range of probabilities. A high value probably means a duplicate whereas a small value probably represents a unique value.
In individual cases, only the user can determine the threshold value for defining whether something is still or no longer is a duplicate. The appropriate solution is an important decision-making aid for the user (for master data maintenance with fewer errors or as possible approach for a future intelligent master data management system).
Validation of master data using AI
It is often difficult to describe materials and their properties as they exist in various characteristic values. The valid dimensioning of a material alone is complicated to map via rule sets in Customizing.
If, however, an AI system was trained with all possible material properties and use it for validation checks of initial creations or master data changes, only valid combinations would be permitted. This means that sophisticated rule sets would no longer be required for a valid master data management.
Intelligent support by assistance systems
A good user experience is part of a future-oriented master data management application. This can be achieved by a great number of automated processes as well as by a convenient interaction with the application. What kind of interaction is more natural than language?
A chatbot assistant, for instance, could take over routine activities (such as the preassignment of entire master data records) due to its skills and the user is only involved in the process with a controlling function.
Conclusion
Companies depend on qualitative ERP master data to remain competitive. It is true that SAP MDG and integrated SAP optimizations can minimize problems; however, even these have their limits in heterogeneous systems.
Master data responsibility, process definition and optimization, user-friendly user interfaces and AI-supported processes and technologies: this way or other, one could imagine the “brave new world” of master data management. But still how far away are we from this?
Questions about this topic? Our team will be happy to assist you personally.
Read, take a look, inform yourself
In our download section, you will find a large number of SAP contents worth knowing – expert talks, white papers and flyers. To find right away what you are looking for, please use the filter function for topics and content type.
