Why Data Cleansing is crucial before ERP implementation

Getting a new ERP

It’s a common belief that companies have clean data that they are going to use with the implementation of their new ERP.  But before you actually go ahead and complete your transfer to the new ERP solution, you might want to ask yourself the following questions.

Do you currently have a blank or incomplete field for your products or generic items?

Do you have old or obsolete product codes in your systems that are no longer in use?

Did you ever enter a price quotation and maybe forgot to delete it?

Do you have data that is very crucial in Excel but not in your legacy system?

Do you have duplicates of multiple accounts of the same customer or supplier?

If you have YES as an answer to any one of these questions then this means that though you are optimistic about your conversion to the new ERP system, your data needs to be cleansed before you complete your transfer to the new ERP solution.
You cannot rely on your ERP software vendors to do the data cleansing for you before or after the ERP implementation, simply because it is very expensive for them to do so and so it will be wrong to blame them for it. Therefore the processes of data cleansing is left to the actual users of the ERP system within the organization. Since the ERP vendors do not have this high on their priority list, the team leading this implementation should make the data cleansing an integral part of the entire implementation process. After all, bringing the clean, precise, relevant and up-to-date records while minimizing the waste will only help you to minimize downtime at conversion as well as reduce the problems that you might face right after going-live.

What data to include in data migration

It a common misconception of organizations to think that all data is to be brought into their new ERP software, but since that is not the case, what identifies as junk or unnecessary data in their legacy system. To put it into non-technical words, imagine you are moving from your old home to a new one. Would you rather bring all your old stuff like unfinished meals in the refrigerator, furniture, trash, really old newspapers to your new home? The more stuff that you bring along, the harder it will be for you to clean it or find something you need later. Similarly, the more data you decide to bring along, the more difficult it will become for you at the time of conversion affecting your go-live date. Moreover, it might become more challenging to later sort this data out. On the other hand, if you leave something crucial behind, you might lose the records completely.

Keep the Regulation in mind while Implementation and data migration

Different industries have to adhere to different regulatory requirements. Some of these regulations demand data to be unchanged to be compliant for example PCI DSS with the data of card and transactions. Whereas, some of the regulations require data to be stored for at least a specific amount of time for auditing purpose. It is best that these requirements are addressed in the paper as by doing so the cost of maintaining, cleaning and migration of the data will be lowered.

The responsibility of data cleansing

Quite often it is believed that it is the IT team’s sole responsibility to take care of the data cleansing process. Though there are great data cleansing tools currently available in the market, However, they can only assist in identifying the potential areas and records that are highly possible to be cleaned, not clean it themselves. For the process to work completely, the involvement of a person is always required, for example, if there are two different records for a company’s address, this person would be required to point out which one of them is correct. In another case, before moving to the new ERP system, the person’s intervention is required to validate the closing balance of the company. This is the key to obtain true data integrity and accuracy. Thus, the responsibility of cleaning the data cannot be placed solely with the IT department, but every department has to play its role in order to make the perfect data as the personals involved in the verification and validation of data needs to be qualified as well as knowledgeable in their department. However, all these departments have to work together to make it a success.

Data cleansing in Excel and in-system

Though they both have their own set of advantages and disadvantages, both are used in the process. The in-system cleansing is done to correct the data in the database of the legacy system. This is a good method if you intend to store a copy of a final screenshot of the legacy system for the record-keeping purpose. But is some field are not available in the legacy system, then you would have to download it and change it to Excel. It is best used on the day of go-live as it represents duplicate data and in case there are changes in the system, it will not be registered after the download of data.

Generating clean data

Once the problem areas have been identified, there might still be some time left till the actual go-live date, which means data is going to keep adding to the legacy system. At this point, it is extremely crucial to have data standardization as wrong data entry can cause more complication on the conversion. Each and every team member who is using the legacy system needs to be thoroughly trained with the standards and closely monitored to eliminate any problems for the future. In fact, data standardization should be a continuous policy to where there is a set format for entering data to avoid entry of bad data even after the successful implementation of the new ERP software.

In this blog, you will find:

Try data matching today

No credit card required


Want to know more?

Check out DME resources

Merging Data from Multiple Sources – Challenges and Solutions

Oops! We could not locate your form.