Missing, duplicated or inaccurate customer and business information – otherwise known as ‘dirty data’ – can be one of the most frustrating challenges for accountants and advisors, particularly when providing assistance on end of year figures.
Recent data from Experian found that 60 per cent of dirty data can be attributed to human error. A further 35 per cent may come from poor interdepartmental communication resulting in inaccurate records. However, there is one aspect that is often overlooked: our current over reliance on technology.
National Audits Group Managing Director, Steven Watson, sat down with My Accounts Managing Partner, Noel Tiufino, to discuss the growing issue of dirty data, how prioritising technology over basic practices is accelerating this problem and how accountants and advisors can strive to help businesses get on top of this once and for all.
Dirty data, done dirt cheap
Every accountant and advisor knows that it’s essential that their clients keep a clean set of records. Every aspect of the business must be accurate, accessible and readable, particularly when it comes to database information.
Unfortunately, our ability to do our job most effectively is made difficult when clients approach us with dirty data, particularly for data-driven companies. The truth is that as an often-external support team, we have little control over the quality of source data client’s are providing us with. But we do play a role in preventing more dirty data from accruing.
Some of the most common ways dirty data occurs are through simple mistakes, such as spelling errors in names and addresses and not checking if a customer is already in a database before re-adding them. And it’s not just isolated to our industry.
Even in a perfect scenario dirty data can still pollute a business. So, outside of human error how does this keep happening?
“It has a lot to do with time and no human intervention,” explained National Audits Group Managing Director, Steven Watson.
“In recent years there’s been a lot of automation that’s come through, and relying on that automation too much and not checking on the figures as they come through has led to these issues.”
“And by the end of year figures, the accountant hasn’t done the additional checks to make sure that the data is accurate,” said Mr Watson.
Put simply, this has resulted in businesses blindly trusting technology and abandoning due processes.
While there isn’t anything wrong with using database management technology, especially for data-driven companies. Advisors may need to reinforce the importance of using that technology correctly and in a uniform way so dirty data doesn’t occur.
“Obviously use technology the best way that you can, but there’s always that level of human intervention needed to check over the data. Technology is a tool, it must be used wisely,” explained Mr Watson.
What is the impact of dirty data?
Dirty data may increase the risk of businesses losing potential revenue by not being diligent about the information that is being recorded.
“Firstly, ask yourself how up-to-date your client base is. What opportunities are you missing out on because that data is not up-to-date,” suggests Mr Watson.
“There are so many lost opportunities because the data is not correct, so you’re not using it in the best way you can.”
Not only will businesses struggle with lost revenue opportunities, but productivity across the business can suffer if dirty data is left to accrue.
Data cleansing is mostly a manual process which means it may be labor intensive, hard to scale and is dependent on accuracy from those performing the cleanse. Employees who are tasked with data cleansing are then losing hours that could be better spent supporting the business.
Dirty data makes the information being used less reliable, and those who rely on it are forced to spend additional hours manually confirming it to be accurate. And not only can this impact end-users on the ground level, but executives may be unable to make future decisions if they cannot rely on their internal data.
Taking responsibility and cleaning up dirty data
Research from MarkLogic shows that data inaccuracies cost companies between 15 – 25 per cent in revenue, amounting to a sizable $400 billion lost due to dirty data.
When it comes to managing dirty data, it can be challenging for accountants and advisors to determine where the blame falls. But Mr Watson argues that everyone should take some responsibility to allow for greater value to be added back into the company.
“We’re partly to blame if clients have dirty data. What we need to do is be a bit more proactive in engaging with clients and assisting them to get it right the first time so you don’t have to go back over and over.”
“We really want to add value in the role that we play as independent auditor, and if our time is consumed with having to fix things and chase up papers, our time becomes limited. This can lead to overruns, which sees the client upset and then everyone loses in this situation.
“For us to perform our role as the auditor we have to get involved and tell them where things have gone wrong. This involves a little hand holding and looking forward to eliminate dirty data,” said Mr Watson.
Thankfully, there are practices that can be put in place to get on top of this growing problem moving forward. Advisers may want to work side-by-side with their clients, or offer recommendations, to help them better manage their data.
“[They] need to be cleaning data continually. One of the first steps you can recommend to clients is simplifying financial statements and getting rid of unnecessary notes that don’t make sense or need to be there.
“Export your client list and do a manual cleansing so it’s as accurate as possible moving forward and you can get the most out of that data. That is a continuous cycle because it can easily get out of control when left unmanaged,” said Mr Watson.
Whether data cleaning occurs biannually, once a quarter or more frequently, involving every department and encouraging them to strive for uniformity in their data entry is a great place to start.
This is where staff training comes into effect. Following a manual cleaning process, staff must be better trained in the process of data entry. The responsibility of which should fall on the client. Human error will occur, but if staff are better engaged with the significance of data quality and how dirty data can hurt the business, it should help to limit future occurrences.
As long as human error may occur, it’s imperative that advisers are more proactive in engaging with clients to ensure they’re manually cleaning their databases and not just relying on the latest technology to maintain accuracy.
