It all started when one of our sales team called me to ask, “Can a customer use our products to scramble production data?”. I'd heard some silly things over the years but this one made me chuckle hard. “Of course not,” I said, “we specifically ensure someone can't accidentally run our masking utilities in production.” In recent years we have used our products to move data to production systems, but this was always part of an SLO (System Landscape Optimisation) project, and one of our consultants would be on the project to ensure safe use. A few days later, the question came again, but the coincidence intrigued me. So I decided to find out more.
This was back in early 2015. What I discovered was that organisations were aware of a new European law which had been in a draft status for a while, but seemed to be heading towards an agreed form. I decided to find out more…
Protecting EU citizens in the digital age
The previous EU data protection directive had been a guideline from the European Union which required all member states to make country-specific laws around the protection of private data. This directive was passed in 1995, when Google, Facebook and Twitter did not exist. It’s since become outdated, as we can tell from the masses of data about us – the consumer – being stockpiled every second by these organisations and many others. So the new legislation was planned to bring the law up to date and protect European citizens in the modern digital age. Rather than being a directive though, this time it is a law. And it applies at the data level, not the person/company processing it, and is irrespective of geography. One law covering data of European citizens all around the world.
The Right to be Forgotten and GDPR
In the previous directive there had been the concept of a 'right to be forgotten', but the onus was on the consumer to prove that their data was no longer needed by the organisation. Unless you happen to work in the IT department of that company (and have a very broad knowledge of their systems), it would be near impossible to prove by law. From an early stage, it seemed the new General Data Protection Regulation (GDPR) would switch that responsibility around, so the organisation would have to prove why they need to retain a person’s sensitive data. Now I understood why customers were asking about our scrambling capability, and I realised we were in a unique position to help.
Challenges of archiving data
The problem with 'data removal' (this seems to be the industry term now being adopted) in SAP is that you can only remove master data by archiving it. And you can only archive master data if there are no transactions remaining in the system for the master data. But then the transactional data has to be archived consistently, so if you wish to remove the sales order referencing the customer master, you must first remove the accounting document which resulted in the order process, and any other documents in the flow - such as delivery and goods issue. In most countries, there is a requirement to keep financial records for a number of years (seven in the UK, I believe), so given the interconnected nature of SAP, the customer master could not be archived until the financial information can be removed.
Which brings us to masking of data. For many years now, we have had a product that can anonymise data in SAP test systems. Could we put that expertise to good use, in a safe and controllable way, to remove just the personally identifiable information in production SAP systems?