Robin Bloor wrote a very interesting piece about why business needs explicit audit trails for data: http://www.dataintegrationblog.com/robin-bloor/the-quality-of-data-is-not-strained/
Here’s the gist: “… data has no explicit audit trails, so we lose information, some of which may be critical. This has the effect of hiding the record of and the impact of data errors.” He says all data should be time-stamped. I agree … but it’s not enough.
I’ve believed for decades (before it was conceivably practical) in the concept of the ‘perfect machine’. In the perfect machine, there are really no such things as changes or deletes; instead, all data just get time-stamped when created and the newer data takes active preference over the older. That gives you built-in audit trails — of data — and more importantly, (mechanized) business memory.
There is, however, at least one circumstance in which data really does need to be deleted (purged). That’s when the business is required for legal reasons to expunge all ‘records’ about some sensitive matter (e.g., the legal transgressions of a minor at age 18). In other words, there are *business rules* that sometimes require true ‘deletes’. Business rules can trump ‘perfect memory’.
Actually, there are business rules for virtually all ‘changes’ made to operational data. Failure to apply the correct business rules consistently is the rrot cause of poor data quality. And good luck trying to reconstruct after-the-fact what really happened (what business rules were actually applied) at some past point in time when things downstream go awry. And they will, of course.
What’s needed is not just an audit trail of changes to data, but an audit trail of all business rules
tested at each point in time some data ‘changed’ (or was created or deleted). If the system automatically logs the relevant business rules, then you have a true, complete and indisputable record of ‘history’.
I’m talking here about nonething less here than built-in (mechanized) methods of addressing compliance. No human intervention. Compliance across the board … with external regulations, contracts, agreements, deals, business policies … whatever. If you want your business to be both extremely agile and highly compliant (which is to say, always meet operational expectations), there’s untimately no alternative. How else can you support one-on-one customized relationships with a million customers?
Of course, first you would have to know your business rules. There’s the rub. Do you?!