The Question: I asked for feedback on LinkedIn about using agile for target business problems focusing on an area subject to regulations, contract terms & conditions, agreement/deal provisions, business policies, etc. (Sound like your area?) The best reply I got back …       Guest Post by Sujatha Ramesh Sr. Manager, Business Solutions at LearningMate “In highly regulated environments, documented evidence of decisions, impact and approvals are a necessity. Agile practices discourage maintaining extensive documenting, relying on ‘sitting next to the team and explaining’ instead. You will generate precious little proof of what transpired and the decisions taken in doing so.”

I am pleased to announce that a comprehensive set of authoritative FAQs about the Business Rules Manifesto has been added to BRcommunity.com: http://www.brcommunity.com/brm.php The Manifesto is celebrating its 10^th anniversary this year – and remains as powerful and as vibrant to today’s business challenges as ever. I will be covering a good many of its insights in my Sunday tutorial, Business Rules: The Why and the What, at this year’s Business Rules Forum conference, part of BBC2012 (Oct. 28 – Nov. 2, Ft. Lauderdale, FL): http://www.buildingbusinesscapability.com/ The FAQs explain in-depth how the Manifesto relates to a great many pressing issues on today’s business agenda, including …

• Requirements
• Business Processes
• Business Analysis
• Business Agility
• Enterprise Architecture
• Zachman Framework  
• Knowledge Retention
• Events
• Enforcement

The new BRCommunity Insider also provides insight into the general positioning and structure of the Manifesto, as well as point-by-point clarification of individual Principles. The Manifesto is just 2-pages and free. It has now been translated into some 15 languages: http://www.businessrulesgroup.org/brmanifesto.htm The Manifesto is a rare thing in our field – a timeless work that seems more and more prescient which each passing year.

1. It is always important where there’s any chance of ever having to say why. 2. It should always be channel-independent. 3. You should never put yourself in a position of having to reconstruct what rules were applied in making an evaluation or decision after-the-fact. Compliance shouldn’t be the issue that it is – it doesn’t have to be. Rules of record should be a built-in feature of architectures. For all these years, we’ve been thinking ‘data’ is the key to compliance. We should have been thinking rules as applied to data. ~~~~~~~~~~~ Follow-up to my post on Rules of Record: http://goo.gl/dbKaD

Business computing today is characterized by a tangled web of interfaces and data movement from system to system, from source to sink. Knowing the official ‘systems of record’ is basic for resolving discrepancies and demonstrating compliance. But that’s merely a start. What your business really needs today is to know the rules of record. A focus on business rules, rule management, and decision management makes that possible. This post explains. First a little background. If every piece of data in the organization had a single, clear home, identifying official versions would be easy. But in today’s world, operational data is extracted, merged, massaged, re-platformed, and reported many times over, often obscuring original sources. Identifying a ‘system of record’ establishes which source is official for each element (or chunk) of data. In other words, it gives you physical traceability back to source.^[1] That’s a first and very basic step for compliance. What physical traceability doesn’t do is explain why that source may have a different value from the copy of the data you’re actually seeing. You might call that difference semantic drift (although that term probably dignifies what often is just plain sloppiness about the meaning and use of data). Since you can’t be sure what rules were applied to create the official version, you can’t easily do a delta on what you’re actually seeing. In other words, you have no semantic traceability back to source. From a compliance point of view, knowing the rules for source data is paramount. Of course, if those rules never changed, you might be able to reconstruct them at an acceptable price when and as needed for compliance. But today, change — at an ever faster pace — is the name of the game. Add massive personalization and customization to the mix and you quickly reach an impossible threshold of complexity and expense. The solution is simply never putting yourself in a position of having to reconstruct rules. Rather, you want to manage rules in such way to keep them ‘on the record’ — i.e., to maintain rules of record for each operational business decision your company makes. Impossible? Not at all. That’s exactly what business rules, rule management, and decision management is all about. There is proven technology to support it.^[2] All you need to do is change how you look at the problem. References [1] For additional background on ‘system of record’ see: http://en.wikipedia.org/wiki/System_of_record http://www.yourwindow.to/information-security/gl_systemofrecord.htm [2] For example, Raden and Taylor write, “Logging rules as they execute to create a record of how a decision was made is a common requirement of decision services. They can be managed like a typical logging requirement, except that using business rules makes it easier. Using business rules also makes it possible to use the logged information in both customer-facing and regulatory conversations, because the rules are more user-friendly.” James Taylor and Neil Raden, Smart (Enough) Systems: How to Deliver Competitive Advantage by Automating Hidden Decisions, Prentice-Hall (June 2007), ISBN: 0132347962, p. 358. ~~~~~~~~~~~~~~~~~~ The original version of this post appeared as: “‘Rules of Record’ ~ Why ‘System of Record’ Isn’t Enough,” Business Rules Journal, Vol. 9, No. 1 (Jan. 2008) http://www.BRCommunity.com/a2008/b385.html

I was reading a fascinating article in The Economist about how robots, including military drones and driverless automobiles, increasingly need ethical guidance[1]. What does that have to do with business rules, you ask? Read on … In the next five years, software systems will begin to appear that bypass programming going more or less straight from regulations, contracts, agreements, deals, certifications, warranties, etc. (written in English or other natural language) to executing code. Think about the economics of the equation! If for no other reason (and there are many others), you’ll quickly see the why a snowballing migration to such platforms is inevitable. And these tools will do the same for business rules based on business policies. I said more or ‘more or less’ above because the tool will have to make certain assumptions about the meaning of what it ‘reads’. For example, if I say, “a person must not be married to more than one other person” most of us would probably assume that means “at a given point in time”. But automated tools could easily be held responsible for making the wrong interpretation. It should therefore err on the safe side, and at the very least, log all its reasoning. That’s where the article comes in. Concerning robots that make liability-laden decisions, it contends that principles are needed …

“… to determine whether the designer, the programmer, the manufacturer or the operator is at fault if an autonomous drone strike goes wrong or a driverless car had an accident. In order to allocate responsibility, autonomous systems must keep detailed logs so that they can explain the reasoning behind their decisions when necessary.” [emphasis added]

That explanation better be in a form that humans (and lawyers too) can actually read. That means structured natural language. The article went on to make the following astute observation …

“This has implications for system design: it may, for example, rule out the use of artificial neural networks … decision-making systems that learn from example rather than obeying predefined rules.”

Right! Where there is social liability, there will always be natural language. P.S. To vendors: If your meaning of ‘business rule’ doesn’t compel you toward this debate, then you’re simply not really doing ‘business rules’(!).

Celebrating the 10^th Anniversary of the Business Rules Manifesto[1] http://www.businessrulesgroup.org/brmanifesto.htm FAQ #6 Question: What role should business rules play in enterprise architecture? Reverse-engineering business rules from legacy systems accurately is virtually impossible. The full, original business intent is simply lost. Reconstruction of business logic has been tried time and time again, often aided by automated tools, but measured against time and cost, seldom achieves satisfactory results. What a waste! The solution is simply to stop hard-coding business rules into procedural languages. Rules will change and they will be needed for new business initiatives and platforms. The opportunity costs of continuing to follow traditional practices – not to mention the ‘maintenance’ costs – is simply too great. The alternative is applying rule technology that can support rules expressed in more natural (declarative) form. The Manifesto summarizes these points as follows …

6.2. Executing rules directly – for example in a rules engine – is a better implementation strategy than transcribing the rules into some procedural form.

With computing power so vastly improved, there is less and less reason every day to support business rules using procedural languages. Why are we still programming the evaluation of rules ourselves?! Just as a DBMS removes data management as an application concern, so too does a rule technology for the evaluation of rules. A related issue is compliance – not just regulatory compliance, but compliance with contractual obligations, deals, agreements, licenses, warranties, and so on. If you want to delight customers, keep your commitments. To do so you must be able to determine how your systems actually got the outcomes they did. That way, if there’s a mistake you can correct it. So the Manifesto says …

6.3. A business rule system must always be able to explain the reasoning by which it arrives at conclusions or takes action.

Today, demonstrating compliance is a largely hit-or-miss affair, always after the fact. Does it have to be? No! A state-of-the-art enterprise architecture is one that logs the rules used to make evaluations and decisions just as a DBMS logs all transactions. Compliance based on rules can and should be built-in.

The guest post by Cecilia Pearce earlier this week (http://goo.gl/QL9zL) stirred up an unexpected controversy, one that deserves clarification – are business rules and expert systems the same? No!  Business Rules  Business rules are organizational rules created for the purpose of running day-to-day business operations. Business rules always have an original source in the form of some law, act, statute, regulation, contract, agreement, business deal, business policy, license, certification, service level agreement, etc. I often like to say that business rules are really about keeping commitments. Expert Rules  In response to Cecilia’s post, Rolando Hernandez explains:

 “If you need to go beyond gathering [i.e., harvesting or mining] business rules to trying to understand “The Knowledge” [sic] that experts know, how experts think and decide, what the expert rules are, or what the higher-level heuristic rules are, then knowledge engineers … will keep using “knowledge acquisition” (KA), “knowledge representation” (KR), and “knowledge engineering” (KE). AI guys know that means.”

In other words, expert rules arise from an individual who is outstanding at his particular knowledge task. That’s very different. Expert Systems  Wikipedia describes expert systems as follows: 

 “software that uses a knowledge base of human expertise for problem solving, or to clarify uncertainties where normally one or more human experts would need to be consulted … a traditional application and/or subfield of artificial intelligence (AI)”

Bob Whyte, a practitioner for a major insurance company, makes the following observation about the difference between business rules and expert systems[1]:

“What makes the real-world challenge of managing business rules so much more tractable than it appeared to academics and researchers in the1980s, the heyday of knowledge engineering and expert systems, is that in the day-to-day business world the institution plays role of ‘god’. 

… for business rules the problem is not one of having to discover and define hidden, unknown or unexpressed rules, which takes you into byzantine solution spaces, but rather one of documenting known rules invented overtly and explicitly by actual historical person(s). 

With business rules you are generally not discovering rules no one has ever consciously considered, but rather uncovering rules that some manager, lawyer or other expert decided on one day, but probably did not record simply for lack of an appropriate infrastructure for rulebook management.”

Excellent clarification!

---

[1] from Building Business Solutions: Business Analysis with Business Rules, by Ronald G. Ross with Gladys S.W. Lam, An IIBA^® Sponsored Handbook, 2011, pp. 257-258http://www.brsolutions.com/bbs 

I am always very careful to talk about ‘business rules’, not ‘rules’. We define ‘business rule’ as a criterion used in business operations to guide behavior, shape judgments or make decisions. We have not perceived any poverty of guidance that fits that definition(!). One reason we always say ‘business rules’ is because we make no attempt to capture rules that mimic intelligent behavior in the original sense of expert systems. That problem is at least an order of magnitude more difficult than capturing organizational (business) rules … which is hard enough as it is. Business rules always trace to some source … even if the source is now unknown (or has gone to work for the competition). Organizations are dependent on people on the inside, and on interactions with people and organizations on the outside. So a great many business rules are about the behavior (of people and organizations). Ensuring conformance with such behavioral rules – and being able to change and redeploy them quickly – is the fundamental failing of traditional software development. It literally boils down to keeping your commitments (as expressed in contracts, agreements, MOUs, deals, certifications, regulations, acts, laws and so on). I’m all for better decision management, but fail to address the root problem of software engineering and we’re simply grasping at straws.

@SergeThorn asks “Are Business Process Management and Business Architecture a perfect match?” http://goo.gl/kLYvs With business rules far better! There is an important overlap of concerns between business rules and enterprise architecture. The overlap actually comes in two forms: 1. Operation-time. Here the issue is really rethinking compliance. I just blogged about this today: http://goo.gl/Gl9wT 2. Business-analysis-time. A business needs to know the rules it plays by. That inevitably brings you to rulebook management. For more: http://www.brcommunity.com/b500.php?zoom_highlight=rulebook (and other articles).

Robin Bloor wrote  a very interesting piece about why business needs explicit audit trails for data: http://www.dataintegrationblog.com/robin-bloor/the-quality-of-data-is-not-strained/ Here’s the gist: “… data has no explicit audit trails, so we lose information, some of which may be critical. This has the effect of hiding the record of and the impact of data errors.” He says all data should be time-stamped. I agree … but it’s not enough. I’ve believed for decades (before it was conceivably practical) in the concept of the ‘perfect machine’. In the perfect machine, there are really no such things as changes or deletes; instead, all data just get time-stamped when created and the newer data takes active preference over the older. That gives you built-in audit trails — of data — and more importantly, (mechanized) business memory.  There is, however, at least one circumstance in which data really does need to be deleted (purged). That’s when the business is required for legal reasons to expunge all ‘records’ about some sensitive matter (e.g., the legal transgressions of a minor at age 18). In other words, there are *business rules* that sometimes require true ‘deletes’. Business rules can trump ‘perfect memory’.  Actually, there are business rules for virtually all ‘changes’ made to operational data. Failure to apply the correct business rules consistently is the rrot cause of poor data quality. And good luck trying to reconstruct after-the-fact what really happened (what business rules were actually applied) at some past point in time when things downstream go awry. And they will, of course.  What’s needed is not just an audit trail of changes to data, but an audit trail of all business rules tested at each point in time some data ‘changed’ (or was created or deleted). If the system automatically logs the relevant business rules, then you have a true, complete and indisputable record of ‘history’.  I’m talking here about nonething less here than built-in (mechanized) methods of addressing compliance. No human intervention. Compliance across the board … with external regulations, contracts, agreements, deals, business policies … whatever.  If you want your business to be both extremely agile and highly compliant (which is to say, always meet operational expectations), there’s untimately no alternative. How else can you support one-on-one customized relationships with a million customers? Of course, first you would have to know your business rules. There’s the rub. Do you?!