RBI Guv Asks Banks To Build Meaningful Human Oversight, Not Blame AI Models For Faulty Decisions

Reserve Bank Governor Sanjay Malhotra on Tuesday asked banks to have "meaningful human oversight" over decisions as a design principle while building artificial intelligence models, making it clear that blaming the technology tools will not be "acceptable" to the regulator.

Addressing the annual Fibac event in the financial capital, Malhotra urged banks to use artificial intelligence tools to augment rather than merely replace human judgment to serve customers better.

He made it clear that the RBI sees AI as a capability to be responsibly harnessed and not as a risk to be contained, and urged banks to adopt the tools rather than sitting on the sidelines.

However, as the system adopts AI, there can be an erosion of human judgement and accountability over a period time, Malhotra warned.

"For a bank's decision, the ultimate responsibility has to lie with the bank and not with the vendor or with the algorithm. We cannot say that it is the model who decided it. That can never be an acceptable answer, (18:35) whether to the bank, to the customer, or to the regulator," Malhotra said.

"Meaningful human oversight, the ability to explain, to intervene, and where necessary to override, must remain a design principle and not an afterthought," he added.

It can be noted that a slew of Indian lenders have been reporting a massive adoption of AI tools for a variety of functions, including credit underwriting and customer service, with some hoping that their reliance on human capital will reduce over a period time.

"(AI must be) used to augment rather than merely to replace human judgement," Malhotra said in the remarks that come amid wide-ranging concerns on the impact of AI on employment intensity across sectors in the economy.

Explaining his point, Malhotra said a relationship manager assisted by an AI system that presents the right product, the right risk flag can serve a higher number of customers more efficiently.

The governor also flagged potential risks and concerns from AI to the bankers, including cyber security risks, pointing to a recent incident of the AI itself attacking the system.

Banks need to guard against other risks from AI, including that of biases in models, Malhotra said, warning that there may be tendencies to have a liking or disliking for certain geographies, occupations or even communities.

Additionally, there is the risk of "herding" which can be triggered if a handful of financial models or tech vendors come to supply platforms, he said, fearing that a bias can perpetuate across the system in such a case.

He asked banks to look at data privacy beyond just complying with the requirements of the law, manage AI-related risks better when collaborating with outside partners and also ensure that AI platforms are able to explain why they recommend a particular decision on a proposal.

The RBI should be knowing as to which are the models that are actually running in banks, Malhotra said, asking banks to maintain a complete inventory of those models.

Banks need to establish a board-approved AI governance policy with clear accountability for outcomes and not merely for technology procurement, he said, adding that they should preserve meaningful human oversight at each and every point wherein AI's system error could cause material harm to a customer or have concerns for financial stability.