Talia Gillis has posted Putting Disclosure to the Test: Toward Better Evidence-Based Policy to SSRN. This is another one of those papers that seems so esoteric, but really addresses an incredibly important topic in consumer protection.  The abstract reads,

Financial disclosures no longer enjoy the immunity from criticism they once had. While disclosures remain the hallmark of numerous areas of regulation, there is increasing skepticism as to whether disclosures are understood by consumers and do in fact improve consumer welfare. Debates on the virtues of disclosures overlook the process by which regulators continue to mandate disclosures. This article fills this gap by analyzing the testing of proposed disclosures, which is an increasingly popular way for regulators to establish the benefits of disclosure. If the testing methodology is misguided then the premise on which disclosures are adopted is flawed, leaving consumers unprotected. This article focuses on two recent major testing efforts: the European Union’s testing of fund disclosure and the Consumer Financial Protection Bureau’s testing of the integrated mortgage disclosures, which will go into effect on August 1, 2015.

Despite the substantial resources invested in these quantitative studies, regulation based on study results is unlikely to benefit consumers since the testing lacks both external and internal validity. The generalizability of the testing is called into question since the isolated conditions of testing overlook the reality of financial transactions. Moreover, the testing method mistakenly assumes a direct link between comprehension and improved decisions, and so erroneously uses comprehension tests.

As disclosure becomes more central to people’s daily lives, from medical decision aids to nutritional labels, greater attention should be given to the testing policies that justify their implementation. This article proposes several ways to improve the content and design of quantitative studies as we enter the era of testing.

One of those clauses bears repeating: “the testing method mistakenly assumes a direct link between comprehension and improved decisions.” I have said repeatedly that the CFPB should rigorously test its financial literacy initiatives because the academic literature does not lend much support to the claim that those initiatives actually help consumers make better financial decisions.

This paper makes a strong case that the CFPB is not paying sufficient attention to the scholarly literature in this area. If so, it may, as a result, lead consumers down a path paved with good intentions that ends at a destination nobody wants to go.