“External audits of AI systems are increasingly recognized as a key mechanism for AI governance. The effectiveness of an audit, however, depends on the degree of system access granted to auditors. Recent audits of state-of-the-art AI systems have primarily relied on black-box access, in which auditors can only query the system and observe its outputs. However, white-box access to the system’s inner workings … allows an auditor to perform stronger attacks. … In this paper, we examine the limitations of black-box audits and the advantages of white- and outside-the-box audits.”

Find the paper and full list of authors at ArXiv.