🦾 Shadow AI - 19 October 2023

Demystifying AI - (Lack of) Transparency at AI Companies

Stanford released a must read research paper introducing the Foundation Model Transparency Index and evaluating over 100 transparency indicators at 10 foundational AI companies.

As the researchers note in the paper, “while the societal impact of foundation models is growing, transparency is on the decline, mirroring the opacity that has plagued past digital technologies (e.g. social media). Reversing this trend is essential: transparency is a vital precondition for public accountability, scientific innovation, and effective governance.”

In a LinkedIn post, co-author Kevin Klyman summarizes several key takeaways:

1. No foundation model developer gets an average transparency score higher than 57% with the open source LLama 2 scoring best.

   

2. No company shares adequate information about any copyrighted data it uses, the environmental impact of building its flagship foundation model, evaluations of how the model might be misused, or the downstream impact of the model. These are all areas where legislation should promote greater transparency.

3. There is no consistency in disclosures with at least one company sharing sufficient information on more than 80 our of 100 indicators. A common framework for transparency disclosures should be leveraged to help users understand model risks, dependencies, and impacts.  

So what?

AI News to Know

• Next Up at the AI Insights Forum: Axios reports that Senate Majority Leader Chuck Schumer’s second of nine planned AI Insights Forum is scheduled for October 24th and will focus on AI innovation. Invitees include a mix of high profile tech executives, venture capitalists, academics, and civil society advocates. Senator Schumer is on pace to hold one forum a month which, at this rate, would put the earliest timeline for U.S. AI legislation at the 2nd half of 2024. Schumer has invited well-known venture capitalist Marc Andreessen to this forum. Marc recently denounced regulation of technology and AI in his “Techno-Optimist Manifesto.” Here’s a snippet:

  

• OWASP Top 10 for LLMs Update: OWASP released its first major update to the Top 10 security risks when deploying and managing LLMs. Major changes in the latest version include:

  1. A high level architectural diagram of a hypothetical LLM

  2. Additional mitigation and detection strategies to address common risks like prompt injection, and

  3. Clarification on the differences between insecure output handling (issues in LLM-generated outputs before passed downstream) and overreliance (overdependence on the accuracy and appropriateness of LLM outputs).

• UK’s AI Safety Summit: The UK released its agenda for the AI Safety Summit scheduled for November 1st and 2nd. The Summit will bring together international governments, leading AI companies, civil society groups and experts in research to discuss topics such as:

  1. Understanding Frontier AI Risks

  2. Improving Frontier AI Safety, and

  3. AI for Good

AI on the Market

• AI Export Restrictions: The Biden Administration has issued new export restrictions on AI chips and manufacturing equipment to China in an effort to maintain military superiority. It will be interesting to see if the “AI Chip War” will prompt more cyber attacks against semiconductor companies, who have already been under stress of attack (Nvidia, Applied Materials).

• Deepfake Detection Funding: Last week, Shadow AI discussed the increasing prevalence of deepfakes and the market is sensing the opportunity. Reality Defender, which began as a non-profit, has raised $15M in Series A funding to detect, text, video, and image deepfakes.