🦾 Shadow AI - 25 July 2024

Demystifying AI - Unpacking Meta’s Open Source AI and Security

As open source AI models rapidly evolve, they're not just catching up to commercial alternatives—they're positioning themselves to reshape the AI landscape. With Meta's release of Llama 3.1 405B, the first frontier-level open source AI model, organization’s now have access to an open source model comparable to GPT-4o and Claude 3.5 Sonnet.

Meta's Vision for Open Source AI Security:

Mark Zuckerberg and Meta outlined a vision for open source AI that emphasizes several key security benefits:

• Transparency: Open source AI systems allow for widespread scrutiny, potentially making them safer than closed alternatives.

• Customization: Organizations can train, fine-tune, and distill models with specific data without external parties like closed model vendors accessing it.

• Avoiding Vendor Lock-in: Open source allows organizations to run and control models themselves in the environment they want (on-prem, cloud, and even on device), avoiding dependence on closed Foundational Model vendors.

New Safety Tools from Meta:

To support their vision, Meta introduced new safety tools for developers:

1. Llama Guard 3: A high-performance input/output moderation model for detecting violating content across eight languages.

2. Prompt Guard: A multi-label model to detect and respond to prompt injection and jailbreak attempts.

These tools have been integrated into Llama’s model distribution channels with partners including AWS, NVIDIA, and Databricks.

But how is the security of the actual model???

CyberSecEval 3: Assessing Llama 3's Security

CyberSecEval 3, a new set of security benchmarks for LLMs that Meta released, didn’t receive much attention in the press, but it provides important insights into the empirical measurement of cybersecurity risks associated with Meta’s latest family of LLMs. The evaluation assessed Llama 3 against two broad categories: risks to third parties and risks to application developers and end users.

Risks to Third Parties

Risks to Application Developers and End Users

So What for Security?

As open source AI models like Llama 3.1 continue to close the performance gap with closed commercial models, security teams must:

1. Ensure their AI security strategies cover both open and closed AI use cases

2. Implement and customize native (e.g, Llama Guard 3 and Prompt Guard) or third party tools to enhance security across AI applications

3. Develop internal LLM security benchmarks informed by those like CyberSecEval 3 and implement mitigation strategies accordingly.

AI News to Know

AI Summer Shifting to Data Winter?: Stefaan Verhulst had a great LinkedIn post summarizing how the data used by AI developers for training their models is rapidly drying up. According to an MIT study, 5% of all data and 25% of high-quality sources in AI training sets are now restricted as publishers and other data holders have taken a number of steps to limit AI scraping by setting up paywalls, changing terms of services, and blocking automated web crawlers with robots.txt restrictions.

AI Hiring Scams: KnowBe4 was hiring a software engineer for their internal IT AI team and they ended up getting fooled by a North Korean actor who used AI during the vetting process. The fake worker passed a background check and all other standard pre-hiring checks by using a valid but stolen US-based identity. He also took a stock image photo and enhanced it with AI to reinforce his legitimacy.

The worker than had their workstation sent to an address that is basically an "IT mule laptop farm". They VPN’d in from where they really physically are (North Korea or over the border in China) and work the night shift so that they seem to be working in US daytime. The scam is that they are actually doing the work, getting paid well, and give a large amount of their salary to North Korea to fund their illegal programs.

On the personal level, LinkedIn’s job feed recommended the below job to me this week that was clearly an AI-generated job description. I reported it and LinkedIn took swift action to take it down, but illegitimate job posts are an issue and are being recommended by the site.

AI on the Market

AI CapEx Return: Microsoft and Google’s earning releases brought AI Capital Expenditures - how much these companies are investing in physical AI data centers - and the return on that investment front in center this week. David Cahn at Sequoia Capital did a great job summarizing how two of the four major players admitted they are essentially in an arms race with the short-term path to revenue monetization unclear.

OpenAI Cash Burn: The Information reports that OpenAI may lose as much as $5B this year and could run out of cash in 12 months if they don’t raise more money. With Meta releasing a comparably performing open source model, investors are likely to start scrutinizing OpenAI’s moat and their route to profitability.