🦾 Shadow AI - 31 August 2023

Demystifying AI

This summer, a friend asked me how I see AI changing the security and IT space.

It was a big question while I was trying to unplug on vacation.

To answer it, we need to take a step back and discuss how AI is transforming businesses. We’re seeing:

• Personalized customer experiences

• Automated decision-making

• Rapid new product and service development

• Enhanced risk mitigation

• Improved employee productivity

• Streamlined business operations

Underpinning these business developments are external, employee, and product Large Language Models (LLMs):

👉 External LLMs: Public, third party LLMs like GPT, PaLM, Llama, and Claude)

👉 Employee LLMs: Company built LLMs ingesting internal data (e.g. HR, legal, and engineering)

👉 Product LLMs: Consumer focused LLMs built into product offerings

Although use of these LLMs will come with significant opportunity for the business, it also introduces significant risks, including:

• Prompt manipulation: manipulating an LLM through deceptive inputs, causing unintended actions by the LLM

• Sensitive data exposure: inadvertently revealing confidential data in its outs, leading to unauthorized data access, privacy violations, and security breaches

• Data poisoning: tampering of LLM training data which introduces vulnerabilities or biases that compromise security, effectiveness, or ethical behavior.

It’s not surprising IT and security teams are struggling to keep up as the WSJ recently noted. Traditional security and IT approaches won’t suffice. Each week this section will provide an informative view (with some expert guest collaborators) into what this means for you and strategies for successfully tackling this rapidly evolving space.

AI News to Know

• The Wh-AI-te House: Artificial Intelligence and National Security, including cyber security, are the top two Research and Development priorities of the Federal Government, further emphasizing the strategic importance of this space.

  • AI: The White House seeks to advance trustworthy AI that protects rights and safety while also harnessing AI to accelerate national progress.

  • Cybersecurity: The White House seeks to mitigate cybersecurity risks through resiliency and secure by design initiatives.

• Enterprise ChatGPT: Over 80% of Fortune 500 companies have employees using ChatGPT and now an enterprise version is available with key security and privacy features including customer data not used for training models, SSO, encryption at rest and in transit, SOC2 compliance, domain verification, and easier admin and user management.

• Lack of Diversity in Data Sets: The Rolling Stone has a feature on 5 woman AI experts who have been raising concern over AI risks and the harm that AI can have by exacerbating racism, sexism, and bias through flawed training of data models. The risks extend to hiring tools, crime-prediction software, and educational access.

• Trending ☝️ - “Nutrition” Labels: On the heels of the planned cybersecurity labeling for smart devices, Twilio is extending the concept to AI to give consumers and businesses a more transparent and clear view into how their data is being used so they can make informed decisions on AI adoption.

• Safeguarding LLMs: LLM Security provides an excellent resource for the types of real world attacks manifested against LLMs, defenses and detections, and best security practices when building and using LLMs.

AI on the Market

• AI and Security at the Cloud 100: Forbes recently released their Cloud 100 list, which highlights the world’s best private cloud companies measured by market leadership, estimated valuation, operating metrics, and people & culture. 

  • 55% of the Cloud 100 companies are already incorporating Generative AI into their products

  • But… based on my research, 41% of the Cloud 100 currently do not have a CISO or equivalent, including 12 companies incorporating Generative AI into their products

  • I’d love to see future rankings like these start to factor in a company’s security commitment.

• Democratizing Good ML: Hugging Face, an AI collaboration platform for developers listed as #98 on the Cloud 100, raised $235M in Series D funding at a valuation of $4.5B. They plan to use the funding to build out their services and team, and it’ll be interesting to watch what that means for their security function.

• AI Startup Fundraising on 🔥: Hugging Face isn’t alone. Crunchbase reports that AI funding for startups more than doubled year-over-year and over 25% of all funding YTD has been for AI-related startups.