🦾 Shadow AI - 23 May 2024

Microsoft

In early May, Satya Nadella, the CEO of Microsoft, said the company would prioritize security above all else in response to the Department of Homeland Security’s Cyber Safety Review Board report. He pledged to double down on Secure by Design and Secure by Default.

Microsoft introduced its AI-optimized Co-Pilot PCs that include a new feature called “Recall.” Recall takes snapshots of a user’s screens every few seconds and stores the data locally on a user’s device. Users will reportedly have the ability to “Opt Out” of Recall, but it should be an “Opt In” feature in keeping with their secure by default commitment.

Additionally, Microsoft’s FAQ states that “Recall does not perform content moderation. It will not hide information such as passwords or financial account numbers.”

Regular screenshots of a user’s PC creates a treasure trove of sensitive information, including financial, health, and personal data.

If a bad actor gains privileged access to your device, they could have all this information at their fingertips.

Moreover, it’s unclear how sensitive software such as password managers, encrypted messaging apps, and HIPAA compliant applications will be able to run safely on Co-Pilot PCs.

To read more on the risks, check out Kevin Beaumont’s write up on how Recall fundamentally undermines Microsoft security.

OpenAI

As part of the AI Seoul Summit this week, OpenAI released a safety update that includes 10 practices they are using to safely develop and deploy frontier AI models, from red teaming, to alignment and safety research, to monitoring for abuse.

OpenAI dissolved their Superalignment AI safety team that was focused on managing long-term risks of AI.

Jan Leike, Head of the Superalignment team at OpenAI, recently announced his departure, and publicly noted that “over the past years, safety culture and processes have taken a back seat to shiny products. We are long overdue in getting incredibly serious about the implications of AGI.”

Over the long term, AI has the potential to build machines that are smarter than humans. Jan’s public comments raises concern that OpenAI is not doing enough to prioritize safety within their foundational model.

Slack

Slack overhauled their AI privacy principles in 2023, which contained text that indicated their AI models were being trained on customer data, including messages, content, and files submitted to Slack.

This change was recently picked up on by the security and privacy community and Slack has since updated their privacy principles to clarify that they only analyze customer data to better parse queries, help with autocomplete and come up with emoji suggestions.

Slack does not offer an Opt-In option for this feature. Instead, to turn it off, Slack requires the workspace owner email its customer experience team requesting an opt-out.

Privacy policies are constantly evolving. It’s critical that security teams work closely with privacy teams to have ongoing monitoring and visibility into how vendors is using your company’s data for AI purposes and the associated risk.