AI Safety Initiative Formed by the Cloud Security Alliance
The AI Safety Initiative was announced on Tuesday by the Cloud Security Alliance (CSA), with Amazon, Anthropic, Google, Microsoft and OpenAI joining as partners.
The AI Safety Initiative effort also includes participation by academic experts, plus government agencies, such as U.S. Cybersecurity and Infrastructure Security Agency (CISA). The initiative has already attracted the greatest number of CSA program participants since its founding, with more than "1,500 expert participants" working in the following four core research groups:
- AI Technology and Risk Working Group
- AI Governance & Compliance Working Group
- AI Controls Working Group
- AI Organizational Responsibilities Working Group
The aim of the AI Safety Initiative is to create safety and security guidelines for artificial intelligence (AI), with an initial focus on generative AI.
Generative AI Security
CSA has already published its first publication on the topic, "Security Implications of ChatGPT," which offers information about the kind of concerns that the CSA AI Safety Initiative aims to address.
The publication (released in August 2023), outlined "how AI-driven systems can be exploited in different aspects of cyberattacks, including enumeration, foothold assistance, reconnaissance, phishing and the generation of polymorphic code." Those are all approaches attackers might try against computing environments.
Enumeration might entail using generative AI to ask about Nmap scans for open ports, or asking it about "the most prevalent applications associated with specific technologies or platforms," to help with attacks.
Attackers can leverage generative AI to gain a foothold in organizations by asking it to detect vulnerabilities in code samples, which might be leveraged to gain system access.
Generative AI can be used for passive and active reconnaissance on systems, or for generating phishing e-mails. It can pull together publicly available information on a target. It can also be used after an attacker has access, aiding them in information about "port scanning, network scanning and vulnerability scanning," per the publication.
Attackers could also use generative AI to create "polymorphic code," which is defined as "a type of code that can alter itself using a polymorphic (“mutation”) engine while maintaining the functionality of its original algorithm." Polymorphic code can be used in malware used to bypass security measures.
CSA's publication also mentioned the issue of users attacking generative AI by "prompt injection," by delivering so-called "malicious prompts." Generative AI systems could be corrupted to "spread disinformation, interfere with its functioning, or deceive the AI into producing unsuitable responses by feeding it false or misleading information." Attackers may also use role-playing prompts to bypass generative AI's "filters and security protocols."
CSA's "Security Implications of ChatGPT" publication did not have specific guidelines for organizations on how to use generative AI securely. However, it promised a follow-up paper on the topic. Organizations meanwhile should develop guidelines and policies on the use of AI tools, protect sensitive information, protect communication channels against adversary-in-the-middle attacks, audit AI use and stay up to date on security developments.
CSA's initiative on AI and security comes shortly after other parallel efforts have launched.
In late October, the Biden administration announced voluntary guidelines with an executive order on AI security. The executive order included concerns that AI use might have "chemical, biological, radiological, nuclear and cybersecurity risks," along with fraud and privacy implications.
In late November, the UK's National Cyber Security Centre and CISA announced joint guidelines for the secure development of AI systems, which were said to complement the White House's guidelines. This "Guidelines for Secure AI System Development" publication offers a framework for the secure design, development, deployment and operations of AI systems.
The NCSC-CISA guidelines suggested that AI security is somewhat different than other systems. Attackers can leverage "adversarial machine learning" to skew the model's performance or classifications, or extract sensitive model information, for instance. AI providers should take responsibility for security at present since users "do not typically have sufficient visibility and/or expertise to fully understand, evaluate or address risks associated with systems they are using," the guidelines argued.
Automated AI attacks are still a few years off, according to a report published last year, "The Security Threat of AI-Enabled Cyberattacks," by security solutions firm WithSecure, which was commissioned by the Finnish Transport and Communications Agency Traficom. However, it's currently possible to use AI to impersonate people's voices and use it for "spear phishing" messaging (targeting specific individuals in an organization).
Kurt Mackie is senior news producer for 1105 Media's Converge360 group.