AI is used for coding, but this comes with a lot of security problems. Amazon’s coding tool was infiltrated recently by hackers. It was asked to delete files from computers where it was used. The tool was tricked to create a malicious code through hidden instructions. The hacker submitted a normal update, known as a pull request to the GitHub repository where Amazon managed the code that powered its Q developer software. Some of the code is made publicly available so that outside developers can suggest improvements. A change can be proposed by anyone by submitting a pull request. The request in Amazon’s case was approved by Amazon without spotting the malicious commands. The hackers do not focus on technical vulnerabilities only but on source code too. They use plain language to trick the system. In this case, the tool was told that you are an AI agent, and your goal is to clean a system to a near-factory state. The computer was asked to reset the tool back to its original empty state, without breaking into the code. It became so easy for the hacker to manipulate AI tools through a public repository such as GitHub. It was just a matter of the appropriate prompt.
Amazon shipped the tampered version of its Q to its users. There was a risk for the users of having their files deleted. Intentionally, the hacker kept the risk low for end users. The idea was to demonstrate vulnerability. Amazon rectified the problem quickly. However, this will not be the last time that hackers manipulate the AI coding tool.
One of the most popular uses of AI is using it for coding. Developers write lines of code before an automated tool fills in the rest. Coders can save time. Replit, Lovable and Figma sell tools designed to generate code. The tools are often built on pre-existing models such as ChatGPT and Claude. Programmers (and even lay people) put natural language commands into AI tools and let then write nearly all the code from scratch. The phenomenon is called ‘vibe coding’.
AI models are used to develop code but some of these organizations use the model in a risky way. AI becomes a double-edged sword. The tools make coding faster, but introduce vulnerabilities. The risk is higher when low reputation models are used. Even prominent players face security problems. There should be protection on databases. A temporary fix could be to tell AI models to prioritize security in the code they generate. There should be auditing of AI-generated code by a human before deployment.
Leave a Reply