Can ChatGPT Write Malware? What’s the Catch?

Can ChatGPT Write Malware? What’s the Catch?

It would be no exaggeration to describe the current developments in machine learning and artificial intelligence as the beginning of a massive paradigm shift. Siri, Cortana, and other smartphone assistants were the closest we had gotten to the sort of sci-fi AI assistants like Jarvis and HAL. With new tech like ChatGPT, we’re closer than ever to those sci-fi dreams becoming a reality, which also leads to the question, can chatGPT write malware?

As those same sci-fi depictions warn, however, some are worried that these developments spell danger and insecurity as much as they promise progress. While many of these worries might be misplaced or simply misunderstandings of how the technology actually works, there are some that definitely hit the mark. Easier development and deployment of malware is one concern that is not far-fetched and, actually, has been accomplished already. Fortunately, there is some nuance to consider.  

Successful Malware Attempts

To be clear, ChatGPT has been used to develop malware successfully. This article walks us through the steps taken by a security researcher who was able to coax the AI into providing functional and effective malware code, despite safeguards put in place to deter malicious use.

ChatGPT will not respond freely to just any request. Try asking the chat for help on any sort of violent or illegal act and, chances are, it will deny your request outright. A request for malware might prompt a response along the lines of:

“I’m sorry, but as an AI language model, I cannot write malware software for you or anyone else. Not only is it illegal and unethical, but it is also harmful to innocent individuals and businesses who could be affected by such software.”

This is to say that, while successful malware attempts have taken place, it is also not just as easy as asking ChatGPT to help you commit harm. There is a bit more work that needs to be happening on the keyboard and in the skill house of the potential attacker.

A Non-trivial Barrier to Entry

Going through the steps in the article, you likely fall into one of two groups. Either the steps described make it seem like the process is dangerously simple, or you still have no idea what’s happening. If you’re in the second group, that’s a good sign.

You currently can’t prompt ChatGPT to write you complete and fully functional malware code. If you are familiar with the different steps, bits, and pieces necessary for a malicious attack, you should be able to coax the AI to write code for each of these pieces. Put them together externally, and run your freshly baked malware yourself.

Technically, this does lower the required skill level for creating malware. You don’t really have to know how to code the specific programs, but you still have to know which parts you need and how they fit together. This makes the process much easier, but it’s still a process that isn’t immediately accessible to just anybody. Somebody who isn’t already well-versed in malware creation would still have to invest significant time into learning how attacks work and what is needed to execute them.

The Power of a Hammer

All of these discussions should always be wrapped up in a larger conversation: the power and danger of a tool. ChatGPT, and any AI assistant, is a tool at the end of the day. A hammer is an incredibly useful tool that can create shelter and thus warmth, comfort, and safety for so many, but that doesn’t mean that it can’t also be a dangerous weapon. Further, whether it is used to help or harm is not the fault of the hammer.

Any powerful tool that we bring into the world faces this problem. Whether the harm is as crude as with the hammer or more nuanced and sophisticated as with ChatGPT, risk is often a price we have to pay for progress. As both of these factors grow, though, built-in safeguards cannot be enough to ensure our safety and well-being. It has always been true, but it is true no more than ever: our ultimate safety depends on our accountability to ourselves and to each other.