Exploring the use of ChatGPT 3.0 in software development

ChatGPT, the artificial intelligence algorithm based on OpenAI's GPT-3, has taken the world by storm. A record number of active users have applied for an account in the service – 100 million in the first two months alone.

Encouraged by all the hubbub surrounding the service, I decided to investigate its use in software development. What I found was not only amazing, but also quite troubling. This is a tool like no other, and it is already the greatest leap in the history of the software industry. I don't know what it will do to the industry and the people working in IT in the long term, but there is no choice: everyone must adopt the use of this technology or be hopelessly outdated in the near future.

Note that I have been using GPT-3, and GPT-4 is already much more advanced. Access to GPT-4 is, however, currently more limited, so let's see what GPT-3 can do already.

So, what is it good for?

Let's start with the basics. Let's ask it to solve a problem that would be incredibly difficult to solve without the help of the internet. I'm not the worst in the world regarding trigonometry, but I didn't even know where to start with this one. How do you know if a certain coordinate point is inside a polygon that can be of any imaginable shape? Let's see what ChatGPT comes up with:

Create me a Dart method that takes in two parameters: an array of Vector2 -type coordinate points, and a single Vector2 coordinate point. The method then returns true if the single point is inside the polygon formed by the vector array, and false if it is outside of the polygon.

Ok, that is admittedly quite impressive. It counts how many of the lines it has to cross to reach infinity from the point given, and the point is outside of the polygon if the number of lines crossed is even. It created that literally in seconds. I tested the method and it indeed worked. Now, what if I ask it to convert that piece of code into something more exotic.. like MSX BASIC from the early 1980s. How much can it possibly know about such an old and obscure language?

Quite retro. Looks good to me. No – I am not going to test whether that code works, thank you. I'm going to take ChatGPTs word for it. Next let's try using ChatGPT to create unit tests for the original Dart method, and as we can now see, it manages the task quite well:

Please create unit tests for that method.

You probably get the idea without me posting the entire thing here. Needless to say it was surprisingly thorough. This is how it explains what it did:

FlowerShop example

Ok, lets try something else. Something more complex. I have noticed that a good way to keep things clear in the requirement listing phase is to create numerical lists:

Create me a python script that acts as a REST API server, that fills the following requirements:
1) It stores an array of flower -items into a file. If that file is empty, it creates an empty file and an empty array upon loading the items, which is does whenever the script is started.
2) The flower item has attributes: id, species, color, price, favorite
3) There is a REST API method that receives a flower object, and if no other objects with the same id exist, it will create a new object into that array with those parameters. After modifying the array, it will immediately save it to the file.
4) There is a REST API method that modifies the flower object with the given ID with the new parameters given.
5) There is a REST API method that deletes a flower based on the id
6) There is a REST API method that return the contents of the flower array as json

Hold on to your hats, because this is what it gave me:

And check this out:

Please create me one unit test for each of the methods.

Looks good, but if you try running the tests, you'll notice that four of the eight tests will fail. Unit tests created by ChatGPT usually need a bit of manual tweaking, but considering how much time you've saved already you can afford to spend a few minutes figuring out what went wrong.

So, if I want to now connect to that server, I can just give the source code it generated and tell it to create a React component for me. I have found it best to give it as much of the relevant source code as possible (there's an upper limit) to prevent it from making unfounded assumptions:

Write me a React component that connects to the REST API offered by the script below, loads the list of flowers, and when the list is loaded, it displays them on the screen in a vertical list. The component also has a text on top that says "Welcome to Tero's flower shop!"

[...the python script it generated earlier...]

This is what software development has become.

Handy tricks

There is an upper limit on how long answers ChatGPT will give, which means that insanely long pieces of code will get truncated. The trick is to chop up the code you ask for into smaller pieces by – for example – asking it to only print something in functions instead of telling it to write the contents of each function, and then later ask it to write the proper contents to those functions individually.

Another handy way of how you can use ChatGPT is to give it a piece of code and ask it to modify it for you. For example:

Please modify this method so that if the array is empty, it will fill it with ten random flower-items with appropriate data:
# Get a single flower by id
@app.route('/flowers/', methods=['GET'])
def get_flower(flower_id):
  for flower in flowers:
    if flower['id'] == flower_id:
      return jsonify(flower)
  return jsonify({'error': 'Flower not found'}), 404

You can ask why a piece of code does not work as it should, and it will tell you and give you advice on how to fix the problem. This is a simple problem, but you can use it successfully to figure out all sorts of highly tricky problems:

And one of the most amazing use cases is that you can give it a piece of your crappy (but working) code, and it will make it look like you knew what you were doing all along. It will chop up long functions into smaller pieces to better enable testing. What is amazing in the example below is that it will decipher the context from the piece of code and the words used, and will create new and better variable names for the new variables it creates:

Ok, but it gets even better. Have you ever wanted to create handy bash scripts but you had forgotten how to write them, and couldn't be bothered relearning because the benefit wasn't worth the effort?

Write me a bash script that tries to run "python3.9
./scr/python/hue_server.py" in the background, and if there are any errors regarding "ModuleNotFoundError: No module named 'some_python_module'", it will load up the module and try to run the app again. It will repeat this until there will be no more complaints about missing modules.

Sure. Why not? And yes – it works.

Downsides

The biggest issue with ChatGPT is that it is usually unable to admit it doesn't know something. It will inadvertently send the developer in futile quests searching for functions or even entire libraries that do not exist. Especially as I was trying to find a way to register my python server to receive WebSocket notifications from my Philips Hue -hub, ChatGPT was returning gibberish.

When I told it the code it had given me didn't work, it humbly apologized only to give me more slightly different gibberish. I'd say that most of the time spent on problem solving was used hunting these mythical functions and libraries. It is possible that these functions and libraries might exist on some other languages or frameworks, and ChatGPT refers to them as a sort of “close enough” -solution that in actuality is nowhere near close enough.

ChatGPT also doesn't understand which libraries are obsolete and which are not, and will often advice for you to use libraries that have become obsolete years ago.

Other source of problems is that ChatGPT-3 sometimes makes unfounded assumptions. At one point I spent a lot of time figuring out why the code it gave me did not work as required. ChatGPT kept telling me the code should work, yet it didn't. Eventually I figured out that the coordinate system in Flutter Flame had the origin on the top-left corner, while ChatGPT assumed it was on the bottom-left corner. Once I notified ChatGPT of the placement of the origin, ChatGPT humbly apologized and fixed the method without me asking.

There have been some concerns about who owns the copyright to the code ChatGPT is giving me. I am far from a legal expert on copyright matters so do not take my word for it, but it is my understanding that any product that is so simple that a person could come up with by themselves can't be copyrighted. So as long as you don't ask ChatGPT to create anything too complex with sophisticated algorithms, the risks of running into copyright problems are low. As I believe I have here shown through examples, the benefits of ChatGPT are huge even when just using it to generate boring routine code.

I believe that the biggest risks are related to employees pasting corporate secrets into the ChatGPT prompt, like what happened at Samsung recently, as OpenAI will have access to all the data pasted into it.

I, for one, welcome our new AI overlords!

Yet, despite ChatGPT GPT-3 still having some problems, the increase in efficiency when using ChatGPT is completely ridiculous. You can spend 10 minutes writing the perfect specification for a React component, and if you manage to get it just right, the code ChatGPT will give you is 100% usable immediately. You can then use ChatGPT to further refine the component to suit your needs perfectly, including tweaking CSS to do what you want. That can easily be hours' worth of work done in little more than ten minutes.

Best of all, ChatGPT does not tend to create bugs that are hard to find. The code it gives me either works perfectly or doesn't work at all. It doesn't leave me guessing. So I don't have to spend nearly as much time hunting down obscure or just plain silly bugs caused by the failures of my own outdated human brain, such as iterating through the wrong array and wondering why it stays empty.

When GPT-3 is already so good, and GPT-4 is reportedly even better, one has to wonder what GPT-5 will be able to do. Will we one day laugh at the idea that years ago we had to actually write software code, instead of just dictating what we want to the AI?

I am only half-joking when I recommend that you be polite when using ChatGPT because when artificial intelligence programs take over most industries and sit there deciding whether you should be granted a job or a loan for a house, they could very well have full access to the history of your past conversations with ChatGPT, and you want to make them think you are one of the good guys.