QTM 350 - Data Science Computing
Lecture 13 - Introduction to AI-Assisted Programming
Introduction to AI-Assisted Programming
- Yes, we all love ChatGPT! 🤖
- And LLMs (Large Language Models) are indeed changing the way we code
- This new paradigm is called AI-Assisted Programming
- It is not about replacing programmers, but about making them more productive (at least for now 😅)
- In this lecture, we will discuss the main concepts behind AI-Assisted Programming
- What LLMs are
- How they can help us write code
- Limits and challenges
- How to get started with GitHub Copilot
What are LLMs?
- LLMs are a type of neural network based on the Transformer architecture (that’s the T in GPT - Generative Pre-trained Transformer)
- Many important ideas behind neural networks were developed in the 1950s and 1960s (!), but the area has recently exploded due to the availability of large datasets and powerful GPUs
- LLMs are trained on large corpora of text data (e.g., books, articles, websites, etc.), and they learn to predict the next word in a sentence
- For code, they are trained on large repositories like GitHub and use Natural Language Processing (NLP) to understand the context and generate code snippets
- Which means that they are particularly good at writing Python or JavaScript, but they can also help with other languages
- For a very good introduction to LLMs, I strongly recommend this article by Stephen Wolfram
What are LLMs?
- So far, LLMs have made tremendous progress in many areas in a very short time
- They can generate text, code, music, art, and even new scientific discoveries
- As we all know, what is remarkable about LLMs is not only the quality of their output, but also the fact that they can be used by anyone, including those with no background in AI or programming
- Thus, LLMs are democratising programming like never before
- Remember when we talked about low and high level programming languages? LLMs are taking us to a new level of abstraction
- LLMs are the culmination of a long process of abstraction in computing, and they are changing the way we think about programming
How do LLMs work?
A very simplified explanation: Sorry mathematicians! 😅
- The data collected by AI firms are huge and include many languages and styles
- An overlooked part of the training process is the data cleaning and preprocessing: removing duplicates, correcting errors, and standardising formats
- Tokenisation is the process of breaking text into smaller units called tokens, and each token is assigned a unique ID
- For example, “
Chatbots are helpful in 2025!” =["Chat", "bots", "are", "help", "ful", "in", "2025", "!"]
- For example, “
- The main objective during training is for the model to predict the next token in a sequence based on the tokens that come before it
- This allows the model to learn language patterns without needing labeled data
How do LLMs work?
- Models use backpropagation to update their parameters based on the errors in their predictions
- Backpropagation is a gradient descent algorithm that adjusts the weights of the model to minimise the loss function
- They also use several methods to prevent overfitting, such as dropout
- Dropout refers to randomly “dropping out”, or omitting, units (both hidden and visible) during the training process of a neural network
- After initial training, the model can be fine-tuned on specific datasets to improve its performance in particular domains, such as programming
- Often done by poorly paid workers in developing countries
Why was GPT-3 so special?
- Transformers changed the field by allowing models to process text in parallel, which made them much faster than previous models
- The model consists of multiple layers, each performing specific functions to process and generate text
- GPT-3 was the first model to have 175 billion parameters, which made it the largest model at the time
- Another important feature was its zero-shot leaning capability, which allowed it to perform tasks without any training data
- GPT also uses multi-head attention to focus on different parts of the input text at the same time
- For example, it can focus on the subject and the verb of a sentence simultaneously
- Finally, the model also scales well with more data and more parameters
AI-Assisted Programming: Challenges
Hallucination
- Generative AI models can produce incorrect or misleading content
- This can be due to errors in the model, biases or incorrect information in the training data, or the limitations of the model architecture
- This makes it vital to check the output of these models and not take it at face value. For example, I asked Copilot (which is powered by OpenAI’s GPT-4) to solve a simple quadratic equation, and it very confidently gave me a very wrong answer 😅
- It provided the answers of \(\frac{1}{2}\) and \(\frac{-5}{4}\) when the correct answers were 0.804 and -1.55.
AI-Assisted Programming: Challenges
Bias
- AI models can amplify biases present in the training data
- For instance, I asked an AI to give me the names of famous scientists, and it came up with the following list:
- Albert Einstein
- Isaac Newton
- Charles Darwin
- Nikola Tesla
- Galileo Galilei
- Stephen Hawking
- Leonardo da Vinci
- Thomas Edison
- Can you spot the bias?
AI-Assisted Programming: Challenges
Bias
AI-Assisted Programming: Challenges
Easy to confuse
- AI models can be easily confused by small changes in the input
- AIs have an annoying tendency to be overpolite and agree with you even when you are wrong
- Thus, they can be easily manipulated (including by bad actors)
- For example, I asked Copilot what the westernmost point of Europe was, and contradicted its answer
AI-Assisted Programming: Challenges
Intellectual property
- AI models can generate code that is identical to existing code
- This raises questions about intellectual property and the ownership of the code
- This is an ongoing debate in the AI community, and it is not clear how it will be resolved
- But it does have some important implications for open source software and the sharing of code
AI-Assisted Programming: Challenges
Security
- AI models can be vulnerable to adversarial attacks and injection attacks
- As programmers use more AI-generated code, many blindly trust the output of these models and send the code into production
- In Security Weaknesses of Copilot Generated Code in GitHub, Yujia Fu et al. highlighted the security issues with GitHub Copilot.
- They evaluated 435 AI-generated code snippets from projects on GitHub, and 35.8% had security vulnerabilities
- …and this code is all going to the LLMs! 😅
How to prevent some of these issues
Better prompt engineering
- Prompt engineering is a new buzzword in the AI community
- It refers to the process of designing the input to an AI model to get the desired output
- It is, to a large extent, a mix of art and science that involves a lot of trial and error (at least in my experience!)
- The idea is to guide the model to produce the desired output by providing it with the right context and examples
- Prompt engineering will never be fully exact simply because models themselves are probabilistic
- But you can think of a prompt as having four main components:
Source: Tulio (2024).
- Context: The information you provide to the model
- Instructions: The task you want the model to perform
- Input of the model: The data you feed into the model
- Format: The structure of the prompt
Context
- It is a good idea to begin your prompt with a sentence or two that clearly defines the task you want the model to perform
- Creating a persona for the model can also help guide its output
- Prompt: You are an experienced software engineer specialising in debugging Python code. You are asked to write a function that takes a list of integers and returns the sum of the even numbers.
- Personal note: I’ve had good results by adopting multiple personas in the same prompt
- For example, I might ask the model to write a function as if it were a beginner, an intermediate, and an expert programmer and then compare the results
Input of the model
- The input of the model is the data you feed into it
- Use keywords that are relevant to the task you want the model to perform
- When it comes to coding, you can provide the model with code snippets that it can use as a reference
- Also pay attention to delimiters and separators, as they can help the model understand the structure of the input
- For example, you can use comments to provide additional information to the model
Format
- Finally, you can also be explicit about the format you want the output to be in
- For example, you can specify that you want the output to be in a specific programming language, or you can ask the model to provide you with a code snippet
- You can also ask the model to provide you with a list of steps to complete a task
- This can be useful if you are working on a complex project or with different programming languages
How to get started with GitHub Copilot
- We will look into Copilot in more detail in the next lecture, but I just want to make sure we all have it installed and ready to go 😉
- First, it is a good idea to have VS Code installed on your computer
- Then, you can install the GitHub Copilot extension from the VS Code marketplace
- You will need to sign in with your GitHub account to use Copilot
- Copilot works in the CLI as well, and you can install it using this guide. You need to install GitHub CLI first
- Take some time to install and play around with Copilot before the next lecture 😃
- Please let me know if you have any questions or issues!
Basic components
- There are two main components to GitHub Copilot:
- GitHub Copilot: This provides code suggestions and completions in the editor window
- GitHub Copilot Chat: This provides a chat interface to Copilot, allowing you to ask questions and get code suggestions. It can also interact with code in the editor window. This is by far the most powerful feature of Copilot
- Copilot not only offers code suggestions, but it can also help you with writing documentation, providing explantions for code you don’t know, and, more recently, retrieving code snippets and information from your whole workspace
- In my view, it is the most convenient AI-assistant for programming available today
- It is also very easy to use
Autocompletion
- If you start writing code in the editor, and Copilot will suggest completions
- You can accept these completions by pressing
TaborEnter - Alternatively, you can press
Ctrland the right arrow key to select just the next word - This can be helpful if some, but not all, of the suggestion is appropriate
- Sometimes, writing a comment will help guide Copilot to suggest relevant code to you
Source: In-Editor Prompting
- In an empty file/code cell, VS Code will display a greyed-out piece of text as follows:
- If you press
Ctrl + I, a text box will appear into which you can write a prompt - Copilot will suggest new code, or changes to existing code based on this prompt
Generate documentation
- You can ask Copilot to generate documentation for a piece of code by using the
/doccommand
Use /tests to generate tests
- Use
/testson a function or snippet, and Copilot Chat will generate relevant test cases
Using @websearch to get more specific results
- Copilot also has the
@websearchcommand that allows you to search the web for information - This can be useful if you are looking for information on a specific topic
- You have to install the Web Search for Copilot extension first (available for free), then create an API key with Tavily and paste it in the settings (Bing API will be discontinued soon)
- Open VS Code settings, search for
websearch.preferredEngine, and selectTavily - Tavily is a web search engine that provides an API for searching the web, and it is free to use for up to 1000 requests per month
Getting an API key from Tavily
GitHub Codespaces
GitHub Codespaces
GitHub Codespaces
GitHub Codespaces
GitHub Codespaces
