Artificial intelligence

"Artificial intelligence (AI) is intelligence - perceiving, synthesizing, and inferring information - demonstrated by machines, as opposed to intelligence displayed by non-human animals and humans. Example tasks in which this is done include speech recognition, computer vision, translation between (natural) languages, as well as other mappings of inputs."

"The effort to automate intellectual tasks normally performed humans."

Machine learning

"Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn' (...) It is seen as a part of artificial intelligence."

"Machine learning is a specific subfield of AI that aims at automatically developing programs (called models) purely from exposure to training data. This process of turning models data into a program is called learning."

Data mining

"Application of machine learning methods to large databases is called data mining. The analogy is that a large volume of earth and raw material is extracted from a mine, which when processed leads to a small amount of very precious material; similarly, in data mining, a large volume of data is processed to construct a simple model with valuable use, for example, having high predictive accuracy."

Deep learning

"Deep learning is the subset of machine learning methods based on neural networks with representation learning. The adjective "deep" refers to the use of multiple layers in the network."

Regression

• Predicts a continuous outcome
• Example: Predicting house prices, GDP growth, temperature

Classification

• Predicts a categorical outcome
• Example: Predicting whether a person will default on a loan, whether an email is spam, whether a patient has a disease

Accuracy

• Accuracy = $\frac{\text{Number of correct predictions}}{\text{Total number of predictions}} = \frac{TP + TN}{TP + TN + FP + FN}$
• Error rate: $1 - \text{Accuracy}$

Usefulness

• Accuracy is a simple and intuitive metric.
• But it can be misleading, especially in imbalanced datasets where the classes are not evenly represented.
• Example: In a dataset with 90% of class A and 10% of class B, a model that predicts all instances as class A will have an accuracy of 90%, but it will not be useful for predicting class B instances.

Precision

• Precision = $\frac{\text{Number of true positive predictions}}{\text{Number of positive predictions}} = \frac{TP}{TP + FP}$

Usefulness

• Precision focuses on the accuracy of positive predictions and is useful when the cost of false positives is high.

Precision

• Precision = $\frac{\text{Number of true positive predictions}}{\text{Number of positive predictions}} = \frac{TP}{TP + FP}$

Usefulness

• Precision focuses on the accuracy of positive predictions and is useful when the cost of false positives is high.

Recall (Sensitivity)

• Recall = $\frac{\text{Number of true positive predictions}}{\text{Number of true positives}} = \frac{TP}{TP + FN}$
• "True positive rate"

Usefulness

• Recall focuses on capturing all positive instances and is important when the cost of false negatives is high.
• Example: In a medical diagnosis, recall is important to ensure that all patients with a disease are correctly identified.
• The complementary measure is specificity (true negative rate; e.g. how many healthy people are identified as not having the condition)

Recall (Sensitivity)

• Recall = $\frac{\text{Number of true positive predictions}}{\text{Number of true positives}} = \frac{TP}{TP + FN}$
• "True positive rate"

Usefulness

• Recall focuses on capturing all positive instances and is important when the cost of false negatives is high.
• Example: In a medical diagnosis, recall is important to ensure that all patients with a disease are correctly identified.
• The complementary measure is specificity (true negative rate; e.g. how many healthy people are identified as not having the condition)

F1 score

• F1 score = $2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}} = \frac{2TP}{2TP + FP + FN}$
• F1 score is the harmonic mean of precision and recall.

Usefulness

• It provides a balance between precision and recall, especially when there is an imbalance between the classes.
• F1 score ranges from 0 to 1, where 1 indicates perfect precision and recall, and 0 indicates poor performance.

F1 score

• F1 score = $2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}} = \frac{2TP}{2TP + FP + FN}$
• F1 score is the harmonic mean of precision and recall.

Usefulness

• It provides a balance between precision and recall, especially when there is an imbalance between the classes.
• F1 score ranges from 0 to 1, where 1 indicates perfect precision and recall, and 0 indicates poor performance.

Background

• Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) is a decision support tool developed by Northpointe (now Equivant) used by U.S. courts to assess the likelihood of recidivism
• Produced several scales (Pretrial release risk, General recidivism, Violent recidivism) based on factors such as age, criminal history, and substance abuse
• The algorithm is proprietary and its inner workings are not public

Source Practitioner's Guide to COMPAS Core

The ProPublica and other investigations

• In 2016, ProPublica published an investigation showing that COMPAS was biased against African Americans
• Bias: The algorithm was more likely for African Americans to wrongly predict that defendants would re-offend.
• Accuracy: only 20% of people predicted to commit violent crimes actually went on to do so (in a later study estimated with 65%, still worse than a group of humans with little expertise)

Source ProPublica 2016

The ProPublica and other investigations

• In 2016, ProPublica published an investigation showing that COMPAS was biased against African Americans
• Bias: The algorithm was more likely for African Americans to wrongly predict that defendants would re-offend.
• Accuracy: only 20% of people predicted to commit violent crimes actually went on to do so (in a later study estimated with 65%, still worse than a group of humans with little expertise)

Source Dressel and Fair, 2018, Science Advances

1. Where do you see potential for AI in public policy-making?

2. Are there applications of AI in Georgian government that you are aware of?

3. What role does AI play in your personal (professional) life?