results.head()

• See a random sample of the data using sample

results.sample(5).sort_values("resultId")

• We can get the column names and types using dtypes

results.dtypes

resultId             int64
raceId               int64
driverId             int64
constructorId        int64
number              object
grid                 int64
position            object
positionText        object
positionOrder        int64
points             float64
laps                 int64
time                object
milliseconds        object
fastestLap          object
rank                object
fastestLapTime      object
fastestLapSpeed     object
statusId             int64
dtype: object

• We can also get the column names using columns

results.columns

Index(['resultId', 'raceId', 'driverId', 'constructorId', 'number', 'grid',
       'position', 'positionText', 'positionOrder', 'points', 'laps', 'time',
       'milliseconds', 'fastestLap', 'rank', 'fastestLapTime',
       'fastestLapSpeed', 'statusId'],
      dtype='object')

• How many rows does the dataset have?
• How many unique values are there for the columns:
  • resultId?
  • raceId?
  • driverId?
• Hint: Use the len() and the pd.nunique() functions (click on their names to see the documentation)
• Appendix 01

• You can split your code into multiple lines to make it easier to read
• You just need to wrap the code in parentheses ()

descriptives_multiline = (results["points"]
                          .describe())
descriptives_multiline

count    25840.000000
mean         1.877053
std          4.169849
min          0.000000
25%          0.000000
50%          0.000000
75%          2.000000
max         50.000000
Name: points, dtype: float64

• This is the same as writing

descriptives_singleline = results["points"].describe()
descriptives_singleline

count    25840.000000
mean         1.877053
std          4.169849
min          0.000000
25%          0.000000
50%          0.000000
75%          2.000000
max         50.000000
Name: points, dtype: float64

• The .agg() subfunction computes aggregate statistics
• The syntax is (column_name, function_name)
• The first argument is the column name
• The second argument is the function_name
• The command works with single quotations ‘…’ or double “…”

# The functions in quotes are pandas functions.  len is not in quotes because it is a Python function
# If you were to use 'len' (a string), pandas would look for a method named len in the DataFrame, 
# which does not exist.
results_agg = results.agg(mean_points = ('points','mean'),
                          sd_points =   ('points','std'),
                          min_points =  ('points','min'),
                          max_points =  ('points','max'),
                          count_obs   = ('points', len))

display(results_agg.round(2)) # round to 2 decimal places

• Split-apply-combine strategy again
• Split: Split the dataset into groups, here by driverId
• Apply: Apply a function to each group, here the mean, standard deviation, minimum, maximum, and count of points
• Combine: Combine the results into a single data frame
• The result is a data frame with the statistics for each driver

• Create a new dataset by chaining groups by constructorId (the team) then compute the average number of points
• Add a chain .sort_values(...,ascending = False) to sort by team points in descending order
• Appendix 02

• You can use different functions for different columns too!
• You should specify the column name and the function for each column in a dictionary
• If you want to add more than one function for a column, you can use a list with ['function1', 'function2']
• The syntax is df.agg({'column1': 'function1', 'column2': 'function2'})

results.agg({'laps': ['min', 'max', len]}).round(2) # one variable

results.agg({'points': 'mean',
             'positionOrder': 'max',
             'laps': ['min', 'max', len]}).round(2) # multiple variables

• Let’s see the first few rows of the merged data frame

results_merge.head()

• We can see that the new columns mean_points, sd_points, min_points, max_points, and count_obs were added to the data frame

results_merge.columns

Index(['resultId', 'raceId', 'driverId', 'constructorId', 'number', 'grid',
       'position', 'positionText', 'positionOrder', 'points', 'laps', 'time',
       'milliseconds', 'fastestLap', 'rank', 'fastestLapTime',
       'fastestLapSpeed', 'statusId', 'mean_points', 'sd_points', 'min_points',
       'max_points', 'count_obs'],
      dtype='object')

• Compute a scatter plot with points (y-axis) vs mean_points (x-axis)
• Note: This plots tells you how much a driver’s performance on individual races deviates from their overall average
• Appendix 03

• Merge the teamrace_agg data into results
• This time use the option: on = ["raceId","constructorId"]
• Appendix 04

• Let’s start by counting the number of rows in the dataset

len(results)

25840

• Now let’s count the number of unique values for the columns resultId, raceId, and driverId

results.resultId.nunique()

25840

results.raceId.nunique()

1079

results.driverId.nunique()

855

• You could also used value_counts() to get the same information

results.resultId.value_counts()

resultId
1        1
17224    1
17234    1
17233    1
17232    1
        ..
8611     1
8610     1
8609     1
8608     1
25845    1
Name: count, Length: 25840, dtype: int64

results.raceId.value_counts()

raceId
800    55
809    47
371    39
359    39
366    39
       ..
657    14
668    14
827    13
660    13
765    10
Name: count, Length: 1079, dtype: int64

results.driverId.value_counts()

driverId
4      358
8      352
22     326
1      310
18     309
      ... 
545      1
331      1
492      1
546      1
856      1
Name: count, Length: 855, dtype: int64

• This function returns the number of times each unique value appears in the column

Back to exercise

• Let’s start by grouping the data by constructorId and computing the average number of points

team_agg = (results.groupby("constructorId")
                   .agg(mean_points = ('points','mean')))

• Now let’s sort the data by mean_points in descending order
• We can use the sort_values() method

team_agg.sort_values("mean_points", ascending = False).round(2)

Back to exercise

plt.figure(figsize=(10,6))
plt.scatter(results_merge["mean_points"], results_merge["points"])
plt.xlabel("Mean Points")
plt.ylabel("Points")
plt.title("Points vs Mean Points")
plt.show()

• This plots tells you how much a driver’s performance on individual races deviates from their overall average

Back to exercise

• Let’s merge the teamrace_agg data into results
• This time use the option: on = ["raceId","constructorId"]
• Since we filtered the data before (raceId >= 500), we will see some missing cases in the merged data frame

results_merge = pd.merge(results,
                         teamrace_agg,
                         on = ["raceId","constructorId"],
                         how = "left")
results_merge.round(2)

• We can remove the missing cases using the dropna() method

results_merge = pd.merge(results,
                         teamrace_agg,
                         on = ["raceId","constructorId"],
                         how = "left")
results_merge.round(2).dropna()

Back to exercise