Lecture 19 - Cloud Computing II
| On-premise | IaaS | PaaS | SaaS |
|---|---|---|---|
| Application | Application | Application | Application |
| Middleware | Middleware | Middleware | Middleware |
| OS | OS | OS | OS |
| Virtualisation | Virtualisation | Virtualisation | Virtualisation |
| Servers | Servers | Servers | Servers |
| Networking | Networking | Networking | Networking |
User manages
Provider manages
apt update, apt upgrade, apt installscp (secure copy) and chmod (change mode)Source: DataCamp
ls, cd, pwd, etc)sudo to run commands as root (superuser)
apt (Advanced Package Tool)apt update refreshes the package listapt install installs a packagesudo apt python3 installs Python 3sudo apt install python3-pip installs pipchmod 400cp and ~ as the destinationsudo to run the commands, including chmod, scp, and ssh (use the password you set when installing WSL)Ubuntu on Windows (WSL)
Name and tags. For example, name your instance qtm350Ubuntu Server 24.04 LTS as the OS image and 64-bit (x86) as the architecturet2.micro or t1.micro, both of which are free tier eligibleCreate a new key pair and give it a name (e.g. qtm350)ED25519 and .pem as the file formatCreate Key Pair and save it to a secure locationNetwork settings, you may check Allow HTTPs traffic from the internet and Allow HTTP traffic from the internet so that you can access the web servers hosted on your EC2 VMRoot volume under Configure storagegp3 is the default volume type, and it works fine for most use casesio2 is the fastest and most expensive volume type, usually used for high-performance databases (which require milisecond latency)sc1 or st1 for cost savingsLaunch instance! 🚀Launch, you will be taken to the Instances pageConnect to instance to see the instructionsSSH client to see the command to connect to your instancechmod 400 "name-of-your-key.pem" and the Example commands provided by AWS to connect to your instanceInstances link on the left to see the details of your instanceActions and then Upload file)sudo apt update to refresh the package listsudo apt get upgrade to install the latest updates
-y flag to automatically answer yes to all promptssudo apt installsudo apt install python3sudo apt install python3-pipsudo apt update && sudo apt upgrade && sudo apt install python3 && sudo apt install -y python3-pip
scp (secure copy)scp is a command-line tool that allows you to copy files securelyscp -i "name-of-your-key.pem" file-to-copy ubuntu@public-ip:/path/to/destinationscp -i "name-of-your-key.pem" ubuntu@public-ip:/path/to/file-to-copy /path/to/destinationscp to copy files between two remote serversecho "print('Hello, QTM350!')" > hello.py
scp -i qtm350.pem hello.py ubuntu@XXXXXX.compute-1.amazonaws.com:~ XXXXXX with your public IP and ~ with the path to your home directoryls) and run (python3) the file on your instance
jupytersudo apt install python3 python3-pip jupytersource ~/.profilewhich python3, which pip3, and which jupyter8000
ssh -i "<your-key>.pem" ubuntu@<public_IPv4_DNS_address_of_your_EC2_instance> -L 8000:localhost:8888jupyter notebookhttp://localhost:8000print('Hello, QTM350!') (or any other code you like!)sudo apt install python3-numpy python3-pandas python3-matplotlib python3-seaborn# weather_data.py
import pandas as pd
import numpy as np
import datetime
# Set seed for reproducibility
np.random.seed(42)
# Generate dates for the past 30 days
dates = pd.date_range(end=datetime.datetime.now(), periods=30).tolist()
dates = [d.strftime('%Y-%m-%d') for d in dates]
# Generate temperature data with some randomness
temp_high = np.random.normal(75, 8, 30)
temp_low = temp_high - np.random.uniform(10, 20, 30)
precipitation = np.random.exponential(0.5, 30)
humidity = np.random.normal(65, 10, 30)
# Create a structured dataset
weather_data = pd.DataFrame({
'date': dates,
'temp_high': temp_high,
'temp_low': temp_low,
'precipitation': precipitation,
'humidity': humidity
})
# Save to a text file
with open('weather_data.txt', 'w') as f:
f.write("# Weather data for the past 30 days\n")
f.write(weather_data.to_string(index=False))
print("Weather data saved to weather_data.txt")python3 weather_data.pyweather_data.txt with 30 days of weather datascp -i <your-key>.pem weather_data.txt ubuntu@<your-instance-ip>:~/# weather_analysis.py
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from io import StringIO # Add proper import for StringIO
# Read the weather data
with open('weather_data.txt', 'r') as f:
lines = f.readlines()
# Skip the header comment
data_str = ''.join(lines[1:])
df = pd.read_csv(StringIO(data_str), sep=r'\s+') # Fix StringIO import and use raw string for regex
# Print basic statistics
print("Weather Data Analysis:")
print("=====================")
print(f"Number of days: {len(df)}")
print(f"Average high temperature: {df['temp_high'].mean():.1f}°F")
print(f"Average low temperature: {df['temp_low'].mean():.1f}°F")
print(f"Maximum temperature: {df['temp_high'].max():.1f}°F on {df.loc[df['temp_high'].idxmax(), 'date']}")
print(f"Minimum temperature: {df['temp_low'].min():.1f}°F on {df.loc[df['temp_low'].idxmin(), 'date']}")
print(f"Days with precipitation > 1 inch: {len(df[df['precipitation'] > 1])}")
# Create a visualisation
plt.figure(figsize=(12, 6))
sns.set_style("whitegrid")
# Plot temperature range
plt.fill_between(df['date'], df['temp_low'], df['temp_high'], alpha=0.3, color='skyblue')
plt.plot(df['date'], df['temp_high'], marker='o', color='red', label='High Temp')
plt.plot(df['date'], df['temp_low'], marker='o', color='blue', label='Low Temp')
# Add precipitation as bars on a secondary axis
ax2 = plt.twinx()
ax2.bar(df['date'], df['precipitation'], alpha=0.3, color='navy', width=0.5, label='Precipitation')
ax2.set_ylabel('Precipitation (inches)', color='navy')
ax2.tick_params(axis='y', labelcolor='navy')
# Formatting
plt.title('30-Day Weather Report: Temperature Range and Precipitation', fontsize=16)
plt.xticks(rotation=45, ha='right')
plt.ylabel('Temperature (°F)')
plt.legend(loc='upper left')
plt.tight_layout()
# Save the figure
plt.savefig('weather_analysis.png')
print("Analysis complete. Results saved to 'weather_analysis.png'")scp -i <your-key>.pem <file> ubuntu@<your-instance-ip>:~/ssh -i <your-key>.pem ubuntu@<your-instance-ip> -L 8000:localhost:8888jupyter notebookhttp://localhost:8000) and run your analysisscp -i <your-key>.pem ubuntu@<your-instance-ip>:~/weather_analysis.png ./When you run the activity, you should get a weather analysis graph similar to this one:
You’ve just completed a full data analysis workflow in the cloud 🎉
This workflow is similar to how data scientists use cloud resources for larger datasets and more complex analyses!