7
Lesson 7: File Handling and Data Processing
Reading and writing data to files
Learning Objectives
By the end of this lesson, students will:
- Read and write text files in Python
- Process CSV files for data analysis
- Handle file errors gracefully
- Create data logging systems
Skills Developed:
- Data persistence concepts
- Error handling and debugging
- Real-world data processing
Lesson Content
1 Basic File Operations (20 minutes)
File Handling Basics:
- • Files store data permanently (persistence)
- • Must open files before reading/writing
- • Always close files when done
- • Different modes: read ('r'), write ('w'), append ('a')
# Writing to a file
robot_log = open("robot_status.txt", "w")
robot_log.write("Robot started successfully\\n")
robot_log.write("Battery level: 100%\\n")
robot_log.write("All systems operational\\n")
robot_log.close()
# Reading from a file
robot_log = open("robot_status.txt", "r")
content = robot_log.read()
print(content)
robot_log.close()
# Better way - using 'with' statement
with open("robot_status.txt", "r") as file:
content = file.read()
print(content)
# File automatically closes!
Instructor Notes:
Emphasize the importance of closing files. Show what happens if you forget. Demonstrate the 'with' statement as best practice. Relate to robot data logging.
2 Processing Files Line by Line (15 minutes)
Line-by-Line Processing:
- • More memory efficient for large files
- • Process data as you read it
- • Use readlines() or iterate directly
- • Perfect for sensor data logs
# Create sample sensor data file
sensor_data = [
"2024-01-15,12:00,25.5,60,1013",
"2024-01-15,12:01,25.7,58,1012",
"2024-01-15,12:02,26.1,55,1011"
]
with open("sensor_log.txt", "w") as file:
for line in sensor_data:
file.write(line + "\\n")
# Read and process line by line
temperatures = []
with open("sensor_log.txt", "r") as file:
for line in file:
data = line.strip().split(",")
temp = float(data[2])
temperatures.append(temp)
print(f"Temperature: {temp}°C")
avg_temp = sum(temperatures) / len(temperatures)
print(f"Average temperature: {avg_temp:.1f}°C")
3 CSV Files and Data Analysis (20 minutes)
CSV (Comma-Separated Values):
- • Standard format for tabular data
- • Each line is a record, commas separate fields
- • First line often contains headers
- • Perfect for robot sensor logs and mission data
# Create robot mission log CSV
import csv
mission_data = [
["Time", "X", "Y", "Battery", "Status"],
["10:00", 0, 0, 100, "Started"],
["10:05", 5, 2, 95, "Moving"],
["10:10", 10, 5, 90, "Scanning"],
["10:15", 15, 8, 85, "Completed"]
]
# Write CSV file
with open("mission_log.csv", "w", newline="") as file:
writer = csv.writer(file)
writer.writerows(mission_data)
# Read and analyze CSV
with open("mission_log.csv", "r") as file:
reader = csv.reader(file)
header = next(reader) # Skip header
print(f"Mission Log: {header}")
total_distance = 0
prev_x, prev_y = 0, 0
for row in reader:
time, x, y, battery, status = row
x, y = int(x), int(y)
distance = ((x-prev_x)**2 + (y-prev_y)**2)**0.5
total_distance += distance
print(f"{time}: ({x},{y}) Battery: {battery}%")
prev_x, prev_y = x, y
print(f"Total distance: {total_distance:.1f} units")
4 Error Handling (10 minutes)
Common File Errors:
- • FileNotFoundError: File doesn't exist
- • PermissionError: Can't access file
- • IOError: General input/output problems
- • Use try-except blocks for graceful handling
# Safe file reading with error handling
def read_robot_config(filename):
try:
with open(filename, "r") as file:
config = file.read()
print("Configuration loaded successfully")
return config
except FileNotFoundError:
print(f"Error: {filename} not found!")
print("Using default configuration...")
return "default_config"
except PermissionError:
print("Error: Permission denied!")
return None
except Exception as e:
print(f"Unexpected error: {e}")
return None
# Test the function
config = read_robot_config("robot_config.txt")
if config:
print("Robot ready to start!")
Hands-On Activity (25 minutes)
Project: Robot Data Logger
Students will create a comprehensive data logging system for robot operations.
Requirements:
- • Create functions to log robot status to files
- • Read and analyze historical log data
- • Generate CSV reports with mission statistics
- • Include error handling for file operations
- • Create a configuration file system
- • Implement data backup and recovery
🤖📁 How to Code This Project (Choose Your Platform):
📱 Pythonista 3:
- Open Pythonista 3
- Tap "+" → "Empty Script"
- Name it "data_logger.py"
- Type the code below
- Tap ▶️ to run
- Check created files
Tip: Files save to Documents folder
🤖 Pydroid 3:
- Open Pydroid 3
- Tap "+" → "Python file"
- Name it "data_logger.py"
- Type the code below
- Tap ▶️ yellow play button
- Check created files
Tip: Use file manager to view logs
💻 PC/Mac:
- Open IDLE or editor
- Create new file
- Save as "data_logger.py"
- Type the code below
- Press F5 to run
- Check created files
Tip: Files save to script directory
# Robot Data Logger System
import csv
from datetime import datetime
def log_robot_status(battery, position, status, filename="robot_log.txt"):
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
log_entry = f"{timestamp},{battery},{position[0]},{position[1]},{status}\n"
try:
with open(filename, "a") as file:
file.write(log_entry)
print(f"Logged: {status} at {position}")
except Exception as e:
print(f"Logging failed: {e}")
def analyze_mission_data(filename="robot_log.txt"):
try:
with open(filename, "r") as file:
lines = file.readlines()
print(f"\n📊 Mission Analysis ({len(lines)} entries):")
battery_levels = []
for line in lines:
parts = line.strip().split(",")
battery = int(parts[1])
battery_levels.append(battery)
if battery_levels:
avg_battery = sum(battery_levels) / len(battery_levels)
print(f"Average battery: {avg_battery:.1f}%")
print(f"Min battery: {min(battery_levels)}%")
print(f"Max battery: {max(battery_levels)}%")
except FileNotFoundError:
print("❌ No log file found. Start logging first!")
# Simulate robot mission
print("🤖 Starting Robot Data Logger")
log_robot_status(100, (0, 0), "Mission started")
log_robot_status(95, (5, 2), "Moving to waypoint 1")
log_robot_status(90, (10, 5), "Scanning area")
log_robot_status(85, (15, 8), "Mission completed")
analyze_mission_data()
Assessment & Homework
Quick Check (In Class):
- • Can student read and write files safely?
- • Do they understand CSV format and processing?
- • Are they implementing error handling?
- • Can they process data from files effectively?
Homework Assignment:
Weather Station Data Processor
Create a program that processes weather station data from CSV files. Include functions to calculate daily averages, identify extreme values, generate monthly reports, and handle missing or corrupted data gracefully.