The Pandas DataFrame

Introduction

A Series holds one column of labelled data. Real data — like the STUDENT table from your SQL chapter — usually needs several columns together. That's exactly what a DataFrame is for.

Concept

A DataFrame is a two-dimensional, labelled table: rows and columns together, each column itself effectively a Series, all sharing one common row index. The most common way to build one directly in code is from a dictionary of lists — each key becomes a column name, each list becomes that column's data.

pd.DataFrame(data)

Worked Example

import pandas as pd

data = {
    "Name": ["Aisha", "Rahul", "Meera"],
    "Marks": [91, 78, 85]
}
students = pd.DataFrame(data)
print(students)

data is a dictionary with two keys, "Name" and "Marks", each mapping to a list of three values. pd.DataFrame(data) turns it into a table: two columns, three rows, with a default integer row index (0, 1, 2) — printed as a neat, aligned grid, not just a raw list of values.

Note: Every list inside the dictionary must be the same length — Pandas builds one row per position across all the columns together, so a mismatched length raises an error.

Explanation

A DataFrame is really a collection of Series sharing one index — students["Name"] on its own is exactly a Series, the same structure the last chapter covered in full, which is exactly why every Series operation (indexing, filtering, sorting) still works once you're looking at just one DataFrame column.

Summary

A DataFrame is a two-dimensional, labelled table — rows and columns together. pd.DataFrame(data), given a dictionary of equal-length lists, builds one with each key as a column and each list as that column's data.

import sys

# SMART CHECK: Only install if missing
if "pandas" not in sys.modules:
    print("Toolkit: Initializing Python Environment...")
    import pyodide_js
    await pyodide_js.loadPackage("micropip")
    import micropip
    await micropip.install(['pandas', 'numpy'])
    print("Toolkit: Pandas & NumPy installed. You are ready to code!")
else:
    print("Toolkit: System is already ready. Proceed to the lesson.")

import pandas as pd

# Data: A list of students [Name, Age]
data = [
    ['Amit', 18],
    ['Sumit', 17],
    ['Ravi', 18]
]

# Create DataFrame (We specify column names manually)
df_list = pd.DataFrame(data, columns=['Name', 'Age'])

print("DataFrame from List of Lists:")
print(df_list)

import pandas as pd

# Data: Dictionary
data_dict = {
    'Name': ['Amit', 'Sumit', 'Ravi'],
    'Marks': [95, 85, 92],
    'City': ['Delhi', 'Mumbai', 'Pune']
}

# Create DataFrame
df_dict = pd.DataFrame(data_dict)

print("DataFrame from Dictionary:")
print(df_dict)

Introduction

Just as a Series carries properties describing it, so does a DataFrame — this topic covers the ones you'll use constantly.

Concept

• shape — a tuple, (rows, columns).
• columns — the column labels.
• index — the row labels.
• dtypes — the data type of each column individually (unlike a Series, which has one shared dtype for the whole thing).

Worked Example

import pandas as pd

data = {"Name": ["Aisha", "Rahul", "Meera"], "Marks": [91, 78, 85]}
students = pd.DataFrame(data)

print(students.shape)
print(students.columns)
print(students.dtypes)

students.shape is (3, 2) — three rows, two columns. students.columns prints the two column labels, Name and Marks. students.dtypes prints one type per column — Name as an object/text type, Marks as an integer type — since a DataFrame's columns don't have to share one single type the way a Series does.

Note: shape is (rows, columns) — in that exact order — matching the same convention ndim/shape used for a NumPy array in Class 11.

Explanation

dtypes (plural) versus a Series' single dtype is the clearest sign a DataFrame is really several Series stacked together, each with its own independent type — Name being text and Marks being numeric in the very same table is exactly what a Series alone could never represent.

Summary

A DataFrame's shape gives (rows, columns); columns and index give the column and row labels; dtypes gives one data type per column — reflecting that a DataFrame is really several Series, of possibly different types, sharing one row index.

import sys

# SMART CHECK: Only install if missing
if "pandas" not in sys.modules:
    print("Toolkit: Initializing Python Environment...")
    import pyodide_js
    await pyodide_js.loadPackage("micropip")
    import micropip
    await micropip.install(['pandas', 'numpy'])
    print("Toolkit: Pandas & NumPy installed. You are ready to code!")
else:
    print("Toolkit: System is already ready. Proceed to the lesson.")

import pandas as pd

# 1. Create Data
data = {
    'Name': ['Amit', 'Sumit', 'Ravi'],
    'Math': [90, 80, 70],
    'Eng': [85, 88, 92]
}
df = pd.DataFrame(data)

print("--- The DataFrame ---")
print(df)

# 2. Check Dimensions
print("\n--- Shape (Rows, Cols) ---")
print(df.shape)  # Expect (3, 3)

# 3. Check Headers
print("\n--- Column Headers ---")
print(df.columns)

print("--- Original ---")
print(df)

print("\n--- Transposed (.T) ---")
print(df.T)

Introduction

Beyond individual attributes, Pandas provides two methods that summarise a whole DataFrame at once — the first thing worth running on any new, unfamiliar dataset.

Concept

• info() — prints a structural summary: column names, how many non-empty values each has, and each column's data type.
• describe() — prints statistical summary measures (count, mean, minimum, maximum, and more) for every numeric column.

Worked Example

import pandas as pd

data = {"Name": ["Aisha", "Rahul", "Meera"], "Marks": [91, 78, 85]}
students = pd.DataFrame(data)

students.info()
print(students.describe())

students.info() prints each column's name, how many non-null entries it has (3 for both, here), and its dtype — a quick structural health check. students.describe() produces statistics for Marks only, since Name is text, not numeric — including its count (3), mean, minimum, and maximum values.

Note: describe() silently skips non-numeric columns by default — Name never appears in its output here, which is expected behaviour, not a missing-data problem.

Explanation

info() and describe() answer two different first questions about new data: info() answers "is anything obviously missing or the wrong type?", while describe() answers "what do the actual numbers in this data look like?" — running both is the standard first step before doing anything else with a real dataset.

Summary

info() gives a structural summary — column names, non-null counts, and dtypes. describe() gives statistical summary measures for every numeric column, silently skipping text columns — both are typically the first thing run on an unfamiliar DataFrame.

import sys

# SMART CHECK: Only install if missing
if "pandas" not in sys.modules:
    print("Toolkit: Initializing Python Environment...")
    import pyodide_js
    await pyodide_js.loadPackage("micropip")
    import micropip
    await micropip.install(['pandas', 'numpy'])
    print("Toolkit: Pandas & NumPy installed. You are ready to code!")
else:
    print("Toolkit: System is already ready. Proceed to the lesson.")

import pandas as pd

# Data with a missing value (None)
data = {
    'Product': ['Apple', 'Banana', 'Cherry'],
    'Price': [100, 50, None],  # One price is missing
    'Stock': [10, 20, 30]
}
df = pd.DataFrame(data)

print("--- Data Health Check ---")
df.info()

# Look at the output:
# 'Price' will show '2 non-null' (meaning 1 is missing/null).

import pandas as pd

# Marks of 5 students
data = {
    'Math': [90, 80, 85, 95, 100],
    'Science': [70, 75, 80, 65, 60]
}
df = pd.DataFrame(data)

print("--- Statistical Summary ---")
print(df.describe())

# Look for 'mean' (Average), 'min' (Lowest), and 'max' (Highest).

Introduction

You've already used .loc[] and .iloc[] on a Series. This topic covers the same two selectors on a full DataFrame, where they also choose which columns to include.

Concept

• df[column] — selects one column, returned as a Series.
• df.loc[row_label, column_label] — selects by label.
• df.iloc[row_position, column_position] — selects by integer position.

Worked Example

import pandas as pd

data = {"Name": ["Aisha", "Rahul", "Meera"], "Marks": [91, 78, 85]}
students = pd.DataFrame(data)

print(students["Name"])
print(students.loc[1, "Marks"])
print(students.iloc[1, 1])

students["Name"] returns the entire Name column as a Series. students.loc[1, "Marks"] selects row label 1, column label "Marks" — Rahul's mark, 78. students.iloc[1, 1] selects row position 1, column position 1 (the second column, zero-indexed) — the same 78, reached by position instead of label.

Note: With the default integer index, row labels and row positions happen to be identical numbers (0, 1, 2) — which is exactly why .loc and .iloc can look interchangeable here, even though they mean genuinely different things. Once a DataFrame gets a custom index (covered later in this chapter), that coincidence disappears.

Explanation

.loc/.iloc on a DataFrame extend the exact same rule from the Series chapter — label-based versus position-based — just now with a second dimension (columns) to specify alongside the row.

Summary

df[column] selects one column as a Series. df.loc[row, col] selects by label; df.iloc[row, col] selects by integer position — with the default index, these coincide numerically, but they select by fundamentally different criteria.

import sys

# SMART CHECK: Only install if missing
if "pandas" not in sys.modules:
    print(" Toolkit: Initializing Python Environment...")
    import pyodide_js
    await pyodide_js.loadPackage("micropip")
    import micropip
    await micropip.install(['pandas', 'numpy'])
    print(" Toolkit: Pandas & NumPy installed. You are ready to code!")
else:
    print(" Toolkit: System is already ready. Proceed to the lesson.")

import pandas as pd

data = {
    'Math': [90, 80, 70],
    'Eng': [85, 88, 92]
}
# Custom Index (Names)
df = pd.DataFrame(data, index=['Amit', 'Sumit', 'Ravi'])

print("--- Data ---")
print(df)

# 1. Select Row 'Sumit' (All columns)
print("\n--- Row: Sumit ---")
print(df.loc['Sumit'])

# 2. Select 'Sumit', Column 'Math'
print("\n--- Sumit's Math Score ---")
print(df.loc['Sumit', 'Math'])

# 1. Select First Row (Index 0)
print("--- First Row (Index 0) ---")
print(df.iloc[0])

# 2. Select First Row, Second Column (0, 1)
print("\n--- Cell at (0, 1) ---")
print(df.iloc[0, 1])

# LOC Slicing: 'Amit' to 'Sumit'
# Result: Amit AND Sumit are included.
print("--- LOC Slice (Inclusive) ---")
print(df.loc['Amit':'Sumit'])

# ILOC Slicing: 0 to 2
# Result: Index 0 and 1. Index 2 (Ravi) is NOT included.
print("\n--- ILOC Slice (Exclusive) ---")
print(df.iloc[0:2])

Introduction

A DataFrame's structure isn't fixed once created — this topic covers adding a new column, and removing an existing one.

Concept

• df["new_column"] = value_or_list — adds a new column. A single value fills every row; a list assigns one value per row.
• df.drop("column", axis=1) — returns a new DataFrame with that column removed. axis=1 specifically means "a column," not a row.

Worked Example

import pandas as pd

data = {"Name": ["Aisha", "Rahul", "Meera"], "Marks": [91, 78, 85]}
students = pd.DataFrame(data)

students["Grade"] = ["A", "B", "B"]
print(students)

reduced = students.drop("Grade", axis=1)
print(reduced)

students["Grade"] = ["A", "B", "B"] adds a third column, one value per existing row — students now has Name, Marks, and Grade. students.drop("Grade", axis=1) returns a new DataFrame, reduced, with Grade removed — students itself is left completely unchanged by drop(), unless its result is explicitly reassigned back.

Note: axis=1 means "operate along columns"; axis=0 (the default, if axis is left out) means "operate along rows." Forgetting axis=1 when dropping a column is a common mistake — without it, drop() tries to remove a row with that name instead.

Explanation

Adding a column with df["new"] = [...] mirrors exactly how a new key gets added to a dictionary — the same "assign to a name that doesn't exist yet" pattern from the Dictionary chapter, just naming a column instead of a key.

Summary

df["column"] = value_or_list adds a new column. df.drop("column", axis=1) returns a new DataFrame with that column removed, leaving the original untouched — axis=1 specifically targets a column, not a row.

import sys # SMART CHECK: Only install if missing if "pandas" not in sys.modules: print("Toolkit: Initializing Python Environment...") import pyodide_js await pyodide_js.loadPackage("micropip") import micropip await micropip.install(['pandas', 'numpy']) print("Toolkit: Pandas & NumPy installed. You are ready to code!") else: print("Toolkit: System is already ready. Proceed to the lesson.")

import pandas as pd

data = {
    'Math': [90, 80, 70],
    'Eng': [85, 88, 92]
}
df = pd.DataFrame(data, index=['Amit', 'Sumit', 'Ravi'])

print("--- Original ---")
print(df)

# 1. Scalar Assignment (Same value for all)
df['City'] = 'Delhi'

# 2. Vector Calculation (Math + Eng)
df['Total'] = df['Math'] + df['Eng']

print("\n--- After Adding Columns ---")
print(df)

# Create the DataFrame again (for safety in this block)
data = {'Math': [90, 80], 'Eng': [85, 88], 'City': ['Delhi', 'Delhi']}
df = pd.DataFrame(data)

print("--- Before Deleting ---")
print(df)

# Drop 'City'. axis=1 is COMPULSORY for columns.
# Note: We must save it back to 'df' (df = ...)
df = df.drop('City', axis=1)

print("\n--- After Deleting ---")
print(df)

Introduction

This closing topic of the chapter covers renaming a column, and changing what a DataFrame actually uses as its row index.

Concept

• df.rename(columns={"old": "new"}) — returns a new DataFrame with the given column(s) renamed.
• df.set_index("column") — makes an existing column the new row index, replacing the default integer one.
• df.reset_index() — reverses that: restores a plain default integer index, turning the old index back into an ordinary column.

Worked Example

import pandas as pd

data = {"Name": ["Aisha", "Rahul", "Meera"], "Marks": [91, 78, 85]}
students = pd.DataFrame(data)

renamed = students.rename(columns={"Marks": "Score"})
print(renamed)

by_name = students.set_index("Name")
print(by_name)
print(by_name.loc["Rahul"])

students.rename(columns={"Marks": "Score"}) returns a new DataFrame with that one column renamed; students itself is unaffected. students.set_index("Name") makes Name the row index instead of the default 0, 1, 2 — so by_name.loc["Rahul"] can now select a row directly by student name, exactly the same .loc you already know, now working against a meaningful custom label.

Note: Both rename() and set_index() return a new DataFrame by default, the same "doesn't change the original unless reassigned" pattern every method in this chapter has followed so far.

Explanation

set_index() is exactly what makes .loc's label-based selection genuinely useful in practice — with only the default integer index, .loc and .iloc behave almost identically; a meaningful custom index (a name, a roll number) is what makes label-based selection actually different from, and often more convenient than, position-based selection.

Summary

rename(columns={...}) renames columns; set_index("column") makes an existing column the row index; reset_index() restores the default integer index — all three return a new DataFrame by default, closing out this chapter's coverage of building and reshaping a DataFrame.

import sys

# SMART CHECK: Only install if missing
if "pandas" not in sys.modules:
    print("Toolkit: Initializing Python Environment...")
    import pyodide_js
    await pyodide_js.loadPackage("micropip")
    import micropip
    await micropip.install(['pandas', 'numpy'])
    print("Toolkit: Pandas & NumPy installed. You are ready to code!")
else:
    print("Toolkit: System is already ready. Proceed to the lesson.")

import pandas as pd

data = {
    'St_Name': ['Amit', 'Sumit'],
    'M': [90, 80]
}
df = pd.DataFrame(data)

print("--- Original Names ---")
print(df)

# Rename Columns using a Dictionary
df = df.rename(columns={'St_Name': 'Name', 'M': 'Math'})

print("\n--- Renamed Columns ---")
print(df)

data = {'Name': ['Amit', 'Sumit']}
df = pd.DataFrame(data, index=['A', 'B'])

print("--- Custom Index ---")
print(df)

# Reset Index (drop=True removes 'A' and 'B' completely)
df = df.reset_index(drop=True)

print("\n--- Reset Index (0, 1...) ---")
print(df)