pandas — Powerful Data Analysis and Manipulation for Python

What it is

pandas is the foundational data analysis library for Python. It provides two primary data structures: DataFrame (2D labeled table) and Series (1D labeled array). With pandas you can read data from CSV, Excel, SQL, JSON, and Parquet files, then filter, group, join, pivot, and reshape it using an expressive API.

pandas is for anyone working with structured data in Python: data analysts, data scientists, backend engineers processing logs, and researchers analyzing experimental results. If your data fits in memory and has rows and columns, pandas is the standard tool.

How it saves time or tokens

This workflow provides ready-to-run pandas snippets for common data tasks. Instead of searching documentation for the right method signature, you get copy-paste code for reading files, filtering rows, grouping aggregations, and merging datasets. Each snippet is a self-contained operation you can adapt to your data.

How to use

1. Install pandas:

pip install pandas

1. Read your data into a DataFrame:

import pandas as pd

# From CSV
df = pd.read_csv('data.csv')

# From Excel
df = pd.read_excel('report.xlsx', sheet_name='Sheet1')

# From SQL
from sqlalchemy import create_engine
engine = create_engine('sqlite:///app.db')
df = pd.read_sql('SELECT * FROM users', engine)

1. Explore and manipulate:

# Basic exploration
df.shape          # (rows, cols)
df.dtypes         # column types
df.describe()     # summary statistics

# Filter rows
active = df[df['status'] == 'active']

# Group and aggregate
by_country = df.groupby('country')['revenue'].sum().sort_values(ascending=False)

Example

import pandas as pd

# Load sales data
df = pd.read_csv('sales.csv', parse_dates=['date'])

# Monthly revenue by product category
monthly = (
    df.assign(month=df['date'].dt.to_period('M'))
    .groupby(['month', 'category'])['amount']
    .sum()
    .unstack(fill_value=0)
)

# Top 5 customers by total spend
top_customers = (
    df.groupby('customer_id')['amount']
    .sum()
    .nlargest(5)
)

print(monthly)
print(top_customers)

Related on TokRepo

• AI tools for research -- Data analysis tools for research workflows
• Automation tools -- Automate data processing pipelines

Common pitfalls

• Using iterrows() for row-by-row processing is slow. Prefer vectorized operations like df['col'].apply() or boolean indexing for better performance.
• Chained assignment (df[df['x'] > 0]['y'] = 1) triggers a SettingWithCopyWarning and may not modify the original DataFrame. Use df.loc[df['x'] > 0, 'y'] = 1 instead.
• Loading large CSV files without specifying dtypes wastes memory. Use the dtype parameter or read in chunks with chunksize for files that approach your RAM limit.