Pd The Ultimate Pandas Cheat Sheet

Pandas hands you two structures — Series (1D) and DataFrame (2D) — that feel like a spreadsheet or a SQL table, except you can script every step.

Here's the itch it scratches: plain Python lists and dicts get slow and awkward once you're past a few thousand rows, and raw NumPy arrays, while fast, only hold one dtype and don't know what their own columns are called. Pandas layers labeled axes, mixed per-column dtypes, and real missing-data handling on top of NumPy's speed, so you get the best of both worlds.

This is why it ended up everywhere — ETL jobs, reporting, feature prep for ML models, log crunching, financial time series, and the kind of "let me just poke at this dataset for five minutes" exploration that turns into an afternoon.

        Excel/CSV/SQL/API/Parquet
                 |
                 v
        +-----------------+
        |   pandas I/O    |  read_csv, read_sql, read_parquet...
        +-----------------+
                 |
                 v
        +-----------------+      +----------------+
        |   DataFrame     | <--> |  NumPy ndarray | (dtype-homogeneous blocks)
        | (labeled table) |      +----------------+
        +-----------------+
                 |
        clean / transform / group / merge
                 |
                 v
        to_csv / to_sql / plot / ML pipeline

Note: Pandas is mostly single-threaded and keeps everything in RAM — that's the wall you'll eventually hit, and it's worth knowing where it is before you're debugging a MemoryError at 2am.

pip install pandas
# or
conda install pandas

# optional speed/IO extras
pip install pandas[performance,excel,parquet]

import pandas as pd
import numpy as np

print(pd.__version__)

One thing that trips up almost everyone early on: pandas will silently truncate wide or tall output in the console, which makes it look like columns are missing when they're not. Fix that once, up front:

pd.set_option('display.max_rows', 100)
pd.set_option('display.max_columns', 50)
pd.set_option('display.width', 120)
pd.set_option('display.float_format', '{:.2f}'.format)

# scoped, temporary options (auto-resets when the block exits)
with pd.option_context('display.max_rows', 10):
    print(df)

Pro Tip: pd.reset_option('all') puts every display setting back to default, and pd.describe_option('display') will show you the full list along with each one's current value — handy when you inherit a notebook that's already been fiddled with.

s = pd.Series([10, 20, 30], index=['a', 'b', 'c'], name='score')

df = pd.DataFrame({
    'name': ['Ada', 'Linus'],
    'score': [92, 88]
})

idx = df.index          # RangeIndex(start=0, stop=2, step=1)
cols = df.columns       # Index(['name', 'score'])

The Index is the quiet workhorse here — it's immutable, and it's what makes label alignment fast. When you write s1 + s2, pandas isn't adding position 0 to position 0; it's matching by label first, which is exactly the behavior that saves you when two datasets are sorted differently but is also exactly the behavior that bites you when you expected positional math instead.

Watch out: A duplicate-valued Index is technically legal, but it turns .loc lookups into surprise multi-row returns and can quietly change the shape of a join downstream. If something looks off after a merge, df.index.is_unique is usually the first thing worth checking.

df['grade'] = df['grade'].astype('category')                 # Categorical
df['ts'] = pd.to_datetime(df['ts'])                          # datetime64[ns]
df['delta'] = pd.to_timedelta(df['delta'])                    # timedelta64[ns]
df['age'] = df['age'].astype('Int64')                         # nullable integer (capital I)
df['name'] = df['name'].astype('string')                      # nullable string dtype
p = pd.Period('2024-01', freq='M')                            # Period

Interview Tip: "Why did my int column turn into a float?" is a classic. The answer: old-style NumPy int64 has no way to represent NaN, so pandas quietly upcasts to float64 the moment a missing value shows up. The nullable Int64 dtype (capital I) exists specifically to fix this.

# dict of lists -> columns
df1 = pd.DataFrame({'a': [1, 2], 'b': [3, 4]})

# dict of Series -> aligns on index automatically
df2 = pd.DataFrame({'a': pd.Series([1, 2], index=[0, 1])})

# list of dicts -> rows
df3 = pd.DataFrame([{'a': 1, 'b': 3}, {'a': 2, 'b': 4}])

# list of tuples/lists + explicit columns
df4 = pd.DataFrame([(1, 3), (2, 4)], columns=['a', 'b'])

# NumPy array
df5 = pd.DataFrame(np.arange(6).reshape(3, 2), columns=['a', 'b'])

Common Mistake: A dict of lists with mismatched lengths raises ValueError right away — good, loud, fast failure. A dict of Series with mismatched lengths instead aligns on the index and quietly fills gaps with NaN. Same-looking code, very different failure mode — know which one you're writing.

from dataclasses import dataclass
from collections import namedtuple

# nested dict -> outer keys become columns, inner keys become index
nested = {'q1': {'ny': 100, 'la': 90}, 'q2': {'ny': 110, 'la': 95}}
df_nested = pd.DataFrame(nested)

# list of namedtuples
Point = namedtuple('Point', ['x', 'y'])
df_nt = pd.DataFrame([Point(1, 2), Point(3, 4)])

# list of dataclasses
@dataclass
class Row:
    x: int
    y: int

df_dc = pd.DataFrame([Row(1, 2), Row(3, 4)])

# generator of dict rows (memory-friendly for large sources)
def gen_rows():
    for i in range(3):
        yield {'a': i, 'b': i * i}

df_gen = pd.DataFrame(gen_rows())

Note: If your real source is deeply nested JSON — lists inside dicts inside lists — reach for pd.json_normalize() instead of fighting the plain DataFrame constructor into submission.

df_csv = pd.read_csv('data.csv')
df_xlsx = pd.read_excel('data.xlsx', sheet_name='Sheet1')
df_json = pd.read_json('data.json')
df_parquet = pd.read_parquet('data.parquet')
df_feather = pd.read_feather('data.feather')

import sqlite3
conn = sqlite3.connect('app.db')
df_sql = pd.read_sql('SELECT * FROM users', conn)

import requests
payload = requests.get('https://api.example.com/records').json()
df_api = pd.json_normalize(payload)

df_clip = pd.read_clipboard()          # whatever is currently copied
df_html = pd.read_html('https://example.com/table-page')[0]   # list of tables on the page

Warning: pd.read_html() needs lxml or html5lib installed, and it grabs every <table> tag on the page into a list — the table you actually want is rarely index 0 on a busy page, so check before you assume.

df = pd.read_csv(
    'file.csv',
    sep=',',
    header=0,
    names=None,
    index_col=0,
    usecols=['a', 'b'],
    dtype={'a': 'int32'},
    parse_dates=['date_col'],
    na_values=['NA', '?'],
    nrows=1000,
    skiprows=1,
    chunksize=None,
    encoding='utf-8',
    compression='infer'
)

Performance: On a wide CSV, usecols plus an explicit dtype dict alone can cut load time and memory by half or more — it's one of the cheapest optimizations available and it's easy to forget.

Note: If you're re-reading the same dataset over and over during development, switch to Parquet or Feather early — they store dtypes and are columnar, so repeated loads are noticeably faster than round-tripping through CSV.

df.to_csv('out.csv', index=False)
df.to_excel('out.xlsx', sheet_name='Sheet1', index=False)
df.to_json('out.json', orient='records', lines=True)
df.to_sql('users', conn, if_exists='replace', index=False)
df.to_parquet('out.parquet')
df.to_feather('out.feather')
df.to_pickle('out.pkl')
df.to_html('out.html')
df.to_markdown()
df.to_dict(orient='records')
df.to_numpy()

Common Mistake: Skip index=False in to_csv/to_excel and you'll bake an extra unnamed index column into the file — which then reappears as a mysterious Unnamed: 0 column the next time someone reads it back in.

df.head(5)          # first 5 rows
df.tail(5)           # last 5 rows
df.sample(5, random_state=42)   # random 5 rows, reproducible

df.shape             # (n_rows, n_cols)
df.size              # total cell count
df.info()            # dtypes, non-null counts, memory
df.describe()        # numeric summary stats
df.describe(include='all')       # include categorical/object columns too

Pro Tip: Make df.info() a reflex right after loading anything new. It surfaces unexpected object columns immediately — usually a sign of mixed types or a stray string sitting in what should be a numeric column — and it's much cheaper to catch that now than three transformations later.

df.dtypes            # dtype per column
df.columns           # column labels
df.index             # row labels
df.axes              # [index, columns]
df.empty             # True if no rows
df.ndim              # 2 for DataFrame, 1 for Series
df.values             # underlying NumPy array (legacy; prefer to_numpy())
df.memory_usage(deep=True)   # accurate memory per column, incl. object payloads

Watch out: df.memory_usage() without deep=True under-reports object columns badly — it's only counting pointer size, not the actual string data sitting behind those pointers. If a memory number looks suspiciously small, this is usually why.

df['col']                      # single column -> Series
df[['a', 'b']]                 # multiple columns -> DataFrame

df.loc[2, 'a']                  # label-based: row label 2, column 'a'
df.loc[2:4, ['a', 'b']]         # label slicing is INCLUSIVE of the end

df.iloc[0, 0]                   # position-based: first row, first column
df.iloc[0:3, 0:2]               # position slicing is EXCLUSIVE of the end, like Python

df.at[2, 'a']                   # fast scalar get by label
df.iat[0, 0]                    # fast scalar get by position

Interview Trap: df.loc[2:4] includes row label 4. df.iloc[2:4] stops before position 4. It's a small asymmetry, but it's asked about constantly, and it's the kind of thing that causes silent off-by-one bugs if you're not paying attention.

df[df['age'] > 30]
df[(df['age'] > 30) & (df['city'] == 'NY')]     # use & / | with parentheses, not 'and'/'or'
df[df['city'].isin(['NY', 'LA'])]
df[df['age'].between(20, 40)]

df.query('age > 30 and city == "NY"')            # readable, supports @variable interpolation
min_age = 18
df.query('age > @min_age')

df.where(df['age'] > 30)          # keeps shape, fills non-matches with NaN
df.mask(df['age'] > 30)           # opposite of where()

Common Mistake: Using Python's plain and/or on a boolean Series throws ValueError: truth value of a Series is ambiguous almost every time. Always &/|, always with parentheses around each condition — it looks slightly uglier and it's non-negotiable.

Performance: With several conditions on a large DataFrame, .query() (running on the default numexpr engine) can genuinely outrun a chain of boolean masks, not just look cleaner.

df.iloc[-1]                       # last row
df.iloc[:, -1]                    # last column

mi_df.xs('2024', level='year')     # cross-section at one level of a MultiIndex
mi_df.xs(('2024', 'Q1'), level=['year', 'quarter'])

SQL comparison:

df.sort_values('age')                                  # ascending
df.sort_values('age', ascending=False)
df.sort_values(['city', 'age'], ascending=[True, False])   # multi-column, mixed order
df.sort_index()                                          # sort by row label

df.nlargest(3, 'age')      # top 3 by age -- faster than sort_values().head(3)
df.nsmallest(3, 'age')

df['rank'] = df['age'].rank(method='min', ascending=False)   # 1 = highest age

Performance: If you only need the top or bottom few rows, nlargest/nsmallest beats sort_values().head(n) on large data — it's a partial sort under the hood instead of sorting everything just to throw most of it away.

df.isna()            # boolean mask, True where missing (alias: isnull())
df.notna()           # opposite
df.isna().sum()      # missing count per column

df.fillna(0)
df.fillna({'age': df['age'].mean(), 'city': 'Unknown'})   # per-column fill
df.ffill()           # forward fill (old name: fillna(method='ffill'))
df.bfill()           # backward fill

df.dropna()                     # drop rows with ANY NaN
df.dropna(how='all')            # drop rows that are ALL NaN
df.dropna(subset=['age'])       # only consider specific columns

df['value'].interpolate(method='linear')   # fill gaps by interpolating

Performance: ffill/bfill are vectorized and quick. Writing a custom row-wise function and running it through apply() to do the same thing will work, but noticeably slower — there's rarely a good reason to hand-roll this.

Interview Tip: NaN (float missing marker), None (a plain Python object), and NaT (missing datetime) are three different things that all show up as "missing" under isna() — worth knowing the distinction even if pandas papers over it for you day to day.

df.rename(columns={'old': 'new'})
df.rename(str.lower, axis='columns')

df.drop(columns=['unused'])
df.drop(index=[0, 1])

df.duplicated()                       # boolean mask of duplicate rows
df.duplicated(subset=['email'])       # duplicates by specific columns
df.drop_duplicates(subset=['email'], keep='first')

Common Mistake: Most cleaning methods return a new DataFrame rather than mutating in place. Forget to reassign (df = df.drop(...)) or pass inplace=True, and the original just sits there unchanged while you wonder why nothing worked.

df['age'] = df['age'].astype('int64')
df = df.convert_dtypes()                     # best-guess nullable dtypes for every column

df['price'] = pd.to_numeric(df['price'], errors='coerce')     # bad values -> NaN instead of raising
df['date'] = pd.to_datetime(df['date'], format='%Y-%m-%d', errors='coerce')
df['duration'] = pd.to_timedelta(df['duration'])

df['region'] = df['region'].astype('category')

Warning: errors='ignore' is on its way out as of pandas 2.x. Use try/except, or lean on errors='coerce' and then go look at the resulting NaNs to see what actually failed — that's usually more informative anyway.

df['name'] = df['name'].str.strip().str.lower()
df['name'] = df['name'].str.replace(r'\s+', ' ', regex=True)
df['sku'] = df['sku'].str.upper()
df['title'] = df['title'].str.title()

Performance: .str methods run as a vectorized loop over the whole column in one pass — noticeably faster than df['name'].apply(lambda x: x.strip().lower()), which calls a Python function once per row and pays the interpreter tax every single time.

s = df['email']

s.str.contains('@gmail.com', case=False, na=False)
s.str.startswith('admin_')
s.str.endswith('.com')
s.str.replace('@old.com', '@new.com', regex=False)
s.str.split('@', expand=True)          # expand=True -> separate DataFrame columns
s.str.len()
s.str.slice(0, 5)                       # same as s.str[0:5]
s.str.find('@')                          # index of first match, -1 if not found
s.str.count('a')

Note: Every .str method returns NaN for a NaN input automatically — one less null-check you have to write yourself.

df['area_code'] = df['phone'].str.extract(r'\((\d{3})\)')
df[['user', 'domain']] = df['email'].str.extract(r'(?P<user>[^@]+)@(?P<domain>.+)')

df['phone'].str.match(r'^\d{3}-\d{4}$')        # anchored match at start
df['phone'].str.fullmatch(r'\d{3}-\d{4}')       # must match the ENTIRE string

Interview Tip: match() only anchors at the start — a trailing garbage character can still sneak through. fullmatch() requires the whole string to line up with the pattern. Mixing these up is a quiet source of "validation" that doesn't actually validate.

df['date'] = pd.to_datetime(df['date'])

df['year'] = df['date'].dt.year
df['month'] = df['date'].dt.month
df['weekday'] = df['date'].dt.day_name()
df['is_month_end'] = df['date'].dt.is_month_end

df['next_week'] = df['date'] + pd.Timedelta(days=7)
df['next_month'] = df['date'] + pd.DateOffset(months=1)
df['days_since'] = (pd.Timestamp.now() - df['date']).dt.days

Common Mistake: Adding pd.Timedelta(days=30) as a stand-in for "one month later" works fine in June and breaks quietly in February. pd.DateOffset(months=1) is the one that actually understands months.

df['ts'] = df['ts'].dt.tz_localize('UTC')
df['ts_ny'] = df['ts'].dt.tz_convert('America/New_York')

df = df.set_index('ts')
monthly = df['sales'].resample('ME').sum()          # month-end resample
daily_filled = df['sales'].resample('D').ffill()     # upsample + forward fill

bdays = pd.bdate_range('2024-01-01', '2024-01-31')   # business days only
us_holidays = pd.tseries.holiday.USFederalHolidayCalendar().holidays('2024-01-01', '2024-12-31')

Warning: Comparing a timezone-naive timestamp against a timezone-aware one raises TypeError outright — no silent wrong answer, just a hard stop. Localize consistently before comparing or merging across sources that might be in different zones.

df['age'].sum(); df['age'].mean(); df['age'].median()
df['age'].std(); df['age'].var()
df['age'].min(); df['age'].max()
df['age'].idxmin(); df['age'].idxmax()     # INDEX LABEL of the min/max, not the value
df['age'].mode()
df['age'].quantile(0.9)
df['age'].count()                          # non-null count
df.agg({'age': ['mean', 'max'], 'salary': 'sum'})

Interview Trap: idxmax() gives you the label of the row holding the maximum, not the maximum value itself. Mixing this up with .max() is one of the most common small errors in a technical screen.

SPLIT                 APPLY                  COMBINE
df.groupby('city') -> mean()/sum()/apply() -> single result per group, re-joined

df.groupby('city')['sales'].sum()
df.groupby(['city', 'year'])['sales'].mean()          # multiple keys
df.groupby(df['date'].dt.month)['sales'].sum()         # group by a derived Series
df.groupby('grade_category')['score'].sum()            # group by categorical column

df.groupby('city').agg(
    total_sales=('sales', 'sum'),
    avg_price=('price', 'mean')
)     # named aggregation, pandas >= 0.25

df.groupby('city')['sales'].transform('mean')          # broadcasts group mean back to every row
df.groupby('city').filter(lambda g: g['sales'].sum() > 1000)   # keep only qualifying groups
df.groupby('city').apply(lambda g: g.nlargest(2, 'sales'))     # arbitrary custom logic per group

Performance: apply() on a groupby falls back to a Python function call per group — it's the most flexible option and also the slowest one. If agg() or transform() can express what you're doing, they almost always should.

pd.merge(orders, customers, on='customer_id', how='inner')
pd.merge(orders, customers, left_on='cust_id', right_on='id', how='left')
pd.merge(a, b, how='outer', indicator=True)     # adds a '_merge' column showing match source
pd.merge(a, b, how='cross')                     # cartesian product, no key needed

Warning: If a join key is less unique than you assumed, a merge can silently multiply your row count and you won't notice until a downstream total looks wrong. Passing validate='one_to_one' (or the _many variants) turns that silent bug into an immediate, loud error — worth adding as a habit, not just when something's already broken.

a.join(b, how='left')                     # joins on INDEX by default (vs merge's column-based join)

pd.concat([df1, df2], axis=0)              # stack rows (like SQL UNION ALL)
pd.concat([df1, df2], axis=1)              # stack columns side-by-side
pd.concat([df1, df2], ignore_index=True)   # reset to a fresh RangeIndex

df1.combine_first(df2)                     # fill df1's NaNs using df2's values
df1.update(df2)                            # overwrite df1's matching values in place

pd.merge_asof(trades, quotes, on='time', direction='backward')   # nearest prior key match, for time series

Note: DataFrame.append() was removed in pandas 2.0. If you're following an old tutorial that uses it, swap in pd.concat([df1, df2]) — it's a drop-in replacement.

pd.pivot_table(df, index='city', columns='year', values='sales', aggfunc='sum', fill_value=0)
pd.crosstab(df['city'], df['product'], normalize='index')     # frequency/contingency table
df.pivot(index='date', columns='city', values='sales')         # like pivot_table but NO aggregation — errors on duplicate keys

long format                  wide format (after pivot)
date  city  sales     --->   date        NY   LA
1/1   NY    100              1/1         100  90
1/1   LA    90               1/2         110  95
1/2   NY    110
1/2   LA    95

Interview Trap: pivot() demands unique index/column combinations and will error on duplicates. pivot_table() is the forgiving cousin — it aggregates duplicates for you (default aggfunc='mean') instead of complaining.

pd.melt(df, id_vars=['date'], value_vars=['NY', 'LA'], var_name='city', value_name='sales')

stacked = df.stack()          # columns -> innermost index level (wide -> long)
unstacked = stacked.unstack()  # innermost index level -> columns (long -> wide), inverse of stack

pd.wide_to_long(df, stubnames='sales', i='id', j='year')   # for prefixed wide columns like sales2022, sales2023

Note: stack()/unstack() work on the Index; melt()/pivot() work on regular columns. Whichever pair matches where your key currently lives is the one to reach for.

df['sales'].rolling(window=7).mean()          # 7-period simple moving average
df['sales'].rolling(window=7, min_periods=1).mean()   # allow partial windows at the start
df['sales'].expanding().sum()                  # cumulative sum from the start, as a rolling stat
df['sales'].ewm(span=10).mean()                # exponentially weighted moving average

Real-world usage: rolling() is the moving average behind most finance and sales dashboards; ewm() shows up constantly in signal smoothing and forecasting features — it reacts to recent change without the jaggedness of a raw daily number.

df['running_total'] = df['sales'].cumsum()
df['running_max'] = df['sales'].cummax()
df['running_min'] = df['sales'].cummin()
df['running_product'] = df['sales'].cumprod()

df['prev_day'] = df['sales'].shift(1)
df['day_over_day_diff'] = df['sales'].diff()          # equivalent to sales - sales.shift(1)
df['pct_change'] = df['sales'].pct_change()            # % change vs previous row

Common Mistake: Running shift()/diff()/pct_change() on data that isn't sorted by date produces numbers that look plausible but mean nothing. Always sort_values('date') (or sort_index()) first — this one's easy to forget and hard to notice after the fact.

df['age'].map({'Y': 1, 'N': 0})                  # Series-only, dict/function element-wise mapping
df['age'].apply(lambda x: x * 2)                  # Series -> element-wise function
df.apply(lambda row: row['a'] + row['b'], axis=1)   # DataFrame -> row-wise (axis=1) or column-wise (axis=0)
df.map(lambda x: x if pd.notna(x) else 0)          # DataFrame element-wise (was DataFrame.applymap, deprecated)

def add_col(frame, col, value):
    return frame.assign(**{col: value})

result = df.pipe(add_col, 'flag', True).pipe(lambda d: d.dropna())   # readable left-to-right chaining

Note: DataFrame.applymap() is deprecated in favor of DataFrame.map() as of pandas 2.1+ — worth updating if you've got older code lying around.

# SLOW: Python-level loop under the hood, one call per row
df['total'] = df.apply(lambda r: r['price'] * r['qty'], axis=1)

# FAST: vectorized C loop across the whole column at once
df['total'] = df['price'] * df['qty']

# FAST: vectorized conditional instead of apply with if/else
df['tier'] = np.where(df['total'] > 100, 'high', 'low')

Pro Tip: Before reaching for .apply(), spend thirty seconds checking whether np.where, np.select, .str, .dt, or a boolean mask can express the same logic. It almost always can, and the speed difference on real-sized data is not subtle.

mi_df = df.set_index(['city', 'year'])                       # from existing columns
arrays = [['NY', 'NY', 'LA'], [2023, 2024, 2023]]
mi = pd.MultiIndex.from_arrays(arrays, names=['city', 'year'])

mi_df.loc['NY']                    # all rows for city='NY'
mi_df.loc[('NY', 2024)]            # a specific (city, year) combination
mi_df.sort_index(level='year')     # sort by one level
mi_df.swaplevel('city', 'year')    # reorder levels

flat = mi_df.reset_index()                    # every index level becomes a regular column

# after groupby.agg with multiple functions, columns become a MultiIndex too:
agg = df.groupby('city')['sales'].agg(['sum', 'mean'])
agg.columns = ['total_sales', 'avg_sales']     # manually flatten
# or, cleaner:
agg = df.groupby('city').agg(total_sales=('sales', 'sum'), avg_sales=('sales', 'mean'))

Common Mistake: Forgetting that a multi-aggregation groupby gives you MultiIndex columns, not flat ones, is a reliable source of confusing KeyErrors a few lines later. Named aggregation sidesteps the whole problem instead of fixing it after the fact.

# 1. Downcast numeric types
df['id'] = pd.to_numeric(df['id'], downcast='integer')

# 2. Use category for low-cardinality text
df['country'] = df['country'].astype('category')

# 3. Load only needed columns
df = pd.read_csv('big.csv', usecols=['id', 'sales'])

# 4. Process in chunks for out-of-core data
for chunk in pd.read_csv('big.csv', chunksize=100_000):
    process(chunk)

# 5. Prefer query()/boolean masks over row-wise apply()
df.query('sales > 100')

Interview Tip: Be ready to explain why category speeds up groupby — the grouping keys become integer codes behind the scenes, so hashing and comparing them is cheaper than doing the same with raw strings over and over.

Warning: SettingWithCopyWarning doesn't necessarily mean the assignment failed — it means pandas genuinely isn't sure whether you're editing a view or a copy. Worth resolving properly rather than reflexively silencing it; the ambiguity it's flagging is real.

Interview Tip: If someone asks "how would you speed this up?", the expected shape of the answer is usually: vectorize first, then fix the dtypes, then trim I/O to only the columns you need, then chunk if it still doesn't fit in memory, and only reach for Dask/Polars/Spark once you're actually beyond one machine.

# Method chaining reads top-to-bottom like a pipeline
result = (
    df
    .query('sales > 0')
    .assign(profit=lambda d: d['sales'] - d['cost'])
    .groupby('region')
    .agg(total_profit=('profit', 'sum'))
    .sort_values('total_profit', ascending=False)
)

• Reach for an explicit .copy() when you're about to mutate a filtered subset — it heads off SettingWithCopyWarning before it happens.
• Set dtypes as early as possible, ideally at read time, rather than patching them up after cleaning.
• Validate merges with validate=, and sanity-check row counts before and after every join.
• Lean on df.pipe() and method chaining rather than accumulating a pile of intermediate variable names.
• Write small, testable transformation functions instead of one long script that does everything.
• Skip inplace=True in anything meant to be reused or maintained — it fights with chaining and makes debugging harder than it needs to be.

Pro Tip: Drop assert checks after critical joins/groupbys (assert len(result) == expected_len, for instance) as cheap, immediate data-validation guardrails — they catch problems the moment they happen, not three steps downstream.

Need to combine two tables?
  -> Same columns, stacking rows?          use concat(axis=0)
  -> Different columns, matching on a key?  use merge()
  -> Matching on the row label (index)?     use join()
  -> Nearest prior timestamp match?         use merge_asof()

Need to reshape?
  -> Long -> wide, with aggregation?        use pivot_table()
  -> Long -> wide, no duplicates?           use pivot()
  -> Wide -> long?                          use melt()
  -> Move index level <-> columns?          use stack() / unstack()

Need to clean missing values?
  -> Fill with a constant/statistic?        use fillna()
  -> Fill using neighboring values?         use ffill() / bfill() / interpolate()
  -> Drop incomplete rows entirely?         use dropna()

Need to select rows/values?
  -> By label?                              use .loc
  -> By position?                           use .iloc
  -> By condition?                          use boolean mask or .query()