Python regex expression is a powerful and flexible way to search, match, extract and manipulate text. The re module is used for performing regular expression operations. Have a good understanding of regex helps a developer to handle unstructured and semi-structured data efficiently.
Re The Ultimate Python Regular Expressions Cheat Sheet
A comprehensive, practical reference for Python's re module — syntax, character classes, quantifiers, groups, lookaround, flags, common validation and extraction patterns, performance pitfalls, and debugging.
40 reference blocks
Regex Basics & the re Module
What a Regex Is & Raw Strings
A regular expression is a tiny pattern language for describing text shapes — and in Python, it almost always needs to live inside a raw string.
A regex (regular expression) is a sequence of characters that defines a search pattern — some characters match themselves literally, others (metacharacters) mean something special, like "one or more of the previous thing" or "the start of the string."
import re
re.search("\\d+", "abc123") # works, but the double backslash is ugly and easy to get wrong
re.search(r"\d+", "abc123") # raw string -- backslashes are passed through literally, exactly as typed
Common Mistake: Forgetting
r"..."is probably the single most common regex bug in Python. Without it,"\d"isn't a Python escape sequence, so it happens to pass through unchanged and still works — but"\b"(word boundary) is a real Python escape for a backspace character, and silently means something completely different without therprefix.
Compiling Patterns & Match Objects
The two core object types you'll be working with — a compiled pattern, and the result of a successful match.
import re
pattern = re.compile(r"\d+") # a Pattern object -- reusable, and faster if used many times
match = pattern.search("abc123") # a Match object, or None if nothing matched
if match:
match.group() # the matched text: '123'
Best Practice:
re.compile()up front and reusing the resulting pattern object is meaningfully faster than callingre.search(pattern, text)repeatedly with the same pattern string in a loop — Python does cache recently-used patterns internally, but an explicit compile is clearer and guaranteed.
Core Matching Functions
match, fullmatch, search & findall
The functions that answer "does this pattern appear, and where" — each anchored differently.
import re
re.match(r"\d+", "123abc") # matches -- match() only anchors at the START of the string
re.match(r"\d+", "abc123") # None -- fails, because 'abc' comes first
re.fullmatch(r"\d+", "123") # matches -- the ENTIRE string must match, start to end
re.fullmatch(r"\d+", "123abc") # None -- trailing text breaks a fullmatch
re.search(r"\d+", "abc123def") # matches -- searches ANYWHERE in the string
re.findall(r"\d+", "a1 b22 c333") # ['1', '22', '333'] -- ALL non-overlapping matches, as strings
| Function | Anchoring | Returns |
|---|---|---|
match() |
Start of string only | Match object or None |
fullmatch() |
Entire string must match | Match object or None |
search() |
Anywhere in the string | First Match object or None |
findall() |
Anywhere, all occurrences | A list of strings (or tuples, if groups are used) |
Interview Trap:
findall()returns plain strings when the pattern has no groups, but tuples of each group's contents when the pattern has one or more capturing groups — the return type silently changes shape depending on the pattern, which trips people up constantly.
finditer, split, sub & subn
Iterating over matches lazily, breaking a string apart, and search-and-replace.
import re
for m in re.finditer(r"\d+", "a1 b22 c333"): # like findall(), but yields Match objects lazily -- keeps span/position info
print(m.group(), m.span())
re.split(r"[,;]", "a,b;c") # ['a', 'b', 'c'] -- split on any of several delimiters
re.split(r"(,)", "a,b,c") # ['a', ',', 'b', ',', 'c'] -- capturing group in the pattern KEEPS the delimiters
re.sub(r"\d+", "#", "a1 b22") # 'a# b#' -- replace every match
re.sub(r"\d+", "#", "a1 b22", count=1) # 'a# b22' -- limit how many replacements happen
new_text, num_subs = re.subn(r"\d+", "#", "a1 b22") # same as sub(), but also returns the replacement count
Note:
re.split()normally throws away the text it split on — wrapping the delimiter pattern in a capturing group( )keeps those delimiters in the output list instead of discarding them.
escape() and purge()
Safely inserting literal, user-supplied text into a pattern, and clearing the internal pattern cache.
import re
user_input = "3.14 (pi)"
re.escape(user_input) # '3\\.14\\ \\(pi\\)' -- every special character is backslash-escaped
pattern = re.compile(re.escape(user_input)) # safely match the LITERAL text, dots and parens included
re.purge() # clears re's internal compiled-pattern cache -- rarely needed, mostly useful in benchmarking
Common Mistake: Interpolating raw user input directly into a pattern string (e.g.,
re.compile(user_input)) treats any regex metacharacters the user happens to type —.,*,(,)— as pattern syntax, not literal text. Always run untrusted or literal text throughre.escape()first if it needs to be matched exactly.
Character Classes
Custom Character Sets & Ranges
Matching "any one of these characters" — the square-bracket syntax that's the workhorse of most patterns.
import re
re.findall(r"[aeiou]", "hello world") # any single vowel -- ['e', 'o', 'o']
re.findall(r"[a-z]", "Hi There") # any lowercase letter, using a RANGE
re.findall(r"[a-zA-Z0-9]", "Room 42B!") # multiple ranges combined inside one class
re.findall(r"[^0-9]", "a1b2c3") # NEGATED set -- anything that ISN'T a digit -- ['a', 'b', 'c']
re.findall(r"[.]", "3.14") # inside [ ], most metacharacters (like .) are just literal
Common Mistake:
^only negates a character class when it's the first character right after[. A^anywhere else inside the brackets, like[0-9^], is just a literal caret — one of the more confusing character-class edge cases.
Predefined Character Classes: \d, \w, \s
Shorthand for the character-class patterns you'd otherwise write out by hand constantly.
| Shorthand | Matches | Negated form |
|---|---|---|
\d |
A digit (0-9, plus Unicode digits by default) |
\D — anything that isn't a digit |
\w |
A 'word' character: letters, digits, underscore | \W — anything that isn't |
\s |
Whitespace: space, tab, newline, etc. | \S — anything that isn't |
re.findall(r"\d+", "Room 42, Floor 3") # ['42', '3']
re.findall(r"\w+", "hello_world 123") # ['hello_world', '123'] -- underscore counts as a word char
re.split(r"\s+", "a b\tc\nd") # ['a', 'b', 'c', 'd']
re.findall(r"\d", "5\u0664", re.ASCII) # ASCII-only mode -- restricts \d to [0-9], excludes Unicode digits
Note: By default in Python 3,
\d,\w, and\sare Unicode-aware —\dmatches Unicode digit characters beyond plain ASCII0-9. Passre.ASCIIif you specifically need the narrower, ASCII-only behavior (e.g., for strict format validation).
Anchors, Quantifiers & Wildcards
Anchors & Word Boundaries
Matching a *position* in the text rather than an actual character — the start, the end, or the edge of a word.
| Anchor | Matches |
|---|---|
^ |
Start of the string (or start of each line, with re.MULTILINE) |
$ |
End of the string (or end of each line, with re.MULTILINE) |
\A |
Start of the string, always — unaffected by re.MULTILINE |
\Z |
End of the string, always — unaffected by re.MULTILINE |
\b |
A word boundary — between a word character and a non-word character |
\B |
NOT a word boundary — inside a word, or inside a run of non-word characters |
import re
re.search(r"^Hello", "Hello world") # matches
re.search(r"world$", "Hello world") # matches
re.findall(r"\bcat\b", "cat catalog cats") # ['cat'] -- word boundary excludes 'catalog' and 'cats'
re.findall(r"\Bcat\B", "concatenate") # matches 'cat' INSIDE a word, no boundary on either side
Interview Trap:
^and$shift meaning underre.MULTILINE(start/end of each line), but\Aand\Znever do — they always mean the absolute start/end of the whole string. Use\A/\Zwhen you specifically need that guarantee regardless of flags.
Quantifiers: *, +, ?, {n,m}
Controlling how many times the preceding element can repeat.
| Quantifier | Meaning |
|---|---|
* |
Zero or more |
+ |
One or more |
? |
Zero or one |
{n} |
Exactly n |
{n,} |
n or more |
{n,m} |
Between n and m, inclusive |
import re
re.findall(r"ab*", "a ab abb abbb") # ['a', 'ab', 'abb', 'abbb'] -- 0 or more b's
re.findall(r"ab+", "a ab abb") # ['ab', 'abb'] -- requires at least one b, so plain 'a' doesn't match
re.findall(r"colou?r", "color colour") # ['color', 'colour'] -- the u is optional
re.findall(r"\d{3}", "12 123 1234") # ['123', '123'] -- exactly 3 digits, taken from '1234' too
re.findall(r"\d{2,4}", "1 12 123 12345") # greedy -- takes as many digits as allowed, up to 4
Greedy, Lazy & Possessive Quantifiers
The same quantifier symbol can grab as much as possible, as little as possible, or refuse to give characters back — and the difference matters a lot.
import re
re.search(r"<.*>", "<a><b>").group() # '<a><b>' -- GREEDY: grabs as much as possible
re.search(r"<.*?>", "<a><b>").group() # '<a>' -- LAZY (add ?): grabs as little as possible
re.search(r"a{2,4}+", "aaaa") # POSSESSIVE quantifier (3.11+) -- like greedy, but never backtracks/gives characters back
| Type | Symbol | Behavior |
|---|---|---|
| Greedy (default) | * + ? {n,m} |
Matches as much as possible, backtracks if needed |
| Lazy | *? +? ?? {n,m}? |
Matches as little as possible, expands only if needed |
| Possessive | *+ ++ ?+ {n,m}+ |
Like greedy, but never backtracks — faster, can fail matches that greedy would find |
| Atomic group | (?>...) |
Wraps a whole group in possessive-style behavior |
Common Mistake: Reaching for
.*when parsing something like HTML tags almost always grabs far more than intended, because it's greedy by default..*?(lazy) is usually the fix — but the real fix for HTML/XML is often to not use regex for that at all (see the Regex Limitations section).
The Dot Wildcard & DOTALL Mode
Matching "any character" — except the one character it doesn't match by default.
import re
re.findall(r"a.c", "abc a c a\nc") # ['abc', 'a c'] -- '.' matches any char EXCEPT a newline, by default
re.findall(r"a.c", "a\nc", re.DOTALL) # ['a\nc'] -- re.DOTALL makes '.' match newlines too
Note: This is a very common source of "my pattern works on one line but not on multi-line text" bugs — if a pattern needs to span across a newline,
.alone won't do it unlessre.DOTALLis set.
Alternation: the | Operator
Matching one option out of several — and the precedence rule that trips people up the first time.
import re
re.findall(r"cat|dog", "I have a cat and a dog") # ['cat', 'dog']
re.search(r"^cat|dog$", "dog").group() # matches -- | has LOW precedence: this reads as (^cat)|(dog$), not ^(cat|dog)$
re.search(r"^(cat|dog)$", "dog").group() # the parens fix it -- now genuinely anchored on both branches
re.findall(r"gr(a|e)y", "gray grey") # order alternatives from most to least specific when they overlap
Common Mistake:
|has the lowest precedence of any regex operator — it applies to the entire pattern on each side, not just the adjacent characters. Wrap alternatives in a group(...)whenever you want|scoped to just part of the pattern.
Grouping & Backreferences
Capturing, Non-Capturing & Named Groups
Parentheses do two jobs at once: they scope part of a pattern, and (unless told not to) they capture what matched.
import re
m = re.search(r"(\d{3})-(\d{4})", "Call 555-1234")
m.group(1); m.group(2) # '555', '1234' -- numbered groups, 1-indexed
re.search(r"(?:\d{3})-(\d{4})", "555-1234").group(1) # '1234' -- (?:...) groups WITHOUT capturing -- group 1 is now the second parens
m = re.search(r"(?P<area>\d{3})-(?P<num>\d{4})", "555-1234")
m.group("area"); m.groupdict() # named groups -- far more readable than remembering numeric positions
re.search(r"colou(r)?", "color").group(1) # None -- an optional GROUP that simply didn't participate in the match
Best Practice: Prefer non-capturing groups
(?:...)for parts of a pattern you're only grouping for precedence or quantifying, not because you actually need the matched text later — it keeps group numbering predictable as a pattern grows and is slightly faster.
Backreferences
Referring back to whatever an earlier group actually matched — inside the same pattern, or inside a replacement.
import re
re.search(r"(\w+) \1", "hello hello") # \1 -- matches the SAME text group 1 captured -- finds repeated words
re.search(r"(?P<word>\w+) (?P=word)", "bye bye") # named backreference, same idea
re.sub(r"(\w+) (\w+)", r"\2 \1", "John Smith") # 'Smith John' -- \1/\2 inside the REPLACEMENT string swaps the groups
re.sub(r"(?P<first>\w+) (?P<last>\w+)", r"\g<last> \g<first>", "John Smith") # same, using named groups in the replacement
Note: A backreference (
\1) matches the exact text the group captured, not the pattern that defined it —(\w+) \1finds literally repeated words like "hello hello", it doesn't just repeat the\w+pattern independently.
Lookaround & Conditional Patterns
Lookahead & Lookbehind Assertions
Matching a position based on what comes before or after it — without actually consuming those characters as part of the match.
| Assertion | Syntax | Matches when... |
|---|---|---|
| Positive lookahead | (?=...) |
...what follows DOES match the pattern inside |
| Negative lookahead | (?!...) |
...what follows does NOT match the pattern inside |
| Positive lookbehind | (?<=...) |
...what precedes DOES match the pattern inside |
| Negative lookbehind | (?<!...) |
...what precedes does NOT match the pattern inside |
import re
re.findall(r"\d+(?= dollars)", "50 dollars, 30 euros") # ['50'] -- number, only if 'dollars' follows -- 'dollars' itself isn't captured
re.findall(r"\d+(?! dollars)", "50 dollars, 30 euros") # matches '3', '0' from '30' -- number NOT followed by 'dollars'
re.findall(r"(?<=\$)\d+", "$50 and 30") # ['50'] -- number, only if preceded by '$'
re.findall(r"(?<!\$)\b\d+\b", "$50 and 30") # ['30'] -- number NOT preceded by '$'
Warning: Lookbehind patterns in Python's
remust be fixed-width —(?<=\d{2,4})raisesre.errorbecause the width isn't fixed. The third-partyregexmodule lifts this restriction if you genuinely need variable-width lookbehind.
Conditional Patterns
Matching one sub-pattern or another, depending on whether an earlier group actually participated in the match.
import re
# (?(id)yes|no) -- if group 'id' matched, require 'yes'-pattern; otherwise require 'no'-pattern
pattern = r"(<)?\w+(?(1)>|$)" # if '<' was matched, require a closing '>'; otherwise require end of string
re.match(pattern, "<tag>") # matches -- opening bracket present, closing bracket required and found
re.match(pattern, "tag") # matches -- no opening bracket, so no closing bracket required
Note: Conditional patterns are a genuinely advanced, rarely-needed feature — they exist for cases like matching optionally-wrapped delimiters where the closing delimiter's requirement depends entirely on whether the opening one was present. Most real-world patterns never need this.
Regex Flags
Module-Level Flags
Modifying how an entire pattern behaves, passed as an argument rather than written inline.
| Flag | Short form | Effect |
|---|---|---|
re.IGNORECASE |
re.I |
Case-insensitive matching |
re.MULTILINE |
re.M |
^/$ match at each line's start/end, not just the string's |
re.DOTALL |
re.S |
. also matches newline characters |
re.VERBOSE |
re.X |
Allows whitespace and comments in the pattern for readability |
re.ASCII |
re.A |
\d \w \s match ASCII only, not full Unicode |
re.LOCALE |
re.L |
Locale-dependent matching — legacy, rarely needed in Python 3 |
re.NOFLAG |
— | Explicit 'no flags' value, mainly for clarity in code that takes a flags parameter |
import re
re.findall(r"hello", "Hello HELLO", re.IGNORECASE)
re.findall(r"^\w+", "line one\nline two", re.MULTILINE) # matches at the start of EACH line
combined = re.IGNORECASE | re.MULTILINE # combine flags with the | operator
re.findall(r"^hello", "Hello\nHELLO", combined)
Inline Flags: (?i), (?m) & Scoped Flags
Setting flags directly inside the pattern string — either for the whole pattern, or scoped to just part of it.
import re
re.findall(r"(?i)hello", "HELLO there") # (?i) at the start -- applies IGNORECASE to the WHOLE pattern
re.findall(r"cat(?i:dog)", "CATDOG catDOG") # (?i:...) -- scoped inline flag, Python 3.11+ -- only 'dog' part is case-insensitive
re.findall(r"(?i:cat)(?-i:dog)", "CATdog CATDOG") # (?-i:...) DISABLES a flag just for that group -- CATDOG doesn't match since 'DOG' fails
Note: In current Python, inline flags like
(?i)are only allowed at the very start of the pattern when applied globally — placing them elsewhere raises a deprecation warning/error in recent versions. Scoped inline flags(?i:...)(3.11+) are the modern way to apply a flag to just part of a pattern.
Match Objects & Compiled Pattern Methods
Match Object Properties
Everything a successful match tells you beyond just "yes, it matched."
import re
m = re.search(r"(?P<year>\d{4})-(?P<month>\d{2})", "Date: 2024-03")
m.group() # '2024-03' -- the whole match
m.group(1) # '2024' -- group 1 by number
m.group("year") # '2024' -- group by name
m.groups() # ('2024', '03') -- all numbered groups as a tuple
m.groupdict() # {'year': '2024', 'month': '03'} -- all named groups as a dict
m.start(); m.end() # 6, 13 -- overall match position
m.span() # (6, 13) -- same, as a tuple
m.span("year") # (6, 10) -- position of just the 'year' group
m.lastindex; m.lastgroup # the number/name of the LAST group that participated in the match
m.string # the original string that was searched
m.re # the compiled pattern object that produced this match
m.pos; m.endpos # the search range that was actually used
Compiled Pattern Object Methods
Everything a `re.compile()` result exposes — mirrors the module-level functions, plus metadata about itself.
import re
pattern = re.compile(r"(?P<num>\d+)", re.IGNORECASE)
pattern.match("123abc"); pattern.fullmatch("123"); pattern.search("a123")
pattern.findall("a1 b22"); pattern.finditer("a1 b22")
pattern.split("a,b,c".replace(',', ' ')); pattern.sub("#", "a1 b22"); pattern.subn("#", "a1 b22")
pattern.flags # the combined integer value of the flags used
pattern.groups # the number of capturing groups in the pattern
pattern.groupindex # {'num': 1} -- maps group NAMES to their numbers
pattern.pattern # the original pattern string it was compiled from
Note: Every method on a compiled
Patternobject mirrors a same-named function in theremodule —re.search(pattern, text)andcompiled_pattern.search(text)do the same thing; the compiled version is just faster to reuse and carries its own metadata.
Search Range, Multiple Matches & Splitting
Controlling the Search Range: pos & endpos
Restricting where in a string a compiled pattern is allowed to search, without slicing the string yourself.
import re
pattern = re.compile(r"\d+")
text = "a1 b22 c333"
pattern.search(text, pos=3) # start searching from index 3 onward -- finds '22'
pattern.search(text, pos=0, endpos=5) # only consider text[0:5] -- restricts the END of the search range too
Note:
pos/endposare available on compiled pattern methods (not the plainre.search()module function) and are more efficient than slicing the string yourself, because^/\Astill correctly refer to the true start of the original string, not the slice.
Finding Multiple & Overlapping Matches
`findall()`/`finditer()` skip past whatever they just matched — here's how to find matches that overlap anyway.
import re
re.findall(r"\d{2}", "123456") # ['12', '34', '56'] -- NON-overlapping by default, consumes as it goes
# to find OVERLAPPING matches, use a lookahead to 'peek' without consuming:
re.findall(r"(?=(\d{2}))", "123456") # ['12', '23', '34', '45', '56'] -- every 2-digit window, overlapping
for m in re.finditer(r"\d+", "a1 b22 c333"):
print(m.group(), m.span()) # span() gives the exact (start, end) of each match
Pro Tip: The zero-width lookahead trick
(?=(pattern))is the standard workaround for overlapping matches — since a lookahead doesn't consume characters, the regex engine's position advances by just one character at a time instead of jumping past the whole match.
Splitting: Capturing Delimiters & maxsplit
Beyond a basic split — keeping the separators, and limiting how many splits happen.
import re
re.split(r"\s+", "a b c") # ['a', 'b', 'c']
re.split(r"(\s+)", "a b c") # ['a', ' ', 'b', ' ', 'c'] -- capturing group keeps the separators
re.split(r",", "a,b,c,d", maxsplit=2) # ['a', 'b', 'c,d'] -- stop after 2 splits
re.split(r"\s*,\s*", "a, b ,c") # handles inconsistent spacing around a delimiter
re.split(r"x*", "abc") # be careful -- a pattern that can match EMPTY can produce surprising results
Warning: A split pattern that can match a zero-length (empty) string — like
x*against text with nox— behaves inconsistently across Python versions and is rarely what you actually want; make sure the split pattern always requires at least one real character.
Search and Replace
Replacement Strings: Static, Grouped & Callable
Three ways to decide what replaces each match — a fixed string, the matched groups themselves, or a function you write.
import re
re.sub(r"\d+", "NUM", "I have 5 apples and 3 oranges") # static replacement
re.sub(r"(\w+)@(\w+)", r"\1 [at] \2", "contact ada@example") # numbered group references in the replacement
re.sub(r"(?P<user>\w+)@(?P<domain>\w+)", r"\g<user> [at] \g<domain>", "ada@example") # named, via \g<name>
def upper_match(m): # callable replacement -- gets the Match object, returns the replacement string
return m.group().upper()
re.sub(r"\b\w+\b", upper_match, "hello world") # 'HELLO WORLD'
re.sub(r"\.", "[DOT]", "a.b.c", count=2) # limit to the first 2 replacements
Warning:
\1in a replacement string can be ambiguous with a literal digit right after it (e.g.,\11— is that group 1 followed by '1', or group 11?).\g<1>is the unambiguous form and is required anyway for numbers 10 and above, or when using named groups.
Verbose Regular Expressions
re.VERBOSE for Readable Patterns
Turning a dense, hard-to-read pattern into something with comments and whitespace — without changing what it matches.
import re
# dense and hard to read:
pattern = re.compile(r"^(\d{3})-(\d{3})-(\d{4})$")
# the same pattern, in VERBOSE mode:
pattern = re.compile(r"""
^ # start of string
(\d{3}) # area code
-
(\d{3}) # exchange
-
(\d{4}) # subscriber number
$ # end of string
""", re.VERBOSE)
# in VERBOSE mode, literal whitespace must be escaped or placed in a character class:
pattern2 = re.compile(r"foo\ bar", re.VERBOSE) # \ before the space -- otherwise VERBOSE ignores it
pattern3 = re.compile(r"foo[ ]bar", re.VERBOSE) # or put the literal space inside a character class
Note: In
re.VERBOSEmode, unescaped whitespace in the pattern is ignored entirely (so you can format it across multiple lines) and#starts a comment — meaning literal spaces and literal#characters both need to be escaped or placed inside a character class to be matched as themselves.
Unicode & Bytes Patterns
Unicode Text Matching
Matching non-ASCII text correctly — accented letters, non-Latin scripts, and Unicode-aware word boundaries.
import re
re.findall(r"\w+", "café naïve") # ['café', 'naïve'] -- \w is Unicode-aware by default, includes accented letters
re.findall(r"\w+", "日本語のテキスト") # matches non-Latin scripts too, by default
re.findall(r"café", "CAFÉ", re.IGNORECASE) # Unicode-aware case folding handles accented case differences correctly
re.findall(r"\w+", "café", re.ASCII) # ['caf'] -- ASCII mode excludes the accented 'é' from \w entirely
Note: Python 3 strings are Unicode by default, and so is
rematching against them — this is a real improvement over Python 2, where handling non-ASCII text in regex required much more careful, explicit handling.
Bytes Patterns
Matching against `bytes` instead of `str` — a different (and non-mixable) mode entirely.
import re
re.findall(rb"\d+", b"abc123") # rb"..." -- a raw BYTES pattern, matched against bytes input
re.sub(rb"\d+", b"#", b"a1 b22") # replacement value must also be bytes
# re.search(r"\d+", b"abc123") # raises TypeError -- can't mix a str pattern with bytes input
Warning: A
strpattern andbytesdata (or vice versa) can never be mixed — Python raisesTypeError: cannot use a string pattern on a bytes-like objectimmediately. Decide up front whether you're working with text or raw bytes, and keep the pattern and the input consistent.
Common Validation Patterns
Email, Phone, URL & IP Address Patterns
Ready-to-adapt patterns for the validation tasks that come up in almost every project — with an honest note about their limits.
| What | Pattern (as a raw string) |
|---|---|
| Simple email-like string | r"^[\w.+-]+@[\w-]+\.[a-zA-Z]{2,}$" |
| US phone number | r"^\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}$" |
| ISO date (YYYY-MM-DD) | r"^\d{4}-\d{2}-\d{2}$" |
| 24-hour time (HH:MM) | r"^([01]\d|2[0-3]):[0-5]\d$" |
| Simple URL | r"^https?://[\w.-]+(?:/[\w./?%&=-]*)?$" |
| IPv4 address | r"^(\d{1,3}\.){3}\d{1,3}$" (needs a range check per octet for full correctness) |
| US ZIP code | r"^\d{5}(-\d{4})?$" |
| Username (letters, digits, underscore) | r"^\w{3,20}$" |
| Hex color code | r"^#?[0-9a-fA-F]{6}$" |
| UUID (v4-style format) | r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$" |
Warning: The email pattern above (and most "email regex" patterns you'll find online) is a reasonable format sanity check, not a full validation of RFC 5322 — the actual email spec is notoriously complex. For anything that matters, pair a simple regex check with actually sending a confirmation email.
Common Extraction Patterns
Numbers, Currency & Word-Level Extraction
Pulling numeric and word-shaped values out of free text.
import re
re.findall(r"-?\d+\.?\d*", "Values: -3, 4.5, 10") # signed integers and decimals: ['-3', '4.5', '10']
re.findall(r"\d+(?:\.\d+)?%", "Up 12.5% this week, down 3%") # percentages, decimals optional
re.findall(r"\$\d+(?:,\d{3})*(?:\.\d{2})?", "Total: $1,250.00") # currency with thousands separators
re.findall(r"\b[a-zA-Z]+\b", "Hello, world! 123") # words only, no digits or punctuation
Note: For anything beyond quick text extraction — real currency parsing, especially across locales — a dedicated library (like
babelor manualDecimalparsing) handles edge cases a hand-written regex won't.
Hashtags, Mentions & Quoted Text
Common social-text and structured-text extraction patterns.
import re
re.findall(r"#\w+", "Loving #python and #regex today") # ['#python', '#regex']
re.findall(r"@\w+", "cc @alice and @bob") # ['@alice', '@bob']
re.findall(r'"([^"]*)"', 'She said "hello there" and left') # ['hello there'] -- text between double quotes
re.findall(r"\((.*?)\)", "note (first) and (second)") # ['first', 'second'] -- text between parens, lazy so it doesn't over-grab
re.findall(r"<(\w+)[^>]*>", "<div class='a'><span>") # ['div', 'span'] -- tag names from HTML-like text (see the warning in Regex Limitations)
File Names, Extensions & Log Fields
Pulling structure out of file paths and log lines — two of the most common real-world regex use cases.
import re
re.findall(r"[\w-]+\.\w+$", "/path/to/report_2024.csv") # ['report_2024.csv'] -- filename with extension
re.match(r"(.+)\.(\w+)$", "report.csv").groups() # ('report', 'csv') -- split into name and extension
log_line = '2024-03-15 14:22:01 ERROR Failed to connect: timeout'
m = re.match(r"(?P<date>\S+) (?P<time>\S+) (?P<level>\w+) (?P<message>.*)", log_line)
m.groupdict() # cleanly split a log line into structured fields
String Cleaning Patterns
Whitespace, Punctuation & Separator Cleanup
The regex patterns behind most "clean this messy text" tasks.
import re
re.sub(r"\s+", " ", "too much \t whitespace").strip() # collapse runs of whitespace to a single space
re.sub(r"(.)\1+", r"\1", "soooo goooood") # 'so god' -- collapse repeated characters using a backreference
re.sub(r"[^\w\s]", "", "Hello, world!!!") # strip punctuation entirely
re.sub(r"[^a-zA-Z0-9\s]", "", "Café #2024!") # strip anything that isn't alphanumeric or whitespace
re.sub(r"[-_\s]+", "-", "my_file name-here") # normalize mixed separators to a single dash
re.sub(r"\n\s*\n", "\n", "line one\n\n\nline two") # collapse multiple blank lines into one
re.sub(r"^(pre_|tmp_)", "", "tmp_data") # strip a known prefix
re.sub(r"(_old|_backup)$", "", "report_backup") # strip a known suffix
Pro Tip:
(.)\1+— 'any character, then one or more repeats of that same character' — is a compact and genuinely useful pattern for collapsing runs of repeated characters, well beyond just letters (works for repeated punctuation, repeated digits, anything).
Pattern Design Principles & Precedence
Designing Robust Patterns
The habits that separate a pattern that works on your test cases from one that actually holds up on real data.
- Anchor validation patterns with
^/$(or\A/\Z) — without them, a "validation" pattern liker"\d{5}"will happily match a ZIP code buried inside a much longer, otherwise invalid string. - Restrict character sets as tightly as the data allows —
[a-zA-Z]is more predictable than\wif you specifically don't want to accidentally allow digits or underscores. - Prefer non-capturing groups
(?:...)for anything you're grouping purely for precedence or quantifying, not because you need the matched text back. - Avoid unnecessary
.*— it's the single biggest cause of both over-matching and catastrophic backtracking; reach for a more specific character class whenever one will do. - Order alternatives from most to least specific when they could otherwise shadow each other — regex alternation picks the first branch that matches, not the longest.
- Explicitly handle optional separators and empty strings rather than assuming they won't occur —
r"a,b"vsr"a, b"vsr"a , b"are all realistic inputs a "simple" comma-split pattern needs to survive.
import re
# unanchored -- 'validates' way too much
re.match(r"\d{5}", "12345-and-then-junk") # matches! -- probably not what a ZIP validator should do
# anchored -- actually validates the whole string
re.fullmatch(r"\d{5}", "12345-and-then-junk") # None -- correctly rejected
Regex Precedence Rules
The order regex operators bind in, from tightest to loosest — and why explicit grouping beats memorizing the table.
| Precedence (highest to lowest) | Operator(s) |
|---|---|
| 1. Grouping | (...), (?:...) |
| 2. Quantifiers | * + ? {n,m} — apply to the single preceding element/group |
| 3. Concatenation | Sequencing characters/groups one after another |
| 4. Alternation | | — applies to everything on either side, at the lowest precedence |
import re
re.findall(r"ab+c|d", "abbc d") # reads as (ab+c)|(d) -- quantifier binds tightly to 'b', alternation is outermost
Best Practice: Rather than relying on memorized precedence rules for anything non-trivial, add explicit grouping
(...)— it costs nothing at runtime for non-capturing groups and removes any ambiguity for the next person reading the pattern (often, future you).
Performance & Optimization
Catastrophic Backtracking & How to Avoid It
The failure mode where a regex engine's runtime explodes exponentially on certain inputs — often from patterns that look completely innocent.
import re
# DANGEROUS: nested quantifiers on overlapping character sets
dangerous = re.compile(r"(a+)+b")
# dangerous.match("a" * 30) # can take an extremely long time -- classic catastrophic backtracking
# SAFER: flatten the nested quantifier, since (a+)+ and a+ match the same strings anyway
safer = re.compile(r"a+b")
# ambiguous alternatives inside a repeated group are the other classic trigger
risky = re.compile(r"(a|a)+b") # each repetition can match 'a' two different ways -- exponential blowup
| Warning sign | Why it's risky |
|---|---|
Nested quantifiers, e.g. (a+)+ |
Exponentially many ways to split the same matched text across repetitions |
| Alternation inside a repeated group with overlapping branches | Same text can be matched multiple different ways, each explored on backtrack |
.* followed by another .* or a specific pattern |
Forces excessive backtracking to find where the second part starts |
Warning: Catastrophic backtracking is a real, exploitable denial-of-service vector (ReDoS) if a pattern like this ever runs against untrusted user input. Test any pattern that accepts external input against a deliberately adversarial string before shipping it.
Compiling, Atomic Groups & Benchmarking
The practical tools for keeping regex fast: reuse compiled patterns, cut off backtracking where it isn't needed, and actually measure.
import re, timeit
# compile once, reuse many times -- avoids re-parsing the pattern on every call
PHONE = re.compile(r"\d{3}-\d{4}")
for line in ["555-1234", "555-5678"]:
PHONE.search(line)
# atomic groups / possessive quantifiers prevent backtracking into a segment once it's matched
re.match(r"(?>a+)b", "aaab") # atomic group -- 3.11+
re.match(r"a++b", "aaab") # possessive quantifier -- equivalent effect, 3.11+
timeit.timeit(lambda: PHONE.search("555-1234"), number=100_000) # measure, don't guess
Performance: Atomic groups and possessive quantifiers both tell the engine "once this part matches, don't ever backtrack into it" — this can turn an exponential-time pattern into a linear-time one, at the cost of occasionally rejecting a match a fully-backtracking pattern would have found.
Error Handling & Debugging
re.error and Invalid Patterns
What happens when a pattern itself is malformed, and how to catch it gracefully.
import re
try:
re.compile(r"[a-") # unbalanced character class
except re.error as e: # re.error is an alias for re.PatternError (Python 3.13+)
print(f"Invalid pattern: {e}")
try:
re.compile(r"(\1)") # invalid backreference -- group 1 referenced before it's ever defined
except re.error as e:
print(e)
Note:
re.erroris the exception type raised for a malformed pattern at compile time — as of Python 3.13 it's also available as the more descriptively namedre.PatternError, withre.errorkept as a backward-compatible alias.
Testing & Debugging Patterns
Treating a regex like any other piece of logic that deserves real test cases — including the ones designed to break it.
import re
pattern = re.compile(r"^\d{3}-\d{4}$")
# positive cases -- things that SHOULD match
assert pattern.fullmatch("555-1234")
# negative cases -- things that should NOT match
assert not pattern.fullmatch("555-12345")
assert not pattern.fullmatch("abc-1234")
# boundary cases -- right at the edges of what's allowed
assert not pattern.fullmatch("55-1234") # one digit short
assert not pattern.fullmatch(" 555-1234") # leading whitespace
re.compile(r"(\d+)", re.DEBUG) # prints the internal parsed representation of the pattern -- useful for understanding how it's actually being interpreted
Best Practice: Treat regex patterns like any other piece of logic worth unit testing — a handful of positive, negative, and boundary test cases catch far more real bugs than eyeballing the pattern, especially once quantifiers and groups start interacting.
Regex Limitations & Common Pitfalls
What Regex Can't Do Well
Regex matches patterns in flat text — the moment structure needs to nest arbitrarily, it stops being the right tool.
Regular expressions describe regular languages — patterns without unbounded, arbitrarily nested structure. That's exactly why they struggle with:
| Limitation | Why |
|---|---|
| Nested/recursive structures | Matching balanced parentheses or nested tags of arbitrary depth isn't expressible in standard regex at all |
| Context-sensitive text | A regex has no memory of what it saw far earlier in a way that scales to arbitrary nesting depth |
| HTML and XML parsing | Real HTML/XML has nested, sometimes malformed tags — a proper parser (BeautifulSoup, lxml, ElementTree) handles this correctly where regex will eventually break |
import re
# looks like it works...
re.findall(r"<(\w+)>", "<div><span></span></div>") # ['div', 'span']
# ...but breaks the moment nesting or attributes get more complex
re.findall(r"<(\w+)>", "<div class='<a>'>") # gives a wrong/misleading result -- regex has no concept of 'nesting'
Best Practice: "Don't parse HTML with regex" is a genuine, well-earned piece of advice, not just a meme — for anything beyond a quick, throwaway text scrape, use an actual HTML/XML parser instead.
Common Pitfalls Checklist
The mistakes that show up constantly, in roughly the order you'll actually run into them.
| Pitfall | What actually happens |
|---|---|
Forgetting r"..." |
Backslash escapes get interpreted by Python first, before the regex engine ever sees them |
Confusing match() with search() |
match() silently only checks the START of the string — a very common false negative |
| Missing anchors on a validation pattern | The pattern matches a SUBSTRING of otherwise-invalid input, not the whole thing |
Overusing .* |
Over-matches greedily, and is a common cause of catastrophic backtracking |
| Greedy matching where lazy was intended | <.*> against "<a><b>" grabs the whole thing, not just <a> |
| Incorrect character ranges | [A-z] accidentally includes several punctuation characters between Z and a in ASCII |
| Unescaped metacharacters | A literal ., (, or $ in the DATA needs re.escape() if it's not meant as regex syntax |
| Capturing-group numbering errors | Adding/removing a group earlier in the pattern shifts every later group's number |
| Empty-match behavior | A pattern that can match zero-length text behaves inconsistently in split()/finditer() |
| Double escaping | r"\\d" matches a literal backslash + 'd', not a digit — one backslash too many |
| Using regex for full semantic validation | A regex can check FORMAT, not meaning — "99999-99-99" can pass a date-shaped pattern while being a nonsense date |
