How to work with higher-order functions¶

This guide covers practical patterns for using higher-order functions in Python. A higher-order function is a function that takes another function as an argument, returns a function, or both. Python provides several built-in higher-order functions, and you can write your own.

Using map() to transform sequences¶

The map() function applies a function to every item in an iterable and returns an iterator of the results.

In [ ]:

temperatures_celsius = [0, 20, 37, 100]


def celsius_to_fahrenheit(c: float) -> float:
    return c * 9 / 5 + 32


temperatures_fahrenheit = list(map(celsius_to_fahrenheit, temperatures_celsius))
print(temperatures_fahrenheit)

temperatures_celsius = [0, 20, 37, 100] def celsius_to_fahrenheit(c: float) -> float: return c * 9 / 5 + 32 temperatures_fahrenheit = list(map(celsius_to_fahrenheit, temperatures_celsius)) print(temperatures_fahrenheit)

You can also use map() with a lambda expression for simple transformations.

In [ ]:

names = ["alice", "bob", "charlie"]
capitalised = list(map(lambda name: name.capitalize(), names))
print(capitalised)

names = ["alice", "bob", "charlie"] capitalised = list(map(lambda name: name.capitalize(), names)) print(capitalised)

map() can also take multiple iterables. The function receives one argument from each iterable.

In [ ]:

bases = [2, 3, 4]
exponents = [3, 2, 5]

powers = list(map(pow, bases, exponents))
print(powers)  # [2**3, 3**2, 4**5]

bases = [2, 3, 4] exponents = [3, 2, 5] powers = list(map(pow, bases, exponents)) print(powers) # [2**3, 3**2, 4**5]

Using filter() to select items¶

The filter() function returns an iterator containing only the items for which the given function returns True.

In [ ]:

numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]


def is_even(n: int) -> bool:
    return n % 2 == 0


evens = list(filter(is_even, numbers))
print(evens)

numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] def is_even(n: int) -> bool: return n % 2 == 0 evens = list(filter(is_even, numbers)) print(evens)

In [ ]:

# Filter with a lambda to select words longer than four characters
words = ["the", "quick", "brown", "fox", "jumps", "over", "a", "lazy", "dog"]
long_words = list(filter(lambda w: len(w) > 4, words))
print(long_words)

# Filter with a lambda to select words longer than four characters words = ["the", "quick", "brown", "fox", "jumps", "over", "a", "lazy", "dog"] long_words = list(filter(lambda w: len(w) > 4, words)) print(long_words)

Using functools.reduce() for accumulation¶

The reduce() function applies a two-argument function cumulatively to the items of an iterable, reducing them to a single value. It is in the functools module.

In [ ]:

from functools import reduce

numbers = [1, 2, 3, 4, 5]

# Calculate the product of all numbers
product = reduce(lambda a, b: a * b, numbers)
print(f"Product: {product}")

# Find the maximum value
maximum = reduce(lambda a, b: a if a > b else b, numbers)
print(f"Maximum: {maximum}")

from functools import reduce numbers = [1, 2, 3, 4, 5] # Calculate the product of all numbers product = reduce(lambda a, b: a * b, numbers) print(f"Product: {product}") # Find the maximum value maximum = reduce(lambda a, b: a if a > b else b, numbers) print(f"Maximum: {maximum}")

In [ ]:

from functools import reduce

# Use an initial value to provide a starting point
words = ["Python", "is", "a", "powerful", "language"]
sentence = reduce(lambda a, b: f"{a} {b}", words)
print(sentence)

from functools import reduce # Use an initial value to provide a starting point words = ["Python", "is", "a", "powerful", "language"] sentence = reduce(lambda a, b: f"{a} {b}", words) print(sentence)

For simple summation or joining, built-in functions such as sum() and str.join() are clearer. Reserve reduce() for cases where no built-in alternative exists.

Sorting with custom key functions¶

The sorted() function and the list.sort() method accept a key parameter — a function that extracts a comparison value from each item.

In [ ]:

students = [
    {"name": "Alice", "grade": 85},
    {"name": "Bob", "grade": 92},
    {"name": "Charlie", "grade": 78},
    {"name": "Diana", "grade": 95},
]

# Sort by grade (ascending)
by_grade = sorted(students, key=lambda s: s["grade"])
for s in by_grade:
    print(f"{s['name']}: {s['grade']}")

students = [ {"name": "Alice", "grade": 85}, {"name": "Bob", "grade": 92}, {"name": "Charlie", "grade": 78}, {"name": "Diana", "grade": 95}, ] # Sort by grade (ascending) by_grade = sorted(students, key=lambda s: s["grade"]) for s in by_grade: print(f"{s['name']}: {s['grade']}")

In [ ]:

# Sort by grade descending, then by name ascending for ties
students_with_ties = [
    {"name": "Alice", "grade": 85},
    {"name": "Bob", "grade": 85},
    {"name": "Charlie", "grade": 92},
]

result = sorted(students_with_ties, key=lambda s: (-s["grade"], s["name"]))
for s in result:
    print(f"{s['name']}: {s['grade']}")

# Sort by grade descending, then by name ascending for ties students_with_ties = [ {"name": "Alice", "grade": 85}, {"name": "Bob", "grade": 85}, {"name": "Charlie", "grade": 92}, ] result = sorted(students_with_ties, key=lambda s: (-s["grade"], s["name"])) for s in result: print(f"{s['name']}: {s['grade']}")

Composing functions¶

Function composition chains multiple functions together so that the return value of one becomes the input to the next. Python does not have a built-in composition operator, but you can create one.

In [ ]:

from functools import reduce
from typing import Callable


def compose(*functions: Callable) -> Callable:
    """Compose functions right to left: compose(f, g, h)(x) == f(g(h(x)))."""
    def apply(f, g):
        return lambda x: f(g(x))
    return reduce(apply, functions)


def double(x: int) -> int:
    return x * 2


def increment(x: int) -> int:
    return x + 1


def square(x: int) -> int:
    return x ** 2


# Read right to left: square, then increment, then double
transform = compose(double, increment, square)
print(f"transform(3) = double(increment(square(3))) = {transform(3)}")

from functools import reduce from typing import Callable def compose(*functions: Callable) -> Callable: """Compose functions right to left: compose(f, g, h)(x) == f(g(h(x))).""" def apply(f, g): return lambda x: f(g(x)) return reduce(apply, functions) def double(x: int) -> int: return x * 2 def increment(x: int) -> int: return x + 1 def square(x: int) -> int: return x ** 2 # Read right to left: square, then increment, then double transform = compose(double, increment, square) print(f"transform(3) = double(increment(square(3))) = {transform(3)}")

Building a data pipeline¶

A more readable approach for data processing is a left-to-right pipeline where each step feeds into the next.

In [ ]:

from typing import Callable


def pipeline(value, *functions: Callable):
    """Apply functions left to right: pipeline(x, f, g, h) == h(g(f(x)))."""
    for func in functions:
        value = func(value)
    return value


# Process a list of prices: remove negatives, apply discount, round
raw_prices = [10.5, -3.0, 25.99, 0, 15.75, -1.0]

result = pipeline(
    raw_prices,
    lambda prices: [p for p in prices if p > 0],     # Remove non-positive
    lambda prices: [p * 0.9 for p in prices],         # Apply 10% discount
    lambda prices: [round(p, 2) for p in prices],     # Round to 2 decimal places
)

print(f"Processed prices: {result}")

from typing import Callable def pipeline(value, *functions: Callable): """Apply functions left to right: pipeline(x, f, g, h) == h(g(f(x))).""" for func in functions: value = func(value) return value # Process a list of prices: remove negatives, apply discount, round raw_prices = [10.5, -3.0, 25.99, 0, 15.75, -1.0] result = pipeline( raw_prices, lambda prices: [p for p in prices if p > 0], # Remove non-positive lambda prices: [p * 0.9 for p in prices], # Apply 10% discount lambda prices: [round(p, 2) for p in prices], # Round to 2 decimal places ) print(f"Processed prices: {result}")

When to use higher-order functions versus list comprehensions¶

Python offers list comprehensions as an alternative to map() and filter(). Here is a comparison to help you decide.

In [ ]:

numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# map + filter approach
result_hof = list(map(lambda n: n ** 2, filter(lambda n: n % 2 == 0, numbers)))

# List comprehension approach
result_comp = [n ** 2 for n in numbers if n % 2 == 0]

print(f"Higher-order functions: {result_hof}")
print(f"List comprehension:    {result_comp}")

numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] # map + filter approach result_hof = list(map(lambda n: n ** 2, filter(lambda n: n % 2 == 0, numbers))) # List comprehension approach result_comp = [n ** 2 for n in numbers if n % 2 == 0] print(f"Higher-order functions: {result_hof}") print(f"List comprehension: {result_comp}")

Prefer list comprehensions when:

• The transformation is simple and fits in a single expression
• You are combining map() and filter() (comprehensions handle both in one expression)
• Readability matters more than functional style

Prefer higher-order functions when:

• You already have a named function to pass (for example, map(str.upper, words))
• You are building reusable pipelines or composing functions
• You want lazy evaluation (both map() and filter() return iterators)

In [ ]:

# When you already have a named function, map() is cleaner
words = ["hello", "world", "python"]

# Clear and concise
print(list(map(str.upper, words)))

# The comprehension version is slightly more verbose
print([w.upper() for w in words])

# When you already have a named function, map() is cleaner words = ["hello", "world", "python"] # Clear and concise print(list(map(str.upper, words))) # The comprehension version is slightly more verbose print([w.upper() for w in words])

Summary¶

The key patterns for working with higher-order functions are as follows:

• Use map() to apply a transformation to every item in a sequence
• Use filter() to select items that satisfy a condition
• Use functools.reduce() for cumulative operations where no built-in alternative exists
• Pass custom key functions to sorted() for flexible sorting
• Build pipelines by composing simple functions together
• Prefer list comprehensions for simple cases and higher-order functions when reusability or lazy evaluation matters