ELIF - Python

List Comprehensions

List comprehensions are fundamental tools in the Python programming language.

If you read open source Python code you'll see them everywhere ([1], [2], [3]), and by relying on them, you'll make your code more declarative and concise (Pythonic).

We usually see our students struggling with them when they're starting our course. We believe it's because they don't understand the fundamental concepts behind them (immutability, collections, map function, etc).

This quick guide tries to walk you through these concepts and explain List comprehensions in a simple way. Let's get started!

Check out our course: Introduction to Programming with Python

Immutability

When you're working with collections, you'll usually have two options to complete your job: either mutate the collection (change/modify it) or create a new one (without changing the original). An example would be to add an element to a list. The mutable option would look something like:

my_list = [1, 2, 3]  # A list containing integers
my_list.append(4)    # Append an element
print(my_list)       # [1, 2, 3, 4]
# ^ my_list has been changed

In the other hand, an immutable solution would be:

my_list = [1, 2, 3]  # A list containing integers
other_list = my_list + [4]
print(my_list)       # [1, 2, 3]
print(other_list)       # [1, 2, 3, 4]
# ^ my_list has NOT been changed

In general terms we'll prefer immutable solutions, as they provide some strong benefits, but sometimes, mutability is required and accepted. Through the rest of this article, we'll use immutable solutions.

Please note that there's no "good" or "bad" in these situations. It's usually a design decision to make.

Map Function

The map function is a fundamental concept in functional programming. The concept behind it is really simple:

given an original collection, create a new collection with the results of applying a given function to each element in the original collection.

Examples are:

Square numbers

You have a collection of numbers and you want to square all the numbers: [1, 2, 3] > [1, 4, 9].

Get email addresses from users

You have a collection of users and you want the email addresses of each of those users: [user1, user2] > ['john@gmail.com', 'mary@live.com'].

Get domain names from email addresses

You have a collection of email addresses and you want the domain name of each one of those emails: ['john@gmail.com', 'mary@live.com'] > ['gmail.com', 'live.com'].

Transform function

In our previous example we wanted to get the domain name from a given email address. To do so, we need to define a specific function that receives an email address (john@gmail.com) and returns its domain name (gmail.com). Such function is simple to implement:

def get_domain_name(email):
    at_position = email.index('@')
    return email[at_position + 1:]

get_domain_name('john@gmail.com')  # 'gmail.com'
get_domain_name('mary@live.com')   # 'live.com'
get_domain_name('Invalid Email')   # Error! Raises ValueError.

This is our long praised transform function. It's a function that takes one element and returns the desired transformed result. From email address to domain name, from number to squared number, etc.

The map function will apply the get_domain_name function to each one of the elements in the original collection to create a new collection with the results:

[                           [
    'john@gmail.com',           'gmail.com',
    'mary@live.com',    >       'live.com',
    'rose@gmail.com             'gmail.com'
]                           ]

Which in Python code, it'd be the equivalent of doing:

domain_names = [
    get_domain_name('john@gmail.com'),
    get_domain_name('mary@live.com'),
    get_domain_name('rose@gmail.com')
]
print(domain_names)  # ['gmail.com', 'live.com', 'gmail.com']

Again, we're applying a given transform function (get_domain_name) to each element in the original collection.

Remember! The transform function is defined for just one element of the original collection, and returns the transformed value you'd expect.

List comprehensions

We've talked about the map function, and yet, we haven't even used it. We wanted to show you the concepts behind it. List comprehensions in Python behave just like the map function (and even more, as we'll see in the next ELI5). We'll jump straight to the code following our previous example of extracting domain names from email addresses using a list comprehension:

original_emails = [
    'john@gmail.com',
    'mary@live.com',
    'rose@gmail.com'
]
domain_names = [get_domain_name(email) for email in original_emails]
print(domain_names)  # ['gmail.com', 'live.com', 'gmail.com']

In this case we're doing exactly the same as we were doing with our map function, but we're using a list comprehension. List comprehensions are just syntactic sugar to write more concise and declarative map functions. Think about it for a second, the map function is such a widely used tool, that the Python developers decided to include it in the language with its special syntax.

Anatomy of a List Comprehension

Let's focus for a second on the syntax of our list comprehension.

List Comprehension Syntax

As you can see, the composing elements of our list comprehension are almost the same from our previous examples. The only difference is the introduction of the special syntax for in. The rest remains the same. You can try reading the list comprehension in this way:

Construct a new list by applying the following transform function to each one of the elements from the original collection.

Comprehensions are immutable

It's really important to remember that a List comprehension will create a new list without modifying the original collection. List comprehensions are immutable. In our previous example, original_emails remains unchanged.

More list comprehensions

Here are a few other interesting examples of List comprehensions

names = ['John', 'Mary', 'Rose']

# Example 1:
duplicated_names = [n for n in names]
print(duplicated_names)  # ['John', 'Mary', 'Rose']

# Example 2:
first_letters = [n[0] for n in names]
print(first_letters)  # ['J', 'M', 'R']

# Example 3:
upper_names = [n.upper() for n in names]
print(upper_names)  # ['JOHN', 'MARY', 'ROSE']

Example 1

In this case we're just creating a new list containing the same elements from the original list. The transform function in this case is just the "identity function", returning the element itself. The transform function is: n (just the element itself) and we read:

Construct a new list with the same elements as the original.

Example 2

In this example we're just using the index notation to access the first character of the name. The transform function is: n[0] and we read:

Construct a new list containing the first character of each element in the original list.

Example 3

In this case we're just invoking the method upper in each one of the names of the collection. The transform function is: n.upper() and we read:

Construct a new list containing the uppercase representation of each element in the original list.

Conclusion and practice

As we said before, list comprehensions are a fundamental feature in the Python programming language. If you're struggling try to go back to the first concepts, make sure you understand them properly before jumping to list comprehensions. If you want to do some practice, check the assignments from the Collections unit in learn.rmotr.com.

Finally, as we'll see in the next ELI5, list comprehensions are more powerful than what you've seen here. They also include a filter part, which will allow you to filter specific items. Stay tuned!