All posts All posts by this author Paper color Change page color Announcements

Python3 maps and list-comprehensions are not interchangable

It has been long since I wrote any blogs. In a world full of people screaming for attention, silence feels like death. By hiding my opinions, I forgot how to participate in healthy discussions. As such, I've made a decision to restart blogging and make my points known.

Is python a functional programming language? Some says yes, others say no. The creator of python, Guido considers functional aspects of python to be something of an after thought. Despite the dispute, python has many functional language features we all enjoy - (almost) first class functions, lambda functions, list comprehensions, data transducers (i.e. map, reduce, filter) etc. But if you always expect functional behavior from python, you will be burned.

Case in hand

You have read some great books about functional programming and learned that any list comprehension can be expressed as some combination of maps and filters. Let's say you have a simple list which needs to be squared. We can do it in idomatic python in two ways:

# Make sure to use python3.

data_in = [1, 2, 3] # Input list.

# Output made with list comprehension.
data_out_a = [x * x for x in data_in]

# Output made by mapping data_in with a lambda function.
data_out_b = map(lambda x: x * x, data_in)

# Let's print the outputs for fun. First round.
for x in data_out_a:
    print(x)
# >> 1
# >> 4
# >> 9

for x in data_out_b:
   print(x)
# >> 1
# >> 4
# >> 9

# So far so good. But we should try printing the outputs again.
for x in data_out_a:
    print(x)
# >> 1
# >> 4
# >> 9

for x in data_out_b:
   print(x)
# >>   

# What just happened?

As you can see, the second time we tried to use data_out_b in a for loop, nothing happened! I've been burned quite a few times by this seemingly unexpected behavior. Also, this script will run as expected in python2.

Curb your expectations

In python3, a list comprehension returns a list, but a map returns a map object which acts like a python generator. It has an internal state which produces finite number of elements for a finite input sequence and then just stops. If you try to get values from it a second time, it returns nothing.

Is it a bug in python3? I don't think so. The documentation of map says it returns an iterator and so it does. What is at fault is our expectations about python being functional. Sometimes it is adequately functional, othertimes it is not.

Software Solutions