Here are some performance hints I learned from doing a deep dive into Python for a work project.
Use map when performance matters AND the functions are complex AND you are using named functions. Use list comprehensions for everything else.
map is a built-in function written in C. Using map produces performance benefits over using list comprehensions in certain cases.
Please note that if you consume an anonymous lambda as your map function, rather than a named function, you lose the optimization benefits of map and it will in fact be much slower than an equivalent list comprehension. I will give you an example of this gotcha below.
To test the performance of these functions, we create an array with 10,000 numbers, and go through the array squaring each value. Pretty simple stuff. Check out the wild differences in runtime and performance:
List Comprehension with anonymous lambda: 5 function calls in 0.001 seconds
For Loop: 10005 function calls in 0.048 seconds
List Comprehension using named function: 10005 function calls in 0.049 seconds
map with named function: 10006 function calls in 0.050 seconds
Moral of the story? If you are doing simple list operations, use list comprehensions with anonymous lambdas. They are faster, more readable, and more pythonic.
When you’re munging complex data in Python, it’s a good idea to handle the data modification in a named function and then use map to call that function. You must always profile your code before and after using map to ensure that you are actually gaining performance and not losing it!
You might be asking so, when should I use map?
A good candidate for map is any long or complex function that will perform conditional operations on the provided arguments. map functions are great for iterating through objects and assigning properties based on data attributes, for example.
Here’s an example of map being significantly faster than list comprehensions (shamelessly taken from Stack Overflow):
Abuse try/except when necessary - but be careful
If you’re using inline try/except statements (where it’s no big deal if the try block fails), just attempt to do the thing you want to do, rather than using extraneous if statements.
Here’s some sample code and real profiling results to guide your decisions.
Our profiler results are below - using an if took 0.098 seconds - using only try/except shaved off one-third of the compute time, down to 0.065 seconds
Notice that our function using if incurs twice as many function calls as our plain old try/except block.