FASTCALL microbenchmarks

For my FASTCALL project (CPython optimization avoiding temporary tuples and dictionaries to pass arguments), I wrote many short microbenchmarks. I grouped them into a new Git repository: pymicrobench. Benchmark results are required by CPython developers to prove that an optimization is worth it. It's not uncommon that I abandon a change because the speedup is not significant, makes CPython slower, or because the change is too complex. Last 12 months, I counted that I abandonned 9 optimization issues, rejected for different reasons, on a total of 46 optimization issues.

This article gives Python 3.7 results of these microbenchmarks compared to Python 3.5 (before FASTCALL). I ignored 3 microbenchmarks which are between 2% and 5% slower: the code was not optimized and the result is not signifiant (less than 10% on a microbenchmark is not significant).

On results below, the speedup is between 1.11x faster (-10%) and 1.92x faster (-48%). It's not easy to isolate the speedup of only FASTCALL. Since Python 3.5, Python 3.7 got many other optimizations.

Using FASTCALL gives a speedup around 20 ns: measured on a patch to use FASTCALL. It's not a lot, but many builtin functions take less than 100 ns, so 20 ns is significant in practice! Avoiding a tuple to pass positional arguments is interesting, but FASTCALL also allows further internal optimizations.

Microbenchmark on calling builtin functions:

Benchmark 3.5 3.7
struct.pack("i", 1) 105 ns 77.6 ns: 1.36x faster (-26%)
getattr(1, "real") 79.4 ns 64.4 ns: 1.23x faster (-19%)

Microbenchmark on calling methods of builtin types:

Benchmark 3.5 3.7
{1: 2}.get(7, None) 84.9 ns 61.6 ns: 1.38x faster (-27%)
collections.deque([None]).index(None) 116 ns 87.0 ns: 1.33x faster (-25%)
{1: 2}.get(1) 79.4 ns 59.6 ns: 1.33x faster (-25%)
"a".replace("x", "y") 134 ns 101 ns: 1.33x faster (-25%)
b"".decode() 71.5 ns 54.5 ns: 1.31x faster (-24%)
b"".decode("ascii") 99.1 ns 75.7 ns: 1.31x faster (-24%)
collections.deque.rotate(1) 106 ns 82.8 ns: 1.28x faster (-22%)
collections.deque.insert() 778 ns 608 ns: 1.28x faster (-22%)
b"".join((b"hello", b"world") * 100) 4.02 us 3.32 us: 1.21x faster (-17%)
[0].count(0) 53.9 ns 46.3 ns: 1.16x faster (-14%)
collections.deque.rotate() 72.6 ns 63.1 ns: 1.15x faster (-13%)
b"".join((b"hello", b"world")) 102 ns 89.8 ns: 1.13x faster (-12%)

Microbenchmark on builtin functions calling Python functions (callbacks):

Benchmark 3.5 3.7
map(lambda x: x, list(range(1000))) 76.1 us 61.1 us: 1.25x faster (-20%)
sorted(list(range(1000)), key=lambda x: x) 90.2 us 78.2 us: 1.15x faster (-13%)
filter(lambda x: x, list(range(1000))) 81.8 us 73.4 us: 1.11x faster (-10%)

Microbenchmark on calling slots (__getitem__, __init__, __int__) implemented in Python:

Benchmark 3.5 3.7
Python __getitem__: obj[0] 167 ns 87.0 ns: 1.92x faster (-48%)
call_pyinit_kw1 348 ns 240 ns: 1.45x faster (-31%)
call_pyinit_kw5 564 ns 401 ns: 1.41x faster (-29%)
call_pyinit_kw10 960 ns 734 ns: 1.31x faster (-24%)
Python __int__: int(obj) 241 ns 207 ns: 1.16x faster (-14%)

Microbenchmark on calling a method descriptor (static method):

Benchmark 3.5 3.7
int.to_bytes(1, 4, "little") 177 ns 103 ns: 1.72x faster (-42%)

Benchmarks were run on speed-python, server used to run CPython benchmarks.

blogroll

social