Benchmarking Elixir and Erlang database drivers
I’ve never used an ORM before, mainly because I’ve had some previous experience watching people use Hibernate and it scared the bejeebus out of me. It seems easier to just write the SQL and there are some performance advantages to do so. Was planning to try Ecto, so at this point, it’s a good time to do some very simple benchmarks for select. So we’ll be benchmarking ecto, postgrex, epgsql, mariaex, and emysql, and mysql-otp.
** Updated benchmarks at this link **
We’ll be using a connection pool of 10 with an overflow of 10 and selecting a single row out of a table. All results are being compiled between two lxc guests on a laptop.
- eredis (1.0.7) - 12000 qps
- mysql-otp - 10500 qps
- epgsql (git:epgsql/epgsql) - 8350 qps
- emysql (git:Eonblast/Emysql) - 7200 qps
- postgrex (0.8.1) - 5800 qps
- ecto (0.12.0-rc) with postgrex (0.8.1) - 5200 qps
- ecto (0.12.0-rc) with mariaex (0.1.6) - 500 qps
Ecto seems to be giving about 85%-90% of the performance of the raw Postgrex. However, the erlang epgsql driver is a bit more mature and about 30% faster. That’s a lot. Interestingly new mysql-otp driver which is still in beta is leading the way over the older emysql.
Regarding Ecto, I wouldn’t mind using it as it seems to come with a lot of free functionality. Migrations, allows setting up tests easily, cleans up queries against injection, at 90% of the speed of the database driver. The problem is that the Elixir driver seems to be less mature than the Erlang version for now. Competition is good, and the more options the better.
The raw results are below and the source code for this is on github.
./wrk -c 100 -d 10 -t 20 http://localhost:8800/redis Running 10s test @ http://localhost:8800/redis 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 8.29ms 1.70ms 32.23ms 88.41% Req/Sec 607.19 54.48 750.00 73.20% 121048 requests in 10.02s, 25.74MB read Requests/sec: 12078.89 Transfer/sec: 2.57MB ./wrk -c 100 -d 10 -t 20 http://localhost:8800/mysqlotp Running 10s test @ http://localhost:8800/mysqlotp 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 9.71ms 3.62ms 82.28ms 95.15% Req/Sec 527.80 61.73 1.01k 81.10% 105228 requests in 10.04s, 22.78MB read Requests/sec: 10483.41 Transfer/sec: 2.27MB ./wrk -c 100 -d 10 -t 20 http://localhost:8800/epgsql Running 10s test @ http://localhost:8800/epgsql 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 12.30ms 5.73ms 131.06ms 97.26% Req/Sec 420.66 49.37 505.00 86.87% 83852 requests in 10.05s, 17.91MB read Requests/sec: 8347.20 Transfer/sec: 1.78MB ./wrk -c 100 -d 10 -t 20 http://localhost:8800/emysql Running 10s test @ http://localhost:8800/emysql 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 14.17ms 5.65ms 113.59ms 96.04% Req/Sec 362.98 46.54 636.00 86.58% 72702 requests in 10.10s, 15.46MB read Requests/sec: 7198.07 Transfer/sec: 1.53MB ./wrk -c 100 -d 10 -t 20 http://localhost:8800/postgrex Running 10s test @ http://localhost:8800/postgrex 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 21.07ms 29.50ms 377.27ms 97.36% Req/Sec 297.64 40.85 370.00 77.03% 58262 requests in 10.04s, 12.61MB read Requests/sec: 5803.25 Transfer/sec: 1.26MB ./wrk -c 100 -d 10 -t 20 http://localhost:8800/ecto Running 10s test @ http://localhost:8800/ecto 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 21.20ms 20.01ms 372.78ms 97.93% Req/Sec 264.36 38.43 323.00 86.00% 52236 requests in 10.05s, 11.06MB read Requests/sec: 5200.08 Transfer/sec: 1.10MB