Comparing cloud VPS

Introduction

Cloud servers are amazing. Devs can easily spin up a server with various CPU/Memory combinations and shut them down when done. Entire companies can be built with a couple of VPS, a load balancer and a managed database.

The three big cloud providers are AWS, Azure, and GCP but I don’t think they are the best fit for a solo dev bootstrapping their project or trying to host their apps. For indie devs, DigitalOcean, Linode, or Vultr are much better options because of their simplicity and better pricing.

I benchmarked various things across the server fleets provided by each provider to get a better feel of their performance. The following are the findings - I hope they help you make an informed decision around performance of the servers from each provider.

Linode

Shared 1 cpu 1gb ram

CPU info

processor : 0
vendor_id : AuthenticAMD
cpu family : 23
model : 1
model name : AMD EPYC 7601 32-Core Processor
stepping : 2
microcode : 0x1000065
cpu MHz : 2199.980
cache size : 512 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw perfctr_core ssbd ibpb vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 virt_ssbd arat arch_capabilities
bugs : fxsave_leak sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4399.96
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

Sysbench CPU

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time

Prime numbers limit: 20000

Initializing worker threads…

Threads started!

CPU speed:
events per second: 462.93

General statistics:
total time: 10.0017s
total number of events: 4631

Latency (ms):
min: 1.94
avg: 2.16
max: 5.14
95th percentile: 2.39
sum: 9988.11

Threads fairness:
events (avg/stddev): 4631.0000/0.00
execution time (avg/stddev): 9.9881/0.00

Sysbench File IO

sysbench –test=fileio –file-total-size=4G prepare

4294967296 bytes written in 8.10 seconds (505.94 MiB/sec).

sysbench –test=fileio –file-total-size=4G –file-test-mode=rndrw –max-time=300 –max-requests=0 run

sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time

Extra file open flags: (none)
128 files, 32MiB each
4GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads…

Threads started!

File operations:
reads/s: 1228.95
writes/s: 819.30
fsyncs/s: 2622.12

Throughput:
read, MiB/s: 19.20
written, MiB/s: 12.80

General statistics:
total time: 300.0156s
total number of events: 1401073

Latency (ms):
min: 0.00
avg: 0.21
max: 242.09
95th percentile: 0.59
sum: 298457.86

Threads fairness:
events (avg/stddev): 1401073.0000/0.00
execution time (avg/stddev): 298.4579/0.00

Postgres test fsync

$/usr/lib/postgresql/13/bin/pg_test_fsync

5 seconds per test
O_DIRECT supported on this platform for open_datasync and open_sync.

Compare file sync methods using one 8kB write:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync 4982.518 ops/sec 201 usecs/op
fdatasync 3670.589 ops/sec 272 usecs/op
fsync 3818.079 ops/sec 262 usecs/op
fsync_writethrough n/a
open_sync 4287.696 ops/sec 233 usecs/op

Compare file sync methods using two 8kB writes:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync 2722.915 ops/sec 367 usecs/op
fdatasync 4732.138 ops/sec 211 usecs/op
fsync 3806.536 ops/sec 263 usecs/op
fsync_writethrough n/a
open_sync 1171.717 ops/sec 853 usecs/op

Compare opensync with different write sizes:
(This is designed to compare the cost of writing 16kB in different write
open_sync sizes.)
1 * 16kB opensync write 4278.156 ops/sec 234 usecs/op
2 * 8kB open
sync writes 2423.489 ops/sec 413 usecs/op
4 * 4kB open*sync writes 1102.054 ops/sec 907 usecs/op
8 * 2kB open_sync writes
504.984 ops/sec 1980 usecs/op
16 \
1kB open_sync writes 289.378 ops/sec 3456 usecs/op

Test if fsync on non-write file descriptor is honored:
(If the times are similar, fsync() can sync data written on a different
descriptor.)
write, fsync, close 4496.443 ops/sec 222 usecs/op
write, close, fsync 3469.971 ops/sec 288 usecs/op

Non-sync’ed 8kB writes:
write 181305.641 ops/sec 6 usecs/op

2cpu 4gb ram

CPU info

root@localhost:~# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 23
model : 1
model name : AMD EPYC 7601 32-Core Processor
stepping : 2
microcode : 0x1000065
cpu MHz : 2199.996
cache size : 512 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw perfctr_core ssbd ibpb vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 virt_ssbd arat arch_capabilities
bugs : fxsave_leak sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4399.99
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

Sysbench CPU

sysbench –test=cpu –cpu-max-prime=20000 run –threads=2

sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 2
Initializing random number generator from current time

Prime numbers limit: 20000

Initializing worker threads…

Threads started!

CPU speed:
events per second: 526.34

General statistics:
total time: 10.0032s
total number of events: 5266

Latency (ms):
min: 2.07
avg: 3.80
max: 8.47
95th percentile: 3.82
sum: 20001.82

Threads fairness:
events (avg/stddev): 2633.0000/7.00
execution time (avg/stddev): 10.0009/0.00

Sysbench File IO

sysbench –test=fileio –file-total-size=8G prepare

8589934592 bytes written in 22.38 seconds (365.99 MiB/sec).

sysbench –test=fileio –file-total-size=8G –file-test-mode=rndrw –max-time=300 –max-requests=0 run –thread=2

Running the test with following options:
Number of threads: 2
Initializing random number generator from current time

Extra file open flags: (none)
128 files, 64MiB each
8GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads…

Threads started!

File operations:
reads/s: 3678.19
writes/s: 2452.12
fsyncs/s: 7847.51

Throughput:
read, MiB/s: 57.47
written, MiB/s: 38.31

General statistics:
total time: 300.0206s
total number of events: 4193410

Latency (ms):
min: 0.00
avg: 0.14
max: 136.02
95th percentile: 0.42
sum: 597957.02

Threads fairness:
events (avg/stddev): 2096705.0000/633.00
execution time (avg/stddev): 298.9785/0.01

Postgres test fsync

5 seconds per test
O_DIRECT supported on this platform for open_datasync and open_sync.

Compare file sync methods using one 8kB write:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync 5767.306 ops/sec 173 usecs/op
fdatasync 5825.664 ops/sec 172 usecs/op
fsync 5391.464 ops/sec 185 usecs/op
fsync_writethrough n/a
open_sync 4703.280 ops/sec 213 usecs/op

Compare file sync methods using two 8kB writes:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync 2911.707 ops/sec 343 usecs/op
fdatasync 5141.467 ops/sec 194 usecs/op
fsync 5574.678 ops/sec 179 usecs/op
fsync_writethrough n/a
open_sync 2950.213 ops/sec 339 usecs/op

Compare opensync with different write sizes:
(This is designed to compare the cost of writing 16kB in different write
open_sync sizes.)
1 * 16kB opensync write 5838.724 ops/sec 171 usecs/op
2 * 8kB open
sync writes 2772.436 ops/sec 361 usecs/op
4 * 4kB open*sync writes 1473.026 ops/sec 679 usecs/op
8 * 2kB open_sync writes 751.673 ops/sec 1330 usecs/op
16 \
1kB open_sync writes 372.101 ops/sec 2687 usecs/op

Test if fsync on non-write file descriptor is honored:
(If the times are similar, fsync() can sync data written on a different
descriptor.)
write, fsync, close 5274.601 ops/sec 190 usecs/op
write, close, fsync 6451.208 ops/sec 155 usecs/op

Non-sync’ed 8kB writes:
write 283026.385 ops/sec 4 usecs/op

64cpu 512gb

CPU info

root@localhost:~# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 23
model : 49
model name : AMD EPYC 7642 48-Core Processor
stepping : 0
microcode : 0x1000065
cpu MHz : 2299.998
cache size : 512 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 16
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves wbnoinvd arat umip rdpid arch_capabilities
bugs : fxsave_leak sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass
bogomips : 4599.99
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40

Sysbench CPU

sysbench –test=cpu –cpu-max-prime=20000 run –threads=64
WARNING: the –test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 64
Initializing random number generator from current time

Prime numbers limit: 20000

Initializing worker threads…

Threads started!

CPU speed:
events per second: 20425.29

General statistics:
total time: 10.0031s
total number of events: 204336

Latency (ms):
min: 1.60
avg: 3.13
max: 63.05
95th percentile: 3.13
sum: 639395.99

Threads fairness:
events (avg/stddev): 3192.7500/472.99
execution time (avg/stddev): 9.9906/0.02

Sysbench File IO

sysbench –test=fileio –file-total-size=550G prepare –threads=64

479917490176 bytes written in 1311.01 seconds (349.11 MiB/sec).

sysbench –test=fileio –file-total-size=550G –file-test-mode=rndrw –max-time=300 –max-requests=0 run –threads=64

Number of threads: 64
Initializing random number generator from current time

Extra file open flags: (none)
128 files, 4.2969GiB each
550GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads…

Threads started!

File operations:
reads/s: 18617.88
writes/s: 12411.95
fsyncs/s: 38577.39

Throughput:
read, MiB/s: 290.90
written, MiB/s: 193.94

General statistics:
total time: 300.0393s
total number of events: 20876780

Latency (ms):
min: 0.00
avg: 0.92
max: 98.86
95th percentile: 3.82
sum: 19145525.56

Threads fairness:
events (avg/stddev): 326199.6875/980.25
execution time (avg/stddev): 299.1488/0.02

Postgres test fsync

N/A

Vultr

1cpu 1gb ram - hfc

CPU info

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 94
model name : Intel Core Processor (Skylake, IBRS)
stepping : 3
microcode : 0x1
cpu MHz : 3792.000
cache size : 16384 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm cpuid_fault invpcid_single pti ssbd ibrs ibpb fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit srbds
bogomips : 7584.00
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual

Sysbench CPU

sysbench –test=cpu –cpu-max-prime=20000 run

Prime numbers limit: 20000

Initializing worker threads…

Threads started!

CPU speed:
events per second: 583.01

General statistics:
total time: 10.0005s
total number of events: 5831

Latency (ms):
min: 1.66
avg: 1.71
max: 5.17
95th percentile: 1.79
sum: 9991.61

Threads fairness:
events (avg/stddev): 5831.0000/0.00
execution time (avg/stddev): 9.9916/0.00

Sysbench File IO

sysbench –test=fileio –file-total-size=16G prepare
17179869184 bytes written in 13.22 seconds (1239.56 MiB/sec).
sysbench –test=fileio –file-total-size=16G –file-test-mode=rndrw –max-time=300 –max-requests=0 run

sysbench 1.0.18 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time

Extra file open flags: (none)
128 files, 128MiB each
16GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads…

Threads started!

File operations:
reads/s: 4498.49
writes/s: 2998.99
fsyncs/s: 9596.77

Throughput:
read, MiB/s: 70.29
written, MiB/s: 46.86

General statistics:
total time: 300.0065s
total number of events: 5128276

Latency (ms):
min: 0.00
avg: 0.06
max: 51.53
95th percentile: 0.15
sum: 297547.46

Threads fairness:
events (avg/stddev): 5128276.0000/0.00
execution time (avg/stddev): 297.5475/0.00

Postgres test fsync

root@counter-client:~# /usr/lib/postgresql/12/bin/pg_test_fsync
5 seconds per test
O_DIRECT supported on this platform for open_datasync and open_sync.

Compare file sync methods using one 8kB write:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync
13609.031 ops/sec 73 usecs/op
fdatasync 13900.469 ops/sec 72 usecs/op
fsync 13775.537 ops/sec 73 usecs/op
fsync_writethrough n/a
open_sync 15541.742 ops/sec 64 usecs/op

Compare file sync methods using two 8kB writes:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync
7932.627 ops/sec 126 usecs/op
fdatasync 13838.045 ops/sec 72 usecs/op
fsync 12999.543 ops/sec 77 usecs/op
fsync_writethrough n/a
open_sync 7724.935 ops/sec 129 usecs/op

Compare opensync with different write sizes:
(This is designed to compare the cost of writing 16kB in different write
open_sync sizes.)
1 * 16kB opensync write 14688.183 ops/sec 68 usecs/op
2 * 8kB open
sync writes 7982.079 ops/sec 125 usecs/op
4 * 4kB open*sync writes 4213.327 ops/sec 237 usecs/op
8 * 2kB open_sync writes 1699.077 ops/sec 589 usecs/op
16 \
1kB open_sync writes 816.131 ops/sec 1225 usecs/op

Test if fsync on non-write file descriptor is honored:
(If the times are similar, fsync() can sync data written on a different
descriptor.)
write, fsync, close 12929.535 ops/sec 77 usecs/op
write, close, fsync 14454.554 ops/sec 69 usecs/op

Non-sync’ed 8kB writes:
write 395160.931 ops/sec 3 usecs/op

6cpu 24gb ram - hfc

CPU info

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 94
model name : Intel Core Processor (Skylake, IBRS)
stepping : 3
microcode : 0x1
cpu MHz : 3696.000
cache size : 16384 KB
physical id : 0
siblings : 6
core id : 0
cpu cores : 3
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm cpuid_fault invpcid_single pti ssbd ibrs ibpb fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit srbds
bogomips : 7392.00
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual

Sysbench CPU

sysbench –test=cpu –cpu-max-prime=20000 run

Number of threads: 1
Initializing random number generator from current time

Prime numbers limit: 20000

Initializing worker threads…

Threads started!

CPU speed:
events per second: 578.74

General statistics:
total time: 10.0016s
total number of events: 5789

Latency (ms):
min: 1.66
avg: 1.73
max: 2.41
95th percentile: 1.86
sum: 9992.89

Threads fairness:
events (avg/stddev): 5789.0000/0.00
execution time (avg/stddev): 9.9929/0.00

sysbench –test=cpu –cpu-max-prime=20000 run –num-threads=6
WARNING: the –test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: –num-threads is deprecated, use –threads instead
sysbench 1.0.18 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:

Number of threads: 6
Initializing random number generator from current time

Prime numbers limit: 20000

Initializing worker threads…

Threads started!

CPU speed:
events per second: 3181.16

General statistics:
total time: 10.0015s
total number of events: 31820

Latency (ms):
min: 1.66
avg: 1.89
max: 10.17
95th percentile: 2.07
sum: 59981.45

Threads fairness:
events (avg/stddev): 5303.3333/88.13
execution time (avg/stddev): 9.9969/0.00

Sysbench File IO

sysbench –test=fileio –file-total-size=30G prepare

15032385536 bytes written in 47.08 seconds (304.47 MiB/sec).

sysbench –test=fileio –file-total-size=30G –file-test-mode=rndrw –max-time=300 –max-requests=0 run
sysbench 1.0.18 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time

Extra file open flags: (none)
128 files, 240MiB each
30GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads…

Threads started!

File operations:
reads/s: 6388.43
writes/s: 4258.95
fsyncs/s: 13628.72

Throughput:
read, MiB/s: 99.82
written, MiB/s: 66.55

General statistics:
total time: 300.0053s
total number of events: 7282857

Latency (ms):
min: 0.00
avg: 0.04
max: 167.54
95th percentile: 0.14
sum: 297085.20

Threads fairness:
events (avg/stddev): 7282857.0000/0.00
execution time (avg/stddev): 297.0852/0.00

Postgres test fsync

root@counter-client:~# /usr/lib/postgresql/12/bin/pg_test_fsync
5 seconds per test
O_DIRECT supported on this platform for open_datasync and open_sync.

Compare file sync methods using one 8kB write:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync 13776.868 ops/sec 73 usecs/op
fdatasync 12903.080 ops/sec 78 usecs/op
fsync 12157.756 ops/sec 82 usecs/op
fsync_writethrough n/a
open_sync 12850.848 ops/sec 78 usecs/op

Compare file sync methods using two 8kB writes:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync 7063.009 ops/sec 142 usecs/op
fdatasync 11983.042 ops/sec 83 usecs/op
fsync 10754.570 ops/sec 93 usecs/op
fsync_writethrough n/a
open_sync 6308.772 ops/sec 159 usecs/op

Compare opensync with different write sizes:
(This is designed to compare the cost of writing 16kB in different write
open_sync sizes.)
1 * 16kB opensync write 12056.793 ops/sec 83 usecs/op
2 * 8kB open
sync writes 6629.434 ops/sec 151 usecs/op
4 * 4kB open*sync writes 2552.389 ops/sec 392 usecs/op
8 * 2kB open_sync writes 1248.317 ops/sec 801 usecs/op
16 \
1kB open_sync writes 821.483 ops/sec 1217 usecs/op

Test if fsync on non-write file descriptor is honored:
(If the times are similar, fsync() can sync data written on a different
descriptor.)
write, fsync, close 12865.907 ops/sec 78 usecs/op
write, close, fsync 12847.286 ops/sec 78 usecs/op

Non-sync’ed 8kB writes:
write 359506.993 ops/sec 3 usecs/op

4cpu/8threads 32gb ram - Bare metal

CPU info

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz
stepping : 9
microcode : 0xea
cpu MHz : 800.455
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit srbds
bogomips : 7599.80
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual

Sysbench CPU

sysbench –test=cpu –cpu-max-prime=20000 run –num-threads=8

sysbench 1.0.18 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 8
Initializing random number generator from current time

Prime numbers limit: 20000

Initializing worker threads…

Threads started!

CPU speed:
events per second: 3217.42

General statistics:
total time: 10.0024s
total number of events: 32186

Latency (ms):
min: 2.15
avg: 2.49
max: 10.63
95th percentile: 2.48
sum: 80005.91

Threads fairness:
events (avg/stddev): 4023.2500/3.63
execution time (avg/stddev): 10.0007/0.00

Sysbench File IO

sysbench –test=fileio –file-total-size=40G prepare

42949672960 bytes written in 173.60 seconds (235.94 MiB/sec).

sysbench –test=fileio –file-total-size=40G –file-test-mode=rndrw –max-time=300 –max-requests=0 run
sysbench 1.0.18 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 8
Initializing random number generator from current time

Extra file open flags: (none)
128 files, 320MiB each
40GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads…

Threads started!

File operations:
reads/s: 10307.02
writes/s: 6871.35
fsyncs/s: 21991.36

Throughput:
read, MiB/s: 161.05
written, MiB/s: 107.36

General statistics:
total time: 300.0213s
total number of events: 11750782

Latency (ms):
min: 0.00
avg: 0.20
max: 10.91
95th percentile: 0.95
sum: 2394693.32

Threads fairness:
events (avg/stddev): 1468847.7500/3955.65
execution time (avg/stddev): 299.3367/0.00

Postgres test fsync

(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync 4340.421 ops/sec 230 usecs/op
fdatasync 4404.267 ops/sec 227 usecs/op
fsync 3266.741 ops/sec 306 usecs/op
fsync_writethrough n/a
open_sync 4110.526 ops/sec 243 usecs/op

Compare file sync methods using two 8kB writes:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync 2079.733 ops/sec 481 usecs/op
fdatasync 2145.310 ops/sec 466 usecs/op
fsync 1863.995 ops/sec 536 usecs/op
fsync_writethrough n/a
open_sync 2054.896 ops/sec 487 usecs/op

Compare opensync with different write sizes:
(This is designed to compare the cost of writing 16kB in different write
open_sync sizes.)
1 * 16kB opensync write 2008.279 ops/sec 498 usecs/op
2 * 8kB open
sync writes 1894.993 ops/sec 528 usecs/op
4 * 4kB open*sync writes 1413.020 ops/sec 708 usecs/op
8 * 2kB open_sync writes 119.911 ops/sec 8340 usecs/op
16 \
1kB open_sync writes 47.912 ops/sec 20872 usecs/op

Test if fsync on non-write file descriptor is honored:
(If the times are similar, fsync() can sync data written on a different
descriptor.)
write, fsync, close 4304.298 ops/sec 232 usecs/op
write, close, fsync 4323.522 ops/sec 231 usecs/op

Non-sync’ed 8kB writes:
write 363031.964 ops/sec 3 usecs/op

1gb cpu 1gb ram

CPU info

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 85
model name : Intel Xeon Processor (Skylake, IBRS)
stepping : 4
microcode : 0x1
cpu MHz : 2593.904
cache size : 16384 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm cpuid_fault invpcid_single pti ssbd ibrs ibpb fsgsbase bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq clwb avx512cd avx512bw avx512vl xsaveopt arat pku ospke
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips : 5187.80
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual

Sysbench CPU

sysbench –test=cpu –cpu-max-prime=20000 run

Number of threads: 1
Initializing random number generator from current time

Prime numbers limit: 20000

Initializing worker threads…

Threads started!

CPU speed:
events per second: 411.41

General statistics:
total time: 10.0004s
total number of events: 4115

Latency (ms):
min: 2.37
avg: 2.43
max: 6.29
95th percentile: 2.52
sum: 9994.44

Threads fairness:
events (avg/stddev): 4115.0000/0.00
execution time (avg/stddev): 9.9944/0.00

Sysbench File IO

sysbench –test=fileio –file-total-size=16G prepare

17179869184 bytes written in 47.43 seconds (345.44 MiB/sec).

sysbench –test=fileio –file-total-size=16G –file-test-mode=rndrw –max-time=300 –max-requests=0 run

sysbench 1.0.18 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time

Extra file open flags: (none)
128 files, 128MiB each
16GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads…

Threads started!

File operations:
reads/s: 3095.89
writes/s: 2063.93
fsyncs/s: 6604.65

Throughput:
read, MiB/s: 48.37
written, MiB/s: 32.25

General statistics:
total time: 300.0090s
total number of events: 3529338

Latency (ms):
min: 0.00
avg: 0.08
max: 106.93
95th percentile: 0.20
sum: 298414.82

Threads fairness:
events (avg/stddev): 3529338.0000/0.00
execution time (avg/stddev): 298.4148/0.00

Postgres test fsync

/usr/lib/postgresql/12/bin/pg_test_fsync

root@counter-client-5:~# /usr/lib/postgresql/12/bin/pg_test_fsync
5 seconds per test
O_DIRECT supported on this platform for open_datasync and open_sync.

Compare file sync methods using one 8kB write:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync 9259.756 ops/sec 108 usecs/op
fdatasync 9078.249 ops/sec 110 usecs/op
fsync 8370.621 ops/sec 119 usecs/op
fsync_writethrough n/a
open_sync 9016.865 ops/sec 111 usecs/op

Compare file sync methods using two 8kB writes:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync 4615.456 ops/sec 217 usecs/op
fdatasync 7648.041 ops/sec 131 usecs/op
fsync 7199.122 ops/sec 139 usecs/op
fsync_writethrough n/a
open_sync 4520.329 ops/sec 221 usecs/op

Compare opensync with different write sizes:
(This is designed to compare the cost of writing 16kB in different write
open_sync sizes.)
1 * 16kB opensync write 7553.132 ops/sec 132 usecs/op
2 * 8kB open
sync writes 4483.100 ops/sec 223 usecs/op
4 * 4kB open*sync writes 2614.184 ops/sec 383 usecs/op
8 * 2kB open_sync writes 1387.879 ops/sec 721 usecs/op
16 \
1kB open_sync writes 677.864 ops/sec 1475 usecs/op

Test if fsync on non-write file descriptor is honored:
(If the times are similar, fsync() can sync data written on a different
descriptor.)
write, fsync, close 8376.666 ops/sec 119 usecs/op
write, close, fsync 8575.923 ops/sec 117 usecs/op

Non-sync’ed 8kB writes:
write 306764.248 ops/sec 3 usecs/op

24 cpu 96gb ram

CPU info

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 85
model name : Intel Xeon Processor (Cascadelake)
stepping : 6
microcode : 0x1
cpu MHz : 2992.968
cache size : 16384 KB
physical id : 0
siblings : 12
core id : 0
cpu cores : 6
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm cpuid_fault invpcid_single pti ssbd ibrs ibpb fsgsbase bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq clflushopt clwb avx512cd avx512bw avx512vl xsaveopt arat pku ospke avx512_vnni
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips : 5985.93
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual

Sysbench CPU

sysbench –test=cpu –cpu-max-prime=20000 run –threads=24

WARNING: the –test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.0.18 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 24
Initializing random number generator from current time

Prime numbers limit: 20000

Initializing worker threads…

Threads started!

CPU speed:
events per second: 10328.75

General statistics:
total time: 10.0024s
total number of events: 103331

Latency (ms):
min: 2.17
avg: 2.32
max: 9.09
95th percentile: 2.66
sum: 239818.80

Threads fairness:
events (avg/stddev): 4305.4583/60.28
execution time (avg/stddev): 9.9925/0.01

Sysbench File IO

sysbench –test=fileio –file-total-size=120G prepare –threads=24

119222452224 bytes written in 130.50 seconds (871.24 MiB/sec).

sysbench –test=fileio –file-total-size=120G –file-test-mode=rndrw –max-time=300 –max-requests=0 run –threads=24

Running the test with following options:
Number of threads: 24
Initializing random number generator from current time

Extra file open flags: (none)
128 files, 960MiB each
120GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads…

Threads started!

File operations:
reads/s: 35913.67
writes/s: 23942.41
fsyncs/s: 76625.98

Throughput:
read, MiB/s: 561.15
written, MiB/s: 374.10

General statistics:
total time: 300.0160s
total number of events: 40944000

Latency (ms):
min: 0.00
avg: 0.17
max: 12.80
95th percentile: 0.60
sum: 7134621.12

Threads fairness:
events (avg/stddev): 1706000.0000/3917.88
execution time (avg/stddev): 297.2759/0.02

Postgres test fsync

5 seconds per test
O_DIRECT supported on this platform for open_datasync and open_sync.

Compare file sync methods using one 8kB write:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync 11673.252 ops/sec 86 usecs/op
fdatasync 11016.637 ops/sec 91 usecs/op
fsync 10199.563 ops/sec 98 usecs/op
fsync_writethrough n/a
open_sync 10496.990 ops/sec 95 usecs/op

Compare file sync methods using two 8kB writes:
(in wal_sync_method preference order, except fdatasync is Linux’s default)
open_datasync 5541.859 ops/sec 180 usecs/op
fdatasync 10443.372 ops/sec 96 usecs/op
fsync 9727.332 ops/sec 103 usecs/op
fsync_writethrough n/a
open_sync 5304.463 ops/sec 189 usecs/op

Compare opensync with different write sizes:
(This is designed to compare the cost of writing 16kB in different write
open_sync sizes.)
1 * 16kB opensync write 9905.875 ops/sec 101 usecs/op
2 * 8kB open
sync writes 5111.736 ops/sec 196 usecs/op
4 * 4kB open*sync writes 2745.272 ops/sec 364 usecs/op
8 * 2kB open_sync writes 1193.440 ops/sec 838 usecs/op
16 \
1kB open_sync writes 625.795 ops/sec 1598 usecs/op

Test if fsync on non-write file descriptor is honored:
(If the times are similar, fsync() can sync data written on a different
descriptor.)
write, fsync, close 9809.055 ops/sec 102 usecs/op
write, close, fsync 10252.799 ops/sec 98 usecs/op

Non-sync’ed 8kB writes:
write 263293.998 ops/sec 4 usecs/op

DigitalOcean

I lost this data and am too tired to rerun all the tests again.

Conclusion

Don’t let the numbers hold you back from starting your next software product. It is extremely unlikely to make or break it. Just pick a provider and deploy your dream.