Selaa lähdekoodia

Updated benchmarks for Go 1.8.1, updated contamination effect discussion.

Peter H. Froehlich 8 vuotta sitten
vanhempi
commit
955741edec
2 muutettua tiedostoa jossa 62 lisäystä ja 15 poistoa
  1. 49 2
      BENCH.md
  2. 13 13
      README.md

+ 49 - 2
BENCH.md

@@ -1,8 +1,8 @@
 # Benchmarks and Test Cases
 
-It seems that Go's otherwise excellent
+It seems that *some* versions of Go's otherwise excellent
 [testing](https://golang.org/pkg/testing/)
-infrastructure is susceptible to certain "contamination effects"
+infrastructure are susceptible to certain "contamination effects"
 in which the presence (or absence) of *test cases* influences the
 performance measured by *benchmarks*.
 Check it out:
@@ -35,6 +35,8 @@ PushBackList-2        2.05k ± 0%     2.05k ± 0%     ~     (all equal)
 PushBackChannel-2     1.03k ± 0%     1.03k ± 0%     ~     (all equal)
 RandomQueue-2         1.63k ± 0%     1.63k ± 0%     ~     (all equal)
 RandomList-2          3.24k ± 0%     3.24k ± 0%     ~     (all equal)
+$ go version
+go version go1.7.5 linux/amd64
 ```
 
 The *only* difference between `old time` and `new time` is that the
@@ -46,3 +48,48 @@ everything is "back to normal" as it were.
 For now I am reporting the *worse* results in the official `README` just
 to be conservative, but I believe something needs to be fixed on the Go
 side of things.
+
+**Update**: Something *was* apparently fixed in a more recent version
+of Go. Check it out:
+
+```
+$ benchstat without.txt with.txt
+name               old time/op    new time/op    delta
+PushFrontQueue-2     82.7µs ± 1%    82.8µs ± 1%    ~     (p=0.832 n=19+16)
+PushFrontList-2       161µs ± 1%     162µs ± 1%  +0.48%  (p=0.003 n=19+18)
+PushBackQueue-2      83.3µs ± 1%    83.4µs ± 1%    ~     (p=0.756 n=19+16)
+PushBackList-2        156µs ± 1%     158µs ± 3%  +1.03%  (p=0.024 n=19+20)
+PushBackChannel-2     110µs ± 2%     110µs ± 2%    ~     (p=0.341 n=20+20)
+RandomQueue-2         158µs ± 4%     161µs ± 2%  +2.34%  (p=0.000 n=20+20)
+RandomList-2          279µs ± 1%     281µs ± 4%    ~     (p=0.223 n=19+19)
+GrowShrinkQueue-2     111µs ± 2%     110µs ± 1%  -1.39%  (p=0.000 n=18+19)
+GrowShrinkList-2      168µs ± 1%     170µs ± 5%  +1.21%  (p=0.017 n=19+20)
+
+name               old alloc/op   new alloc/op   delta
+PushFrontQueue-2     40.9kB ± 0%    40.9kB ± 0%    ~     (all equal)
+PushFrontList-2      57.4kB ± 0%    57.4kB ± 0%    ~     (all equal)
+PushBackQueue-2      40.9kB ± 0%    40.9kB ± 0%    ~     (all equal)
+PushBackList-2       57.4kB ± 0%    57.4kB ± 0%    ~     (all equal)
+PushBackChannel-2    24.7kB ± 0%    24.7kB ± 0%    ~     (all equal)
+RandomQueue-2        45.7kB ± 0%    45.7kB ± 0%    ~     (all equal)
+RandomList-2         90.8kB ± 0%    90.8kB ± 0%    ~     (all equal)
+GrowShrinkQueue-2    57.2kB ± 0%    57.2kB ± 0%    ~     (all equal)
+GrowShrinkList-2     57.4kB ± 0%    57.4kB ± 0%    ~     (all equal)
+
+name               old allocs/op  new allocs/op  delta
+PushFrontQueue-2      1.03k ± 0%     1.03k ± 0%    ~     (all equal)
+PushFrontList-2       2.05k ± 0%     2.05k ± 0%    ~     (all equal)
+PushBackQueue-2       1.03k ± 0%     1.03k ± 0%    ~     (all equal)
+PushBackList-2        2.05k ± 0%     2.05k ± 0%    ~     (all equal)
+PushBackChannel-2     1.03k ± 0%     1.03k ± 0%    ~     (all equal)
+RandomQueue-2         1.63k ± 0%     1.63k ± 0%    ~     (all equal)
+RandomList-2          3.24k ± 0%     3.24k ± 0%    ~     (all equal)
+GrowShrinkQueue-2     1.04k ± 0%     1.04k ± 0%    ~     (all equal)
+GrowShrinkList-2      2.05k ± 0%     2.05k ± 0%    ~     (all equal)
+$ go version
+go version go1.8.1 linux/amd64
+```
+
+The additional test cases still have *some* effect on the benchmarks, but
+it's nowhere near as extreme as it was before.
+I am not perfectly happy with this, but I am happy enough.

+ 13 - 13
README.md

@@ -44,7 +44,7 @@ insertion.
 **Please read
 [`BENCH.md`](https://github.com/phf/go-queue/blob/master/BENCH.md)
 for some perspective.
-The numbers below are most likely "contaminated" in a way that makes
+Some numbers below are most likely "contaminated" in a way that makes
 our queues appear *worse* than they are.**
 
 Here are the numbers for my (ancient) home machine:
@@ -53,15 +53,15 @@ Here are the numbers for my (ancient) home machine:
 $ go test -bench=. -benchmem -count=10 >bench.txt
 $ benchstat bench.txt
 name               time/op
-PushFrontQueue-2   97.7µs ± 1%
-PushFrontList-2     163µs ± 1%
-PushBackQueue-2    98.0µs ± 1%
-PushBackList-2      165µs ± 3%
-PushBackChannel-2   145µs ± 1%
-RandomQueue-2       172µs ± 1%
-RandomList-2        292µs ± 1%
-GrowShrinkQueue-2   121µs ± 1%
-GrowShrinkList-2    174µs ± 1%
+PushFrontQueue-2   82.8µs ± 1%
+PushFrontList-2     162µs ± 1%
+PushBackQueue-2    83.4µs ± 1%
+PushBackList-2      158µs ± 3%
+PushBackChannel-2   110µs ± 2%
+RandomQueue-2       161µs ± 2%
+RandomList-2        281µs ± 4%
+GrowShrinkQueue-2   110µs ± 1%
+GrowShrinkList-2    170µs ± 5%
 
 name               alloc/op
 PushFrontQueue-2   40.9kB ± 0%
@@ -85,15 +85,15 @@ RandomList-2        3.24k ± 0%
 GrowShrinkQueue-2   1.04k ± 0%
 GrowShrinkList-2    2.05k ± 0%
 $ go version
-go version go1.7.5 linux/amd64
+go version go1.8.1 linux/amd64
 $ cat /proc/cpuinfo | grep "model name" | uniq
 model name	: AMD Athlon(tm) 64 X2 Dual Core Processor 6000+
 ```
 
 That's a [speedup](https://en.wikipedia.org/wiki/Speedup) of
-1.45-1.70
+1.55-1.96
 over [container/list](https://golang.org/pkg/container/list/) and a speedup of
-1.48
+1.31
 over Go's channels.
 We also consistently allocate less memory in fewer allocations than
 [container/list](https://golang.org/pkg/container/list/).