|
|
@@ -1,8 +1,8 @@
|
|
|
# Benchmarks and Test Cases
|
|
|
|
|
|
-It seems that Go's otherwise excellent
|
|
|
+It seems that *some* versions of Go's otherwise excellent
|
|
|
[testing](https://golang.org/pkg/testing/)
|
|
|
-infrastructure is susceptible to certain "contamination effects"
|
|
|
+infrastructure are susceptible to certain "contamination effects"
|
|
|
in which the presence (or absence) of *test cases* influences the
|
|
|
performance measured by *benchmarks*.
|
|
|
Check it out:
|
|
|
@@ -35,6 +35,8 @@ PushBackList-2 2.05k ± 0% 2.05k ± 0% ~ (all equal)
|
|
|
PushBackChannel-2 1.03k ± 0% 1.03k ± 0% ~ (all equal)
|
|
|
RandomQueue-2 1.63k ± 0% 1.63k ± 0% ~ (all equal)
|
|
|
RandomList-2 3.24k ± 0% 3.24k ± 0% ~ (all equal)
|
|
|
+$ go version
|
|
|
+go version go1.7.5 linux/amd64
|
|
|
```
|
|
|
|
|
|
The *only* difference between `old time` and `new time` is that the
|
|
|
@@ -46,3 +48,48 @@ everything is "back to normal" as it were.
|
|
|
For now I am reporting the *worse* results in the official `README` just
|
|
|
to be conservative, but I believe something needs to be fixed on the Go
|
|
|
side of things.
|
|
|
+
|
|
|
+**Update**: Something *was* apparently fixed in a more recent version
|
|
|
+of Go. Check it out:
|
|
|
+
|
|
|
+```
|
|
|
+$ benchstat without.txt with.txt
|
|
|
+name old time/op new time/op delta
|
|
|
+PushFrontQueue-2 82.7µs ± 1% 82.8µs ± 1% ~ (p=0.832 n=19+16)
|
|
|
+PushFrontList-2 161µs ± 1% 162µs ± 1% +0.48% (p=0.003 n=19+18)
|
|
|
+PushBackQueue-2 83.3µs ± 1% 83.4µs ± 1% ~ (p=0.756 n=19+16)
|
|
|
+PushBackList-2 156µs ± 1% 158µs ± 3% +1.03% (p=0.024 n=19+20)
|
|
|
+PushBackChannel-2 110µs ± 2% 110µs ± 2% ~ (p=0.341 n=20+20)
|
|
|
+RandomQueue-2 158µs ± 4% 161µs ± 2% +2.34% (p=0.000 n=20+20)
|
|
|
+RandomList-2 279µs ± 1% 281µs ± 4% ~ (p=0.223 n=19+19)
|
|
|
+GrowShrinkQueue-2 111µs ± 2% 110µs ± 1% -1.39% (p=0.000 n=18+19)
|
|
|
+GrowShrinkList-2 168µs ± 1% 170µs ± 5% +1.21% (p=0.017 n=19+20)
|
|
|
+
|
|
|
+name old alloc/op new alloc/op delta
|
|
|
+PushFrontQueue-2 40.9kB ± 0% 40.9kB ± 0% ~ (all equal)
|
|
|
+PushFrontList-2 57.4kB ± 0% 57.4kB ± 0% ~ (all equal)
|
|
|
+PushBackQueue-2 40.9kB ± 0% 40.9kB ± 0% ~ (all equal)
|
|
|
+PushBackList-2 57.4kB ± 0% 57.4kB ± 0% ~ (all equal)
|
|
|
+PushBackChannel-2 24.7kB ± 0% 24.7kB ± 0% ~ (all equal)
|
|
|
+RandomQueue-2 45.7kB ± 0% 45.7kB ± 0% ~ (all equal)
|
|
|
+RandomList-2 90.8kB ± 0% 90.8kB ± 0% ~ (all equal)
|
|
|
+GrowShrinkQueue-2 57.2kB ± 0% 57.2kB ± 0% ~ (all equal)
|
|
|
+GrowShrinkList-2 57.4kB ± 0% 57.4kB ± 0% ~ (all equal)
|
|
|
+
|
|
|
+name old allocs/op new allocs/op delta
|
|
|
+PushFrontQueue-2 1.03k ± 0% 1.03k ± 0% ~ (all equal)
|
|
|
+PushFrontList-2 2.05k ± 0% 2.05k ± 0% ~ (all equal)
|
|
|
+PushBackQueue-2 1.03k ± 0% 1.03k ± 0% ~ (all equal)
|
|
|
+PushBackList-2 2.05k ± 0% 2.05k ± 0% ~ (all equal)
|
|
|
+PushBackChannel-2 1.03k ± 0% 1.03k ± 0% ~ (all equal)
|
|
|
+RandomQueue-2 1.63k ± 0% 1.63k ± 0% ~ (all equal)
|
|
|
+RandomList-2 3.24k ± 0% 3.24k ± 0% ~ (all equal)
|
|
|
+GrowShrinkQueue-2 1.04k ± 0% 1.04k ± 0% ~ (all equal)
|
|
|
+GrowShrinkList-2 2.05k ± 0% 2.05k ± 0% ~ (all equal)
|
|
|
+$ go version
|
|
|
+go version go1.8.1 linux/amd64
|
|
|
+```
|
|
|
+
|
|
|
+The additional test cases still have *some* effect on the benchmarks, but
|
|
|
+it's nowhere near as extreme as it was before.
|
|
|
+I am not perfectly happy with this, but I am happy enough.
|