Yeah, this seemed fishy to me too (I made a similar comment here: https://www.reddit.com/r/golang/comments/uo5mix/comment/i8dn...). It seems the mattn/cgo version is doing a constant amount of work, but the modernc/non-cgo version is doing an amount of work linear(ish) to the number of rows. The latter makes more sense for this query, so I wonder if there's something wrong here?
How could one be doing constant work and the other O(rows) work if it's the same code (just compiled from one language to another)?
I also thought the sub-ms numbers can't be right — SQLite is fast but not that fast on millions of rows, but didn't look into it until now.
Turns out the benchmarking code is wrong: it didn't read the rows returned from db.Query, so the mattn version simply didn't wait for the results to arrive. Once you apply this patch:
diff --git a/cgo/main.go b/cgo/main.go
index 8796b3d..9a74a2f 100644
--- a/cgo/main.go
+++ b/cgo/main.go
@@ -82,11 +82,15 @@ CREATE TABLE people (
panic(err)
}
}
- fmt.Printf("%f,%d,insert,cgo\n", float64(time.Now().Sub(t1)) / 1e9, rows)
+ fmt.Printf("%f,%d,insert,cgo\n", float64(time.Now().Sub(t1))/1e9, rows)
t1 = time.Now()
- _, err = db.Query("SELECT COUNT(1), age FROM people GROUP BY age ORDER BY COUNT(1) DESC")
- fmt.Printf("%f,%d,group_by,cgo\n", float64(time.Now().Sub(t1)) / 1e9, rows)
+ res, _ := db.Query("SELECT COUNT(1), age FROM people GROUP BY age ORDER BY COUNT(1) DESC")
+ for res.Next() {
+ var count, age int
+ _ = res.Scan(&count, &age)
+ }
+ fmt.Printf("%f,%d,group_by,cgo\n", float64(time.Now().Sub(t1))/1e9, rows)
}
}
}
modernc SELECT performance becomes pretty comparable, actually a little bit faster than mattn on my Intel Mac with high row count.
Not only that, modernc INSERT is noticeably faster on my Intel Mac...
Good point, thanks for checking out the code! I replaced db.Query with a db.Exec for the SELECT count (just to avoid the row iteration/deserialization) and I'm seeing closer performance.
How could one be doing constant work and the other O(rows) work if it's the same code (just compiled from one language to another)?