If one removes the two outliers from the results, it turns out that 44ms latency shows up in more or less regular patterns - every 14th request in average. My wild guess is therefore that it's most likely related to memory allocation patterns or network.