If you're implementing CSP, you should only include the header on text/html or other rendered responses, so the overhead is more per-navigation than per-request. I've seen a lot of guides where CSP is added globally at the webserver level which can waste a lot of bandwidth with images etc.
Twitter sends over 6kb of CSP headers on every single request. This is what happens if you run loads of different advertisement and tracking vendors.