If you are self-employed (the situation we're discussing), and you work for the man (your assertion), then by logical deduction you are the man — which is what I said.
You're having a major case of the "whoosh". In the context of this conversation, if you are self-employed and/or own your own company, then "the man" is the government, your clients, possibly even your suppliers.