That's the issue though. Libertarians often say things like "If you don't like Walmart, don't shop there," which is fine, but puts the responsibility on the consumer to know the various ideologies of the businesses they choose to support. If we want to live in a society in which this responsibility lies with the consumer, then the people who make up a company's board of directors are an important part of this decision making process.