Yelp actually does have an algorithm for computing the final star rating. It is not a simple average across all ratings. It takes contributor status on Yelp, age of review and other signals into account before the final score is published. You can, of course, dig into the rating distribution to see the spread between 1 and 5 but I doubt that most users go that far.
Absolutely. So one part of the solution is to stop putting restaurants on an "objective" 1-5 scale, and averaging every human together.
Instead, cluster restaurants so you can "people who liked the overrated, pretentious mess also liked X..."