Housing prices are already being "inflated" by the low cost of mortgage debt. If interest rates climbed to something like 6-7% for a 30-year mortgage you'd see values drop.