While mortgage rates remain elevated for potential homebuyers, factors outside borrowing costs are keeping housing unaffordable for many, experts say.