Rates on fixed mortgages dropped for the second straight week at the same time Treasury yields are falling.

The average rate on the 30-year mortgage sank to 4.71 percent this week from 4.77 percent the preceding week, Freddie Mac said on Thursday. In November, it had reached its lowest rate of 4.17 percent in 40 years.

The 15-year loan average rate dropped to 4.08 percent from 4.13 percent. It hit also hit 3.57 percent in November, the lowest rate it has ever recorded since 1991.

After the December employment report showed weaker than expected, Treasury yields dropped which moved investors to buy safer Treasury bonds.  This increased its prices and lowered the yields. The 10-year Treasury note shows mortgage rates tend to trail behind the yields.

After the 40-year low in November, rates have been rising. However, investors moved their money out of Treasury and into stocks as they anticipated faster economic growth and higher inflation. Yields are apt to increase along with inflation fears.

The latest drop in rates have convinced some borrowers to refinance, but upcoming buyers remain cautious. On Wednesday, the Mortgage Bankers Association said that the number of homeowners looking to refinance increased last week. Yet, the number of people applying for a purchase mortgage fell a week ago.

Higher rates are just another obstacle that struggling housing market face. High unemployment rate, increasing foreclosures, and falling home prices are the other factors that are slowing down the market’s recovery.