Your ARM Is Adjusting Lower. Is There A Downside To Letting It? Adjusting conforming mortgages are adjusting to as low as 3 percent. If you have one, should you let your rate fall? Or refinance out?