My husband and I didn't have a preference when we bought our 2nd house. Our realtor had a mortgage company that shared their office building and was in the center of our town. They had very competitive interest rates and it was very convenient to be able to do business all within one building. One less worry for us when we were working with school deadlines, getting out of paperwork from a deal that fell through, getting our new paperwork and the paperwork on our old house figured out, etc. Since this was a bigger 'brand', I'm kind of surprised (but very glad) that they have not sold our mortgage in the 10 yrs we've had our house. Was disappointed recently now that we're dealing with the mortgage company face-to-face again (due to Hurricane Sandy damage), that they are no longer affiliated with our real estate agent and don't have an office in our town anymore. It means anything we do takes at least 1.5 to 2 hrs - even if it's dropping off an insurance check to be put in our escrow account.
For our first house, we went with the absolute lowest price ARM mortgage we could find (fresh out of school and just starting jobs). It was a company I hadn't ever heard of and we found by going through the mortgage rate listings in the newspaper. I don't know if it makes a difference or not, but it seemed our mortgage was sold at least 8-10 times in the 18 yrs we had the house. We were always having to get used to new payment procedures, etc. and it was a pain in the butt.
Best of luck!