Artashes answered this pretty well.
Location is often a key factor. However, this really depends on the type of service you wish to run. These days networking is pretty stable between a lot of locations and with things such as webhosting it doesn't matter quite as much as it used to. Games and low latency needs are typically the main reasons people pick specific locations these days. Other reasons might be if you have a office in Manhattan NY you might want servers in New York if your staff are connecting to those for data.
Price seems to be a big factor for some companies or people, others rather spend extra and believe paying more gets you a better service which isn't always the case. That being said sometimes too cheap = not a great option either. A happy balance is always nice.
Support is another reason people want to choose a company. Do they have 24/7 support or just business hours? This can be a tricky one. Knowing where staff are can sometimes help. For example we have a few staff in Europe and other locations to help cover different hours of business.
Relationship is also a big one finding a company you trust. Word of mouth is quite often times really helpful to find who is a great company to work with. I've seen headaches from customers who have tried multiple places only to leave after issues. That being said it's always suggested to try and work out problems with a provider before just leaving. This is how relationships can be formed and broken. Some customers will have one single issue and rage leaving their provider. There are issues that sometimes come up in this industry such as network routing issues, power outages, equipment failures you name it. While those should all be rare issues sometimes come up.