You should have a few criteria's that you should weigh your decision on. I find that these are the most important and after these, you'll have to search for answers that go with your specific needs or niche.
1. Uptime. A provider must have 99.9% uptime rating and hold this rating for at least 6 months. I want to see a good track record of uptime before I go with them. Everyone has downtime and 99% of the time it is unavoidable. I do not judge people because they had downtime but rather how long they went without one. Also, how they dealt with the downtime is something I look for as well.
2. Years in business. I would prefer a provider who has at least 1-2 years under their belt. This shows me they can stay in business and most likely will remain in business. Now, if this is an important project that demands high attention, funds, etc...I would go with a provider who has a minimum of 5-10 years under their belt. For example, a forum like Hosting Discussion is pretty important and serious, a host who has been around a year or two will not work. You would want a provider who has years in the business.
3. Reviews. You will want to see if the provider has reviews and how many of them are good. Everyone will get a bad review from time to time so it is best not to judge them based off those. However, if there are a lot of bad reviews then yes, judge them on that. I will use an example, if a provider has 100 reviews and 10 are bad, I find that generally to be good. Now, if the number of bad reviews hits 30-40 out of 100, then that provider is probably not going to work out. Read the reviews and see what people say about a company and go from there.
4. Budget. You must consider a provider that works with your budget. Now, there is a saying, "You pay for what you get." and I believe this to be true. Consider what the host is offering and for how much. Hosts try to be competitive and will offer insane deals but be cautious with these. If a host who has been around 15 years offers a competitive price it generally means they can support it as they have the excess hardware and resources to handle it. A host who has been around a year or two offering the same kind of service might not last long. Do your research and pick a quality provider. Find a provider that can work with your budget and that will also provider quality service. There are a lot out there, just got to find them.
5. Research. If you want a simple website then any host will do pretty much. But if this is a major project, business, or career path for you, do your research. Do not skimp on this as it will affect you going forward. Doing your due diligence now will save you headaches later and possibly financial savings too. This is your path, how you go about it is on you and only you. If crap hits the fan because you failed to do proper research, you only have yourself to blame. Google is free and so is this forum. Use them both.
6. Customer Support. How they deal with clients and how they deal with issues is an important thing to consider. You want a host to take your issues seriously, no matter what it is. Not saying they have to jump through hoops with you, but they need to be able to respond to tickets in a timely manner and be able to resolve it in a timely manner. If a provider takes 2 days to get back to you then they are not a good host. The max amount of time that should go by is 24 hours. Though, my preference is 12 hours but that is just me.
I hope this helps you and everyone else here. Thanks for coming to my Ted talk.