Uptime:
This is very important. Your site needs to be up atleast 99.9% up of the time. Uptime is very important and also shows your visitors that you mean business if your site is up when they get there.
Money back guarantee:
If you do not like the host you can always get a refund before the guarantee is over. Never pick a host without a money back guarantee.
Security:
Your files need to be safe on the server. So make sure the host is up-to-date with all their security settings, software, etc. Also, make sure your software is up-to-date as well as you could cuase a problem with security.
Established:
It is ok to go ith a host who has just started but if this is something serious then you should consider a host who has been up atleast a year or so. This way you know you are getting reliable service.
Support:
Now, if you can get 24/7 live support then by all means go for it, but the most reliable form of support is ticket support. So make sure the host you go with has 24/7 ticket support. This way if a problem should occur you can have it resolved quick.
100Mbit Port:
This is not a must, but is good to have. This way you can upload files to your account a lot faster than the standard 10Mbit Port you get with most companies.
Overselling:
Never go with a host that oversells. Overselling is when a host sells more than they can hold. For example: 300GB space, 2TB bandwidth for 3.99 is overselling. This kind of setup requires a dedicated server which will run you well over $100+
Upgradable:
Make sure the host is scablable. What happens when your shared account is no use to you anymore, can your host sell you a VPS? Make sure the host you choose has room for you to grow.
Anything Else:
Anything else you need should be included here. If you need extra IPs can the host sell them to you? need extra space or bandwidth; can the host sell you that too? So, just plan for the future and if the host can guarantee the stuff you need then by all means go with them.
Feel free to add to my list guys