Hell yeah! Most my customers are UK based so wish to give them the best speeds and latency as possible, which being close helps alot. Not just that it helps with SEO ranking, I doubt I get as many UK customers from Google if I had an Asian or an American IP.
Yes, a data center that is closed to your target audience make a different effect on the business. If you are targeting the audience in India and you have USA based data center then the network ping will be high and also impacts your website's SEO. Therefore, you should get a data center that is near to your target audience.
Depending on where your datacenter is physically located affects how fast your website loads. It is likely that if you have servers located far away, they will cause data transfer delays. If this is the case, your users will experience website latency.
If to speak general ,The physical location of server's data center can affect website's speed and latency. If server is far away from its users, information and data will have to travel greater distances. Its matter .
Me personally, location really matters to us. Latency and where you are located matters more to the customer actually than it does to us. We are located in Ashburn, VA as it's basically the hub of the internet!