I still very much pay attention to page sizes and how fast things load, and you should too. Google also pays attention to how fast a website is.
In web hosting, you're dealing with a world wide market, so the smaller the page sizes, the faster it will be for other countries such as Australia, Germany, England, Canada, US, Mexico etc etc.. The further away someone is, the longer the page loads, so the smaller the better.
Also, if your host has not already enabled gzip compression, have them do it, it's a great piece of technology to help reduce page loading times too!