From a webmaster point of view I would choose wordpress. I have worked with both and eventually switch anything I have running Joomla over to wordpress. I just like it better and like how it seems easier to manage things like categories, tags, friendly URL's. I also have a list of basic plugins I use on all of my sites which include the all in one SEO pack, feedburner feeds to name just two out of a list of 10 or so.
From a readers point of view I really don't think they would care what you use as long as the site worked, was functional and delivered content that they would come back daily for and subscribe to. For 99.9% of people who come to your site they probably don't even know what wordpress or Joomla is and I would say that a lot wouldn't know the difference between a blog and a regular site... your site is just a website to them regardless of what backend powers the site.
So... you just need to use what you are comfortable with. For me, that is wordpress.
You could also look in to the technical side a bit more like the security of each, how much they can scale (if you are planning a website in a very busy sector). For me, wordpress has been good for me too.