Google Analytics should NOT go in the header - you want this to be the last thing to load. You want all scripts as close to the bottom as possible so that it doesn't negatively affect the page loading times on your site.
There are a number of Google Anayltics plugins that you can install into Wordpress, but as you mentioned, you'll want to check the Theme that you're using. Sometimes it's included in the Theme, sometimes it's not.
Either way, you can easily just add the code into the FOOTER of your site - not the header.