From an SEO perspective, as long as you fill out every area, then either work just fine.
The problem many run into these days with HTML is that they don't fill out all the different aspects (where WordPress or Drupal have plugins to take care of things for you). Things like SCHEMA markup, JSON information, OpenGraph, Titles, Descriptions, navigation breadcrumbs etc, to name a few.
You can certainly do this on a small site, but it gets very time consuming on large sites.
Something to do if you're looking at an HTML would be using includes and creating a separate header and footer. This way, you can insert much of the extras needed for SEO in a common header, and then just override with certain items on the individual page itself.
If you're going to be changing things regularly, CMS is the way to go. If you're setting something up and probably won't touch it for a year (or more), then HTML is an easy route.
Site Speed is your friend, and not having to make a database call reduces one more step in the page build. With all that said, you could always use a CMS like WordPress, and then have it dump an HTML version which you would direct your site to. We used this method in the past and having pre-rendered pages shaved a ton off speed. These days, with the advances in PHP, MySQL, Memory handling, and optimized themes, we no longer run an HTML dumped site, and rely heavily on site and page cache (pre-rendered).