On Windows there are lots of "single install" bundles available. These are easy-to-install files that package together popular web technologies like Apache, PHP and MySQL and install them all for you at once, often with a handy system-tray icon to manage them too.
We would recommend either WAMP Server or XAMPP. Both have easy set-up guides available. WAMP specifically installs an icon into your system-try from which you can stop and restart the services as well as modify Apache settings, such as creating a new folder alias for a project.
Cesanta provide the Mongoose web server. This is a really small application that requires no installation and can run as a single EXE file. Without all of the additional bundles like SSI and WebDAV (none of which you'll need for an HTML5 game) the EXE is a paltry 45KB in size. Even the fully featured one is only 355KB.
Instead of an 'all in one' bundle you could also download just a web server on its own. Both Microsoft IIS and Apache can be downloaded for free from their respective sites.
Note: Skype likes to steal port 80 by default. This is the traditional port for a web server to run over and it might interfer with WAMP or similar being able to start. To disable this within Skype go to "Tools - Options - Connection" and uncheck the "Use port 80 and 443 as alternatives for incoming connections" checkbox.
Being a Unix environment at heart there are more options available of OS X. But if you'd like an "all in one" approach like WAMP for Windows, with a nice clean and easy to use interface then we'd strongly recommend MAMP. This comes in two versions: one free and one paid for.
Naturally there are also guides for setting up a local web server manually, such as this guide written for Mountain Lion. Pick whichever approach you feel most comfortable with.
Grunt is an extremely powerful tool to have installed, regardless if you use it as a web server or not. At its essence it's a JavaScript based task runner and allows you to automate tedious time consuming tasks. We use it in Phaser to build our distribution scripts for example. But it can also be configured with Connect to serve local files, acting as a web server, and here's a guide on doing just that.
If you need a quick web server running and you don't want to mess around with setting up Apache or downloading an app, then Python can help. Python comes with a simple built-in HTTP server. With the help of this little HTTP server you can turn any directory on your system into your web server directory. Naturally the only thing you need to have installed is Python. Read the full guide here.
http-server is a simple, zero-configuration command-line http server for node.js. It is powerful enough for production usage, but it's simple and hackable enough to be used for testing, local development, and learning. Or as the web site says "Serving up static files like they were turtles strapped to rockets". Get the npm and instructions from the http-server web site.
As of PHP 5.4.0, the CLI SAPI provides a built-in web server. It's only really suitable for development purposes and serves all files sequentially, but it's easily powerful enough for testing HTML5 games. It's invoked from a single command-line call, and you can find details on how to do this in the PHP Manual.
Rather than running a web server locally you could build your HTML5 game fully in the cloud. We explore some cloud options in Part 3.