It appears that there is a bug in versions of Outlook beyond 2010 that makes the “All Mail” folder not show up, even when subscribed to it in the IMAP settings.

It turns out, as long as you are subscribed, you can create a new folder named “All Mail” under the GMAIL heading and refresh the IMAP connection.

It will now be populated!