Welcome to the Q&A Forum

Yes, your understanding of x-default is correct. The purpose of including it everywhere you have alternate HREFLANG links, is to handle any locales you don't explicitly include (to tell the search engine which is the default version of the page for other non-specified locales). And it should be included on each version of the page, along with the other specified alternate links for each locale. Alternatively, you could collect all of these centrally into the sitemap file, rather than inserting into each page. Both types of implementation are valid (but anecdotally I've had better luck with on-page tags instead of sitemap implementation).