If the agency is working on your behalf, then it depends on the agreed contract. Ultimately, you might decide that it's worth switching to an agency which isn't as protective about the accounts.
Personally as someone who runs Adwords and Youtube accounts for various clients, I tend to group them within the account for my agency simply because it means we have a bigger overall spend, so we might get a slightly better response from Google etc, and it also means that we never have access issues when we're checking them daily (and the client generally doesn't have the time or desire to monitor as closely). But I also give them top level account access as a matter of course, even if they never use it - optimising ad accounts isn't a massive secret - it's mainly long, dull and tedious optimisation!
And if a client prefers, I'm always fine with letting them maintain the account as long as I can get the access I need - the only difference really is that it doesn't then count towards the totals for Google Partner status etc, but that's not a massive issue...
Having had agencies hand over accounts in the past, I've seen good and bad examples, and I certainly wouldn't want to work with an agency that wasn't willing to let me have a quick look at negative keyword lists etc every so often, espencially if you're spending a reasonable amount...