I'm very doubtful hosting the video off-site would have much effect on the site speed especially YouTube, Personally I use Wistia mainly due to the level of analytics that they provide. The only time this may be an issue if you have a quantity on a single page, in that case I would try and split it onto several different pages by means of categories or something.
To me it sounds like there may be a programming problem.
The other thing is it may not be the videos that is slowing the site down.
Just a few thoughts don't know if it helps.