From what I understand it most certainly does. The SE use it to correlate the Search Users experience with your site. Bouncing back to the SERPS is an indictor that they may not have found the website trustworthy, or that the pages returned to them via the SERPS wasn't really relevant to their search query.
The best way to increase time on site is to provide quality content that engages the vistor.