Hi Dana,
Yes TTFB is something you can control with the type of server you use. And where that server is in relation to your visitors. You cannot control the browsers they use, but hear are some thoughts on possible optimizations:
Server side: http://createdevelop.blog.com/2010/10/12/how-to-reduce-time-to-first-byte/
Location (plus other suggestions): http://msdn.microsoft.com/en-us/magazine/dd188562.aspx
