I have had the same happen to two of our sites. Same language, but different country and Tld.
Google ignores the hreflang element. We even made sure to include the country in the page title and in the footer. Since the we use country specific .tld we have no option and shouldn't need to set specific market in webmaster tools.
I guess it is just the way google works when you have two sites with basically the same content.
In our case we werent penalized, it's just that google think our foreign site is more relevant.

