So if I wait long enough randomly the VMs on hosts will show unavailable (four hosts never at the same time). Reviewing VMM 2012 SP1 never seems to lose a connection to them as I can right click and do status on hosts when the VMs show unavailable and under status everything is ok. If I wait they will show available again.
Could it be because I have two gateways? (Both subnets I have are reachable on either subnet so it shouldn't be a problem, and actually they are connected to same switch).