Thank you for your help, greatly appreciated. I will forward this to our network team and see what they think.
I was told they have done a wireshark to see what was going on and they saw the following for a vmotion (as I recall):
Host A NIC is talking to the virtual machine NIC on Host B to kick in the vmotion however all the ACKS are returned by Host B NIC.
Not sure if it is releavant but it was like the Host was "spoofing" the vmNIC in order to answer on its behalf.
If something else comes in mind, feel free to add on.
Thanks!