can't build or assign villagers or organize feast in multiplayer.

Multiplayer loads fine, and then when in game, a +- 2 min "waiting for players" time elapses, then the game starts. However, only my friend can actually play. My game looks like it has a network problem (laging behind the game progress, villagers walking in place, and not beiong able to build/interact with the game), however i have a fast internet with low latency (2ms). None of the multiplayer options work. This is after recently being able to play. I have also switched wi-fi connections, and this also didnt resolve the problem.