Hm. Maybe if these vendors truly are using it to retrieve things from our worlds there is something about the portal that forces their return. Otherwise I would assume they would simply leave.

Unless they are committed to fighting the World Eaters, which would be beneficial to us, but highly unlikely.