Royal row could damage relations