There have been several reports of this issue as far as I can tell. The answer seems to be that it is an InDesign bug, meaning that even if you apply the master pages manually in InDesign the issue with the anchored objects occurs.
See here, for example: https://www.id-extras.com/forums/topic/possible-issue-with-mastermatic-and-linked-objects/
Is that the case for you? If you apply the master page manually (without Mastermatic), do the anchored objects still shrink?
I haven’t see this happen myself, but if you could send me a document where it’s happening, I’ll be happy to take a look.