We have been using large dynamic groups, 10k plus members, though even on the smaller one this is a problem. Rebuilds are really problematic because a read by any service, in the middle of a rebuild, will cause that service to assume the, as yet to be reinserted members don't have access.
Is any body else experiencing this? I though this had been addressed. Dynamic groups could be really useful, and a whole lot safer, if they did delta forever (ads append/delete).