The mgr dispatch throttle (ms_dispatch_throttle_bytes, default 100MB)
was far smaller than the OSD throttle (mgr_osd_bytes, 512MB),
making the dispatch queue the effective bottleneck on large clusters.
When the dispatch queue fills, OSD connections stall waiting for
throttle space. These connections time out and reconnect, causing
connection churn and heap growth from repeated connection setup/teardown.
The MGR also falls behind on PG stats processing, leading to unknown
and inactive PGs.
Add mgr_dispatch_throttle_bytes (default 1GB) and a
Messenger::set_dispatch_throttle_size() API to configure it at init time.
The new default is sized to accommodate the sum of all per-type policy
throttles, ensuring the dispatch queue is never the bottleneck.