mgr/balancer: tolerate pgs outside of target weight map

We build a target weight map based on the primary crush weights, and
ignore weights that are 0.  However, it's possible that existing PGs are
on other OSDs that have weight 0 because the weight-set weight is >0.
That leads to a KeyError exception when we

                     pgs_by_osd[osd] += 1

and the key isn't present.  Fix by simply populating those keys as we
encounter OSDs.  Drop the old initialization loop.  The net of this is
we may have OSDs outside of target_by_root (won't matter, as far as I can
tell) and we won't have keys for osds with weight 0 (also won't matter,
as far as I can tell).

Fixes: https://tracker.ceph.com/issues/42721
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c66e9acafd68215c42755067867c509247759bcb)

diff --git a/src/pybind/mgr/balancer/module.py b/src/pybind/mgr/balancer/module.py
index cd0d25d4ec62ecb392dd73123cd4626d51bcc6f9..2592305a935c296709f99627d85a2bd40bee2ce5 100644 (file)
--- a/src/pybind/mgr/balancer/module.py
+++ b/src/pybind/mgr/balancer/module.py
@@ -758,15 +758,14 @@ class Module(MgrModule):
             pgs_by_osd = {}
             objects_by_osd = {}
             bytes_by_osd = {}
-            for root in pe.pool_roots[pool]:
-                for osd in pe.target_by_root[root]:
-                    pgs_by_osd[osd] = 0
-                    objects_by_osd[osd] = 0
-                    bytes_by_osd[osd] = 0
             for pgid, up in six.iteritems(pm):
                 for osd in [int(osd) for osd in up]:
                     if osd == CRUSHMap.ITEM_NONE:
                         continue
+                    if osd not in pgs_by_osd:
+                        pgs_by_osd[osd] = 0
+                        objects_by_osd[osd] = 0
+                        bytes_by_osd[osd] = 0
                     pgs_by_osd[osd] += 1
                     objects_by_osd[osd] += ms.pg_stat[pgid]['num_objects']
                     bytes_by_osd[osd] += ms.pg_stat[pgid]['num_bytes']