mgr/dashboard: Update PG update test

It is now commented out like it was before,
but I've added a comment what happened during this test with the QA
system. The problem was that even with only a increase of 1 PG the QA
cluster went into a cluster warning state and did not recover in time.
The QA coverage timeout is 2 minutes.

I could not reproduce this behavior with a local cluster, but I've
added a loop to wait until pgp and pg number are equal and the cluster
is in a healthy state again. This can take locally about 5 seconds.
The internal loop has a timeout of 3 minutes.

Fixes: https://tracker.ceph.com/issues/36362
Signed-off-by: Stephan Müller <smueller@suse.com>

diff --git a/qa/tasks/mgr/dashboard/test_pool.py b/qa/tasks/mgr/dashboard/test_pool.py
index b9f28ff34669ffbcb2779a27e6b7b6024f5fd60b..91b63810d9e4a7c714f9b1aa3b088b8543af9320 100644 (file)
--- a/qa/tasks/mgr/dashboard/test_pool.py
+++ b/qa/tasks/mgr/dashboard/test_pool.py
@@ -4,6 +4,7 @@ from __future__ import absolute_import
 import logging
 
 import six
+import time
 
 from .helper import DashboardTestCase, JObj, JList
 
@@ -78,8 +79,19 @@ class PoolTest(DashboardTestCase):
             self.assertEqual(pool[prop], value, '{}: {} != {}'.format(prop, pool[prop], value))
 
     def _check_pg_num(self, value, pool):
+        # If both properties have not the same value, the cluster goes into a warning state,
+        # which will only happen during a pg update on a existing pool.
+        # The test that does that is currently commented out because
+        # our QA systems can't deal with the change.
+        # Feel free to test it locally.
         prop = 'pg_num'
         pgp_prop = 'pg_placement_num'
+        health = lambda: self._get('/api/dashboard/health')['health']['status'] == 'HEALTH_OK'
+        t = 0;
+        while (int(value) != pool[pgp_prop] or not health()) and t < 180:
+            time.sleep(2)
+            t += 2
+            pool = self._get_pool(pool['pool_name'])
         for p in [prop, pgp_prop]:  # Should have the same values
             self.assertEqual(pool[p], int(value), '{}: {} != {}'.format(p, pool[p], value))
 
@@ -198,8 +210,12 @@ class PoolTest(DashboardTestCase):
             {
                 'application_metadata': ['rbd', 'sth'],
             },
+            # The following test case is currently commented out because
+            # our QA systems can't deal with the change and will fail because
+            # they can't recover from the resulting warning state.
+            # Feel free to test it locally.
             # {
-            #     'pg_num': '12',
+            #     'pg_num': '8',
             # },
             {
                 'application_metadata': ['rgw'],