qa: fix issue with fn unable to fetch port and ip

_get_port_ip_info() fails to fetch port and ip due to empty 'backend' key:

2023-02-24T20:49:09.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph nfs cluster info test
2023-02-24T20:49:09.471 INFO:teuthology.orchestra.run.smithi042.stdout:{
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:  "test": {
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:    "backend": [],
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:    "virtual_ip": null
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:  }
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:}

it then raises:

2023-02-24T20:49:10.323 INFO:tasks.cephfs_test_runner:    info_output = json.loads(self._nfs_cmd('cluster', 'info', self.cluster_id))['test']['backend'][0]
2023-02-24T20:49:10.323 INFO:tasks.cephfs_test_runner:IndexError: list index out of range

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 310286fa185194fdd27bfe7692d516703e18fa11)

diff --git a/qa/tasks/cephfs/test_nfs.py b/qa/tasks/cephfs/test_nfs.py
index 5effb98b79116bdac036fec90d973a5b77bfb576..24374630fd3e12a1ca90ab230369dd860ca9db6e 100644 (file)
--- a/qa/tasks/cephfs/test_nfs.py
+++ b/qa/tasks/cephfs/test_nfs.py
@@ -300,9 +300,20 @@ class TestNFS(MgrTestCase):
         '''
         Return port and ip for a cluster
         '''
-        #{'test': {'backend': [{'hostname': 'smithi068', 'ip': '172.21.15.68', 'port': 2049}]}}
-        info_output = json.loads(self._nfs_cmd('cluster', 'info', self.cluster_id))['test']['backend'][0]
-        return info_output["port"], info_output["ip"]
+        #{'test': {'backend': [{'hostname': 'smithi068', 'ip': '172.21.15.68',
+        #'port': 2049}]}}
+        with contextutil.safe_while(sleep=5, tries=6) as proceed:
+            while proceed():
+                try:
+                    info_output = json.loads(
+                        self._nfs_cmd('cluster', 'info',
+                                      self.cluster_id))['test']['backend'][0]
+                    return info_output["port"], info_output["ip"]
+                except (IndexError, CommandFailedError) as e:
+                    if 'list index out of range' in str(e):
+                        log.warning('no port and/or ip found, retrying')
+                    else:
+                        log.warning(f'{e}, retrying')
 
     def _test_mnt(self, pseudo_path, port, ip, check=True):
         '''