]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agomgr/dashboard: python unit tests refactoring
Alfonso Martínez [Wed, 3 Nov 2021 09:17:21 +0000 (10:17 +0100)]
mgr/dashboard: python unit tests refactoring

* Controller tests: cherrypy config: authentication disabled by default; ability to pass custom config (e.g. enable authentication).
* Auth controller: add tests; test that unauthorized request fails when authentication is enabled.
* DocsTest: clear ENDPOINT_MAP so the test_gen_tags test becomes deterministic.
* pylint: disable=no-name-in-module: fix imports in tests.

Fixes: https://tracker.ceph.com/issues/53083
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit a6aeded5141ec3a959a86aa4002ccf5ac8d0a523)

 Conflicts:
src/pybind/mgr/dashboard/tests/test_docs.py
    - Resolved conflicts.
src/pybind/mgr/dashboard/tests/test_iscsi.py
    - Resolved conflicts.

3 years agomgr/dashboard: improve error handling for gather_facts
Avan Thakkar [Wed, 27 Oct 2021 19:30:28 +0000 (01:00 +0530)]
mgr/dashboard: improve error handling for gather_facts

Fixes: https://tracker.ceph.com/issues/53084
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 75c8d578ed106c4584c166d8ccd9153c69667d01)

3 years agomgr/dashboard: Cluster Creation Add multiple hosts at once
Aashish Sharma [Thu, 23 Sep 2021 10:52:56 +0000 (16:22 +0530)]
mgr/dashboard: Cluster Creation Add multiple hosts at once

Add multiple hosts at once in cluster creation wizard

Fixes: https://tracker.ceph.com/issues/52759
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit caa177265c34530b1de10f9fe3664c23202d1b5f)

3 years agomgr/dashboard: gather facts should only be fetched when orch backend is cephadm
Avan Thakkar [Wed, 20 Oct 2021 14:07:26 +0000 (19:37 +0530)]
mgr/dashboard: gather facts should only be fetched when orch backend is cephadm

Fixes: https://tracker.ceph.com/issues/52981
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Gather facts in UI should only be fetched if there is orch available and
get_facts feature is available for that orch backend.

(cherry picked from commit 78b8af2bda39dea8aa46a42e0cea4c5fd53c1d42)

3 years agomgr/dashboard: Fix for form inside form closing issue
Nizamudeen A [Sat, 23 Oct 2021 19:06:51 +0000 (00:36 +0530)]
mgr/dashboard: Fix for form inside form closing issue

After the angular 11 upgrade the form in form behaviour got broken when
one tries to close that "embedded" form. This commit is to fix that
behaviour.

Fixes: https://tracker.ceph.com/issues/53020
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit b4ebca28d82eeb927c30720b97ba14838482c497)

3 years agomgr/dashboard: NFS exports: API + UI: integration with mgr/nfs; cleanups
Alfonso Martínez [Thu, 26 Aug 2021 10:05:54 +0000 (12:05 +0200)]
mgr/dashboard: NFS exports: API + UI: integration with mgr/nfs; cleanups

mgr/dashboard: move NFS_GANESHA_SUPPORTED_FSALS to mgr_module.py

Importing from nfs module throws AttributeError because as a side effect the dashboard module is impersonating the nfs module.
https://gist.github.com/varshar16/61ac26426bbe5f5f562ebb14bcd0f548

mgr/dashboard: 'Create NFS export' form: list clusters from nfs module

mgr/dashboard: frontend+backend cleanups for NFS export

Removed all code and references related to daemons. UI cleanup and adopted unit-testing for
nfs-epxort create form for CEPHFS backend. Cleanup for export list/get/create/set/delete endpoints.

mgr/dashboard: rm set-ganesha ref + update docs

Remove existing set-ganesha-clusters-rados-pool-namespace references as
they are no longer required. Moreover, nfs doc in dashboard doc is
updated accordingly to the current nfs status.

mgr/dashboard: add nfs-export e2e test coverage

mgr/dashboard: 'Create NFS export' form: remove RGW user id field.

- Improve bucket typeahead behavior.
- Increase version for bucket list endpoint.
- Some refactoring.

mgr/dashboard: 'Create NFS export' form: allow RGW backend only when default realm is selected.

When RGW multisite is configured, the NFS module can only handle buckets in the default realm.

mgr/dashboard: 'Create service' form: fix NFS service creation.

After https://github.com/ceph/ceph/pull/42073, NFS pool and namespace are not customizable.

mgr/dashboard: 'Create NFS export' form: add bucket validation.

- Allow only existing buckets.
- Refactoring:
  - Moved bucket validator from bucket form to cd-validators.ts
  - Split bucket validator into 2: bucket name validator and bucket existence (that checks either existence or non-existence).

mgr/dashboard: 'Create NFS export' form: path validation refactor: allow only existing paths.

Fixes: https://tracker.ceph.com/issues/46493
Fixes: https://tracker.ceph.com/issues/51479
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
(cherry picked from commit 58a6ab2147c34d5b3f14bf48f9b47b03bea8a672)

 Conflicts:
doc/mgr/dashboard.rst
  - Resolved conflicts.
src/pybind/mgr/dashboard/services/cephx.py
  - Deleted as in master.

3 years agomgr/dashboard: directly use ExportMgr and NFSCluster objects
Varsha Rao [Mon, 9 Aug 2021 19:00:16 +0000 (00:30 +0530)]
mgr/dashboard: directly use ExportMgr and NFSCluster objects

Using the objects directly provides access to other methods and helps in
avoiding repeatition.

mgr/dashboard/nfsganesha: remove tag

Since NFS v3 is no longer supported. We can remove tag.

mgr/nfs: define global constant to list supported FSALs

mgr/dashboard: directly list nfs clusters by directly importing available_cluster() method

The current dashboard api returns a list of following dictionary

{
   'pool': 'nfs-ganesha',
   'namespace': cluster_id,
   'type': 'orchestrator',
   'daemon_conf': None
}

None of these values are required for listing nfs cluster by mgr/nfs module.
Instead directly list available cluster names

mgr/dashboard: add comment to remove listing of daemons

As the configs are per cluster. There is no need to list daemons per cluster.

mgr/dashboard/controllers/nfsganesha: Add comments to update/remove status endpoint

This endpoint can be updated in suggested way or even removed. As it was
initially[1] introduced to check if dashboard pool and namespace configuration was
set.

[1] https://github.com/ceph/ceph/commit/824726393b185b8e5a8f17e66487dfde9f3c8b5c

mgr/nfs: remove fetch_cluster_obj()

There is no need to fetch NFSCluster class object. Directly
available_clusters() can be imported to list nfs clusters.

mgr/dashboard/controllers/nfsganesha: list exports based on cluster id

As mgr/nfs module lists based on cluster id.

mgr/dashboard/nfs: get and delete export by export id

Fixes: https://tracker.ceph.com/issues/46493
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 64178e1f5b0f26f5c17898e85c3d5b439e5f9dbf)

3 years agomgr/nfs: stick to lazy evaluation of logger messages
Ramana Raja [Thu, 30 Sep 2021 01:13:09 +0000 (21:13 -0400)]
mgr/nfs: stick to lazy evaluation of logger messages

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit e1c738cefe84c2b6dd15a6667d2df051a0f34e31)

3 years agomgr/nfs: change _cmd_rgw_export_create_rgw() name
Varsha Rao [Mon, 16 Aug 2021 05:46:45 +0000 (11:16 +0530)]
mgr/nfs: change _cmd_rgw_export_create_rgw() name

To maintain uniformity in naming, modify _cmd_rgw_export_create_rgw() name to
_cmd_nfs_export_create_rgw().

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 4137f88d0c4278ad1a8a7bce9c6d432d1c9e7c7c)

3 years agomgr/nfs: don't log fsal keys
Varsha Rao [Mon, 16 Aug 2021 05:39:18 +0000 (11:09 +0530)]
mgr/nfs: don't log fsal keys

FSAL keys are written to export objects. Don't log it, instead log the initial
export before generating keys and user config object mostly will not contain
export block. It's okay to log user config object content.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit b8c47c8d223f45092325cb8c02fa137eeebefb25)

3 years agomgr/nfs: Add more debug log messages
Varsha Rao [Mon, 16 Aug 2021 05:20:48 +0000 (10:50 +0530)]
mgr/nfs: Add more debug log messages

Fixes: https://tracker.ceph.com/issues/52274
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 719e7024682d64da9bb2c22b90725ff311e83860)

3 years agomgr/dashboard: consume mgr/nfs via mgr.remote()
Sage Weil [Wed, 28 Jul 2021 14:29:47 +0000 (10:29 -0400)]
mgr/dashboard: consume mgr/nfs via mgr.remote()

Stop using the dashboard version of the Ganesha config classes; consume
mgr/nfs instead via remote().

mgr/nfs/export: return Export from _apply_export

Future callers will want this.

mgr/nfs: new module methods for dashboard consumption

Add some new methods that are easy for the dashboard API to consume.  These
are very similar to the CLI methods but do now have the @CLICommand and
related decorators, and have slightly different interfaces (e.g., returning
the created/modified Export dict).

mgr/dashboard: remove old ganesha code (and tests)

Fixes: https://tracker.ceph.com/issues/46493
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6e5a1eefd0b48614f7c95f86fd9fadab1b2b03fe)

Conflicts:
src/pybind/mgr/dashboard/services/ganesha.py
    - Deleted as in master.

3 years agomgr/dashboard: Cluster expansion review page minor bug fixes
Nizamudeen A [Tue, 26 Oct 2021 07:46:51 +0000 (13:16 +0530)]
mgr/dashboard: Cluster expansion review page minor bug fixes

Fix the Total Memory coming up as NaN Undefined
Remove `Add` Button in the Cluster Details table

Fixes: https://tracker.ceph.com/issues/53038
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 81430251edbb07e73155c7b194cfe3d6d60c7905)

3 years agomgr/dashboard: Proper error handling in module status guard
Nizamudeen A [Mon, 25 Oct 2021 07:25:35 +0000 (12:55 +0530)]
mgr/dashboard: Proper error handling in module status guard

I've recently introduced a check that verifies if the orch_backend we need for a feature matches the orch_backend of the cluster. But to verify this I need to call configOpt API and some user's don't have the permission to access it. So it'll show this Access Denied page.

Fixes: https://tracker.ceph.com/issues/53021
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit dc80f26a50a972c71ca45c09f808ed5434f3a085)

3 years agomgr/dashboard: s/hosts.clickHostTab/hosts.clickTab/
Kefu Chai [Thu, 14 Oct 2021 22:44:55 +0000 (06:44 +0800)]
mgr/dashboard: s/hosts.clickHostTab/hosts.clickTab/

this change is a follow-up fix of
3276147049511a9a8d45d84c95154a4ec014cd83

otherwise we have following test failure:

[lint:tsc     ] cypress/integration/orchestrator/01-hosts.e2e-spec.ts(29,13): error TS2339: Property 'clickHostTab' does not exist on type 'HostsPageHelper'.

also change "host" to "hostname" to be more consistent

Fixes: https://tracker.ceph.com/issues/52645
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit a6bb0e9771eec7067ba03582583c78ded6eed8ee)

3 years agomgr/dashboard: Cluster expansion e2e cleanups
Nizamudeen A [Fri, 17 Sep 2021 13:13:01 +0000 (18:43 +0530)]
mgr/dashboard: Cluster expansion e2e cleanups

Fixes: https://tracker.ceph.com/issues/52645
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 3276147049511a9a8d45d84c95154a4ec014cd83)

3 years agomgr/dashboard: introduce gather facts in host list
Avan Thakkar [Tue, 3 Aug 2021 09:01:57 +0000 (14:31 +0530)]
mgr/dashboard: introduce gather facts in host list

Fixes: https://tracker.ceph.com/issues/52017
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 6644a00a2c6d6a7fa709ee82f5ada0d73a0929a8)

3 years agomgr/dashboard: Cluster Creation Add Services Section
Aashish Sharma [Tue, 7 Sep 2021 06:30:45 +0000 (12:00 +0530)]
mgr/dashboard: Cluster Creation Add Services Section

Add Services section in cluster creation wizard
Create Cluster OSD Section Followups

1. The device preview disappearing when going to next step and coming back to the previous step
2. Even when clearing the device preview, the Storage Capacity count and the drive group spec doesn't get cleared.
3. Expanding the cluster without selecting any devices gives a 400
   error.
4. Renamed "Delete Host" to "Remove Host"
5. Generalizing most of the sub component code

Fixes: https://tracker.ceph.com/issues/52499
Fixes: https://tracker.ceph.com/issues/51991
Signed-off-by: Nizamudeen A <nia@redhat.com>
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit b914f59ff123b9e006d7d11dc786e967c0271dbf)

3 years agomgr/dashboard: Cluster Creation Create OSDs Section
Nizamudeen A [Tue, 17 Aug 2021 13:34:31 +0000 (19:04 +0530)]
mgr/dashboard: Cluster Creation Create OSDs Section

Create OSDs section in cluster creation wizard

Fixes: https://tracker.ceph.com/issues/51991
Fixes: https://tracker.ceph.com/issues/52298
Signed-off-by: Nizamudeen A <nia@redhat.com>
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit ad1535eea465f5e9f096600e01df1e95813ebb18)

3 years agomgr/dashboard: Review Section for the Create Cluster Workflow
Avan Thakkar [Fri, 9 Jul 2021 13:27:35 +0000 (18:57 +0530)]
mgr/dashboard: Review Section for the Create Cluster Workflow

Fixes: https://tracker.ceph.com/issues/50566
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 2e31ba54273732db5d93a843d22345b676fcecf7)

3 years agomgr/dashboard: Cluster Creation Add Host Section and e2es
Nizamudeen A [Sun, 4 Jul 2021 13:16:45 +0000 (18:46 +0530)]
mgr/dashboard: Cluster Creation Add Host Section and e2es

Add host section of the cluster creation workflow.

1. Fix bug in the modal where going forward one step on the wizard and coming back opens up the add host modal.
2. Rename Create Cluster to Expand Cluster as per the discussions
3. A skip confirmation modal to warn the user when he tries to skip the
   cluster creation
4. Adapted all the tests
5. Did some UI improvements like fixing and aligning the styles,
   colors..
- Used routed modal for host Additon form
- Renamed the Create to Add in Host Form

Fixes: https://tracker.ceph.com/issues/51517
Fixes: https://tracker.ceph.com/issues/51640
Fixes: https://tracker.ceph.com/issues/50336
Fixes: https://tracker.ceph.com/issues/50565
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 59cbf97e6ceedee0f9d682a94a749fbb01e9f787)

3 years agomgr/dashboard: Create Cluster Workflow welcome screen and e2e tests
Avan Thakkar [Tue, 1 Jun 2021 12:55:15 +0000 (18:25 +0530)]
mgr/dashboard: Create Cluster Workflow welcome screen and e2e tests

A module option called CLUSTER_STATUS has two option. INSTALLED
AND POST_INSTALLED. When CLUSTER_STATUS is INSTALLED it will allow to show the
create-cluster-wizard after login the initial time.  After the cluster
creation is succesfull this option is set to POST_INSTALLED
Also has the e2e codes for the Review Section

Fixes: https://tracker.ceph.com/issues/50336
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit b9f38cadc46338c9f1527beeaaba3d42639badf9)

3 years agocephadm: expose gather-facts api method
Avan Thakkar [Fri, 11 Jun 2021 11:07:10 +0000 (16:37 +0530)]
cephadm: expose gather-facts api method

Fixes: https://tracker.ceph.com/issues/51209
This PR intends to expose host metadata(gather-facts) api method

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit f41eae16af9fa6725d5b714f068fab3f946d2927)

3 years agomgr/dashboard: Remove i18n Warning about duplicate id
Nizamudeen A [Thu, 30 Sep 2021 11:48:14 +0000 (17:18 +0530)]
mgr/dashboard: Remove i18n Warning about duplicate id

Fixes: https://tracker.ceph.com/issues/50268
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit fa6fb0087cf6733c6b70bcd38a408978e9ab14de)

3 years agomgr/dashboard: Taking care of deprecation warnings and other errors
Nizamudeen A [Thu, 30 Sep 2021 07:47:15 +0000 (13:17 +0530)]
mgr/dashboard: Taking care of deprecation warnings and other errors

1. deprecation  group is deprecated: This API is not typesafe and can result in issues with Closure Compiler renaming. Use the `FormBuilder#group` overload with `AbstractControlOptions` instead.

2. cypress/integration/rgw/users.po.ts(113,23): error TS2345: Argument of type 'void' is not assignable to parameter of type 'string'.

3. Cannot find module 'isomorphic-form-data' from 'node_modules/swagger-ui/dist/swagger-ui.js'

Fixes: https://tracker.ceph.com/issues/50268
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit f7d73d0420147300cbd6189620495498d27ec861)

3 years agomgr/dashboard: Angular v11 Migration
Nizamudeen A [Tue, 29 Jun 2021 09:30:34 +0000 (15:00 +0530)]
mgr/dashboard: Angular v11 Migration

Fixes: https://tracker.ceph.com/issues/50268
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit de2648501838891144a1cd85dc52527e1a28726f)
  Conflicts
src/pybind/mgr/dashboard/frontend/package.json
  - Resolved the conflicts
src/pybind/mgr/dashboard/frontend/package-lock.json
  - Generated a new package-lock.json file

3 years agoMerge pull request #43615 from batrick/i52999
Yuri Weinstein [Tue, 2 Nov 2021 13:34:49 +0000 (06:34 -0700)]
Merge pull request #43615 from batrick/i52999

pacific: mon/MDSMonitor: avoid crash when decoding old FSMap epochs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #43614 from batrick/i52998
Yuri Weinstein [Tue, 2 Nov 2021 13:33:41 +0000 (06:33 -0700)]
Merge pull request #43614 from batrick/i52998

pacific: mds/FSMap: do not assert allow_standby_replay on old FSMaps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #43748 from tchaikov/pacific-doc-build
Sebastian Wagner [Mon, 1 Nov 2021 13:38:31 +0000 (14:38 +0100)]
Merge pull request #43748 from tchaikov/pacific-doc-build

pacific: admin/doc-requirements.txt: pin Sphinx at 3.5.4

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoadmin/doc-requirements.txt: pin Sphinx at 3.5.4 43748/head
Kefu Chai [Sat, 30 Oct 2021 03:18:17 +0000 (11:18 +0800)]
admin/doc-requirements.txt: pin Sphinx at 3.5.4

* pin Sphinx at 3.5.4
* pin docutils at 0.18

at least the combination of these two versions
is known to compile.

to address the bug reported at
https://sourceforge.net/p/docutils/bugs/431/

the backtrace looks like:

/home/jenkins-build/build/workspace/ceph-pr-docs/build-doc/virtualenv/lib/python3.8/site-packages/sphinx/util/docutils.py:285:
RemovedInSphinx30Warning: function based directive support is now
deprecated. Use class based directive instead.
  warnings.warn('function based directive support is now deprecated. '

Exception occurred:
  File
"/home/jenkins-build/build/workspace/ceph-pr-docs/build-doc/virtualenv/lib/python3.8/site-packages/docutils/writers/html5_polyglot/__init__.py",
line 445, in section_title_tags
    if (ids and self.settings.section_self_link
AttributeError: 'Values' object has no attribute 'section_self_link'

please note this change is not cherry-picked from
master, because master already bumped Sphinx to 3.5.4
in 4968baa2523bd2a5ca6be147b26bc28906a864c9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #43543 from rhcs-dashboard/wip-52870-pacific
Yuri Weinstein [Thu, 28 Oct 2021 20:02:43 +0000 (13:02 -0700)]
Merge pull request #43543 from rhcs-dashboard/wip-52870-pacific

pacific: mgr/dashboard: clean-up controllers and API backward versioning compatibility

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
3 years agoMerge pull request #43417 from trociny/wip-51646-pacific
Yuri Weinstein [Wed, 27 Oct 2021 13:16:54 +0000 (06:16 -0700)]
Merge pull request #43417 from trociny/wip-51646-pacific

pacific: osd/OSD: mkfs need wait for transcation completely finish

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #43562 from lxbsz/vino_fix
Yuri Weinstein [Wed, 27 Oct 2021 13:15:50 +0000 (06:15 -0700)]
Merge pull request #43562 from lxbsz/vino_fix

Pacific: test/libcephfs: put inodes after lookup

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #43559 from batrick/i52654-pacific
Yuri Weinstein [Wed, 27 Oct 2021 13:14:59 +0000 (06:14 -0700)]
Merge pull request #43559 from batrick/i52654-pacific

pacific: pybind/mgr/cephadm: set allow_standby_replay during CephFS upgrade

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
3 years agoMerge pull request #43475 from lxbsz/tracker_52876
Yuri Weinstein [Wed, 27 Oct 2021 13:13:26 +0000 (06:13 -0700)]
Merge pull request #43475 from lxbsz/tracker_52876

pacific: test: shutdown the mounter after test finishes

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #43644 from aaSharma14/wip-52965-pacific
Ernesto Puerta [Wed, 27 Oct 2021 10:23:54 +0000 (12:23 +0200)]
Merge pull request #43644 from aaSharma14/wip-52965-pacific

pacific: mgr/dashboard: monitoring: grafonnet refactoring for radosgw dashboards

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #43619 from smithfarm/wip-53005-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:38:12 +0000 (13:38 -0700)]
Merge pull request #43619 from smithfarm/wip-53005-pacific

pacific: rgw/tracing: unify SO version numbers within librgw2 package

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #43512 from neha-ojha/wip-52770-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:29:56 +0000 (13:29 -0700)]
Merge pull request #43512 from neha-ojha/wip-52770-pacific

pacific: os/bluestore: list obj which equals to pend

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
3 years agoMerge pull request #43513 from neha-ojha/wip-52620-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:29:11 +0000 (13:29 -0700)]
Merge pull request #43513 from neha-ojha/wip-52620-pacific

pacific: osd: fix partial recovery become whole object recovery after restart osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge pull request #43511 from neha-ojha/wip-52843-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:28:38 +0000 (13:28 -0700)]
Merge pull request #43511 from neha-ojha/wip-52843-pacific

pacific: msg/async/ProtocolV2: Set the recv_stamp at the beginning of receiving a message

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
3 years agoMerge pull request #43445 from k0ste/wip-52848-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:27:30 +0000 (13:27 -0700)]
Merge pull request #43445 from k0ste/wip-52848-pacific

pacific: mgr: Add check to prevent mgr from crashing

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #43437 from trociny/wip-52831-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:26:46 +0000 (13:26 -0700)]
Merge pull request #43437 from trociny/wip-52831-pacific

pacific: osd: re-cache peer_bytes on every peering state activate

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43421 from callithea/wip-52289-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:26:16 +0000 (13:26 -0700)]
Merge pull request #43421 from callithea/wip-52289-pacific

pacific: qa/tasks/mgr: skip test_diskprediction_local on python>=3.8

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43353 from kamoltat/wip-ksirivad-backport-pacific-37544
Yuri Weinstein [Tue, 26 Oct 2021 20:24:37 +0000 (13:24 -0700)]
Merge pull request #43353 from kamoltat/wip-ksirivad-backport-pacific-37544

pacific: mgr/progress: optimize global recovery && introduce 5 seconds interval

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agomgr/dashboard: monitoring: grafonnet refactoring for hosts dashboards 43644/head
Aashish Sharma [Fri, 8 Oct 2021 10:03:13 +0000 (15:33 +0530)]
mgr/dashboard: monitoring: grafonnet refactoring for hosts dashboards

This PR intends to refactor hosts dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit f7714de294dd7376a9a8ae5131aa429322b459c3)

Conflicts:
monitoring/grafana/dashboards/jsonnet/grafana_dashboards.jsonnet(merging all the jsonnet dashboards in one PR)

3 years agoMerge pull request #43646 from rhcs-dashboard/wip-53026-pacific
Ernesto Puerta [Mon, 25 Oct 2021 14:00:47 +0000 (16:00 +0200)]
Merge pull request #43646 from rhcs-dashboard/wip-53026-pacific

pacific: mgr/dashboard: pin a version for autopep8 and pyfakefs

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: pin a version for autopep8 and pyfakefs 43646/head
Nizamudeen A [Mon, 25 Oct 2021 08:42:57 +0000 (14:12 +0530)]
mgr/dashboard: pin a version for autopep8 and pyfakefs

Fixes: https://tracker.ceph.com/issues/53024
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 946dab4f608ec47e0a3cfefdf8e7d1afda69117f)

3 years agomgr/dashboard: monitoring: grafonnet refactoring for cephfs dashboards
Aashish Sharma [Fri, 8 Oct 2021 10:07:17 +0000 (15:37 +0530)]
mgr/dashboard: monitoring: grafonnet refactoring for cephfs dashboards

This PR intends to refactor cephfs dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit ed954b0e6ce24fbae66f78f7e4f90416b9ed7749)

3 years agomgr/dashboard: monitoring: grafonnet refactoring for osds dashboards
Aashish Sharma [Fri, 8 Oct 2021 09:58:13 +0000 (15:28 +0530)]
mgr/dashboard: monitoring: grafonnet refactoring for osds dashboards

This PR intends to refactor osds dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit e490e2f3abe707a2e891171f3c230d44e282c601)

3 years agomgr/dashboard: monitoring: grafonnet refactoring for pools dashboards
Aashish Sharma [Fri, 8 Oct 2021 09:52:46 +0000 (15:22 +0530)]
mgr/dashboard: monitoring: grafonnet refactoring for pools dashboards

This PR intends to refactor pools dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 8c48821c21f7a6b248de10ff6750a63bab1e4948)

3 years agomgr/dashboard: monitoring: grafonnet refactoring for rbd dashboards
Aashish Sharma [Fri, 8 Oct 2021 09:42:41 +0000 (15:12 +0530)]
mgr/dashboard: monitoring: grafonnet refactoring for rbd dashboards

This PR intends to refactor rbd dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit e737aaa000a31e2f37ca90eb813f031a42edef3b)

3 years agomgr/dashboard: monitoring: grafonnet refactoring for radosgw dashboards
Aashish Sharma [Fri, 8 Oct 2021 09:30:09 +0000 (15:00 +0530)]
mgr/dashboard: monitoring: grafonnet refactoring for radosgw dashboards

This PR intends to refactor radosgw dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit eb01954cd999430417555628e0099f645d371746)

3 years agorgw/tracing: unify SO version numbers within librgw2 package 43619/head
Nathan Cutler [Wed, 20 Oct 2021 10:51:02 +0000 (12:51 +0200)]
rgw/tracing: unify SO version numbers within librgw2 package

The librgw2 package contains several SO files. Two of those - librgw_op_tp.so
and librgw_rados_tp.so - had a different version number than the main librgw.

This was a violation of the openSUSE Shared Library Packaging Policy [1] but it
also seems like a "violation" of common sense.

[1] https://en.opensuse.org/openSUSE:Shared_library_packaging_policy#Package_naming

Fixes: https://tracker.ceph.com/issues/52979
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 172d6e01d5079f445044da9fe0823ceb353bdc86)

3 years agoMerge pull request #43548 from rzarzynski/pacific-50483
Yuri Weinstein [Thu, 21 Oct 2021 13:41:46 +0000 (06:41 -0700)]
Merge pull request #43548 from rzarzynski/pacific-50483

pacific: msgr/async: fix unsafe access in unregister_conn()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43610 from rhcs-dashboard/wip-pr_triage_dashboard-pacific
Ernesto Puerta [Thu, 21 Oct 2021 08:53:19 +0000 (10:53 +0200)]
Merge pull request #43610 from rhcs-dashboard/wip-pr_triage_dashboard-pacific

.github: add dashboard PRs to Dashboard project

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #43440 from rhcs-dashboard/wip-52835-pacific
Ernesto Puerta [Thu, 21 Oct 2021 08:52:42 +0000 (10:52 +0200)]
Merge pull request #43440 from rhcs-dashboard/wip-52835-pacific

pacific: qa/mgr/dashboard/test_pool: don't check HEALTH_OK

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agomon/MDSMonitor: avoid crash when decoding old FSMap epochs 43615/head
Patrick Donnelly [Tue, 12 Oct 2021 18:00:50 +0000 (14:00 -0400)]
mon/MDSMonitor: avoid crash when decoding old FSMap epochs

Fixes: https://tracker.ceph.com/issues/52820
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 631360a41ab2ad93445d35a9730200beef2bebab)

3 years agomds/FSMap: do not assert allow_standby_replay on old FSMaps 43614/head
Patrick Donnelly [Tue, 12 Oct 2021 18:39:42 +0000 (14:39 -0400)]
mds/FSMap: do not assert allow_standby_replay on old FSMaps

Octopus and older may have standby-replay daemons with
allow_standby_replay==false.

Fixes: https://tracker.ceph.com/issues/52874
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5655329b3ad6a46514ca89ef1ce8436b50d09b94)

3 years ago.github/pr-triage: rename GH token 43610/head
Ernesto Puerta [Mon, 11 Oct 2021 11:05:34 +0000 (13:05 +0200)]
.github/pr-triage: rename GH token

Repo projects use GITHUB_TOKEN instead of MY_GITHUB_TOKEN:
https://github.com/srggrs/assign-one-project-github-action/blob/master/entrypoint.sh#L19

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit 2220646c2085f6967e61d21ff19145666f5a1285)

3 years ago.github: add dashboard PRs to Dashboard project
Ernesto Puerta [Fri, 8 Oct 2021 16:43:25 +0000 (18:43 +0200)]
.github: add dashboard PRs to Dashboard project

This action automatically adds PRs with 'dashboard' label to the
'Dashboard' project (https://github.com/ceph/ceph/projects/6).

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit ed55c527f10237c0ab48038639a971e85f8e1377)

3 years agoMerge pull request #43200 from batrick/i52639
Yuri Weinstein [Wed, 20 Oct 2021 15:35:09 +0000 (08:35 -0700)]
Merge pull request #43200 from batrick/i52639

pacific: MDSMonitor: handle damaged state from standby-replay

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoqa/tasks/backfill_toofull: make test work when compression on 43437/head
Mykola Golub [Wed, 13 Oct 2021 15:22:09 +0000 (18:22 +0300)]
qa/tasks/backfill_toofull: make test work when compression on

The osd backfill reservation does not take compression into account so
we need to operate with "uncompressed" bytes when calculating nearfull
ratio.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 429ac06cbb44b8a8263beb0d0780a01cedb517ba)

3 years agoMerge pull request #43267 from cfsnyder/wip-52588-pacific
Guillaume Abrioux [Mon, 18 Oct 2021 15:55:31 +0000 (17:55 +0200)]
Merge pull request #43267 from cfsnyder/wip-52588-pacific

pacific: ceph-volume: fix lvm activate --all --no-systemd

3 years agoMerge pull request #43523 from rhcs-dashboard/wip-52911-pacific
Ernesto Puerta [Mon, 18 Oct 2021 15:11:27 +0000 (17:11 +0200)]
Merge pull request #43523 from rhcs-dashboard/wip-52911-pacific

pacific:  mgr/dashboard: replace "Ceph-cluster" Client connections with active-standby MGRs

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #43541 from rhcs-dashboard/wip-52931-pacific
Ernesto Puerta [Mon, 18 Oct 2021 15:09:29 +0000 (17:09 +0200)]
Merge pull request #43541 from rhcs-dashboard/wip-52931-pacific

pacific: mgr/dashboard: Fix orchestrator/01-hosts.e2e-spec.ts failure

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #43240 from callithea/wip-52292-pacific
Ernesto Puerta [Mon, 18 Oct 2021 15:08:22 +0000 (17:08 +0200)]
Merge pull request #43240 from callithea/wip-52292-pacific

pacific: mgr/dashboard: visual tests: Add more ignore regions for dashboard component

Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: replace string version with class 43543/head
Ernesto Puerta [Fri, 24 Sep 2021 15:46:42 +0000 (17:46 +0200)]
mgr/dashboard: replace string version with class

* APIVersion:
  * Moved to a separate file
  * Added doctests
  * Added sentinel values:
    * DEFAULT = 1.0
    * EXPERIMENTAL = 0.1
    * NONE = 0.0
  * Added to_mime_type() helper method
* Controllers.__init__:
  * Added type hints
  * Replaced string versions with APIVersions
* Feedback controller:
  * Replaced with EXPERIMENTAL (probably it should be NONE)

Fixes: https://tracker.ceph.com/issues/52480
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
 Conflicts:
src/pybind/mgr/dashboard/controllers/__init__.py
   - Remove the current changes and keep the incoming new changes
src/pybind/mgr/dashboard/controllers/crush_rule.py
   - Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/controllers/docs.py
   - Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/controllers/feedback.py
   - Deleted the file since feedback module isn't backported to pacific
src/pybind/mgr/dashboard/controllers/host.py
   - Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/openapi.yaml
   - Generated a new openapi yaml file
src/pybind/mgr/dashboard/tests/__init__.py
   - Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/tests/test_docs.py
   - Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/tests/test_host.py
   - Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/tests/test_tools.py
   - Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/tests/test_versioning.py
   - Changes related to the versioning like importing the APIVersion
src/pybind/mgr/dashboard/controllers/crush_rule.py
   - Removed the MethodMap decorator which updates the version of the
     enpoint to 2.0 because those changes which caused that version
     updating were not backported to pacific

3 years agotest: shutdown the mounter after test finishes 43475/head
Xiubo Li [Sat, 9 Oct 2021 03:12:18 +0000 (11:12 +0800)]
test: shutdown the mounter after test finishes

In the previous backport commit (5772641cb9bde083), when resolving
the conflicts, this has been missed.

Fixes: https://tracker.ceph.com/issues/52876
Signed-off-by: Xiubo Li <xiubli@redhat.com>
3 years agotest/libcephfs: put inodes after lookup 43562/head
Patrick Donnelly [Tue, 14 Sep 2021 17:02:12 +0000 (13:02 -0400)]
test/libcephfs: put inodes after lookup

Otherwise, the client umount will hang due to inability to trim the
inodes looked up using the low-level interface. This results in slow-op
warnings and an eviction:

2021-09-11T17:23:31.097+0000 7f99c3522700  0 log_channel(cluster) log [WRN] : evicting unresponsive client smithi176 (9756), after 303.924 seconds
2021-09-11T17:23:31.097+0000 7f99c3522700 10 mds.0.server autoclosing stale session client.9756 172.21.15.176:0/3891214934 last renewed caps 303.924s ago

From: /ceph/teuthology-archive/yuriw-2021-09-11_16:21:09-smoke-pacific-distro-basic-smithi/6385038/remote/smithi175/log/ceph-mds.b.log.gz

Fixes: https://tracker.ceph.com/issues/52572
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c0252063b94d811dc7863058999856ac5614d1eb)

 Conflicts:
src/test/libcephfs/test.cc

3 years agoqa: add test for cephfs upgrade sequence 43559/head
Patrick Donnelly [Fri, 1 Oct 2021 16:06:50 +0000 (12:06 -0400)]
qa: add test for cephfs upgrade sequence

This also checks max_mds>1 and allow_standby_replay are restored to
previous values.

Future work can add tests for multiple file systems (or volumes).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b1420e5771927f5c659e0e5edbc5714035f3df09)

3 years agoqa: add tasks to check mds upgrade state
Patrick Donnelly [Fri, 1 Oct 2021 16:05:42 +0000 (12:05 -0400)]
qa: add tasks to check mds upgrade state

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5a7382214fe4dbd4b79773c6e732512ade22793a)

3 years agoqa: add note about where caps are generated
Patrick Donnelly [Fri, 1 Oct 2021 16:05:12 +0000 (12:05 -0400)]
qa: add note about where caps are generated

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit dbe5573ed4781cb4b214e701c77be7bc2cddabf3)

3 years agoqa: move CephManager cluster instantiation to subtask
Patrick Donnelly [Tue, 5 Oct 2021 17:31:02 +0000 (13:31 -0400)]
qa: move CephManager cluster instantiation to subtask

This needs to be available for the cephfs_setup task so administration
mounts can run ceph commands, potentially through `cephadm shell`.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7812cfb6744fc3bce50e26aa7dd6a4e47a43bb23)

3 years agopybind/mgr/cephadm: disable allow_standby_replay during CephFS upgrade
Patrick Donnelly [Sat, 18 Sep 2021 00:15:01 +0000 (20:15 -0400)]
pybind/mgr/cephadm: disable allow_standby_replay during CephFS upgrade

Following procedure in [1].

Also: harden checks for active. Ensure "up" and "in" are both [0]. There
should be no standby-replay daemon.

[1] https://docs.ceph.com/en/pacific/cephfs/upgrading/

Fixes: https://tracker.ceph.com/issues/52654
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit bca21f01ce3bb32e0951f0fe15da88a81750a191)

3 years agopybind/mgr/cephadm: always do mds upgrade sequence
Patrick Donnelly [Thu, 23 Sep 2021 23:49:31 +0000 (19:49 -0400)]
pybind/mgr/cephadm: always do mds upgrade sequence

Minor versions also require this sequence.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 4affb5c7029f6b83d640aa7b7206d9cf61e75f1d)

3 years agomgr/dashboard: make modified API endpoints backward compatible
Avan Thakkar [Thu, 23 Sep 2021 11:15:16 +0000 (16:45 +0530)]
mgr/dashboard: make modified API endpoints backward compatible

Fixes: https://tracker.ceph.com/issues/52480
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Introducing APIVersion class to handle versioning for API-endpints and making
them backward compatible.

3 years agomgr/dashboard: clean-up controllers
Ernesto Puerta [Tue, 7 Sep 2021 15:07:48 +0000 (17:07 +0200)]
mgr/dashboard: clean-up controllers

Fixes: https://tracker.ceph.com/issues/52589
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
 Conflicts:
src/pybind/mgr/dashboard/CMakeLists.txt
   - Added some testts in the CephTest section

3 years agomsgr/async: fix unsafe access in unregister_conn() 43548/head
Sage Weil [Mon, 19 Apr 2021 14:26:30 +0000 (09:26 -0500)]
msgr/async: fix unsafe access in unregister_conn()

We were looking at anon_conns and accepting_conns without holding
the lock (deleted_lock is not sufficient).

Drop this test, and move the decrements:

- inc when we add to conns or anon_conns (no changes there)
- dec when we remove from deleted_conns (several different paths!)

Fixes: https://tracker.ceph.com/issues/49237
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d51d80b3234e17690061f65dc7e1515f4244a5a3)
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agomgr/dashboard: Fix orchestrator/01-hosts.e2e-spec.ts failure 43541/head
Nizamudeen A [Thu, 7 Oct 2021 15:36:29 +0000 (21:06 +0530)]
mgr/dashboard: Fix orchestrator/01-hosts.e2e-spec.ts failure

The test is failing on deleting a host because the agent daemon is
present in that host. Its not possible to simply delete a host. We need
to drain it first and then delete it.

Fixes: https://tracker.ceph.com/issues/52764
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit db5cfb15e55dadf7bd5c381f53a4ea548fcea152)

3 years agomgr/dashboard: replace Client connections with active-stdby mgrs 43523/head
Avan Thakkar [Thu, 30 Sep 2021 22:26:42 +0000 (03:56 +0530)]
mgr/dashboard: replace Client connections with active-stdby mgrs

Fixes: https://tracker.ceph.com/issues/52121
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit d388c5e958ddf5447c78db50ca2061bb443d2227)

3 years agoosd: fix partial recovery become whole object recovery after restart osd 43513/head
Jianwei Zhang [Mon, 13 Sep 2021 10:13:18 +0000 (18:13 +0800)]
osd: fix partial recovery become whole object recovery after restart osd

support SERVER_OCTOPUS feature for pg_missing_item::encode()

Fixes: https://tracker.ceph.com/issues/52583
Signed-off-by: Jianwei Zhang <jianwei1216@qq.com>
(cherry picked from commit dcdb188b6f577551fb377ba34145419f81322b03)

3 years agoos/bluestore: list obj which equals to pend 43512/head
Kefu Chai [Fri, 24 Sep 2021 15:33:03 +0000 (23:33 +0800)]
os/bluestore: list obj which equals to pend

otherwise we could have failures like

scrub : stat mismatch, got 3/4 objects, 1/2 clones, 3/4 dirty, 3/4 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 49/56 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes."

where the numbers of scrubbed object, clones, dirty and omap are always
less than the total number of corresponding numbers, if the PG contains
object(s) whose hash happens to be 0xffffffff.

in this change, if the calculated hash of the upper bound is greater
than the maximum possible number represented by uint32_t, in addition to
setting the hash of the upper bound hobj to 0xffffffff, we also set the
nspace of hobj of the upper bound to "\xff", so that the upper bound
is greater than an hobj whose hash happens to be 0xfffffff. please note,
the nspace of "\xff" is not an ascii string, so it's not likely to be
less than a real-world nspace of an hobj.

with this new *greater* upper bound, we are able to include the previous
missing hobj when listing the objects in a PG. so the scrub won't be
annoyed when the number of objects does not match.

Fixes: https://tracker.ceph.com/issues/52705
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit ffab13bcd9006c1f961a24b8016df9d1fe06ba1d)

3 years agoos/bluestore: use scope_guard to log latency
Kefu Chai [Wed, 22 Sep 2021 16:42:33 +0000 (00:42 +0800)]
os/bluestore: use scope_guard to log latency

simpler this way, and avoid using `goto`.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 715a83822ebc1a3d102d1ec13323b69db0600719)

3 years agomsg/async/ProtocolV2: replace ltt_recv_stamp with recv_stamp 43511/head
Dongdong Tao [Tue, 28 Sep 2021 06:40:43 +0000 (14:40 +0800)]
msg/async/ProtocolV2: replace ltt_recv_stamp with recv_stamp

Fixes: https://tracker.ceph.com/issues/52739
Signed-off-by: dongdong tao <dongdong.tao@canonical.com>
(cherry picked from commit 1b1a91c31ba6078caff045c499b8737e0068460f)

3 years agomsg/async/ProtocolV2: Set the recv_stamp at the beginning of receiving a message...
taodd [Sat, 25 Sep 2021 03:56:02 +0000 (11:56 +0800)]
msg/async/ProtocolV2: Set the recv_stamp at the beginning of receiving a message instead of after receiving.

Fixes: https://tracker.ceph.com/issues/52739
Signed-off-by: dongdong tao <dongdong.tao@canonical.com>
(cherry picked from commit 5ca30f396bface2a8e95a0efb1b97f8c1b64de1c)

3 years agoMerge pull request #43368 from tchaikov/pacific-pr-39602
Yuri Weinstein [Tue, 12 Oct 2021 12:44:53 +0000 (05:44 -0700)]
Merge pull request #43368 from tchaikov/pacific-pr-39602

pacific: mgr/influx: use "N/A" for unknown hostname

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #43351 from rhcs-dashboard/wip-52772-pacific
Yuri Weinstein [Tue, 12 Oct 2021 12:44:18 +0000 (05:44 -0700)]
Merge pull request #43351 from rhcs-dashboard/wip-52772-pacific

pacific: qa/mgr/dashboard: add extra wait to test

Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #43347 from rhcs-dashboard/wip-52763-pacific
Yuri Weinstein [Tue, 12 Oct 2021 12:43:31 +0000 (05:43 -0700)]
Merge pull request #43347 from rhcs-dashboard/wip-52763-pacific

pacific: mgr/dashboard: Move force maintenance test to the workflow test suite

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #43167 from ktdreyer/pacific-52610-cmake-thread-libs-init
Yuri Weinstein [Tue, 12 Oct 2021 12:41:57 +0000 (05:41 -0700)]
Merge pull request #43167 from ktdreyer/pacific-52610-cmake-thread-libs-init

pacific: cmake: link Threads::Threads instead of CMAKE_THREAD_LIBS_INIT

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #43199 from vshankar/wip-52627
Yuri Weinstein [Fri, 8 Oct 2021 13:34:17 +0000 (06:34 -0700)]
Merge pull request #43199 from vshankar/wip-52627

pacific: mgr/mirroring: remove unnecessary fs_name arg from daemon status command

Reviewed-by: Xiubo Li <xiubli@redhat.com>
3 years agoMerge pull request #43198 from vshankar/wip-52444
Yuri Weinstein [Fri, 8 Oct 2021 13:33:42 +0000 (06:33 -0700)]
Merge pull request #43198 from vshankar/wip-52444

pacific: cephfs-mirror: shutdown ClusterWatcher on termination

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
3 years agoMerge pull request #43148 from lxbsz/fair_mutex
Yuri Weinstein [Fri, 8 Oct 2021 13:32:12 +0000 (06:32 -0700)]
Merge pull request #43148 from lxbsz/fair_mutex

pacific: mds: switch mds_lock to fair mutex to fix the slow performance issue

Reviewed-by: Jeff Layton <jlayton@redhat.com>
3 years agoMetricCollector.h: Add check to prevent mgr from crashing 43445/head
Aswin Toni [Fri, 1 Oct 2021 14:12:22 +0000 (16:12 +0200)]
MetricCollector.h: Add check to prevent mgr from crashing

Fixes: https://tracker.ceph.com/issues/52801
Signed-off-by: Aswin Toni <aswin.toni@cern.ch>
(cherry picked from commit 9a05872fdd499575961ee1a8d188d19054841eb8)

3 years agoqa/mgr/dashboard/test_pool: don't check HEALTH_OK 43440/head
Ernesto Puerta [Wed, 22 Sep 2021 12:25:44 +0000 (14:25 +0200)]
qa/mgr/dashboard/test_pool: don't check HEALTH_OK

Fixes: https://tracker.ceph.com/issues/48845
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit 2283cb068b82033b14587c7bac6a28440221dcd8)

3 years agoqa/suites/rados: add backfill_toofull test
Mykola Golub [Thu, 9 Sep 2021 11:44:25 +0000 (14:44 +0300)]
qa/suites/rados: add backfill_toofull test

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 76743e005866664795e9240460734b31108824e2)

3 years agoqa/tasks/ceph_manager: fix assertion
Mykola Golub [Sun, 23 May 2021 08:55:33 +0000 (11:55 +0300)]
qa/tasks/ceph_manager: fix assertion

The osd may be 0.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit e0a926a2c18d76225fd4d4051bc19b9a1917b932)

3 years agoosd: re-cache peer_bytes on every peering state activate
Mykola Golub [Mon, 30 Aug 2021 06:58:04 +0000 (07:58 +0100)]
osd: re-cache peer_bytes on every peering state activate

peer_bytes is used for backfill reservation request and may be
reset if backfill is interrupted, and we want it set back before
continuing backfill and re-sending the reservation request.

Fixes: https://tracker.ceph.com/issues/52448
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit bdfdf96d2f6c3cf7e5595ae5b8238fd4c0b3c6bc)

3 years agoMerge pull request #43348 from cfsnyder/wip-52350-pacific
Yuri Weinstein [Tue, 5 Oct 2021 15:00:24 +0000 (08:00 -0700)]
Merge pull request #43348 from cfsnyder/wip-52350-pacific

pacific: rgw: fix sts memory leak

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #42643 from cfsnyder/wip-51803-pacific
Yuri Weinstein [Tue, 5 Oct 2021 14:59:37 +0000 (07:59 -0700)]
Merge pull request #42643 from cfsnyder/wip-51803-pacific

pacific: rgw/notifications: send correct size in case of delete marker creation

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoqa/tasks/mgr: skip test_diskprediction_local on python>=3.8 43421/head
Kefu Chai [Wed, 7 Apr 2021 05:38:27 +0000 (13:38 +0800)]
qa/tasks/mgr: skip test_diskprediction_local on python>=3.8

query the python version before trying to test diskprediction_local

Fixes: https://tracker.ceph.com/issues/50196
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 39b2b5edc008900d531be95ece1ce75a1e036914)