]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agomon/MDSMonitor: do not ignore mds's down:dne request 37858/head
chencan [Sat, 17 Oct 2020 07:26:51 +0000 (15:26 +0800)]
mon/MDSMonitor: do not ignore mds's down:dne request

Fixes: https://tracker.ceph.com/issues/47881
Signed-off-by: chencan <chen.can2@zte.com.cn>
(cherry picked from commit 768d7fc4e8b74c88ea2a623ee4d21ac1f20d8c7a)

4 years agoMerge pull request #37722 from rishabh-d-dave/wip-47845-octopus
Yuri Weinstein [Fri, 23 Oct 2020 18:51:21 +0000 (11:51 -0700)]
Merge pull request #37722 from rishabh-d-dave/wip-47845-octopus

octopus: ceph-volume: add no-systemd argument to zap

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
4 years agoMerge pull request #37705 from smithfarm/wip-47898-octopus
Yuri Weinstein [Fri, 23 Oct 2020 14:03:21 +0000 (07:03 -0700)]
Merge pull request #37705 from smithfarm/wip-47898-octopus

octopus: mon: have 'mon stat' output json as well

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
4 years agoMerge pull request #37697 from s0nea/wip-47602-octopus
Yuri Weinstein [Fri, 23 Oct 2020 14:02:49 +0000 (07:02 -0700)]
Merge pull request #37697 from s0nea/wip-47602-octopus

octopus: Enable per-RBD image monitoring

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
4 years agoMerge pull request #37674 from Vicente-Cheng/wip-47706-octopus
Yuri Weinstein [Fri, 23 Oct 2020 14:00:22 +0000 (07:00 -0700)]
Merge pull request #37674 from Vicente-Cheng/wip-47706-octopus

octopus: osdc/ObjectCacher: overwrite might cause stray read request callbacks

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
4 years agoMerge pull request #37686 from bk201/wip-47832-octopus
Lenz Grimmer [Fri, 23 Oct 2020 11:55:16 +0000 (13:55 +0200)]
Merge pull request #37686 from bk201/wip-47832-octopus

octopus: mgr/dashboard: fix the error when exporting CephFS path "/" in NFS exports

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
4 years agoMerge pull request #37673 from Vicente-Cheng/wip-47815-octopus
Yuri Weinstein [Wed, 21 Oct 2020 16:36:30 +0000 (09:36 -0700)]
Merge pull request #37673 from Vicente-Cheng/wip-47815-octopus

octopus: rgw: fix setting of namespace in ordered and unordered bucket listing

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #37672 from Vicente-Cheng/wip-47850-octopus
Yuri Weinstein [Wed, 21 Oct 2020 16:35:58 +0000 (09:35 -0700)]
Merge pull request #37672 from Vicente-Cheng/wip-47850-octopus

octopus: rgw/rgw_file: Fix the incorrect lru object eviction

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #37640 from pritha-srivastava/wip-47836-octopus
Yuri Weinstein [Wed, 21 Oct 2020 16:35:30 +0000 (09:35 -0700)]
Merge pull request #37640 from pritha-srivastava/wip-47836-octopus

octopus: rgw: adds code for creating and managing oidc provider entities in rgw and for offline validation of OpenID Connect Access and ID Token.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #37434 from rhcs-dashboard/wip-47675-octopus
Tatjana Dehler [Wed, 21 Oct 2020 16:28:31 +0000 (18:28 +0200)]
Merge pull request #37434 from rhcs-dashboard/wip-47675-octopus

octopus: mgr/dashboard: cluster > manager modules

4 years agoMerge pull request #37365 from smithfarm/wip-47147-octopus
Yuri Weinstein [Wed, 21 Oct 2020 16:25:38 +0000 (09:25 -0700)]
Merge pull request #37365 from smithfarm/wip-47147-octopus

octopus: qa/tasks/nfs: Test mounting of export created with nfs command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37578 from callithea/wip-47770-octopus
Lenz Grimmer [Wed, 21 Oct 2020 10:56:04 +0000 (12:56 +0200)]
Merge pull request #37578 from callithea/wip-47770-octopus

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
4 years agoMerge pull request #37331 from callithea/wip-47461-octopus
Lenz Grimmer [Wed, 21 Oct 2020 09:57:26 +0000 (11:57 +0200)]
Merge pull request #37331 from callithea/wip-47461-octopus

octopus: mgr/dashboard: Update datatable only when necessary

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agoMerge pull request #37367 from smithfarm/wip-47247-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:35:01 +0000 (08:35 -0700)]
Merge pull request #37367 from smithfarm/wip-47247-octopus

octopus: qa: ignore expected mds failover message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37366 from smithfarm/wip-47151-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:34:34 +0000 (08:34 -0700)]
Merge pull request #37366 from smithfarm/wip-47151-octopus

octopus: pybind/mgr/volumes: add global lock debug

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37363 from smithfarm/wip-47089-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:33:54 +0000 (08:33 -0700)]
Merge pull request #37363 from smithfarm/wip-47089-octopus

octopus: mds: standy-replay mds remained in the "resolve" state after resta…

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37360 from smithfarm/wip-47083-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:33:20 +0000 (08:33 -0700)]
Merge pull request #37360 from smithfarm/wip-47083-octopus

octopus: mds: fix 'forward loop' when forward_all_requests_to_auth is set

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37354 from smithfarm/wip-46940-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:32:47 +0000 (08:32 -0700)]
Merge pull request #37354 from smithfarm/wip-46940-octopus

octopus: mds: place MDSGatherBuilder on the stack

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37358 from smithfarm/wip-47021-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:27:27 +0000 (08:27 -0700)]
Merge pull request #37358 from smithfarm/wip-47021-octopus

octopus: cephfs: libcephfs: ignore restoring the open files limit

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37357 from smithfarm/wip-47018-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:27:01 +0000 (08:27 -0700)]
Merge pull request #37357 from smithfarm/wip-47018-octopus

octopus: mds: fix kcephfs parse dirfrag's ndist is always 0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37356 from smithfarm/wip-47016-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:26:29 +0000 (08:26 -0700)]
Merge pull request #37356 from smithfarm/wip-47016-octopus

octopus: mds: revert the decode version

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
4 years agoMerge pull request #37355 from smithfarm/wip-46942-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:25:24 +0000 (08:25 -0700)]
Merge pull request #37355 from smithfarm/wip-46942-octopus

octopus: mds: resolve SIGSEGV in waiting for uncommitted fragments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37353 from smithfarm/wip-46859-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:24:52 +0000 (08:24 -0700)]
Merge pull request #37353 from smithfarm/wip-46859-octopus

octopus: mds: do not raise "client failing to respond to cap release" when client working set is reasonable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37352 from smithfarm/wip-46857-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:24:16 +0000 (08:24 -0700)]
Merge pull request #37352 from smithfarm/wip-46857-octopus

octopus: qa: add debugging for volumes plugin use of libcephfs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37351 from smithfarm/wip-46855-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:23:46 +0000 (08:23 -0700)]
Merge pull request #37351 from smithfarm/wip-46855-octopus

octopus: cephfs: client: use non-static dirent for thread-safety

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37350 from smithfarm/wip-46463-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:22:19 +0000 (08:22 -0700)]
Merge pull request #37350 from smithfarm/wip-46463-octopus

octopus: pybind/cephfs: fix custom exception raised by cephfs.pyx

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37334 from smithfarm/wip-47087-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:21:51 +0000 (08:21 -0700)]
Merge pull request #37334 from smithfarm/wip-47087-octopus

octopus: mds: don't recover files after normal session close

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37249 from Vicente-Cheng/wip-46786-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:21:19 +0000 (08:21 -0700)]
Merge pull request #37249 from Vicente-Cheng/wip-46786-octopus

octopus: cephfs: client: fix extra open ref decrease

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37248 from Vicente-Cheng/wip-46783-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:20:46 +0000 (08:20 -0700)]
Merge pull request #37248 from Vicente-Cheng/wip-46783-octopus

octopus: mds/CInode: Optimize only pinned by subtrees check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37247 from Vicente-Cheng/wip-46637-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:20:19 +0000 (08:20 -0700)]
Merge pull request #37247 from Vicente-Cheng/wip-46637-octopus

octopus: mds: optimize random threshold lookup for dentry load

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37243 from Vicente-Cheng/wip-46636-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:19:52 +0000 (08:19 -0700)]
Merge pull request #37243 from Vicente-Cheng/wip-46636-octopus

octopus: mds: fix nullptr dereference in MDCache::finish_rollback

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37240 from Vicente-Cheng/wip-46634-octopus
Yuri Weinstein [Tue, 20 Oct 2020 15:19:22 +0000 (08:19 -0700)]
Merge pull request #37240 from Vicente-Cheng/wip-46634-octopus

octopus: mds: fix mds forwarding request 'no_available_op_found'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #37520 into octopus
Jan Fajerski [Tue, 20 Oct 2020 13:05:50 +0000 (15:05 +0200)]
Merge PR #37520 into octopus

* refs/pull/37520/head:
ceph-volume: don't exit before empty report can be printed
PendingReleaseNotes: add note about batch refactor
pybind/mgr/orchestrator: fix preview for new c-v batch output
ceph-volume batch: return valid empty json reports
python-common: do not skip unavailable devices
ceph-volume: pass filter_for_batch as keyword argument
ceph-volume batch: fix very_fast_allocation plan and add tests
ceph-volume: batch: call the right prepare method
python-common drive-groups: allow strings for wal and db size
ceph-volume inventory: add option to filter unwanted devices
ceph-volume: address review comments
ceph-volume: batch: fix size retrieval for lvs
ceph-volume: include encryption in batch report
ceph-volume lvm batch: use namedtuple instead of tuple
ceph-volume: address review comments, mostly tidying, clarification
idempotency must result in the same outcome
ceph-volume: batch test should pass --journal-devices with filestore
ceph-volume: make --journal optional, add --journal-slots
doc: update ceph-volume lvm batch docs
py-common/deployment: drop workarounds for old batch
ceph-volume batch: add deprecation warning for auto behaviour
ceph-volume batch: add ceph.conf mocking to pass tests
ceph-volume batch: use disk.Size for size args
ceph-volume batch: Fix osd_ids passing and improve plan formatting
ceph-volume batch: track rel_size in percent, more tests
ceph-volume batch: improve backwards compatibility
ceph-volume: batch - enable legacy auto behaviour
ceph-volume: batch - major refactor
ceph-volume: Device - available_lvm if 10 extents are free.
ceph-volume: Device - add vg_free property
ceph-volume: prepare/create - size args as Size class
ceph-volume: disk.Size - add cast to bool
ceph-volume: api/lvm - add VolumeGroup.free_percent property
ceph-volume: util.device -  add vg_free_percent property
ceph-volume: api/lvm - query LV units in bytes
ceph-volume: lvm/common - refactor common arg specification
ceph-volume: batch: fix argument help message

Reviewed-by: Rishabh Dave <ridave@redhat.com>
4 years agoceph-volume: add no-systemd argument to zap 37722/head
wanghongxu [Fri, 18 Sep 2020 03:28:23 +0000 (11:28 +0800)]
ceph-volume: add no-systemd argument to zap

In some container deployment scenarios, systemd services are not
used, so the no-systemd parameter is useful.

Fixes: https://tracker.ceph.com/issues/47541
Signed-off-by: wanghongxu <wanghongxu@t2cloud.net>
(cherry picked from commit 54c011b3d44fd29000918bef8de389bc1784bb07)

4 years agomgr/dashboard: Show warning when replicated size is 1 37578/head
Sebastian Krah [Fri, 2 Oct 2020 09:07:40 +0000 (11:07 +0200)]
mgr/dashboard: Show warning when replicated size is 1

Issues a warning when creating a replicated pool and the replicated size
is 1. It won't stop the user from creating the pool, but will give the
user a hint that no replication will be created.

Fixes: https://tracker.ceph.com/issues/42404
Signed-off-by: Sebastian Krah <skrah@suse.com>
(cherry picked from commit af1a7dbfe2b0e2e40650d64d4c3247f4961743c2)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/styles/defaults/_bootstrap-defaults.scss
- The file does not exist in octopus, moving a slightly changed version of the diff
  to src/pybind/mgr/dashboard/frontend/src/styles.scss instead

4 years agomgr/dashboard: Fix issue in test: delay lower than expected 37697/head
Patrick Seidensal [Mon, 14 Sep 2020 12:32:10 +0000 (14:32 +0200)]
mgr/dashboard: Fix issue in test: delay lower than expected

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 746bac29afd915bb85279234c665d38c807ef2b7)

4 years agomgr/dashboard: update dashboard docs with RBD image monitoring paragraph
Patrick Seidensal [Mon, 14 Sep 2020 10:41:56 +0000 (12:41 +0200)]
mgr/dashboard: update dashboard docs with RBD image monitoring paragraph

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 0c0e6ee496520145782d82a250c0d0f59378222f)

Conflicts:
doc/mgr/dashboard.rst
Fixed conflict because https://github.com/ceph/ceph/commit/d6369ac159a759b00769c0af34131ecba422e8d5
has not been backported to octopus.

4 years agoqa/cephtool: test 'mon stat' commands 37705/head
Joao Eduardo Luis [Tue, 13 Oct 2020 08:24:34 +0000 (08:24 +0000)]
qa/cephtool: test 'mon stat' commands

Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit 122388429d01ef2f294dc2846d16d88aa0bdba68)

Conflicts:
qa/workunits/cephtool/test.sh
- drop unrelated "# test elector" comment (elector test not backported)

4 years agomon: have 'mon stat' output json as well
Joao Eduardo Luis [Mon, 12 Oct 2020 11:29:16 +0000 (11:29 +0000)]
mon: have 'mon stat' output json as well

Fixes: https://tracker.ceph.com/issues/46816
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit c148a3cde5c256576d0a67a40321e543fdf891bf)

4 years agomgr/dashboard: Enable per RBD graphs in details of RBD
Patrick Seidensal [Fri, 11 Sep 2020 09:47:34 +0000 (11:47 +0200)]
mgr/dashboard: Enable per RBD graphs in details of RBD

Fixes: https://tracker.ceph.com/issues/47433
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 3d3230b99bf075020256bdc7cf082fd54fc61992)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-details/rbd-details.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-details/rbd-details.component.ts
Fixed conflicts because https://github.com/ceph/ceph/pull/35290,
https://github.com/ceph/ceph/pull/35311 and https://github.com/ceph/ceph/pull/35574
have not been backported to octopus.

4 years agomgr/dashboard: fix the error when exporting CephFS path "/" in NFS exports 37686/head
Kiefer Chang [Tue, 15 Sep 2020 08:27:44 +0000 (16:27 +0800)]
mgr/dashboard: fix the error when exporting CephFS path "/" in NFS exports

Fixes: https://tracker.ceph.com/issues/47397
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 5d0d7f89285c43d70d0918a0ae71e77f602809cd)

4 years agorgw: remove unused librgw_admin_user 37640/head
Casey Bodley [Wed, 25 Mar 2020 14:50:07 +0000 (10:50 -0400)]
rgw: remove unused librgw_admin_user

unused and not worth the effort to maintain

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c89b59428e66ae6d89d1fd27a4280045c57251cb)

4 years agolibrbd: skip cache initialization if data IoCtx is invalid 37674/head
Jason Dillaman [Mon, 21 Sep 2020 18:05:17 +0000 (14:05 -0400)]
librbd: skip cache initialization if data IoCtx is invalid

This resolves the potential for an assertion failure in the cache
constructors that occurs when the data IoCtx is not valid. Errors
are deferred until actual IOs are issued against the invalid data
pool.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ad1016b014d4c33dee3be66376bf9f3394160501)

Conflicts:
src/librbd/plugin/ParentCache.cc
  - parent_cache_enabled is not split to above file.
    We just add another data ioctx checking before
    sending parent cache

4 years agoosdc/ObjectCacher: overwrite might cause stray read request callbacks
Jason Dillaman [Mon, 21 Sep 2020 16:53:37 +0000 (12:53 -0400)]
osdc/ObjectCacher: overwrite might cause stray read request callbacks

In librbd, if readahead is active, there might be a pending read request
for the cache which is then (partially) overwritten by a write request.
This overwrite will cause bh splits and merges which can cause the
bh read callback to fail to invoke the pending read callbacks.

Fixes: https://tracker.ceph.com/issues/46822
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 94d43165ed7319d163640f38d154f8f80408eb14)

4 years agorgw: fix setting of namespace in ordered and unordered bucket listing 37673/head
J. Eric Ivancich [Tue, 6 Oct 2020 16:42:22 +0000 (12:42 -0400)]
rgw: fix setting of namespace in ordered and unordered bucket listing

The namespace is not always set correctly during bucket listing. This
can, for example, cause the listing of incomplete multipart uploads,
which are in the _multipart_ namespace, to not paginate correctly, and
cause entries to be re-listed.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit bd6f163f366753e8ec42b85a53334f4bf78916bd)

4 years agorgw/rgw_file: Fix the incorrect lru object eviction 37672/head
luo rixin [Tue, 1 Sep 2020 09:06:40 +0000 (17:06 +0800)]
rgw/rgw_file: Fix the incorrect lru object eviction

In func lookup_fh, when RGWFileHandle not be found in fh_cache, it
need to recycle an object and create an new RGWFileHandle. When there
are multi threads use lookup_fh to find and create RGWFileHandle concurrently,
it must to make sure evict lru object from the partiton of fh_cache which new
RGWFileHandle will be inserted to.

Fixes: https://tracker.ceph.com/issues/47235
Signed-off-by: luo rixin <luorixin@huawei.com>
(cherry picked from commit f2097338722d7f2526bb815da47695f2da17fcce)

4 years agoMerge pull request #37562 from smithfarm/wip-47762-octopus
Nathan Cutler [Tue, 13 Oct 2020 16:37:47 +0000 (18:37 +0200)]
Merge pull request #37562 from smithfarm/wip-47762-octopus

octopus: mgr/prometheus: add pool compression stats

Reviewed-by: Boris Ranto <branto@redhat.com>
4 years agoMerge pull request #37504 from smithfarm/wip-arm-isa-l-octopus
Josh Durgin [Tue, 13 Oct 2020 16:15:13 +0000 (09:15 -0700)]
Merge pull request #37504 from smithfarm/wip-arm-isa-l-octopus

octopus: erasure-code: enable isa-l EC for aarch64 platform

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37450 from callithea/wip-47621-octopus
Yuri Weinstein [Mon, 12 Oct 2020 23:50:32 +0000 (16:50 -0700)]
Merge pull request #37450 from callithea/wip-47621-octopus

octopus: mgr/dashboard: fix security scopes of some NFS-Ganesha endpoints

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
4 years agoMerge pull request #37383 from ukernel/octopus-47608
Yuri Weinstein [Mon, 12 Oct 2020 18:39:21 +0000 (11:39 -0700)]
Merge pull request #37383 from ukernel/octopus-47608

octopus: mds: reduce memory usage of open file table prefetch #37382

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37372 from smithfarm/wip-47604-octopus
Yuri Weinstein [Mon, 12 Oct 2020 18:38:58 +0000 (11:38 -0700)]
Merge pull request #37372 from smithfarm/wip-47604-octopus

octopus: mds: fix purge_queue's _calculate_ops is inaccurate

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37371 from smithfarm/wip-47601-octopus
Yuri Weinstein [Mon, 12 Oct 2020 18:38:33 +0000 (11:38 -0700)]
Merge pull request #37371 from smithfarm/wip-47601-octopus

octopus: mgr/volumes/nfs: Check if orchestrator spec service_id is valid

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37370 from smithfarm/wip-47260-octopus
Yuri Weinstein [Mon, 12 Oct 2020 18:38:07 +0000 (11:38 -0700)]
Merge pull request #37370 from smithfarm/wip-47260-octopus

octopus: cephfs: client: handle readdir reply without Fs cap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37369 from smithfarm/wip-47255-octopus
Yuri Weinstein [Mon, 12 Oct 2020 18:37:33 +0000 (11:37 -0700)]
Merge pull request #37369 from smithfarm/wip-47255-octopus

octopus: cephfs: client: make Client::open() pass proper cap mask to path_walk

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #37368 from smithfarm/wip-47253-octopus
Yuri Weinstein [Mon, 12 Oct 2020 18:37:03 +0000 (11:37 -0700)]
Merge pull request #37368 from smithfarm/wip-47253-octopus

octopus: cephfs: osdc/Journaler: do not call onsafe->complete() if onsafe is 0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agorgw: adding code for policy evaluation for ops
Pritha Srivastava [Sat, 18 Apr 2020 17:08:47 +0000 (22:38 +0530)]
rgw: adding code for policy evaluation for ops
like getbucketversioning, putbucketversioning etc

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 5f86459381ed9f3c5565d1b74fc064f11b55c275)

4 years agorgw: Correcting pool name for oidc provider for older
Pritha Srivastava [Fri, 17 Apr 2020 03:49:49 +0000 (09:19 +0530)]
rgw: Correcting pool name for oidc provider for older
versions of osd.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 1e50f8cac9c7e16322ec7d8177d8679ab29cc28a)

4 years agorgw: fix for make failure in test folder.
Pritha Srivastava [Mon, 13 Apr 2020 13:17:29 +0000 (18:47 +0530)]
rgw: fix for make failure in test folder.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 47dc509366162121a392651be81010235e8f5154)

4 years agorgw: adds code for role session to be used an ARN.
Pritha Srivastava [Tue, 31 Mar 2020 10:07:20 +0000 (15:37 +0530)]
rgw: adds code for role session to be used an ARN.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 039a2b9ce52e3ed2c5422423deaf27c32de3773e)

4 years agorgw: adding new iam ops for iam oidc provider
Pritha Srivastava [Mon, 30 Mar 2020 10:43:31 +0000 (16:13 +0530)]
rgw: adding new iam ops for iam oidc provider
to correct unittest faliure.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 9f3d772d8061563dd189bb6442d68fff576e162e)

4 years agorgw: modifying documentation to include explanation of
Pritha Srivastava [Sun, 29 Mar 2020 17:46:30 +0000 (23:16 +0530)]
rgw: modifying documentation to include explanation of
open id connect provider related REST APIs and removing
references to token introspection for validating incoming
web token.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit d228d613b00067e1c72f65afdbf90c3cf2fee396)

4 years agorgw: adds a commom method to remove prefix(www, http and https)
Pritha Srivastava [Sun, 29 Mar 2020 10:20:32 +0000 (15:50 +0530)]
rgw: adds a commom method to remove prefix(www, http and https)
from a url.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit ca96ae806f79063922f27c3b4628026b9930fd89)

4 years agorgw: adds code to check for client id and cert thumbprint
Pritha Srivastava [Sun, 29 Mar 2020 04:56:03 +0000 (10:26 +0530)]
rgw: adds code to check for client id and cert thumbprint
while validating incoming web token in AssumeRoleWithWebIdentity.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 75b4060c7732bb00ea0811771920658a6492b168)

4 years agorgw: Added code for offline OpenId Connect/ OAuth 2.0 tokens
Pritha Srivastava [Thu, 26 Mar 2020 12:53:58 +0000 (18:23 +0530)]
rgw: Added code for offline OpenId Connect/ OAuth 2.0 tokens
if the token is a JWT.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 7566664f89be062e0c9f3519dc60b94c8af5e2a4)

4 years agorgw: Adding jwt-cpp header only library for JWT validation.
Pritha Srivastava [Thu, 26 Mar 2020 12:52:01 +0000 (18:22 +0530)]
rgw: Adding jwt-cpp header only library for JWT validation.
https://github.com/Thalhammer/jwt-cpp

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 8052d6e322dc662ea9cf83431b662a414e11ea1d)

4 years agorgw: adds REST APIs for OpenID connect providers manipulation.
Pritha Srivastava [Sat, 28 Mar 2020 02:47:54 +0000 (08:17 +0530)]
rgw: adds REST APIs for OpenID connect providers manipulation.

This commit implements:
1. CreateOpenIDConnectProvider
2. DeleteOpenIDConnectProvider
3. ListOpenIDConnectProviders
4. GetOpenIDConnectProviders

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit aec4815c21dccd3be5766c985488fe4f68a41079)

4 years agomds/OpenFileTable: add option that disables dirfrags prefetch 37383/head
Yan, Zheng [Mon, 13 Jul 2020 02:50:27 +0000 (10:50 +0800)]
mds/OpenFileTable: add option that disables dirfrags prefetch

Dirfrags prefetch may speedup rejoin stage of mds recovery. But it may
use lots of memory and cause mds out of memory.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit cc19fc624b1ee4d7e3248d1dfc8f89f8879a46bf)

4 years agomds: use parent/d_name in cap_reconnect_t to speedup MDCache::open_ino()
Yan, Zheng [Mon, 13 Jul 2020 01:55:29 +0000 (09:55 +0800)]
mds: use parent/d_name in cap_reconnect_t to speedup MDCache::open_ino()

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 3e7c5b4077b68ec519f9e3e98fea8fd9dcf9cbc3)

4 years agoclient: encode inode's parent/d_name in cap_reconnect_t
Yan, Zheng [Mon, 6 Jul 2020 09:04:51 +0000 (17:04 +0800)]
client: encode inode's parent/d_name in cap_reconnect_t

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 7f43840fd3f9a8f88f809dcad5d3d381e76ba7f3)

4 years agomds: explicitly track files with client writeable ranges
Yan, Zheng [Fri, 10 Jul 2020 09:56:19 +0000 (17:56 +0800)]
mds: explicitly track files with client writeable ranges

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 44f67be02b8e8aa8fc69283bf12107c2df3f8277)

 Conflicts:
src/mds/CInode.cc
src/mds/Locker.cc
src/mds/MDCache.cc
src/mds/Server.cc
src/mds/StrayManager.cc

4 years agomds: properly set CLIENTWRITEABLE flag for imported caps
Yan, Zheng [Fri, 10 Jul 2020 08:43:14 +0000 (16:43 +0800)]
mds: properly set CLIENTWRITEABLE flag for imported caps

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 8c56adabb36f2983437fa73f88c00b7d62d0a880)

4 years agomds: in-place update projected inode's client ranges
Yan, Zheng [Fri, 10 Jul 2020 01:42:34 +0000 (09:42 +0800)]
mds: in-place update projected inode's client ranges

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 994017b6155af7fbb5a53f76f6a4525b363ba07f)

 Conflicts:
src/mds/Locker.cc
src/mds/Server.cc

4 years agomds: only add inodes that clients want Frw or excl caps to open file table
Yan, Zheng [Thu, 9 Jul 2020 02:43:12 +0000 (10:43 +0800)]
mds: only add inodes that clients want Frw or excl caps to open file table

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit e6de138f3334adc26cac4c01dfbbdf8a2472679a)

4 years agomds/OpenFileTable: reduce anchor map lookup during prefetch
Yan, Zheng [Tue, 7 Jul 2020 08:14:22 +0000 (16:14 +0800)]
mds/OpenFileTable: reduce anchor map lookup during prefetch

reduce one anchor map lookup for each inode

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 1adc21e1b46744fa810b9d29530a04369af303e7)

4 years agomds/OpenFileTable: merge dirfrag set into anchor map
Yan, Zheng [Mon, 6 Jul 2020 07:35:30 +0000 (15:35 +0800)]
mds/OpenFileTable: merge dirfrag set into anchor map

Track dirfrags whose child inodes have caps in corresponding dir
inodes' anchor objects.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 12691168dfed4c6cfb97ad3f4841a078b84a1f11)

4 years agoMerge pull request #37090 from shyukri/wip-46961-octopus
Lenz Grimmer [Fri, 9 Oct 2020 12:00:09 +0000 (14:00 +0200)]
Merge pull request #37090 from shyukri/wip-46961-octopus

octopus: mgr/dashboard: Improve notification badge

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agoMerge pull request #37300 from alvistack/octopus-aio_graceful_reboot
Yuri Weinstein [Thu, 8 Oct 2020 19:38:37 +0000 (12:38 -0700)]
Merge pull request #37300 from alvistack/octopus-aio_graceful_reboot

octopus: systemd: Support Graceful Reboot for AIO Node

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
4 years agoMerge pull request #37533 from smithfarm/wip-47739-octopus
Yuri Weinstein [Thu, 8 Oct 2020 18:58:00 +0000 (11:58 -0700)]
Merge pull request #37533 from smithfarm/wip-47739-octopus

octopus: mgr/devicehealth: device_health_metrics pool gets created even without any OSDs in the cluster

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37225 from smithfarm/wip-47531-octopus
Yuri Weinstein [Thu, 8 Oct 2020 18:56:20 +0000 (11:56 -0700)]
Merge pull request #37225 from smithfarm/wip-47531-octopus

octopus: ceph.in: ignore failures to flush stdout

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37128 from k0ste/wip-47426-octopus
Yuri Weinstein [Thu, 8 Oct 2020 18:55:54 +0000 (11:55 -0700)]
Merge pull request #37128 from k0ste/wip-47426-octopus

octopus: mgr/zabbix: indent the output of "zabbix config-show"

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37048 from trociny/wip-47363-octopus
Yuri Weinstein [Thu, 8 Oct 2020 18:54:02 +0000 (11:54 -0700)]
Merge pull request #37048 from trociny/wip-47363-octopus

octopus: bluestore: fix collection_list ordering

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #37436 from sebastian-philipp/octopus-backport-36800-36894-36929...
Nathan Cutler [Thu, 8 Oct 2020 17:18:29 +0000 (19:18 +0200)]
Merge pull request #37436 from sebastian-philipp/octopus-backport-36800-36894-36929-36995-36946-36432-36903-36766-37046-37071-36980-3693

octopus: cephadm batch backport September (2)

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
4 years agoceph-volume: don't exit before empty report can be printed 37520/head
Jan Fajerski [Thu, 8 Oct 2020 06:45:26 +0000 (08:45 +0200)]
ceph-volume: don't exit before empty report can be printed

get_plan() called exit in case of an empty plan. This prevented a report
being printed under these circumstances. Avoid exit in this case. Also
adds tests to ensure an empty report is printed.

Fixes: https://tracker.ceph.com/issues/47760
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 0cc5604843b215709a681fa402145c9fa403b1dd)

4 years agoPendingReleaseNotes: add note about batch refactor
Jan Fajerski [Wed, 7 Oct 2020 07:45:42 +0000 (09:45 +0200)]
PendingReleaseNotes: add note about batch refactor

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 485f0d797e4b21ffb7ac742f0949e8c0a23d43f2)

 Conflicts:
PendingReleaseNotes
        Sort ceph-volume entry under 15.2.6 heading

4 years agopybind/mgr/orchestrator: fix preview for new c-v batch output
Jan Fajerski [Wed, 30 Sep 2020 14:16:05 +0000 (16:16 +0200)]
pybind/mgr/orchestrator: fix preview for new c-v batch output

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 996df56a4aebe29e32fd529ae926ae484d240943)

 Conflicts:
src/pybind/mgr/orchestrator/tests/test_orchestrator.py
        drop test_handle_command(), not added by us

4 years agoMerge PR #37345 into octopus
Jan Fajerski [Wed, 7 Oct 2020 13:32:57 +0000 (15:32 +0200)]
Merge PR #37345 into octopus

* refs/pull/37345/head:
ceph-volume: avoid format strings for now

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
4 years agodoc: replace luminous with |stable-release| 37436/head
Kefu Chai [Fri, 28 Aug 2020 10:26:26 +0000 (18:26 +0800)]
doc: replace luminous with |stable-release|

"luminous" is a magic number in these contexts, and we should use a
constant for representing it.

the "sphinx_substitution_extensions" sphinx extension is introduced for
performing the global subsitution.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7ca85e77d040e990fc60d7fe99447d3fb1e7f69e)

4 years agoceph-volume batch: return valid empty json reports
Jan Fajerski [Sat, 3 Oct 2020 07:40:33 +0000 (09:40 +0200)]
ceph-volume batch: return valid empty json reports

Fixes: https://tracker.ceph.com/issues/47729
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ab59269a6ca5bb80c28e94beef0338f23fc10fff)

4 years agopython-common: do not skip unavailable devices
Joshua Schmid [Fri, 2 Oct 2020 09:37:58 +0000 (11:37 +0200)]
python-common: do not skip unavailable devices

Fixes: https://tracker.ceph.com/issues/47726
We should assemble the device set based on the drivegroups in
mgr/cephadm (regardless of their availability) and pass it to c-v.

This patch is the missing piece to actually do this.

related changes: https://github.com/ceph/ceph/pull/34740/commits/7d168ad7bdbb6d6d5231a4ae540ab03040b49a38

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit b9e3da39193419c3fd606a4cba586bea58c6e961)

4 years agoceph-volume: pass filter_for_batch as keyword argument
Jan Fajerski [Mon, 5 Oct 2020 10:56:26 +0000 (12:56 +0200)]
ceph-volume: pass filter_for_batch as keyword argument

This PR also removes an unused ctor argument in the Devices class.

Fixes: 7d168ad7bdbb6d6d5231a4ae540ab03040b49a38
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c5a711e5b6499915b7c2a7b7869f890fa7dc7e2d)

4 years agoceph-volume batch: fix very_fast_allocation plan and add tests
Jan Fajerski [Fri, 25 Sep 2020 09:35:19 +0000 (11:35 +0200)]
ceph-volume batch: fix very_fast_allocation plan and add tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 8178d5c48ac1a7f3915f0003abab6d625385bd78)

4 years agoceph-volume: batch: call the right prepare method
Jan Fajerski [Wed, 16 Sep 2020 13:43:00 +0000 (15:43 +0200)]
ceph-volume: batch: call the right prepare method

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit e75ef77f23ae07463510ec213ac4007f29cbe2da)

4 years agopython-common drive-groups: allow strings for wal and db size
Jan Fajerski [Mon, 14 Sep 2020 09:32:38 +0000 (11:32 +0200)]
python-common drive-groups: allow strings for wal and db size

This enables user to pass sizes like "10G", which batch now understands.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 28e33f3025d03008b46b56989e5d939682b5720d)

4 years agoceph-volume inventory: add option to filter unwanted devices
Jan Fajerski [Fri, 11 Sep 2020 14:35:00 +0000 (16:35 +0200)]
ceph-volume inventory: add option to filter unwanted devices

Some device we never want to pass to the batch subcommand. For now this
includes devices that have a partition or are mounted on the machine.
One goal is to filter the root device, so it is not included on a batch
command and thus would contribute to its implicit sizing calculation.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 7d168ad7bdbb6d6d5231a4ae540ab03040b49a38)

4 years agoceph-volume: address review comments
Jan Fajerski [Fri, 11 Sep 2020 08:36:43 +0000 (10:36 +0200)]
ceph-volume: address review comments

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 6f1592a1146529d352184c795aae8ce12f66e554)

4 years agoceph-volume: batch: fix size retrieval for lvs
Jan Fajerski [Thu, 10 Sep 2020 14:45:34 +0000 (16:45 +0200)]
ceph-volume: batch: fix size retrieval for lvs

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 24e4aa1296608ef861d4ea4b6a1892246a53ef76)

4 years agoceph-volume: include encryption in batch report
Jan Fajerski [Wed, 9 Sep 2020 11:04:14 +0000 (13:04 +0200)]
ceph-volume: include encryption in batch report

Fixes: https://tracker.ceph.com/issues/44783
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit fce184cf9b2d8f15543e1adee49e1fe6cc17437d)

4 years agoceph-volume lvm batch: use namedtuple instead of tuple
Jan Fajerski [Wed, 9 Sep 2020 07:41:15 +0000 (09:41 +0200)]
ceph-volume lvm batch: use namedtuple instead of tuple

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 98c991fc6fd17b18d5bfbebe4b8febe5ff8fa2f0)

4 years agoceph-volume: address review comments, mostly tidying, clarification
Jan Fajerski [Tue, 8 Sep 2020 14:53:53 +0000 (16:53 +0200)]
ceph-volume: address review comments, mostly tidying, clarification

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit d0735ce1c90c952a6d2e1b805c1326d13ff7b06c)

4 years agoidempotency must result in the same outcome
Jan Fajerski [Tue, 8 Sep 2020 12:11:15 +0000 (14:11 +0200)]
idempotency must result in the same outcome

...not should

Co-authored-by: Joshua Schmid <jschmid@suse.de>
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit fcacd0b96ab195e939f6f879b0a0362a06385f9a)