]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agorgw/notifications: version id was not sent in versioned buckets 35254/head
Yuval Lifshitz [Fri, 8 May 2020 07:09:32 +0000 (10:09 +0300)]
rgw/notifications: version id was not sent in versioned buckets

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Fixes: https://tracker.ceph.com/issues/45440
(cherry picked from commit 973b0b1ed8767767f0c4446d97248c1b8002f1d3)

5 years agoMerge pull request #35120 from Vicente-Cheng/wip-45500-octopus
Yuri Weinstein [Wed, 20 May 2020 18:32:26 +0000 (11:32 -0700)]
Merge pull request #35120 from Vicente-Cheng/wip-45500-octopus

octopus: rgw: anonomous swift to obj that dont exist should 401

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #34993 from smithfarm/wip-45492-octopus
Yuri Weinstein [Wed, 20 May 2020 18:31:57 +0000 (11:31 -0700)]
Merge pull request #34993 from smithfarm/wip-45492-octopus

octopus: rgw: fix bug where bucket listing end marker not always set correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #34989 from smithfarm/wip-45485-octopus
Yuri Weinstein [Wed, 20 May 2020 18:31:33 +0000 (11:31 -0700)]
Merge pull request #34989 from smithfarm/wip-45485-octopus

octopus: rgw: radosgw-admin: fix infinite loops in 'datalog list'

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #34988 from smithfarm/wip-45484-octopus
Yuri Weinstein [Wed, 20 May 2020 18:31:08 +0000 (11:31 -0700)]
Merge pull request #34988 from smithfarm/wip-45484-octopus

octopus: rgw: fix rgw tries to fetch anonymous user

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #35126 from smithfarm/wip-45598-octopus
Yuri Weinstein [Wed, 20 May 2020 17:08:08 +0000 (10:08 -0700)]
Merge pull request #35126 from smithfarm/wip-45598-octopus

octopus: librbd: avoid completing mirror:DisableRequest while holding its lock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
5 years agoMerge pull request #34994 from smithfarm/wip-45364-octopus
Yuri Weinstein [Wed, 20 May 2020 17:07:21 +0000 (10:07 -0700)]
Merge pull request #34994 from smithfarm/wip-45364-octopus

octopus: qa/workunits/rbd: fix list-mapped filter in unmap_device

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #34985 from smithfarm/wip-45498-octopus
Yuri Weinstein [Wed, 20 May 2020 17:06:21 +0000 (10:06 -0700)]
Merge pull request #34985 from smithfarm/wip-45498-octopus

octopus: rgw: fix some list buckets handle leak

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
5 years agoMerge pull request #34940 from Vicente-Cheng/wip-45366-octopus
Yuri Weinstein [Wed, 20 May 2020 17:05:11 +0000 (10:05 -0700)]
Merge pull request #34940 from Vicente-Cheng/wip-45366-octopus

octopus: rgw/notifications: fix zero size in notifications

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
5 years agoMerge pull request #34992 from tspmelo/wip-45470-octopus
Laura Paduano [Wed, 20 May 2020 08:42:23 +0000 (10:42 +0200)]
Merge pull request #34992 from tspmelo/wip-45470-octopus

octopus: mgr/dashboard: Reduce requests in Mirroring page

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34850 from s0nea/wip-45334-octopus
Volker Theile [Wed, 20 May 2020 08:10:03 +0000 (10:10 +0200)]
Merge pull request #34850 from s0nea/wip-45334-octopus

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34947 from s0nea/wip-45429-octopus
Nathan Cutler [Tue, 19 May 2020 16:36:20 +0000 (18:36 +0200)]
Merge pull request #34947 from s0nea/wip-45429-octopus

octopus: doc/mgr/dashboard: Add troubleshooting guide

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34987 from smithfarm/wip-45479-octopus
Nathan Cutler [Tue, 19 May 2020 16:33:33 +0000 (18:33 +0200)]
Merge pull request #34987 from smithfarm/wip-45479-octopus

octopus: doc/rgw: document 'rgw gc max concurrent io'

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
5 years agolibrbd: avoid completing mirror:DisableRequest while holding its lock 35126/head
Jason Dillaman [Thu, 14 May 2020 18:01:35 +0000 (14:01 -0400)]
librbd: avoid completing mirror:DisableRequest while holding its lock

Ensure that the lock is released before another thread has the chance
to complete the state machine and attempt to destruct the in-use lock.

Fixes: https://tracker.ceph.com/issues/45544
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c45324a14f6bb02896549d5f79ad42f942afb5d1)

5 years agoMerge pull request #34893 from sebastian-philipp/octopus-backport-34718-34695-34676...
Sebastian Wagner [Tue, 19 May 2020 10:15:01 +0000 (12:15 +0200)]
Merge pull request #34893 from sebastian-philipp/octopus-backport-34718-34695-34676-34698-34747-34590-34733-34707-34690-34824-34723-34728-34779

octopus: cephadm: Batch backport May (1)

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agorgw: anonomous swift to obj that dont exist should 401 35120/head
Matthew Oliver [Wed, 26 Feb 2020 06:15:22 +0000 (06:15 +0000)]
rgw: anonomous swift to obj that dont exist should 401

Currently, if you attempt to GET and object in the Swift API that
doesn't exist and you don't pass a `X-Auth-Token` it will 404 instead of
401.

This is actually a rather big problem as it means someone can leak data
out of the cluster, not object data itself, but if an object exists or
not.

This is caused by the SwiftAnonymousEngine's, frankly wide open
is_applicable acceptance. When we get to checking the bucket or object
for user acceptance we  deal with it properly, but if the object doesn't
exsit, because the user has been "authorised" rgw returns a 404.

Why? Because we always override the user with the Swift account.
Meaning as far as checks are concerned the auth user is the user, not
and anonymous user.

I assume this is because a swift container could have world readable
reads or writes and in slight s3 and swift api divergents can make these
interesting edge cases leak in.

This patch doesn't change the user to the swift account if they are
anonymous. So we can do some anonymous checks when it suits later in the
request processing path.

Fixes: https://tracker.ceph.com/issues/43617
Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit b03d9754e113d24221f1ce0bac17556ab0017a8a)

5 years ago15.2.2 v15.2.2
Jenkins Build Slave User [Mon, 18 May 2020 16:25:12 +0000 (16:25 +0000)]
15.2.2

5 years agoMerge branch 'wip-joshd-fix-octopus' into octopus
Josh Durgin [Mon, 18 May 2020 14:35:19 +0000 (07:35 -0700)]
Merge branch 'wip-joshd-fix-octopus' into octopus

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #35067 from cbodley/wip-qa-rgw-too-many-shards-octopus
Yuri Weinstein [Thu, 14 May 2020 18:33:10 +0000 (11:33 -0700)]
Merge pull request #35067 from cbodley/wip-qa-rgw-too-many-shards-octopus

octopus: qa/rgw: remove 1999-shard tests from rgw/verify

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoqa/rgw: remove 1999-shard tests from rgw/verify 35067/head
Casey Bodley [Fri, 27 Mar 2020 21:43:07 +0000 (17:43 -0400)]
qa/rgw: remove 1999-shard tests from rgw/verify

causing test failures because the host runs out of disk space

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 01516061ece6ebe4d265a994e5f1f8d255334fcd)

5 years agoPendingReleaseNotes: note about security fix
Josh Durgin [Thu, 14 May 2020 04:34:56 +0000 (21:34 -0700)]
PendingReleaseNotes: note about security fix

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 06f239fc35f35865d2cf92dda1ac8f4d5fe82bde)

5 years agomgr: require all caps for pre-octopus tell commands
Josh Durgin [Mon, 4 May 2020 21:03:35 +0000 (17:03 -0400)]
mgr: require all caps for pre-octopus tell commands

This matches the requirements for admin socket commands
sent via tell elsewhere.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 347003e13167c428187a5450517850f4d85e09ad)

5 years agomon: enforce caps for pre-octopus client tell commands
Josh Durgin [Thu, 23 Apr 2020 04:22:10 +0000 (00:22 -0400)]
mon: enforce caps for pre-octopus client tell commands

This affects only the commands whitelisted here - in particular
injectargs requires write access to the monitors.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit fc5e56b75a97c4652c87e9959aad1c4dec45010d)

5 years agoqa/rgw: fix issue error in tests_ps.py 34940/head
J. Eric Ivancich [Mon, 4 May 2020 15:35:25 +0000 (11:35 -0400)]
qa/rgw: fix issue error in tests_ps.py

The Python interpreter on teuthology complains about the lack of
parens in a print statement.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 92180651095924baaaf33286998e32c687744a2b)

5 years agorgw/notifications: fix zero size in notifications
Yuval Lifshitz [Mon, 20 Apr 2020 14:37:17 +0000 (17:37 +0300)]
rgw/notifications: fix zero size in notifications

delete notifications and multipart upload complete notifications
send the correct object size

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Fixes: https://tracker.ceph.com/issues/45150
(cherry picked from commit f04f01ad429ff158f2da8346f884335a600c0e41)

Conflicts:
src/test/rgw/rgw_multi/tests_ps.py
  - NamedTemporaryFile is not introduced to octopus, use TemporaryFile

5 years agoqa/workunits/rbd: fix list-mapped filter in unmap_device 34994/head
Mykola Golub [Tue, 28 Apr 2020 10:45:33 +0000 (11:45 +0100)]
qa/workunits/rbd: fix list-mapped filter in unmap_device

In recent versions `rbd list-mapped` does not print the white space
at the end of the line.

Fixes: https://tracker.ceph.com/issues/45305
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d667e385a12ea4125dbf74da97a9ba4911a3f7bc)

5 years agorgw: fix bug where bucket listing end marker not always set correctly 34993/head
J. Eric Ivancich [Tue, 10 Mar 2020 17:47:43 +0000 (13:47 -0400)]
rgw: fix bug where bucket listing end marker not always set correctly

Due to using std::move a second time, often
RGWRados::cls_bucket_list_ordered would not set its end marker
correctly. This affected the behavior of
RGWRados::Bucket::List::list_objects_ordered, and might have caused it
return fewer elements than it may have otherwise.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 76043e00851be0b131448a7a2ab4fed2c22426fe)

5 years agomgr/dashboard: Reduce requests in Mirroring page 34992/head
Tiago Melo [Tue, 5 May 2020 11:48:10 +0000 (11:48 +0000)]
mgr/dashboard: Reduce requests in Mirroring page

Fixes: https://tracker.ceph.com/issues/45385
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit f1a020eaf5644d9c7b43c40445a74f89e0e8b536)

5 years agoradosgw-admin: fix infinite loop in 'datalog list' 34989/head
Casey Bodley [Tue, 24 Mar 2020 19:01:31 +0000 (15:01 -0400)]
radosgw-admin: fix infinite loop in 'datalog list'

affects datalog shards with more than 1000 entries

Fixes: https://tracker.ceph.com/issues/44751
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 166f38fe70e6afd2831bdb87a27c8fe757899b95)

5 years agorgw: fix rgw tries to fetch anonymous user 34988/head
Or Friedmann [Tue, 31 Mar 2020 14:30:52 +0000 (17:30 +0300)]
rgw: fix rgw tries to fetch anonymous user

rgw tries to fetch the anonymous user although it does not exist.
Fixes: https://tracker.ceph.com/issues/44772
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit a221274ffce356d776010706deb1005ef1d68ee5)

5 years agodoc/rgw: document 'rgw gc max concurrent io' 34987/head
Casey Bodley [Thu, 7 May 2020 14:41:22 +0000 (10:41 -0400)]
doc/rgw: document 'rgw gc max concurrent io'

Fixes: https://tracker.ceph.com/issues/44958
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b0563a093ada6d8d618a76f2df4617ee4190b999)

5 years agorgw: fix some list buckets handle leak 34985/head
Tianshan Qu [Wed, 21 Aug 2019 04:33:37 +0000 (12:33 +0800)]
rgw: fix some list buckets handle leak

Fixes: https://tracker.ceph.com/issues/44283
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit 8955ca2c0c804f6c43bb33c97bbe263bac1cc36d)

5 years agoMerge pull request #34828 from rhcs-dashboard/wip-45328-octopus
Yuri Weinstein [Fri, 8 May 2020 20:12:26 +0000 (13:12 -0700)]
Merge pull request #34828 from rhcs-dashboard/wip-45328-octopus

octopus: monitoring: fix decimal precision in Grafana %percentages

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #34924 from smithfarm/wip-45401-octopus
Yuri Weinstein [Fri, 8 May 2020 19:29:11 +0000 (12:29 -0700)]
Merge pull request #34924 from smithfarm/wip-45401-octopus

octopus: mon/OSDMonitor: allow trimming maps even if osds are down

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34917 from sseshasa/wip-45392-octopus
Yuri Weinstein [Fri, 8 May 2020 19:27:58 +0000 (12:27 -0700)]
Merge pull request #34917 from sseshasa/wip-45392-octopus

octopus: mon/OSDMonitor: Always tune priority cache manager memory on all mons

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #34449 from rhcs-dashboard/wip-44979-octopus
Yuri Weinstein [Fri, 8 May 2020 19:20:42 +0000 (12:20 -0700)]
Merge pull request #34449 from rhcs-dashboard/wip-44979-octopus

octopus: monitoring: Fix pool capacity incorrect

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
5 years agoMerge pull request #34912 from batrick/i45389
Yuri Weinstein [Thu, 7 May 2020 23:32:19 +0000 (16:32 -0700)]
Merge pull request #34912 from batrick/i45389

octopus: qa: install task runs twice with double unwind causing fatal errors

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34766 from smithfarm/wip-45214-octopus
Yuri Weinstein [Thu, 7 May 2020 23:30:20 +0000 (16:30 -0700)]
Merge pull request #34766 from smithfarm/wip-45214-octopus

octopus: client: reset requested_max_size if file write is not wanted

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
5 years agoMerge pull request #34273 from ukernel/octopus-44800
Yuri Weinstein [Thu, 7 May 2020 23:29:11 +0000 (16:29 -0700)]
Merge pull request #34273 from ukernel/octopus-44800

octopus: mds: fix 'if there is lock cache on dir' check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #34528 from smithfarm/wip-45039-octopus
Yuri Weinstein [Thu, 7 May 2020 22:37:36 +0000 (15:37 -0700)]
Merge pull request #34528 from smithfarm/wip-45039-octopus

octopus: mon: calculate min_size on osd pool set size

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
5 years agoMerge pull request #34416 from smithfarm/wip-44836-octopus
Yuri Weinstein [Thu, 7 May 2020 22:33:22 +0000 (15:33 -0700)]
Merge pull request #34416 from smithfarm/wip-44836-octopus

octopus: mgr/DaemonServer: fetch metadata for new daemons (e.g., mons)

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34769 from smithfarm/wip-45211-octopus
Yuri Weinstein [Thu, 7 May 2020 18:27:12 +0000 (11:27 -0700)]
Merge pull request #34769 from smithfarm/wip-45211-octopus

octopus: ceph-fuse: link to libfuse3 and pass `-o big_writes` to libfuse if libfuse < 3.0.0

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
5 years agoMerge pull request #34775 from ajarr/octopus-fix-fs-commands
Yuri Weinstein [Thu, 7 May 2020 15:27:16 +0000 (08:27 -0700)]
Merge pull request #34775 from ajarr/octopus-fix-fs-commands

octopus: mon/FSCommands: Fix 'add_data_pool' command and 'fs new' command

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoMerge pull request #34803 from smithfarm/wip-45220-octopus
Ramana Raja [Thu, 7 May 2020 15:16:53 +0000 (20:46 +0530)]
Merge pull request #34803 from smithfarm/wip-45220-octopus

octopus: tools/cephfs: add accounted_rstat/rstat when building file dentry

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34802 from smithfarm/wip-45219-octopus
Ramana Raja [Thu, 7 May 2020 15:16:00 +0000 (20:46 +0530)]
Merge pull request #34802 from smithfarm/wip-45219-octopus

octopus: qa/cephfs: update TestClusterFull:test_barrier

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
5 years agoMerge pull request #34801 from smithfarm/wip-45216-octopus
Ramana Raja [Thu, 7 May 2020 14:52:30 +0000 (20:22 +0530)]
Merge pull request #34801 from smithfarm/wip-45216-octopus

octopus: qa/tasks/cephfs: umount the mountpoints when tearDown

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34816 from vumrao/wip-vumrao-45317
Yuri Weinstein [Thu, 7 May 2020 14:41:52 +0000 (07:41 -0700)]
Merge pull request #34816 from vumrao/wip-vumrao-45317

octopus: radosgw-admin: add support for --bucket-id in bucket stats command

5 years agoMerge pull request #34800 from smithfarm/wip-45049-octopus
Ramana Raja [Thu, 7 May 2020 14:41:18 +0000 (20:11 +0530)]
Merge pull request #34800 from smithfarm/wip-45049-octopus

octopus: mgr: force purge normal ceph entities from service map

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34799 from smithfarm/wip-44844-octopus
Ramana Raja [Thu, 7 May 2020 14:35:37 +0000 (20:05 +0530)]
Merge pull request #34799 from smithfarm/wip-44844-octopus

octopus: qa/test_config_session_timeout: fix options conflicting

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34798 from smithfarm/wip-44843-octopus
Ramana Raja [Thu, 7 May 2020 14:24:30 +0000 (19:54 +0530)]
Merge pull request #34798 from smithfarm/wip-44843-octopus

octopus: test/libcephfs: retry and wait for caps when setting the RD deleg

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agodoc/mgr/dashboard: Add troubleshooting guide 34947/head
Tatjana Dehler [Tue, 21 Apr 2020 14:07:21 +0000 (16:07 +0200)]
doc/mgr/dashboard: Add troubleshooting guide

Fixes: https://tracker.ceph.com/issues/25196
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit a241a00452f20e2c511ee9d10c6689a8d63c9ef5)

5 years agoMerge pull request #34804 from smithfarm/wip-45222-octopus
Ramana Raja [Thu, 7 May 2020 11:23:57 +0000 (16:53 +0530)]
Merge pull request #34804 from smithfarm/wip-45222-octopus

octopus: cephfs-journal-tool: correctly parse --dry_run argument

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34805 from smithfarm/wip-45227-octopus
Ramana Raja [Thu, 7 May 2020 11:17:32 +0000 (16:47 +0530)]
Merge pull request #34805 from smithfarm/wip-45227-octopus

octopus: test_cephfs_shell: fix UnicodeDecodeError

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34681 from kotreshhr/octopus-subvolume-info
Ramana Raja [Thu, 7 May 2020 11:11:21 +0000 (16:41 +0530)]
Merge pull request #34681 from kotreshhr/octopus-subvolume-info

octopus: mgr/volumes: Add interface to get subvolume metadata

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34477 from jan--f/wip-44999-octopus
Jan Fajerski [Thu, 7 May 2020 10:20:45 +0000 (12:20 +0200)]
Merge pull request #34477 from jan--f/wip-44999-octopus

octopus: ceph-volume/batch: return success when all devices are filtered

5 years agoMerge pull request #34480 from jan--f/wip-45003-octopus
Jan Fajerski [Thu, 7 May 2020 10:20:18 +0000 (12:20 +0200)]
Merge pull request #34480 from jan--f/wip-45003-octopus

octopus: ceph-volume/batch: check lvs list before access

5 years agoMerge pull request #34886 from jan--f/wip-34341-no-tracker-octopus
Jan Fajerski [Thu, 7 May 2020 10:19:57 +0000 (12:19 +0200)]
Merge pull request #34886 from jan--f/wip-34341-no-tracker-octopus

octopus: ceph-volume: update functional testing deploy.yml playbook

5 years agoMerge pull request #34446 from jan--f/wip-44975-octopus
Jan Fajerski [Wed, 6 May 2020 16:27:24 +0000 (18:27 +0200)]
Merge pull request #34446 from jan--f/wip-44975-octopus

octopus: devices/simple/scan: Fix string in log statement

5 years agoceph-volume: update functional testing deploy.yml playbook 34886/head
Guillaume Abrioux [Wed, 1 Apr 2020 09:32:40 +0000 (11:32 +0200)]
ceph-volume: update functional testing deploy.yml playbook

This commit adds a call to `ceph-facts` role in the first play of this
playbook. This is needed so `ceph-validate` won't fail because of
following error:

```
fatal: [osd0]: FAILED! => {}

MSG:

'osd_pool_default_size' is undefined
```

`osd_pool_default_size` is set in ceph-facts.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5bf7cc87ec91e128bd106e101b43328cacef1c7b)

5 years agomon/OSDMonitor: ensure lec only accounts for up osds 34924/head
Joao Eduardo Luis [Tue, 21 Nov 2017 10:18:24 +0000 (10:18 +0000)]
mon/OSDMonitor: ensure lec only accounts for up osds

If we also consider down osds, we may very well be in a healthy state
but keeping maps as far back as the last epoch when a given osd went
down. If said osd stays down for eons, we will be keeping bajillions of
maps that we shouldn't.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit e62269c8929e414284ad0773c4a3c82e43735e4e)

5 years agoqa/standalone: exercise osdmon's last epoch clean
Joao Eduardo Luis [Mon, 20 Nov 2017 15:57:10 +0000 (15:57 +0000)]
qa/standalone: exercise osdmon's last epoch clean

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit 3d682c21f6246276fa30d179766a49f0eb78d73a)

5 years agomon/OSDMonitor: dump last epoch clean info on report
Joao Eduardo Luis [Tue, 21 Nov 2017 10:15:10 +0000 (10:15 +0000)]
mon/OSDMonitor: dump last epoch clean info on report

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit bd2e5c62757c17d00f30620c043ed06abb8a0bca)

5 years agomon/OSDMonitor: Always tune priority cache manager memory on all mons 34917/head
Sridhar Seshasayee [Wed, 29 Apr 2020 14:02:35 +0000 (19:32 +0530)]
mon/OSDMonitor: Always tune priority cache manager memory on all mons

Always call into priority cache manager (pcm) to tune the memory on the
leader and on all the followers (peons) as part of each tick(). This
ensures that the pcm on all the mons continuously tunes the tcmalloc
memory thereby ensuring that the peons don't run out of memory eventually.

Fixes: https://tracker.ceph.com/issues/45266
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 5083e21192c3cfdec274c8bafdc4d73e7c843af9)

5 years agoqa: move fsx deps into common cephfs install yaml 34912/head
Patrick Donnelly [Tue, 5 May 2020 16:25:07 +0000 (09:25 -0700)]
qa: move fsx deps into common cephfs install yaml

The double install task causes errors during unwind.

Fixes: fe3961fc36e4d0af896fe140ba2450e1556ab563
Fixes: https://tracker.ceph.com/issues/45387
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d791daa732e122c4b81ec5693eeb3971a87199cb)

5 years agoMerge pull request #34797 from smithfarm/wip-45207-octopus
Josh Durgin [Mon, 4 May 2020 20:55:31 +0000 (13:55 -0700)]
Merge pull request #34797 from smithfarm/wip-45207-octopus

octopus: ceph-object-corpus: update to octopus

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #34790 from smithfarm/wip-drop-is_opensuse-octopus
Yuri Weinstein [Mon, 4 May 2020 20:34:38 +0000 (13:34 -0700)]
Merge pull request #34790 from smithfarm/wip-drop-is_opensuse-octopus

octopus: rpm: drop "is_opensuse" conditional in SUSE-specific bcond block

Reviewed-by: Tim Serong <tserong@suse.com>
5 years agoMerge pull request #34610 from ifed01/wip-ifed-better-bluefs-expand-octopus
Yuri Weinstein [Mon, 4 May 2020 20:32:11 +0000 (13:32 -0700)]
Merge pull request #34610 from ifed01/wip-ifed-better-bluefs-expand-octopus

octopus: os/bluestore: open DB in read-only when expanding DB/WAL

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
5 years agoMerge pull request #34830 from smithfarm/wip-45314-octopus
Yuri Weinstein [Mon, 4 May 2020 20:13:27 +0000 (13:13 -0700)]
Merge pull request #34830 from smithfarm/wip-45314-octopus

octopus: qa/standalone/scrub/osd-scrub-repair.sh: fix race in TEST_auto_repair_bluestore_failed

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #34809 from smithfarm/wip-45041-octopus
Yuri Weinstein [Mon, 4 May 2020 20:12:56 +0000 (13:12 -0700)]
Merge pull request #34809 from smithfarm/wip-45041-octopus

octopus: osd/PrimaryLogPG: fix SPARSE_READ stat

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #34807 from smithfarm/wip-44842-octopus
Yuri Weinstein [Mon, 4 May 2020 20:11:51 +0000 (13:11 -0700)]
Merge pull request #34807 from smithfarm/wip-44842-octopus

octopus: osd/PeeringState: do not trim pg log past last_update_ondisk

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agoMerge pull request #34806 from smithfarm/wip-44685-octopus
Yuri Weinstein [Mon, 4 May 2020 20:10:51 +0000 (13:10 -0700)]
Merge pull request #34806 from smithfarm/wip-44685-octopus

octopus: qa/standalone/osd/osd-backfill-stats.sh: get_latest_osdmap to propagate map change

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #34353 from neha-ojha/wip-44757-octopus
Yuri Weinstein [Mon, 4 May 2020 20:09:57 +0000 (13:09 -0700)]
Merge pull request #34353 from neha-ojha/wip-44757-octopus

octopus: common/options: Disable bluefs_buffered_io by default again.

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
5 years agoMerge pull request #34795 from smithfarm/wip-45044-octopus
Yuri Weinstein [Mon, 4 May 2020 19:34:37 +0000 (12:34 -0700)]
Merge pull request #34795 from smithfarm/wip-45044-octopus

octopus: os/bluestore: Don't pollute old journal when add new device

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
5 years agoqa/workunits/cephadm/test_cephadm.sh: wait longer for monitoring stack 34893/head
Michael Fritch [Mon, 27 Apr 2020 21:11:49 +0000 (15:11 -0600)]
qa/workunits/cephadm/test_cephadm.sh: wait longer for monitoring stack

Give the monitoring stack (node-exporter, prom, grafana) a few more
retries to become available before giving up

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit c60ec03e47a596f9732f79e7c008f707f822cf4c)

5 years agocephadm: Infer config on "cephadm shell"
Ricardo Marques [Fri, 24 Apr 2020 09:59:21 +0000 (10:59 +0100)]
cephadm: Infer config on "cephadm shell"

Fixes: https://tracker.ceph.com/issues/44792
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit eb732dcbc5afd0218849afbf70c09a7f9c06d731)

5 years agocephadm: remove fqdn_enabled from ceph-iscsi
Matthew Oliver [Fri, 24 Apr 2020 06:17:50 +0000 (06:17 +0000)]
cephadm: remove fqdn_enabled from ceph-iscsi

The fqdn_enabled option never actually landed in the ceph-iscsi master
branch. So this patch removes it from the cephadm ceph-iscsi spec.

Fixes: https://tracker.ceph.com/issues/45196
Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit fb295911ac318c0c063a8f03e7d2ccfc3ea0b241)

5 years agopython-common: Fix wrong type annotation
Sebastian Wagner [Wed, 29 Apr 2020 11:22:22 +0000 (13:22 +0200)]
python-common: Fix wrong type annotation

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit ea085d6839256a8086ebe0587fe5fbf1d2f0a1b9)

5 years agomgr/cephadm: raise, if crash keyring fails
Sebastian Wagner [Wed, 22 Apr 2020 13:59:24 +0000 (15:59 +0200)]
mgr/cephadm: raise, if crash keyring fails

Fixes: https://tracker.ceph.com/issues/44826
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit c681fbb41df975325ef2b7dc6a2a12c96a4d7022)

5 years agodoc/orchestrator: update docs to new syntax and updated examples
Joshua Schmid [Thu, 23 Apr 2020 12:04:45 +0000 (14:04 +0200)]
doc/orchestrator: update docs to new syntax and updated examples

Signed-off-by: Joshua Schmid <jschmid@suse.de>
Co-Authored-By: Sebastian Wagner <sebastian@spawnhost.de>
(cherry picked from commit cc5bf59413a66a8af8c783f47d1fdb229565a638)

5 years agoPendingReleaseNotes: Add Cephadm for >15.2.1
Sebastian Wagner [Fri, 24 Apr 2020 11:47:03 +0000 (13:47 +0200)]
PendingReleaseNotes: Add Cephadm for >15.2.1

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a057467ede498c39ad7310419eeac772c1dfc32e)

conflicts:
PendingReleaseNotes

5 years agodoc/cephadm/install.rst: Add troubleshooting for 'CIDR' error
Zac Dover [Wed, 22 Apr 2020 12:59:56 +0000 (22:59 +1000)]
doc/cephadm/install.rst: Add troubleshooting for 'CIDR' error

add troubleshooting for 'failed to infer CIDR network...' error

See also
https://tracker.ceph.com/issues/44828

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 4571ee26a05dc7f4b1e266017df48faafb0c1329)

5 years agodoc/mgr/orchestrator: Remove duplicate ps from support table
Varsha Rao [Fri, 24 Apr 2020 18:02:52 +0000 (23:32 +0530)]
doc/mgr/orchestrator: Remove duplicate ps from support table

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 37bd6920752d3ec6de122f318c244ee973cedb7e)

5 years agocephadm: Check for pool existance for iscsi And NFS
Matthew Oliver [Thu, 23 Apr 2020 01:07:19 +0000 (11:07 +1000)]
cephadm: Check for pool existance for iscsi And NFS

Currently both iscsi and NFS require pools to be specified when they are
deployed. However, we don't actaully check these pools exist. Leading to
broken containers.

This patch uses the rados client that is part of the mgrmodule to check
that the specified pool exists. As we need to check in 2 different
daemons, a helper method:

   def _check_pool_exists(self, pool, service_name):

was added to `cephadm/module.py`.

Fixes: https://tracker.ceph.com/issues/45161
Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit d3de8697e6d389c514d26f5eb62675ac061a56c6)

5 years agomgr/cephadm: add Inventory class
Sebastian Wagner [Fri, 17 Apr 2020 15:12:55 +0000 (17:12 +0200)]
mgr/cephadm: add Inventory class

This is just a minor cleanup.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 814787b70129a51c006869e21333503122b9eac6)

5 years agomgr/cephadm: allow config for an iscsi `container_image`
Michael Fritch [Wed, 22 Apr 2020 19:28:43 +0000 (13:28 -0600)]
mgr/cephadm: allow config for an iscsi `container_image`

ceph config set client.iscsi container_image <container_image>

Fixes: https://tracker.ceph.com/issues/45162
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 4e73255ffa4161e4ceaf54cf9f8c569b2a8ecfc3)

5 years agocephadm: catch Exception during is_mgr_available
Michael Fritch [Thu, 23 Apr 2020 21:09:03 +0000 (15:09 -0600)]
cephadm: catch Exception during is_mgr_available

Traceback (most recent call last):
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 4578, in <module>
    r = args.func()
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 1121, in _default_image
    return func()
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 2489, in command_bootstrap
    is_available('mgr', is_mgr_available)
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 895, in is_available
    if func():
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 2486, in is_mgr_available
    out = cli(['status', '-f', 'json-pretty'], timeout=timeout)
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 2403, in cli
    return CephContainer(
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 2140, in run
    out, _, _ = call_throws(
  File "test_tmp/tmp.ykMmPiFqIT/cephadm", line 838, in call_throws
    raise RuntimeError('Failed command: %s' % ' '.join(command))

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit b454b995cbfc658e116eed23832ebe029eb553ef)

5 years agoMerge pull request #34739 from votdev/wip-45262-octopus
Yuri Weinstein [Mon, 4 May 2020 15:49:32 +0000 (08:49 -0700)]
Merge pull request #34739 from votdev/wip-45262-octopus

octopus: mgr/dashboard: RGW auto refresh is not working

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34742 from sebastian-philipp/octopus-backport-34691-34600-34620...
Abhishek L [Mon, 4 May 2020 15:35:59 +0000 (17:35 +0200)]
Merge pull request #34742 from sebastian-philipp/octopus-backport-34691-34600-34620-34674-34678-34673-34675-34693-34619

octopus: cephadm: Batch backport April (3)

Reviewed-By: Michael Fritch <mfritch@suse.com>
5 years agomgr/dashboard: add telemetry report component 34850/head
Tatjana Dehler [Fri, 21 Feb 2020 09:08:10 +0000 (10:08 +0100)]
mgr/dashboard: add telemetry report component

Add a telemetry component in order to give the user the
possibility to configure the telemetry module in a more
guided fashion. The component offers broader explanations,
shows a preview of the generated report and asks the user
to accept the license before enabling the module.

Fixes: https://tracker.ceph.com/issues/43956
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 18875ef337036696914a9056c666db66882cbe22)

5 years agomgr/dashboard: handle `textarea` and add tooltip
Tatjana Dehler [Fri, 17 Apr 2020 12:49:05 +0000 (14:49 +0200)]
mgr/dashboard: handle `textarea` and add tooltip

Enhance `Copy2ClipboardButtonDirective` in order to
handle `textarea`s and copy a formatted input.
Add a tooltip to the button, too.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit d2f76c91af13789ac95a3de0041d2d47c37e0173)

5 years agomgr/telemetry: refactor/enhance CLI commands
Tatjana Dehler [Tue, 7 Apr 2020 15:18:27 +0000 (17:18 +0200)]
mgr/telemetry: refactor/enhance CLI commands

1. Move the implementations of the `on` and `off`
command into dedicated methods so that they can
be accessed by other mgr modules via `mgr.remote`.

2. Add a `show-all` command to return the report
and device report at once. Also add a method
`get_report` to make the reports accessible for
other mgr modules as mentioned above.

3. Add two  minor changes and a note where the
gathered data can be found to the Telemetry docu
file.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 8d91443c8b932f8d24017e72cbbae58616928ab9)

5 years agorgw/rgw_bucket: clear tenant string for non tenanted buckets 34816/head
Vikhyat Umrao [Fri, 1 May 2020 15:28:55 +0000 (08:28 -0700)]
rgw/rgw_bucket: clear tenant string for non tenanted buckets
in rgw_bucket_parse_bucket_key function.

Fixes: https://tracker.ceph.com/issues/45355
Signed-off-by: Vikhyat Umrao <vikhyat@redhat.com>
(cherry picked from commit 6c4594a6aa6db0aee05287a82eaa937f715dc8b0)

5 years agoradosgw-admin: add support for --bucket-id in bucket stats command
Vikhyat Umrao [Fri, 30 Aug 2019 07:16:46 +0000 (00:16 -0700)]
radosgw-admin: add support for --bucket-id in bucket stats command
Fixes: https://tracker.ceph.com/issues/41061
Signed-off-by: Vikhyat Umrao <vikhyat@redhat.com>
(cherry picked from commit 4cd16e13ca0c8709091737ad2cb2b37a3b19840d)

5 years agoMerge pull request #34793 from smithfarm/wip-45063-octopus
Yuri Weinstein [Thu, 30 Apr 2020 23:13:08 +0000 (16:13 -0700)]
Merge pull request #34793 from smithfarm/wip-45063-octopus

octopus: os/bluestore: fix 'unused' calculation

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
5 years agoMerge pull request #34734 from smithfarm/wip-45259-octopus
Yuri Weinstein [Thu, 30 Apr 2020 20:56:04 +0000 (13:56 -0700)]
Merge pull request #34734 from smithfarm/wip-45259-octopus

octopus: rgw: reshard: skip stale bucket id entries from reshard queue

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #34825 from smithfarm/wip-44895-octopus
Yuri Weinstein [Thu, 30 Apr 2020 19:19:29 +0000 (12:19 -0700)]
Merge pull request #34825 from smithfarm/wip-44895-octopus

octopus: rgw: pubsub sync module ignores ERR_USER_EXIST

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #34826 from smithfarm/wip-45052-octopus
Yuri Weinstein [Thu, 30 Apr 2020 19:18:39 +0000 (12:18 -0700)]
Merge pull request #34826 from smithfarm/wip-45052-octopus

octopus: rgw: Disable prefetch of entire head object when GET request with range header

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #34819 from idryomov/wip-krbd-unmap-spacing-octopus
Yuri Weinstein [Thu, 30 Apr 2020 15:21:57 +0000 (08:21 -0700)]
Merge pull request #34819 from idryomov/wip-krbd-unmap-spacing-octopus

octopus: tests: update unmap.t for table spacing changes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #34810 from smithfarm/wip-45047-octopus
Yuri Weinstein [Thu, 30 Apr 2020 15:20:36 +0000 (08:20 -0700)]
Merge pull request #34810 from smithfarm/wip-45047-octopus

octopus: rbd-mirror: improved replication statistics

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #34425 from smithfarm/wip-44839-octopus
Yuri Weinstein [Thu, 30 Apr 2020 15:19:43 +0000 (08:19 -0700)]
Merge pull request #34425 from smithfarm/wip-44839-octopus

octopus: pybind/rbd: ensure image is open before permitting operations

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>