]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agomgr/dashboard: displaying Service detail inside table 35269/head
Kiefer Chang [Thu, 16 Apr 2020 05:27:21 +0000 (13:27 +0800)]
mgr/dashboard: displaying Service detail inside table

Adapt the changes in https://github.com/ceph/ceph/pull/32747.

Fixes: https://tracker.ceph.com/issues/45106
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 6a627481025aa70664635e95b4eca8c5bdfff294)

5 years agoMerge pull request #35150 from Vicente-Cheng/wip-45599-octopus
Yuri Weinstein [Tue, 26 May 2020 22:16:32 +0000 (15:16 -0700)]
Merge pull request #35150 from Vicente-Cheng/wip-45599-octopus

octopus: client: expose Client::ll_register_callback via libcephfs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #35148 from Vicente-Cheng/wip-45603-octopus
Yuri Weinstein [Tue, 26 May 2020 22:15:28 +0000 (15:15 -0700)]
Merge pull request #35148 from Vicente-Cheng/wip-45603-octopus

octopus: mds: Handle blacklisted error in purge queue

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #35147 from Vicente-Cheng/wip-45601-octopus
Yuri Weinstein [Tue, 26 May 2020 22:14:33 +0000 (15:14 -0700)]
Merge pull request #35147 from Vicente-Cheng/wip-45601-octopus

octopus: mds: don't shallow copy when decoding xattr map

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #34999 from smithfarm/wip-45495-octopus
Yuri Weinstein [Tue, 26 May 2020 22:13:30 +0000 (15:13 -0700)]
Merge pull request #34999 from smithfarm/wip-45495-octopus

octopus: cephfs: Client: fix Finisher assert failure

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #34997 from smithfarm/wip-45477-octopus
Yuri Weinstein [Tue, 26 May 2020 22:10:06 +0000 (15:10 -0700)]
Merge pull request #34997 from smithfarm/wip-45477-octopus

octopus: client: only set MClientCaps::FLAG_SYNC when flushing dirty auth caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
5 years agoMerge pull request #34996 from smithfarm/wip-45473-octopus
Yuri Weinstein [Tue, 26 May 2020 22:08:36 +0000 (15:08 -0700)]
Merge pull request #34996 from smithfarm/wip-45473-octopus

octopus: bash_completion: Do not auto complete obsolete and hidden cmds

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #34727 from kotreshhr/octopus-ceph-fs-status-output
Yuri Weinstein [Tue, 26 May 2020 22:07:57 +0000 (15:07 -0700)]
Merge pull request #34727 from kotreshhr/octopus-ceph-fs-status-output

octopus: mgr/status: Fix "ceph fs status" json format writing to stderr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #34509 from smithfarm/wip-45028-octopus
Yuri Weinstein [Tue, 26 May 2020 22:07:25 +0000 (15:07 -0700)]
Merge pull request #34509 from smithfarm/wip-45028-octopus

octopus: mds: don't assert empty io context list when shutting down

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #35120 from Vicente-Cheng/wip-45500-octopus
Yuri Weinstein [Wed, 20 May 2020 18:32:26 +0000 (11:32 -0700)]
Merge pull request #35120 from Vicente-Cheng/wip-45500-octopus

octopus: rgw: anonomous swift to obj that dont exist should 401

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #34993 from smithfarm/wip-45492-octopus
Yuri Weinstein [Wed, 20 May 2020 18:31:57 +0000 (11:31 -0700)]
Merge pull request #34993 from smithfarm/wip-45492-octopus

octopus: rgw: fix bug where bucket listing end marker not always set correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #34989 from smithfarm/wip-45485-octopus
Yuri Weinstein [Wed, 20 May 2020 18:31:33 +0000 (11:31 -0700)]
Merge pull request #34989 from smithfarm/wip-45485-octopus

octopus: rgw: radosgw-admin: fix infinite loops in 'datalog list'

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #34988 from smithfarm/wip-45484-octopus
Yuri Weinstein [Wed, 20 May 2020 18:31:08 +0000 (11:31 -0700)]
Merge pull request #34988 from smithfarm/wip-45484-octopus

octopus: rgw: fix rgw tries to fetch anonymous user

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #35126 from smithfarm/wip-45598-octopus
Yuri Weinstein [Wed, 20 May 2020 17:08:08 +0000 (10:08 -0700)]
Merge pull request #35126 from smithfarm/wip-45598-octopus

octopus: librbd: avoid completing mirror:DisableRequest while holding its lock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
5 years agoMerge pull request #34994 from smithfarm/wip-45364-octopus
Yuri Weinstein [Wed, 20 May 2020 17:07:21 +0000 (10:07 -0700)]
Merge pull request #34994 from smithfarm/wip-45364-octopus

octopus: qa/workunits/rbd: fix list-mapped filter in unmap_device

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #34985 from smithfarm/wip-45498-octopus
Yuri Weinstein [Wed, 20 May 2020 17:06:21 +0000 (10:06 -0700)]
Merge pull request #34985 from smithfarm/wip-45498-octopus

octopus: rgw: fix some list buckets handle leak

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
5 years agoMerge pull request #34940 from Vicente-Cheng/wip-45366-octopus
Yuri Weinstein [Wed, 20 May 2020 17:05:11 +0000 (10:05 -0700)]
Merge pull request #34940 from Vicente-Cheng/wip-45366-octopus

octopus: rgw/notifications: fix zero size in notifications

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
5 years agotest: add a new program for testing ino_release_cb 35150/head
Jeff Layton [Tue, 28 Apr 2020 18:00:13 +0000 (14:00 -0400)]
test: add a new program for testing ino_release_cb

Create a bunch of files and get their inode numbers. Remount, look them
all up by inode number and hold references. Stop looking up inodes as
soon as we get a callback from libcephfs. If we got the callback, return
success. Fail otherwise.

Since this has the same cluster setup as the other client_trim_caps
testcase, we can piggyback onto that task.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit c0db8a01f0a04cf9e10f3715bfb802d619bc32b9)

5 years agoclient: add a new inode release request callback
Jeff Layton [Fri, 17 Apr 2020 13:55:41 +0000 (09:55 -0400)]
client: add a new inode release request callback

trim_caps() walks the list of caps on the session, and releases
non-auth caps, and attempts to trim dentries until the cache
size is under the max_caps value requested by MDS.

This is fine for FUSE, but doesn't really match the use-case of
nfs-ganesha. Ganesha typically looks up inodes by inode number, not
by dentry. It's quite possible that after a restart, we may have a
ton of outstanding inodes with no dentries associated with them.

Ganesha holds a reference to each inode, so libcephfs can't release
them, and we don't have a way to request that ganesha do so.

Add a new ino_release_callback and finisher. The intent is to allow
libcephfs to "upcall" to the application and request that it release
references to a specific inode.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit e3b9df78fa42add2dfea1f8bff1e00c832a35697)

5 years agoclient: only override umask_cb with non-NULL values
Jeff Layton [Tue, 21 Apr 2020 12:50:54 +0000 (08:50 -0400)]
client: only override umask_cb with non-NULL values

Client::init sets this, but if we later call ll_register_callbacks again
with a new set of function pointers that has umask_cb set to nullptr,
it'll override the value in the cmount.

Only reset umask_cb if the one in args is not nullptr.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 4cb17bb12feaa2b1a78a08612f8e80a191c87e5e)

5 years agoMerge pull request #34992 from tspmelo/wip-45470-octopus
Laura Paduano [Wed, 20 May 2020 08:42:23 +0000 (10:42 +0200)]
Merge pull request #34992 from tspmelo/wip-45470-octopus

octopus: mgr/dashboard: Reduce requests in Mirroring page

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoclient: expose ceph_ll_register_callbacks via libcephfs
Jeff Layton [Thu, 16 Apr 2020 16:19:59 +0000 (12:19 -0400)]
client: expose ceph_ll_register_callbacks via libcephfs

Allow applications based on libcephfs to register callbacks, like we
do for ceph-fuse.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit adcf12d6c329a3bd534e089bc0c8b42dfd3e4515)

5 years agoclient: move callback typedefs and arg struct into ceph_ll_client.h
Jeff Layton [Thu, 16 Apr 2020 16:09:54 +0000 (12:09 -0400)]
client: move callback typedefs and arg struct into ceph_ll_client.h

Put them in a common interface header file. This also allows us to
eliminate the duplicate definition of ceph_deleg_cb_t in Delegation.h.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 8370f70cacfb0bccc96d121cab687376155e8b8d)

5 years agoclient: rename ceph_statx.h to ceph_ll_client.h
Jeff Layton [Thu, 16 Apr 2020 16:01:05 +0000 (12:01 -0400)]
client: rename ceph_statx.h to ceph_ll_client.h

...so we can use it to include other definitions and types that need to
be shared with in-tree code that doesn't want to include libcephfs.h.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 4436f27a0476e96d10ed3367d738e1679d8b0a38)

5 years agoclient: make client_dentry_callback_t more friendly for C
Jeff Layton [Thu, 16 Apr 2020 14:05:20 +0000 (10:05 -0400)]
client: make client_dentry_callback_t more friendly for C

C doesn't have the string type, and doesn't understand references.
Change client_dentry_callback_t to take separate pointer and length
arguments.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 79b588211017d8af15a0e49628563dbe45b55ccb)

5 years agoFiler: Add error handler in _do_purge_range() 35148/head
Varsha Rao [Thu, 16 Apr 2020 14:58:54 +0000 (20:28 +0530)]
Filer: Add error handler in _do_purge_range()

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 39b25efb26963180a7e0403174868f5a9940ff55)

5 years agomds: Handle blacklisted error in purge queue
Varsha Rao [Fri, 28 Feb 2020 06:43:31 +0000 (12:13 +0530)]
mds: Handle blacklisted error in purge queue

This patch adds check to catch blacklisted error from filer. On catching this
error, mds will respawn.

Fixes: https://tracker.ceph.com/issues/43598
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit c144d66afa74e8e45a2fc96bb650219d001538bf)

5 years agomds: don't shallow copy when decoding xattr map 35147/head
Yan, Zheng [Wed, 15 Apr 2020 10:49:36 +0000 (18:49 +0800)]
mds: don't shallow copy when decoding xattr map

Otherwise inodes' xattr maps may reference a large shared buffer
(from omap fetch or journal read). If mds trims and loads inode
repeatly, each inode can reference different large buffer in the
worst case.

Fixes: https://tracker.ceph.com/issues/45090
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit deff94c8f43bb3734b688ccb828d942b8f150638)

5 years agoMerge pull request #34850 from s0nea/wip-45334-octopus
Volker Theile [Wed, 20 May 2020 08:10:03 +0000 (10:10 +0200)]
Merge pull request #34850 from s0nea/wip-45334-octopus

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34947 from s0nea/wip-45429-octopus
Nathan Cutler [Tue, 19 May 2020 16:36:20 +0000 (18:36 +0200)]
Merge pull request #34947 from s0nea/wip-45429-octopus

octopus: doc/mgr/dashboard: Add troubleshooting guide

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34987 from smithfarm/wip-45479-octopus
Nathan Cutler [Tue, 19 May 2020 16:33:33 +0000 (18:33 +0200)]
Merge pull request #34987 from smithfarm/wip-45479-octopus

octopus: doc/rgw: document 'rgw gc max concurrent io'

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
5 years agolibrbd: avoid completing mirror:DisableRequest while holding its lock 35126/head
Jason Dillaman [Thu, 14 May 2020 18:01:35 +0000 (14:01 -0400)]
librbd: avoid completing mirror:DisableRequest while holding its lock

Ensure that the lock is released before another thread has the chance
to complete the state machine and attempt to destruct the in-use lock.

Fixes: https://tracker.ceph.com/issues/45544
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c45324a14f6bb02896549d5f79ad42f942afb5d1)

5 years agoMerge pull request #34893 from sebastian-philipp/octopus-backport-34718-34695-34676...
Sebastian Wagner [Tue, 19 May 2020 10:15:01 +0000 (12:15 +0200)]
Merge pull request #34893 from sebastian-philipp/octopus-backport-34718-34695-34676-34698-34747-34590-34733-34707-34690-34824-34723-34728-34779

octopus: cephadm: Batch backport May (1)

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agorgw: anonomous swift to obj that dont exist should 401 35120/head
Matthew Oliver [Wed, 26 Feb 2020 06:15:22 +0000 (06:15 +0000)]
rgw: anonomous swift to obj that dont exist should 401

Currently, if you attempt to GET and object in the Swift API that
doesn't exist and you don't pass a `X-Auth-Token` it will 404 instead of
401.

This is actually a rather big problem as it means someone can leak data
out of the cluster, not object data itself, but if an object exists or
not.

This is caused by the SwiftAnonymousEngine's, frankly wide open
is_applicable acceptance. When we get to checking the bucket or object
for user acceptance we  deal with it properly, but if the object doesn't
exsit, because the user has been "authorised" rgw returns a 404.

Why? Because we always override the user with the Swift account.
Meaning as far as checks are concerned the auth user is the user, not
and anonymous user.

I assume this is because a swift container could have world readable
reads or writes and in slight s3 and swift api divergents can make these
interesting edge cases leak in.

This patch doesn't change the user to the swift account if they are
anonymous. So we can do some anonymous checks when it suits later in the
request processing path.

Fixes: https://tracker.ceph.com/issues/43617
Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit b03d9754e113d24221f1ce0bac17556ab0017a8a)

5 years ago15.2.2 v15.2.2
Jenkins Build Slave User [Mon, 18 May 2020 16:25:12 +0000 (16:25 +0000)]
15.2.2

5 years agoMerge branch 'wip-joshd-fix-octopus' into octopus
Josh Durgin [Mon, 18 May 2020 14:35:19 +0000 (07:35 -0700)]
Merge branch 'wip-joshd-fix-octopus' into octopus

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #35067 from cbodley/wip-qa-rgw-too-many-shards-octopus
Yuri Weinstein [Thu, 14 May 2020 18:33:10 +0000 (11:33 -0700)]
Merge pull request #35067 from cbodley/wip-qa-rgw-too-many-shards-octopus

octopus: qa/rgw: remove 1999-shard tests from rgw/verify

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoqa/rgw: remove 1999-shard tests from rgw/verify 35067/head
Casey Bodley [Fri, 27 Mar 2020 21:43:07 +0000 (17:43 -0400)]
qa/rgw: remove 1999-shard tests from rgw/verify

causing test failures because the host runs out of disk space

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 01516061ece6ebe4d265a994e5f1f8d255334fcd)

5 years agoPendingReleaseNotes: note about security fix
Josh Durgin [Thu, 14 May 2020 04:34:56 +0000 (21:34 -0700)]
PendingReleaseNotes: note about security fix

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 06f239fc35f35865d2cf92dda1ac8f4d5fe82bde)

5 years agomgr: require all caps for pre-octopus tell commands
Josh Durgin [Mon, 4 May 2020 21:03:35 +0000 (17:03 -0400)]
mgr: require all caps for pre-octopus tell commands

This matches the requirements for admin socket commands
sent via tell elsewhere.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 347003e13167c428187a5450517850f4d85e09ad)

5 years agomon: enforce caps for pre-octopus client tell commands
Josh Durgin [Thu, 23 Apr 2020 04:22:10 +0000 (00:22 -0400)]
mon: enforce caps for pre-octopus client tell commands

This affects only the commands whitelisted here - in particular
injectargs requires write access to the monitors.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit fc5e56b75a97c4652c87e9959aad1c4dec45010d)

5 years agoqa/rgw: fix issue error in tests_ps.py 34940/head
J. Eric Ivancich [Mon, 4 May 2020 15:35:25 +0000 (11:35 -0400)]
qa/rgw: fix issue error in tests_ps.py

The Python interpreter on teuthology complains about the lack of
parens in a print statement.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 92180651095924baaaf33286998e32c687744a2b)

5 years agorgw/notifications: fix zero size in notifications
Yuval Lifshitz [Mon, 20 Apr 2020 14:37:17 +0000 (17:37 +0300)]
rgw/notifications: fix zero size in notifications

delete notifications and multipart upload complete notifications
send the correct object size

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Fixes: https://tracker.ceph.com/issues/45150
(cherry picked from commit f04f01ad429ff158f2da8346f884335a600c0e41)

Conflicts:
src/test/rgw/rgw_multi/tests_ps.py
  - NamedTemporaryFile is not introduced to octopus, use TemporaryFile

5 years agoClient: make sure the Finisher's mutex lock not held during it being distructed 34999/head
Xiubo Li [Tue, 31 Mar 2020 09:09:45 +0000 (05:09 -0400)]
Client: make sure the Finisher's mutex lock not held during it being distructed

The objecter_finisher is already started in Client::Client(), but
in the failure path when initializing and starting the Client object,
we may not get a chance to call the Client::shutdown() to stop the
Finisher thread, which maybe still holding the mutex lock in it. Then
when destrucing the Finisher object the pthread_mutex_destroy() will
fail.

This fix will delay the objecter_finisher thread to start in ::init()
until we're ready to call Client::shutdown on any errors instead.

Fixes: https://tracker.ceph.com/issues/44389
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit fbff4ee153f17da51c3b6675eb0616f2b2062f5e)

5 years agoclient: only set MClientCaps::FLAG_SYNC when flushing dirty auth caps 34997/head
Jeff Layton [Mon, 6 Apr 2020 16:26:30 +0000 (12:26 -0400)]
client: only set MClientCaps::FLAG_SYNC when flushing dirty auth caps

Sidharth noticed that we were probably doing more FLAG_SYNC cap flushes
than necessary.

This flag is intended for sync/fsync type activity, and in those cases
all we care about is cleaning dirty caps. We have no need to expedite
other types of cap calls (releases, revokes, etc.).

Fixes: https://tracker.ceph.com/issues/44963
Reported-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 63e4ee0e9fe3654dcd680e7e8271870fbd295215)

5 years agobash_completion: Do not auto complete obsolete and hidden cmds 34996/head
Kotresh HR [Sat, 18 Apr 2020 18:33:37 +0000 (00:03 +0530)]
bash_completion: Do not auto complete obsolete and hidden cmds

This patch fixes two things.

1. Do not auto complete obsolete and hidden cmds.
2. sub command completions often failed due to the
   use of associative arrays which does not keep the
   order. Hence used non-associative arrays.

Fixes: https://tracker.ceph.com/issues/45141
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 41082930f33fa43391986ac05394dbfff3b5dfd7)

5 years agoqa/workunits/rbd: fix list-mapped filter in unmap_device 34994/head
Mykola Golub [Tue, 28 Apr 2020 10:45:33 +0000 (11:45 +0100)]
qa/workunits/rbd: fix list-mapped filter in unmap_device

In recent versions `rbd list-mapped` does not print the white space
at the end of the line.

Fixes: https://tracker.ceph.com/issues/45305
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d667e385a12ea4125dbf74da97a9ba4911a3f7bc)

5 years agorgw: fix bug where bucket listing end marker not always set correctly 34993/head
J. Eric Ivancich [Tue, 10 Mar 2020 17:47:43 +0000 (13:47 -0400)]
rgw: fix bug where bucket listing end marker not always set correctly

Due to using std::move a second time, often
RGWRados::cls_bucket_list_ordered would not set its end marker
correctly. This affected the behavior of
RGWRados::Bucket::List::list_objects_ordered, and might have caused it
return fewer elements than it may have otherwise.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 76043e00851be0b131448a7a2ab4fed2c22426fe)

5 years agomgr/dashboard: Reduce requests in Mirroring page 34992/head
Tiago Melo [Tue, 5 May 2020 11:48:10 +0000 (11:48 +0000)]
mgr/dashboard: Reduce requests in Mirroring page

Fixes: https://tracker.ceph.com/issues/45385
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit f1a020eaf5644d9c7b43c40445a74f89e0e8b536)

5 years agoradosgw-admin: fix infinite loop in 'datalog list' 34989/head
Casey Bodley [Tue, 24 Mar 2020 19:01:31 +0000 (15:01 -0400)]
radosgw-admin: fix infinite loop in 'datalog list'

affects datalog shards with more than 1000 entries

Fixes: https://tracker.ceph.com/issues/44751
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 166f38fe70e6afd2831bdb87a27c8fe757899b95)

5 years agorgw: fix rgw tries to fetch anonymous user 34988/head
Or Friedmann [Tue, 31 Mar 2020 14:30:52 +0000 (17:30 +0300)]
rgw: fix rgw tries to fetch anonymous user

rgw tries to fetch the anonymous user although it does not exist.
Fixes: https://tracker.ceph.com/issues/44772
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit a221274ffce356d776010706deb1005ef1d68ee5)

5 years agodoc/rgw: document 'rgw gc max concurrent io' 34987/head
Casey Bodley [Thu, 7 May 2020 14:41:22 +0000 (10:41 -0400)]
doc/rgw: document 'rgw gc max concurrent io'

Fixes: https://tracker.ceph.com/issues/44958
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b0563a093ada6d8d618a76f2df4617ee4190b999)

5 years agorgw: fix some list buckets handle leak 34985/head
Tianshan Qu [Wed, 21 Aug 2019 04:33:37 +0000 (12:33 +0800)]
rgw: fix some list buckets handle leak

Fixes: https://tracker.ceph.com/issues/44283
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit 8955ca2c0c804f6c43bb33c97bbe263bac1cc36d)

5 years agoMerge pull request #34828 from rhcs-dashboard/wip-45328-octopus
Yuri Weinstein [Fri, 8 May 2020 20:12:26 +0000 (13:12 -0700)]
Merge pull request #34828 from rhcs-dashboard/wip-45328-octopus

octopus: monitoring: fix decimal precision in Grafana %percentages

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #34924 from smithfarm/wip-45401-octopus
Yuri Weinstein [Fri, 8 May 2020 19:29:11 +0000 (12:29 -0700)]
Merge pull request #34924 from smithfarm/wip-45401-octopus

octopus: mon/OSDMonitor: allow trimming maps even if osds are down

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34917 from sseshasa/wip-45392-octopus
Yuri Weinstein [Fri, 8 May 2020 19:27:58 +0000 (12:27 -0700)]
Merge pull request #34917 from sseshasa/wip-45392-octopus

octopus: mon/OSDMonitor: Always tune priority cache manager memory on all mons

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #34449 from rhcs-dashboard/wip-44979-octopus
Yuri Weinstein [Fri, 8 May 2020 19:20:42 +0000 (12:20 -0700)]
Merge pull request #34449 from rhcs-dashboard/wip-44979-octopus

octopus: monitoring: Fix pool capacity incorrect

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
5 years agoMerge pull request #34912 from batrick/i45389
Yuri Weinstein [Thu, 7 May 2020 23:32:19 +0000 (16:32 -0700)]
Merge pull request #34912 from batrick/i45389

octopus: qa: install task runs twice with double unwind causing fatal errors

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34766 from smithfarm/wip-45214-octopus
Yuri Weinstein [Thu, 7 May 2020 23:30:20 +0000 (16:30 -0700)]
Merge pull request #34766 from smithfarm/wip-45214-octopus

octopus: client: reset requested_max_size if file write is not wanted

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
5 years agoMerge pull request #34273 from ukernel/octopus-44800
Yuri Weinstein [Thu, 7 May 2020 23:29:11 +0000 (16:29 -0700)]
Merge pull request #34273 from ukernel/octopus-44800

octopus: mds: fix 'if there is lock cache on dir' check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #34528 from smithfarm/wip-45039-octopus
Yuri Weinstein [Thu, 7 May 2020 22:37:36 +0000 (15:37 -0700)]
Merge pull request #34528 from smithfarm/wip-45039-octopus

octopus: mon: calculate min_size on osd pool set size

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
5 years agoMerge pull request #34416 from smithfarm/wip-44836-octopus
Yuri Weinstein [Thu, 7 May 2020 22:33:22 +0000 (15:33 -0700)]
Merge pull request #34416 from smithfarm/wip-44836-octopus

octopus: mgr/DaemonServer: fetch metadata for new daemons (e.g., mons)

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34769 from smithfarm/wip-45211-octopus
Yuri Weinstein [Thu, 7 May 2020 18:27:12 +0000 (11:27 -0700)]
Merge pull request #34769 from smithfarm/wip-45211-octopus

octopus: ceph-fuse: link to libfuse3 and pass `-o big_writes` to libfuse if libfuse < 3.0.0

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
5 years agoMerge pull request #34775 from ajarr/octopus-fix-fs-commands
Yuri Weinstein [Thu, 7 May 2020 15:27:16 +0000 (08:27 -0700)]
Merge pull request #34775 from ajarr/octopus-fix-fs-commands

octopus: mon/FSCommands: Fix 'add_data_pool' command and 'fs new' command

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoMerge pull request #34803 from smithfarm/wip-45220-octopus
Ramana Raja [Thu, 7 May 2020 15:16:53 +0000 (20:46 +0530)]
Merge pull request #34803 from smithfarm/wip-45220-octopus

octopus: tools/cephfs: add accounted_rstat/rstat when building file dentry

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34802 from smithfarm/wip-45219-octopus
Ramana Raja [Thu, 7 May 2020 15:16:00 +0000 (20:46 +0530)]
Merge pull request #34802 from smithfarm/wip-45219-octopus

octopus: qa/cephfs: update TestClusterFull:test_barrier

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
5 years agoMerge pull request #34801 from smithfarm/wip-45216-octopus
Ramana Raja [Thu, 7 May 2020 14:52:30 +0000 (20:22 +0530)]
Merge pull request #34801 from smithfarm/wip-45216-octopus

octopus: qa/tasks/cephfs: umount the mountpoints when tearDown

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34816 from vumrao/wip-vumrao-45317
Yuri Weinstein [Thu, 7 May 2020 14:41:52 +0000 (07:41 -0700)]
Merge pull request #34816 from vumrao/wip-vumrao-45317

octopus: radosgw-admin: add support for --bucket-id in bucket stats command

5 years agoMerge pull request #34800 from smithfarm/wip-45049-octopus
Ramana Raja [Thu, 7 May 2020 14:41:18 +0000 (20:11 +0530)]
Merge pull request #34800 from smithfarm/wip-45049-octopus

octopus: mgr: force purge normal ceph entities from service map

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34799 from smithfarm/wip-44844-octopus
Ramana Raja [Thu, 7 May 2020 14:35:37 +0000 (20:05 +0530)]
Merge pull request #34799 from smithfarm/wip-44844-octopus

octopus: qa/test_config_session_timeout: fix options conflicting

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34798 from smithfarm/wip-44843-octopus
Ramana Raja [Thu, 7 May 2020 14:24:30 +0000 (19:54 +0530)]
Merge pull request #34798 from smithfarm/wip-44843-octopus

octopus: test/libcephfs: retry and wait for caps when setting the RD deleg

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agodoc/mgr/dashboard: Add troubleshooting guide 34947/head
Tatjana Dehler [Tue, 21 Apr 2020 14:07:21 +0000 (16:07 +0200)]
doc/mgr/dashboard: Add troubleshooting guide

Fixes: https://tracker.ceph.com/issues/25196
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit a241a00452f20e2c511ee9d10c6689a8d63c9ef5)

5 years agoMerge pull request #34804 from smithfarm/wip-45222-octopus
Ramana Raja [Thu, 7 May 2020 11:23:57 +0000 (16:53 +0530)]
Merge pull request #34804 from smithfarm/wip-45222-octopus

octopus: cephfs-journal-tool: correctly parse --dry_run argument

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34805 from smithfarm/wip-45227-octopus
Ramana Raja [Thu, 7 May 2020 11:17:32 +0000 (16:47 +0530)]
Merge pull request #34805 from smithfarm/wip-45227-octopus

octopus: test_cephfs_shell: fix UnicodeDecodeError

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34681 from kotreshhr/octopus-subvolume-info
Ramana Raja [Thu, 7 May 2020 11:11:21 +0000 (16:41 +0530)]
Merge pull request #34681 from kotreshhr/octopus-subvolume-info

octopus: mgr/volumes: Add interface to get subvolume metadata

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #34477 from jan--f/wip-44999-octopus
Jan Fajerski [Thu, 7 May 2020 10:20:45 +0000 (12:20 +0200)]
Merge pull request #34477 from jan--f/wip-44999-octopus

octopus: ceph-volume/batch: return success when all devices are filtered

5 years agoMerge pull request #34480 from jan--f/wip-45003-octopus
Jan Fajerski [Thu, 7 May 2020 10:20:18 +0000 (12:20 +0200)]
Merge pull request #34480 from jan--f/wip-45003-octopus

octopus: ceph-volume/batch: check lvs list before access

5 years agoMerge pull request #34886 from jan--f/wip-34341-no-tracker-octopus
Jan Fajerski [Thu, 7 May 2020 10:19:57 +0000 (12:19 +0200)]
Merge pull request #34886 from jan--f/wip-34341-no-tracker-octopus

octopus: ceph-volume: update functional testing deploy.yml playbook

5 years agoMerge pull request #34446 from jan--f/wip-44975-octopus
Jan Fajerski [Wed, 6 May 2020 16:27:24 +0000 (18:27 +0200)]
Merge pull request #34446 from jan--f/wip-44975-octopus

octopus: devices/simple/scan: Fix string in log statement

5 years agoceph-volume: update functional testing deploy.yml playbook 34886/head
Guillaume Abrioux [Wed, 1 Apr 2020 09:32:40 +0000 (11:32 +0200)]
ceph-volume: update functional testing deploy.yml playbook

This commit adds a call to `ceph-facts` role in the first play of this
playbook. This is needed so `ceph-validate` won't fail because of
following error:

```
fatal: [osd0]: FAILED! => {}

MSG:

'osd_pool_default_size' is undefined
```

`osd_pool_default_size` is set in ceph-facts.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5bf7cc87ec91e128bd106e101b43328cacef1c7b)

5 years agomon/OSDMonitor: ensure lec only accounts for up osds 34924/head
Joao Eduardo Luis [Tue, 21 Nov 2017 10:18:24 +0000 (10:18 +0000)]
mon/OSDMonitor: ensure lec only accounts for up osds

If we also consider down osds, we may very well be in a healthy state
but keeping maps as far back as the last epoch when a given osd went
down. If said osd stays down for eons, we will be keeping bajillions of
maps that we shouldn't.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit e62269c8929e414284ad0773c4a3c82e43735e4e)

5 years agoqa/standalone: exercise osdmon's last epoch clean
Joao Eduardo Luis [Mon, 20 Nov 2017 15:57:10 +0000 (15:57 +0000)]
qa/standalone: exercise osdmon's last epoch clean

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit 3d682c21f6246276fa30d179766a49f0eb78d73a)

5 years agomon/OSDMonitor: dump last epoch clean info on report
Joao Eduardo Luis [Tue, 21 Nov 2017 10:15:10 +0000 (10:15 +0000)]
mon/OSDMonitor: dump last epoch clean info on report

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit bd2e5c62757c17d00f30620c043ed06abb8a0bca)

5 years agomon/OSDMonitor: Always tune priority cache manager memory on all mons 34917/head
Sridhar Seshasayee [Wed, 29 Apr 2020 14:02:35 +0000 (19:32 +0530)]
mon/OSDMonitor: Always tune priority cache manager memory on all mons

Always call into priority cache manager (pcm) to tune the memory on the
leader and on all the followers (peons) as part of each tick(). This
ensures that the pcm on all the mons continuously tunes the tcmalloc
memory thereby ensuring that the peons don't run out of memory eventually.

Fixes: https://tracker.ceph.com/issues/45266
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 5083e21192c3cfdec274c8bafdc4d73e7c843af9)

5 years agoqa: move fsx deps into common cephfs install yaml 34912/head
Patrick Donnelly [Tue, 5 May 2020 16:25:07 +0000 (09:25 -0700)]
qa: move fsx deps into common cephfs install yaml

The double install task causes errors during unwind.

Fixes: fe3961fc36e4d0af896fe140ba2450e1556ab563
Fixes: https://tracker.ceph.com/issues/45387
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d791daa732e122c4b81ec5693eeb3971a87199cb)

5 years agoMerge pull request #34797 from smithfarm/wip-45207-octopus
Josh Durgin [Mon, 4 May 2020 20:55:31 +0000 (13:55 -0700)]
Merge pull request #34797 from smithfarm/wip-45207-octopus

octopus: ceph-object-corpus: update to octopus

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #34790 from smithfarm/wip-drop-is_opensuse-octopus
Yuri Weinstein [Mon, 4 May 2020 20:34:38 +0000 (13:34 -0700)]
Merge pull request #34790 from smithfarm/wip-drop-is_opensuse-octopus

octopus: rpm: drop "is_opensuse" conditional in SUSE-specific bcond block

Reviewed-by: Tim Serong <tserong@suse.com>
5 years agoMerge pull request #34610 from ifed01/wip-ifed-better-bluefs-expand-octopus
Yuri Weinstein [Mon, 4 May 2020 20:32:11 +0000 (13:32 -0700)]
Merge pull request #34610 from ifed01/wip-ifed-better-bluefs-expand-octopus

octopus: os/bluestore: open DB in read-only when expanding DB/WAL

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
5 years agoMerge pull request #34830 from smithfarm/wip-45314-octopus
Yuri Weinstein [Mon, 4 May 2020 20:13:27 +0000 (13:13 -0700)]
Merge pull request #34830 from smithfarm/wip-45314-octopus

octopus: qa/standalone/scrub/osd-scrub-repair.sh: fix race in TEST_auto_repair_bluestore_failed

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #34809 from smithfarm/wip-45041-octopus
Yuri Weinstein [Mon, 4 May 2020 20:12:56 +0000 (13:12 -0700)]
Merge pull request #34809 from smithfarm/wip-45041-octopus

octopus: osd/PrimaryLogPG: fix SPARSE_READ stat

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #34807 from smithfarm/wip-44842-octopus
Yuri Weinstein [Mon, 4 May 2020 20:11:51 +0000 (13:11 -0700)]
Merge pull request #34807 from smithfarm/wip-44842-octopus

octopus: osd/PeeringState: do not trim pg log past last_update_ondisk

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agoMerge pull request #34806 from smithfarm/wip-44685-octopus
Yuri Weinstein [Mon, 4 May 2020 20:10:51 +0000 (13:10 -0700)]
Merge pull request #34806 from smithfarm/wip-44685-octopus

octopus: qa/standalone/osd/osd-backfill-stats.sh: get_latest_osdmap to propagate map change

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #34353 from neha-ojha/wip-44757-octopus
Yuri Weinstein [Mon, 4 May 2020 20:09:57 +0000 (13:09 -0700)]
Merge pull request #34353 from neha-ojha/wip-44757-octopus

octopus: common/options: Disable bluefs_buffered_io by default again.

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
5 years agoMerge pull request #34795 from smithfarm/wip-45044-octopus
Yuri Weinstein [Mon, 4 May 2020 19:34:37 +0000 (12:34 -0700)]
Merge pull request #34795 from smithfarm/wip-45044-octopus

octopus: os/bluestore: Don't pollute old journal when add new device

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
5 years agoqa/workunits/cephadm/test_cephadm.sh: wait longer for monitoring stack 34893/head
Michael Fritch [Mon, 27 Apr 2020 21:11:49 +0000 (15:11 -0600)]
qa/workunits/cephadm/test_cephadm.sh: wait longer for monitoring stack

Give the monitoring stack (node-exporter, prom, grafana) a few more
retries to become available before giving up

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit c60ec03e47a596f9732f79e7c008f707f822cf4c)

5 years agocephadm: Infer config on "cephadm shell"
Ricardo Marques [Fri, 24 Apr 2020 09:59:21 +0000 (10:59 +0100)]
cephadm: Infer config on "cephadm shell"

Fixes: https://tracker.ceph.com/issues/44792
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit eb732dcbc5afd0218849afbf70c09a7f9c06d731)

5 years agocephadm: remove fqdn_enabled from ceph-iscsi
Matthew Oliver [Fri, 24 Apr 2020 06:17:50 +0000 (06:17 +0000)]
cephadm: remove fqdn_enabled from ceph-iscsi

The fqdn_enabled option never actually landed in the ceph-iscsi master
branch. So this patch removes it from the cephadm ceph-iscsi spec.

Fixes: https://tracker.ceph.com/issues/45196
Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit fb295911ac318c0c063a8f03e7d2ccfc3ea0b241)

5 years agopython-common: Fix wrong type annotation
Sebastian Wagner [Wed, 29 Apr 2020 11:22:22 +0000 (13:22 +0200)]
python-common: Fix wrong type annotation

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit ea085d6839256a8086ebe0587fe5fbf1d2f0a1b9)

5 years agomgr/cephadm: raise, if crash keyring fails
Sebastian Wagner [Wed, 22 Apr 2020 13:59:24 +0000 (15:59 +0200)]
mgr/cephadm: raise, if crash keyring fails

Fixes: https://tracker.ceph.com/issues/44826
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit c681fbb41df975325ef2b7dc6a2a12c96a4d7022)

5 years agodoc/orchestrator: update docs to new syntax and updated examples
Joshua Schmid [Thu, 23 Apr 2020 12:04:45 +0000 (14:04 +0200)]
doc/orchestrator: update docs to new syntax and updated examples

Signed-off-by: Joshua Schmid <jschmid@suse.de>
Co-Authored-By: Sebastian Wagner <sebastian@spawnhost.de>
(cherry picked from commit cc5bf59413a66a8af8c783f47d1fdb229565a638)