]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agoosd/PrimaryLogPG: rebuild attrs from clients 22960/head
Sage Weil [Wed, 4 Jul 2018 19:19:04 +0000 (14:19 -0500)]
osd/PrimaryLogPG: rebuild attrs from clients

Ensure that buffers coming in via client ops are rebuilt before populating
the PGTransaction.  This ensures that we don't pin the raw buffers for
the entire incoming message in memory.

In the past we've addressed this issue at the ObjectStore layer, but we
did not consider the attr_cache in ObjectContext.  Rebuilding the buffers
at this point will sanitize any incoming attribute before it reaches
either PGBackend or ObjectContext object_cache or the ObjectStore
implementation.

Fixes: http://tracker.ceph.com/issues/24768
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0a0f8d5e6fa8276f06a9b7a7b18ea6f15fa8cb86)

7 years agoMerge pull request #22700 from yuriw/wip-yuriw-22532-backport
Yuri Weinstein [Mon, 9 Jul 2018 20:31:20 +0000 (13:31 -0700)]
Merge pull request #22700 from yuriw/wip-yuriw-22532-backport

mimic: qa/suites: Add supported-random-distro$ links.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by:  Warren Usui <wusui@redhat.com>

7 years agoMerge pull request #22791 from pdvian/wip-24534-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:28:24 +0000 (13:28 -0700)]
Merge pull request #22791 from pdvian/wip-24534-mimic

mimic: client: fix bug #24491 _ll_drop_pins may access invalid iterator

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22819 from pdvian/wip-24539-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:27:59 +0000 (13:27 -0700)]
Merge pull request #22819 from pdvian/wip-24539-mimic

mimic: client:  update inode fields according to issued caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22841 from pdvian/wip-24541-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:27:26 +0000 (13:27 -0700)]
Merge pull request #22841 from pdvian/wip-24541-mimic

mimic: mds: add magic to header of open file table

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22882 from smithfarm/wip-24310-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:25:59 +0000 (13:25 -0700)]
Merge pull request #22882 from smithfarm/wip-24310-mimic

mimic: pjd: cd: too many arguments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22900 from pdvian/wip-24585-mimic
Alfredo Deza [Mon, 9 Jul 2018 19:36:42 +0000 (15:36 -0400)]
Merge pull request #22900 from pdvian/wip-24585-mimic

mimic: ceph-volume lvm.common use destroy-new, doesn't need admin keyring

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #22849 from smithfarm/wip-24764-mimic
Lenz Grimmer [Mon, 9 Jul 2018 18:52:50 +0000 (20:52 +0200)]
Merge pull request #22849 from smithfarm/wip-24764-mimic

mimic: mgr/dashboard: Missing breadcrumb on monitor performance counters page

Reviewed-by: Ricardo Marques <rimarques@suse.com>
7 years agoMerge remote-tracking branch 'private/wip-mon-snap-caps-mimic' into mimic
Sage Weil [Mon, 9 Jul 2018 13:29:13 +0000 (08:29 -0500)]
Merge remote-tracking branch 'private/wip-mon-snap-caps-mimic' into mimic

7 years agoMerge remote-tracking branch 'private/wip-cx-mimic' into mimic
Sage Weil [Mon, 9 Jul 2018 13:09:38 +0000 (08:09 -0500)]
Merge remote-tracking branch 'private/wip-cx-mimic' into mimic

7 years agoMerge pull request #22297 from tchaikov/mimic-24322
Yuri Weinstein [Fri, 6 Jul 2018 21:50:12 +0000 (14:50 -0700)]
Merge pull request #22297 from tchaikov/mimic-24322

mimic: mon/OSDMonitor: no_reply on MOSDFailure messages

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22333 from pdvian/wip-24246-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:49:43 +0000 (14:49 -0700)]
Merge pull request #22333 from pdvian/wip-24246-mimic

mimic: mon/MgrMonitor: change 'unresponsive' message to info level

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22361 from tchaikov/mimic-24361
Yuri Weinstein [Fri, 6 Jul 2018 21:48:58 +0000 (14:48 -0700)]
Merge pull request #22361 from tchaikov/mimic-24361

mimic: mon: enable level_compaction_dynamic_level_bytes for rocksdb

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22382 from pdvian/wip-24261-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:48:27 +0000 (14:48 -0700)]
Merge pull request #22382 from pdvian/wip-24261-mimic

mimic: bluestore: flush_commit is racy

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22394 from tchaikov/mimic-24406
Yuri Weinstein [Fri, 6 Jul 2018 21:47:30 +0000 (14:47 -0700)]
Merge pull request #22394 from tchaikov/mimic-24406

mimic: osd: retry to read object attrs at EC recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22406 from pdvian/wip-24292-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:44:44 +0000 (14:44 -0700)]
Merge pull request #22406 from pdvian/wip-24292-mimic

mimic: common: fix typo in rados bench write JSON output

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #22525 from tchaikov/mimic-pr-22336
Yuri Weinstein [Fri, 6 Jul 2018 21:43:42 +0000 (14:43 -0700)]
Merge pull request #22525 from tchaikov/mimic-pr-22336

mimic: os/bluestore: firstly delete db then delete bluefs if open db met error.

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
7 years agoMerge pull request #22552 from smithfarm/wip-24468-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:42:40 +0000 (14:42 -0700)]
Merge pull request #22552 from smithfarm/wip-24468-mimic

mimic: tell ... config rm <foo> not idempotent

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22559 from pdvian/wip-24332-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:42:08 +0000 (14:42 -0700)]
Merge pull request #22559 from pdvian/wip-24332-mimic

mimic: PG: add custom_reaction Backfilled and release reservations after bac…

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoMerge pull request #22651 from liewegas/wip-24423-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:40:33 +0000 (14:40 -0700)]
Merge pull request #22651 from liewegas/wip-24423-mimic

mimic: osd: fix getting osd maps on initial osd startup

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22722 from liewegas/wip-hobject-hash-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:39:55 +0000 (14:39 -0700)]
Merge pull request #22722 from liewegas/wip-hobject-hash-mimic

mimic: common/hobject: improved hash calculation

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
7 years agoMerge pull request #22730 from pdvian/wip-24494-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:38:59 +0000 (14:38 -0700)]
Merge pull request #22730 from pdvian/wip-24494-mimic

mimic: osd/Session: fix invalid iterator dereference in Sessoin::have_backoff()

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoceph-volume lvm.common use destroy-new, doesn't need admin keyring 22900/head
Alfredo Deza [Fri, 8 Jun 2018 19:36:17 +0000 (15:36 -0400)]
ceph-volume lvm.common use destroy-new, doesn't need admin keyring

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 57fb124ce74fe4fb091091be6b4fa63e36d567a2)

7 years agoqa/workunits/suites/pjd.sh: use correct dir name 22882/head
Neha Ojha [Fri, 25 May 2018 00:03:01 +0000 (17:03 -0700)]
qa/workunits/suites/pjd.sh: use correct dir name

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 8fbab63436467aeb72b43f9db8c6b182e48d993c)

7 years agoMerge pull request #22854 from jcsp/wip-mimic-cephfs-doc
John Spray [Thu, 5 Jul 2018 13:41:06 +0000 (14:41 +0100)]
Merge pull request #22854 from jcsp/wip-mimic-cephfs-doc

[mimic] doc/cephfs: remove lingering "experimental" note about multimds

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agomgr/dashboard: Improve max-line-length tslint rule 22849/head
Tiago Melo [Mon, 28 May 2018 13:19:26 +0000 (14:19 +0100)]
mgr/dashboard: Improve max-line-length tslint rule

Prettier is not breaking strings if they are bigger
then the defined max line length.

With this rule we can make sure lines are not exceeding it,
with the exception of import/export lines.

Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit fc67af05b67cb2bbb76fcbea35cf10b1284d25de)

Conflicts:
src/pybind/mgr/dashboard/frontend/tslint.json

7 years agodoc/cephfs: remove lingering "experimental" note about multimds 22854/head
John Spray [Wed, 4 Jul 2018 16:03:05 +0000 (17:03 +0100)]
doc/cephfs: remove lingering "experimental" note about multimds

Signed-off-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #22803 from joscollin/wip-24746-mimic
Nathan Cutler [Wed, 4 Jul 2018 13:06:34 +0000 (15:06 +0200)]
Merge pull request #22803 from joscollin/wip-24746-mimic

mimic: doc: update experimental features - snapshots

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agodoc: update experimental features - snapshots 22803/head
Jos Collin [Thu, 21 Jun 2018 06:03:41 +0000 (11:33 +0530)]
doc: update experimental features - snapshots

* drop the experimental warning of multi-MDS
* update snapshotting blocked off version
* update Multiple filesystems availability in Jewel

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 9ed1158e1de3addf0e768fdb489fc128d09335eb)

7 years agodoc: highlight the flags
Jos Collin [Thu, 21 Jun 2018 06:07:08 +0000 (11:37 +0530)]
doc: highlight the flags

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 1586580d33ff3f1ca4b168d9e3bd1182643e031f)

7 years agomgr/dashboard: Add performance counters breadcrumb
Ricardo Marques [Thu, 17 May 2018 12:15:23 +0000 (13:15 +0100)]
mgr/dashboard: Add performance counters breadcrumb

Fixes: https://tracker.ceph.com/issues/23806
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit e89d5bef7fb768d508db6600174be6388a314b15)

7 years agomgr/dashboard: Remove useless observable unsubscriptions
Ricardo Marques [Thu, 10 May 2018 09:40:09 +0000 (10:40 +0100)]
mgr/dashboard: Remove useless observable unsubscriptions

Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit ae75f15dedf6e9f19744dd4ba568ceb9b2c768e4)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-user-form/rgw-user-form.component.ts
        (retain mimic "Create an observable to add the S3 key when the form is submitted" logic
        but add whitespace so it blends in with the rest of the refactor)

7 years agomgr/dashboard: Add support for queryParams on routerLinkTpl
Ricardo Marques [Thu, 17 May 2018 12:13:45 +0000 (13:13 +0100)]
mgr/dashboard: Add support for queryParams on routerLinkTpl

Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit 9e710e1ad785c86ec32c5c46a6c63349f8615709)

7 years agoMerge pull request #22807 from smithfarm/wip-24699-mimic
Lenz Grimmer [Wed, 4 Jul 2018 08:52:37 +0000 (10:52 +0200)]
Merge pull request #22807 from smithfarm/wip-24699-mimic

mimic: mgr/dashboard: Replace Pool with Pools

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
7 years agomds: add magic to header of open file table 22841/head
Yan, Zheng [Fri, 25 May 2018 02:47:12 +0000 (10:47 +0800)]
mds: add magic to header of open file table

Fixes: http://tracker.ceph.com/issues/24240
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit c60d55d19276c5c65c18bebbfd5939bb8c53f57c)

Conflicts:
src/mds/OpenFileTable.cc: Resolved in _load_finish

7 years agoMerge pull request #22832 from ceph/backport-mimic-22828
Alfredo Deza [Tue, 3 Jul 2018 22:44:25 +0000 (18:44 -0400)]
Merge pull request #22832 from ceph/backport-mimic-22828

mimic: ceph-volume: provide a nice errror message when missing ceph.conf

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume: always ignore a missing ceph conf in main.py 22832/head
Andrew Schoen [Tue, 3 Jul 2018 11:45:24 +0000 (06:45 -0500)]
ceph-volume: always ignore a missing ceph conf in main.py

Now that we have a nice error message when a ceph.conf is missing
and we try to use values from it, maintaining a list of commands that
don't need ceph.conf isn't as helpful. We had actually missed 'simple
trigger' when we first implemented this causing all our luminous tests
for simple to fail when we backported.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 0487a9ac60c5ba289b7dd7c730b014a298c5998f)

7 years agoceph-volume: show a nice error message when ceph.conf is not loaded
Andrew Schoen [Mon, 2 Jul 2018 20:58:44 +0000 (15:58 -0500)]
ceph-volume: show a nice error message when ceph.conf is not loaded

If the ceph configuration file is not loaded correctly and then values
from it are used then an undescript error message is shown, e.g.

AttributeError: 'property' object has no attribute 'get'

With this change that same error condition shows the message:

RuntimeError: No valid ceph configuration file was loaded.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit e42ecb831914b07c7c8559d4556a8aabd2accd0c)

7 years agoMerge pull request #22747 from ceph/backport-mimic-22724
Alfredo Deza [Tue, 3 Jul 2018 15:31:35 +0000 (11:31 -0400)]
Merge pull request #22747 from ceph/backport-mimic-22724

mimic: ceph-volume: error on commands that need ceph.conf to operate

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoclient: update inode fields according to issued caps 22819/head
Yan, Zheng [Sat, 26 May 2018 08:46:12 +0000 (16:46 +0800)]
client:  update inode fields according to issued caps

Cap message and request reply from non-auth MDS may carry stale
information (corresponding locks are in LOCK states) even they
have the newest inode version. So client should update inode fields
according to issued caps.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/24269
(cherry picked from commit 0c51508dd2028abd9031e40206cf1753cbf3c412)

7 years agomgr/dashboard: Replaced "Pool" with "Pools" in navigation bar 22807/head
Lenz Grimmer [Mon, 25 Jun 2018 11:38:55 +0000 (13:38 +0200)]
mgr/dashboard: Replaced "Pool" with "Pools" in navigation bar

Replaced the string "Pool" with "Pools" in the navigation component.

Fixes: https://tracker.ceph.com/issues/24623
Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>
(cherry picked from commit ccd59a467864b44a5eb8a4da671f611e0ada7ff9)

7 years agoMerge pull request #22311 from pdvian/wip-24215-mimic
Yuri Weinstein [Mon, 2 Jul 2018 16:52:38 +0000 (09:52 -0700)]
Merge pull request #22311 from pdvian/wip-24215-mimic

mimic: common: partially revert 95fc248 to make get_process_name work

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #22545 from smithfarm/wip-24500-mimic
Yuri Weinstein [Mon, 2 Jul 2018 16:52:04 +0000 (09:52 -0700)]
Merge pull request #22545 from smithfarm/wip-24500-mimic

mimic: osd: eternal stuck PG in 'unfound_recovery'

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22621 from pdvian/wip-24355-mimic
Yuri Weinstein [Mon, 2 Jul 2018 16:51:30 +0000 (09:51 -0700)]
Merge pull request #22621 from pdvian/wip-24355-mimic

mimic: osd: increase default hard pg limit

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22775 from theanalyst/wip-mimic-24716
Yuri Weinstein [Mon, 2 Jul 2018 16:50:49 +0000 (09:50 -0700)]
Merge pull request #22775 from theanalyst/wip-mimic-24716

mimic: qa/workunits/suites/blogbench.sh: use correct dir name

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoclient: fix bug #24491 _ll_drop_pins may access invalid iterator 22791/head
LiuYangkuan [Mon, 11 Jun 2018 15:51:16 +0000 (11:51 -0400)]
client: fix bug #24491 _ll_drop_pins may access invalid iterator
Fixes: https://tracker.ceph.com/issues/24491
Signed-off-by: Liu Yangkuan <liuyangkuan@gmail.com>
(cherry picked from commit 97188fef8596f098aa43aa745a217b40a6c29ea4)

7 years agoqa/workunits/suites/blogbench.sh: use correct dir name 22775/head
Neha Ojha [Tue, 26 Jun 2018 22:30:38 +0000 (18:30 -0400)]
qa/workunits/suites/blogbench.sh: use correct dir name

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 24513c39f1253e6fda2f77aa71aabc0769008a1b)

7 years agoqa/suites: Add supported-random-distro$ links. 22700/head
Warren Usui [Sat, 19 May 2018 00:27:17 +0000 (00:27 +0000)]
qa/suites: Add supported-random-distro$ links.

Add supported-random-distro$ links for fs, kcephfs, and multimds
Removed qa/suites/fs/upgrade/supported-random-distros$ per Patrick's comment

Fixes: http://tracker.ceph.com/issues/24138
Signed-off-by: Warren Usui <wusui@redhat.com>
(cherry picked from commit affaae045f4ce090ea614b693e47d47bdf9b7c95)

7 years agoMerge pull request #22697 from yuriw/wip-24316-mimic
Josh Durgin [Wed, 27 Jun 2018 23:13:24 +0000 (16:13 -0700)]
Merge pull request #22697 from yuriw/wip-24316-mimic

qa/tests - added skeleton for mimic point to point upgrades testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoqa/tests - added skeleton for mimic point to point upgrades testing 22697/head
Yuri Weinstein [Mon, 25 Jun 2018 21:03:22 +0000 (14:03 -0700)]
qa/tests - added skeleton for mimic point to point upgrades testing

Fixes https://tracker.ceph.com/issues/24316
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoceph-volume: error on commands that need ceph.conf to operate 22747/head
Andrew Schoen [Tue, 26 Jun 2018 19:19:43 +0000 (14:19 -0500)]
ceph-volume: error on commands that need ceph.conf to operate

We had been ignoring the failure to load ceph.conf for all subcommands
but most of them require that a ceph.conf be present. This changes that
so only commands that do not need ceph.conf ignore the failure to find
it.

Fixes: http://tracker.ceph.com/issues/23941
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit de8b301ac20ae07f59a19bc1b3a083e9c010667d)

7 years agoMerge pull request #22713 from tspmelo/mimic-fix-codelyzer
Lenz Grimmer [Wed, 27 Jun 2018 13:04:40 +0000 (15:04 +0200)]
Merge pull request #22713 from tspmelo/mimic-fix-codelyzer

mimic: mgr/dashboard: fix lint error caused by codelyzer update

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
7 years agoosd/Session: fix invalid iterator dereference in Sessoin::have_backoff() 22730/head
Sage Weil [Sun, 10 Jun 2018 18:37:56 +0000 (13:37 -0500)]
osd/Session: fix invalid iterator dereference in Sessoin::have_backoff()

If p is i->second.end(), we do want to back up a position, but we
shouldn't dereference p for p->first.

Fixes: http://tracker.ceph.com/issues/24486
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2d7bfafa879ca1475173a74a84c66fd7f5e1b6bc)

7 years agocommon/performance: Now hobject_t's hash no longer depends directly on oid. 22722/head
Adam Kupczyk [Mon, 4 Jun 2018 10:50:49 +0000 (12:50 +0200)]
common/performance: Now hobject_t's hash no longer depends directly on oid.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit c14504c157d3d6a8eecddaf6c83280ee254c069d)

7 years agocommon/performance: improved hash calculation for hobject_t and ghobject_t
Adam Kupczyk [Tue, 29 May 2018 10:43:33 +0000 (12:43 +0200)]
common/performance: improved hash calculation for hobject_t and ghobject_t

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 0cdeb682af87a0703f8650e013e4685af0633f3f)

7 years agoqa/workunits/rados: test pool op permissions
Jason Dillaman [Thu, 21 Jun 2018 02:20:14 +0000 (22:20 -0400)]
qa/workunits/rados: test pool op permissions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 97e3f0aa013dad49de1a55f8025327de7e801822)

7 years agoqa/workunits/rbd: test self-managed snapshot create/remove permissions
Jason Dillaman [Tue, 5 Jun 2018 19:40:44 +0000 (15:40 -0400)]
qa/workunits/rbd: test self-managed snapshot create/remove permissions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b12dd0bf419ae834abb31c712830fa9c4b5cda9c)

7 years agopybind/rados: new methods for manipulating self-managed snapshots
Jason Dillaman [Tue, 5 Jun 2018 18:48:17 +0000 (14:48 -0400)]
pybind/rados: new methods for manipulating self-managed snapshots

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 21538304d78df9f3b9f54a5b3c9b6b62fc3e6b48)

7 years agomon/OSDMonitor: enforce caps for all remaining pool ops
Jason Dillaman [Thu, 21 Jun 2018 01:30:47 +0000 (21:30 -0400)]
mon/OSDMonitor: enforce caps for all remaining pool ops

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 61c06200fe59996bba2bb65fc402207bc10fd459)

7 years agomon/OSDMonitor: enforce caps when creating/deleting unmanaged snapshots
Jason Dillaman [Tue, 5 Jun 2018 17:24:48 +0000 (13:24 -0400)]
mon/OSDMonitor: enforce caps when creating/deleting unmanaged snapshots

The entity will require write access to the OSD service or permission
for the synthetic "osd pool op unmanaged-snap" command.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4972e054b32c8200600f27564d50d443e683153e)

Conflicts:
src/mon/OSDMonitor.cc: use 'bufferlist::begin' instead of 'bufferlist::cbegin'

7 years agomgr/dashboard: fix lint error caused by codelyzer update 22713/head
Tiago Melo [Mon, 25 Jun 2018 16:23:00 +0000 (17:23 +0100)]
mgr/dashboard: fix lint error caused by codelyzer update

fixes: https://tracker.ceph.com/issues/24663

Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 4e7f4cbd3fa1d89420a3521888ca7583fd7284fb)

7 years agoMerge pull request #22649 from liewegas/wip-24319-24550-mimic
Josh Durgin [Fri, 22 Jun 2018 00:18:45 +0000 (17:18 -0700)]
Merge pull request #22649 from liewegas/wip-24319-24550-mimic

mimic: os/bluestore: backport 24319 and 24550

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22653 from yuriw/wip-yuriw-distros-mimic
Yuri Weinstein [Thu, 21 Jun 2018 19:59:10 +0000 (12:59 -0700)]
Merge pull request #22653 from yuriw/wip-yuriw-distros-mimic

qa/tests: changed distro symlink to point to new way using supported OSes

7 years agoMerge pull request #22640 from ceph/backport-mimic-22627
Alfredo Deza [Thu, 21 Jun 2018 13:57:27 +0000 (09:57 -0400)]
Merge pull request #22640 from ceph/backport-mimic-22627

mimic: lvm: when osd creation fails log the exception

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoqa/tests: changed disto symlink to point to new way using supporeted OS'es 22653/head
Yuri Weinstein [Tue, 12 Jun 2018 20:49:55 +0000 (13:49 -0700)]
qa/tests: changed disto symlink to point to new way using supporeted OS'es

backport of https://github.com/ceph/ceph/pull/22536

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit 37ac8df5559a6084091670e6a1f479ab9332c24b)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoosd: fix getting osd maps on initial osd startup 22651/head
Paul Emmerich [Fri, 15 Jun 2018 22:45:26 +0000 (00:45 +0200)]
osd: fix getting osd maps on initial osd startup

Commit 89d0c357964ff28b83ac541c4d946f712a7bee47 introduced a check for
deleted pools that relies on getting an older osd map that might not be
available in some situations on OSDs starting up for the very first
time.

fixes https://tracker.ceph.com/issues/24423

Signed-off-by: Paul Emmerich <paul.emmerich@croit.io>
(cherry picked from commit 02180f644094d520d5763afa0e76403591a90b9b)

7 years agoos/bluestore: use vector instead of set for zombies 22649/head
Sage Weil [Tue, 19 Jun 2018 12:15:53 +0000 (07:15 -0500)]
os/bluestore: use vector instead of set for zombies

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ef34d2dc8f3dd40c8426b7373d13dae84f90ff73)

7 years agoos/bluestore: reuse zombie OpSequencers by collection id
Sage Weil [Mon, 18 Jun 2018 12:32:08 +0000 (07:32 -0500)]
os/bluestore: reuse zombie OpSequencers by collection id

We can get a sequence that deletes and then recreates a collection where
the transaction removing the collection is delayed (due to pending IO on
its sequencer) but colleciton create is not (new sequencer).

Avoid any such reordering by recycling the old collection's sequencer if
the zombie_osr has not been reaped yet.

Fixes: http://tracker.ceph.com/issues/24550
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 199a110c9e2a42be64bd7fd873b6b172ce1347a8)

7 years agoqa/suites/rados/objecstore/backends/objectstore: capture coredumps
Sage Weil [Mon, 18 Jun 2018 20:35:38 +0000 (15:35 -0500)]
qa/suites/rados/objecstore/backends/objectstore: capture coredumps

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 55f390d0eb0c96cc563190371a35c41f103eb662)

7 years agoos/bluestore: more debug output
Sage Weil [Fri, 15 Jun 2018 19:13:14 +0000 (14:13 -0500)]
os/bluestore: more debug output

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fd3036d2e186eb0299785c43aefe6f2ce88f244d)

7 years agoos/bluestore: print cnode from _open_collections
Sage Weil [Thu, 14 Jun 2018 19:15:20 +0000 (14:15 -0500)]
os/bluestore: print cnode from _open_collections

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b308b6b1554b3203e02e64e3963ef4901274f23f)

7 years agoos/bluestore: print cnode on fsck
Sage Weil [Fri, 15 Jun 2018 15:41:25 +0000 (10:41 -0500)]
os/bluestore: print cnode on fsck

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f3c89ef6e7ce313a2eda2136be5181735e5684d8)

7 years agoqa/suites/rados/objecstore: preserve data dir for ceph_test_objecstore
Sage Weil [Wed, 13 Jun 2018 11:55:18 +0000 (06:55 -0500)]
qa/suites/rados/objecstore: preserve data dir for ceph_test_objecstore

If we fail, preserve the data directory.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 23dd6279bb518709b0719132d850c70b7fc554a8)

7 years agoos/bluestore: fix SharedBlobSet deregister race
Sage Weil [Sun, 10 Jun 2018 20:57:31 +0000 (15:57 -0500)]
os/bluestore: fix SharedBlobSet deregister race

In commit 8c8944b2c45ca9dc5b8fd4db1590e1d24206c0b3 we fixed one narrow race
and introduced a new (probably less narrow) one:

A: put shared blob foo, nref = 0, start removing self from set
B: open_shared_blob -> lookup gets nullptr (nref==null), creates a new shared blob
  B: takes lock, sets sb_map[sbid] = newblob
A: gets lock, erases sb_map[sbid]
B: open_shared_blob -> lookup gets null, creates another new shared blob

Fix by only removing the sb_map entry for the nref=0 sb if it still points
to us.  If it doesn't, that means some new blob has shown up in its place.

Fixes: http://tracker.ceph.com/issues/24319
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 345ea53bd30426d4e6159d0866497d5a8bf6f327)

7 years agolvm: when osd creation fails log the exception 22640/head
Andrew Schoen [Tue, 19 Jun 2018 19:34:57 +0000 (14:34 -0500)]
lvm: when osd creation fails log the exception

Fixes: http://tracker.ceph.com/issues/24456
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit d622dadef09a53d24c6a7b9119051594f8da1cae)

7 years agoMerge pull request #22341 from b-ranto/wip-branto-mgr-mimic
Boris Ranto [Tue, 19 Jun 2018 08:06:42 +0000 (10:06 +0200)]
Merge pull request #22341 from b-ranto/wip-branto-mgr-mimic

[mimic] Sync up ceph-mgr prometheus related changes

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
7 years agoMerge pull request #22224 from yuriw/wip-yuri-powercycle-mimic
Sage Weil [Mon, 18 Jun 2018 19:42:12 +0000 (14:42 -0500)]
Merge pull request #22224 from yuriw/wip-yuri-powercycle-mimic

mimic: qa/tests:  added supported distro for powercycle suite

7 years agoosd: increase default hard pg limit 22621/head
Josh Durgin [Wed, 23 May 2018 16:17:04 +0000 (09:17 -0700)]
osd: increase default hard pg limit

The hard limit was too easy to hit during failure cases, especially
due to mismatched device sizes causing high variance in pgs/osd.

Fixes: https://tracker.ceph.com/issues/24243
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit dfdedce89521f89046fb13287d9fdd683c91e242)

7 years agoMerge pull request #22468 from votdev/fix_rgw_create_key
Lenz Grimmer [Fri, 15 Jun 2018 07:46:48 +0000 (09:46 +0200)]
Merge pull request #22468 from votdev/fix_rgw_create_key

mimic: mgr/dashboard: Fix bug when creating S3 keys

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
7 years agomgr/dashboard: Fix bug when creating S3 keys 22468/head
Volker Theile [Fri, 8 Jun 2018 11:44:45 +0000 (13:44 +0200)]
mgr/dashboard: Fix bug when creating S3 keys

Currently a new subuser is created if a key is created for the main user. That's because the uid is set as subuser parameter, too.
To fix that the key's user name, e.g. test2 or test1:subuser2, must be splitted into its user and subuser parts.

Signed-off-by: Volker Theile <vtheile@suse.com>
7 years agoPG: add custom_reaction Backfilled and release reservations after backfill 22559/head
Neha Ojha [Mon, 21 May 2018 19:34:31 +0000 (12:34 -0700)]
PG: add custom_reaction Backfilled and release reservations after backfill

After backfill completes, we directly go to the Recovered state without
releasing reservations. The outstanding reservations cause double reservation
issues.

Creating a custom_reaction Backfilled, allows us to release reservations,
before transiting to the Recovered state.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 1abc2320283e9247bec7b0821a6134f31b9b5e29)

7 years agoMerge pull request #22515 from ceph/backport-mimic-21803
Alfredo Deza [Wed, 13 Jun 2018 17:47:48 +0000 (13:47 -0400)]
Merge pull request #22515 from ceph/backport-mimic-21803

mimic: ceph-volume initial take on auto sub-command

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoosd,mds,common: make 'config rm ...' idempotent 22552/head
Sage Weil [Mon, 4 Jun 2018 16:00:08 +0000 (11:00 -0500)]
osd,mds,common: make 'config rm ...' idempotent

Fixes: http://tracker.ceph.com/issues/24408
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 11044f5f1b6221eeb8a7837a9b90cd2a4c39936e)

7 years agocommon/config: fix rm_val error codes
Sage Weil [Mon, 4 Jun 2018 15:59:51 +0000 (10:59 -0500)]
common/config: fix rm_val error codes

EINVAL if the config option is invalid.
ENOENT if the config option wasn't set.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4f957a4ad7fb7029435f38a1da086d59f34f3428)

7 years agoMerge pull request #22477 from alfredodeza/mimic-bluestore-no-meta-paths
Andrew Schoen [Wed, 13 Jun 2018 16:06:13 +0000 (16:06 +0000)]
Merge pull request #22477 from alfredodeza/mimic-bluestore-no-meta-paths

mimic: os/bluestore: don't store/use path_block.{db,wal} from meta

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoceph-volume tests verify get_devices captures device metadata 22515/head
Alfredo Deza [Thu, 3 May 2018 18:33:08 +0000 (14:33 -0400)]
ceph-volume tests verify get_devices captures device metadata

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8deb088f101894763d27a6c31a3e8a2e96fc8320)

7 years agoceph-volume tests.conftest allow custom dirs in tmpfile fixture
Alfredo Deza [Thu, 3 May 2018 18:32:00 +0000 (14:32 -0400)]
ceph-volume tests.conftest allow custom dirs in tmpfile fixture

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9d820e82ac6bc0f46890df1cdaa104e88cab2a7e)

7 years agoceph-volume tests ensure human_readable_size works up to terabytes
Alfredo Deza [Thu, 3 May 2018 16:52:21 +0000 (12:52 -0400)]
ceph-volume tests ensure human_readable_size works up to terabytes

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4ad42f4244062995de003d8141cddcdc1d2b80fc)

7 years agoceph-volume tests verify mapper and /dev devices
Alfredo Deza [Thu, 3 May 2018 16:47:53 +0000 (12:47 -0400)]
ceph-volume tests verify mapper and /dev devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9cfd37344c69ebf1f3d8440f295d79ae05b545e9)

7 years agoceph-volume tests add tests for get_block_devs
Alfredo Deza [Thu, 3 May 2018 16:45:05 +0000 (12:45 -0400)]
ceph-volume tests add tests for get_block_devs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 672c4c4d1c31b4631b1238ed0efb98fef34f2a3d)

7 years agoceph-volume tests verify _map_dev_paths mappings
Alfredo Deza [Thu, 3 May 2018 16:40:17 +0000 (12:40 -0400)]
ceph-volume tests verify _map_dev_paths mappings

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2c4d46955e8d7f52b090cd8b5b5561a2dba443e5)

7 years agoceph-volume devices.auto initial take on auto sub-command
Alfredo Deza [Thu, 3 May 2018 15:05:42 +0000 (11:05 -0400)]
ceph-volume devices.auto initial take on auto sub-command

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e3b04c9677c273f91e0a9de35dcd68c578d124db)

7 years agoceph-volume util.disk create a device metadata utility
Alfredo Deza [Thu, 3 May 2018 14:51:47 +0000 (10:51 -0400)]
ceph-volume util.disk create a device metadata utility

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c6559538db28d6b8bc8529e6b2a62e8619c69bd6)

7 years agoceph-volume util.disk add dev mapping helpers for path resolution
Alfredo Deza [Thu, 3 May 2018 11:38:29 +0000 (07:38 -0400)]
ceph-volume util.disk add dev mapping helpers for path resolution

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c06e63fed8a11e80444e8e7ec2e82373d75aa501)

7 years agoceph-volume tests verify get_file_contents utility
Alfredo Deza [Wed, 2 May 2018 23:48:23 +0000 (19:48 -0400)]
ceph-volume tests verify get_file_contents utility

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0177f71caba3fd64de285d6808b5f859c55ed2fe)

7 years agoceph-volume util.system capture file contents
Alfredo Deza [Wed, 2 May 2018 23:15:10 +0000 (19:15 -0400)]
ceph-volume util.system capture file contents

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 39792573dabac3b85a41760c965b60746ca1f059)

7 years agoceph-volume devices.auto initial take on auto sub-command
Alfredo Deza [Fri, 4 May 2018 17:16:05 +0000 (13:16 -0400)]
ceph-volume devices.auto initial take on auto sub-command

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5649bd724b5d300c7080052433f9b9d1c6774cff)

7 years agoceph-volume main include commented out auto subcommand
Alfredo Deza [Wed, 2 May 2018 23:01:10 +0000 (19:01 -0400)]
ceph-volume main include commented out auto subcommand

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ad1b8cae055ee6c822370d91027613680ccd3dd3)

7 years agoceph-volume tests ensure is_lv detection on LVs
Alfredo Deza [Wed, 2 May 2018 22:59:51 +0000 (18:59 -0400)]
ceph-volume tests ensure is_lv detection on LVs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 47060a1ef4562b29771bdfb5a375a225f18c228c)

7 years agoceph-volume api.lvm create helpers to detect an lv from a device path
Alfredo Deza [Wed, 2 May 2018 20:39:16 +0000 (16:39 -0400)]
ceph-volume api.lvm create helpers to detect an lv from a device path

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 68ca7a691e6e333209b67eb5cea8594f37fb93c1)