]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agoMerge pull request #22882 from smithfarm/wip-24310-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:25:59 +0000 (13:25 -0700)]
Merge pull request #22882 from smithfarm/wip-24310-mimic

mimic: pjd: cd: too many arguments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22900 from pdvian/wip-24585-mimic
Alfredo Deza [Mon, 9 Jul 2018 19:36:42 +0000 (15:36 -0400)]
Merge pull request #22900 from pdvian/wip-24585-mimic

mimic: ceph-volume lvm.common use destroy-new, doesn't need admin keyring

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #22849 from smithfarm/wip-24764-mimic
Lenz Grimmer [Mon, 9 Jul 2018 18:52:50 +0000 (20:52 +0200)]
Merge pull request #22849 from smithfarm/wip-24764-mimic

mimic: mgr/dashboard: Missing breadcrumb on monitor performance counters page

Reviewed-by: Ricardo Marques <rimarques@suse.com>
7 years agoMerge remote-tracking branch 'private/wip-mon-snap-caps-mimic' into mimic
Sage Weil [Mon, 9 Jul 2018 13:29:13 +0000 (08:29 -0500)]
Merge remote-tracking branch 'private/wip-mon-snap-caps-mimic' into mimic

7 years agoMerge remote-tracking branch 'private/wip-cx-mimic' into mimic
Sage Weil [Mon, 9 Jul 2018 13:09:38 +0000 (08:09 -0500)]
Merge remote-tracking branch 'private/wip-cx-mimic' into mimic

7 years agoMerge pull request #22297 from tchaikov/mimic-24322
Yuri Weinstein [Fri, 6 Jul 2018 21:50:12 +0000 (14:50 -0700)]
Merge pull request #22297 from tchaikov/mimic-24322

mimic: mon/OSDMonitor: no_reply on MOSDFailure messages

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22333 from pdvian/wip-24246-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:49:43 +0000 (14:49 -0700)]
Merge pull request #22333 from pdvian/wip-24246-mimic

mimic: mon/MgrMonitor: change 'unresponsive' message to info level

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22361 from tchaikov/mimic-24361
Yuri Weinstein [Fri, 6 Jul 2018 21:48:58 +0000 (14:48 -0700)]
Merge pull request #22361 from tchaikov/mimic-24361

mimic: mon: enable level_compaction_dynamic_level_bytes for rocksdb

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22382 from pdvian/wip-24261-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:48:27 +0000 (14:48 -0700)]
Merge pull request #22382 from pdvian/wip-24261-mimic

mimic: bluestore: flush_commit is racy

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22394 from tchaikov/mimic-24406
Yuri Weinstein [Fri, 6 Jul 2018 21:47:30 +0000 (14:47 -0700)]
Merge pull request #22394 from tchaikov/mimic-24406

mimic: osd: retry to read object attrs at EC recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22406 from pdvian/wip-24292-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:44:44 +0000 (14:44 -0700)]
Merge pull request #22406 from pdvian/wip-24292-mimic

mimic: common: fix typo in rados bench write JSON output

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #22525 from tchaikov/mimic-pr-22336
Yuri Weinstein [Fri, 6 Jul 2018 21:43:42 +0000 (14:43 -0700)]
Merge pull request #22525 from tchaikov/mimic-pr-22336

mimic: os/bluestore: firstly delete db then delete bluefs if open db met error.

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
7 years agoMerge pull request #22552 from smithfarm/wip-24468-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:42:40 +0000 (14:42 -0700)]
Merge pull request #22552 from smithfarm/wip-24468-mimic

mimic: tell ... config rm <foo> not idempotent

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22559 from pdvian/wip-24332-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:42:08 +0000 (14:42 -0700)]
Merge pull request #22559 from pdvian/wip-24332-mimic

mimic: PG: add custom_reaction Backfilled and release reservations after bac…

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoMerge pull request #22651 from liewegas/wip-24423-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:40:33 +0000 (14:40 -0700)]
Merge pull request #22651 from liewegas/wip-24423-mimic

mimic: osd: fix getting osd maps on initial osd startup

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22722 from liewegas/wip-hobject-hash-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:39:55 +0000 (14:39 -0700)]
Merge pull request #22722 from liewegas/wip-hobject-hash-mimic

mimic: common/hobject: improved hash calculation

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
7 years agoMerge pull request #22730 from pdvian/wip-24494-mimic
Yuri Weinstein [Fri, 6 Jul 2018 21:38:59 +0000 (14:38 -0700)]
Merge pull request #22730 from pdvian/wip-24494-mimic

mimic: osd/Session: fix invalid iterator dereference in Sessoin::have_backoff()

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoceph-volume lvm.common use destroy-new, doesn't need admin keyring 22900/head
Alfredo Deza [Fri, 8 Jun 2018 19:36:17 +0000 (15:36 -0400)]
ceph-volume lvm.common use destroy-new, doesn't need admin keyring

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 57fb124ce74fe4fb091091be6b4fa63e36d567a2)

7 years agoqa/workunits/suites/pjd.sh: use correct dir name 22882/head
Neha Ojha [Fri, 25 May 2018 00:03:01 +0000 (17:03 -0700)]
qa/workunits/suites/pjd.sh: use correct dir name

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 8fbab63436467aeb72b43f9db8c6b182e48d993c)

7 years agoMerge pull request #22854 from jcsp/wip-mimic-cephfs-doc
John Spray [Thu, 5 Jul 2018 13:41:06 +0000 (14:41 +0100)]
Merge pull request #22854 from jcsp/wip-mimic-cephfs-doc

[mimic] doc/cephfs: remove lingering "experimental" note about multimds

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agomgr/dashboard: Improve max-line-length tslint rule 22849/head
Tiago Melo [Mon, 28 May 2018 13:19:26 +0000 (14:19 +0100)]
mgr/dashboard: Improve max-line-length tslint rule

Prettier is not breaking strings if they are bigger
then the defined max line length.

With this rule we can make sure lines are not exceeding it,
with the exception of import/export lines.

Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit fc67af05b67cb2bbb76fcbea35cf10b1284d25de)

Conflicts:
src/pybind/mgr/dashboard/frontend/tslint.json

7 years agodoc/cephfs: remove lingering "experimental" note about multimds 22854/head
John Spray [Wed, 4 Jul 2018 16:03:05 +0000 (17:03 +0100)]
doc/cephfs: remove lingering "experimental" note about multimds

Signed-off-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #22803 from joscollin/wip-24746-mimic
Nathan Cutler [Wed, 4 Jul 2018 13:06:34 +0000 (15:06 +0200)]
Merge pull request #22803 from joscollin/wip-24746-mimic

mimic: doc: update experimental features - snapshots

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agodoc: update experimental features - snapshots 22803/head
Jos Collin [Thu, 21 Jun 2018 06:03:41 +0000 (11:33 +0530)]
doc: update experimental features - snapshots

* drop the experimental warning of multi-MDS
* update snapshotting blocked off version
* update Multiple filesystems availability in Jewel

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 9ed1158e1de3addf0e768fdb489fc128d09335eb)

7 years agodoc: highlight the flags
Jos Collin [Thu, 21 Jun 2018 06:07:08 +0000 (11:37 +0530)]
doc: highlight the flags

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 1586580d33ff3f1ca4b168d9e3bd1182643e031f)

7 years agomgr/dashboard: Add performance counters breadcrumb
Ricardo Marques [Thu, 17 May 2018 12:15:23 +0000 (13:15 +0100)]
mgr/dashboard: Add performance counters breadcrumb

Fixes: https://tracker.ceph.com/issues/23806
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit e89d5bef7fb768d508db6600174be6388a314b15)

7 years agomgr/dashboard: Remove useless observable unsubscriptions
Ricardo Marques [Thu, 10 May 2018 09:40:09 +0000 (10:40 +0100)]
mgr/dashboard: Remove useless observable unsubscriptions

Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit ae75f15dedf6e9f19744dd4ba568ceb9b2c768e4)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-user-form/rgw-user-form.component.ts
        (retain mimic "Create an observable to add the S3 key when the form is submitted" logic
        but add whitespace so it blends in with the rest of the refactor)

7 years agomgr/dashboard: Add support for queryParams on routerLinkTpl
Ricardo Marques [Thu, 17 May 2018 12:13:45 +0000 (13:13 +0100)]
mgr/dashboard: Add support for queryParams on routerLinkTpl

Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit 9e710e1ad785c86ec32c5c46a6c63349f8615709)

7 years agoMerge pull request #22807 from smithfarm/wip-24699-mimic
Lenz Grimmer [Wed, 4 Jul 2018 08:52:37 +0000 (10:52 +0200)]
Merge pull request #22807 from smithfarm/wip-24699-mimic

mimic: mgr/dashboard: Replace Pool with Pools

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
7 years agoMerge pull request #22832 from ceph/backport-mimic-22828
Alfredo Deza [Tue, 3 Jul 2018 22:44:25 +0000 (18:44 -0400)]
Merge pull request #22832 from ceph/backport-mimic-22828

mimic: ceph-volume: provide a nice errror message when missing ceph.conf

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoceph-volume: always ignore a missing ceph conf in main.py 22832/head
Andrew Schoen [Tue, 3 Jul 2018 11:45:24 +0000 (06:45 -0500)]
ceph-volume: always ignore a missing ceph conf in main.py

Now that we have a nice error message when a ceph.conf is missing
and we try to use values from it, maintaining a list of commands that
don't need ceph.conf isn't as helpful. We had actually missed 'simple
trigger' when we first implemented this causing all our luminous tests
for simple to fail when we backported.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 0487a9ac60c5ba289b7dd7c730b014a298c5998f)

7 years agoceph-volume: show a nice error message when ceph.conf is not loaded
Andrew Schoen [Mon, 2 Jul 2018 20:58:44 +0000 (15:58 -0500)]
ceph-volume: show a nice error message when ceph.conf is not loaded

If the ceph configuration file is not loaded correctly and then values
from it are used then an undescript error message is shown, e.g.

AttributeError: 'property' object has no attribute 'get'

With this change that same error condition shows the message:

RuntimeError: No valid ceph configuration file was loaded.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit e42ecb831914b07c7c8559d4556a8aabd2accd0c)

7 years agoMerge pull request #22747 from ceph/backport-mimic-22724
Alfredo Deza [Tue, 3 Jul 2018 15:31:35 +0000 (11:31 -0400)]
Merge pull request #22747 from ceph/backport-mimic-22724

mimic: ceph-volume: error on commands that need ceph.conf to operate

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agomgr/dashboard: Replaced "Pool" with "Pools" in navigation bar 22807/head
Lenz Grimmer [Mon, 25 Jun 2018 11:38:55 +0000 (13:38 +0200)]
mgr/dashboard: Replaced "Pool" with "Pools" in navigation bar

Replaced the string "Pool" with "Pools" in the navigation component.

Fixes: https://tracker.ceph.com/issues/24623
Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>
(cherry picked from commit ccd59a467864b44a5eb8a4da671f611e0ada7ff9)

7 years agoMerge pull request #22311 from pdvian/wip-24215-mimic
Yuri Weinstein [Mon, 2 Jul 2018 16:52:38 +0000 (09:52 -0700)]
Merge pull request #22311 from pdvian/wip-24215-mimic

mimic: common: partially revert 95fc248 to make get_process_name work

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #22545 from smithfarm/wip-24500-mimic
Yuri Weinstein [Mon, 2 Jul 2018 16:52:04 +0000 (09:52 -0700)]
Merge pull request #22545 from smithfarm/wip-24500-mimic

mimic: osd: eternal stuck PG in 'unfound_recovery'

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22621 from pdvian/wip-24355-mimic
Yuri Weinstein [Mon, 2 Jul 2018 16:51:30 +0000 (09:51 -0700)]
Merge pull request #22621 from pdvian/wip-24355-mimic

mimic: osd: increase default hard pg limit

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22775 from theanalyst/wip-mimic-24716
Yuri Weinstein [Mon, 2 Jul 2018 16:50:49 +0000 (09:50 -0700)]
Merge pull request #22775 from theanalyst/wip-mimic-24716

mimic: qa/workunits/suites/blogbench.sh: use correct dir name

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoqa/workunits/suites/blogbench.sh: use correct dir name 22775/head
Neha Ojha [Tue, 26 Jun 2018 22:30:38 +0000 (18:30 -0400)]
qa/workunits/suites/blogbench.sh: use correct dir name

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 24513c39f1253e6fda2f77aa71aabc0769008a1b)

7 years agoMerge pull request #22697 from yuriw/wip-24316-mimic
Josh Durgin [Wed, 27 Jun 2018 23:13:24 +0000 (16:13 -0700)]
Merge pull request #22697 from yuriw/wip-24316-mimic

qa/tests - added skeleton for mimic point to point upgrades testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoqa/tests - added skeleton for mimic point to point upgrades testing 22697/head
Yuri Weinstein [Mon, 25 Jun 2018 21:03:22 +0000 (14:03 -0700)]
qa/tests - added skeleton for mimic point to point upgrades testing

Fixes https://tracker.ceph.com/issues/24316
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoceph-volume: error on commands that need ceph.conf to operate 22747/head
Andrew Schoen [Tue, 26 Jun 2018 19:19:43 +0000 (14:19 -0500)]
ceph-volume: error on commands that need ceph.conf to operate

We had been ignoring the failure to load ceph.conf for all subcommands
but most of them require that a ceph.conf be present. This changes that
so only commands that do not need ceph.conf ignore the failure to find
it.

Fixes: http://tracker.ceph.com/issues/23941
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit de8b301ac20ae07f59a19bc1b3a083e9c010667d)

7 years agoMerge pull request #22713 from tspmelo/mimic-fix-codelyzer
Lenz Grimmer [Wed, 27 Jun 2018 13:04:40 +0000 (15:04 +0200)]
Merge pull request #22713 from tspmelo/mimic-fix-codelyzer

mimic: mgr/dashboard: fix lint error caused by codelyzer update

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
7 years agoosd/Session: fix invalid iterator dereference in Sessoin::have_backoff() 22730/head
Sage Weil [Sun, 10 Jun 2018 18:37:56 +0000 (13:37 -0500)]
osd/Session: fix invalid iterator dereference in Sessoin::have_backoff()

If p is i->second.end(), we do want to back up a position, but we
shouldn't dereference p for p->first.

Fixes: http://tracker.ceph.com/issues/24486
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2d7bfafa879ca1475173a74a84c66fd7f5e1b6bc)

7 years agocommon/performance: Now hobject_t's hash no longer depends directly on oid. 22722/head
Adam Kupczyk [Mon, 4 Jun 2018 10:50:49 +0000 (12:50 +0200)]
common/performance: Now hobject_t's hash no longer depends directly on oid.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit c14504c157d3d6a8eecddaf6c83280ee254c069d)

7 years agocommon/performance: improved hash calculation for hobject_t and ghobject_t
Adam Kupczyk [Tue, 29 May 2018 10:43:33 +0000 (12:43 +0200)]
common/performance: improved hash calculation for hobject_t and ghobject_t

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 0cdeb682af87a0703f8650e013e4685af0633f3f)

7 years agoqa/workunits/rados: test pool op permissions
Jason Dillaman [Thu, 21 Jun 2018 02:20:14 +0000 (22:20 -0400)]
qa/workunits/rados: test pool op permissions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 97e3f0aa013dad49de1a55f8025327de7e801822)

7 years agoqa/workunits/rbd: test self-managed snapshot create/remove permissions
Jason Dillaman [Tue, 5 Jun 2018 19:40:44 +0000 (15:40 -0400)]
qa/workunits/rbd: test self-managed snapshot create/remove permissions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b12dd0bf419ae834abb31c712830fa9c4b5cda9c)

7 years agopybind/rados: new methods for manipulating self-managed snapshots
Jason Dillaman [Tue, 5 Jun 2018 18:48:17 +0000 (14:48 -0400)]
pybind/rados: new methods for manipulating self-managed snapshots

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 21538304d78df9f3b9f54a5b3c9b6b62fc3e6b48)

7 years agomon/OSDMonitor: enforce caps for all remaining pool ops
Jason Dillaman [Thu, 21 Jun 2018 01:30:47 +0000 (21:30 -0400)]
mon/OSDMonitor: enforce caps for all remaining pool ops

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 61c06200fe59996bba2bb65fc402207bc10fd459)

7 years agomon/OSDMonitor: enforce caps when creating/deleting unmanaged snapshots
Jason Dillaman [Tue, 5 Jun 2018 17:24:48 +0000 (13:24 -0400)]
mon/OSDMonitor: enforce caps when creating/deleting unmanaged snapshots

The entity will require write access to the OSD service or permission
for the synthetic "osd pool op unmanaged-snap" command.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4972e054b32c8200600f27564d50d443e683153e)

Conflicts:
src/mon/OSDMonitor.cc: use 'bufferlist::begin' instead of 'bufferlist::cbegin'

7 years agomgr/dashboard: fix lint error caused by codelyzer update 22713/head
Tiago Melo [Mon, 25 Jun 2018 16:23:00 +0000 (17:23 +0100)]
mgr/dashboard: fix lint error caused by codelyzer update

fixes: https://tracker.ceph.com/issues/24663

Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 4e7f4cbd3fa1d89420a3521888ca7583fd7284fb)

7 years agoMerge pull request #22649 from liewegas/wip-24319-24550-mimic
Josh Durgin [Fri, 22 Jun 2018 00:18:45 +0000 (17:18 -0700)]
Merge pull request #22649 from liewegas/wip-24319-24550-mimic

mimic: os/bluestore: backport 24319 and 24550

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22653 from yuriw/wip-yuriw-distros-mimic
Yuri Weinstein [Thu, 21 Jun 2018 19:59:10 +0000 (12:59 -0700)]
Merge pull request #22653 from yuriw/wip-yuriw-distros-mimic

qa/tests: changed distro symlink to point to new way using supported OSes

7 years agoMerge pull request #22640 from ceph/backport-mimic-22627
Alfredo Deza [Thu, 21 Jun 2018 13:57:27 +0000 (09:57 -0400)]
Merge pull request #22640 from ceph/backport-mimic-22627

mimic: lvm: when osd creation fails log the exception

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoqa/tests: changed disto symlink to point to new way using supporeted OS'es 22653/head
Yuri Weinstein [Tue, 12 Jun 2018 20:49:55 +0000 (13:49 -0700)]
qa/tests: changed disto symlink to point to new way using supporeted OS'es

backport of https://github.com/ceph/ceph/pull/22536

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit 37ac8df5559a6084091670e6a1f479ab9332c24b)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoosd: fix getting osd maps on initial osd startup 22651/head
Paul Emmerich [Fri, 15 Jun 2018 22:45:26 +0000 (00:45 +0200)]
osd: fix getting osd maps on initial osd startup

Commit 89d0c357964ff28b83ac541c4d946f712a7bee47 introduced a check for
deleted pools that relies on getting an older osd map that might not be
available in some situations on OSDs starting up for the very first
time.

fixes https://tracker.ceph.com/issues/24423

Signed-off-by: Paul Emmerich <paul.emmerich@croit.io>
(cherry picked from commit 02180f644094d520d5763afa0e76403591a90b9b)

7 years agoos/bluestore: use vector instead of set for zombies 22649/head
Sage Weil [Tue, 19 Jun 2018 12:15:53 +0000 (07:15 -0500)]
os/bluestore: use vector instead of set for zombies

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ef34d2dc8f3dd40c8426b7373d13dae84f90ff73)

7 years agoos/bluestore: reuse zombie OpSequencers by collection id
Sage Weil [Mon, 18 Jun 2018 12:32:08 +0000 (07:32 -0500)]
os/bluestore: reuse zombie OpSequencers by collection id

We can get a sequence that deletes and then recreates a collection where
the transaction removing the collection is delayed (due to pending IO on
its sequencer) but colleciton create is not (new sequencer).

Avoid any such reordering by recycling the old collection's sequencer if
the zombie_osr has not been reaped yet.

Fixes: http://tracker.ceph.com/issues/24550
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 199a110c9e2a42be64bd7fd873b6b172ce1347a8)

7 years agoqa/suites/rados/objecstore/backends/objectstore: capture coredumps
Sage Weil [Mon, 18 Jun 2018 20:35:38 +0000 (15:35 -0500)]
qa/suites/rados/objecstore/backends/objectstore: capture coredumps

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 55f390d0eb0c96cc563190371a35c41f103eb662)

7 years agoos/bluestore: more debug output
Sage Weil [Fri, 15 Jun 2018 19:13:14 +0000 (14:13 -0500)]
os/bluestore: more debug output

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fd3036d2e186eb0299785c43aefe6f2ce88f244d)

7 years agoos/bluestore: print cnode from _open_collections
Sage Weil [Thu, 14 Jun 2018 19:15:20 +0000 (14:15 -0500)]
os/bluestore: print cnode from _open_collections

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b308b6b1554b3203e02e64e3963ef4901274f23f)

7 years agoos/bluestore: print cnode on fsck
Sage Weil [Fri, 15 Jun 2018 15:41:25 +0000 (10:41 -0500)]
os/bluestore: print cnode on fsck

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f3c89ef6e7ce313a2eda2136be5181735e5684d8)

7 years agoqa/suites/rados/objecstore: preserve data dir for ceph_test_objecstore
Sage Weil [Wed, 13 Jun 2018 11:55:18 +0000 (06:55 -0500)]
qa/suites/rados/objecstore: preserve data dir for ceph_test_objecstore

If we fail, preserve the data directory.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 23dd6279bb518709b0719132d850c70b7fc554a8)

7 years agoos/bluestore: fix SharedBlobSet deregister race
Sage Weil [Sun, 10 Jun 2018 20:57:31 +0000 (15:57 -0500)]
os/bluestore: fix SharedBlobSet deregister race

In commit 8c8944b2c45ca9dc5b8fd4db1590e1d24206c0b3 we fixed one narrow race
and introduced a new (probably less narrow) one:

A: put shared blob foo, nref = 0, start removing self from set
B: open_shared_blob -> lookup gets nullptr (nref==null), creates a new shared blob
  B: takes lock, sets sb_map[sbid] = newblob
A: gets lock, erases sb_map[sbid]
B: open_shared_blob -> lookup gets null, creates another new shared blob

Fix by only removing the sb_map entry for the nref=0 sb if it still points
to us.  If it doesn't, that means some new blob has shown up in its place.

Fixes: http://tracker.ceph.com/issues/24319
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 345ea53bd30426d4e6159d0866497d5a8bf6f327)

7 years agolvm: when osd creation fails log the exception 22640/head
Andrew Schoen [Tue, 19 Jun 2018 19:34:57 +0000 (14:34 -0500)]
lvm: when osd creation fails log the exception

Fixes: http://tracker.ceph.com/issues/24456
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit d622dadef09a53d24c6a7b9119051594f8da1cae)

7 years agoMerge pull request #22341 from b-ranto/wip-branto-mgr-mimic
Boris Ranto [Tue, 19 Jun 2018 08:06:42 +0000 (10:06 +0200)]
Merge pull request #22341 from b-ranto/wip-branto-mgr-mimic

[mimic] Sync up ceph-mgr prometheus related changes

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
7 years agoMerge pull request #22224 from yuriw/wip-yuri-powercycle-mimic
Sage Weil [Mon, 18 Jun 2018 19:42:12 +0000 (14:42 -0500)]
Merge pull request #22224 from yuriw/wip-yuri-powercycle-mimic

mimic: qa/tests:  added supported distro for powercycle suite

7 years agoosd: increase default hard pg limit 22621/head
Josh Durgin [Wed, 23 May 2018 16:17:04 +0000 (09:17 -0700)]
osd: increase default hard pg limit

The hard limit was too easy to hit during failure cases, especially
due to mismatched device sizes causing high variance in pgs/osd.

Fixes: https://tracker.ceph.com/issues/24243
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit dfdedce89521f89046fb13287d9fdd683c91e242)

7 years agoMerge pull request #22468 from votdev/fix_rgw_create_key
Lenz Grimmer [Fri, 15 Jun 2018 07:46:48 +0000 (09:46 +0200)]
Merge pull request #22468 from votdev/fix_rgw_create_key

mimic: mgr/dashboard: Fix bug when creating S3 keys

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
7 years agomgr/dashboard: Fix bug when creating S3 keys 22468/head
Volker Theile [Fri, 8 Jun 2018 11:44:45 +0000 (13:44 +0200)]
mgr/dashboard: Fix bug when creating S3 keys

Currently a new subuser is created if a key is created for the main user. That's because the uid is set as subuser parameter, too.
To fix that the key's user name, e.g. test2 or test1:subuser2, must be splitted into its user and subuser parts.

Signed-off-by: Volker Theile <vtheile@suse.com>
7 years agoPG: add custom_reaction Backfilled and release reservations after backfill 22559/head
Neha Ojha [Mon, 21 May 2018 19:34:31 +0000 (12:34 -0700)]
PG: add custom_reaction Backfilled and release reservations after backfill

After backfill completes, we directly go to the Recovered state without
releasing reservations. The outstanding reservations cause double reservation
issues.

Creating a custom_reaction Backfilled, allows us to release reservations,
before transiting to the Recovered state.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 1abc2320283e9247bec7b0821a6134f31b9b5e29)

7 years agoMerge pull request #22515 from ceph/backport-mimic-21803
Alfredo Deza [Wed, 13 Jun 2018 17:47:48 +0000 (13:47 -0400)]
Merge pull request #22515 from ceph/backport-mimic-21803

mimic: ceph-volume initial take on auto sub-command

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoosd,mds,common: make 'config rm ...' idempotent 22552/head
Sage Weil [Mon, 4 Jun 2018 16:00:08 +0000 (11:00 -0500)]
osd,mds,common: make 'config rm ...' idempotent

Fixes: http://tracker.ceph.com/issues/24408
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 11044f5f1b6221eeb8a7837a9b90cd2a4c39936e)

7 years agocommon/config: fix rm_val error codes
Sage Weil [Mon, 4 Jun 2018 15:59:51 +0000 (10:59 -0500)]
common/config: fix rm_val error codes

EINVAL if the config option is invalid.
ENOENT if the config option wasn't set.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4f957a4ad7fb7029435f38a1da086d59f34f3428)

7 years agoMerge pull request #22477 from alfredodeza/mimic-bluestore-no-meta-paths
Andrew Schoen [Wed, 13 Jun 2018 16:06:13 +0000 (16:06 +0000)]
Merge pull request #22477 from alfredodeza/mimic-bluestore-no-meta-paths

mimic: os/bluestore: don't store/use path_block.{db,wal} from meta

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoceph-volume tests verify get_devices captures device metadata 22515/head
Alfredo Deza [Thu, 3 May 2018 18:33:08 +0000 (14:33 -0400)]
ceph-volume tests verify get_devices captures device metadata

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8deb088f101894763d27a6c31a3e8a2e96fc8320)

7 years agoceph-volume tests.conftest allow custom dirs in tmpfile fixture
Alfredo Deza [Thu, 3 May 2018 18:32:00 +0000 (14:32 -0400)]
ceph-volume tests.conftest allow custom dirs in tmpfile fixture

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9d820e82ac6bc0f46890df1cdaa104e88cab2a7e)

7 years agoceph-volume tests ensure human_readable_size works up to terabytes
Alfredo Deza [Thu, 3 May 2018 16:52:21 +0000 (12:52 -0400)]
ceph-volume tests ensure human_readable_size works up to terabytes

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4ad42f4244062995de003d8141cddcdc1d2b80fc)

7 years agoceph-volume tests verify mapper and /dev devices
Alfredo Deza [Thu, 3 May 2018 16:47:53 +0000 (12:47 -0400)]
ceph-volume tests verify mapper and /dev devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9cfd37344c69ebf1f3d8440f295d79ae05b545e9)

7 years agoceph-volume tests add tests for get_block_devs
Alfredo Deza [Thu, 3 May 2018 16:45:05 +0000 (12:45 -0400)]
ceph-volume tests add tests for get_block_devs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 672c4c4d1c31b4631b1238ed0efb98fef34f2a3d)

7 years agoceph-volume tests verify _map_dev_paths mappings
Alfredo Deza [Thu, 3 May 2018 16:40:17 +0000 (12:40 -0400)]
ceph-volume tests verify _map_dev_paths mappings

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2c4d46955e8d7f52b090cd8b5b5561a2dba443e5)

7 years agoceph-volume devices.auto initial take on auto sub-command
Alfredo Deza [Thu, 3 May 2018 15:05:42 +0000 (11:05 -0400)]
ceph-volume devices.auto initial take on auto sub-command

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e3b04c9677c273f91e0a9de35dcd68c578d124db)

7 years agoceph-volume util.disk create a device metadata utility
Alfredo Deza [Thu, 3 May 2018 14:51:47 +0000 (10:51 -0400)]
ceph-volume util.disk create a device metadata utility

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c6559538db28d6b8bc8529e6b2a62e8619c69bd6)

7 years agoceph-volume util.disk add dev mapping helpers for path resolution
Alfredo Deza [Thu, 3 May 2018 11:38:29 +0000 (07:38 -0400)]
ceph-volume util.disk add dev mapping helpers for path resolution

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c06e63fed8a11e80444e8e7ec2e82373d75aa501)

7 years agoceph-volume tests verify get_file_contents utility
Alfredo Deza [Wed, 2 May 2018 23:48:23 +0000 (19:48 -0400)]
ceph-volume tests verify get_file_contents utility

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0177f71caba3fd64de285d6808b5f859c55ed2fe)

7 years agoceph-volume util.system capture file contents
Alfredo Deza [Wed, 2 May 2018 23:15:10 +0000 (19:15 -0400)]
ceph-volume util.system capture file contents

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 39792573dabac3b85a41760c965b60746ca1f059)

7 years agoceph-volume devices.auto initial take on auto sub-command
Alfredo Deza [Fri, 4 May 2018 17:16:05 +0000 (13:16 -0400)]
ceph-volume devices.auto initial take on auto sub-command

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5649bd724b5d300c7080052433f9b9d1c6774cff)

7 years agoceph-volume main include commented out auto subcommand
Alfredo Deza [Wed, 2 May 2018 23:01:10 +0000 (19:01 -0400)]
ceph-volume main include commented out auto subcommand

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ad1b8cae055ee6c822370d91027613680ccd3dd3)

7 years agoceph-volume tests ensure is_lv detection on LVs
Alfredo Deza [Wed, 2 May 2018 22:59:51 +0000 (18:59 -0400)]
ceph-volume tests ensure is_lv detection on LVs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 47060a1ef4562b29771bdfb5a375a225f18c228c)

7 years agoceph-volume api.lvm create helpers to detect an lv from a device path
Alfredo Deza [Wed, 2 May 2018 20:39:16 +0000 (16:39 -0400)]
ceph-volume api.lvm create helpers to detect an lv from a device path

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 68ca7a691e6e333209b67eb5cea8594f37fb93c1)

7 years agoosd/PG: normalize query processing in Stray and ReplicaActive 22545/head
Sage Weil [Thu, 7 Jun 2018 13:33:46 +0000 (08:33 -0500)]
osd/PG: normalize query processing in Stray and ReplicaActive

A stray PG may end up in ReplicaActive if it is participating in backfill.
However, whether it is or isn't, we should treat queries the same.
Otherwise we end up with weird behaviors like:

 - osd's stray pg moves to ReplicaActive (gets info+log from primary)
 - osd goes down and back up
 - primary restarts peering, request FULLLOG to find missing objects
 - osd ignores FULLLOG because it is ReplicaActive and not Stray

Fixes: http://tracker.ceph.com/issues/24373
Reported-by: Kouya Shimura <kouya@jp.fujitsu.com>
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4e14a380becd61d823a1c86e2dbb41dc8e3a5834)

7 years agoosd/PG: move send_notify() into RecoveryCtx
Sage Weil [Thu, 7 Jun 2018 13:20:54 +0000 (08:20 -0500)]
osd/PG: move send_notify() into RecoveryCtx

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 856fd0243346ca99fa4a3ba5014a7bcdf2c29450)

7 years agoosd/PG: reset PG peering if osd transitions from down -> up
Sage Weil [Thu, 7 Jun 2018 12:07:19 +0000 (07:07 -0500)]
osd/PG: reset PG peering if osd transitions from down -> up

Consider a PG that is stray and ends up in ReplicaActive (because it is
participating as a recovery source).  If it is marked down wrongly and
then comes back up, then the PG will not reset, because there was not
an interval change (the PG is not part of the up or acting sets).
This can leave the PG in an odd state, leading to questionable behavior.
(For example, a stray might be in ReplicaActive and then ignore some
types of query messages.)

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8d3b4ca591090a5ab3b5940f43a86866f2f5b9bc)

7 years agoceph-volume util make chown links affect links as well as paths 22477/head
Alfredo Deza [Fri, 8 Jun 2018 12:54:07 +0000 (08:54 -0400)]
ceph-volume util make chown links affect links as well as paths

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7f79932f2c522da222a27c15182434914ca9a738)

7 years agoceph-volume lvm.activate chown bluestore symlinks
Alfredo Deza [Fri, 8 Jun 2018 12:57:54 +0000 (08:57 -0400)]
ceph-volume lvm.activate chown bluestore symlinks

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2a0911d04cc1eeb19ea2e1d6069095f9e2af26fe)

7 years agoos/bluestore: make bad symlink target error messages more informative
Sage Weil [Fri, 8 Jun 2018 13:34:53 +0000 (08:34 -0500)]
os/bluestore: make bad symlink target error messages more informative

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit aea215103caf669c3e289cc07f978511d61f68b0)

7 years agoos/bluestore: don't store/use path_block.{db,wal} from meta
Sage Weil [Thu, 7 Jun 2018 17:12:46 +0000 (12:12 -0500)]
os/bluestore: don't store/use path_block.{db,wal} from meta

This reverts most of 8d07fa5ad0772b4d979dc10695d230074d1f1e69.

ceph-volume goes to great lengths to ensure that the symlinks in the
osd dir are accurate.  Having these values here is an opportunity to
get them out of sync.  And that can happen very easily if the initial
mkfs was performed using a /dev/sdX device name (which is unstable
across reboots).  Even after ceph-volume corrects the symlink, bluestore
will continue to use the stale device path.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ef510e43f0ee14b49e99beed9ae8feda6db3429a)

7 years agoMerge pull request #22507 from ceph/backport-mimic-22437
Alfredo Deza [Tue, 12 Jun 2018 21:05:25 +0000 (17:05 -0400)]
Merge pull request #22507 from ceph/backport-mimic-22437

mimic: ceph-volume tests destroy osds on monitor hosts

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #22535 from alfredodeza/mimic-cv-ansible-deps
Andrew Schoen [Tue, 12 Jun 2018 18:33:12 +0000 (18:33 +0000)]
Merge pull request #22535 from alfredodeza/mimic-cv-ansible-deps

mimic: ceph-volume tests.functional install new ceph-ansible dependencies

Reviewed-by: Andrew Schoen <aschoen@redhat.com>