ceph.git
5 years ago12.2.11 v12.2.11
Jenkins Build Slave User [Wed, 30 Jan 2019 15:51:26 +0000 (15:51 +0000)]
12.2.11

5 years agoMerge pull request #26117 from ceph/backport-luminous-26084 26163/head
Alfredo Deza [Fri, 25 Jan 2019 15:59:11 +0000 (10:59 -0500)]
Merge pull request #26117 from ceph/backport-luminous-26084

luminous: ceph-volume: mark a device not available if it belongs to ceph-disk

5 years agoMerge pull request #26114 from alfredodeza/luminous-rm37963
Andrew Schoen [Fri, 25 Jan 2019 15:59:02 +0000 (09:59 -0600)]
Merge pull request #26114 from alfredodeza/luminous-rm37963

luminous ceph-volume fix TypeError on dmcrypt when using Python3

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
5 years agoceph-volume: mark a device not available if it belongs to ceph-disk 26117/head
Andrew Schoen [Tue, 22 Jan 2019 15:32:00 +0000 (09:32 -0600)]
ceph-volume: mark a device not available if it belongs to ceph-disk

The `ceph-volume inventory` command will now show if a device is being
used by ceph-disk and mark it not available if so.

Fixes: https://tracker.ceph.com/issues/24871
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit d14a6b284c44a3f61c0dd8c60ccc474203003653)

5 years agoceph-volume tests verify behavior of create_dmcrypt_key in python 3 26114/head
Alfredo Deza [Fri, 18 Jan 2019 17:51:04 +0000 (12:51 -0500)]
ceph-volume tests verify behavior of create_dmcrypt_key in python 3

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 69613cc5a8b2e111e24b75ad5b69f924c89fd75b)

5 years agoceph-volume util.encryption ensure int is used for urandom (py3 compat)
Alfredo Deza [Fri, 18 Jan 2019 17:50:38 +0000 (12:50 -0500)]
ceph-volume util.encryption ensure int is used for urandom (py3 compat)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0ab9fa62d6d46464b16f08d3a5bb3da50f5030e7)

5 years agoMerge pull request #26088 from ukernel/luminous-37977
Abhishek L [Wed, 23 Jan 2019 15:42:14 +0000 (16:42 +0100)]
Merge pull request #26088 from ukernel/luminous-37977

luminous: mds: fix infinite loop in OpTracker::check_ops_in_flight

Reviewed-By: Patrick Donelly <pdonelly@redhat.com>
5 years agomds: fix infinite loop in OpTracker::check_ops_in_flight 26088/head
Yan, Zheng [Wed, 23 Jan 2019 09:32:05 +0000 (17:32 +0800)]
mds: fix infinite loop in OpTracker::check_ops_in_flight

introduced by backport commit 02faf3d
"mds: don't report slow request for blocked filelock request"

Fixes: http://tracker.ceph.com/issues/37977
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
5 years agoRevert "mds: fix infinite loop in OpTracker::check_ops_in_flight"
Yan, Zheng [Wed, 23 Jan 2019 09:29:27 +0000 (17:29 +0800)]
Revert "mds: fix infinite loop in OpTracker::check_ops_in_flight"

This reverts commit adad2d873a37770e4910b04dfddf92f70964be4a.
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
5 years agoMerge pull request #26037 from yuriw/wip-yuriw-p2p-luminous
Yuri Weinstein [Tue, 22 Jan 2019 16:38:04 +0000 (08:38 -0800)]
Merge pull request #26037 from yuriw/wip-yuriw-p2p-luminous

luminous: qa/tests: changed to `supported' distro and moved all `p2p` suites un…

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #25867 from ashishkumsingh/wip-37827-luminous
Sage Weil [Tue, 22 Jan 2019 16:09:06 +0000 (10:09 -0600)]
Merge pull request #25867 from ashishkumsingh/wip-37827-luminous

luminous: mgr: fix crash due to multiple sessions from daemons with same name

5 years agoqa/tests: changed to `supported' distro and moved all `p2p` suites under the same... 26037/head
Yuri Weinstein [Fri, 18 Jan 2019 20:53:05 +0000 (12:53 -0800)]
qa/tests: changed to `supported' distro and moved all `p2p` suites under the same folder

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoMerge pull request #26061 from yuriw/wip-yuriw-p2p-luminous_3
Yuri Weinstein [Mon, 21 Jan 2019 23:53:56 +0000 (15:53 -0800)]
Merge pull request #26061 from yuriw/wip-yuriw-p2p-luminous_3

qa/tests: changed start point from `luminous` to v12.2.10

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #26048 from ukernel/luminous-37977
Yuri Weinstein [Mon, 21 Jan 2019 23:25:33 +0000 (15:25 -0800)]
Merge pull request #26048 from ukernel/luminous-37977

luminous: mds: fix infinite loop in OpTracker::check_ops_in_flight

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #26036 from noahdesu/luminous-pr26015
Noah Watkins [Mon, 21 Jan 2019 21:33:01 +0000 (13:33 -0800)]
Merge pull request #26036 from noahdesu/luminous-pr26015

luminous: cli: dump osd-fsid as part of osd find <id>

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #26030 from alfredodeza/luminous-fix_raw_input
Andrew Schoen [Mon, 21 Jan 2019 21:28:07 +0000 (15:28 -0600)]
Merge pull request #26030 from alfredodeza/luminous-fix_raw_input

luminous ceph-volume: Adapt code to support Python3

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
5 years agoqa/tests: changed start point from `luminous` to v12.2.10 26061/head
Yuri Weinstein [Mon, 21 Jan 2019 19:10:39 +0000 (11:10 -0800)]
qa/tests: changed start point from `luminous` to v12.2.10

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoMerge pull request #26014 from alfredodeza/luminous-bz1644847
Andrew Schoen [Mon, 21 Jan 2019 19:39:41 +0000 (13:39 -0600)]
Merge pull request #26014 from alfredodeza/luminous-bz1644847

luminous ceph-volume zap devices associated with an OSD ID and/or OSD FSID

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
5 years agoMerge pull request #25553 from jan--f/25238-luminous
Alfredo Deza [Mon, 21 Jan 2019 17:15:45 +0000 (12:15 -0500)]
Merge pull request #25553 from jan--f/25238-luminous

luminous ceph-volume: introduce class hierachy for strategies

Reviewed-by: Alfredo Deza <adeza@redhat.com>
5 years agomds: fix infinite loop in OpTracker::check_ops_in_flight 26048/head
Yan, Zheng [Mon, 21 Jan 2019 02:08:51 +0000 (10:08 +0800)]
mds: fix infinite loop in OpTracker::check_ops_in_flight

introduced by backport commit 02faf3dc321
"mds: don't report slow request for blocked filelock request"

Fixes: http://tracker.ceph.com/issues/37977
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
5 years agoMerge pull request #25307 from trociny/wip-37438-luminous
Yuri Weinstein [Fri, 18 Jan 2019 21:56:15 +0000 (13:56 -0800)]
Merge pull request #25307 from trociny/wip-37438-luminous

luminous: crushtool: add --reclassify operation to convert legacy crush maps to use device classes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #25949 from neha-ojha/wip-36686-luminous
Yuri Weinstein [Fri, 18 Jan 2019 20:38:00 +0000 (12:38 -0800)]
Merge pull request #25949 from neha-ojha/wip-36686-luminous

luminous: osd/mon: pg log hard limit with upgrades fixed

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #25804 from ashishkumsingh/wip-37758-luminous
Yuri Weinstein [Fri, 18 Jan 2019 20:33:59 +0000 (12:33 -0800)]
Merge pull request #25804 from ashishkumsingh/wip-37758-luminous

luminous: mds: clean up log messages for standby-replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
5 years agoMerge pull request #25904 from pdvian/wip-37829-luminous
Yuri Weinstein [Fri, 18 Jan 2019 20:33:13 +0000 (12:33 -0800)]
Merge pull request #25904 from pdvian/wip-37829-luminous

luminous : client: fix fuse client hang because its pipe to mds is not ok4

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #26011 from batrick/i37953
Yuri Weinstein [Fri, 18 Jan 2019 20:30:04 +0000 (12:30 -0800)]
Merge pull request #26011 from batrick/i37953

luminous: qa: test_damage needs to silence MDS_READ_ONLY

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agocli: dump osd-fsid as part of osd find <id> 26036/head
Noah Watkins [Thu, 17 Jan 2019 19:16:44 +0000 (11:16 -0800)]
cli: dump osd-fsid as part of osd find <id>

Dumps the osd-fsid uuid as part of the `osd find <id>` command.
Currently this uuid is only available as part of `osd dump` but
ceph-ansible has a use case to interrogate a single osd without needing
the entire osdmap dump.

Fixes: http://tracker.ceph.com/issues/37966
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
(cherry picked from commit 4d01b2c96e0ea1122225c30da21bc39c40e15c0e)

5 years agodoc: pglog_hardlimit flag recommendations 25949/head
Neha Ojha [Fri, 18 Jan 2019 19:01:18 +0000 (14:01 -0500)]
doc: pglog_hardlimit flag recommendations

Signed-off-by: Neha Ojha <nojha@redhat.com>
5 years agoqa/suites/upgrade/jewel-x/stress-split*: require-osd-release luminous after upgrade
Neha Ojha [Fri, 11 Jan 2019 23:35:50 +0000 (18:35 -0500)]
qa/suites/upgrade/jewel-x/stress-split*: require-osd-release luminous after upgrade

Signed-off-by: Neha Ojha <nojha@redhat.com>
5 years agoinclude/rados.h: hide CEPH_OSDMAP_PGLOG_HARDLIMIT from ceph -s
Neha Ojha [Thu, 10 Jan 2019 02:12:17 +0000 (18:12 -0800)]
include/rados.h: hide CEPH_OSDMAP_PGLOG_HARDLIMIT from ceph -s

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit dfc292923548fe1e1b5329555dd46949feb96b99)

5 years agoqa/suites/upgrade/luminous-p2p-stress-split: add split scenario
Neha Ojha [Fri, 11 Jan 2019 02:40:30 +0000 (21:40 -0500)]
qa/suites/upgrade/luminous-p2p-stress-split: add split scenario

This commit adds stress-split test cases to test luminous against a point
release of luminous.

Signed-off-by: Neha Ojha <nojha@redhat.com>
5 years agoqa/suites/upgrade/jewel-x: add pg log settings
Neha Ojha [Fri, 11 Jan 2019 00:25:32 +0000 (19:25 -0500)]
qa/suites/upgrade/jewel-x: add pg log settings

- vary pg log lengths
- test pglog_hardlimit flag

These are qa suites changes specific to luminous.

Signed-off-by: Neha Ojha <nojha@redhat.com>
5 years agomon/OSDMonitor.cc: make a note about reusing jewel feature bit
Neha Ojha [Mon, 7 Jan 2019 23:26:27 +0000 (15:26 -0800)]
mon/OSDMonitor.cc: make a note about reusing jewel feature bit

For OSD_PGLOG_HARDLIMIT, we have reused a jewel feature bit that was retired
in luminous. Therefore, we need to check the release version for
>= CEPH_RELEASE_LUMINOUS, before using it.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 6abcc20dca0ee5a08a3fe7c560750f904fe3fa65)

Conflicts:
src/mon/OSDMonitor.cc: trivial resolution

5 years agomon: add and use OSD_PGLOG_HARDLIMIT feature bit
Neha Ojha [Thu, 20 Dec 2018 17:27:34 +0000 (09:27 -0800)]
mon: add and use OSD_PGLOG_HARDLIMIT feature bit

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 6b0a3ded5baabb19809618de16cdf67c925a8e5a)

Conflicts:
src/mon/OSDMonitor.cc: trivial resolution

5 years agoosd/mon: fix upgrades for pg log hard limit
Neha Ojha [Tue, 18 Dec 2018 00:20:10 +0000 (16:20 -0800)]
osd/mon: fix upgrades for pg log hard limit

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 6ceeab6e204286148a69eb147fbc4045beddef49)

Conflicts:
src/include/rados.h
src/mon/MonCommands.h
src/mon/OSDMonitor.cc
src/osd/OSDMap.cc
          Luminous does not have CEPH_OSDMAP_NOSNAPTRIM flag.
          In nautilus, CEPH_OSDMAP_PGLOG_HARDLIMIT is set by default,
          which is not the case in luminous.

5 years agoosd: bring back old calc_trim_to and rename new method
Neha Ojha [Fri, 14 Dec 2018 23:59:24 +0000 (15:59 -0800)]
osd: bring back old calc_trim_to and rename new method

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 0536aef761fefda9a86d5bc72a12ca8cab931ccb)

5 years agoosd/PrimaryLogPG: fix potential pg-log overtrimming
xie xingguo [Mon, 30 Jul 2018 10:56:56 +0000 (18:56 +0800)]
osd/PrimaryLogPG: fix potential pg-log overtrimming

In https://github.com/ceph/ceph/pull/21580 I set a trap to catch some wired
and random segmentfaults and in a recent QA run I was able to observe it was
successfully triggered by one of the test case, see:

```
http://qa-proxy.ceph.com/teuthology/xxg-2018-07-30_05:25:06-rados-wip-hb-peers-distro-basic-smithi/2837916/teuthology.log
```

The root cause is that there might be holes on log versions, thus the
approx_size() method should (almost) always overestimate the actual number of log entries.
As a result, we might be at the risk of overtrimming log entries.

https://github.com/ceph/ceph/pull/18338 reveals a probably easier way
to fix the above problem but unfortunately it also can cause big performance regression
and hence comes this pr..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 3654d56985c67d15506fa37b56ef5b0c04e01a65)

Conflicts:
src/osd/PrimaryLogPG.cc: trivial resolution

5 years agoosd/PrimaryLogPG: avoid dereferencing invalid complete_to
xie xingguo [Mon, 3 Sep 2018 07:37:36 +0000 (15:37 +0800)]
osd/PrimaryLogPG: avoid dereferencing invalid complete_to

For the auto-repair (EIO caused) case, we will not reinitialize
**complete_to** (because last_complete is equal to last_update!)
and hence there is chance that **complete_to** should aleady
point to **log.end()** before we call recover_got.

We could simply drop it here as we (already) logged the **complete_to**
iterator change in a more compatible way a few lines below.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 69a2cc35840939436da09691ca62476d7f599de4)

5 years agoosd/PrimaryLogPG.cc: limit trimming at can_rollback_to
Neha Ojha [Thu, 16 Aug 2018 18:48:19 +0000 (11:48 -0700)]
osd/PrimaryLogPG.cc: limit trimming at can_rollback_to

This change is motivated by the failures seen in the multimds suite,
where we hit assert(s <= can_rollback_to), while trimming the log in ec
pools.

This is due to the fact that we had removed limits on the trim_to value to
address https://tracker.ceph.com/issues/23979.

But, seems that this could be dangerous for ec pools. So, keep the
can_rollback_to limit, while calculating the trim_to value.

Fixes: http://tracker.ceph.com/issues/21416
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 4b5c6b88d444e2173e716fe4890717873c8dc8e5)

5 years agoosd/PGLog.cc: check if complete_to points to log.end()
Neha Ojha [Sat, 4 Aug 2018 00:38:22 +0000 (17:38 -0700)]
osd/PGLog.cc: check if complete_to points to log.end()

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 630daa1d7d233aa01a2d49bf1bc7b858bf73348a)

5 years agosrc/osd/PG.cc: remove redundant call to trim_log()
Neha Ojha [Tue, 31 Jul 2018 00:09:51 +0000 (17:09 -0700)]
src/osd/PG.cc: remove redundant call to trim_log()

This change is motived by the failure tracked in
https://tracker.ceph.com/issues/25198. The failure highlights a case, when a
call to trim_log() after the PG has recovered, races with the previous op,
on a replica OSD. Since the previous operation has not completed, the
last_complete value for that OSD is not valid, when we try to trim the
log. It is also worth noting that the race is due to MOSDPGTrim going through
the strict queue as a peering message vs regular ops going through the
non-strict queue.

During the investigation of this bug, we noticed that, with
https://tracker.ceph.com/issues/23979, we allow pg log trimming to
happen on the primary and replicas, whenever we cross the upper bound of
the pg log. This also ensures that pg log trimming happens while processing
any new op.

Therefore, the function trim_log(), which earlier served the purpose of
trimming logs on the primary and replicas, just before the PG went into
the Recovered state, is no more required. This acted like a last line of
defense to trim logs, when we did not need the logs any more. But, this call
seems redundant now, because, we are limiting the pg log length at all times.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 283b0bde4a52128c1590afe8e5011b266a2e334b)

Conflicts:
src/osd/PG.cc: We do not need the trim_log() call any longer,
        due to the explanation provided in the commit message.

5 years agoosd/PGLog.cc: use lgeneric_subdout instead of generic_dout
Neha Ojha [Mon, 30 Jul 2018 23:42:55 +0000 (16:42 -0700)]
osd/PGLog.cc: use lgeneric_subdout instead of generic_dout

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 1b6dafb351b4f80e1e2c9a304f9e1d508ae8bf72)

5 years agoosd/PGLog: allow pg log trim when complete_to is less than trim_to
Neha Ojha [Tue, 17 Jul 2018 01:11:27 +0000 (18:11 -0700)]
osd/PGLog: allow pg log trim when complete_to is less than trim_to

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit a5329ba8dd169e55deaff47d042354e53d8e722d)

Conflicts:
src/osd/PGLog.cc: Now it is possible to have complete_to version
        less than or equal to trim version, because the pg log length upper
        limit is a hard limit, and trim can proceed even when there is
        pending recovery/backfill. So do not complain when this happens.

5 years agoosd: reset complete_to when trimming the log past it
Neha Ojha [Tue, 17 Jul 2018 01:01:26 +0000 (18:01 -0700)]
osd: reset complete_to when trimming the log past it

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 38170cdb1b8c3ea7e8b411fabe1fe99abd06cf52)

5 years agoosd: allow trim() to proceed when there are missing items
Neha Ojha [Mon, 16 Jul 2018 23:48:58 +0000 (16:48 -0700)]
osd: allow trim() to proceed when there are missing items

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit de42fee0dff299f4d0377961d05e02fd8f49f21b)

Conflicts:
src/osd/PGLog.cc: The async recovery feature is not present
        in luminous. Remove async recovery requirements from this commit.

5 years agoosd: handle trim() during backfill
Neha Ojha [Mon, 16 Jul 2018 23:31:22 +0000 (16:31 -0700)]
osd: handle trim() during backfill

Remove async recovery components: The async recovery feature is not present
in luminous. We do not need commit 22d17fb5aad6ab9d7525d9492c0e96a36d02879e,
which adds a flag to remember async recovery. We have also removed async
recovery requirements from this commit and modified the commit message to
only reflect backfill.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit e538c31f0f3133f811a8e478fcb25b575cad66bf)

5 years agoosd: print pg log length and trim_to
Neha Ojha [Mon, 16 Jul 2018 22:06:12 +0000 (15:06 -0700)]
osd: print pg log length and trim_to

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit f48584a5b468949c31bffba1b507fb13a8756284)

5 years agoosd: make calc_trim_to() independent of min_last_complete_ondisk
Neha Ojha [Mon, 16 Jul 2018 21:46:21 +0000 (14:46 -0700)]
osd: make calc_trim_to() independent of min_last_complete_ondisk

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 1ae5fd32c92ea2f025344c663535d00f71f2cdda)

Conflicts:
src/osd/PrimaryLogPG.cc: min_last_complete_ondisk and
        pg_log.get_can_rollback_to() are no longer the limit of the pg log.
        Make the head of the pg log the new limit for pg log trimming.

5 years agoMerge pull request #25833 from vshankar/wip-37762
Yuri Weinstein [Fri, 18 Jan 2019 15:47:26 +0000 (07:47 -0800)]
Merge pull request #25833 from vshankar/wip-37762

luminous: config: drop config::lock when invoking config observer

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoceph-volume: Adapt code to support Python3 26030/head
Volker Theile [Thu, 29 Nov 2018 12:48:30 +0000 (13:48 +0100)]
ceph-volume: Adapt code to support Python3

- raw_input() has been renamed to input() in Python3
- Changed signature of prompt_bool. Variables that are named like built-ins must be named like xxx_ and not _xxx

Fixes: https://tracker.ceph.com/issues/37470
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit fe25a0ea625e75c598f6d0749e7259eef167fa8e)

5 years agoceph-volume tests.functional.batch symlink test_zap to all batch scenarios 26014/head
Alfredo Deza [Fri, 7 Dec 2018 17:29:45 +0000 (12:29 -0500)]
ceph-volume tests.functional.batch symlink test_zap to all batch scenarios

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d3c0058911fd0b3e273a7afb1a9f4f6072942d75)

5 years agoceph-volume tests.functional.batch add test_zap yaml to tox.ini
Alfredo Deza [Fri, 7 Dec 2018 17:28:37 +0000 (12:28 -0500)]
ceph-volume tests.functional.batch add test_zap yaml to tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 25843b762953e53c08e6475d1f9dbfd5102a283f)

5 years agoceph-volume tests.functional.batch create a separate test_zap playbook
Alfredo Deza [Fri, 7 Dec 2018 17:08:44 +0000 (12:08 -0500)]
ceph-volume tests.functional.batch create a separate test_zap playbook

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6b8f0d483a6203de204fcc605ade536749a9f6c8)

5 years agoceph-volume lvm.zap update success message for OSD IDs
Alfredo Deza [Fri, 7 Dec 2018 12:53:43 +0000 (07:53 -0500)]
ceph-volume lvm.zap update success message for OSD IDs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 73bb17f79a5b67b28ceace9dc7dcc2dde38dad5e)

5 years agodoc/man/ceph-volume add zapping by osd-id examples
Alfredo Deza [Thu, 6 Dec 2018 20:30:38 +0000 (15:30 -0500)]
doc/man/ceph-volume add zapping by osd-id examples

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit acbf7c9b2ad2810621d7ed3987e814cd8c2a2f65)

5 years agodoc/ceph-volume add zapping by osd-id examples
Alfredo Deza [Thu, 6 Dec 2018 20:30:22 +0000 (15:30 -0500)]
doc/ceph-volume add zapping by osd-id examples

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f82304f4b5f5ecd865b6a208f894b59cec6386e2)

5 years agoceph-volume tests.lvm verify associated lvs by osd id+fsid behavior
Alfredo Deza [Thu, 6 Dec 2018 20:00:00 +0000 (15:00 -0500)]
ceph-volume tests.lvm verify associated lvs by osd id+fsid behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 40a414464727375d5c558532aed3b70766ec5b78)

5 years agoceph-volume lvm.zap initial take on zapping by OSD ID, FSID
Alfredo Deza [Thu, 6 Dec 2018 15:54:11 +0000 (10:54 -0500)]
ceph-volume lvm.zap initial take on zapping by OSD ID, FSID

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 59f6cec6dd0687a2528f75a518bd0d4617c27fe0)

5 years agoqa: silence read-only WRN for damage testing 26011/head
Patrick Donnelly [Wed, 16 Jan 2019 18:52:09 +0000 (10:52 -0800)]
qa: silence read-only WRN for damage testing

Fixes: http://tracker.ceph.com/issues/37944
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ac302de7b7725c259fee1852e3f402533d69f8cf)

5 years agoMerge pull request #25719 from ashishkumsingh/wip-37553-luminous
Yuri Weinstein [Thu, 17 Jan 2019 18:06:56 +0000 (10:06 -0800)]
Merge pull request #25719 from ashishkumsingh/wip-37553-luminous

luminous: osdc/Objecter: update op_target_t::paused in _calc_target

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agoMerge pull request #25967 from batrick/i37922
Yuri Weinstein [Wed, 16 Jan 2019 20:31:23 +0000 (12:31 -0800)]
Merge pull request #25967 from batrick/i37922

luminous: qa: test_damage fixes

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoMerge pull request #25968 from batrick/i37899
Yuri Weinstein [Wed, 16 Jan 2019 20:30:52 +0000 (12:30 -0800)]
Merge pull request #25968 from batrick/i37899

luminous: mds: purge queue recovery hangs during boot if PQ journal is damaged

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoMerge pull request #25762 from pdvian/wip-37635-luminous
Yuri Weinstein [Wed, 16 Jan 2019 13:03:19 +0000 (05:03 -0800)]
Merge pull request #25762 from pdvian/wip-37635-luminous

luminous: cephfs: race of updating wanted caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
5 years agoMerge pull request #25826 from Vicente-Cheng/wip-37092-luminous
Yuri Weinstein [Wed, 16 Jan 2019 13:02:35 +0000 (05:02 -0800)]
Merge pull request #25826 from Vicente-Cheng/wip-37092-luminous

luminous: mds: "src/mds/MDLog.cc: 281: FAILED ceph_assert(!capped)" during max_mds thrashing

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

5 years agoceph-volume lvm.strategies remove unused import 25553/head
Alfredo Deza [Mon, 17 Dec 2018 18:34:58 +0000 (13:34 -0500)]
ceph-volume lvm.strategies remove unused import

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d127ae3e5901c6477ed44f4d50be834040c44b6f)

  Conflicts:
  src/ceph-volume/ceph_volume/devices/lvm/strategies/bluestore.py
          resolved by removing str_to_int import.

5 years agoceph-volume: filestore strategy use strategy class hierarchy
Jan Fajerski [Mon, 26 Nov 2018 13:03:56 +0000 (14:03 +0100)]
ceph-volume: filestore strategy use strategy class hierarchy

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 34df12ea6a8686f161b4bdfde92e557743f59cf4)

5 years agoceph-volume: bluestore strategy use strategy class hierarchy
Jan Fajerski [Mon, 26 Nov 2018 13:03:03 +0000 (14:03 +0100)]
ceph-volume: bluestore strategy use strategy class hierarchy

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 302334ac6a4375125b6c1e68bf7160f8bc0597c2)

5 years agoceph-volume: add strategies.py to for shared code
Jan Fajerski [Mon, 26 Nov 2018 13:01:57 +0000 (14:01 +0100)]
ceph-volume: add strategies.py to for shared code

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit b38987ee9652527e4845f2beb6406cbd398dac9b)

5 years agoMerge pull request #25384 from ifed01/wip-ifed-fix2-expand-luminous
Yuri Weinstein [Wed, 16 Jan 2019 00:02:57 +0000 (16:02 -0800)]
Merge pull request #25384 from ifed01/wip-ifed-fix2-expand-luminous

luminous: core: os/bluestore_tool: fix bluefs expand

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #25965 from yuriw/wip-yuriw-p2p-luminous
Yuri Weinstein [Tue, 15 Jan 2019 23:29:22 +0000 (15:29 -0800)]
Merge pull request #25965 from yuriw/wip-yuriw-p2p-luminous

luminous: qa/tests: added v12.2.9 and v12.2.10 to the mix

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agomds: allow boot on read-only 25968/head
Patrick Donnelly [Sun, 23 Dec 2018 22:22:49 +0000 (14:22 -0800)]
mds: allow boot on read-only

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c7ce967b778a0b86b335f6801301e484aaf6ebc3)

Conflicts:
src/mds/MDSRank.cc

5 years agomds: setup readonly mode for PurgeQueue
Patrick Donnelly [Tue, 18 Dec 2018 23:11:02 +0000 (15:11 -0800)]
mds: setup readonly mode for PurgeQueue

If the PQ faces an error, it should go read-only along with the MDS rank.

Fixes: http://tracker.ceph.com/issues/37543
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 4cccc4dffb0915ef9e7d3b446e9a32f277646562)

Conflicts:
src/mds/PurgeQueue.cc

5 years agomds: add missing locks for PurgeQueue methods
Patrick Donnelly [Tue, 18 Dec 2018 23:08:11 +0000 (15:08 -0800)]
mds: add missing locks for PurgeQueue methods

These could race with the asynchronous workings of the PQ.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c7350ac23c73867b52cd9b7bb23b6c618eebe44d)

Conflicts:
src/mds/PurgeQueue.cc

5 years agomds: delete on_error context on des
Patrick Donnelly [Tue, 18 Dec 2018 22:00:29 +0000 (14:00 -0800)]
mds: delete on_error context on des

Otherwise it leaks.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 33279822eabb64380f5968cc645735a8f99a3ac1)

5 years agoqa: fix damage expectation setting 25967/head
Patrick Donnelly [Wed, 9 Jan 2019 00:26:14 +0000 (16:26 -0800)]
qa: fix damage expectation setting

The purge queue expectation was being ignored.

Fixes: http://tracker.ceph.com/issues/37837
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 360550b1ab143bb59fd38c77569f7f5ef762801a)

5 years agoqa: fix loop variable reference
Patrick Donnelly [Tue, 8 Jan 2019 23:51:53 +0000 (15:51 -0800)]
qa: fix loop variable reference

Otherwise the Mutation for Truncate is done on obj_id of the last iteration of the previous loop.

Fixes: http://tracker.ceph.com/issues/37836
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 909d0f1333a730c648c08cda047bb5f356a61986)

5 years agoqa/tests: added v12.2.9 and v12.2.10 to the mix 25965/head
Yuri Weinstein [Tue, 15 Jan 2019 17:37:17 +0000 (09:37 -0800)]
qa/tests: added v12.2.9 and v12.2.10 to the mix

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoMerge branch 'luminous' into wip-ifed-fix2-expand-luminous 25384/head
Josh Durgin [Mon, 14 Jan 2019 21:12:58 +0000 (13:12 -0800)]
Merge branch 'luminous' into wip-ifed-fix2-expand-luminous

5 years agoMerge pull request #25829 from badone/wip-examples-link-order-fix
Yuri Weinstein [Sat, 12 Jan 2019 22:05:44 +0000 (14:05 -0800)]
Merge pull request #25829 from badone/wip-examples-link-order-fix

luminous: examples: fix link order in librados example Makefile

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #25845 from pdvian/wip-37811-luminous
Yuri Weinstein [Sat, 12 Jan 2019 22:05:03 +0000 (14:05 -0800)]
Merge pull request #25845 from pdvian/wip-37811-luminous

luminous: mon/OSDMonitor: do not populate void pg_temp into nextmap

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agoMerge pull request #25847 from pdvian/wip-37813-luminous
Yuri Weinstein [Sat, 12 Jan 2019 22:04:30 +0000 (14:04 -0800)]
Merge pull request #25847 from pdvian/wip-37813-luminous

luminous: mon: shutdown messenger early to avoid accessing deleted logger

5 years agoMerge pull request #25922 from alfredodeza/luminous-ceph-volume-fix-json
Alfredo Deza [Sat, 12 Jan 2019 20:19:31 +0000 (15:19 -0500)]
Merge pull request #25922 from alfredodeza/luminous-ceph-volume-fix-json

luminous ceph-volume: fix JSON output in `inventory`

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
5 years agoMerge pull request #25677 from joscollin/wip-37700-luminous
Yuri Weinstein [Sat, 12 Jan 2019 00:09:05 +0000 (16:09 -0800)]
Merge pull request #25677 from joscollin/wip-37700-luminous

luminous: mds: fix bug filelock stuck at LOCK_XSYN leading client can't read data

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #25682 from joscollin/wip-37633-luminous
Yuri Weinstein [Sat, 12 Jan 2019 00:08:24 +0000 (16:08 -0800)]
Merge pull request #25682 from joscollin/wip-37633-luminous

luminous: mds: remove duplicated l_mdc_num_strays perfcounter set

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #25684 from joscollin/wip-37631-luminous
Yuri Weinstein [Sat, 12 Jan 2019 00:08:07 +0000 (16:08 -0800)]
Merge pull request #25684 from joscollin/wip-37631-luminous

luminous: client: do not move f->pos untill success write

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #25686 from joscollin/wip-37737-luminous
Yuri Weinstein [Sat, 12 Jan 2019 00:07:44 +0000 (16:07 -0800)]
Merge pull request #25686 from joscollin/wip-37737-luminous

luminous: MDSMonitor: allow beacons from stopping MDS that was laggy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #25695 from joscollin/wip-37625-luminous
Yuri Weinstein [Sat, 12 Jan 2019 00:07:22 +0000 (16:07 -0800)]
Merge pull request #25695 from joscollin/wip-37625-luminous

luminous: pybind/mgr/status: fix ceph fs status in py3 environments.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #25696 from joscollin/wip-36502-luminous
Yuri Weinstein [Sat, 12 Jan 2019 00:06:57 +0000 (16:06 -0800)]
Merge pull request #25696 from joscollin/wip-36502-luminous

luminous: qa: increase timeout for cleanup

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #25779 from pdvian/wip-37694-luminous
Yuri Weinstein [Sat, 12 Jan 2019 00:06:30 +0000 (16:06 -0800)]
Merge pull request #25779 from pdvian/wip-37694-luminous

luminous: mon: mark REMOVE_SNAPS messages as no_reply

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #25784 from ukernel/luminous-37739
Yuri Weinstein [Sat, 12 Jan 2019 00:05:56 +0000 (16:05 -0800)]
Merge pull request #25784 from ukernel/luminous-37739

luminous: extend reconnect period when mds is busy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #25805 from ashishkumsingh/wip-36504-luminous
Yuri Weinstein [Sat, 12 Jan 2019 00:05:28 +0000 (16:05 -0800)]
Merge pull request #25805 from ashishkumsingh/wip-36504-luminous

luminous: qa: use timeout for fs asok operations

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #25928 from neha-ojha/wip-whitelist-slow-request
Yuri Weinstein [Sat, 12 Jan 2019 00:00:21 +0000 (16:00 -0800)]
Merge pull request #25928 from neha-ojha/wip-whitelist-slow-request

luminous: qa/tasks/thrashosds-health.yaml: whitelist 'slow request'

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoqa/tasks/thrashosds-health.yaml: whitelist 'slow request' 25928/head
Neha Ojha [Fri, 11 Jan 2019 23:19:27 +0000 (18:19 -0500)]
qa/tasks/thrashosds-health.yaml: whitelist 'slow request'

https://github.com/ceph/ceph/pull/25824 adds slow request to OSD logs.
To deal with it, whitelist 'slow request' instead of 'slow requests'.
This PR is specific to luminous because later versions whitelist it correctly.

Signed-off-by: Neha Ojha <nojha@redhat.com>
5 years agoceph-volume: fix JSON output in `inventory` 25922/head
Sebastian Wagner [Thu, 22 Nov 2018 17:01:50 +0000 (18:01 +0100)]
ceph-volume: fix JSON output in `inventory`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a3e6f569b4fa0419dff4690a72e9be6fe0a255c1)

5 years agoMerge pull request #25889 from pdvian/wip-37820-luminous
Yuri Weinstein [Fri, 11 Jan 2019 13:44:35 +0000 (05:44 -0800)]
Merge pull request #25889 from pdvian/wip-37820-luminous

luminous: mds: create heartbeat grace config option

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

5 years agoMerge pull request #25890 from vshankar/wip-purge-single-mds-multifs-test
Yuri Weinstein [Fri, 11 Jan 2019 13:43:42 +0000 (05:43 -0800)]
Merge pull request #25890 from vshankar/wip-purge-single-mds-multifs-test

luminous:  qa: remove single mds yaml for cephfs multifs test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #25431 from batrick/i37540
Yuri Weinstein [Fri, 11 Jan 2019 13:42:43 +0000 (05:42 -0800)]
Merge pull request #25431 from batrick/i37540

luminous: mds: obsolete MDSMap option configs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by:  Venky Shankar <vshankar@redhat.com>

5 years agoMerge pull request #25824 from neha-ojha/wip-1659156-luminous
Yuri Weinstein [Fri, 11 Jan 2019 13:39:55 +0000 (05:39 -0800)]
Merge pull request #25824 from neha-ojha/wip-1659156-luminous

luminous: osd/OSD.cc: log slow requests in OSD logs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoclient: fix fuse client hang because its pipe to mds is not ok 25904/head
IvanGuan [Fri, 4 Jan 2019 04:22:27 +0000 (12:22 +0800)]
client: fix fuse client hang because its pipe to mds is not ok

If fuse client session had been killed by mds and the mds daemon restart
or hot-standby switch happens right away but the client did not receive
any message from monitor due to network or other whatever reason untill
the mds become active again.Thus cause client didn't do closed_mds_session
lead the seession still is STATE_OPEN but client can't send any message to
mds because its pipe is not ok.So we should close the stale session so that
it can be reopened again.

Fixes: http://tracker.ceph.com/issues/36079
Signed-off-by: Guan yunfei <yunfei.guan@xtaotech.com>
(cherry picked from commit 0e137de26e85942f8b40f7b13e564bd4c31b37f9)

Conflicts:
src/client/Client.cc : Resolved in handle_mds_map

5 years agorgw: sanitize customer encryption keys from log output in v4 auth
Casey Bodley [Mon, 10 Dec 2018 17:38:01 +0000 (12:38 -0500)]
rgw: sanitize customer encryption keys from log output in v4 auth

Fixes: http://tracker.ceph.com/issues/37847
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ba55e2a96c9dfcc7aa2311431beaaa23cb05c30d)

5 years agorgw: mimic gconf changes
Abhishek Lekshmanan [Mon, 10 Dec 2018 23:30:46 +0000 (00:30 +0100)]
rgw: mimic gconf changes

As the largeish change from master g_conf() isn't in mimic yet, use the g_conf
global structure, also make rgw_op use the value from req_info ceph context as
we do for all the requests

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 01d647310ec2d7d423df1196eb2a7aef685d832e)