Fix inventory file name

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add ansible based upgrade test

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add upgrades task for ceph-ansible

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

fix duplicate osd numbering

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Move to 6 node cluster

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add rgw system tests

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

initial calamari api tests configured for downstream

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

few ceph-ansible and ceph-deploy tests in smoke

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

downstream upgrade suite

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add rh downstream package list

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Remove ceph-deploy test from smoke suite

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #17383 from liewegas/wip-luminous-mon-bits-luminous

mon: final luminous mon bits

Reviewed-by: Kefu Chai <kchai@redhat.com>

PendingReleaseNotes: must upgrade to 12.2.1 before mimic

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #17429 from ceph/wip-luminous-x-luminous

qa: add upgrade/luminous-x suite

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Initial check in luminous-x suite

same from master https://github.com/ceph/ceph/pull/17160

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #17408 from ceph/luminous-bz1485011

luminous: ceph-volume Persist non-lv devices for journals

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests add new ceph_* ansible dev variables required (centos)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d3abfd42ef8e558a1bbb31e24d0ab3c44ee7f4c4)

ceph-volume tests add new ceph_* ansible dev variables required (xenial)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ebb48c58123a1cfcfc6eccdf9f6c2b3a720f1233)

ceph-volume util create a disk utility for blkid operations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a19e1288e8ae044f41dff9a1770126aa540cc729)

ceph-volume lvm.prepare store the blkid uuid of a partition journal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5160c5398bbf107bcc826696664c503aa52d0020)

ceph-volume lvm.activate use the partuuid of a partition to link the journal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a1080f833b35c5de30d7b577deabf0a0e82fbbf2)

ceph-volume tests update the ansible version for functional/tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 007655afa3def6788e07d2db10c7cd9cb4ab45b7)

ceph-volume tests add pv* related unit tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7c22743c9aa1306242fc09ccae5dedf94e025cc4)

ceph-volume lvm.api include lv_uuid as output fields

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d85656da2531471e11cb90fcc74570c9c2b804e0)

ceph-volume lvm.activate always update the link to the journal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6a1de41838cb9dbbdc4a8f59eb07e4340ccb9680)

ceph-volume lvm.activate retrieve the journal uuid if journal is a device

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a368fa1ad6d6265c5094eacbe7e72f642b43559d)

ceph-volume exceptions create a specifc error for multiple pvs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 73315c49115dbb45f3bb6c9e46955b333602cec2)

ceph-volume lvm.prepare make a journal a pv, use uuids always

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 913cef2d01b91e12c16e42d268043886b62c72fa)

ceph-volume lvm.api create the PVolumes class and helpers

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f2bc125d980485c7b456e0193ec9c66ea0acbe54)

ceph-volume lvm.api create the PVolume class

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1725a5f3589ba2faddc9ce633997e5563fc66a01)

ceph-volume lvm.api add a helper to create pvs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 00f3fa401871f3108558fea71f6ac03660d00ba1)

qa/suites/rados/thrash/d-luminous/at-end: check for purged_snapdirs

Signed-off-by: Sage Weil <sage@redhat.com>

mon: set purged_snapdirs OSDMap flag once snapsets have all converted

This makes it easier to test whether the upgrade + conversion has
completed.  In particular, mimic+ will be able to simply test for this
flag without waiting for complete PG stats to know whether it is safe to
upgrade beyond luminous.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c3cbaad2a1a4df68fe39394e07c2f3ac9818b9c8)

mon: CEPH_MON_FEATURE_INCOMPAT_LUMINOUS

- set this once we have a luminous quorum
- require luminous if we have this set locally
- include this on mkfs

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5b6c451e3fdaf0785b197fdfacf1c782b932bec9)

Merge pull request #17381 from dillaman/ritz303-wip-doc-20437

luminous: doc/rbd: iSCSI Gateway Documentation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: tweaks for the RBD iSCSI configuration documentation

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b0abab17e3fa96ca973791a3afe077406d066035)

doc/rbd: Replaced numbered steps with a # for dynamic numbering

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit f109031cb5c8150f042f9754f16bfa5216411eda)

doc/rbd: Updated the content based on engineering feedback

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 2a59c82b3d4bbb13bc82cf62fce1d9cb5ba7e6be)

doc/rbd: Updated the content based on engineering feedback

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 1d4f5b458d127065d92652c323c73704cf775754)

doc/rbd: Updated the content based on engineering feedback

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 441820d95890d95ea79bae55afb3ca3b1a8a36e3)

doc/rbd: Updated the requirements based on engineering feedback

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit c57266bf82c554b93653489a4b355f53b4b86424)

doc/rbd: Added the iSCSI gateway requirements content

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 9335742140a2b4e8b408560c600a490b2749bd9d)

doc/rbd: Added the iSCSI gateway overview content

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit d022034bb3a6e1b00cf185cae0b5fa3f4b617a38)

doc/rbd: Added the iSCSI gateway 'gwtop' monitoring content

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 7e8233429ba158c1a4cfe6e58a7ce24ffe8759c9)

doc/rbd: Added the iSCSI targets, Ansible and CLI docs

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit cbf654c702e7b7a8e7023c4707c4ea38d584be3e)

doc/rbd: Added the iSCSI initiators, RHEL, Windows, and ESX docs

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit f0e177fa3d8807cfa6ae17f8a0dc759a045623f9)

doc/rbd: Updated the "toctree" adding the iSCSI gateway overview doc

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 4ef8426bff9a32e00093f7c5d2b6b9620252293b)

doc/rbd: Added the unmodified iSCSI initiator images for ESX and Windows 2016

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 2fd61f8de1587d2bfbbb7e15c952a29566b390a1)

Merge pull request #17359 from tchaikov/wip-luminous-ceph-release-stable

luminous: ceph_release: luminous is now 'stable' (12.2.x)

ceph_release: luminous is now 'stable' (12.2.x)

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge tag 'v12.2.0' into luminous

v12.2.0

Merge pull request #17332 from tchaikov/wip-luminous-pr-17269

luminous: mon: fix two stray legacy get_health() callers

Reviewed-by: Sage Weil <sage@redhat.com>

12.2.0

mon: fix two stray legacy get_health() callers

These need to call the new method when appropriate.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 69038a18070d0cd47dc857020cce115514beaa3d)

Merge pull request #17327 from tchaikov/wip-luminous-pr-17326

luminous: doc: fix order of options in osd new

Reviewed-by: Sage Weil <sage@redhat.com>

doc: fix order of options in osd new

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit e47463028921f0414e2f64c467f74cf37a1de085)

12.2.0

Merge pull request #17302 from tchaikov/wip-luminous-pr-17242

luminous: mon: warn about using osd new instead of osd create

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17303 from tchaikov/wip-luminous-pr-17225

luminous: os/bluestore: print leaked extents to debug output

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17264 from liewegas/wip-21127-luminous

luminous: mon/pgmap: add objects prefix for unfound type

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

qa/suites/rados/upgrade/jewel-x-singleton: tolerate sloppy past_intervals

See-also: d5d5d7d1
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b2d7f4f4c7a457f9282743c36413c8c8127ca94f)

Merge pull request #17306 from liewegas/wip-powercycle-slow-trim

qa/suites/powercycle/osd/whitelist_health: whitelist more

qa/suites/powercycle/osd/whitelist_health: whitelist more

"2017-08-26 16:09:27.704418 mon.a mon.0 172.21.15.169:6789/0 876 : cluster [WRN] MDS health message (mds.0): Behind on trimming (66/30)" in cluster log

Signed-off-by: Sage Weil <sage@redhat.com>

mon: warn about using osd new instead of osd create

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 834ab275b703637b1c3f1a466f610ef672f5df73)

os/bluestore: print leaked extents to debug output

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 13577a4f35d7b5ec4fad5d68ec3f750f8eb4e60e)

Merge pull request #17282 from dillaman/wip-21017-luminous

luminous: mgr/dashboard: fix duplicate images listed on iSCSI status page

Reviewed-by: John Spray <john.spray@redhat.com>

mgr/dashboard: fix duplicate images listed on iSCSI status page

Fixes: http://tracker.ceph.com/issues/21017
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e16e52ee1d30a86f1e9ddbbff6add8dec6deb7c4)

Merge pull request #17274 from xiexingguo/wip-luminous-pr-17271

luminous: mon/MonCommands: fix copy-and-paste error

Merge pull request #17273 from xiexingguo/wip-luminous-pr-17268

luminous: os/bluestore: compensate for bad freelistmanager size/blocks metadata

os/bluestore: compensate for bad freelistmanager size/blocks metadata

This repairs bluestores created before http://tracker.ceph.com/issues/21089
was fixed in f6f1ae3724d593d3709d982c973ec18a25a47b6e.

In both cases, the freelistmanager's size is off by one block (4k).  In
one case, it is just a matter of fixing the size and twiddling the trailing
bit.  In the second case, the size delta causes freelistmanager to need
a new row, which means the blocks count also changes, and we have lots
of bits to zero (all but one in the new row).

Both are silently corrected by fsck in this patch.

Fixes: http://tracker.ceph.com/issues/21089
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c029a9645b13d0c0cf412940010b90ac10638ec3)

mon/MonCommands: fix copy-and-paste error

Class is definitely required by default for the "crush rule ls-by-class"
command.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 49e293d5523840af4ae6ea589f6b015602a63c40)

Merge pull request #17233 from tchaikov/wip-luminous-pr-17183

luminous: osd/PGBackend: delete reply if fails to complete delete request

Merge pull request #17257 from tchaikov/wip-luminous-pr-17248

luminous: mon/OSDMonitor: check creating_pgs.last_scan_epoch instead when sending creates

Merge pull request #17260 from liewegas/wip-pr-17029-luminous

luminous: mon: "ceph osd crush rule rename" support

qa/standalone/scrub/osd-scrub-repair: fix grep pattern

PGMap shows

    ss << pg_sum.stats.sum.num_objects_unfound
       << "/" << pg_sum.stats.sum.num_objects << " objects unfound (" << b << "%)";

but we were grepping for "1/1 unfound" instead of "1/1 objects
unfound".

Introduced by fe81b7e3a5034ce855303f93f3e413f3f2dc74a8.

Fixes: http://tracker.ceph.com/issues/21127
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 84465bf5a583651436df7813d3b24857dc49460d)

mon/pgmap: add objects prefix for unfound type

the objects prefix display indicates the scope of the "unfound",
like objects "misplaced" err.

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
(cherry picked from commit fe81b7e3a5034ce855303f93f3e413f3f2dc74a8)

Merge pull request #17232 from tchaikov/wip-luminous-pr-17179

luminous: mon/OSDMonitor: fix improper input/testing range of crush somke testing

Merge pull request #17234 from theanalyst/wip-21097-luminous

luminous: multisite: FAILED assert(prev_iter != pos_to_prev.end()) in RGWMetaSyncShardCR::collect_children()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

crush, mon: "ceph osd crush rule ls-by-class" support

This command returns all crush rules that are currently
referencing the device class specified by user.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 7c67f95201316a240c7cdf1d8619b7642ff8fc33)

mon: "ceph osd crush rule rename" support

User may specify a rule with the same name of the pool that it serves.
Since a pool can be renamed, so does the rule.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit a5075ed253940471b347ba0773f66ea6e61398d0)

Merge pull request #17259 from cbodley/wip-luminous-qa-rgw-pool-application

luminous: qa/rgw: use 'ceph osd pool application enable' on created pools

qa/rgw: enable 'rgw' application on created pools

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa: add optional 'application' to pool creation helpers

Signed-off-by: Casey Bodley <cbodley@redhat.com>

mon/OSDMonitor: check creating_pgs.last_scan_epoch instead when sending creates

we cannot be sure that the creating_pgs_by_osd_epoch is in sync with
creating_pgs.pgs even if mapping.get_epoch() is less or equal to
creating_pgs_epoch. because, 1) the access to mapping.epoch is not
protected by lock, 2) even worse, the mapping might be not finished yet
when we are trying to send pg-creates to subscribers.

so instead of comparing creating_pgs_epoch with mapping's epoch, we
should compare it with creating_pgs.last_scan_epoch. the former is
updated once the creating_pgs_by_osd_epoch is updated with the latest
mapping's epoch and creating_pgs.pgs; the latter is updated with current
osdmap's epoch when creating_pgs is being updated with the inc osd map.
if we are using a creating_pgs_epoch in sync, creating_pgs_epoch should
be creating_pgs.last_scan_epoch + 1.

Fixes: http://tracker.ceph.com/issues/20785
Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/PGBackend: release a msg using msg->put() not delete

fix the regression introduced by 1c18b5cb

Fixes: http://tracker.ceph.com/issues/20913
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d2d941dd19b9dd6e41429d92cdab8390f9c3084d)

osd/PGBackend: delete reply if fails to complete delete request

if any of the objects fails to be deleted due to pg reset after latest
osdmap, the pg recovery delete reply won't be sent to the primary OSD.
in that case, we should delete the reply.

Fixes: http://tracker.ceph.com/issues/20913
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1c18b5cb0c27d7976e6d3d5e4ea6c3935685019b)

Merge pull request #17214 from tchaikov/wip-luminous-pr-17014

luminous: crush: various weight-set fixes

Merge pull request #17215 from tchaikov/wip-luminous-pr-17099

luminous: mon/PGMap: fix "0 stuck requests are blocked > 4096 sec" warn

Merge pull request #17228 from tchaikov/wip-luminous-pr-17058

luminous: crush: fix CrushCompiler won't compile maps with empty shadow tree

Merge pull request #17229 from tchaikov/wip-luminous-pr-17083

luminous: crush: force rebuilding shadow hierarchy after swapping buckets

Merge pull request #17230 from tchaikov/wip-luminous-pr-17034

luminous: mon/OSDMonitor: add plain output for "crush class ls-osd" command

Merge PR #17240 into luminous

* refs/remotes/upstream/pull/17240/head:
mds: check cap string only if !allow_all
mds/MDSDaemon: add 'is_valid=false' when failed to parse caps

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #17238 into luminous

* refs/remotes/upstream/pull/17238/head:
mon: get writeable osdmap for added data pool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #17237 into luminous

* refs/remotes/upstream/pull/17237/head:
fuse: use c++ allocations for group list

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge PR #17236 into luminous

* refs/remotes/upstream/pull/17236/head:
client: fix compat version on MStatfs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #17235 into luminous

* refs/remotes/upstream/pull/17235/head:
client: fix locking in Client::getcwd

Reviewed-by: Sage Weil <sage@redhat.com>

qa/tasks/ceph_deploy: gatherkeys before mgr deploy

Otherwise we may be missing the bootstrap-mgr key.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 800fdd9953013f975acd70de060bb828c83f30bc)

Merge pull request #17196 from theanalyst/wip-21051-luminous

luminous: Improve size scrub error handling and ignore system attrs in xattr checking

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17241 from dzafman/wip-pidfile-luminous

test/CMakeLists: disable test_pidfile.sh

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mds: check cap string only if !allow_all

This corrects a regression introduced by #16891 which fixes
http://tracker.ceph.com/issues/20990. Not using cephx would
cause all clients to fail auth with:

    2017-08-17 12:21:05.191958 7f5b788d4700  0 -- 127.0.0.1:0/65887226 >> 127.0.0.1:6805/3339248996 conn(0x1004be8a0 :-1 s=STATE_CONNECTING_WAIT_CONNECT_REPLY_AUTH pgs=0 cs=0 l=0).handle_connect_reply connect got BADAUTHORIZER

Fixes: http://tracker.ceph.com/issues/21027
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ce1995fc63829f854f2da16c68bee09c03efa180)