ceph-volume lvm.create rollback osd creation when prepare or activate fails

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 07be6fe0ab7981ebdc34779a8ccd6220d640a549)

ceph-volume lvm.create fallback to bluestore when objectstore is not specified

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6781d6498e01bfc89189c235540e5c2a2e6bc8ca)

ceph-volume lvm.prepare fallback to bluestore when objectstore is not specified

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9b17ad2f8fe617d219fdb2f46dd2675669f32f08)

ceph-volume lvm.prepare rollback osd when prepare fails and an osd id was generated

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 579d12806b2ffc693a4e70ba3eaf663fa281d35e)

ceph-volume lvm.common create a rollback_osd utility to cleanup failed osd prepare/create calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e533792e0dbafc160238c0f5783f4430e2705571)

Merge pull request #19483 from smithfarm/wip-22375-luminous

luminous: build/ops: luminous build fails with --without-radosgw

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20211 from dillaman/wip-22198-luminous

luminous: librbd: compare and write against a clone can result in failure

Reviewed-by: Mykola Golub <mgolub@suse.com>

test/librbd: disable compare and write if skipping partial discard

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4e8b0b60a72fe1e1ac10a2beb46b77d4b58c91c4)

librbd: refactor object write request state machines

Fixes: http://tracker.ceph.com/issues/20789
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0e643fb926f0484bb4f860740c7f85b692de6737)

librbd: consolidate all object discard-related logic into single state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 318797f59adcedcc386e01f1975011e0086434ac)

librbd: simplify interface between object IO and copyup state machines

The initial copyup was not receiving a write hint and the
code for hints was duplicated multiple times. Additionally,
the object map state should match the last executed write op.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7117aa4ecaf0b619b527b2783fa1d09b11f3dd55)

Conflicts:
src/librbd/io/ObjectRequest.h: trivial resolution

librbd: copyup state machine needs to handle empty write ops

The compare-and-write object operation cannot be executed
concurrently within a copyup operation since the object might not
exist yet (if not performing a deep-copy).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f6db9b8027b6978b4d28fcf9f0389c88f9109e75)

librbd: object cacher should re-use read state machine

This adds support for sparse-reads and ensures all object reads
utilize a single, tested code path.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e79c0cf2009c69681ed4c70f52ade2f5fd570567)

librbd: refactor io::ObjectReadRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 62c3282b875f7345d7b71228d1e35e0cc87d44fa)

librbd: default template parameter for C_SparseReadRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ab0e558961ed9b63c00ad1d09ee0f9f79fd62295)

librbd: reduce lock scope when assembling read results

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 284feb6a32795c64b1c5ec3b3bf0d959e72dd940)

librbd: ObjectReadRequest does not require buffer extents

The ReadResult wrapper requires the buffer extents to know
how to properly reconstruct the out buffer.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 35ce1c2d39302011adbf494ada6248495681a436)

librbd: templatize IO object request state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 654a78fa54520370f5466f9d282d4e9f8365ad48)

test/librados_test_stub: mocked remainder of IO ops used by librbd

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c4e5613e4a74316ba9667654e887a835b240609a)

test/librados_test_stub: simulate set_alloc_hint creating a non-existent file

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4b851ce0fa534ca043aca835057b9d68660971a4)

test/librados_test_stub: pass snap context to zero op

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 78ca0fde22076e8b156feb8dfd60aaee316a1895)

Merge pull request #20209 from dillaman/wip-22033-luminous

luminous: mgr/dashboard: added iSCSI IOPS/throughput metrics

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20210 from dillaman/wip-22169-luminous

luminous: librbd: set deleted parent pointer to null

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20208 from dillaman/wip-21920-luminous

luminous: librbd: default to sparse-reads for any IO operation over 64K

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20207 from dillaman/wip-21793-luminous

luminous: rbd-mirror: primary image should register in remote, non-primary image's journal

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20206 from dillaman/wip-21694-luminous

luminous: librbd: journal should ignore -EILSEQ errors from compare-and-write

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20205 from dillaman/wip-22577-luminous

luminous: qa/workunits/rbd: simplify split-brain test to avoid potential race

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20153 from pdvian/wip-22809-luminous

luminous: librbd: fix snap create/rm may taking long time

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20135 from dillaman/wip-22806-luminous

luminous: librbd: force removal of a snapshot cannot ignore dependent children

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #19123 from pdvian/wip-22174-luminous

luminous: possible deadlock in various maintenance operations

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17875 from smithfarm/wip-21359-luminous

luminous: cephfs: racy is_mounted() checks in libcephfs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20121 from batrick/i21252p2

luminous: mds: fix return value of MDCache::dump_cache

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20213 from dillaman/wip-21868-luminous

luminous: doc/rbd: tweaks for the LIO iSCSI gateway

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc/rbd: tweaks to the Windows iSCSI initiator directions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 997bb80ba231c4a21237485ac6780d573be992c2)

doc: Added CHAP cofiguration instructions for iSCSI

Added details to specify the CHAP username and password while
discovering/login the iSCSI target.

Signed-off-by: Ashish Singh <assingh@redhat.com>
(cherry picked from commit 2a5755e0767108aefa6ccfb8a85e5d63e8c04fe8)

doc: fix ceph-iscsi-config version number

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 3ab71020a5e420a333cb4a6b0be67ade185c1593)

doc: add ceph iscsi test kernel info

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 5c30318660eebc882326d87eec39b4fbe4cc85d7)

doc: add rbd pool requirement for gwcli

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit b56d994f3addcd9c6a7848dec9be93568cdb2d0e)

doc: add daemon-reload for rbd-target-api setup

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 4ca0d508bb9b68c791abc5799d0195f4baace455)

doc: add gwcli manual install instructions

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit f45aafed1c7a5393e712f9fbfd7479fc444ad3d8)

doc: separate gwcli install from post install setup steps

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit f80f083d0a982d7e89c77e3a6f8d71c2871a8f2c)

doc: add note to clarify trusted_ip_list use

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 3cc367dc6ebca18e884d7c75d2cd1fa7907b7139)

doc: update esx instructions

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 3806f9750d29c6d930a7c8a003902e79b99766f6)

Conflicts:
doc/rbd/iscsi-initiator-esx.rst: trivial resolution

doc: add warning about SCSI PGRs/reservations

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 62661a2e23ce98432f3c037fa9a8155a633ee182)

doc: add esx web based client images

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 78e12135eeb5829cad65cebd3c2be9df4d582b1f)

doc: remove windows based vsphere client images

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 426b562fdc1b373b72de2d69c4851be22ec12008)

doc: fix gwcli examples

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 480f6a82a2b9dcd0179951ea16432d8d32f2c96d)

doc: Update ceph iscsi kernel/distro requirements.

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit f436e993d359dfdad26b7d569d555a3b3a5b798d)

doc/rbd: tweaks for the LIO iSCSI gateway

Fixes: http://tracker.ceph.com/issues/21763
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4f227e108d40271d5d6b2b1e88659d86cbc6c914)

mgr/dashboard: added iSCSI IOPS/throughput metrics

Fixes: http://tracker.ceph.com/issues/21391
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e55ef24c9140c23c832a2205a2154e68452545a3)

mgr/dashboard: include A/O start relative timestamp for iSCSI

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e62f186613a1de64a155a852b21ec9033c6e73ea)

librbd: track image open and lock acquire time via perf counter

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8583270fd2d0930c1fccd2285378de0844b1d928)

librbd: export read and writes performance counters for daemons

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7a9d10a4e61ed168678f38f6a224859e0df44099)

librbd: moved performance counter enums to common location

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2224e5ca1dacef1e3f8ec4854f08b4abe86f5d75)

Conflicts:
src/librbd/internal.h: trivial resolution

librbd: set deleted parent pointer to null

Fixes: http://tracker.ceph.com/issues/22158
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7b53256bb825383fc604a96d71bcd51c36668cc4)

librbd: track the child of an image in-memory

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ce2ae1d7c0f8818c86cbc746921e8eb6b0f89913)

rbd-mirror: journal debug message should not be logged as error

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 625222edbce66c98b86fc87fa3bd12ea40db17a2)

rbd-mirror: avoid attempting to decode an empty client data buffer

Fixes: http://tracker.ceph.com/issues/21961
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1e1d15eb0c14a6be82a6d75ab0088a6f0f4f8125)

librbd: default to sparse-reads for any IO operation over 64K

Testing BlueStore against both HDDs and OSDs with fully allocated
and sparse-allocated objects shows a performance improvement with
sparse-read between 32K and 64K.

Fixes: http://tracker.ceph.com/issues/21849
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 251658471eab6d8cd968d678922bab437f72a9c7)

librbd: templatize io::CopyupRequest and io::ObjectRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit dd5452004047c31e0f3eb813d5dd0b50bd68e76f)

rbd-mirror: removed duplicate client registration logic

The client will now be registered before the bootstrap state machine
is invoked.

Fixes: http://tracker.ceph.com/issues/21561
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d774d4702fbb9ff0b88797392b6673964d6b8c7c)

rbd-mirror: local primary images should still register to remote images

Fixes: http://tracker.ceph.com/issues/21561
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d74b95dc10a5073ee7980583d94628ded7850ebb)

librbd: journal should ignore -EILSEQ errors from compare-and-write

Fixes: http://tracker.ceph.com/issues/21628
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c7a9b045808ba741f6aadd2c2e65bd84135347c4)

qa/workunits/rbd: simplify split-brain test to avoid potential race

Fixes: http://tracker.ceph.com/issues/22485
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8c2ee6dd0f4c77b9fbbc24a673841b04d3e175d6)

Conflicts:
qa/workunits/rbd/rbd_mirror.sh: trivial resolution

Merge pull request #19555 from shinobu-x/wip-22455-luminous

 luminous: balancer crush-compat sends "foo" command

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20156 from pdvian/wip-22811-luminous

luminous: mon: fix mgr using auth_client_required policy

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19481 from jan--f/wip-22421-luminous

luminous: mon: reenable timer to send digest when paxos is temporarily inactive

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19553 from shinobu-x/wip-22453-luminous

 luminous: mgr/balancer/upmap_max_iterations must be cast to integer

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19624 from shinobu-x/wip-22496-luminous

 luminous: KeyError: ('name',) in balancer rm

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19834 from pdvian/wip-22585-luminous

luminous: pybind/mgr/prometheus: fix metric type undef -> untyped

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20084 from pdvian/wip-22760-luminous

luminous: mgr: prometheus: added osd commit/apply latency metrics (#22718).

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20089 from pdvian/wip-22768-luminous

luminous: mgr: disconnect unregistered service daemon when report received

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20182 from jcsp/wip-22831

luminous: pybind/mgr/dashboard: fix reverse proxy support

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #17869 from smithfarm/wip-21479-luminous

luminous: mgr: Services reported with blank hostname

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #18832 from shinobu-x/wip-22075-luminous

luminous: mgr tests don't indicate failure if exception thrown from serve()

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #18842 from shinobu-x/wip-21863-luminous

luminous: ceph-conf: dump parsed config in plain text or as json

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19002 from tchaikov/wip-pr-18976-luminous

luminous: udev: Fix typo in udev OSD rules file

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19023 from pdvian/wip-22164-luminous

luminous: cluster [ERR] Unhandled exception from module 'balancer' while running on mgr.x: 'NoneType' object has no attribute 'iteritems'" in cluster log

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19031 from pdvian/wip-22167-luminous

luminous: Various odd clog messages for mons

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19039 from jan--f/wip-22199-luminous

luminous: tools/crushtool: skip device id if no name exists

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19046 from shinobu-x/wip-22189-luminous

luminous: osdc/Objecter: objecter op_send_bytes perf counter always 0

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19059 from pdvian/wip-22176-luminous

luminous: osd: subscribe osdmaps if any pending pgs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19083 from smithfarm/wip-22213-luminous

luminous: On pg repair the primary is not favored as was intended

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #19084 from shinobu-x/wip-22216-luminous

 luminous: "osd status" command exception if OSD not in pgmap stats

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19118 from tangwenjun3/wip-backport-19030

luminous: os/bluestore: fix the allocate in bluefs

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Shinobu Kinjo <skinjo@redhat.com>

Merge pull request #19133 from smithfarm/wip-22194-luminous

luminous: Default kernel.pid_max is easily exceeded during recovery on high OSD-count system

Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #19631 from shinobu-x/wip-22509-luminous

 luminous: osd: "sudo cp /var/lib/ceph/osd/ceph-0/fsid ..." fails

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19839 from tchaikov/wip-22558-luminous

luminous: rpm: adjust ceph-{osdomap,kvstore,monstore}-tool feature move

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #18838 from shinobu-x/wip-21973-luminous

luminous: [test] UpdateFeatures RPC message should be included in test_notify.py

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18840 from shinobu-x/wip-21970-luminous

luminous: [journal] tags are not being expired if no other clients are registered

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18949 from jcsp/wip-luminous-rgw-config-desc

luminous: options.cc: document rgw config options

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19051 from shinobu-x/wip-22185-luminous

luminous: rbd: abort in listing mapped nbd devices when running in a container

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

pybind/mgr/dashboard: fix reverse proxy support

This fixes http redirection for reverse http proxies

Fixes: http://tracker.ceph.com/issues/22557
Signed-off-by: Nick Erdmann <n@nirf.de>
(cherry picked from commit 95e1963cb5327f0699081c4c4b0b355d109ff0e3)

pybind/mgr: fix typo

Signed-off-by: Nick Erdmann <n@nirf.de>
(cherry picked from commit fdd64bb2dac33085b47c737032a525d89319c866)

pybind/mgr/mgr_module: add default param for MgrStandbyModule.get_config()

this matches its counterpart of MgrModule.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit bc7de8b124c32a2ee6910a3bac6a06f1c566a2ac)

Merge pull request #20062 from cbodley/wip-luminous-rgw-bilog-trim

luminous: rgw multisite: automated trimming for bucket index logs

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

librbd: added missing locks for snap rollback and rename

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0ccd26f9989b892be2ab620bcf7f4fbf14980e8e)

librbd: possible deadlock with synchronous maintenance operations

Fixes: http://tracker.ceph.com/issues/22120
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 90b7ecd8a8aac9a5c282d44004752ade0c195331)

Merge pull request #19848 from cbodley/wip-luminous-boost-166

luminous: backport beast frontend and boost 1.66 update

Merge pull request #19500 from shinobu-x/wip-22401-luminous

luminous: rgw: make HTTP dechunking compatible with Amazon S3

Reviewed-by: Casey Bodley <cbodley@redhat.com>