test/librados_test_stub: mocked remainder of IO ops used by librbd

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c4e5613e4a74316ba9667654e887a835b240609a)

test/librados_test_stub: simulate set_alloc_hint creating a non-existent file

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4b851ce0fa534ca043aca835057b9d68660971a4)

test/librados_test_stub: pass snap context to zero op

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 78ca0fde22076e8b156feb8dfd60aaee316a1895)

Merge pull request #20209 from dillaman/wip-22033-luminous

luminous: mgr/dashboard: added iSCSI IOPS/throughput metrics

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20210 from dillaman/wip-22169-luminous

luminous: librbd: set deleted parent pointer to null

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20208 from dillaman/wip-21920-luminous

luminous: librbd: default to sparse-reads for any IO operation over 64K

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20207 from dillaman/wip-21793-luminous

luminous: rbd-mirror: primary image should register in remote, non-primary image's journal

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20206 from dillaman/wip-21694-luminous

luminous: librbd: journal should ignore -EILSEQ errors from compare-and-write

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20205 from dillaman/wip-22577-luminous

luminous: qa/workunits/rbd: simplify split-brain test to avoid potential race

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20153 from pdvian/wip-22809-luminous

luminous: librbd: fix snap create/rm may taking long time

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20135 from dillaman/wip-22806-luminous

luminous: librbd: force removal of a snapshot cannot ignore dependent children

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #19123 from pdvian/wip-22174-luminous

luminous: possible deadlock in various maintenance operations

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17875 from smithfarm/wip-21359-luminous

luminous: cephfs: racy is_mounted() checks in libcephfs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20121 from batrick/i21252p2

luminous: mds: fix return value of MDCache::dump_cache

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20213 from dillaman/wip-21868-luminous

luminous: doc/rbd: tweaks for the LIO iSCSI gateway

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc/rbd: tweaks to the Windows iSCSI initiator directions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 997bb80ba231c4a21237485ac6780d573be992c2)

doc: Added CHAP cofiguration instructions for iSCSI

Added details to specify the CHAP username and password while
discovering/login the iSCSI target.

Signed-off-by: Ashish Singh <assingh@redhat.com>
(cherry picked from commit 2a5755e0767108aefa6ccfb8a85e5d63e8c04fe8)

doc: fix ceph-iscsi-config version number

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 3ab71020a5e420a333cb4a6b0be67ade185c1593)

doc: add ceph iscsi test kernel info

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 5c30318660eebc882326d87eec39b4fbe4cc85d7)

doc: add rbd pool requirement for gwcli

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit b56d994f3addcd9c6a7848dec9be93568cdb2d0e)

doc: add daemon-reload for rbd-target-api setup

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 4ca0d508bb9b68c791abc5799d0195f4baace455)

doc: add gwcli manual install instructions

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit f45aafed1c7a5393e712f9fbfd7479fc444ad3d8)

doc: separate gwcli install from post install setup steps

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit f80f083d0a982d7e89c77e3a6f8d71c2871a8f2c)

doc: add note to clarify trusted_ip_list use

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 3cc367dc6ebca18e884d7c75d2cd1fa7907b7139)

doc: update esx instructions

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 3806f9750d29c6d930a7c8a003902e79b99766f6)

Conflicts:
doc/rbd/iscsi-initiator-esx.rst: trivial resolution

doc: add warning about SCSI PGRs/reservations

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 62661a2e23ce98432f3c037fa9a8155a633ee182)

doc: add esx web based client images

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 78e12135eeb5829cad65cebd3c2be9df4d582b1f)

doc: remove windows based vsphere client images

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 426b562fdc1b373b72de2d69c4851be22ec12008)

doc: fix gwcli examples

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 480f6a82a2b9dcd0179951ea16432d8d32f2c96d)

doc: Update ceph iscsi kernel/distro requirements.

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit f436e993d359dfdad26b7d569d555a3b3a5b798d)

doc/rbd: tweaks for the LIO iSCSI gateway

Fixes: http://tracker.ceph.com/issues/21763
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4f227e108d40271d5d6b2b1e88659d86cbc6c914)

mgr/dashboard: added iSCSI IOPS/throughput metrics

Fixes: http://tracker.ceph.com/issues/21391
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e55ef24c9140c23c832a2205a2154e68452545a3)

mgr/dashboard: include A/O start relative timestamp for iSCSI

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e62f186613a1de64a155a852b21ec9033c6e73ea)

librbd: track image open and lock acquire time via perf counter

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8583270fd2d0930c1fccd2285378de0844b1d928)

librbd: export read and writes performance counters for daemons

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7a9d10a4e61ed168678f38f6a224859e0df44099)

librbd: moved performance counter enums to common location

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2224e5ca1dacef1e3f8ec4854f08b4abe86f5d75)

Conflicts:
src/librbd/internal.h: trivial resolution

librbd: set deleted parent pointer to null

Fixes: http://tracker.ceph.com/issues/22158
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7b53256bb825383fc604a96d71bcd51c36668cc4)

librbd: track the child of an image in-memory

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ce2ae1d7c0f8818c86cbc746921e8eb6b0f89913)

rbd-mirror: journal debug message should not be logged as error

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 625222edbce66c98b86fc87fa3bd12ea40db17a2)

rbd-mirror: avoid attempting to decode an empty client data buffer

Fixes: http://tracker.ceph.com/issues/21961
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1e1d15eb0c14a6be82a6d75ab0088a6f0f4f8125)

librbd: default to sparse-reads for any IO operation over 64K

Testing BlueStore against both HDDs and OSDs with fully allocated
and sparse-allocated objects shows a performance improvement with
sparse-read between 32K and 64K.

Fixes: http://tracker.ceph.com/issues/21849
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 251658471eab6d8cd968d678922bab437f72a9c7)

librbd: templatize io::CopyupRequest and io::ObjectRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit dd5452004047c31e0f3eb813d5dd0b50bd68e76f)

rbd-mirror: removed duplicate client registration logic

The client will now be registered before the bootstrap state machine
is invoked.

Fixes: http://tracker.ceph.com/issues/21561
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d774d4702fbb9ff0b88797392b6673964d6b8c7c)

rbd-mirror: local primary images should still register to remote images

Fixes: http://tracker.ceph.com/issues/21561
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d74b95dc10a5073ee7980583d94628ded7850ebb)

librbd: journal should ignore -EILSEQ errors from compare-and-write

Fixes: http://tracker.ceph.com/issues/21628
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c7a9b045808ba741f6aadd2c2e65bd84135347c4)

qa/workunits/rbd: simplify split-brain test to avoid potential race

Fixes: http://tracker.ceph.com/issues/22485
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8c2ee6dd0f4c77b9fbbc24a673841b04d3e175d6)

Conflicts:
qa/workunits/rbd/rbd_mirror.sh: trivial resolution

Merge pull request #19555 from shinobu-x/wip-22455-luminous

 luminous: balancer crush-compat sends "foo" command

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20156 from pdvian/wip-22811-luminous

luminous: mon: fix mgr using auth_client_required policy

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19481 from jan--f/wip-22421-luminous

luminous: mon: reenable timer to send digest when paxos is temporarily inactive

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19553 from shinobu-x/wip-22453-luminous

 luminous: mgr/balancer/upmap_max_iterations must be cast to integer

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19624 from shinobu-x/wip-22496-luminous

 luminous: KeyError: ('name',) in balancer rm

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19834 from pdvian/wip-22585-luminous

luminous: pybind/mgr/prometheus: fix metric type undef -> untyped

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20084 from pdvian/wip-22760-luminous

luminous: mgr: prometheus: added osd commit/apply latency metrics (#22718).

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20089 from pdvian/wip-22768-luminous

luminous: mgr: disconnect unregistered service daemon when report received

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20182 from jcsp/wip-22831

luminous: pybind/mgr/dashboard: fix reverse proxy support

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #17869 from smithfarm/wip-21479-luminous

luminous: mgr: Services reported with blank hostname

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #18832 from shinobu-x/wip-22075-luminous

luminous: mgr tests don't indicate failure if exception thrown from serve()

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #18842 from shinobu-x/wip-21863-luminous

luminous: ceph-conf: dump parsed config in plain text or as json

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19002 from tchaikov/wip-pr-18976-luminous

luminous: udev: Fix typo in udev OSD rules file

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19023 from pdvian/wip-22164-luminous

luminous: cluster [ERR] Unhandled exception from module 'balancer' while running on mgr.x: 'NoneType' object has no attribute 'iteritems'" in cluster log

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19031 from pdvian/wip-22167-luminous

luminous: Various odd clog messages for mons

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19039 from jan--f/wip-22199-luminous

luminous: tools/crushtool: skip device id if no name exists

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #19046 from shinobu-x/wip-22189-luminous

luminous: osdc/Objecter: objecter op_send_bytes perf counter always 0

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19059 from pdvian/wip-22176-luminous

luminous: osd: subscribe osdmaps if any pending pgs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19083 from smithfarm/wip-22213-luminous

luminous: On pg repair the primary is not favored as was intended

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #19084 from shinobu-x/wip-22216-luminous

 luminous: "osd status" command exception if OSD not in pgmap stats

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19118 from tangwenjun3/wip-backport-19030

luminous: os/bluestore: fix the allocate in bluefs

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Shinobu Kinjo <skinjo@redhat.com>

Merge pull request #19133 from smithfarm/wip-22194-luminous

luminous: Default kernel.pid_max is easily exceeded during recovery on high OSD-count system

Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #19631 from shinobu-x/wip-22509-luminous

 luminous: osd: "sudo cp /var/lib/ceph/osd/ceph-0/fsid ..." fails

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19839 from tchaikov/wip-22558-luminous

luminous: rpm: adjust ceph-{osdomap,kvstore,monstore}-tool feature move

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #18838 from shinobu-x/wip-21973-luminous

luminous: [test] UpdateFeatures RPC message should be included in test_notify.py

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18840 from shinobu-x/wip-21970-luminous

luminous: [journal] tags are not being expired if no other clients are registered

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18949 from jcsp/wip-luminous-rgw-config-desc

luminous: options.cc: document rgw config options

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19051 from shinobu-x/wip-22185-luminous

luminous: rbd: abort in listing mapped nbd devices when running in a container

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

pybind/mgr/dashboard: fix reverse proxy support

This fixes http redirection for reverse http proxies

Fixes: http://tracker.ceph.com/issues/22557
Signed-off-by: Nick Erdmann <n@nirf.de>
(cherry picked from commit 95e1963cb5327f0699081c4c4b0b355d109ff0e3)

pybind/mgr: fix typo

Signed-off-by: Nick Erdmann <n@nirf.de>
(cherry picked from commit fdd64bb2dac33085b47c737032a525d89319c866)

pybind/mgr/mgr_module: add default param for MgrStandbyModule.get_config()

this matches its counterpart of MgrModule.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit bc7de8b124c32a2ee6910a3bac6a06f1c566a2ac)

Merge pull request #20062 from cbodley/wip-luminous-rgw-bilog-trim

luminous: rgw multisite: automated trimming for bucket index logs

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

librbd: added missing locks for snap rollback and rename

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0ccd26f9989b892be2ab620bcf7f4fbf14980e8e)

librbd: possible deadlock with synchronous maintenance operations

Fixes: http://tracker.ceph.com/issues/22120
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 90b7ecd8a8aac9a5c282d44004752ade0c195331)

Merge pull request #19848 from cbodley/wip-luminous-boost-166

luminous: backport beast frontend and boost 1.66 update

Merge pull request #19500 from shinobu-x/wip-22401-luminous

luminous: rgw: make HTTP dechunking compatible with Amazon S3

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17816 from smithfarm/wip-hadoop-s3a

luminous: tests: qa/rgw: hadoop-s3a suite targets centos_latest

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #18972 from linuxbox2/luminous-22084

luminous: rgw: Fix swift object expiry not deleting objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19086 from shinobu-x/wip-22210-luminous

luminous: radosgw-admin zonegroup get and zone get should return defaults when there is no realm

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19088 from pdvian/wip-22177-luminous

luminous: rgw: check going_down() when lifecycle processing

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

Merge pull request #19636 from shinobu-x/wip-22506-luminous

luminous: rgw usage trim only trims a few entries

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19784 from linuxbox2/luminous-20201

luminous: RGW: S3 POST policy should not require Content-Type

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #19799 from smithfarm/wip-22581-luminous

luminous: rgw: multisite: 'radosgw-admin sync error list' contains temporary EBUSY errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19867 from smithfarm/wip-22591-luminous

luminous: rgw: refuses upload when Content-Type missing from POST policy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20051 from ceph/revert-19538-wip-22434-luminous

Revert "luminous: rgw: user stats increased after bucket reshard"

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20056 from linuxbox2/luminous-rgw-latchfix

luminous: rgw_file: alternate fix deadlock on lru eviction

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20073 from smithfarm/wip-22767-luminous

luminous: rgw: librgw: fix shutdown error with resources uncleaned

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20075 from smithfarm/wip-22773-luminous

luminous: rgw: file deadlock on lru evicting

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #20107 from dreamhost/wip-22792

luminous: rgw: When a system object is created exclusively, do not distribute the

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr: apply auth_service_required to client conns

Previously was using auth_cluster_required for all
connections, which meant that if someone had
disabled client cephx, they'd get BADAUTHORIZER
from their CLI when it tried to load mgr
command descriptions.

Disabling cephx on the admin CLI is odd, but the mon
tolerates it so the mgr should too.

Fixes: https://tracker.ceph.com/issues/22096
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 86ee30c33a06a13e6bb360b80dd1e41ba928147b)

mon: fix mgr using auth_client_required policy

This caused mgr daemons to fail to authenticate
when auth_client_required was set to something
different to auth_cluster_required.

Fixes: https://tracker.ceph.com/issues/22096
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 1e06fe003e50d360b5c5ce9824bba044b6406ec5)

librbd: fix snap create/rm may taking long time

  fix snap create/rm may taking long time
  http://tracker.ceph.com/issues/22716

Signed-off-by: Song Shun <song.shun3@zte.com.cn>
(cherry picked from commit d04ed348a1b20e5ea5bedada2462cb41f0f1d85a)

Conflicts:
test_mock_SnapshotRemoveRequest.cc: define om as a ObjectMap<ImageCtx> obj

osd: Scrub of object with only object error(s) are not logged

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 1ad05b1068ddd5d3312af45af1a60587200ddcd7)

osd: Primary shard needs to be at the front to be preferred

Caused by: cd0d8b0714d8684cf61b4650e170027ef46f489b

Fixes: http://tracker.ceph.com/issues/21907
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b7d2f27da1a68046a688749f95093a88e6362b28)