common/LogClient: make last_log non-atomic

Signed-off-by: Sage Weil <sage@redhat.com>

common/LogClient: fix indentation

Signed-off-by: Sage Weil <sage@redhat.com>

common/LogClient: assign seq and queue atomically

The _get_mon_log_message() assumes that log_last and log_queue
are in sync, but it was previously possible to increment log_last
setting e.seq in do_log(), and only later queue it.  If a racing
thread ran get_mon_log_message() in the meantime it would fail
an assertion.

Fix by assigning the seq and queueing it atomically.  If the
cluster log is not enabled, use the get_next_seq() helper so that
graylog or syslog messages still have a seq assigned.

Fixes: http://tracker.ceph.com/issues/18209
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16861 from wido/docs-mgr-zabbix-enable

doc: Update mgr doc on how to enable Zabbix module

Reviewed-By: John Spray <jcsp@redhat.com>

Merge pull request #16908 from dillaman/wip-20701

doc: update rbd-mirroring documentation

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16910 from liewegas/wip-iana

doc/dev/iana: document our official IANA numbers

Merge pull request #16911 from liewegas/wip-lead

doc/dev/index: update rados lead

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #16879 from smithfarm/wip-release-notes-11-2-1

doc: release-notes.rst: add Kraken v11.2.1 and update releases.rst

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #16894 from liupan1111/wip-remove-friendly

bluestore: drop unused friend class in SharedDriverQueueData

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #16907 from theanalyst/doc/rgw/s3-api-tagging

doc: rgw mention about tagging & bucket policies in s3api

Reviewed-By: Adam Emerson <aemerson@redhat.com>
Reviewed-By: Matt Benjamin <mbenjami@redhat.com>

doc/dev/index: update rados lead

Signed-off-by: Sage Weil <sage@redhat.com>

doc/dev/iana: document our official IANA numbers

PEN 50495
port 3300

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: indicate that rbd-mirror should use unique IDs

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16782 from wjwithagen/wip-wjw-FreeBSD-Release

PendingReleaseNotes:Announce FreeBSD availability

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-By: Jason Dillaman <jdillaman@redhat.com>

Merge pull request #16734 from theanalyst/rgw-lc-awsv4

rgw_lc: support for AWSv4 authentication

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc: updated rbd-mirror daemon instructions

Fixes: http://tracker.ceph.com/issues/20701
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16821 from theanalyst/rgw-lc-log-drop-oldlc

rgw_op: Drop the Old LifecycleConfiguration from logs

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

doc: re-ordered rbd table of contents

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16854 from yaozongyou/fix-rgw-not-responding

rgw: fix not responding when receiving SIGHUP signal.

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

doc: rgw mention about tagging & bucket policies in s3api

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge PR #16378 into master

* refs/remotes/upstream/pull/16378/head:
doc: remove accidental additions to release notes
qa/cephfs: Fix race in test_volume_client
qa/cephfs: Test filtered df
PendingReleaseNotes: add note about df filtering
client: Support new, filtered MStatfs
objecter: Support new, filtered MStatfs
mon/PGMap stats: Support new, filtered MStatfs
messages: Add optional data pool to MStatfs

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

doc: remove accidental additions to release notes

Presumably this was caused by a bad rebase.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #16877 from dillaman/wip-20918

test/librbd: fix race condition with OSD map refresh

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

PendingReleaseNotes:Announce FreeBSD availability

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #16882 from dillaman/wip-20941

librbd: default localize parent reads to false

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15339 from trociny/wip-rbd-ggate

rbd-ggate: tool to map images on FreeBSD via GEOM Gate

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

bluestore/NVMEDevice: remove unused friendly class declaration.

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>

Merge pull request #16846 from amitkumar50/cov-messages-4

messages: Initialize member variables

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #16793 from amitkumar50/cov-717286

messages: initialize variable tid in MMDSFindIno

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #16890 from Songweibin/wip-librados-Wreorder

libradosstriper: silence warning from -Wreorder

Reviewed-by: Jos Collin <jcollin@redhat.com>

test: add wrapper to run rbd-ggate test on FreeBSD

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd-ggate: tool to map images on FreeBSD via GEOM Gate

rbd-ggate spawns a process responsible for the creation of ggate
device and forwarding I/O requests between the GEOM Gate kernel
subsystem and RADOS.

On FreeBSD it provides functionality similar to rbd-nbd on Linux.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

libradosstriper: silence warning from -Wreorder

Signed-off-by: songweibin <song.weibin@zte.com.cn>

Merge pull request #16881 from liewegas/wip-kraken-x

qa/suites/upgrade/kraken-x: fixes

qa/suites/upgrade/kraken-x/stress-split*: whitelist

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/upgrade/kraken-x/parallel: whitelist

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16823 from adamemerson/wip-use-after-free

rgw: Fix use after free in IAM policy parser

Merge pull request #16866 from amitkumar50/cov-librbd

librbd: initialization of state member variables

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16867 from amitkumar50/cov-librbd-1

librbd: Initializing member variables

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: Fix the last policy use-after-free

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Fix another use after free

This one was caused by iterator invalidation in set operations. In
this case just replace the set entirely with a bitfield.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Fix use after free in IAM policy parser

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

librbd: default localize parent reads to false

Fixes: http://tracker.ceph.com/issues/20941
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16848 from amitkumar50/cov-messages-5

messages: Initialization of uninitialized members various classes

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #16876 into master

* refs/remotes/upstream/pull/16876/head:
doc: add new fs auth command to release notes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16871 from liewegas/wip-20920

mon: fix commands advertised during mon cluster upgrade

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

doc: add new fs auth command to release notes

Introduced by #16761.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

doc: update releases.rst

Add links to v12.1.2, v12.1.1 and v11.2.1. Also fix v12.1.1 heading in
release-notes.rst.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: release-notes.rst: add Kraken v11.2.1

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/suites/upgrade/jewel-x/parallel: fix POOL_APP_NOT_ENABLED disable

This code runs on the mgr.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonCommands: mark 'pg force_create_pg' deprecated

It's deprecated.

Also, this avoids a dup when we have an upgrading mon cluster
and it's also in PGMonitorCommands.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: include PGMonitor commands with mixed-version mons

While we have a mixed version cluster, we have to advertise our
PGMonitor commands to our peons or else commands like 'pg dump'
won't work.

Once the mon feature flag is set, we can drop that because each
mon will include the mgr commands (either those stored in paxos
or the statically compiled ones until that point).

Signed-off-by: Sage Weil <sage@redhat.com>

mon: use vector<MonCommand> throughput for commands

The old code was pretty messy. This is standardizes on std::vector
throughout.  We also drop the win_election command args because
when we win an election we always set the leader commands to our
commands, and we can do that inside win_command() without passing
them in from here.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16642 from dillaman/wip-rbd-mirror-image-ids

rbd-mirror: simplify notifications for image assignment

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

test/librbd: fix race condition with OSD map refresh

Fixes: http://tracker.ceph.com/issues/20918
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: Initializing member variables

Fixes the coverity issues:

** 1351727 Uninitialized scalar field
CID 1351727 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member m_state is not initialized in
this constructor nor in any functions that it calls.

** 1351728 Uninitialized scalar field
CID 1351728 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member m_state is not initialized in
this constructor nor in any functions that it calls.

** 1351733 Uninitialized scalar field
CID 1351733 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member snap_id is not initialized in
this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com

Merge pull request #16799 from liewegas/wip-smithi-have-4

qa/clusters/fixed-[23]: 4 osds per node, not 3

Vasu Kulkarni <vasu@redhat.com>

qa/clusters/fixed-[23]: 4 osds per node, not 3

Smithi have 4 nvme partitions available for use.

Signed-off-by: Sage Weil <sage@redhat.com>

librbd: Initialization of state inside module

Fixes the coverity issues:

** 1351715 Uninitialized scalar field
5. uninit_member: Non-static class member m_state is not initialized in this
constructor nor in any functions that it calls.
7. uninit_member: Non-static class member m_copyup_start is not initialized
in this constructor nor in any functions that it calls.
CID 1351715 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
9. uninit_member: Non-static class member m_copyup_end is not initialized
in this constructor nor in any functions that it calls.

** 1351724 Uninitialized scalar field
CID 1351724 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member m_state is not initialized in
this constructor nor in any functions that it calls.

** 1351725 Uninitialized scalar field
CID 1351725 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member m_state is not initialized in
this constructor nor in any functions that it calls.

** 1351726 Uninitialized scalar field
CID 1351726 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
3. uninit_member: Non-static class member m_state is not initialized in
this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com

messages: Member initialization for class MMDSFindIno

Fixes the coverity Issue:

** 717286 Uninitialized scalar field
CID 717286 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member tid is not initialized
 in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com

Merge pull request #16873 from liewegas/wip-4-nodes

qa/suites: change fixed-2.yaml users to get 4 openstack disks

Reviewed-by: Zack Cerza <zcerza@redhat.com>

Merge pull request #16864 from smithfarm/wip-big-openstack

qa: big: add openstack.yaml

qa/suites: change fixed-2.yaml users to get 4 openstack disks

Follow-up for 4203c4f88785d8149235dd34d37f87e471084d71

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16842 from liewegas/wip-more-ec-map-discon

qa/suites/rados/thrash: fix thrashing with ec vs map discon

Merge pull request #16693 from alex-mikheev/wip_rdma_rxpool_1

msg/async/rdma: improves RX buffer management

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #16869 from liewegas/wip-crush-ids

crush/CrushWrapper: fix out of bounds access

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16845 from amitkumar50/cov-librados-aio

test/librados: Initialize member variables in aio.cc

Reviewed-by: Jos Collin <jcollin@redhat.com>

test/librados: Initializing left members from aio.cc

Fixes the coverity Issue:

** 1322815 Uninitialized pointer field
CID 1322815 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
2. uninit_member: Non-static class member m_sem is not initialized
in this constructor nor in any functions that it calls.

** 1322816 Uninitialized pointer field
CID 1322816 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
2. uninit_member: Non-static class member m_sem is not initialized
in this constructor nor in any functions that it calls.

** 1322817 Uninitialized pointer field
CID 1322817 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
2. uninit_member: Non-static class member m_sem is not initialized
in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com

rbd-mirror: clean up logging during test shut down

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: avoid silently failing to register image replayer admin sockets

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: restore deletion propagation and image replayer cleanup

The previous intermediate commits removed handling for deletion
propagation and image replayer cleanup since this logic has been
moved from instance to image replayer. Note that eventually the
policy's release notification will be responsible for the cleanup
of image replayers.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: ensure threads are mockable for testing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: moved wait for deletion logic within image replayer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: simplified acquire/release notification messages

The other instances should already know about the possible peers, so
we can co-opt these messages for use by the policy mapper.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: co-opt image acquire/release notifications

This notifications will now be used by the policy when a (re)map is
required.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: new 'peer image removed' notification to instance watchers

Note: this commit intentionally breaks backwards compatibility with
Luminous RC instance watcher notifications.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: simplify resync handling within image replayer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: pre-register image id before creating image

This fixes a potential race condition that could occur previously
if rbd-mirror daemon failed between creating an image and recording
the image id to the remote journal.

Fixes: http://tracker.ceph.com/issues/15764
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: clean up bootstrap recovery from missing local image

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: image replayer should query remote mirror uuid / image id

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: helper state machine for querying remote peer uuid / image id

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: helper state machine for retrieving image id from global id

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16393 from PCzhangPC/fix3

rbd: 'image-meta remove' for missing key does not return error

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

crush/CrushWrapper: fix out of bounds access

Fixes: http://tracker.ceph.com/issues/20926
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16851 from xiexingguo/wip-doc-update-class-cli

doc/release-notes: update device class cli

Merge pull request #16551 from fangyuxiangGL/data-notifier-interval

rgw: add a separate configuration for data notify interval

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16726 from theanalyst/rgw-crypt-debug-log

rgw_crypt: log error messages during failures

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@mirantis.com>

Merge pull request #16770 from tianshan/fix_create_bucket

rgw: should not restrict location_constraint same when user not provide

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

qa: big: add openstack.yaml

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #16862 from scienceluo/wip-luo-check-fix-branch

common/cmdparse.cc: remove unused variable 'argnum' in dump_cmd_to_json()

Reviewed-by: Jos Collin <jcollin@redhat.com>

Common: remove variable 'argnum' that is modified but its new value is never used.

Signed-off-by: Luo Kexue <luo.kexue@zte.com.cn>

doc: Update mgr doc on how to enable Zabbix module

Signed-off-by: Wido den Hollander <wido@42on.com>

Fix rgw not responding occasionally when receiving SIGHUP signal.

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>

doc/release-notes: update device class cli

These commands are nearly stable and are unlikely to change for now...

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16847 from amitkumar50/cov-1054870

rgw: Initialize of member variable admin_specified in RGWUserAdminOpState

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #16822 from liewegas/wip-20849

os/bluestore: _do_remove: dirty shard individually as each blob is unshared

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16844 from xiexingguo/wip-bluestore-core

os/bluestore: default journal media to store media if bluefs is disabled

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14053 from jdurgin/wip-19191

osd/PG: reset the missing set when restarting backfill

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16837 from xiexingguo/wip-still-more-class-fixes

crush: more class fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16838 from xiexingguo/wip-fix-purge

mon/OSDMonitor: sanity check osd before performing 'osd purge'

Reviewed-by: Sage Weil <sage@redhat.com>