rgw : fix race in RGWCompleteMultipart

Fixes : http://tracker.ceph.com/issues/20861

Signed-off-by: Abhishek Varshney <abhishek.varshney@flipkart.com>

Merge pull request #16377 from theanalyst/doc/release-notes/12.1.1

doc: 12.1.1 & 12.1.2 release notes

Merge pull request #16308 from joscollin/wip-rbd-nbd-return-values

tools/rbd, rgw: Removed unreachable returns

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16370 from rzarzynski/wip-rgw-swift-prefixed-tempurls

rgw: add support for Swift's TempURLs with prefix-based scope

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16442 from theanalyst/rgw-admin-tenant-validate

rgw: validate tenant names during user create.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #16485 from rzarzynski/wip-rgw-s3anonauthengine

rgw: move the S3 anonymous auth handling to a dedicated engine.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16690 from scienceluo/wip-luo-unused-var-branch

common/rgw/rbd: remove some unused variables

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16692 from oritwas/wip-rgw-dir-suggest

rgw: Fix up to 1000 entries at a time in check_bad_index_multipart

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc: release notes for v12.1.2

Signed-off-by: Abhishek Lekshmanan <alekshmanan@suse.com>

doc: 12.1.1 release notes add major changes

Also cleanup PendingReleasenotes to an empty file so that only newer
changes are tracked, adding the relevant section back to
RC1 where relevant. Moving all the RC1 announcements back to RC2, when
we go to 12.2.0 we'll collapse all of these back to the release
announcments

Signed-off-by: Abhishek Lekshmanan <alekshmanan@suse.com>

doc: 12.1.1 release notes

Adding the changelog from 12.1.0 -> 12.1.1

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #16759 from liewegas/wip-powercycle-health

qa/suites/powercycle: whitelist health for thrashing

qa/suites/powercycle: whitelist health for thrashing

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16758 from ceph/luminous

sync luminous tag back to master

Merge pull request #16535 from liewegas/wip-osd-shard-init

osd: initialize waiting_for_pg_osdmap on startup

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #16684 from tchaikov/wip-pybind-replace-deprecated-apis

pybind/rados: use new APIs instead of deprecated ones

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16735 from dillaman/wip-20869

common: avoid statically allocating configuration options

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #16747 from tchaikov/wip-doc-fixes

doc: document bluestore compression settings

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

doc: document bluestore compression settings

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/man/8/radosgw-admin: document --compression option

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16741 from liewegas/wip-more-doc-links

doc/release-notes: fix links, formatting; add crush device class docs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16722 from tchaikov/wip-qa-fixes

qa/suites: escape the parenthesis of the whitelist text

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16474 from shun-s/wip-fix-deactive

ceph-disk: do not remove mount point if deactive --once

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites: more whitelisting

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/release-notes: clean up links

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/crush-map: more device class docs

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16700 from jcsp/wip-20803

osd/mon/mds: fix `config set` tell command

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16701 from jcsp/wip-health-format

mon: extensible output format for health checks

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16706 from jdurgin/wip-20850

osd: do not try to set device class before luminous

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16712 from gregsfortytwo/wip-20808

osd: pg: be more careful with locking around forced pg recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16688 from amitkumar50/cov-1395347-1

os: Argument cannot be negative

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

12.1.2

Merge pull request #15801 from zhangsw/feature-bucket-sync

rgw multisite: feature of bucket sync enable/disable

Reviewed-by: Casey Bodley <cbodley@redhat.com>

common: avoid statically allocating configuration options

This was resulting in seg faults within rbd-fuse during shutdown. This also
fixes the "variable tracking size limit exceeded with -fvar-tracking-assignments,
retrying without" compiler warning".

Fixes: http://tracker.ceph.com/issues/20869
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16616 from tchaikov/wip-doc-cleanup

doc: remove docs on non-existant command

rbd-fuse: remove dependency on ceph-common

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16727 from jcsp/wip-doc-config-hel

doc/qa: cover `config help` command

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16707 from ivancich/wip-more-dmclock-docs

doc: Describe mClock's use within Ceph in great detail.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16730 from liewegas/wip-more-doc-links

doc/release-notes: fix upmap and osd replacement links; add fixme

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/release-notes: fix a few doc links

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16653 from amitkumar50/Coverity-1409697

rgw: Use comparison instead of assignment

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

os: Argument cannot be negative
Fixed:

** CID 1395347 (#1 of 1): Argument cannot be negative (NEGATIVE_RETURNS)
3. negative_returns: fd is passed to a parameter that cannot be negative

Signed-off-by: Amit Kumar amitkuma@redhat.com

Merge pull request #16656 from idryomov/wip-qa-newer-fio

qa/tasks/rbd_fio: bump default fio version to 2.21

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

osd: initialize shardedwq osdmap references on startup

The _process methods assumes the shard osdmap refs are
non-null, but that leaves a window between startup and
the first call to prune_pg_waiters (via consume_map()),
which can lead to a seg fault.

Calling prune during startup, before we have any work,
will initialize those references.

Fixes: http://tracker.ceph.com/issues/20748
Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites: escape the parenthesis of the whitelist text

so we can avoid the warnings like

grep: Unmatched ( or \(

because we pass the whitelisted string to `egrep -v "$1"` directly.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/workunits/ceph-disk: fix the path to ceph-helpers-root.sh

partially reverts 841f3bd

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16652 from shangzhong/wip-update-quick-guide

doc: update the pool names created by vstart.sh by default

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16637 from ceph/wip-doc-links

doc: upmap docs; various missing links for release notes

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/release-notes: add various links to docs

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/upmap: document new upmap feature

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15838 from theanalyst/doc-pool-ns-update

doc: update the rados namespace docs

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16623 from xiexingguo/wip-weightset

crush: a couple of weight-set fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16611 from liewegas/wip-doc-health

doc/rados/operations/health-checks: osd section

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

doc/rados/operations/health-checks: add PG health check commentary

Include a link to pg-repair.rst, although there is no
content there yet.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMap: include which pgs have unfound objects in detail

Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMap: put the _toofull states under DEGRADED_FULL

I think this was an oversight?

Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMap: 'incomplete' means data is unavailable

Well, data is certainly unavailable, and may also be
degraded in the sense that we can't peer.  I think
unavailable is the more severe of the two, though, so
let's put it there!

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/operations/health-checks: osd section

First paragraph: explain what the error means.

Second or later paragraph: describe steps to fix or mitigate.

Signed-off-by: Sage Weil <sage@redhat.com>

doc: change the default filesystem/pool names
- s/cephfs_data/cephfs_data_a
- s/cephfs_metadata/cephfs_metadata_a
- s#./rados df#bin/rados df
- update the 'bin/rados df' output
- remove the rbd pool, it isn't created by default anymore.

Signed-off-by: Zhu Shangzhong <zhu.shangzhong@zte.com.cn>

qa: include `config help` in admin socket test

Signed-off-by: John Spray <john.spray@redhat.com>

doc: document `config help` command

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #16721 from joscollin/wip-config-rst-typo

doc: fix typo in config.rst

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16723 from tchaikov/wip-doc-kill-warnings

doc: various fixes

Merge pull request #16471 from jan--f/janf-mds-deactivate-doc

doc: reword mds deactivate docs; add optional fs_name argument

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #16620 from cfanz/master

common: use std::move() for better performance

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #16526 from TsaiJin/wip-fix-msg-typo

msg/async: fix the bug of inaccurate calculation of l_msgr_send_bytes

Reviewed-by: Pan Liu <wanjun.lp@alibaba-inc.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #16666 from scienceluo/wip-luo-auth-branch

auth: Remove unused function in AuthSessionHandler

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: various fixes

- radosgw/s3/bucketops.rst: fix Malformed table.
- operations/health-checks.rst: Title underline too short
- rbd/rados-rbd-cmds.rst: Title underline too short
- rados/operations/index.rst: include health-checks in toc

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fix typo in config.rst

Fixed the typo introduced by PR#16681

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #16622 from dillaman/wip-cmpext-dne

osd: cmpext operator should ignore -ENOENT on read

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd: pg: be more careful with locking around forced pg recovery

This does several little things that add up to big concurrency and safety
improvements:
* Switch to passing around PGRefs instead of raw pointers, which is
  generally a good idea
* drop the pg_map_lock once we're done looking up the PGRefs, since
  we don't need it and holding the PG pointer alive was the only previous
  thing that might have made it necessary
* don't hold the recovery_lock since we don't need any OSD-level
  synchronization
* make sure the PG is not being deleted before we do a force-change of its
  state

Fixes: http://tracker.ceph.com/issues/20808
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #16677 from tchaikov/wip-20785

osd: do not send pg_created unless luminous

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mon/OSDMonitor: make setting require-osd-release idempotent

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

osd: do not try to set device class before luminous

This avoids crashing when older monitors do not support it.

Fixes: http://tracker.ceph.com/issues/20850
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16663 from dzafman/wip-ec-enoent

osd: EC read handling: don't grab an objectstore error to use as the read error

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #16681 from shun-s/wip-doc-config

doc: fix typos in config.rst

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #16695 from tchaikov/wip-cmake-disable-vta

cmake: disable -fvar-tracking-assignments for config.cc

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #16697 from wjwithagen/wip-wjw-ENODATA-HashIndex.cc

HashIndex.cc: add compat.h for ENODATA

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #16689 from pritha-srivastava:wip-rgw-policy-parse-error

rgw: Fix for Policy Parse exception in case of multiple statements.

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #16661 from adamemerson/wip-throttle-minimal

throttle: Minimal destructor fix for Luminous

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

doc: Describe mClock's use within Ceph in great detail.

It seems many are initially unclear as to how the current
implementations of mclock op queues work, so we need to document it to
avoid confusion.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

throttle: Minimal destructor fix for Luminous

Get rid of the undefined behavior of destroying condition variables
while they're being waited on.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #16547 from dillaman/wip-dashboard-iscsi

mgr/dashboard: RBD iSCSI daemon status page

Reviewed-by: John Spray <john.spray@redhat.com>

doc: fix sevral typos

  fix sevral typos

Signed-off-by: Song Shun <song.shun3@zte.com.cn>

Merge pull request #16670 from Yan-waller/wip-walle-0729rbdcompile

cls/rbd: silence warning from -Wunused-variable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #16583 from lxbsz/ws_macro

librbd: add LIBRBD_SUPPORTS_WRITESAME support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mon: extensible output format for health checks

The summary field can be extended with the fields
used to construct the message (e.g. including
the down osd count in the message about osds
being down).

The detail entries, similarly, can be extended
with machine-readable fields like the PG ID
for a damaged PG.

For the moment all the internal stuff is just
strings still, but we change the output format
so that we don't break it later when we
add things.

Signed-off-by: John Spray <john.spray@redhat.com>

osd/mon/mds: fix `config set` tell command

This was working for setting values, but failing to call
the config observers, so some values didn't take effect.

Fixes: http://tracker.ceph.com/issues/20803
Signed-off-by: John Spray <john.spray@redhat.com>

cls/rbd: silence warning from -Wunused-variable

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

Merge pull request #16694 from Songweibin/wip-doc-rbd

rbd: modified some commands' description into imperative sentence

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

HashIndex.cc: add compat.h for ENODATA

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

mon/OSDMonitor: assert(pg in creating_pgs.pg) in send_pg_creates()

and mark OSDMonitor::send_pg_creates() a const method.

See-also: http://tracker.ceph.com/issues/20785
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16686 from amitkumar50/cov-1351525-1

rbd: fixed coverity 'Argument cannot be negative' warning

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

cmake: disable -fvar-tracking-assignments for config.cc

there are too many variables involved in config.cc, but
-fvar-tracking-assignments is enabled by default, when the compiler
finds that the number of variables exceeds the limit
"max-vartrack-size", it gives up, and tries again disabling this
option.

this change avoids the recompilation of config.cc and also
silences the warning of

src/common/config.cc:77:1: note: variable tracking size limit exceeded
with -fvar-tracking-assignments, retrying without
 md_config_t::md_config_t(bool is_daemon)
 ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

rbd: 1351525 Argument cannot be negative

Fixed:

** CID 1351525 (#1 of 1): Argument cannot be negative (NEGATIVE_RETURNS)
7. negative_returns: sd is passed to a parameter that cannot be negative.

Signed-off-by: Amit Kumar amitkuma@redhat.com

rbd: modified some commands' description into imperative sentence

Signed-off-by: songweibin <song.weibin@zte.com.cn>

rgw: Fix up to 1000 entries at a time in check_bad_index_multipart

Fixes: http://tracker.ceph.com/issues/20772
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

rgw: remove some unused variable

Signed-off-by: Luo Kexue <luo.kexue@zte.com.cn>

Merge pull request #16660 from nrdmn/patch-2

doc: added mgr caps to manual deployment documentation

Reviewed-by: John Spray <john.spray@redhat.com>

rgw: Fix for Policy Parse exception in case of multiple statements.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>