common: fix option "rados_mon_op_timeout" comment spell error

Signed-off-by: Leo Zhang <nguzcf@gmail.com>

Merge pull request #14885 from rzarzynski/wip-rgw-auth-rework-cont-3-awsv4

rgw: continuation of the auth rework -- AWSv4

Merge pull request #14344 from rzarzynski/wip-rgw-repect-negative-refacls

rgw: respect Swift's negative, HTTP referer-based ACL grants.

Merge pull request #15586 from adamemerson/wip-ex-ML

rgw: Remove spurious XML header for GetBucketPolicy

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rgw: Remove spurious XML header for GetBucketPolicy

Fixes: http://tracker.ceph.com/issues/20247
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #15564 from Jing-Scott/fix-formpost-auth

rgw: fix the signature mismatch of FormPost in swift API

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #15581 from neha-ojha/wip-fix-objclass

objclass-sdk: use namespace ceph for bufferlist

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15595 from ztczll/zl

os/bluestore/BlueFS: .slow should be compared with dirname

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15590 from Linbing/wip-osd-check-osdmap-twice

osd: the condition of last epoch <= superblock.newest_map epoch has been check twice

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15661 from tchaikov/wip-qa-upgrade-set-sortbitwise

qa/suites/upgrade: set "sortbitwise" for jewel clusters

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15613 from cbodley/wip-20244

rgw: only log metadata on metadata master zone

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #15280 from ZVampirEM77/wip-bucket-policy-cleanup

rgw: fix get bucket policy s3 compatible issue

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #15663 from zealoussnow/wip-fix-build

cmake: exclude *.css while generating ctags

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15655 from guihecheng/rgw_file-fix-mkdir

rgw_file: prevent conflict of mkdir between restarts

Merge pull request #15659 from lixiaoy1/fio_ceph

test/fio: Fix assert in set_cache_shards in bluestore fio

Fix assert in set_cache_shards in bluestore fio

The config osd_op_num_shards is changed so it needs to change in
fio_ceph_objectstore.cc.

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

Merge pull request #15593 from chenfangxian/rbd-nbd-fix-unmap

rbd-nbd: ensure unmap returns error code

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Pan Liu <wanjun.lp@alibaba-inc.com>

build: add css file to excludes while generate tags

Signed-off-by: Leo Zhang <nguzcf@gmail.com>

rgw: fix the signature mismatch of FormPost in swift API

Fixes: http://tracker.ceph.com/issues/20220
Signed-off-by: Jing Wenjun <jingwenjun@cmss.chinamobile.com>

qa/suites/upgrade: remove duplicated upgrade task

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/upgrade: set "sortbitwise" for jewel clusters

so ceph.healthy or wait-for-healthy won't be blocked by this warning.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw_file: prevent conflict of mkdir between restarts

Fixes: http://tracker.ceph.com/issues/20275
Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>

Merge pull request #15648 from joscollin/wip-CID1412575-incorrect-expression

mon: Incorrect expression in PGMap::get_health()

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15638 from liewegas/wip-19948

ceph_test_rados_api_*: wait for snap trim on ENOENT during cleanup

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

rbd-nbd: fix nbd do_unmap error handling

when nbd disconnect failed, ensure do_unmap return error logic.

Signed-off-by: chenfangxian <chenfangxian@cmss.chinamobile.com>
Signed-off-by: guojiannan <guojiannan@cmss.chinamobile.com>

Merge pull request #15519 from liewegas/wip-19595

osd/PrimaryLogPG: clear oi from trim_object()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mon: Incorrect expression in PGMap::get_health()

Fixes:

CID 1412575:  Incorrect expression  (COPY_PASTE_ERROR)
ceph/src/mon/PGMap.cc: 2764 in PGMap::get_health()
"num_err" in "ss2 << num_err" looks like a copy-paste error.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #15629 from Linbing/wip-msg-async-op-threads

msg/async: add assert of ms_async_op_threads > 0

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #15426 from zhangsw/fix-rgw-gc-loglevel

rgw: lower some log's level in gc process.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15571 from ceph/wip-fix-fio-shard-hint

test/fio: fix lack of setting for Sequencer::shard_hint

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15603 from idryomov/wip-rados-df-typo

rados: fix typo in 'df' column name

Merge pull request #15615 from joscollin/wip-CID717210-Uninitialized-members

common: initialize _hash in LogEntryKey()

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15627 from theanalyst/12.1.0/release-notes-2

doc: 12.1.0/release notes 2

ceph_test_rados_api_*: wait for snap trim on ENOENT during cleanup

Assume that an ENOENT removing an object during cleanup means the
head is a whiteout because there are clones.  When that happens
just sleep for a bit and then retry in the hopes that snap
trimming has happened.

Time out (and fail) after retrying for 10 minutes' worth of tries.

Fixes: http://tracker.ceph.com/issues/19948
Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: clear oi from trim_object()

trim_object() is unique in that its resulting OpContxtUPtr is fed to
simple_opc_submit() but does not first pass through finish_ctx().  Almost
none of what finish_ctx is helpful for us because we are the only place
where we delete a clone, and for that reason it is not terribly useful to
put what we do into finish_ctx.  However, *one* thing that finish_ctx()
does do that we don't (didn't) is clear out the object_info_t io on
delete.  This would leave things like version with a value that would
bleed into future instantiations of the object and break things like
the prior_version in pg_log_entry_t's and, much later, break asserts
when loading the missing set off of disk.

Fix by clearing oi.

Fixes: http://tracker.ceph.com/issues/19947
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15604 from liewegas/wip-rest-test-qa

qa/suites/rados/rest: test restful mgr module

Reviewed-by: Boris Ranto <branto@redhat.com>

qa/suites/rados/rest/mgr-restful: test on centos

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15584 from liewegas/wip-20219

mon/MgrStatMonitor: keep mgrstat version ahead of pgmon

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15620 from tchaikov/wip-20249

test/vstart_wrapper.sh: display_log on test failure

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

qa/workunits/rest/test_mgr_rest_api.py: tolerate old distros

Work with older requests and/or missing urllib3.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/rest/mgr-restful: test restful module

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15634 from liewegas/wip-pyformatter

mgr/PyFormatter: implement dump_format_va

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15580 from liewegas/wip-bluestore-memory

os/bluestore: configure rocksdb cache via bluestore_cache_kv_ratio

Reviewed-by: Mark Nelson <mnelson@redhat.com>

mgr/PyFormatter: implement dump_format_va

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15624 from Adirl/wip

msg/async/rdma: check if fin message completed

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: check if fin message completed

Change-Id: I9cb1e523be7b21127503a9252b09331a37bea204
Signed-off-by: Alexander Mikheev <alexm@mellanox.com>
Signed-off-by: Adir Lev <adirl@mellanox.com>

mon/PGMonitor: assert PGMonitor stops proposing after luminous

One hacky exception here for the rebuild-mondb case.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MgrStatMonitor: keep mgrstat version ahead of pgmon

Pre-luminous, statfs and getpoolstats come from pgmon, and clients will
see and track the paxos version for that.  After the switchover, those
move to mgrstatmon.  If a clients sends a version ahead of the current
mgrstat version, it will block until the paxos version catches up, which
may be years for an aged cluster.

Fix by keeping mgrstat version ahead of pgmon's every time it updates.
This only really matters right at the transition point (we don't update
before that, and after that pgmon doesn't update).

Fixes: http://tracker.ceph.com/issues/20219
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14753 from theanalyst/v12.0.2/mailmap

doc: mailmap for v12.0.2

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #15328 from jimifm/wip-RGWPutObj

rgw/rgw_op:Prevents memory leaks when calling func swift_versioning_copy() fails

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #15323 from fangyuxiangGL/compress_bug_fix

rgw: get wrong content when download object with specific range with compression

Reviewed-by: Adam Kupczyk <akupczyk@mirantis.com>
Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

msg/async: add assert of ms_async_op_threads > 0

Signed-off-by: linbing <linbing@t2cloud.net>

qa: timeout if flush_pg_stats() takes too long

a "timeout" which defaults to 300 seconds is added to flush_pg_stats()

Signed-off-by: Kefu Chai <kchai@redhat.com>

mailmap: Yunfei Guan affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Andrey Parfenov affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Yao Zongyou affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Patrick Dinnen affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Henrik Korkuc affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Sven Seeberg affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Mohamad Gebai affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Fabian Grünbichler affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Vedant Nanda affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Andrew Solomon affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Yu Shengzuo affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Yanchun Bai affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Tone Zhang affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Liu Hong affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Liu Hong affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Li Hongjie name normalization

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Josh Durgin name normalization

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Jesse Williamson name normalization

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Hongtong Liu name normalization

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: He Chuang affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mailmap: Chu Hua-Rong name normalization

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

doc: release notes: notable change for 12.1.0

Adding the changelog from 12.0.3 to 9f259cb923e7474229bb2e5e54b59f3b7f068bc5

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

doc: v12.1.0 major changes for release

Adding a section for major changes in 12.1.0 release, sorted by sections
so that we can have a consolidated announce when announcing the release

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

qa: remove duplicated flush_pg_stats() helper

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/vstart_wrapper.sh: display_log on test failure

Signed-off-by: Kefu Chai <kchai@redhat.com>

common: initialize _hash in LogEntryKey()

Fixed:

** CID 717210:  Uninitialized members  (UNINIT_CTOR)
ceph/src/common/LogEntry.h: 70 in LogEntryKey::LogEntryKey()()
Non-static class member "_hash" is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge branch 'ceph' into zl

rgw: only log metadata on metadata master zone

Fixes: http://tracker.ceph.com/issues/20244
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15610 from wjwithagen/wip-wjw-freebsd-py-pecan

core: Install Pecan for FreeBSD

Merge pull request #15574 from dillaman/wip-20223

pybind/rbd: OSError should be picklable

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15576 from dachary/wip-ceph-disk

ceph-disk: sensible default for block.db

Reviewed-by: Sage Weil <sage@redhat.com>

core: Install Pecan for FreeBSD

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #15609 from stiwari/wip-19879

Merge pull request from stiwari/wip-19879

Merge pull request #12851 from zhangsw/fix-rgw-bucketcheck-bug

rgw: bucket index check in radosgw-admin removes valid index.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #15606 from stiwari/fix_1

Documentation Fixes for http://tracker.ceph.com/issues/19879

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15588 from liewegas/wip-20216

pybind/mgr/dashboard: monkeypatch os.exit to stop cherrypy from taking down mgr

Reviewed-by: John Spray <jspray@redhat.com>

Merge pull request #15550 from jcsp/wip-doc-cephfs-objsize

doc/cephfs: mention RADOS object size limit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ceph-disk: sensible default for block.db

If bluestore_block_db_size is unset or zero, fallback to a sensible
default which is block_size / 100. With a minimum of 1GB.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #13395 from dang/wip-quota-cache-doc

Document RGW quota cache options

Merge pull request #15568 from guihecheng/rgw_file-cleanup-names

rgw: rgw_file cleanup names

Merge pull request #15541 from guihecheng/rgw_file-cleanup

rgw_file: replace raw fs->fh_lru.unref with predefined fs->unref

rados: fix typo in 'df' column name

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #15558 from Linbing/wip-osd-optimize-send-failues

osd: when osd in not in failure_pending, we don't need to get osd inst from osdmap.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15223 from badone/wip-ceph_test_admin_socket_output

tests: Add integration tests for admin socket output

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15573 from liewegas/wip-20208

mon/PGMap: fix osd_epoch update when removing osd_stat

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Loic Dachary <loic@dachary.org>

os/bluestore: set bluestore_cache_kv_ratio to configure rocksdb cache size

Devote 40% to kv (rocksdb), 50% to metadata (onodes etc), 10% to data.

Note that if we don't consume the data portion (e.g., no cache hints) that
the onode metadata will "borrow" that space.

Signed-off-by: Sage Weil <sage@redhat.com>

kv/RocksDBStore: rocksdb_cache_row_ratio

Specify portion of cache to devote to row cache (the rest goes to the
block cache).

Signed-off-by: Sage Weil <sage@redhat.com>