test/store_test: adjust OnodeSizeTracking test case to measure full object metadata using empty Onode as a basis

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: remove excessive 0x in logging

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: add test cases for BlueStore blob reuse.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: try to reuse blob rather than create new one on overwrite.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: do garbage collection after main write path to be able to call set_lextent from _do_alloc_write (will be required later for blob alignment).

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: make punch_hole call from set_lextent optional.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: refactor bluestore_blob_t to encapsulate extents/*length fields.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: store blob logical length rather than recalculate it on pextent vector

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #10412 from stiopaa1/mon_monitorh_addConstToGetLeader

mon/Monitor.h: add const to member function

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14146 from liupan1111/wip-fix-comment

rbd-nbd: clean up the doc and help information

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12893 from rzarzynski/wip-rgw-auth-rework-cont-2

rgw: continuation of the auth rework

mon/Monitor.h: add const to member functions

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #10121 from theanalyst/wip-16357

rgw: cls_user don't clobber existing bucket stats when creating bucket
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rbd-nbd: polish the doc and help information.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

Merge pull request #14134 from wangzhengyong/doc

doc: add some undocumented options to rbd-nbd

Reviewed-by: Pan Liu <liupan1111@gmail.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14091 from trociny/wip-prepare_async_request

librbd: potential use of uninitialised value in ImageWatcher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13740 from Adirl/forksafe

msg/async/rdma: Add fork safe on RDMA

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13965 from liewegas/wip-bluestore-pc

os/bluestore: fix perf counters

Merge pull request #13962 from Liuchang0812/wip-add-override-in-osd-headers

osd: add override in headers files

Reviewed-by: Sage Weil <sage@redhat.com>

doc: add some undocumented options to rbd-nbd

Signed-off-by: wangzhengyong@cmss.chinamobile.com

osd: add override in headers files

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #14114 from dmick/wip-boost-j

debian/rules, ceph.spec.in: invoke cmake with -DBOOST_J

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: implement the dynamic reconfiguration of auth strategies.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: the S3's local v2 auth engine becomes a fallback conditionally.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement SwiftAnonymousEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: parametrize the implicit tenancy of rgw::auth::RemoteApplier.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: RGWPostObj_ObjStore_S3 doesn't instantiate auth strategy for each request anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: fix appending '\0' in the rgw::auth::LocalApplier::to_str().

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #13889 from liewegas/wip-denc-nullptr

include/denc: remove nullptr runtime magic boundedness check

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14096 from baiyanchun/remove_useless_parameter

common: remove useless parameter

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Pan Liu <liupan1111@gmail.com>

Merge pull request #14131 from liewegas/wip-crush-encode

crush: only encode class info if SERVER_LUMINOUS

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13960 from wangzhengyong/kstore

os/kstore: some error handling

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13973 from shinobu-x/wp-sk-primarylogpg-null-nullptr

osd/PrimaryLogPG: nullptr not NULL

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13995 from liuhongtong/wip-config

common/config: set rocksdb_cache_size to OPT_U64

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14013 from ShiqiCooperation/newshiqi

test/unittest_bluefs: check whether add_block_device success

Reviewed-by: Sage Weil <sage@redhat.com>

crush: only encode class info if SERVER_LUMINOUS

This fixes OSDMap reencode crc mismatches on jewel to
luminous upgrades.

Fixes: http://tracker.ceph.com/issues/19361
Signed-off-by: Sage Weil <sage@redhat.com>

ceph.spec.in: derive _smp_ncpus and use it for -DBOOST_J

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph.spec.in: move lowmem_build setting of _smp_mflags

Signed-off-by: Dan Mick <dan.mick@redhat.com>

debian/rules: invoke cmake with -DBOOST_J

Allow boost build during toplevel cmake from Debian package build
to benefit from multiple processors.  Should speed build a lot
on many-proc machines (say, arm64).  Use argument passed to
debhelper.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

rgw: improve logs in the RGWAccessControlPolicy class.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: improve logs in rgw::auth::Strategy.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: RGWOp is responsible now for the authentication process.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: refactor error handling in rgw::auth::keystone.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement descriptive authentication failure reasons.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: unify handling S3's ::authorize_v2 and ::get_policy methods.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: remove the parts of auth framework that aren't necessary anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: move ACL Strategies to the newer auth framework.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: switch from req_state::auth_identity to req_state::auth::identity.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: remove rgw/rgw_auth_decoimpl.h as it isn't necessary anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: add rgw::auth::s3::AWSv2AuthStrategy and switch the S3 auth to it.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: replace RGWDummyIdentityApplier with rgw::auth::DummyIdentityApplier.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: apply the new filters over appliers in the Swift's auth strategy.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement the rgw::auth::SysReqApplier over IdentityApplier.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement rgw::auth::ThirdPartyAccountApplier.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: add Control::FALLBACK mode to rgw::auth::AuthStrategy.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: aggregate S3's external auth engine as the ExternalAuthStrategy.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: make rgw::keystone::TokenEnvelope::expired() to be const-correct.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: switch S3-specific auth engines to the new infra.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: integrate the new EC2Engine with current S3 auth code.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: introduce basement for all S3 auth engines.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: fix indentation in RGWPostObj_ObjStore_S3::get_policy().

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: improve const-correctness and refactor S3 canonized string crafting.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: add rgw::auth::keystone::EC2Engine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: the authorize() method of Swift's handler operates on auth strategy now.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: the Swift's auth strategy is instantiated and injected from rgw_main.cc.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: settle rgw::auth::{IdentityApplier,Completer} in req_state.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: port the TempURL auth engine to the new auth infrastructure.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: port Swift's Signed and External auth engines.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement rgw::auth::DecoratedApplier.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement rgw::auth::swift::DefaultStrategy.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement rgw::auth::AnonymousEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement rgw::auth::LocalApplier.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: inject token cache as a dependency in rgw::auth::keystone::TokenEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement the rgw::auth::KeystoneEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: Keystone implementation can support multiple instances now.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: only move Keystone-related things into rgw::keystone namespace.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: add the rgw::auth::TokenExtractor interface.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: introduce the rgw::auth::RemoteApplier interface.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #14082 from idealguo/update-bucket-acl

rgw: enable to update acl of bucket created in slave zonegroup

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14043 from zhangsw/fix-rgw-deletebucket

rgw: delete non-empty buckets in slave zonegroup works not well

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13991 from Liuchang0812/wip-rgw-optimization

rgw: avoid listing user buckets for rgw_delete_user

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13504 from rzarzynski/wip-rgw-chunkingfilter-cleanup

rgw: clean up the unneeded rgw::io::ChunkingFilter::has_content_length.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: introduce the rgw::auth::Applier interface.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement the rgw::auth::Strategy.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: introduce and document new interfaces to the auth subsystem.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #13847 from wjwithagen/wip-wjw-ceph-disk-tests-2

ceph-disk/tests/test_main.py: FreeBSD does not do multipath

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13974 from tchaikov/wip-vstart-start-mgr

vstart: do not start mgr if not start_all

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13197 from asheplyakov/master-18740

systemd/ceph-disk: make it possible to customize timeout

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14103 from tchaikov/wip-https-github

script: ceph-release-notes: use https instead of http

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #14085 from wjwithagen/wip-wjw-bluestore-fixture

test/objectstore/store_test_fixture.cc: Exclude bluestore code if required.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13931 from wangzhengyong/extent

os/bluestore: fix bug for calc extent_avg in reshard function

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14073 from liewegas/wip-bluestore-nullptr

os/bluestore: avoid nullptr in bluestore_extent_ref_map_t::bound_encode

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13577 from yonghengdexin735/wip-zzz-openalloc

os/bluestore: fix bug in _open_alloc()

Reviewed-by: Varada Kari <varada.kari@sandisk.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14110 from dachary/wip-crush-cleanup

crush: builder: clean the arguments of crush_reweight* methods

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

vstart.sh: do not init fsmap if "$new == 0"

we cannot create a new cephfs using a non-empty pool without '--force'
option now, so the "ceph fs new" command fails with "vstart.sh -k".

Signed-off-by: Kefu Chai <kchai@redhat.com>

tests: remove mds,osd,mon args passed to vstart.sh

Signed-off-by: Kefu Chai <kchai@redhat.com>

crush: builder: clean the arguments of crush_reweight* methods

This commit is just a cleanup to make the arguments of the method
around crush_reweight all coherent.

Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>

vstart.sh: remove start_*

so there are only two ways to override the number of daemons to start
- using the env var CEPH_NUM_{MON|OSD|MGR|MDS} or {MON|OSD|MGR|MDS}
- command line options: --{mon,osd,mds}_num

do prevent a daemon from running, set the corrresponding env var to 0.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14050 from ovh/bp-dump-ops-by-duration

common/TrackedOp: allow dumping historic ops sorted by duration

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14060 from LiumxNL/wip-170321

osd: combine unstable stats with info.stats when publish stats to osd

Reviewed-by: Sage Weil <sage@redhat.com>