common: Make spinlock delay more conventional

The accepted method of implementing a spinlock delay is the "pause"
instruction.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13423 from dzafman/wip-18533-tool

Fixes: http://tracker.ceph.com/issues/18533
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14132 from dmick/wip-arm-crc

arch: fix cmake's ARM CRC intrinsics test to handle duplicitous gcc 4.8.5

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake/modules/SIMDExt.cmake: armv8 crypto intrinsics

Test not only for -march support, but also the actual
presence of the intrinsic routines.  Not sure why, but gcc
4.8.5 passes the first but not the second.

Fixes: http://tracker.ceph.com/issues/19386
Signed-off-by: Dan Mick <dan.mick@redhat.com>

filestore, tools: Fix logging of DBObjectMap check() repairs

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Simplify DBObjectMap by no longer creating complete tables

Bump the version for new maps to 3
Make clone less efficient but simpler
Add rename operation (use instead of clone/unlink)
For now keep code that understands version 2 maps

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-osdomap-tool: Fix seg fault with large amount of check error output

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Add automatic repair for DBObjectMap bug

Add repair command to ceph-osdomap-tool too

Under some situations the previous rm_keys() code would
generated a corrupt complete table.  There is no way to
figure out what the table should look like now.  By removing
the entries we fix the corruption and aren't much worse off
because the corruption caused some deleted keys to re-appear.

This doesn't breaking the parent/child relationship during
repair because some of the keys may still be contained
in the parent.

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-osdomap-tool: Fix tool exit status

Signed-off-by: David Zafman <dzafman@redhat.com>

DBObjectMap: rewrite rm_keys and merge_new_complete

Leverage the updated in_complete_region and needs_parent to simplify
these methods.

Signed-off-by: Samuel Just <sjust@redhat.com>

DBObjectMap: strengthen in_complete_region post condition

Previously, in_complete_region didn't guarantee anything about
where it left complete_iter pointing.  It will be handy for
complete_iter to be pointing at the lowest interval which ends
after to_test.  Make it so.

Signed-off-by: Samuel Just <sjust@redhat.com>

DBObjectMap: fix next_parent()

The previous implementation assumed that
lower_bound(parent_iter->key()) always leaves the iterator
on_parent().  There isn't any guarantee, however, that that
key isn't present on the child as well.

Signed-off-by: Samuel Just <sjust@redhat.com>

test_object_map: add tests to trigger some bugs related to 18533

Signed-off-by: Samuel Just <sjust@redhat.com>

test: Add ceph_test_object_map to make check tests

Signed-off-by: David Zafman <dzafman@redhat.com>

ceph-osdomap-tool: Add --debug and only show internal logging if enabled

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: DBOjectMap::check: Dump complete mapping when inconsistency found

Signed-off-by: David Zafman <dzafman@redhat.com>

test_object_map: Use ASSERT_EQ() for check() so failure doesn't stop testing

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Check for overlaps in internal "complete" table for DBObjectMap

Changed check to return an error count and fix tool error message

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Add dump-headers command to ceph-osdomap-tool

Signed-off-by: David Zafman <dzafman@redhat.com>

tools: Add --oid option to ceph-osdomap-tool

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Remove unnecessary assert and assignment in DBObjectMap

Fix and add comment(s)

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #13754 from liewegas/wip-objectstore-random

os: allow 'osd objectstore = random' to pick either filestore or bluestore

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: remove unuse perfcounter:l_bluestore_read_wait_flush_lat.

Now read don't need flush(). See commit 83e33a32fde7f83.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14088 from Adirl/rdma-cm-2

msg/async/rdma: Move resource handling to Device

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13915 from jcsp/wip-19253

qa: whitelist new fullness messages in fs tests

Merge pull request #14159 from tchaikov/wip-rbd-size_t-is-not-uint64_t-on-arm32

librbd: pass an uint64_t to clip_io() as the third param

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: pass an uint64_t to clip_io() as the 3rd param

because, on arm32, size_t is "unsigned", while uint64_t is "long long
unsigned int", compiler is not able to promote the former to the latter
automatically. and ImageRequest::Extents::value_type::second_type is
indeed uint64_t, which is expected by clip_io(). so let's just use
"auto" here.

Fixes: http://tracker.ceph.com/issues/18938
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10412 from stiopaa1/mon_monitorh_addConstToGetLeader

mon/Monitor.h: add const to member function

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14146 from liupan1111/wip-fix-comment

rbd-nbd: clean up the doc and help information

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12893 from rzarzynski/wip-rgw-auth-rework-cont-2

rgw: continuation of the auth rework

mon/Monitor.h: add const to member functions

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #10121 from theanalyst/wip-16357

rgw: cls_user don't clobber existing bucket stats when creating bucket
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rbd-nbd: polish the doc and help information.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

Merge pull request #14134 from wangzhengyong/doc

doc: add some undocumented options to rbd-nbd

Reviewed-by: Pan Liu <liupan1111@gmail.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14091 from trociny/wip-prepare_async_request

librbd: potential use of uninitialised value in ImageWatcher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13740 from Adirl/forksafe

msg/async/rdma: Add fork safe on RDMA

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13965 from liewegas/wip-bluestore-pc

os/bluestore: fix perf counters

Merge pull request #13962 from Liuchang0812/wip-add-override-in-osd-headers

osd: add override in headers files

Reviewed-by: Sage Weil <sage@redhat.com>

doc: add some undocumented options to rbd-nbd

Signed-off-by: wangzhengyong@cmss.chinamobile.com

osd: add override in headers files

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #14114 from dmick/wip-boost-j

debian/rules, ceph.spec.in: invoke cmake with -DBOOST_J

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake/modules/SIMDExt.cmake: add whitespace

readability, man, readability

Signed-off-by: Dan Mick <dan.mick@redhat.com>

rgw: implement the dynamic reconfiguration of auth strategies.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: the S3's local v2 auth engine becomes a fallback conditionally.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement SwiftAnonymousEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: parametrize the implicit tenancy of rgw::auth::RemoteApplier.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: RGWPostObj_ObjStore_S3 doesn't instantiate auth strategy for each request anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: fix appending '\0' in the rgw::auth::LocalApplier::to_str().

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #13889 from liewegas/wip-denc-nullptr

include/denc: remove nullptr runtime magic boundedness check

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14096 from baiyanchun/remove_useless_parameter

common: remove useless parameter

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Pan Liu <liupan1111@gmail.com>

Merge pull request #14131 from liewegas/wip-crush-encode

crush: only encode class info if SERVER_LUMINOUS

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #13960 from wangzhengyong/kstore

os/kstore: some error handling

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13973 from shinobu-x/wp-sk-primarylogpg-null-nullptr

osd/PrimaryLogPG: nullptr not NULL

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13995 from liuhongtong/wip-config

common/config: set rocksdb_cache_size to OPT_U64

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14013 from ShiqiCooperation/newshiqi

test/unittest_bluefs: check whether add_block_device success

Reviewed-by: Sage Weil <sage@redhat.com>

crush: only encode class info if SERVER_LUMINOUS

This fixes OSDMap reencode crc mismatches on jewel to
luminous upgrades.

Fixes: http://tracker.ceph.com/issues/19361
Signed-off-by: Sage Weil <sage@redhat.com>

ceph.spec.in: derive _smp_ncpus and use it for -DBOOST_J

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph.spec.in: move lowmem_build setting of _smp_mflags

Signed-off-by: Dan Mick <dan.mick@redhat.com>

debian/rules: invoke cmake with -DBOOST_J

Allow boost build during toplevel cmake from Debian package build
to benefit from multiple processors.  Should speed build a lot
on many-proc machines (say, arm64).  Use argument passed to
debhelper.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

rgw: improve logs in the RGWAccessControlPolicy class.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: improve logs in rgw::auth::Strategy.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: RGWOp is responsible now for the authentication process.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: refactor error handling in rgw::auth::keystone.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement descriptive authentication failure reasons.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: unify handling S3's ::authorize_v2 and ::get_policy methods.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: remove the parts of auth framework that aren't necessary anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: move ACL Strategies to the newer auth framework.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: switch from req_state::auth_identity to req_state::auth::identity.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: remove rgw/rgw_auth_decoimpl.h as it isn't necessary anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: add rgw::auth::s3::AWSv2AuthStrategy and switch the S3 auth to it.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: replace RGWDummyIdentityApplier with rgw::auth::DummyIdentityApplier.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: apply the new filters over appliers in the Swift's auth strategy.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement the rgw::auth::SysReqApplier over IdentityApplier.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement rgw::auth::ThirdPartyAccountApplier.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: add Control::FALLBACK mode to rgw::auth::AuthStrategy.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: aggregate S3's external auth engine as the ExternalAuthStrategy.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: make rgw::keystone::TokenEnvelope::expired() to be const-correct.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: switch S3-specific auth engines to the new infra.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: integrate the new EC2Engine with current S3 auth code.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: introduce basement for all S3 auth engines.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: fix indentation in RGWPostObj_ObjStore_S3::get_policy().

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: improve const-correctness and refactor S3 canonized string crafting.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: add rgw::auth::keystone::EC2Engine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: the authorize() method of Swift's handler operates on auth strategy now.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: the Swift's auth strategy is instantiated and injected from rgw_main.cc.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: settle rgw::auth::{IdentityApplier,Completer} in req_state.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: port the TempURL auth engine to the new auth infrastructure.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: port Swift's Signed and External auth engines.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement rgw::auth::DecoratedApplier.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement rgw::auth::swift::DefaultStrategy.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement rgw::auth::AnonymousEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement rgw::auth::LocalApplier.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: inject token cache as a dependency in rgw::auth::keystone::TokenEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: implement the rgw::auth::KeystoneEngine.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: Keystone implementation can support multiple instances now.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: only move Keystone-related things into rgw::keystone namespace.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: add the rgw::auth::TokenExtractor interface.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: introduce the rgw::auth::RemoteApplier interface.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #14082 from idealguo/update-bucket-acl

rgw: enable to update acl of bucket created in slave zonegroup

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14043 from zhangsw/fix-rgw-deletebucket

rgw: delete non-empty buckets in slave zonegroup works not well

Reviewed-by: Casey Bodley <cbodley@redhat.com>