osdc/Objecter: unify disparate EAGAIN handling paths into one

We had two different code paths for EAGAIN that both
seemed to resend the same request again (with a new
tid), but implemneted it in two different ways. Further,
the first one was conditioned on the BALANCE_READS
flag but not LOCALIZED_READS, which should (I think) be
equivalent in this context.

Unify them into a single path.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16384 from mikulely/2-checkout-placmenet-target-existness

rgw: check placement target existence during bucket creation

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16389 from prallabh/wip-20661

rgw: Do not decrement stats cache when the cache values are zero

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #16463 from cbodley/wip-20708

rgw: fix parse/eval of policy conditions with IfExists

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #16495 from weiqiaomiao/wip_weiqm_rgw_crash

rgw: fix radosgw will crash when service is restarted during lifecycl…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #16531 from rzarzynski/wip-rgw-20757

rgw: fix memory leaks during Swift Static Website's error handling.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16601 from rzarzynski/wip-rgw-20774

rgw: never let http_redirect_code of RGWRedirectInfo to stay uninitialized

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #16604 from rzarzynski/wip-rgw-20779

rgw: fix the UTF8 check on bucket entry name in rgw_log_op().

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16575 from liewegas/wip-20693

qa/suites/rados: at-end: ignore PG_{AVAILABILITY,DEGRADED}

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16398 from dillaman/wip-20655

rbd-mirror: guard the deletion of non-primary images

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #16574 from batrick/rgw-rbd-stripped-build

CMakeLists: trim rbd/rgw forced dependencies

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16603 from dillaman/wip-rbd-mirror-valgrind

rbd-mirror: initialize timer context pointer to null

Merge pull request #16613 from liewegas/wip-vstart-mgr

vstart.sh: start mgr after mon, before osds

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #16584 from gregsfortytwo/wip-deletes-coverity

messages: default-initialize MOSDPGRecoveryDelete[Reply] members

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

vstart.sh: start mgr after mon, before osds

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16610 from dzafman/wip-fix-reg11184

test: reg11184 might not always find pg 2.0 prior to import

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16530 from xiexingguo/wip-fix-pgtemp

mon: prime pg_temp and a few health warning fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16587 from dzafman/wip-20770

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16600 from dingdangzhang/master

bluestore/BlueFS: pass string as const ref

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16563 from hjwsm1989/readable-thread-info

common/WorkQueue: use threadpoolname + threadaddr for heartbeat_han…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16576 from liewegas/wip-20371

mon/MonClient: scale backoff interval down when we have a healthy mon session

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16559 from hjwsm1989/dump-stuck

qa/tasks/dump_stuck: fix dump_stuck test bug

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16516 from Liuchang0812/nit-close-formatter-section

mon/MDSMonitor: close object section of formatter

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16553 from wumingqiao/wip-leveldb-store-crash

kv: resolve a crash issue in ~LevelDBStore()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16578 from liewegas/wip-20771

common/common_init: disable default dout logging for UTILITY_NODOUT too

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16522 from majianpeng/bluestore-remove-unused-code

os/bluestore/BlueStore: remove unused code.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

test: reg11184 might not always find pg 2.0 prior to import

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #16571 from ceph/wip-cd-bluestore-2

qa/tasks/ceph-deploy: Fix bluestore options for ceph-deploy

Reviewed-by: Tamil Muthamizhan <tmuthami@redhat.com>

Merge pull request #16452 from wjwithagen/wjw-warning-unused

buffer: silence unused var warning on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16483 from ddiss/cmake_install_includedir

cmake: use CMAKE_INSTALL_INCLUDEDIR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15033 from rzarzynski/wip-rgw-19831

rgw: fix segfault in RevokeThread during its shutdown procedure.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16013 from xiexingguo/wip-blue-tool

os/bluestore/bluestore_tool: add sanity check to get rid of occasionally crash

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16528 from tchaikov/wip-ceph-release-notes

tools: ceph-release-notes: escape _ for unintended links

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #16404 from dzafman/wip-20041

osd: scrub_to specifies clone ver, but transaction include head write…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16579 from liewegas/wip-fix-nonregression

qa/suites/rados/singleton/all/erasure-code-nonregression: fix typo

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Amik Kumar <amitkuma@redhat.com>

qa/workunits/erasure-code/encode-decode-nonregression: do not require git checkout

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: fix the UTF8 check on bucket entry name in rgw_log_op().

Fixes: http://tracker.ceph.com/issues/20779
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #16580 from liewegas/wip-20767

pybind/mgr/zabbix: fix health in non-compat mode

Reviewed-by: Wido den Hollander <wido@widodh.nl>

Merge pull request #16573 from liewegas/wip-osd-manual

doc/install/manual-deployment: update osd creation steps

Reviewed-by: Alfredo Deza <adeza@redhat.com>

rbd-mirror: initialize timer context pointer to null

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

 bluestore/BlueFS: pass string as const ref

Signed-off-by: dingdangzhang <boqian.zy@alibaba-inc.com>

test: Fixes for test_pidfile

Add a short delay for one test
Fix test that had unreliable log output

Fixes: http://tracker.ceph.com/issues/20770
Signed-off-by: David Zafman <dzafman@redhat.com>

rgw: rgw_website.h doesn't assume inclusion of the std namespace anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: never let http_redirect_code of RGWRedirectInfo to stay uninitialized.

Fixes: http://tracker.ceph.com/issues/20774
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #14868 from wangzhengyong/librbd

librbd: add compare and write API

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: fixed compiler warnings

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: improved compare_and_write testing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: add compare_and_write test for fsx
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

test/librbd: add compare_and_write testcase
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

test/librbd: add Mock Image cache for compare_and_write
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

test/librbd: handle compare_and_write op in test_mock_Replay
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd/journal: handle compare_and_write event
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

test/librbd: handle cmpext request in LibradosTestStub
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: add compare_and_write/aio_compare_and_write API
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: add compare_and_write perfcounter
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: handle compare_and_write imagerequest in ImageRequestWQ
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: add compare_and_write ImageRequest
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

librbd: add CompareAndWrite ObjectRequest
Signed-off-by: Zhengyong Wang <wangzhengyong@cmss.chinamobile.com>

Merge pull request #16595 from wjwithagen/wip-wjw-freebsd-run-standalone

qa: make run-standalone work on FreeBSD

Reviewed-by: Sage Weil <sage@redhat.com>

mon/MonClient: do not send_log if conn is not active anymore

the log message to be sent will be appended to waiting_for_session
instead. but we will send the logs anyway when the MonClient is
authorized with the new connection. so, avoid doing this as it's not
necessary.

also refactor the schedule_tick() call into a scope_guard, so it is
always called upon the return of the tick() method.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/MonClient: un-backoff on tick when we have a authenticated session

This means that if we do some backoff, then authenticate, and are healthy
for an extended period of time, a subsequent failure won't leave us
starting our hunting sequence with a large backoff.

Fixes: http://tracker.ceph.com/issues/20371
Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: factor un-backoff into helper; add missing config option

- move this into a helper
- use the backoff factor for the division instead of hard-coding 2.0
- add a min config option instead of hard-coding 1.0

Signed-off-by: Sage Weil <sage@redhat.com>

qa: make run-standalone work on FreeBSD

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #16543 from dillaman/wip-service-daemon-c-api

librados: add missing implementations for C service daemon API methods

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16519 from liewegas/wip-20754

osd: populate last_epoch_split during build_initial_pg_history

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #16446 from xiexingguo/wip-destroyed

mon: show destroyed status in tree view; do not auto-out destroyed osds

Reviewed-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: introduce mon_osd_destroyed_out_interval for destroyed out control

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSD: abort booting if osd is marked as destroyed

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: apply new 'destroyed' status to 'osd tree' filter

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSDMap: show "destroyed" status in tree view

It would be a pain if we destroyed an osd and there is no obvious way
to find it out.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16123 from ddiss/fix-journal-supression

ceph-disk: don't activate suppressed journal devices

Reviewed-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/zabbix: fix health in non-compat mode

This was apparently written/tested with mon_health_preluminous_compat
enabled.  Fix to behave with or without that option.

Fixes: http://tracker.ceph.com/issues/20767
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16498 from Songweibin/wip-doc-rbd-trash-cli

doc: add rbd new trash cli and cleanups  in release-notes.rst

Merge pull request #16552 from tchaikov/wip-doc-fixes

doc: document mClock related options

Reviewed-by: Sage Weil <sage@redhat.com>

doc/install/manual-deployment: update osd creation steps

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16534 from liewegas/wip-20751

mon/OSDMonitor: ensure UP is not set for newly-created OSDs

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/singleton/all/erasure-code-nonregression: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>

common/common_init: disable default dout logging for UTILITY_NODOUT too

This was broken in 4d06719f101f49019be25bd6137f72f2cb85bd5a.

Fixes: http://tracker.ceph.com/issues/20771
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16469 from xiexingguo/wip-fix-test

test: s/osd_objectstore_type/osd_objectstore

messages: default-initialize MOSDPGRecoveryDelete[Reply] members

This makes Coverity happy and is good practice, even though we do
a member-by-member initialization in the only place we create these messages.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

doc: add new trash cli in rados-rbd-cmds.rst

Signed-off-by: songweibin <song.weibin@zte.com.cn>

doc: cleanups in release-notes.rst

Signed-off-by: songweibin <song.weibin@zte.com.cn>

os/bluestore/BlueStore: remove unused code.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

CMakeLists: trim rbd/rgw accidental dependencies

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa/suites/rados: at-end: ignore PG_{AVAILABILITY,DEGRADED}

With the peering deletes change, setting luminous sets the osdmap flag
which triggers a new peering interval.  That can lead to health warnings
about PG_AVAILABILITY or PG_DEGRADED.  Ignore those!

Fixes: http://tracker.ceph.com/issues/20693
Signed-off-by: Sage Weil <sage@redhat.com>

Add both filestore and bluestore options for tests

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add ceph-deploy overrides options

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #15693 from dillaman/wip-osd-cmpext-ec-pool

osd: support cmpext operation on EC-backed pools

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #16527 from jcsp/wip-options-followup

common: follow up to new options infrastructure

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16511 from c744402859/master

OSDMonitor: check mon_max_pool_pg_num when set pool pg_num

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16560 from nrdmn/patch-3

pybind/mgr/restful: fix typo

Reviewed-by: Sage Weil <sage@redhat.com>

Fixed typo in src/pybind/mgr/restful/api/__init__.py

Signed-off-by: Nick Erdmann <n@nirf.de>
Fixed typo

bluestore config options for tests

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add bluestore overrides for ceph-deploy

ceph-deploy doesn't use ceph overrides, Add same overrides for ceph-deploy

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Move ceph-deploy config options into its own folder

The old structure of link at top folder is pretty much outdated, the test
config option needs to be specific to cluster yaml.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add option to specify bluestore/filestore options

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #16536 from jdurgin/wip-20747

osd: PrimaryLogPG, PGBackend: complete callback even if interval changes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16539 from jdurgin/wip-20753

osd/PGLog: fix inaccurate missing assert

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16541 from liewegas/wip-20761

qa/workunits/cephtool/test.sh: disable 'fs status' until bug is fixed

Merge pull request #16544 from dillaman/wip-service-daemon-null-dereference

mgr/MgrClient: do not attempt to access a global variable for config

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>