doc: add release note for libcephsqlite

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

doc: remove aggressive scrubbing from pacific release notes

This was reverted in pacific by 673b5ea4ae988096d7165f7ff166f6c556345386

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

doc: Add Pacific release notes

Signed-off-by: David Galloway <dgallowa@redhat.com>

Merge pull request #39776 from alimaredia/wip-rgw-log-req-latency

rgw: add latency to the request summary of an op

Merge PR #40366 into master

* refs/pull/40366/head:
mgr/cephadm: Fix dashboard gateway configuration when using IPV6

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #40395 from rhcs-dashboard/fix-49952-master

mgr/dashboard:Simplify some complex calculations in test_alerts.yml

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #40400 from tchaikov/wip-ctest-jenkins

run-make-check.sh: let ctest generate XML output

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #40327 from racpatel/wip-cephfs-top-refresh-interval

cephfs-top: allow configurable stats refresh interval

Merge pull request #40210 from joscollin/wip-B49736-missing-client-metadata

cephfs-top: be resilient to missing client metadata keys

run-make-check.sh: let ctest generate XML output

to enable XUnit plugin of jenkins to consume the ctest output and
publish it in the dashboard, we need to

* let ctest generate XML output instead of plain text output
* do not fail the test if any test case fails. this allows the publisher
  to do its job by checking the XML output.
* prevent ctest from compressing the output. see
  https://issues.jenkins.io/browse/JENKINS-21737

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard:Simplify some complex calculations in test_alerts.yml

run-promtool-unittests is failing with difference in floating point values in some complex calculations. This PR intends to simplify those calculations and fix this issue.

Fixes: https://tracker.ceph.com/issues/49952
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #40377 from tchaikov/wip-doc-fixes

doc/man: fix formatting

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #38044 from dvanders/dvanders_buffered

common/options: bluefs_buffered_io=true by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40380 from adamemerson/wip-num-rados-handles-die-die-die

Die, `num_rados_handles`, die.

Reviewed-by: Ali Maredia <amaredia@redhat.com>

rgw: Remove spurious `num_rados_handles` option

While any code that used the option is gone, it appears the option
itself was accidentally resurrected in various options.cc refactors
and transitions.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

doc/install/windows-troubleshooting: fix typo

and tweak the format a little bit

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/man/8/ceph-volume: fix formatting

Signed-off-by: Kefu Chai <kchai@redhat.com>

admin/build-doc: build man page in proper directory

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #40350 into master

* refs/pull/40350/head:
doc/cephadm: explicitly show host requirments in adding host section

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #40235 from tchaikov/wip-cmake-test-fixture

cmake: use fixture for preparing venv

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/cephadm: Fix dashboard gateway configuration when using IPV6

Fixes: https://tracker.ceph.com/issues/49957
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

cmake: use fixture for preparing venv

this change should allow us to decouple "ninja tests" from "ctest".
in other words, we can just run

ctest -R run-tox-python-common -V

without running "ninja tests" first. before this change

${name}-venv is added as a dependency of "tests" target.

after this change,

setup-venv-for-${name} is added as a test, which is in turn a test of
run-tox-${name}, so we can just

ctest -R run-tox-${name}

now for preparing the venv and then testing the tox test of ${name}.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40344 from zdover23/wip-doc-cephadm-index-rewrite-2021-Mar-23

doc/cephadm: rewrite "index.rst"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #40353 into master

* refs/pull/40353/head:
cephsqlite: add julian day offset in milliseconds

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #40291 from tchaikov/wip-debian-cleanups

debian: cleanups and exclude ceph_crypto from dh_shlibdep

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

tools/cephfs: fix missing keys in cephfs-top client_metadata

Fixes: https://tracker.ceph.com/issues/49736
Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #39683 from maydaycc/wip-49487

osd: fix scrub reschedule bug

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

debian/rules: add with_system_libs option

so the maintainer can choose to use the packages shipped by distro.

Signed-off-by: Kefu Chai <kchai@redhat.com>

debian/control: remove cython from Build-Depends

as cython3 is enough. and we've dropped the python2 support.

Signed-off-by: Kefu Chai <kchai@redhat.com>

debian/rules: exclude ceph_crypto from dh_shlibdep

as libceph_crypto* are plugins, and they are not self-contained. they
reference symbols offered by the executable loading them. dh_shlibdep
should not complain when checking them, so add them to the exclude list.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40346 from zdover23/wip-doc-cephadm-osd-service-2021-Mar-23-first

doc/cephadm: rewrite osd.rst - list devices

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/cephadm: rewrite "index.rst"

This PR rewrites the three paragraphs at the
front of the cephadm guide, increasing their
elegance and removing ambiguities.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #40357 from ybwang0211/quota_doc

doc: remove duplicated 'quota' parameter in URL

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: remove duplicated 'quota' parameter in URL

Only one "quota" parameter is needed in the URL to indicate that it is a quota request.

Signed-off-by: wangyingbin <ybwang0211@163.com>

cephfs-top: signal main thread to exit on SIGINT/SIGTERM

Fixes: http://tracker.ceph.com/issues/49953
Signed-off-by: Rachana Patel <racpatel@redhat.com>

cephfs-top: allow configurable stats refresh interval

Signed-off-by: Rachana Patel <racpatel@redhat.com>

Merge PR #40314 into master

* refs/pull/40314/head:
qa/tasks/cephadm: use 'orch apply mon' to deploy mons

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>

Merge PR #40324 into master

* refs/pull/40324/head:
mgr/cephadm/upgrade: do not crash if error races with user cancellation

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Adam King <adking@redhat.com>

Merge PR #40305 into master

* refs/pull/40305/head:
doc/cephfs/nfs: Add note about cephadm NFS-Ganesha daemon port

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

cephsqlite: add julian day offset in milliseconds

This magic number was copied from another VFS but was not adjusted for
the xCurrentTimeInt64 interface.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40320 into master

* refs/pull/40320/head:
mgr/orchestrator: move PORTS column in 'orch ps' output

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #40322 into master

* refs/pull/40322/head:
cephadm: only bootstrap using image that matches cephadm version

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #40216 from rhcs-dashboard/fix-role-cloning

mgr/dashboard: Fix for broken User management role cloning

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #36402 from rhcs-dashboard/wip-45965-users-quota-usage

mgr/dashboard: Display users current quota usage

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

doc/cephadm: explicitly show host requirments in adding host section

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>

doc/cephadm: rewrite osd.rst - list devices

This PR rewrites the "List Devices" section of
the OSD chapter of the Cephadm guide. This PR
is a simple grammar-and-elegance improvement.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #40208 from CongMinYin/fix-flush-block-bug

librbd/cache/pwl: fix bug of flush request blocked by defered IO

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #40005 from rhcs-dashboard/49655-handle-no-rgw-daemons-running

mgr/dashboard: fix error shown when no rgw daemons are running.

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: fix error notification shown when no rgw daemons are running.

- Adapted code to changes introduced in: https://github.com/ceph/ceph/pull/40220
- Improved error handling.
- Increased test coverage.
- Some refactoring.
- Simplified documentation about setting default daemon host and port.

Fixes: https://tracker.ceph.com/issues/49655
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #40300 from tchaikov/test-promtool

ceph.spec,install-deps: use golang-github-prometheus for promtools

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #40318 from batrick/libcephsqite-label

.github/labeler: add libcephsqlite label

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #40192 into master

* refs/pull/40192/head:
test: reduce number of threads to 32 in LibCephFS.ShutdownRace

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40174 into master

* refs/pull/40174/head:
qa/vstart_runner: fix the admin_socket args parsing

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #40158 into master

* refs/pull/40158/head:
test: add test for removing non-existent xattr
mds: fix removexattr logic when there aren't any

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #40146 into master

* refs/pull/40146/head:
test: pass peer uuid when adding cephfs mirror peers
mon: check cephfs mirror peer based on remote cluster spec and file system name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40145 into master

* refs/pull/40145/head:
doc: add note about disabling standby-replay during upgrades
qa: add test for standby-replay disable
mon: fail standby-replay daemons when flag is turned off

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>

Merge PR #38443 into master

* refs/pull/38443/head:
qa: set "shell" to False for run_ceph_w()
vstart_runner: make "shell" a default argument

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #39777 from xxhdx1985126/wip-alienstore-thread-pin

crimson/os/alienstore: scatter alienstore threads onto specified cpu cores

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #39435 into master

* refs/pull/39435/head:
mgr/cephadm: redeploy daemons deployed using old image during upgrade
mgr/cephadm: add container digests of mgr that deployed daemon to unit.meta

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #40125 from ceph/wip-14218-notes

doc: 14.2.18 Release Notes

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/cephadm/upgrade: do not crash if error races with user cancellation

If the user cancels the upgrade just before the upgrade thread runs into
a problem (and these things may be correlated!), ignore the failure
instead of crashing the module.

Signed-off-by: Sage Weil <sage@newdream.net>

cephadm: only bootstrap using image that matches cephadm version

Only allow bootstrap to deploy if the cephadm version matches the
ceph version in the container.  Allow the master branch version of cephadm
to deploy the latest stable version as well (at least for now).

Provide a flag to force bootstrap to continue despite the check.

Move the _pull_image call up into bootstrap so that it is easier to see
when it happens.

Fixes: https://tracker.ceph.com/issues/49884
Signed-off-by: Sage Weil <sage@newdream.net>

qa/tasks/cephadm: use 'orch apply mon' to deploy mons

The 'orch daemon add ...' command is not idempotent and can cause
duplicate (and failing) attempts to add the same mon.

Maintain the ability to add mons the old way for the benefit of testing
upgrades from early octopus versions of cephadm.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge PR #40203 into master

* refs/pull/40203/head:
mgr/cephadm: retry after JSONDecodeError in wait_for_mgr_restart()

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/orchestrator: move PORTS column in 'orch ps' output

Put it next to HOST.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge PR #40290 into master

* refs/pull/40290/head:
qa/suites/rados/cephadm/upgrade: wait for rgw servicemap entries to refresh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #40288 into master

* refs/pull/40288/head:
mon/PGMap: change hard coded 0x10000 to macro CEPH_OSD_IN

Reviewed-by: Kefu Chai <kchai@redhat.com>

.github/labeler: add libcephsqlite label

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #39191 into master

* refs/pull/39191/head:
pybind/mgr/snap_schedule: use ceph VFS
pybind/mgr/snap_schedule: idempotentize table creation
mgr: add ceph sqlite VFS
doc: add libcephsqlite
ceph.spec,debian: package libcephsqlite
test/libcephsqlite,qa: add tests for libcephsqlite
libcephsqlite: rework architecture and backend
SimpleRADOSStriper: wait for finished aios after write
SimpleRADOSStriper: add new minimal async striper
mon: define simple-rados-client-with-blocklist profile
librados: define must renew lock flag
common: add timeval conversion for durations
Revert "libradosstriper: add function to read into char*"
test_libcephsqlite: test random inserts
cephsqlite: fix compiler errors
cmake: improve build inst for cephsqlite
libcephsqlite: sqlite interface to RADOS
libradosstriper: add function to read into char*

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #40201 from ceph/wip-tracker-48697

ceph-volume: show devices with GPT headers as not available

Merge pull request #40295 from tchaikov/wip-doc-links

doc: fix broken links

Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge PR #40245 into master

* refs/pull/40245/head:
mgr/cephadm: identify iscsi service by the pool

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>

Merge pull request #40293 from tchaikov/wip-cmake-cleanup

cmake: define BOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT globaly

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #40301 into master

* refs/pull/40301/head:
qa/distros/podman: install containernetworking-plugins along with podman

Reviewed-by: Sage Weil <sage@redhat.com>

doc/cephfs/nfs: Add note about cephadm NFS-Ganesha daemon port

Fixes: https://tracker.ceph.com/issues/49921
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #40282 from rzarzynski/wip-crimson-doc-waitstates

doc/crimson: document wait states

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/crimson: document wait states.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #40198 from jdurgin/wip-quick-fix-on-mount

common/options: turn off bluestore_fsck_quick_fix_on_mount by default

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40239 from tchaikov/wip-osd-silence-warning

osd/PeeringState: remove unused variable

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40143 from neha-ojha/wip-objectstore-debug

qa/suites/rados/objectstore/backends: use default level for filestore

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #40243 from liewegas/fix-pg-health-messages

mon/PGMap: nicely format pg acting in health messages

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40238 from ygtzf/wip-blob-info

src/os: Remove redundant hexadecimal symbols from the log output

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40112 from lxbsz/lockdep2

BackTrace: reduce the array max number to 32

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39412 from tchaikov/wip-rocksdb

bluestore: use string_view and bump rocksdb to v6.15.5

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #40033 from tchaikov/wip-47380

mon/OSDMonitor: drop stale failure_info after a grace period

Reviewed-by: Sage Weil <sage@redhat.com>

qa/distros/podman: install containernetworking-plugins along with podman

/etc/cni/net.d/87-podman-bridge.conflist tries to load "bridge",
"firewall", "tuning" and "portmap" plugins, which are provided by
containernetworking-plugins package.

Fixes: https://tracker.ceph.com/issues/49909
Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph.spec,install-deps: use golang-github-prometheus for promtools

instead of installing docker for using promtools, install
golang-github-prometheus.

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Fix for broken User management role cloning

Cloning a role in user management gets hit with a 415 error.

Fixes: https://tracker.ceph.com/issues/49880
Signed-off-by: Nizamudeen A <nia@redhat.com>

doc/cephfs: fix broken links

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: use :ref: instead relative path for links

path is fragile when we move the file around, but link is more stable.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/man: replace http://ceph.com/docs with https://docs.ceph.com

the former brings us to a 404 page

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/cephadm/upgrade: wait for rgw servicemap entries to refresh

rgw changed the way it registered in the service map.  Wait a bit for
the old entries to be flushed out.

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm: retry after JSONDecodeError in wait_for_mgr_restart()

'ceph mgr dump' does not always return valid JSON so cephadm
will throw an exception sometimes when applying a spec as per
the issue this PR closes. Add a try/except to catch a possible
JSONDecodeError and retry after sleeping.

Fixes: https://tracker.ceph.com/issues/49870
Signed-off-by: John Fulton <fulton@redhat.com>

cmake: define BOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT globaly

turns out we also need it for compiling librados tests with libboost
1.75, so just define it globally

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: use list(APPEND..) when appropriate

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40102 from tchaikov/wip-doc-fixes

doc: theme, cmake and formatting related fixes

Reviewed-by: Zac Dover <zac.dover@gmail.com>

crimson/os/alienstore: add default behaviour for alien threads affinities

currently, we allow alienstore to be scheduled on to any cpu cores other than
the starting three, as in most current tests we use the those cores for crimson-osd
seastar threads

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

mon/PGMap: change hard coded 0x10000 to macro CEPH_OSD_IN

0x10000 is same with CEPH_OSD_IN for osd weight, so change them to
that macro, more meaningful.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>

Merge pull request #40272 from tchaikov/wip-install-dep-remove-existing-boost

install-deps.sh: remove existing ceph-libboost of different version

Reviewed-by: David Galloway <dgallowa@redhat.com>

Merge PR #40147 into master

* refs/pull/40147/head:
python-common: Validate characters in service_id for container names

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>