test: remove fs/test-volume.sh workunit

.. since its ported to python now.

Fixes: http://tracker.ceph.com/issues/39949
Signed-off-by: Venky Shankar <vshankar@redhat.com>

test: port fs/volume related tests to python

... and add subvolume related tests.

Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #28508 from smithfarm/wip-40295

cmake: set empty-string RPATH for ceph-osd

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #22446 into master

* refs/pull/22446/head:
relicense LGPL-2.1 code as LGPL-2.1 or LGPL-3.0
COPYING: add reference to a few Apache-2.0 licensed source files
debian/copyright: sync with COPYING
COPYING: LGPL2.1 -> LGPL-2.1
COPYING: extra bits from debian/copyright

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #28484 from tchaikov/wip-test_librados_build

qa/workunits/rados/test_librados_build.sh: install build deps

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #28486 from tchaikov/wip-remove-digest-from-buffer.h

buffer,crypto,tools: extract digest methods out of bufferlist

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Myoungwon Oh <omwmw@sk.com>

Merge pull request #28494 from ricardoasmarques/update-iscsi-dependencies-versions

doc: Update 'python-rtslib' and 'tcmu-runner' min versions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

cmake: set empty-string RPATH for ceph-osd

Fixes: http://tracker.ceph.com/issues/40295
Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/workunits/rados/test_envlibrados_for_rocksdb: support SUSE distros

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/workunits/rados/test_librados_build.sh: install build deps

to build helloworld of librados, we need to install the build
dependencies.

Fixes: https://tracker.ceph.com/issues/40155
Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/workunits/rados/test_envlibrados_for_rocksdb: use helper script

instead of having its own copy of `install()`, try to leverage the
helper in ceph-helpers-root.sh.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/workunits/ceph-helpers-root: use /etc/os-release instead

as lsb_release does not work on SUSE distros

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28465 from tchaikov/wip-xio-cleanup

include/config-h.in.cmake: remove HAVE_XIO

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28495 from tchaikov/wip-dashboard-portal_ip_addresses

mgr/dashboard: take portal_ip_addresses as a list

Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: s/portal_ip_address/portal_ip_addresses/

and take portal_ip_addresses as a list

it's a follow-up fix of #28084

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28290 from b-ranto/wip-selinux-rhel8

selinux: Update the policy for RHEL8

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27507 from pritha-srivastava/wip-rgw-cross-tenant-doc

rgw: Improving doc for Cross Project(Tenant) access with Openstack Ke…

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

doc: Update 'python-rtslib' and 'tcmu-runner' min versions

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #28084 from ricardoasmarques/improve-portal-ips-calculation

mgr/dashboard: Optimize portal IPs calculation

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #28454 from tchaikov/wip-gcc-9-on-bionic

install-deps.sh: use GCC-9 on bionic

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #28246 from kungf/mgr_default_addr

mgr: use ipv4 default when ipv6 was disabled

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28481 from iotcg/common

include: buffer_raw.h: Copyright time fix

Reviewed-by: Kefu Chai <kchai@redhat.com>

buffer,crypto,osd,tools: extract digest methods out of bufferlist

they were consumers of ceph::list::buffers(), and there is no need to
make them member functions of ceph::list, so move them into ceph::crypto
namespace. and update the unit test and ceph_dedup_tool.cc accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/ceph_crypto: add digest_size to digest classes

prepare for extracting the digest methods from ceph::buffer::list, so we
can use them in a templated implementation.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28443 from dzafman/wip-40198

osd: Don't randomize deep scrubs when noscrub set

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28431 from cbodley/wip-40188

common: OutputDataSocket retakes mutex on error path

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #28427 from tchaikov/wip-40174

mgr/influx: try to call close()

Reviewed-by: Wido den Hollander <wido@42on.com>

Merge pull request #28254 from myoungwon/wip-hashfuncs-dedup

common,osd: add hash algorithms for dedup fingerprint

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28176 from p-na/wip-pna-fix-fixmod

mgr/dashboard: Fix `npm run fixmod` command

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #26909 from vitalif/wal-sync-fix

os/bluestore: Actually wait until completion in write_sync

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28167 from dillaman/wip-39972

librados: move buffer free functions to inline namespace

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28463 from tchaikov/wip-yaml.safe_load

admin,doc/_ext/ceph_releases.py: use yaml.safe_load()

Reviewed-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #28074 from falcon78921/wip-doc-39712

doc: fixed broken links in nautilus release page

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

include: buffer_raw.h: Copyright time fix

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>

Merge pull request #28479 from yuriw/wip-yuriw-crontab-master

qa.tests: added "smoke" suite to the schedule on "mimic","nautilus"

qa.tests: added "smoke" suite to the schedule on "mimic","nautilus"

Note: frequency can/will reduce later

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #28451 from linuxbox2/wip-rgwfile-dirinv

rgw_file: all directories are virtual with respect to contents

Merge pull request #28434 from dillaman/wip-40110

rbd-mirror: link against the specified alloc library

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #28414 from smithfarm/wip-40175

build/ops: make "patch" build dependency explicit

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28388 from dillaman/wip-msg-async-1

msg/async: avoid unnecessary costly wakeups for outbound messages

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

build/ops: make "patch" build dependency explicit

This commit adds "patch" as an explicit build dependency. Before, the RPM
build was installing it in the build environment via some mechanism that I
don't entirely understand.

The "patch" command is used by the BuildBoost cmake module.

Fixes: http://tracker.ceph.com/issues/40175
Signed-off-by: Nathan Cutler <ncutler@suse.com>

include/config-h.in.cmake: remove HAVE_XIO

* remove the HAVE_XIO template
* remove README.xio

as XioMessenger was removed back in cc9a9142 .

Signed-off-by: Kefu Chai <kchai@redhat.com>

admin,doc/_ext/ceph_releases.py: use yaml.safe_load()

see
https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23312 from sylvia0409/master

doc: Fixes OSD node labels which based on the osd_devices name

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28461 from tchaikov/wip-cmake-pg-meta-set

crimson/osd/pg_meta: use initializer list for passing set<>

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd/pg_meta: use initializer list for passing set<>

we cannot assume that the data in string views are always nul
terminated.

this change partially reverts 82fedbd0089073cfe86640eaa7d73ed1e2545c31.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28397 from tchaikov/wip-38403

osdc/Objecter: use unique_ptr<OSDMap> for Objecter::osdmap

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28403 from xiexingguo/wip-unify-is-noup

osd: unify sources of no{up,down,in,out} flags into singleton helpers

Reviewed-by: Sage Weil <sage@redhat.com>

osdc/Objecter: move Objecter::Objecter() into .cc

to speed up compilation.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osdc/Objecter: use unique_ptr<OSDMap> for Objecter::osdmap

Fixes: http://tracker.ceph.com/issues/38403
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28379 from jtlayton/wip-jlayton-40114

msg: fix addr2 encoding for sockaddrs

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28368 from kamoltat/wip-progress-module-pg-osdmap-comparison

mgr/progress: Look at PG state when PG epoch >= OSDMap epoch

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28352 from smithfarm/wip-rpm-obsolete-python

rpm: have pybind RPMs provide/obsolete their python2 predecessors

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27512 from jmolmo/tls_auth

mgr/ansible: TLS Mutual Authentication

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #28171 into master

* refs/pull/28171/head:
test_volume_client: simplify test_get_authorized_ids()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28190 into master

* refs/pull/28190/head:
mds: cleanup unneeded client_snap_caps when splitting snap inode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28293 into master

* refs/pull/28293/head:
mds: avoid passing null to SessionmMap::hit_session()
qa/cephfs: add test for blacklisted client eviction
mds: fix 'is session in blacklist' check in Server::apply_blacklist()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28376 into master

* refs/pull/28376/head:
mds: count purge queue items left in journal

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28406 into master

* refs/pull/28406/head:
mds: reset heartbeat inside big loop

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28411 into master

* refs/pull/28411/head:
qa/cephfs: fix test_evict_client

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28433 into master

* refs/pull/28433/head:
cephfs-shell: Remove unnecessary empty line
cephfs-shell: Fix flake8 bare 'except' warnings
cephfs-shell: Remove unused variable 'e'
cephfs-shell: Fix flake8 use != to compare str literals warning
cephfs-shell: Fixe flake8 line too long error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

osd: Don't randomize deep scrubs when noscrub set

Fixes: http://tracker.ceph.com/issues/40198
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #28455 from yuriw/wip-yuriw-40208-master

qa/tests: removed `1node` and `systemd` tests as ceph-deploy is not actively developed

qa/tests: removed `1node` and `systemd` tests as ceph-deploy is not actively developed

Fixes: http://tracker.ceph.com/issues/40207
Fixes: http://tracker.ceph.com/issues/40208
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

rgw_file: all directories are virtual with respect to contents

This change causes directory handles to always report an mtime of
"now."  This is not an invalidate per se--it interacts with the
nfs implementation to produce that result when the implementation
updates its cached attributes.  Hence, it can be modulated by timers
or other rules governing attribute caching at the upper layer.

Fixes: http://tracker.ceph.com/issues/40204
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #28440 from linuxbox2/wip-rgwfile-invbckt

rgw_file: permit lookup_handle to lookup root_fh

Merge pull request #28442 from neha-ojha/wip-40120-port

qa/tasks/cbt.py: change port to work with client_endpoints

Reviewed-by: Mark Nelson <mnelson@redhat.com>

install-deps.sh: update ubuntu-toolchain-r mirrors

the removed two do not have GCC-9, but the newly added mirros do.

Signed-off-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: use GCC-9 on bionic

two reasons

* GCC-9 features more checks. so let's use it!
* crimson targets the hardware + toolchain + kernel after 1+ years,
  so it would be great if we can compile and test crimson using
  the toolchain which is ubiquitous on most mainstream distos.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28435 from yuriw/wip-yuriw-smoke-master

qa/tests: reduced distro to run to be random

Merge pull request #28428 from tchaikov/wip-package-crimson-osd

rpm,debian,install-deps: package crimson-osd

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

ceph.spec.in: align "Requires" of ceph-osd

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph.spec.in: package crimson-osd

Signed-off-by: Kefu Chai <kchai@redhat.com>

debian/: add ceph-crimson-osd package

this package is commented out using `# Crimson`. and it will be enabled
only if we are building "crimson" flavor builds.

this package depends on ceph-osd, because it'll basically be a drop-in
replacement of ceph-osd. and it shares all the dependencies of ceph-osd.
so let's just reuse ceph-osd package for now.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: install crimson-osd if WITH_SEASTAR

Signed-off-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: populate WITH_SEASTAR in the same way as for_make_check

because `install-deps.sh` is executed using `source`, we have to pass
these options using env variables. but before this change, `WITH_SEASTAR` is used directly,
while `FOR_MAKE_CHECK` is checked and translated to a local variable
`for_make_check`. which, in my opinion, has better readability.

so, in this change, `WITH_SEASTAR` is translated to `with_seastar`
variable in `install-deps.sh`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cbt.py: change port to work with client_endpoints

hack to work with https://github.com/ceph/cbt/pull/160/

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28439 from yuriw/wip-yuriw-crontab-master

qa/tests:  filtered in only "trusty"

Merge PR #27731 into master

* refs/pull/27731/head:
qa: use mimic-O upgrade process

Reviewed-by: Sage Weil <sage@redhat.com>

qa/tests:  filtered in only "trusty"

Fixes: http://tracker.ceph.com/issues/40195
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

rgw_file: permit lookup_handle to lookup root_fh

Originally, root_fh was reachable from RGWLibFS::fh_cache, but
this was problematic as it has infinite lifetime and so is not
present in RGWLibFS::fs_lru.

To fix the above, RGWLibFS::root_fh came to live unhooked from the
handle caches which works because nfs-ganesha/the application
always has an instance handle.  However, this silently breaks
READDIR event invalidation.

Fixes: https://tracker.ceph.com/issues/40196
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

qa: use mimic-O upgrade process

Fixes: https://tracker.ceph.com/issues/39436
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #27015 into master

* refs/pull/27015/head:
OSD: rollforward may need to mark pglog dirty

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #28242 into master

* refs/pull/28242/head:
osdc/Objecter: fix OSDMap leak in handle_osd_map

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #28363 into master

* refs/pull/28363/head:
osd/ClassHandler: use std::variant for storing func
osd/ClassHandler: no need to add `struct` before struct type
osd/ClassHandler: use ceph::mutex instead of Mutex
osd/ClassHandler: remove unused function
osd/ClassHandler: s/string/std::string/ in header

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

qa/tests: reduced distro to run to be random

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

rbd-mirror: link against the specified alloc library

As a daemon, use the build-time specified alloc library
(tcmalloc/jemalloc) when linking the application.

Fixes: http://tracker.ceph.com/issues/40110
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cephfs-shell: Remove unnecessary empty line

This patch fixes flake8 too many blank lines error.

Fixes: https://tracker.ceph.com/issues/40191
Signed-off-by: Varsha Rao <varao@redhat.com>

cephfs-shell: Fix flake8 bare 'except' warnings

Instead of bare 'except', catch OSError and libcephfs exceptions.

Fixes: https://tracker.ceph.com/issues/40191
Signed-off-by: Varsha Rao <varao@redhat.com>

cephfs-shell: Remove unused variable 'e'

Fixes: https://tracker.ceph.com/issues/40191
Signed-off-by: Varsha Rao <varao@redhat.com>

cephfs-shell: Fix flake8 use != to compare str literals warning

Fixes: https://tracker.ceph.com/issues/40191
Signed-off-by: Varsha Rao <varao@redhat.com>

cephfs-shell: Fixe flake8 line too long error

Break the line into multiple lines and add new variable wherever necessary, to
fix the line too long error.

Fixes: https://tracker.ceph.com/issues/40191
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #28313 from Songweibin/wip-fix-get-args

rbd/action: fix error getting positional argument

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28253 from Songweibin/wip-39973-rbd

osd/OSDCap: rbd profile permits use of rbd_info

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27902 from mikechristie/rbd-nbd-netlink-support

rbd-nbd: add netlink map/unmap support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

common: OutputDataSocket retakes mutex on error path

the list of buffers is protected by mutex m_lock. when dump_data() fails
and puts unwritten buffers back into the list, it needs to reaquire the
lock

Fixes: http://tracker.ceph.com/issues/40188
Signed-off-by: Casey Bodley <cbodley@redhat.com>

test: add unittests

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

mgr/influx: try to call close()

let's at least try to call influxclient.close() which in turn closes the
requests.Session

see
https://github.com/influxdata/influxdb-python/blob/v4.1.1/influxdb/client.py
and
https://github.com/influxdata/influxdb-python/blob/v5.0.0/influxdb/client.py

Fixes: http://tracker.ceph.com/issues/40174
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28370 from linuxbox2/wip-rgw-tntbucket

rgw_file: include tenant when hashing bucket names

common,osd: add hash algorithms for dedup (sha256, sha512)

Signed-off-by: Myoungwon Oh <omwmw@sk.com>