rpm,debian: use download.ceph.com/tarballs instead of ceph.com/download

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fix warning reported by "build-doc linkcheck"

all the HTTP 301 (moved permanently) should be killed.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: update manual-deployment with latest code

rbd pool is not created by default any more.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15811 from x-ion-de/wip-20309

doc: Update manual deployment

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #18372 into master

* refs/pull/18372/head:
ptl-tool: Fix more readme examples

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #18368 from kungf/scrub_week_day_bound

osd: add scrub week day constraint

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18400 from dillaman/wip-21844

osdc/Objecter: skip sparse-read result decode if bufferlist is empty

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18418 from dillaman/wip-21860

common/common_init: disable ms subsystem log gathering for clients

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18427 from dillaman/wip-21845

osdc/Objecter: delay initialization of hobject_t in _send_op

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #18450 from liewegas/wip-osd-removing

osd: add numpg_removing metric

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #18452 from liewegas/wip-21880

ceph_test_objectstore: disable filestore_fiemap for tests

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18454 from liewegas/wip-21882

messages/MOSDMap: do compat reencode of crush map, too

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18457 from liewegas/wip-restful-health

qa/suites/rados/rest/mgr-restful: whitelist more health

qa/suites/rados/rest/mgr-restful: whitelist more health

The test is fiddling with OSDs.

Signed-off-by: Sage Weil <sage@redhat.com>

messages/MOSDMap: do compat reencode of crush map, too

If we are reencoding an incremental, and it embeds a crush map, we need
to reencode that in a compatible way too.  This is especially true now
because we have the compat crush weight-sets.  Otherwise, a client may
learn the crush map through an incremental but not understand choose_args,
and not see/understand the alternate weight set.  It will send requests
to the wrong OSDs where they will just get dropped.

Fixes: http://tracker.ceph.com/issues/21882
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18406 from adamemerson/wip-insecticide

test,osd,kvstore_tool: silence warnings and prepare test buffer in the right way

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18379 from liewegas/wip-kill-replay-interval

osd/osd_types: pg_pool_t: remove crash_replay_interval member

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #18407 from adamemerson/wip-upgrade-zstd

zstd: Upgrade to v1.3.2

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18357 from asomers/have_libaio

Build: create a proper WITH_BLUESTORE option

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-By: Kefu Chai <kchai@redhat.com>

Merge pull request #18354 from kungf/no_scurb_deadline

osd: make scrub no deadline when max interval is zero

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #18396 from xiexingguo/wip-scrub-mismatch

osd/osd_types.cc: copy extents map too while making clone

Reviewed-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: disable filestore_fiemap for tests

This option was enabled in 87f33376d977962ab7438c46873ea9b6292390d1 but
causes ObjectStore/StoreTest.Synthetic/1 (filestore) to fail.  Revert that
bit for now until we fix fiemap properly.

See http://tracker.ceph.com/issues/21880

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16958 from mikulely/fix-full-uri

rgw: fix opslog uri as per Amazon s3

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17546 from mikulely/rgw-cleanup-min

rgw: cleanup MIN macro with std::min

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #17934 from joke-lee/rgw_list_objecst_wrong_result_with_marker_when_enable_versioning

rgw:fix list objects with marker wrong result  when bucket is enable versioning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #17673 from mogeb/blkin

blkin: link against lttng-ust-fork

Merge pull request #18414 from majianpeng/pmem-work

libmpem: Revert "submodule: make libmpem as a submodule."

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: add numpg_removing metric

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18335 from linuxbox2/wip-rgw-readdir2

rgw_file: implement variant offset readdir processing

Reviewed-by: Casey Bodley <cbodley@redhat.com>

common/common_init: disable ms subsystem log gathering for clients

The log gathering causes large performance degradation to clients
with high message throughputs. This is hopefully a short-term
workaround until per-message logging can be replaced with an
efficient data recording system for post-incident analysis
use-cases.

Fixes: http://tracker.ceph.com/issues/21860
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18392 from tchaikov/wip-21842

os/bluestore: implement BlueRocksEnv::AreFilesSame()

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

osdc/Objecter: delay initialization of hobject_t in _send_op

Fixes: http://tracker.ceph.com/issues/21845
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #18251 from jan--f/mgr-py-module-constants

pybind/mgr_module: move PRIO_* and PERFCOUNTER_* to MgrModule class

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #18021 from jcsp/wip-prometheus-disk-occupation

mgr/prometheus: add ceph_disk_occupation series

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Revert "submodule: make libmpem as a submodule."

This reverts commit 6a14159ee4b910bf83b38316f98ef07edf5f9a01.
Using ExternalProject_Add::GIT to download nvml source .

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #18250 from dillaman/wip-21763

doc/rbd: tweaks for the LIO iSCSI gateway

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

os/bluestore: implement BlueRocksEnv::AreFilesSame()

it is used by the "repair" feature to dedup the files to be searched for
MANIFEST-* files. the default implementation is the POSIX one, which
tries to look at the local fs, but we should be looking for the files in
the bluefs. in this very use case, wal and db do not share the same device,
so we can just compare the paths. actually, it should aways return
"false". as the files being compared are always "db" and "db.wal".

Fixes: http://tracker.ceph.com/issues/21842
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18387 from tchaikov/wip-dmclock-client-info-ptr

osd,dmclock: use pointer to ClientInfo instead of a copy of it

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #18350 from ovh/bp-ceph-conf-dump-all

tools/ceph-conf: dump parsed config in plain text or as json

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18367 from zealoussnow/wip-10180

ceph-bluestore-tool: create out_dir before create full path  of kvdb

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: add scrub week day constraint

if add week day constraint, we can set scrub permit time
more flexible. eg. we can set scurb in Monday-Wednesday 0-12 o'clock
according set this parameter:
osd_scrub_begin_week_day = 1
osd_scrub_end_week_day = 3
osd_scrub_begin_hour = 0
osd_scrub_end_hour = 12

Signed-off-by: kungf <yang.wang@easystack.cn>

osdc/Objecter: skip sparse-read result decode if bufferlist is empty

If the OSD does not execute sub-ops due to errors encountered prior to
the sub-op, the sub-op result remains zeroed with empty out data.
Attempting to decode the empty bufferlist results in large exception
handling CPU overhead.

Fixes: http://tracker.ceph.com/issues/21844
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17441 from yanyixing/update-umcloud-affiliation

mailmap: Update umcloud affiliation

Reviewed-by: Kefu Chai <kchai@redhat.com>

zstd: Upgrade to v1.3.2

This buys us fixes for a few memory errors and asserts and also
improves the CMake build script so it doesn't mostly ignore the
CXXFLAGS we pass in.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #18374 from tchaikov/wip-ceph-C-d

ceph.in: do not panic at control+d in interactive mode

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>

OSD: Silence unused variable warnings

Even though extending the lifetime of this temporary delays execution
of its destructor, the compiler does not consider the reference to
have a side-effecting constructor.

The compiler is wrong, but we can silence its complaints.

This will cease to be an issue once we adopt C++17, since mandatory
RVO will solve the inability to bind a variable to a returned
move-only object and the need to extend a temporary.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

kvstore_tool: Create default constructor

Clang4 does not recognize a constructor with all arguments defaulted
as a default constructor in some circumstances.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

test/cls_journal: Fix reversed std::string constructor arguments

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Update docs for manual deployment

Fixes: http://tracker.ceph.com/issues/20309
Signed-off-by: Jens Rosenboom <j.rosenboom@x-ion.de>

Merge pull request #17462 from majianpeng/pmem-work

cmake: compile nvml as an external project

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: Build libpmem from source code.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

doc/rbd: tweaks for the LIO iSCSI gateway

Fixes: http://tracker.ceph.com/issues/21763
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: make scrub no deadline when max interval is zero

some times, we only want scrub at out permitted time, avoid
scrub mainly deep scrub affecting business io. this patch
make no scrub deadline when osd_scrub_max_interval == 0, then
we can make sure scrub can only do in expect time.

Signed-off-by: kungf <yang.wang@easystack.cn>

Merge pull request #18389 from yaozongyou/fix-type-in-bluestore-migration

doc: fix typo in bluestore-migration.rst

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #18395 from gaosibei/mailmap

mailmap: Add Sibei, XueYu Affiliation

Reviewed-by: Jos Collin <jcollin@redhat.com>

mailmap, organizationmap: add Sibei, XueYu

Signed-off-by: Sibei Gao <gaosb@inspur.com>

osd: update mClock.*Queue to incorporate the changes on dmclock

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fix typo in bluestore-migration.rst

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>

Merge pull request #18232 from jdurgin/wip-filestore-rocksdb

common/options: enable multiple rocksdb compaction threads for filestore

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18311 from shinobu-x/sk-osd_20171015

osd: Don't initialze pointers by NULL or 0

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #14213 from xiaoxichen/fix_wbthrottle_assert

os/Filestore: fix wbthrottle assert.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18256 from liewegas/wip-bluestore-superblock

os/bluestore: allow reconstruction of osd data dir from bluestore bdev label

Reviewed-By: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18346 from tchaikov/wip-log-if-do_read_entry-fails

os/filestore: print out the error if do_read_entry() fails

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

dmclock: pickup latest dmclock subtree

to include the change of https://github.com/ceph/dmclock/pull/41
Merge commit '060a7777cb6fe3f052259e1324490cdbf7b49980'

Signed-off-by: Kefu Chai <kchai@redhat.com>

Squashed 'src/dmclock/' changes from 93f760c57c..1bbd42b8fe

1bbd42b8fe Merge pull request #41 from tchaikov/wip-client-info-ptr
cb7901e99a ClientRec: instead of a copy of ClientInfo, keep a pointer of it

git-subtree-dir: src/dmclock
git-subtree-split: 1bbd42b8fe212dbff68c2b470813e936859705d6

ceph-bluestore-tool: create out_dir before create full path  of kvdb

Signed-off-by: Leo Zhang <nguzcf@gmail.com>

Merge pull request #18145 from dzafman/wip-18162

osd: object added to missing set for backfill, but is not in recovering, error!

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/osd_types: pg_pool_t: remove crash_replay_interval member

We removed the dual ack before luminous, so this option is completely
obsolete now.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #18377 from joscollin/wip-cleanup-rgw-function-names

rgw: Fix getter function names in RGWEnv

rgw_file: disable FLAG_EXACT_MATCH enforcement

a. upstream motivating behavior no longer reproduces
b. incorrect stat behavior observed, traced to exact match enforcement

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file: implement variant offset readdir processing

Introduce new rgw_readdir2(...), which in which continues
from an arbitrary dirent name, which presumably has been
seen in a prior partial enumeration.

Add single-file unit test for READDIR cases, librgw_file_marker.cc.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #18380 from zmc/wip-master-init

Changes required for teuthology's systemd support

Merge pull request #18365 from linuxbox2/wip-rgw-v3open-write

rgw_file: explicit NFSv3 open() emulation

qa/tasks/ceph: Set owner on mds mount point

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/tasks/cephfs/filesystem: Check for mds failure

... inside Filesystem.are_daemons_healthy()

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/tasks/ceph: Remove /var/lib/ceph copypasta

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/tasks/ceph: Set owner on osd mount point

This is needed when running with systemd

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/tasks/ceph: Set owner on mon mount point

This is needed when running with systemd

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #15554 from cbodley/wip-rgw-parse-cr-dump

tools/rgw: add script to inspect admin socket "cr dump"

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

test: Cleanup test-erasure-eio.sh code

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Add replicated recovery/backfill test

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Better recovery/backfill restart for mark_unfound_lost delete/revert

Signed-off-by: David Zafman <dzafman@redhat.com>

osd, mon: Add new pg states recovery_unfound and backfill_unfound

Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Add new UnfoundBackfill and UnfoundRecovery pg transitions

Signed-off-by: David Zafman <dzafman@redhat.com>

test: Test case that reproduces tracker 18162

recover_replicas: object added to missing set for backfill, but is not in recovering, error!

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #18378 from ceph/wip-yuriw-21822-master

tests - ceph-ansible vars additions

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>

Build: create a proper WITH_BLUESTORE option

Using Bluestore is not the same thing as having libaio, especially on
non-Linux platforms.

Signed-off-by: Alan Somers <asomers@gmail.com>

tests - ceph-ansible vars additions
added symlinks for distros

Fixes http://tracker.ceph.com/issues/21822
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit f30e9a5e6bd47129f8a35c1f9067216cfe8a6a70)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

rgw: Fix getter function names in RGWEnv

Fixed the getter function names as per the review comment in
https://github.com/ceph/ceph/pull/17432

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #18376 from tchaikov/wip-authors

doc: update Component Technical Leads and maintainers to canonical location

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18355 from gaosibei/wip-rgw-refrence

RGW: change function parameters from value to refrence

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17432 from joscollin/wip-cleanup-fix-rgwconf

rgw: Access RGWConf through RGWEnv

Merge pull request #17440 from mikulely/cleanup-iterate

rgw: update outdated debug func name

Merge pull request #17956 from zhangsw/fix-rgw-encry-torrent-bug

rgw: Torrents are not supported for objects encrypted using SSE-C

Merge pull request #18057 from baixueyu/branch1

rgw: fix radosgw-admin quota enable return value bug

Merge pull request #18375 from tchaikov/wip-ceph-disk-check-log

ceph-disk: write log to /var/log/ceph not to /var/run/ceph

Reviewed-by: Alfredo Deza <adeza@redhat.com>

AUTHORS: Update Component Technical Leads and maintainers to canonical location of ceph.com.

Signed-off-by: Patrick McGarry <pmcgarry@redhat.com>

Merge pull request #18315 from dillaman/wip-21797

librbd: object map batch update might cause OSD suicide timeout

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>

Merge pull request #18345 from wido/doc-osd-replace

doc: Remove additional arguments when replacing OSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

tools/ceph-conf: dump parsed config in plain text or as json

This is useful for finding differences between ceph.conf on disk
and in osd/mon memory.

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>