mailmap: Luo Kexue name normalization

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Ning Yao affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Vitja Makarov affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: sunspot affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Qinghua Jin affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Richard W.M. Jones affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Ira Cooper affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Star Guo affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Xiaoxi Chen affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Jenkins name normalization

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

Merge pull request #8519 from dachary/wip-xenial

tests: add Ubuntu 16.04 xenial dockerfile

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8517 from kjurka/rgw-admin-doc-region-list

doc: rgw admin uses "region list" not "regions list"

Merge pull request #8518 from wjin/fix

crush: fix typo

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #8462 from xiexingguo/xxg-wip-scrub

osd: fix two scrub relevant issues

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8485 from liewegas/wip-fuse-log

ceph-fuse: rotate log file

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #8487 from liewegas/wip-copy-from-fail-cleanup

osd: clean up temp object if copy-from fails

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8486 from liewegas/wip-14695

librados: Revert "rados: Add new field flags for ceph_osd_op.copy_get."

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #7956 from xiexingguo/xxg-wip-fixfstat

common: buffer: put a guard for stat() syscall during read_file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8516 from runsisi/wip-fix-ret

journal: fix final result for JournalTrimmer::C_RemoveSet

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tests: add Ubuntu 16.04 xenial dockerfile

Signed-off-by: Loic Dachary <loic@dachary.org>

crush: fix typo

Signed-off-by: Wei Jin <wjin.cn@gmail.com>

doc: rgw admin uses "region list" not "regions list"

journal: fix final result for JournalTrimmer::C_RemoveSet

if the first journal object got removed successfully while later objects
failed then the final result should be set to fail

Signed-off-by: runsisi <runsisi@zte.com.cn>

Merge pull request #8453 from cbodley/wip-rgw-sync-master-down

rgw: retry RGWRemoteMetaLog::read_log_info() while master is down
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8434 from theanalyst/rgw/typo-fix

rgw: fix a typo in error message

Merge pull request #8481 from weiqiaomiao/wq

rgw: the map 'headers' is assigned a wrong value
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8488 from oritwas/wip-15318

rgw: fix problem deleting objects begining with double underscores

rbd-mirror: fixed infinite loop when disabling image mirroringReviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: fixed infinite loop when disabling image mirroring

rbd-mirror: fixed bug that caused infinite loop when disabling image mirroring

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #8501 from dachary/wip-releases

releases: update the v10.1.0 release link

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

release-notes: v10.1.1 release notes

Signed-off-by: Loic Dachary <loic@dachary.org>

release-notes: v10.1.1 release notes (draft)

cd src/script
virtualenv v
source v/bin/activate
pip install githubpy GitPython requests
GITHUB_ACCESS_TOKEN=xxxxxxx ./ceph-release-notes \
      -r tags/v10.1.0..tags/v10.1.1 $(git rev-parse --show-toplevel)

Signed-off-by: Loic Dachary <loic@dachary.org>

releases: firefly was EOL december 2015

Signed-off-by: Loic Dachary <loic@dachary.org>

releases: update the v10.1.0 release link

And add the v10.1.1 release notes as well.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #8500 from linuxbox2/ldap-include

rgw_ldap: make ldap.h inclusion conditional

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #8495 from athanatos/wip-14777

osd/ReplicatedPG::_rollback_to: update the OMAP flag

Reviewed-by: Sage Weil <sage@redhat.com>

rgw_ldap: make ldap.h inclusion conditional

The feature implementation is conditional, embarassingly, the
header inclusion is not.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: fix problem deleting objects begining with double underscore

Fixes: http://tracker.ceph.com/issues/15318
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8493 from dillaman/wip-15354

test: fix valgrind memcheck issues for rbd-mirror test cases

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8490 from dillaman/wip-15097

rbd: journal reset should disable/re-enable journaling feature

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8477 from dillaman/wip-15364

journal: fix race condition between Future and journal shutdown

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

test: fix memory leaks in rbd-mirror test cases

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: avoid leaking librados connections when creating pools

Fixes: http://tracker.ceph.com/issues/15354
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: journal reset should disable/re-enable journaling feature

Fixes: http://tracker.ceph.com/issues/15097
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: warn if 'sortbitwise' flag is not set and no legacy OSDs are presentReviewed-by: Samuel Just <sjust@redhat.com>

mon: warn if 'sortbitwise' flag is not set and no legacy OSDs are present

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8476 from dillaman/wip-rbd-mirror-lockdep

rbd-mirror: workaround for intermingled lockdep singletons

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/ReplicatedPG: clean up temp object if copy-from fails

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_rados_api_misc: make CopyFrom omap test be a big object

This makes it so we check single-shot path and the one where we have a
temp object to clean up.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8456 from dreamhost/wip-authtool-help

authtool: update --help and manpage to match code.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8446 from liewegas/wip-15366

osd: fix/clean up full map request handling

Reviewed-by: Kefu Chai <kchai@redhat.com>

Revert "rados: Add new field flags for ceph_osd_op.copy_get."

This reverts commit 2b4acfb1b808b98132a771cf1089063c0f7a75b5.

We fixed this on the receiving end in
fc51ce2a837e8e878d46d8ca54531aa7bd5d01b2 a few weeks after this commit
was added.  It's ugly and no longer needed.

This *does* mean that we'll start to copy some data and then abort
partway through, but that's better than an kludgey protocol/interface.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8472 from jmunhoz/fix-aws4-subdomain-calling

rgw: aws4 subdomain calling bugfix
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rbd-mirror: workaround for intermingled lockdep singletons

If librados constructs the CephContext, lockdep will be initialized
in both the librados shared address space and also within the app
address space.  This breaks lockdep's functionality and results in
a crash upon shutdown.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

logrotate.conf: poke ceph-fuse after log rotation

This still leaves other loggers with un-reopened log files,
unfortunately.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-fuse: reopen log file on SIGHUP

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8460 from linuxbox2/jewel-nfs-fixes-3

Jewel nfs fixes 3

librbd: restore out-of-band future callbacks to avoid lock cycles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: aws4 subdomain calling bugfix

Fixes: http://tracker.ceph.com/issues/15369
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

rgw: the map 'headers' is assigned a wrong value

Signed-off-by: weiqiaomiao <wei.qiaomiao@zte.com.cn>

Merge pull request #8457 from dreamhost/wip-virtualenv-tmpdir

build: Respect TMPDIR for virtualenv.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

authtool: fix test output validation & minor whitespace.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

journal: fix context memory leak when shutting down live replay

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: Future does not require metadata shared pointer

Avoid keeping the metadata object alive through the lifespan of the
future.  Callers can expect to receive a re-entrant callback if the
future is already safe and flush/wait is invoked.

Fixes: http://tracker.ceph.com/issues/15364
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: warn if 'sortbitwise' flag is not set and no legacy OSDs are present

If the user has upgraded, they should set sortbitwise.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8418 from liewegas/wip-blk-probe

os/ObjectStore: make device uuid probe output something friendly

Merge pull request #8428 from liewegas/wip-rest-mds

ceph-rest-api: fix fs/flag/set

Merge pull request #7981 from liewegas/wip-14364

osdc/Objecter: fix narrow race with tid assignment

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8403 from dx9/wip-ceph-dencoder-esessions-fix

mds: Add cmapv to ESessions default constructor initializer list

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8419 from adamemerson/wip-32bit-time

common: fix time_t cast in decode

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8431 from liewegas/wip-bluestore

os/bluestore: revamp BlueFS bdev management and add perfcounters

Merge pull request #8445 from jmunhoz/fix-aws4-uri-encoding

rgw: aws4 uri encoding bugfix
Review-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8450 from javacruft/tasksmax-infinity

systemd: drop any systemd imposed process/thread limits

Reviewed-by: Sage Weil <sage@redhat.com>

debian/rules: include ceph-mds-*.conf upstart files in ceph-mds

These were lost by a typo in 0cbe3dea69604730bc7dba7bc3844294f4c9ac5a

Fixes: http://tracker.ceph.com/issues/15395
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8459 from jdurgin/wip-rbd-op-threads

librbd: disallow unsafe rbd_op_threads values

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8455 from liewegas/wip-legacy-layout-zero

mds: fix file_layout_t legacy encoding snafu

Reviewed-by: John Spray <john.spray@redhat.com>

osd: cancel scrub if noscrub is set for pool or all

The sched_scrub() method can be called by various code path, such as
OSD::tick() or triggered by a scrub_reserve_reply message.

The sched_scrub() will check whether or not the noscrub is globally set or
set for a specified pool before really starting to schedule a scrub job.
However, if we set noscrub flag for a specified pool, there are other pools
for which scrub are still legal and thus shall be granted.

The problem here is that we may stopping a pg's scrub in an intermidate stage
due to setting of the corresponding pool's noscrub flag whithout releasing
the reservation. Which as a result shall prevent other pgs of a different
pool from going scrubbing because we have already hit the reservation limit.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: reset tp handle when search for boundary of chunky-scrub

One of our tests in our local testbed shows that if the number of snapshots
become extremely huge, the process of chunky-scrub() may encouter
heart-beat failure. This is because it takes a real long time for the
procedure to traverse and determine the boundary for a single run of
chunk scrub under this case.

This pr tries to solve the above the problem by resetting the tp handle
passed in once in a while(after a certain number of loops, 64 by default)
since the search can become very time-consumptive. Furthermore, the
BUILD_MAP stage later on shall encouter the same problem but has already
got fixed in the same way. Therefore, although the test case is rare,
but this change is defensive and make our code strong and thus shall be
considered as worthwhile.

Fixes: tracker.ceph.com/issues/12892
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: use short, relative paths with bluefs

If we're using bluefs, only pass in the short relative
path (db, db.wal, db.slow).  The leading components
are ignored and only lead to errors if the configuration
provides relative paths that do not match (e.g., if one
if using ceph-objectstore-tool).

Fixes: http://tracker.ceph.com/issues/15376
Signed-off-by: Sage Weil <sage@redhat.com>

librbd: disallow unsafe rbd_op_threads values

Don't use this config option in librbd until
http://tracker.ceph.com/issues/15034 is avoided.

The option itself is still useful for mirroring threads, where
ordering is unimportant.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

librgw/rgw_file:  correctly handle object permissions

Implement the full object permission model for librgw (aka, NFS
and similar) operations.

Fixes DIRS1 unit tests.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file: print DIRS1 read parameters at verbose

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file:  fix attributes for "special" test cases

If a caller does an atomic create using rgw_lookup() and
RGW_LOOKUP_FLAG_CREATE, it needs to fix up the attributes using
create_stat().

For use outside of test cases, it probably needs an interlock also,
but for now, do just enough to satisfy existing attribute checks.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file unit tests: validate Unix owners in DIRS1

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

authtool: update --help and manpage to match code.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

build: Respect TMPDIR for virtualenv.

Gentoo's normal build process uses a sandbox to catch writes outside the
build environment; this includes providing a value other than /tmp for
TMPDIR. Use TMPDIR by default for CEPH_BUILD_VIRTUALENV.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

common/fs_types: dump pool_id signed

Signed-off-by: Sage Weil <sage@redhat.com>

mds: fix legacy layout decode with pool 0

If you data pool was pool 0, this was transforming
that to -1 unconditionally, which broke upgrades.  We
only want do that for a fully zeroed ceph_file_layout,
so that it still maps to a file_layout_t.  If any fields
are set, though, we trust the fl_pgpool to be a valid
pool.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8447 from cbodley/wip-cmake-mrun

mrun: update path to cmake binaries

rgw: retry read_log_info() while master is down

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #8430 from wjin/fix

crush: fix error log

Reviewed-by: Kefu Chai <kchai@redhat.com>

Drop any systemd imposed process/thread limits

If systemd has task accounting enabled, a default of 512 tasks
will be applied to all systemd units.

For ceph, this is way to low even for a modest cluster, so stop
this restriction being applied and allow administrators to apply
limits using sysctl.

Signed-off-by: James Page <james.page@ubuntu.com>

mrun: update path to cmake binaries

Signed-off-by: Casey Bodley <cbodley@redhat.com>

os/bluestore/BlueFS: add some perfcounters

Most utilization-related.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: revamp bdev ids

You cannot tell from the old bdev vector which device
was which.

- use a fixed id for each type/slot
- go from fast(small) to slow(big)
- normalize the allocation fallback to try any slower
  device.
- clean up the BlueStore instantiation/setup accordingly

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: make device uuid probe output something friendly

Otherwise, all you see is errors about the probes that failed (e.g., a
failure to decode a non-bluestore superblock as bluestore).

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: aws4 uri encoding bugfix

Fixes: http://tracker.ceph.com/issues/15358
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

mon/OSDMonitor: fix off-by-one for osd_map_message_max

For most messages we were sending osd_map_message_max + 1 maps.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8131 from ErwanAliasr1/evelu-fast-check

tests: Improving 'make check' execution time

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: improve full map requests

If we don't get all the full maps we want, request more
immediately.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: create rerequest_full_maps() helper

Signed-off-by: Sage Weil <sage@redhat.com>