osd: add option to disable startup check

Signed-off-by: Sage Weil <sage@redhat.com>

osd: refuse to start if configure object limits don't work

Fail early if the configured max object name cannot be stored on
the (FileStore) backend.  This avoids rude surprises down the
line when some random rados op get ENAMETOOLONG.

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: --short to set ext4-friendly object name and namespace limits

So devs can work on ext4.  Note that this will also mean s3-tests
will fail under vstart.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8544 from xiexingguo/xxg-wip-15464

test: fix ut test failure caused by lfn change

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: fix ut test failure caused by lfn change

"https://github.com/ceph/ceph/pull/8496/commits/755c685f2d09c36b53bc39589a77fa73faba5149"
change the lfn_parse_object_name() return type from bool to int but does not
change the relevant test correspondingly.

Fixes: http://tracker.ceph.com/issues/15464
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Fixed ceph-common install.

Fixes: #15438
Signed-off-by: John Wilkins <jowilkin@redhat.com>

librbd: fixed bug in disabling non-primary image mirroring

librbd: fixed bug in disabling non-primary image mirroring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8496 from athanatos/wip-15312

os/filestore: force lfn attrs to be written atomically, restructure name length limits

Merge pull request #8529 from oritwas/wip-rgw-fix-zone-set

radosgw-admin: allow

librbd: Fixed bug in disabling non-primary image mirroring

It is only possible to remove image from rbd_mirroring object if the
mirroring image state is set to DISABLING.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #8508 from JiYou/fix-ceph-disk

ceph-disk: do not always fail when re-using a partition

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'gh/jewel'

ceph-disk: fix PrepareData.set_type() args should be class member.

Signed-off-by: You Ji <youji@ebay.com>

ceph-disk: fix spelling mistake geattr to getattr.

Signed-off-by: You Ji <youji@ebay.com>

ceph-disk: fix lockbox set_or_create_partition()

Signed-off-by: You Ji <youji@ebay.com>

Merge pull request #8507 from kmroz/wip-rgw-admin-cleanup

rgw-admin: remove unused iterator and fix error message

radosgw-admin: fix name checking

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: allow setting zone when there is no realm

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8522 from swamireddy/10.1.1-mailmap

mailmap update

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #8519 from dachary/wip-xenial

tests: add Ubuntu 16.04 xenial dockerfile

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8517 from kjurka/rgw-admin-doc-region-list

doc: rgw admin uses "region list" not "regions list"

Merge pull request #8518 from wjin/fix

crush: fix typo

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #8462 from xiexingguo/xxg-wip-scrub

osd: fix two scrub relevant issues

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8485 from liewegas/wip-fuse-log

ceph-fuse: rotate log file

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #8487 from liewegas/wip-copy-from-fail-cleanup

osd: clean up temp object if copy-from fails

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8486 from liewegas/wip-14695

librados: Revert "rados: Add new field flags for ceph_osd_op.copy_get."

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #7956 from xiexingguo/xxg-wip-fixfstat

common: buffer: put a guard for stat() syscall during read_file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8516 from runsisi/wip-fix-ret

journal: fix final result for JournalTrimmer::C_RemoveSet

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tests: add Ubuntu 16.04 xenial dockerfile

Signed-off-by: Loic Dachary <loic@dachary.org>

crush: fix typo

Signed-off-by: Wei Jin <wjin.cn@gmail.com>

doc: rgw admin uses "region list" not "regions list"

journal: fix final result for JournalTrimmer::C_RemoveSet

if the first journal object got removed successfully while later objects
failed then the final result should be set to fail

Signed-off-by: runsisi <runsisi@zte.com.cn>

test/rados/misc.cc: add long locator key and namespace tests

Signed-off-by: Samuel Just <sjust@redhat.com>

LFNIndex: use chain_getxattr_buf

LFN attrs can be longer than these hard coded limits, let's just do
buffer doubling everywhere.

Signed-off-by: Samuel Just <sjust@redhat.com>

chain_xattr: add chain_getxattr_buf

Otherwise callers need to implement buffer doubling in a lot of cases,
which is error prone.

Signed-off-by: Samuel Just <sjust@redhat.com>

LFNIndex::lfn_parse_object_name: return int rather than bool

All current callers (except list_objects) already expect
ret < 0 for error.  Adjust list_objects to skip on EINVAL
instead of 0.

Signed-off-by: Samuel Just <sjust@redhat.com>

LFNIndex::list_objects: lfn_translate does not set errno

Signed-off-by: Samuel Just <sjust@redhat.com>

FileStore::set_xattr_limits_via_conf: add warning if max xattr size smaller than max name

Signed-off-by: Samuel Just <sjust@redhat.com>

test/objectstore/chain_xattr.cc: add test for ensure_single_attr

Signed-off-by: Samuel Just <sjust@redhat.com>

LFNIndex: ensure that lfn and directory attrs are written atomically

We leave skip_chain_cleanup = false since the object may have been
written using code that split the lfn attr over multiple attrs.

Signed-off-by: Samuel Just <sjust@redhat.com>

chain_[f]getxattr: always use size, no reaon to consider CHAIN_XATTR_MAX_BLOCK_LEN

Users of setxattr can now set attrs larger than that size.

Signed-off-by: Samuel Just <sjust@redhat.com>

chain_xattr: s/onechunk/skip_chain_cleanup, add ensure_single_attr

onechunk was a deceptive name since it didn't actually ensure that
a single chunk would be used.  Rename to ensure_single_attr.  Also,
add a parameter to ensure that we use a single attribute.  We need
these to be distinct since we have LFN attrs which have been split
over 254 byte xattrs since after hammer which we need to correctly
clean up.  However, we need to ensure going forward that those attrs
are never split over more than one chunk -- it's not atomic.
Further, skip_chain_cleanup should imply ensure_single_attr, so make
them template params and add a static check to make sure we don't
mess this up in the future.

Signed-off-by: Samuel Just <sjust@redhat.com>

os/,osd/: restructure the rados name length check

Enforce locator length vs the max name length and also
introduce a namespace length limit.

In addition to these checks, also pass the head object to the
ObjectStore implementation to validate.  This allows LFNIndex to account
for the idiosyncracies of its filename escaping and for different xattr
value max sizes.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #8453 from cbodley/wip-rgw-sync-master-down

rgw: retry RGWRemoteMetaLog::read_log_info() while master is down
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

ceph-disk: fix set_data_partition() when data is partition.

Signed-off-by: You Ji <youji@ebay.com>

rgw-admin: fix period delete error message

Signed-off-by: Karol Mroz <kmroz@suse.com>

rgw-admin: remove unused iterator

Signed-off-by: Karol Mroz <kmroz@suse.com>

Merge pull request #8434 from theanalyst/rgw/typo-fix

rgw: fix a typo in error message

Merge pull request #8481 from weiqiaomiao/wq

rgw: the map 'headers' is assigned a wrong value
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8488 from oritwas/wip-15318

rgw: fix problem deleting objects begining with double underscores

rbd-mirror: fixed infinite loop when disabling image mirroringReviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: fixed infinite loop when disabling image mirroring

rbd-mirror: fixed bug that caused infinite loop when disabling image mirroring

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #8501 from dachary/wip-releases

releases: update the v10.1.0 release link

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Luo Kexue name normalization

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Ning Yao affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Vitja Makarov affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

release-notes: v10.1.1 release notes

Signed-off-by: Loic Dachary <loic@dachary.org>

release-notes: v10.1.1 release notes (draft)

cd src/script
virtualenv v
source v/bin/activate
pip install githubpy GitPython requests
GITHUB_ACCESS_TOKEN=xxxxxxx ./ceph-release-notes \
      -r tags/v10.1.0..tags/v10.1.1 $(git rev-parse --show-toplevel)

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: sunspot affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Qinghua Jin affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Richard W.M. Jones affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Ira Cooper affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Star Guo affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Xiaoxi Chen affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Jenkins name normalization

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

releases: firefly was EOL december 2015

Signed-off-by: Loic Dachary <loic@dachary.org>

releases: update the v10.1.0 release link

And add the v10.1.1 release notes as well.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #8500 from linuxbox2/ldap-include

rgw_ldap: make ldap.h inclusion conditional

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #8495 from athanatos/wip-14777

osd/ReplicatedPG::_rollback_to: update the OMAP flag

Reviewed-by: Sage Weil <sage@redhat.com>

rgw_ldap: make ldap.h inclusion conditional

The feature implementation is conditional, embarassingly, the
header inclusion is not.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: fix problem deleting objects begining with double underscore

Fixes: http://tracker.ceph.com/issues/15318
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8493 from dillaman/wip-15354

test: fix valgrind memcheck issues for rbd-mirror test cases

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8490 from dillaman/wip-15097

rbd: journal reset should disable/re-enable journaling feature

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8477 from dillaman/wip-15364

journal: fix race condition between Future and journal shutdown

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

test: fix memory leaks in rbd-mirror test cases

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: avoid leaking librados connections when creating pools

Fixes: http://tracker.ceph.com/issues/15354
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: journal reset should disable/re-enable journaling feature

Fixes: http://tracker.ceph.com/issues/15097
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: warn if 'sortbitwise' flag is not set and no legacy OSDs are presentReviewed-by: Samuel Just <sjust@redhat.com>

mon: warn if 'sortbitwise' flag is not set and no legacy OSDs are present

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8476 from dillaman/wip-rbd-mirror-lockdep

rbd-mirror: workaround for intermingled lockdep singletons

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/ReplicatedPG: clean up temp object if copy-from fails

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_rados_api_misc: make CopyFrom omap test be a big object

This makes it so we check single-shot path and the one where we have a
temp object to clean up.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8456 from dreamhost/wip-authtool-help

authtool: update --help and manpage to match code.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8446 from liewegas/wip-15366

osd: fix/clean up full map request handling

Reviewed-by: Kefu Chai <kchai@redhat.com>

Revert "rados: Add new field flags for ceph_osd_op.copy_get."

This reverts commit 2b4acfb1b808b98132a771cf1089063c0f7a75b5.

We fixed this on the receiving end in
fc51ce2a837e8e878d46d8ca54531aa7bd5d01b2 a few weeks after this commit
was added.  It's ugly and no longer needed.

This *does* mean that we'll start to copy some data and then abort
partway through, but that's better than an kludgey protocol/interface.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8472 from jmunhoz/fix-aws4-subdomain-calling

rgw: aws4 subdomain calling bugfix
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

rbd-mirror: workaround for intermingled lockdep singletons

If librados constructs the CephContext, lockdep will be initialized
in both the librados shared address space and also within the app
address space.  This breaks lockdep's functionality and results in
a crash upon shutdown.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

logrotate.conf: poke ceph-fuse after log rotation

This still leaves other loggers with un-reopened log files,
unfortunately.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-fuse: reopen log file on SIGHUP

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8460 from linuxbox2/jewel-nfs-fixes-3

Jewel nfs fixes 3

librbd: restore out-of-band future callbacks to avoid lock cycles

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: aws4 subdomain calling bugfix

Fixes: http://tracker.ceph.com/issues/15369
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

rgw: the map 'headers' is assigned a wrong value

Signed-off-by: weiqiaomiao <wei.qiaomiao@zte.com.cn>

Merge pull request #8457 from dreamhost/wip-virtualenv-tmpdir

build: Respect TMPDIR for virtualenv.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

authtool: fix test output validation & minor whitespace.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

journal: fix context memory leak when shutting down live replay

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: Future does not require metadata shared pointer

Avoid keeping the metadata object alive through the lifespan of the
future.  Callers can expect to receive a re-entrant callback if the
future is already safe and flush/wait is invoked.

Fixes: http://tracker.ceph.com/issues/15364
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon: warn if 'sortbitwise' flag is not set and no legacy OSDs are present

If the user has upgraded, they should set sortbitwise.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8418 from liewegas/wip-blk-probe

os/ObjectStore: make device uuid probe output something friendly

Merge pull request #8428 from liewegas/wip-rest-mds

ceph-rest-api: fix fs/flag/set

Merge pull request #7981 from liewegas/wip-14364

osdc/Objecter: fix narrow race with tid assignment

Reviewed-by: Josh Durgin <jdurgin@redhat.com>