systemd: Use the same restart limits as upstart

Currently, the systemd daemons are not restarted on failure. This patch
adds this functionality and sets the defaults to those defined in
upstart. This resolves to 3 fails per 30 minutes for osd, mon and mds
and 5 fails per 30 seconds for radosgw.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #8498 from liewegas/wip-bluestore

os/bluestore: fix shortened bluefs paths in env mirror case

os/bluestore: fix shortened bluefs paths in env mirror case

If we are mirroring the rocksdb env, don't shorten the
paths.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8556 from liewegas/wip-ext4

doc: recommend against ext4

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.cz>

Merge pull request #8574 from tchaikov/wip-readable-test-fix

test/encoding/readable: use [ for "test" not ((

Reviewed-by: Erwan Velu <erwan@redhat.com>

Merge pull request #8567 from cbodley/wip-rgw-period-pull-url

radosgw-admin: allow 'period pull --url' without full multisite config
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

test/encoding/readable: use [ for "test" not ((

and use '&&' instead, easier to understand.

this also fixes following error in the ubuntu gitbuilder

error: Added files:
+ cat .git/added-files
src/0

where sh points to dash.

Signed-off-by: Kefu Chai <kchai@redhat.com>

radosgw-admin: 'realm pull' only accepts --url

'realm pull' is defined as a 'raw storage op', so cannot use the
--remote argument to refer to endpoints from the period or local
zonegroup

refactored send_to_url() slightly to make it easier to call directly

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: allow 'period pull --url' without full multisite config

to support 'period pull --remote', radosgw-admin needs a complete
multisite configuration to look up remote endpoints by name

however, 'period pull' will also accept a --url argument, which doesn't
require a working configuration. this patch splits the handling of these
two cases, so that 'period pull --url' can succeed when run on a cluster
that hasn't yet been fully configured

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #8520 from yuyuyu101/wip-15440

msg/async: AsyncConnection: avoid is_connected require connection's lock

Merge pull request #8559 from liewegas/wip-15643

rgw: fix unsafe c_str() usage

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc/configuration/filesystem-recommendations: recommend against ext4

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: mention ext4 in the release notes

Signed-off-by: Sage Weil <sage@redhat.com>

doc/start/os-recommendations: drop ancient 'syncfs' note

...and with it it's mention of ext4.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/rados/configuration: minor cleanup of inline xattr description

- whitespace
- it's not omap

Signed-off-by: Sage Weil <sage@redhat.com>

rgw/rgw_rados: use to_str() instead of c_str()

bufferlist::c_str() doesn't guarantee null termination.

Fixes: http://tracker.ceph.com/issues/15463
Signed-off-by: Sage Weil <sage@redhat.com>

buffer: add list::to_str()

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #8552 from tchaikov/wip-15426

PG: set epoch_created and parent_split_bits for child pg

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8521 from stiopaa1/checkgenerated

check-generated.sh: can't source bash from sh

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge branch 'jewel'

Merge remote-tracking branch 'origin/greg-fs-testing-411' into jewel

doc: do not include ext4 as an example

Signed-off-by: Sage Weil <sage@redhat.com>

doc/8/ceph-deploy: do not include ext4 as an example value

Signed-off-by: Sage Weil <sage@redhat.com>

doc: list ext4 last

Signed-off-by: Sage Weil <sage@redhat.com>

doc/dev: remove ancient filestore-filesystem-compat doc

This describes a problem we fixed years ago.

Signed-off-by: Sage Weil <sage@redhat.com>

PG: set epoch_created and parent_split_bits for child pg

if we have a wrong "created", OSD receiving an osd_pg_create message
could fail to project its history and crash. a bad split_bits could
mislead the monitor and osd, and hence cause problems.
* set child.info.history.epoch_created to osdmap.get_epoch() instead of
  using the parent's epoch_created
* set child.info.stats.parent_split_bits to the calculated split_bits
  instead using the parent's split_bits

Fixes: http://tracker.ceph.com/issues/15426
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8544 from xiexingguo/xxg-wip-15464

test: fix ut test failure caused by lfn change

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: fix ut test failure caused by lfn change

"https://github.com/ceph/ceph/pull/8496/commits/755c685f2d09c36b53bc39589a77fa73faba5149"
change the lfn_parse_object_name() return type from bool to int but does not
change the relevant test correspondingly.

Fixes: http://tracker.ceph.com/issues/15464
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge branch 'jewel-15402' of git://github.com/ukernel/ceph into greg-fs-testing

Do not ignore standby_for_fscid when selecting a standby.

https://github.com/ceph/ceph/pull/8536

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Fixed ceph-common install.

Fixes: #15438
Signed-off-by: John Wilkins <jowilkin@redhat.com>

librbd: fixed bug in disabling non-primary image mirroring

librbd: fixed bug in disabling non-primary image mirroring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8496 from athanatos/wip-15312

os/filestore: force lfn attrs to be written atomically, restructure name length limits

mds: take standby_for_fscid into account in FSMap::find_unused()

Fixes: http://tracker.ceph.com/issues/15402
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #8529 from oritwas/wip-rgw-fix-zone-set

radosgw-admin: allow

librbd: Fixed bug in disabling non-primary image mirroring

It is only possible to remove image from rbd_mirroring object if the
mirroring image state is set to DISABLING.

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #8508 from JiYou/fix-ceph-disk

ceph-disk: do not always fail when re-using a partition

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'gh/jewel'

ceph-disk: fix PrepareData.set_type() args should be class member.

Signed-off-by: You Ji <youji@ebay.com>

ceph-disk: fix spelling mistake geattr to getattr.

Signed-off-by: You Ji <youji@ebay.com>

ceph-disk: fix lockbox set_or_create_partition()

Signed-off-by: You Ji <youji@ebay.com>

Merge pull request #8507 from kmroz/wip-rgw-admin-cleanup

rgw-admin: remove unused iterator and fix error message

radosgw-admin: fix name checking

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: allow setting zone when there is no realm

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8522 from swamireddy/10.1.1-mailmap

mailmap update

Reviewed-by: Loic Dachary <ldachary@redhat.com>

AsyncConnection: avoid is_connected require connection's lock

Fixes: http://tracker.ceph.com/issues/15440
Signed-off-by: Haomai Wang <haomai@xsky.com>

check-generated.sh: can't source bash from sh

'source' is not sh command and executing bash commands in sh won't
work. We could try to replace it with:
'. ../qa/workunits/ceph-helpers.sh'
but the above file uses bash.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #8519 from dachary/wip-xenial

tests: add Ubuntu 16.04 xenial dockerfile

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8517 from kjurka/rgw-admin-doc-region-list

doc: rgw admin uses "region list" not "regions list"

Merge pull request #8518 from wjin/fix

crush: fix typo

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #8462 from xiexingguo/xxg-wip-scrub

osd: fix two scrub relevant issues

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8485 from liewegas/wip-fuse-log

ceph-fuse: rotate log file

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #8487 from liewegas/wip-copy-from-fail-cleanup

osd: clean up temp object if copy-from fails

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8486 from liewegas/wip-14695

librados: Revert "rados: Add new field flags for ceph_osd_op.copy_get."

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #7956 from xiexingguo/xxg-wip-fixfstat

common: buffer: put a guard for stat() syscall during read_file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8516 from runsisi/wip-fix-ret

journal: fix final result for JournalTrimmer::C_RemoveSet

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tests: add Ubuntu 16.04 xenial dockerfile

Signed-off-by: Loic Dachary <loic@dachary.org>

crush: fix typo

Signed-off-by: Wei Jin <wjin.cn@gmail.com>

doc: rgw admin uses "region list" not "regions list"

journal: fix final result for JournalTrimmer::C_RemoveSet

if the first journal object got removed successfully while later objects
failed then the final result should be set to fail

Signed-off-by: runsisi <runsisi@zte.com.cn>

test/rados/misc.cc: add long locator key and namespace tests

Signed-off-by: Samuel Just <sjust@redhat.com>

LFNIndex: use chain_getxattr_buf

LFN attrs can be longer than these hard coded limits, let's just do
buffer doubling everywhere.

Signed-off-by: Samuel Just <sjust@redhat.com>

chain_xattr: add chain_getxattr_buf

Otherwise callers need to implement buffer doubling in a lot of cases,
which is error prone.

Signed-off-by: Samuel Just <sjust@redhat.com>

LFNIndex::lfn_parse_object_name: return int rather than bool

All current callers (except list_objects) already expect
ret < 0 for error.  Adjust list_objects to skip on EINVAL
instead of 0.

Signed-off-by: Samuel Just <sjust@redhat.com>

LFNIndex::list_objects: lfn_translate does not set errno

Signed-off-by: Samuel Just <sjust@redhat.com>

FileStore::set_xattr_limits_via_conf: add warning if max xattr size smaller than max name

Signed-off-by: Samuel Just <sjust@redhat.com>

test/objectstore/chain_xattr.cc: add test for ensure_single_attr

Signed-off-by: Samuel Just <sjust@redhat.com>

LFNIndex: ensure that lfn and directory attrs are written atomically

We leave skip_chain_cleanup = false since the object may have been
written using code that split the lfn attr over multiple attrs.

Signed-off-by: Samuel Just <sjust@redhat.com>

chain_[f]getxattr: always use size, no reaon to consider CHAIN_XATTR_MAX_BLOCK_LEN

Users of setxattr can now set attrs larger than that size.

Signed-off-by: Samuel Just <sjust@redhat.com>

chain_xattr: s/onechunk/skip_chain_cleanup, add ensure_single_attr

onechunk was a deceptive name since it didn't actually ensure that
a single chunk would be used.  Rename to ensure_single_attr.  Also,
add a parameter to ensure that we use a single attribute.  We need
these to be distinct since we have LFN attrs which have been split
over 254 byte xattrs since after hammer which we need to correctly
clean up.  However, we need to ensure going forward that those attrs
are never split over more than one chunk -- it's not atomic.
Further, skip_chain_cleanup should imply ensure_single_attr, so make
them template params and add a static check to make sure we don't
mess this up in the future.

Signed-off-by: Samuel Just <sjust@redhat.com>

os/,osd/: restructure the rados name length check

Enforce locator length vs the max name length and also
introduce a namespace length limit.

In addition to these checks, also pass the head object to the
ObjectStore implementation to validate.  This allows LFNIndex to account
for the idiosyncracies of its filename escaping and for different xattr
value max sizes.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #8453 from cbodley/wip-rgw-sync-master-down

rgw: retry RGWRemoteMetaLog::read_log_info() while master is down
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

ceph-disk: fix set_data_partition() when data is partition.

Signed-off-by: You Ji <youji@ebay.com>

rgw-admin: fix period delete error message

Signed-off-by: Karol Mroz <kmroz@suse.com>

rgw-admin: remove unused iterator

Signed-off-by: Karol Mroz <kmroz@suse.com>

Merge pull request #8434 from theanalyst/rgw/typo-fix

rgw: fix a typo in error message

Merge pull request #8481 from weiqiaomiao/wq

rgw: the map 'headers' is assigned a wrong value
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8488 from oritwas/wip-15318

rgw: fix problem deleting objects begining with double underscores

rbd-mirror: fixed infinite loop when disabling image mirroringReviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: fixed infinite loop when disabling image mirroring

rbd-mirror: fixed bug that caused infinite loop when disabling image mirroring

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #8501 from dachary/wip-releases

releases: update the v10.1.0 release link

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Luo Kexue name normalization

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Ning Yao affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Vitja Makarov affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

release-notes: v10.1.1 release notes

Signed-off-by: Loic Dachary <loic@dachary.org>

release-notes: v10.1.1 release notes (draft)

cd src/script
virtualenv v
source v/bin/activate
pip install githubpy GitPython requests
GITHUB_ACCESS_TOKEN=xxxxxxx ./ceph-release-notes \
      -r tags/v10.1.0..tags/v10.1.1 $(git rev-parse --show-toplevel)

Signed-off-by: Loic Dachary <loic@dachary.org>

mailmap: sunspot affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Qinghua Jin affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Richard W.M. Jones affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Ira Cooper affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Star Guo affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Xiaoxi Chen affiliation

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

mailmap: Jenkins name normalization

Signed-off-by: M Ranga Swami Reddy <swamireddy@gmail.com>

releases: firefly was EOL december 2015

Signed-off-by: Loic Dachary <loic@dachary.org>

releases: update the v10.1.0 release link

And add the v10.1.1 release notes as well.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #8500 from linuxbox2/ldap-include

rgw_ldap: make ldap.h inclusion conditional

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge branch 'wip-fuse-log' of git://github.com/liewegas/ceph into greg-fs-testing

Enable log rotation: handle SIGHUP in ceph_fuse.cc and add it to the conf.

https://github.com/ceph/ceph/pull/8485

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'wip-layout-snafu-followup' of git://github.com/jcsp/ceph into greg-fs-testing

Be more transparent about several failure modes and do more validation on replay

https://github.com/ceph/ceph/pull/8468

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'jewel-fs-pool-perm' of git://github.com/ukernel/ceph into greg-fs-testing

Extend the existing pool access checking to include specific RADOS namespacse.

https://github.com/ceph/ceph/pull/8444

Reviewed-by: John Spray <john.spray@redhat.com>

Merge branch 'xxg-wip-mds' of git://github.com/xiexingguo/ceph into greg-fs-testing

Fix several message leaks and handle clock skew more politely.

https://github.com/ceph/ceph/pull/8408

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'renhw-wip-mds-clean_up_admin_socket' of git://github.com/renhwsky/ceph into greg-fs-testing

Clean up some admin socket commands on shutdown.

https://github.com/ceph/ceph/pull/8483

Reviewed-by: Greg Farnum <gfarnum@redhat.com>