Fix debug message in osd::is_healthy

Display the actual vaule of osd_heartbeat_min_healthy_ratio instead
of 1/3 (default value).

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Merge pull request #5065 from wonzhq/optracker-histogram

common: OpTracker age histogram calculation is not correct

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6030 from rzarzynski/wip-11241

rgw: fix wrong etag calculation during POST on S3 bucket.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6173 from dachary/wip-13355-docker-test

tests: allow docker-test.sh to run under root

Reviewed-by: Nathan Cutler <ncutler@suse.cz>

Merge pull request #6206 from rzarzynski/wip-rgw-manifest-skip

rgw: add support for getting Swift's DLO without manifest handling

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: add support for skipping manifest parsing during GET on Swift object.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #6200 from ceph/wip-13364

mds/Session: use projected parent for auth path check

Merge pull request #6195 from dachary/wip-release-notes

release-notes: draft v0.94.4 release notes

Reviewed-by: Sage Weil <sage@redhat.com>

mds/Session: use projected parent for auth path check

In particular, a file's current path might indicate no parent
because the initial link into the tree isn't committed yet.

Fixes: #13364
Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_libcephfs: parse env properly (access)

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_libcephfs: parse CEPH_ARGS properly

This has to happen after the config parse.

Signed-off-by: Sage Weil <sage@redhat.com>

release-notes: draft v0.94.4 release notes

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge branch 'infernalis'

Merge branch 'wip-yrf-fix1' of git://github.com/H3C/ceph into infernalis

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'wip-13256' of git://github.com/ukernel/ceph into infernalis

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

qa: avoid using sudo in fsstress

This test required root in order to copy its built
binary into /usr (presumably to avoid rebuilding it).

That's not really a good thing anyway because there's
no guarantee that a binary in that path is the binary
we wanted, so just run the thing straight out of /tmp.  The
build is really quick anyway.

Signed-off-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #6186 from ceph/wip-12913-2

cls_rgw: fix bucket listing when dealing with invisible entries

Merge pull request #6185 from liewegas/wip-13382

osd/PG: fix generate_past_intervals

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #6182 from liewegas/wip-13379

mon: do not remove proxied sessions

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

osd/PG: fix generate_past_intervals

We may be only calculating older past intervals and have a valid
history.same_interval_since value, in which case the local
same_interval_since value will end at the newest old interval we had to
generate.

This was introduced by 70316541bbb115d9a35954bfba373cf1dc084b7e.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: fix math error in firefly notes

Reported-by: Christopher Kunz <chris@filoo.de>
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6171 from SUSE/wip-infernalis-obs

Get infernalis to build in openSUSE Build Service

Reviewed-by: Boris Ranto <branto@redhat.com>

mon: do not remove proxied sessions

A proxied session (see handle_forward) isn't registered, so it doesn't
need remove_session.  Moreover, s->con is null, so it will crash in
remove_session.

Fixes: #13379
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6181 from athanatos/wip-13323

ReplicatedPG: consider IGNORE_CACHE for all maybe_handle_cache calls

Reviewed-by: Sage Weil <sage@redhat.com>

ceph.spec.in: remove comments regarding ceph UID/GID in SUSE

It is possible that the ceph user/group will not have fixed UID/GID in SUSE.
Instead, it is possible that the ceph package will depend on a separate package
whose sole purpose will be to create the ceph user/group if they do not exist.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: enable OBS post-build-checks to find systemd-tmpfiles

The openSUSE Build Service runs a number of "post-build checks" after the RPMs
have been generated. One of these tests the RPM scriptlets for idempotence.
Without this line in the specfile, the check fails on SLE_12 because it cannot
find the systemd-tmpfiles binary.

http://tracker.ceph.com/issues/13375 Fixes: #13375

Signed-off-by: Nathan Cutler <ncutler@suse.com>

etc/sysconfig/ceph: add CEPH_AUTO_RESTART_ON_UPGRADE

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: Standardize systemd preun and postun scripts

Currently, the main ceph package and the ceph-radosgw behave
differently on upgrade. This commit unifies their behavior
to the following:

On package removal, disable and stop all related systemd units.

On package upgrade, do nothing unless there is a file /etc/sysconfig/ceph
containing a parameter CEPH_AUTO_RESTART_ON_UPGRADE. If parameter is set
to "yes", restart the systemd units iff they are running.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: fix for out-of-memory errors in OBS

Add "--param ggc-min-expand=20 --param ggc-min-heapsize=32768"
to RPM_OPT_FLAGS, ensuring gcc does not add debug symbols and is
more aggressive about garbage collection.

Thanks to Berthold Gunreben for debugging this issue.

http://tracker.ceph.com/issues/13352 Fixes: #13352

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: Use _smp_mflags instead of _NPROCESSORS_ONLN

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Revert "install-deps.sh: use %bcond_with selinux on SLE/openSUSE"

This reverts commit 1df2cc2bf625a1f65ef4968d9cf313cc8ee9acc5.

ceph.spec.in: selinux by default on RHEL/Fedora, non-default on SUSE

Surround it with an appropriate conditional.

http://tracker.ceph.com/issues/13324 Fixes: #13324

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: use %fillup_only macro in SLE/openSUSE

SUSE does not allow /etc/sysconfig/ceph to be owned by the ceph
package.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: distro-conditional groupadd and useradd

The -o option to groupadd/useradd is not recognized in SLE12. For
SLE/openSUSE, follow the openSUSE packaging guidelines.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: move systemd-tmpfiles stuff to ceph-common

Since the ceph user and group are created in the ceph-common
%pre scriptlet, the systemd-tmpfiles --create command needs
to run in that package.

tracker.ceph.com/issues/13346 Fixes: #13346

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: remove SUSE-specific curses hack

Unclear what this was supposed to fix.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: lttng in SLES12 only

Over in the SUSE sector, we are trying to enable the SLE_12 and openSUSE_13.2
build targets. The lttng/babeltrace stuff is currently available only in
SLE_12.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: drop %insserv_prereq (obsoleted by systemd)

The %post, etc. scriptlets are no longer calling %insserv, et al.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: fix boost-random build dependency for SLE/openSUSE

In Fedora, RHEL and CentOS there is a package called boost-random.
In SLE/openSUSE, there is no such package.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc/release-notes: initial v9.1.0 notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6110 from ceph/wip-13239-infernalis

rgw: set default value for env->get() call

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6142 from ceph/wip-13061

ceph.spec.in: Do not always restart the daemons on upgrades/removal

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #6105 from liewegas/wip-13262

mon: ignore messages from disconnected clients; clean up session checks

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #6176 from liewegas/wip-13283

osdc/Objecter: send FULL_TRY and FULL_FORCE ops despite full flag

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osdc/Objecter: send FULL_TRY and FULL_FORCE ops despite full flag

If a request has the FULL_TRY or FULL_FORCE flag, send it despite a full
pool or cluster.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: allow ping through despite synch/quorum status, with session

Attach sessions to ping messages, too.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: drop ops on closed sessions early

We will throw it out in dispatch_op() anyway.  Remove the ancient (~2009)
reuse_caps logic too, which doesn't make any sense nowadays anyway.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: clean up _ms_dispatch

Lots of cruft here.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: all MonOpRequests have an attached Session

...except in dispatch, where we initially set it up.

Signed-off-by: Sage Weil <sage@redhat.com>

mon: drop any ops from closed sessions in dispatch_op

If we've closed the session we shouldn't waste our time on this op.

Fixes: #13262 (we should not add a subscription to a disconnected session)
Signed-off-by: Sage Weil <sage@redhat.com>

mon: always set up session; move waitlist logic

Unconditionally attach a Session, so that when we push an op back through
dispatch_op it isn't in a weird broken state.

Move the waitinglist check to the bottom.  Use s->global_id == 0 to tell
whether a client has authenticated, and zap sessions based on that.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

mon: fix msg leak in resend_routed_requests

MForward takes its own ref so we can drop ours.

Signed-off-by: Sage Weil <sage@redhat.com>

Mon: Fix decoded message leak when this monitor is leader

Fix #13109
Signed-off-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6139 from liewegas/wip-13283

librados: add FULL_TRY flag; EDQUOT != ENOSPC

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: allow docker-test.sh to run under root

Do not attempt to create a user if it is root because it already exists.

http://tracker.ceph.com/issues/13355 Fixes: #13355

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: remove fedora docker files

They are not used (missing version number at the end of the directory).

Signed-off-by: Loic Dachary <ldachary@redhat.com>

librados: document new flag

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6169 from rzarzynski/wip-13326

rgw: improve handling of already removed buckets in object expirer.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6168 from SUSE/wip-13333-infernalis

ceph.spec.in: correctly declare systemd dependency for SLE/openSUSE

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

ceph.spec.in: correctly declare systemd dependency for SLE/openSUSE

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 7b1882f87509092380d11ec68b504a7bc1f77830)

osd/ReplicatedPG: exempt MDS from the failsafe check, too

This mimics the prior behavior.  We probably want to change this at some
point!

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: improve handling of already removed buckets in object expirer.

Fixes: #13326
Backport: infernalis
Signed-off-by: Radoslaw Zarzynski <rzarzynski@github.com>

Merge pull request #5907 from dachary/wip-release-notes

release-notes: draft v0.94.4 release notes

Reviewed-by: Sage Weil <sage@redhat.com>

release-notes: draft v0.94.4 release notes

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tools: ceph-release-notes handle multiple issues

* handle the case where a single pull request is associated with
  multiple pull requests
* encapsulate the quest for the original in a function for easier
  maintenance
* add the --verbose flag for debugging

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6096 from liewegas/wip-13260

osd/ReplicatedPG: preserve (some) flags when proxying reads

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6107 from liewegas/wip-13206

osd: init started to 0

Merge pull request #6136 from liewegas/wip-13114

osd/librados: fix notify completion race

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ceph.spec.in: Do not always restart the daemons on removal

There is no need to restart the daemons when removing ceph-selinux
package if SELinux is not enabled.

Fixes: #13061
Signed-off-by: Boris Ranto <branto@redhat.com>

ceph.spec.in: Do not always restart the daemons on upgrades

This patch minimizes the amount of daemon stop/start procedures when
upgrading ceph-selinux package. With this patch, the daemons get
restarted only if SELinux is enabled and the SELinux policy version
changed in the meantime.

Fixes: #13061
Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #6137 from SUSE/wip-13324-infernalis

install-deps.sh: use %bcond_with selinux on SLE/openSUSE

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'jdurgin/wip-13281' into infernalis

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

ReplicatedPG: consider IGNORE_CACHE for all maybe_handle_cache calls

Otherwise, we could erroneously promote a snap after not promoting
the missing head for which we have a cached obc and ssc.

Fixes: #13323
Signed-off-by: Samuel Just <sjust@redhat.com>

install-deps.sh: use %bcond_with selinux on SLE/openSUSE

http://tracker.ceph.com/issues/13324 Fixes: #13324

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph_test_rados_api_aio: test pool full gets EDQUOT when FULL_TRY flag is set

Signed-off-by: Sage Weil <sage@redhat.com>

librados: expose OPERATION_FULL_TRY flag

Allow librados users to opt to receive ENOSPC or EDQUOT when they submit
an operation against a full cluster.  This should only be used if the
librados app can handle those errors gracefully (librbd, for example,
cannot).

Also note that this allows savvy librados users to send delete operations;
they will get either a success or EDQUOT, depending on whether the
operation results in a net drop in space utilization.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: return -EDQUOT instead of -ENOSPC if it is a pool quota

The pool quota error should return EDQUOT intead of ENOSPC.

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: distinguish between multiple notify completions

We may send a notify to the cluster multiple times due to OSDMap
changes.  In some cases, earlier notify attempts may complete with
an error, while later attempts succeed.  We need to only pay
attention to the most-recently send notify's completion.

Do this by making note of the notify_id in the initial ACK (only
present when talking to newer OSDs).  When we get a notify
completion, match it against our expected notify_id (if we have
one) or else discard it.

This is important because in some cases an early notify completion
may be an error while a later one succeeds.

Note that if we are talking to an old cluster we will simply not record a
notify_id and our behavior will be the same as before (we will trust any
notify completion we get).

Fixes: #13114
Signed-off-by: Sage Weil <sage@redhat.com>

osd: reply to notify request with our unique notify_id

The OSD assigns a unique ID to each notify it queues for
processing.  Include this in the reply to the notifier so that
they can match it up with the eventual completions they receive.

This is necessary to distinguish between multiple completions
they may receive if there is PG peering and the notify is resent.
In particular, an earlier notify may return an error when a later
attempt succeeds.

This is forwards and backwards compatible: new clients will make use of
this reply payload but older clients ignore it.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge branch 'wip-mdscap'

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #6114 from SUSE/wip-suse-spec-fixes

ceph.spec.in: correctly declare systemd dependency for SLE/openSUSE

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #6134 from SUSE/wip-13318-infernalis

install-deps.sh: openSUSE-release/sles-release/sled-release are always present

install-deps.sh: openSUSE-release/sles-release/sled-release are always present

http://tracker.ceph.com/issues/13318 Fixes: #13318

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph_test_libcephfs: remove remaining pool name assumptions

Signed-off-by: Sage Weil <sage@redhat.com>

client: clarify setattr forced sync behavior

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/fs/test_auth_caps: superceded by ceph_test_libcephfs

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_libcephfs: fix LibCephFS.OpenLayout test

We can't make assumptions about the name of the data pool.

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_mds_authcap: improve user tests

Signed-off-by: Sage Weil <sage@redhat.com>

mds: drop MAY_CREATE

The check is a no-op.  We already verify the uid/gid combo is valid and
that the dir is writeable with MAY_WRITE.  The new file is always set to
the caller uid:gid.

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDSAuthCap: verify the caller_gid is valid

Verify both the caller uid and gid are a match for the given rule.

Signed-off-by: Sage Weil <sage@redhat.com>

mds: fix chown/chgrp check and tests

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDSAuthCaps: rename args for is_capable

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_libcephfs: add AccessTest.User

Covers read/write/create.

Signed-off-by: Sage Weil <sage@redhat.com>

mds/SessionMap: fix MAY_CREATE check

It's about caller_uid/gid.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_libcephfs: make foo, path tests use unique paths, users

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits: drop bash path tests

ceph_test_libcephfs ones are better.

Signed-off-by: Sage Weil <sage@redhat.com>

mds/SessionMap: fix check_access for stray inodes

Signed-off-by: Sage Weil <sage@redhat.com>

mds/SessionMap: move Session method definitions together

Signed-off-by: Sage Weil <sage@redhat.com>