client: drop prefix from int types

Signed-off-by: John Coyle dx9err@gmail.com

Merge pull request #6241 from JevonQ/fix-cephfs-doc

doc: remove toctree items under Create CephFS

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: remove toctree items under Create CephFS

The two toctree items under Create CephFS should be changed into hyper links.

Signed-off-by: Jevon Qiao <qiaojianfeng@unitedstack.com>

Merge branch 'infernalis'

doc/release-notes: 9.1.0

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6180 from jcsp/wip-tell-sessions

New filtered MDS tell commands for sessions

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #6108 from ceph/wip-dang-cmake

More CMake package check fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6204 from jcsp/wip-client-mark-down

client: don't mark_down on command reply

Merge pull request #6226 from xiaoxichen/osd_misc

osd: fix debug message in osd::is_healthy

Reviewed-by: Kefu Chai <kchai@redhat.com>

Fix debug message in osd::is_healthy

Display the actual vaule of osd_heartbeat_min_healthy_ratio instead
of 1/3 (default value).

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Merge pull request #5065 from wonzhq/optracker-histogram

common: OpTracker age histogram calculation is not correct

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6220 from dreamhost/wip-pybind-rados-omap-binary

pybind/rados: Fix binary omap values.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

pybind/rados, get_omap_vals: Fix pydoc type.

pydoc type for filter_prefix did not match code.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

test: pybind/test_rados: add binary data.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

pybind/rados: Fix binary omap values.

The prior code caused binary omap values to be discarded. This fixes
them to use the same model as the xattr iterator, and correctly return
binary data as python strings, eg:
'object_prefix': '\x15\x00\x00\x00rbd_data.449d2ae8944a'

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

Merge pull request #6193 from athanatos/wip-13240

ReplicatedPG::maybe_handle_cache: do not promote before checking full

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6190 from SUSE/wip-drop-my-conf-opt

ceph.spec.in: drop MY_CONF_OPTS

Reviewed-by: Ken Dreyer, <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #6183 from ceph/wip-fix-selinux-man-page

Fix selinux man page

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6191 from SUSE/wip-concentrate-buildrequires

ceph.spec.in: move BuildRequires out of subpackages

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #6184 from dillaman/wip-13372-infernalis

librbd: invalidate object map on error even w/o holding lock

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: fix wrong etag calculation during POST on S3 bucket.

Fixes: #11241
Backport: infernalis, hammer, firefly
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
(cherry picked from commit 742906ab5ee644d5932570bf95342697b673a498)

Merge pull request #6030 from rzarzynski/wip-11241

rgw: fix wrong etag calculation during POST on S3 bucket.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

LibRBD: Adjust correct op latency scope

Op latency need to cover from queuing to finishing op.

Signed-off-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Jason Dillaman <jdillaman@redhat.com>

librados_test_stub: prevent interleaving of operations

It was possible for unrelated ops to interleave within a different
transaction for a given object.

Fixes: #13313
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6044 from ceph/wip-12958

rgw: orphan tool shouldn't clean up head objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: orphan tool shouldn't clean up head objects

Fixes: #12958
Head objects are mutable, so removing them can race with object removal
and a later recreation, so we might end up cleaning them up when we don't
need to.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6187 from SUSE/wip-13317-infernalis

doc: remove mention of --lazy-remove from radosgw-admin manpage

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6211 from SUSE/wip-dead-bool

CephxServiceHandler.cc: fix get_auth conditional

Reviewed-by: Kefu Chai <kchai@redhat.com>

CephxServiceHandler.cc: fix get_auth conditional

The KeyServer class has a public method get_auth() that returns a boolean
value. This value is being checked here - fix the conditional so it triggers
when get_auth() returns false.

http://tracker.ceph.com/issues/9756 References: #9756

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6173 from dachary/wip-13355-docker-test

tests: allow docker-test.sh to run under root

Reviewed-by: Nathan Cutler <ncutler@suse.cz>

Merge pull request #6194 from dillaman/wip-13220-infernalis

rbd-replay-prep and rbd-replay improvements

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ReplicatedPG::maybe_handle_cache: do not promote before checking full

Otherwise, ops with need_promote() can order past ops without it
causing an rgw write sequence like:

writefull 0~0
writefull 0~512kb
write 512kb~512kb

to reorder to

writefull 0~0 -- delayed: full
writefull 0~512kb -- delayed: full
write 512kb~512kb -- promote and execute
writefull 0~0 -- execute
writefull 0~512kb -- execute

Fixes: 13240
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #6206 from rzarzynski/wip-rgw-manifest-skip

rgw: add support for getting Swift's DLO without manifest handling

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

tests: removed obsolete rbd_replay test cases

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

ceph-dencoder: new rbd_replay trace file types

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay: added version control to trace output file

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: add support for skipping manifest parsing during GET on Swift object.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

client: don't mark_down on command reply

I guess this was handy when issuing single commands
from the CLI, but it breaks things badly when
trying to issue commands from a client
that's also going to carry on and do client IO.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #6200 from ceph/wip-13364

mds/Session: use projected parent for auth path check

Merge pull request #6116 from jcsp/wip-cephfs-direntry

pybind/cephfs: fix DirEntry helpers

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6195 from dachary/wip-release-notes

release-notes: draft v0.94.4 release notes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6188 from liewegas/wip-13395

os/FileStore: kludge sloppy hammer temp objects into temp collection

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

mds/Session: use projected parent for auth path check

In particular, a file's current path might indicate no parent
because the initial link into the tree isn't committed yet.

Fixes: #13364
Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_libcephfs: parse env properly (access)

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_libcephfs: parse CEPH_ARGS properly

This has to happen after the config parse.

Signed-off-by: Sage Weil <sage@redhat.com>

release-notes: draft v0.94.4 release notes

Signed-off-by: Loic Dachary <ldachary@redhat.com>

rbd-replay-prep: added --verbose command line option

This new command-line will dump all the processed events as they
are extracted from the trace file.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay-prep: stream events to the prep file

Avoid building the entire prep event history in memory before
attempting to write the prepped trace file.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay-prep: simplify IO dependency calculation

Only track read-after-write and write-after-write IO dependencies
via the associated write completions.  All IO events after a write
completion are considered to be dependent and can be pruned down
to at most the number of concurrent IOs.  This reduces the prep
time from a simple 'rbd bench-write' from over 4 hrs down to seconds.

Fixes: #13378, #13384
Backport: hammer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay: improve error messages

Fixes: #13221
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay-prep: support new read/write APIs

Added support for read2, write2, aio_read2, and aio_write2 APIs.

Fixes: 13325
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay-prep: avoid using assert statements on user inputs

Fixes: #13220
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge branch 'infernalis'

Merge pull request #6170 from jcsp/wip-spurious-warning

mds: avoid emitting cap warnings before evicting session

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #6128 from jcsp/wip-snapserver-crash

mds: fix SnapServer crash on deleted pool

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'wip-yrf-fix1' of git://github.com/H3C/ceph into infernalis

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'wip-13256' of git://github.com/ukernel/ceph into infernalis

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

qa: avoid using sudo in fsstress

This test required root in order to copy its built
binary into /usr (presumably to avoid rebuilding it).

That's not really a good thing anyway because there's
no guarantee that a binary in that path is the binary
we wanted, so just run the thing straight out of /tmp.  The
build is really quick anyway.

Signed-off-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #6186 from ceph/wip-12913-2

cls_rgw: fix bucket listing when dealing with invisible entries

os/FileStore: kludge sloppy hammer temp objects into temp collection

When we are running with a mixed hammer cluster, hammer primaries
will generate temp object names that are sloppy.  Make sure we still
put them into the temp collection.

Note that this isn't a problem on write because the primary (hammer)
OSD generated the transaction and explicitly specified a temp
collection; it's only transactions we do on our own with the sloppy
temp ghobject_t that trip over this.

Fixes: #13395
Signed-off-by: Sage Weil <sage@redhat.com>

ceph.spec.in: move python-sphinx BuildRequires to the right section

These belong in the section for distro-conditional dependencies.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: move BuildRequires out of subpackages

Since the main package and all the subpackages are built in the
same environment, concentrate all the BuildRequires in the main
package.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: drop MY_CONF_OPTS

http://tracker.ceph.com/issues/13402 Fixes: #13402

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6185 from liewegas/wip-13382

osd/PG: fix generate_past_intervals

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #6182 from liewegas/wip-13379

mon: do not remove proxied sessions

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

doc: remove mention of --lazy-remove from radosgw-admin manpage

"The --lazy-remove flag has been removed over 3 years ago. Docs need to be
updated."

http://tracker.ceph.com/issues/13317 Fixes: #13317

Signed-off-by: Nathan Cutler <ncutler@suse.com>

osd/PG: fix generate_past_intervals

We may be only calculating older past intervals and have a valid
history.same_interval_since value, in which case the local
same_interval_since value will end at the newest old interval we had to
generate.

This was introduced by 70316541bbb115d9a35954bfba373cf1dc084b7e.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: fix math error in firefly notes

Reported-by: Christopher Kunz <chris@filoo.de>
Signed-off-by: Sage Weil <sage@redhat.com>

librbd: invalidate object map on error even w/o holding lock

If there is a deep bug that prevents the object map from owning
the lock before objects are updated, flag the object map as invalid
regardless.

Fixes: #13372
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

selinux: Fix man page location

The SELinux man page was previously located in two places and the man
page that was supposed to be updated when rgw selinux changes were
proposed did not get updated properly. Fixing this by moving
selinux/ceph_selinux.8 to man/ceph_selinux.8. Also, populate EXTRA_DIST
with ceph_selinux.8.

Signed-off-by: Boris Ranto <branto@redhat.com>

man/Makefile-server.am: conditionalize make ceph_selinux manpage

http://tracker.ceph.com/issues/13353 Fixes: #13353

Modified-by: Boris Ranto <branto@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6171 from SUSE/wip-infernalis-obs

Get infernalis to build in openSUSE Build Service

Reviewed-by: Boris Ranto <branto@redhat.com>

mon: do not remove proxied sessions

A proxied session (see handle_forward) isn't registered, so it doesn't
need remove_session.  Moreover, s->con is null, so it will crash in
remove_session.

Fixes: #13379
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6181 from athanatos/wip-13323

ReplicatedPG: consider IGNORE_CACHE for all maybe_handle_cache calls

Reviewed-by: Sage Weil <sage@redhat.com>

test: add TestSessionFilter

Tests for SessionFilter, the class filters
allows session listing and eviction in the
new MDS tell commands.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: implement filtered "session ls" tell command

So that one can efficiently query client session
state remotely.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: call through to MDSRank in handle_command

...so that we can implement things like
session ls in MDSRankDispatcher.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: add auth_name to session_info_t

Record the name of the entity who's authorisation
credentials were used during connection.  This
allows us to refer to it later when listing or
evicting sessions, for the user's benefit.

Signed-off-by: John Spray <john.spray@redhat.com>

ceph.spec.in: remove comments regarding ceph UID/GID in SUSE

It is possible that the ceph user/group will not have fixed UID/GID in SUSE.
Instead, it is possible that the ceph package will depend on a separate package
whose sole purpose will be to create the ceph user/group if they do not exist.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: enable OBS post-build-checks to find systemd-tmpfiles

The openSUSE Build Service runs a number of "post-build checks" after the RPMs
have been generated. One of these tests the RPM scriptlets for idempotence.
Without this line in the specfile, the check fails on SLE_12 because it cannot
find the systemd-tmpfiles binary.

http://tracker.ceph.com/issues/13375 Fixes: #13375

Signed-off-by: Nathan Cutler <ncutler@suse.com>

etc/sysconfig/ceph: add CEPH_AUTO_RESTART_ON_UPGRADE

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: Standardize systemd preun and postun scripts

Currently, the main ceph package and the ceph-radosgw behave
differently on upgrade. This commit unifies their behavior
to the following:

On package removal, disable and stop all related systemd units.

On package upgrade, do nothing unless there is a file /etc/sysconfig/ceph
containing a parameter CEPH_AUTO_RESTART_ON_UPGRADE. If parameter is set
to "yes", restart the systemd units iff they are running.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: fix for out-of-memory errors in OBS

Add "--param ggc-min-expand=20 --param ggc-min-heapsize=32768"
to RPM_OPT_FLAGS, ensuring gcc does not add debug symbols and is
more aggressive about garbage collection.

Thanks to Berthold Gunreben for debugging this issue.

http://tracker.ceph.com/issues/13352 Fixes: #13352

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: Use _smp_mflags instead of _NPROCESSORS_ONLN

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Revert "install-deps.sh: use %bcond_with selinux on SLE/openSUSE"

This reverts commit 1df2cc2bf625a1f65ef4968d9cf313cc8ee9acc5.

ceph.spec.in: selinux by default on RHEL/Fedora, non-default on SUSE

Surround it with an appropriate conditional.

http://tracker.ceph.com/issues/13324 Fixes: #13324

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: use %fillup_only macro in SLE/openSUSE

SUSE does not allow /etc/sysconfig/ceph to be owned by the ceph
package.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: distro-conditional groupadd and useradd

The -o option to groupadd/useradd is not recognized in SLE12. For
SLE/openSUSE, follow the openSUSE packaging guidelines.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: move systemd-tmpfiles stuff to ceph-common

Since the ceph user and group are created in the ceph-common
%pre scriptlet, the systemd-tmpfiles --create command needs
to run in that package.

tracker.ceph.com/issues/13346 Fixes: #13346

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: remove SUSE-specific curses hack

Unclear what this was supposed to fix.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: lttng in SLES12 only

Over in the SUSE sector, we are trying to enable the SLE_12 and openSUSE_13.2
build targets. The lttng/babeltrace stuff is currently available only in
SLE_12.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: drop %insserv_prereq (obsoleted by systemd)

The %post, etc. scriptlets are no longer calling %insserv, et al.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: fix boost-random build dependency for SLE/openSUSE

In Fedora, RHEL and CentOS there is a package called boost-random.
In SLE/openSUSE, there is no such package.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc/release-notes: initial v9.1.0 notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6110 from ceph/wip-13239-infernalis

rgw: set default value for env->get() call

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6142 from ceph/wip-13061

ceph.spec.in: Do not always restart the daemons on upgrades/removal

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #6105 from liewegas/wip-13262

mon: ignore messages from disconnected clients; clean up session checks

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #6176 from liewegas/wip-13283

osdc/Objecter: send FULL_TRY and FULL_FORCE ops despite full flag

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

CMake - Fix perftools checks

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

CMake - fix yasm check

The yasm check was incorrectly including yasm-build assembler files even
when yasm wasn't present.  Fix this, and make the output nicer.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>