doc/release-notes: v9.1.0

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6275 from dx9/wip-12406-prefixed-ints

client: drop prefix from ints

client: drop prefix from int types

Signed-off-by: John Coyle dx9err@gmail.com

Merge pull request #6254 from jcsp/wip-cmake-tweaks

cmake tweaks

Reviewed-by: Kefu Chai <kchai@redhat.com>

CMake: add vstart convenience targets

These are for developers like me, who want to
quickly build just the bits they need for
interactive vstart stuff, and don't wait for wait
for a full make.

Signed-off-by: John Spray <john.spray@redhat.com>

CMake: make mon and osd depend on EC plugins

...because the services won't start if their
git version stamps don't match those of the
plugins.  This is for the benefit of developers,
so that we can do "make ceph-mon" and get a
vstart-able result.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #6255 from jcsp/wip-vstart-uids

vstart: set cephfs root uid/gid to caller

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #6256 from YankunLi/patch-2

update radosgw-admin command

update radosgw-admin command

the command "radosgw-admin caps remove" should be "radosgw-admin caps rm"
radosgw-admin:
commands
caps rm                    remove user capabilities

vstart: set cephfs root uid/gid to caller

So that when running cluster as unprivileged user,
can do a ceph-fuse mount and then access it without
going root.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #6221 from athanatos/wip-13434

Wip 13434

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6217 from dillaman/wip-13380-infernalis

cls: new force-promotion flag for class methods

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6230 from liewegas/wip-crushtool-err

mon/OSDMonitor: put crushtool error in log

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6235 from dzafman/wip-13462

osd: Correct the object_info_t::decode() version

Reviewed-by: Samuel Just <sjust@redhat.com>

doc/infernalis: hate hate

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: i hate rst

Signed-off-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

doc/release-notes: final infernalis notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge branch 'infernalis' of jenkins.front.sepia.ceph.com:ceph/ceph into infernalis

doc/release-notes: fix some attributions

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

doc/release-notes: infernalis notable changes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6237 from ceph/wip-13458

rgw: location constraints should return api name

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6241 from JevonQ/fix-cephfs-doc

doc: remove toctree items under Create CephFS

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: remove toctree items under Create CephFS

The two toctree items under Create CephFS should be changed into hyper links.

Signed-off-by: Jevon Qiao <qiaojianfeng@unitedstack.com>

9.1.0

Merge pull request #6245 from liewegas/wip-setuptools

debian/control: python-setuptools is a build dependency

Reviewed-by: Alfredo Deza <adeza@redhat.com>

debian/control: python-setuptools is a build dependency

cd ./ceph-detect-init ; python setup.py build
Traceback (most recent call last):
  File "setup.py", line 23, in <module>
    from setuptools import setup
ImportError: No module named setuptools

Signed-off-by: Sage Weil <sage@redhat.com>

Merge branch 'infernalis'

doc/release-notes: 9.1.0

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6180 from jcsp/wip-tell-sessions

New filtered MDS tell commands for sessions

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #6108 from ceph/wip-dang-cmake

More CMake package check fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

ReplicatedPG::maybe_handle_cache_detail: always populate missing_oid

Signed-off-by: Samuel Just <sjust@redhat.com>

ReplicatedPG::_rollback_to: handle block on full correctly

If we fail to promote the clone due to a full cache tier, we need
to block writes on the head just as if we were waiting on the
promote.  Generally therefore, we block writes on head if we block
an op on a full cache tier.  Add a new blocked set to support this.

Fixes: 13434
Signed-off-by: Samuel Just <sjust@redhat.com>

osd: Correct the object_info_t::decode() version

Caused by: 13b9dc70

Fixes: #13462
Backport: Hammer, Firefly

Signed-off-by: David Zafman <dzafman@redhat.com>

rgw: location constraints should return api name

Fixes: #13458
The S3 location constraints request should return the api name,
not the region name.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

mon/OSDMonitor: put crushtool error in log

Putting it in *ss is useless for mkpool callers who don't return
an error string.  Log it to the mon log, too.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6204 from jcsp/wip-client-mark-down

client: don't mark_down on command reply

Merge pull request #6226 from xiaoxichen/osd_misc

osd: fix debug message in osd::is_healthy

Reviewed-by: Kefu Chai <kchai@redhat.com>

Fix debug message in osd::is_healthy

Display the actual vaule of osd_heartbeat_min_healthy_ratio instead
of 1/3 (default value).

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

ReplicatedPG: allow maybe_handle_cache to return status detailing what happened

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #5065 from wonzhq/optracker-histogram

common: OpTracker age histogram calculation is not correct

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6220 from dreamhost/wip-pybind-rados-omap-binary

pybind/rados: Fix binary omap values.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

pybind/rados, get_omap_vals: Fix pydoc type.

pydoc type for filter_prefix did not match code.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

test: pybind/test_rados: add binary data.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

pybind/rados: Fix binary omap values.

The prior code caused binary omap values to be discarded. This fixes
them to use the same model as the xattr iterator, and correctly return
binary data as python strings, eg:
'object_prefix': '\x15\x00\x00\x00rbd_data.449d2ae8944a'

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

Merge pull request #6193 from athanatos/wip-13240

ReplicatedPG::maybe_handle_cache: do not promote before checking full

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6190 from SUSE/wip-drop-my-conf-opt

ceph.spec.in: drop MY_CONF_OPTS

Reviewed-by: Ken Dreyer, <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

rgw: fix wrong etag calculation during POST on S3 bucket.

Fixes: #11241
Backport: infernalis, hammer, firefly
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
(cherry picked from commit 742906ab5ee644d5932570bf95342697b673a498)

Merge pull request #6183 from ceph/wip-fix-selinux-man-page

Fix selinux man page

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6191 from SUSE/wip-concentrate-buildrequires

ceph.spec.in: move BuildRequires out of subpackages

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #6184 from dillaman/wip-13372-infernalis

librbd: invalidate object map on error even w/o holding lock

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6030 from rzarzynski/wip-11241

rgw: fix wrong etag calculation during POST on S3 bucket.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

LibRBD: Adjust correct op latency scope

Op latency need to cover from queuing to finishing op.

Signed-off-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Jason Dillaman <jdillaman@redhat.com>

librados_test_stub: prevent interleaving of operations

It was possible for unrelated ops to interleave within a different
transaction for a given object.

Fixes: #13313
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cls: new force-promotion flag for class methods

Class methods that use the cls_get_request_origin should not be
proxied to a base tier -- otherwise the origin will reflect the
cache tier instead of the client.

Fixes: #13380
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6044 from ceph/wip-12958

rgw: orphan tool shouldn't clean up head objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: orphan tool shouldn't clean up head objects

Fixes: #12958
Head objects are mutable, so removing them can race with object removal
and a later recreation, so we might end up cleaning them up when we don't
need to.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6187 from SUSE/wip-13317-infernalis

doc: remove mention of --lazy-remove from radosgw-admin manpage

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6211 from SUSE/wip-dead-bool

CephxServiceHandler.cc: fix get_auth conditional

Reviewed-by: Kefu Chai <kchai@redhat.com>

CephxServiceHandler.cc: fix get_auth conditional

The KeyServer class has a public method get_auth() that returns a boolean
value. This value is being checked here - fix the conditional so it triggers
when get_auth() returns false.

http://tracker.ceph.com/issues/9756 References: #9756

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6173 from dachary/wip-13355-docker-test

tests: allow docker-test.sh to run under root

Reviewed-by: Nathan Cutler <ncutler@suse.cz>

Merge pull request #6194 from dillaman/wip-13220-infernalis

rbd-replay-prep and rbd-replay improvements

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ReplicatedPG::maybe_handle_cache: do not promote before checking full

Otherwise, ops with need_promote() can order past ops without it
causing an rgw write sequence like:

writefull 0~0
writefull 0~512kb
write 512kb~512kb

to reorder to

writefull 0~0 -- delayed: full
writefull 0~512kb -- delayed: full
write 512kb~512kb -- promote and execute
writefull 0~0 -- execute
writefull 0~512kb -- execute

Fixes: 13240
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #6206 from rzarzynski/wip-rgw-manifest-skip

rgw: add support for getting Swift's DLO without manifest handling

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

tests: removed obsolete rbd_replay test cases

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

ceph-dencoder: new rbd_replay trace file types

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay: added version control to trace output file

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: add support for skipping manifest parsing during GET on Swift object.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

client: don't mark_down on command reply

I guess this was handy when issuing single commands
from the CLI, but it breaks things badly when
trying to issue commands from a client
that's also going to carry on and do client IO.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #6200 from ceph/wip-13364

mds/Session: use projected parent for auth path check

Merge pull request #6116 from jcsp/wip-cephfs-direntry

pybind/cephfs: fix DirEntry helpers

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6195 from dachary/wip-release-notes

release-notes: draft v0.94.4 release notes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6188 from liewegas/wip-13395

os/FileStore: kludge sloppy hammer temp objects into temp collection

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

mds/Session: use projected parent for auth path check

In particular, a file's current path might indicate no parent
because the initial link into the tree isn't committed yet.

Fixes: #13364
Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_libcephfs: parse env properly (access)

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_libcephfs: parse CEPH_ARGS properly

This has to happen after the config parse.

Signed-off-by: Sage Weil <sage@redhat.com>

release-notes: draft v0.94.4 release notes

Signed-off-by: Loic Dachary <ldachary@redhat.com>

rbd-replay-prep: added --verbose command line option

This new command-line will dump all the processed events as they
are extracted from the trace file.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay-prep: stream events to the prep file

Avoid building the entire prep event history in memory before
attempting to write the prepped trace file.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay-prep: simplify IO dependency calculation

Only track read-after-write and write-after-write IO dependencies
via the associated write completions.  All IO events after a write
completion are considered to be dependent and can be pruned down
to at most the number of concurrent IOs.  This reduces the prep
time from a simple 'rbd bench-write' from over 4 hrs down to seconds.

Fixes: #13378, #13384
Backport: hammer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay: improve error messages

Fixes: #13221
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay-prep: support new read/write APIs

Added support for read2, write2, aio_read2, and aio_write2 APIs.

Fixes: 13325
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-replay-prep: avoid using assert statements on user inputs

Fixes: #13220
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge branch 'infernalis'

Merge pull request #6170 from jcsp/wip-spurious-warning

mds: avoid emitting cap warnings before evicting session

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #6128 from jcsp/wip-snapserver-crash

mds: fix SnapServer crash on deleted pool

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'wip-yrf-fix1' of git://github.com/H3C/ceph into infernalis

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'wip-13256' of git://github.com/ukernel/ceph into infernalis

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

qa: avoid using sudo in fsstress

This test required root in order to copy its built
binary into /usr (presumably to avoid rebuilding it).

That's not really a good thing anyway because there's
no guarantee that a binary in that path is the binary
we wanted, so just run the thing straight out of /tmp.  The
build is really quick anyway.

Signed-off-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #6186 from ceph/wip-12913-2

cls_rgw: fix bucket listing when dealing with invisible entries

os/FileStore: kludge sloppy hammer temp objects into temp collection

When we are running with a mixed hammer cluster, hammer primaries
will generate temp object names that are sloppy.  Make sure we still
put them into the temp collection.

Note that this isn't a problem on write because the primary (hammer)
OSD generated the transaction and explicitly specified a temp
collection; it's only transactions we do on our own with the sloppy
temp ghobject_t that trip over this.

Fixes: #13395
Signed-off-by: Sage Weil <sage@redhat.com>

ceph.spec.in: move python-sphinx BuildRequires to the right section

These belong in the section for distro-conditional dependencies.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: move BuildRequires out of subpackages

Since the main package and all the subpackages are built in the
same environment, concentrate all the BuildRequires in the main
package.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: drop MY_CONF_OPTS

http://tracker.ceph.com/issues/13402 Fixes: #13402

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6185 from liewegas/wip-13382

osd/PG: fix generate_past_intervals

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #6182 from liewegas/wip-13379

mon: do not remove proxied sessions

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

doc: remove mention of --lazy-remove from radosgw-admin manpage

"The --lazy-remove flag has been removed over 3 years ago. Docs need to be
updated."

http://tracker.ceph.com/issues/13317 Fixes: #13317

Signed-off-by: Nathan Cutler <ncutler@suse.com>

osd/PG: fix generate_past_intervals

We may be only calculating older past intervals and have a valid
history.same_interval_since value, in which case the local
same_interval_since value will end at the newest old interval we had to
generate.

This was introduced by 70316541bbb115d9a35954bfba373cf1dc084b7e.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/release-notes: fix math error in firefly notes

Reported-by: Christopher Kunz <chris@filoo.de>
Signed-off-by: Sage Weil <sage@redhat.com>