pybind/ceph_daemon.py: fix Termsize.update

it's a regression introduced by 84f9c12

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15238 from xiexingguo/wip-fix-blue-mkfs

os/bluestore: write "mkfs_done" into disk only if we pass fsck() tests

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15228 from chardan/jfw-wip-halflife_atomic_t-filestore

filestore: migrate atomic_t to std::atomic<>

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9307 from cxwshawn/wip-tc7-fix

crush: optimize header file dependency

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14035 from liewegas/wip-bluestore-kv-finisher

os/bluestore: separate kv_sync_thread into two parts

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #14176 from LiumxNL/wip-170321

mon: show io status quickly if no update in a long period

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15169 from joscollin/wip-cleanup-redundant-headers-2

common: Remove redundant includes - 2

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #15177 from liewegas/wip-bluestore-nonexperimental

os: remove experimental status for BlueStore

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15244 from liewegas/wip-crush-features

crush/CrushWrapper: fix has_incompat_choose_args()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15245 from liewegas/wip-20058

qa/suites/rados/*/at-end: wait for healthy before scrubbing

Merge pull request #15190 from tchaikov/wip-ceph-term-width

ceph.in: adjust usage width according to user's tty

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #15246 from linuxbox2/wip-unlink-safe

rgw_file: release rgw_fh lock and ref on ENOTEMPTY

Merge pull request #15202 from liewegas/wip-19935

qa/suites/rados: temporarily remove scrub_test from basic/ until post-luminous

Merge pull request #15207 from liewegas/wip-20043

qa/suites/rados/singleton-nomsg/health-warnings: behave on ext4

Merge pull request #15092 from yuyuyu101/wip-msgr-writelock

msg/async: reduce write_lock contention

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15172 from badone/wip-erasure-code-optimize-header-file-dependency

erasure-code: optimize header file dependency

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15187 from badone/wip-compressor-optimize-header-file-dependency

compressor: optimize header file dependency

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15201 from liewegas/wip-19990

qa/workunits/rados/test_health_warning: misc fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #15098 from cbodley/wip-rgw-default-frontend

rgw: remove fastcgi from default rgw frontends

Reviewed-by: Sage Weil <sage@redhat.com>

rgw_file: release rgw_fh lock and ref on ENOTEMPTY

An early return in new unlink bucket num_entries check was
missing a conditional unlock and unref.

Fixes: http://tracker.ceph.com/issues/20061
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #15216 from linuxbox2/wip-rgw-file-unlink

rgw_file:  remove post-unlink lookup check

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15179 from liewegas/wip-fix-osd-health

mon: fix a few bugs with the osd health reporting

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #15188 from ShiqiCooperation/master

mon: delete useless function definition

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15204 from joscollin/wip-cleanup-redundant-headers-3

common: Remove redundant includes - 3

qa/suites/rados/*/at-end: wait for healthy before scrubbing

The scrub_pgs command also waits for healthy for a while, but fails
silently if it times out, which means the subsequent scrubs will also
fail to clean up.

This forces an earlier failure that does not obscure the root cause.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14389 from mingshuaiwang/master

rgw: segment fault when shard id out of range

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12774 from shashalu/gc-list

rgw: fix 'gc list --include-all' command infinite loop the first items

Reviewed-by: Casey Bodley <cbodley@redhat.com>

crush/CrushWrapper: fix has_incompat_choose_args()

Special case empty and return false.  Otherwise the rest of the method
will dereference an invalid iterator on an empty choose_args
map.

Fixes broken fix in b1e4295570b6af4d844cf3ec55d77f5c287b29e6.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15242 from liupan1111/wip-fix-msg

msg/async: Lower down the AsyncMessenger's standby warning from debug

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #15218 from liewegas/wip-crush-features

crush/CrushWrapper: fix has_incompat_choose_args

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

msg/async: Lower down the AsyncMessenger's standby warning from debug
level 0 to 10.

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>

os/bluestore: save "mkfs_done" only if we pass fsck() tests

In our local test bed, we found mkfs() sometimes can fail
due to errors discovered by fsck(), and is therefore unrecoverable
by redoing mkfs() as the "mkfs_done" flag has been successfully saved into disk.

This patch fixes the above case.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #14610 from jcsp/wip-17980

#17980: MDS client blacklisting and blacklist on eviction

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #14550 from ukernel/wip-multimds-misc

mds: miscellaneous multimds fixes

Reviewed-by: John Spray <john.spray@redhat.com>

qa: refine assert_session_count (don't count killing)

Signed-off-by: John Spray <john.spray@redhat.com>

qa: update TestVolumeClient for new blacklisting

Blacklisted clients will now proactively fail
outstanding operations, rather than blocking.

Signed-off-by: John Spray <john.spray@redhat.com>

qa: clean up whitespace in test_misc.py

Signed-off-by: John Spray <john.spray@redhat.com>

qa: remove outdated TODO in TestVolumeClient

Signed-off-by: John Spray <john.spray@redhat.com>

qa: log whitelist for eviction messages

Signed-off-by: John Spray <john.spray@redhat.com>

mds: improve eviction usability and update docs

The "session ls" and "session evict" are now
"client ls" and "client evict" (the old ones
are still there for backwards compatibility).

The automatic client eviction now emits
cluster logs that call the client by its
friendly name (usually the hostname).

Signed-off-by: John Spray <john.spray@redhat.com>

mds: tidy up 'kill_session' naming

We had two kill_sessions.  Leave Server::kill_session (the
one that actually operates on MDS sessions only), and rename
the MDSRank one to evict_client, because it does more than
just kill the MDS session.

Signed-off-by: John Spray <john.spray@redhat.com>

qa: add test_filelock_eviction

To check that eviction is releasing flocks.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: hook up autoclose to blacklisting eviction

Signed-off-by: John Spray <john.spray@redhat.com>

mds: make blacklisting configurable in kill_session

This makes the code a bit awkward because we now have
4x paths through this function (blocking, noblocking, blacklisting,
nonblacklisting).

Signed-off-by: John Spray <john.spray@redhat.com>

client: fix unmounting while blacklisted

Previously, if you ever made it as far as ~Client
with any inodes in your cache, you'd hit the ObjectCacher
is_locked_by_me assertion during release_set on those inode's osets.

Signed-off-by: John Spray <john.spray@redhat.com>

client: detect blacklisting of myself

EIO all operations when this happens, and do a cut-down
unmount() so that once the user has noticed his mount
is broken, he can unmount it without getting stuck
waiting for flushes/caps/sessions.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: update osd epoch barrier when handling blacklist

Signed-off-by: John Spray <john.spray@redhat.com>

mds: update osdmap epoch barrier on eviction

Signed-off-by: John Spray <john.spray@redhat.com>

mds: blacklist clients on eviction

Fixes: http://tracker.ceph.com/issues/9754
Signed-off-by: John Spray <john.spray@redhat.com>

mon: permit MDS daemons to do blacklisting

Signed-off-by: John Spray <john.spray@redhat.com>

mds: remove stale comment

The code the comment is talking about went
away in 2490a0a1e.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: apply OSDMap blacklist to CephFS clients

Signed-off-by: John Spray <john.spray@redhat.com>

osd/OSDMap: std::set version of get_blacklist

Signed-off-by: John Spray <john.spray@redhat.com>

osdc: expose new blacklist entries from Objecter

For the MDS which needs to know about new blacklist entries
in order to evict corresponding MDS sessions.

Fixes: http://tracker.ceph.com/issues/17980
Signed-off-by: John Spray <john.spray@redhat.com>

filestore: migrate atomic_t to std::atomic<>

Signed-off-by: Jesse Williamson <jwilliamson@suse.de>

Merge pull request #15220 from joscollin/wip-cleanup-rados-typo

include/rados: Fix typo in rados_ioctx_cct() doc

include/rados: Fix typo in rados_ioctx_cct() doc

This fixes PR#15077.

Signed-off-by: Jos Collin <jcollin@redhat.com>

ceph.in: adjust usage width according to user's tty

fixed a pep8 warning also

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/ceph_daemon.py: move _gettermsize() into Termsize

as the latter is the only consumer of _gettermsize(). and a little bit
refactor to improve the readability and be more pep8 compliant.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crush/CrushWrapper: fix has_incompat_choose_args

If the map has *no* choose_args it does not have incompat choose args.
This makes the OSDMap::get_features() check

  if (crush->has_incompat_choose_args())
    features |= CEPH_FEATURE_CRUSH_CHOOSE_ARGS;

happier.

Signed-off-by: Sage Weil <sage@redhat.com>

rgw_file:  remove post-unlink lookup check

This could induce asserts in multi-nfs-gateway scenarios.

Fixes: http://tracker.ceph.com/issues/20047
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #15210 from linuxbox2/wip-rgw-rmbuck

rgw_file: remove hidden uxattr objects from buckets on delete

rgw_file: remove hidden uxattr objects from buckets on delete

If a setattr (e.g., chown) has been performed on a bucket, then like
any directory it has a hidden object storing its attributes. This must
be deleted before attempting bucket delete, otherwise, actually empty
buckets will not be removable via NFS.

Fixes: http://tracker.ceph.com/issues/20045
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #15195 from theanalyst/doc/12.0.3-typo-fix

doc: fixing an error in 12.0.3 release notes

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Loic Dachary <loic@dachary.org>

common: Remove redundant includes

Fixes: http://tracker.ceph.com/issues/19883 (Partially)
Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #15186 from adamemerson/wip-clangtastic-mr-clocks

rgw: Remove pessimizing move
common: Forward-declare container I/O overloads

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/singleton-nomsg/health-warnings: behave on ext4

We may land on an ext4 root partition.

Fixes: http://tracker.ceph.com/issues/20043
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #12861 from theanalyst/wip-absolute-urls

rgw: pass authentication domain to civetweb

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #15150 from ovh/bp-doc-perf-histograms

doc: document perf historgrams

Reviewed-by: Sage Weil <sage@redhat.com>

doc: document perf historgrams

Documentation for perf histograms schema and dump.

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>

common: Remove redundant includes

Fixes: http://tracker.ceph.com/issues/19883 (Partially)
Signed-off-by: Jos Collin <jcollin@redhat.com>

rgw: Remove pessimizing move

Do not std::move values from returned functions.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

common: Forward-declare container I/O overloads

This will allow strict ADL to find the right overload in the case of a
container of containers and unbreak compilation with Clang.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

qa/suites/rados: temporarily remove scrub_test from basic/ until post-luminous

We don't want to do the at-end.yaml scrubbing business with this test.
Move it into a separate collection until after luminous.

I have a todo item on the post-luminous cleanup list to avoid forgetting
to move this back.

Fixes: http://tracker.ceph.com/issues/19935
Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/rados/test_health_warning: misc fixes

- use 'noup' flag
- wait for healthy between each test
- check counts for each type

Fixes: http://tracker.ceph.com/issues/19990
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15162 from dillaman/wip-19970

librbd: reduce potential of erroneous blacklisting on image close

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

doc: fixing an error in 12.0.3 release notes

Thanks @smithfarm for pointing this out

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #15191 from Linbing/modify-note-word-error

mon: Fixed typo in @post of _active()

Note word error of _active(), take iif with if

Signed-off-by: Linbing <hawkerous@gmail.com>

mon: delete useless function definition

Signed-off-by: shiqi <1454927420@qq.com>

compressor: optimize header file dependency

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #15181 from tchaikov/wip-cxx1z-size

common: add ceph::size()

Reviewed-By: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #14946 from dmick/wip-mgr-dashboard

mgr: dashboard GUI module

Merge pull request #14994 from theanalyst/f/tier-fix

rgw_rados: create sync module instances only if run_sync_thread is set

Reviewed-by: Casey Bodley <cbodley@redhat.com>

test: add mgr/ and smoke test for mgr dashboard

Signed-off-by: Dan Mick <dan.mick@redhat.com>

rgw_rados: create sync module instances only if run_sync_thread is set

This allows for radosgw-admin to run even if there was a
misconfiguration in setting tier types which would otherwise make it
difficult to revert an invalid tier type setting. Since sync modules are
necessary only for rgw and not for the admin itself, this allows for
invalid tier configurations to be corrected as rgw-admin wouldn't fail

Fixes: http://tracker.ceph.com/issues/19830
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #15153 from zhangsw/optimize-torrent

rgw: optimize generating torrent file. Object data won't stay in memory  now.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #15001 from chardan/jfw-wip-halflife_atomic_t-kamakuras

rgw: migrate atomic_t to std::atomic<>

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14827 from liuyuhong66/rgw-swift-auth-improve

rgw/rgw_swift_auth.cc: using string::back() instead as the C++11 recommend

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15025 from mikulely/jiaying-fix-lc-list

rgw: fix lc list failure when shards not be all created

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12300 from idealguo/rename-s3_code

rgw: rename s3_code to err_code for swift

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15045 from shashalu/fix-slo

rgw: fix X-Object-Meta-Static-Large-Object in SLO download

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14800 from theanalyst/doc/release-notes/sse-c-correction

doc: release-notes clarify about rgw encryption

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14922 from ukernel/wip-kcephfs-whitelist

qa: update log whitelists for kcephfs suite

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14941 from wjwithagen/wip-wjw-freebsd-doc-manual-install

doc: add FreeBSD manual install

Merge pull request #14988 from ShiqiCooperation/master

test/unittest_bluefs: check whether mounted success

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14688 from cbodley/wip-rgw-multi-suite

qa/rgw: add multisite suite to configure and run multisite tests

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #15071 from cbodley/wip-qa-dnsmasq

qa: add task for dnsmasq configuration

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #15090 from theanalyst/release/v12.0.3/notes

doc: release notes for v12.0.3 (dev)

Merge pull request #15133 from cbodley/wip-doc-rgw-remove-fcgi

doc/rgw: remove fastcgi page and sample configs

Reviewed-by: Sage Weil <sage@redhat.com>