From: Abhishek Lekshmanan Date: Thu, 22 Jun 2017 20:57:05 +0000 (+0200) Subject: doc: v12.1.0 release notes notable changes addition again X-Git-Tag: ses5-milestone7~3^2~16^2 X-Git-Url: http://git.apps.os.sepia.ceph.com/?a=commitdiff_plain;h=570fa825fd77bb8916fe39892964a5048101d92d;p=ceph.git doc: v12.1.0 release notes notable changes addition again commmits from 9f259cb923e7474229bb2e5e54b59f3b7f068bc5..luminous && sort Signed-off-by: Abhishek Lekshmanan --- diff --git a/doc/release-notes.rst b/doc/release-notes.rst index e466614f04658..7386c06847532 100644 --- a/doc/release-notes.rst +++ b/doc/release-notes.rst @@ -111,20 +111,37 @@ Major Changes from Kraken Notable Changes --------------- + * bluestore: ceph-disk: add --filestore argument, default to --bluestore (`pr#15437 `_, Loic Dachary, Sage Weil) * bluestore,core: os/bluestore: fix warning (`pr#15435 `_, Sage Weil) * bluestore,core: os/bluestore: improve mempool usage (`pr#15402 `_, Sage Weil) * bluestore,core: os/bluestore: write "mkfs_done" into disk only if we pass fsck() tests (`pr#15238 `_, xie xingguo) * bluestore,core: os: remove experimental status for BlueStore (`pr#15177 `_, Sage Weil) +* bluestore: os/bluestore: better debug output on unsharing blobs (`issue#20227 `_, `pr#15746 `_, Sage Weil) * bluestore: os/bluestore/BlockDevice: support pmem device as bluestore backend (`pr#15102 `_, Jianpeng Ma) +* bluestore: os/bluestore/BlueFS: Rebuild memcopy for bufferlist::page_aligned_app… (`pr#15728 `_, Jianpeng Ma, Sage Weil) +* bluestore: os/bluestore/BlueFS: .slow should be compared with dirname (`pr#15595 `_, zhanglei) +* bluestore: os/bluestore/BlueStore: no device no symlink. (`pr#15721 `_, Jianpeng Ma) +* bluestore: os/bluestore: cleanup bluestore_types (`pr#15680 `_, xie xingguo) +* bluestore: os/bluestore: configure rocksdb cache via bluestore_cache_kv_ratio (`pr#15580 `_, Sage Weil) * bluestore: os/bluestore: fix a typo about bleustore (`pr#15357 `_, Dongsheng Yang) * bluestore: os/bluestore: fix BitMapAllocator assert on out-of-bound hint value (`pr#15289 `_, Igor Fedotov) * bluestore: os/bluestore: fix buffers pinned by indefinitely deferred writes (`pr#15398 `_, Sage Weil) * bluestore: os/bluestore: fix false assert in IOContext::aio_wake (`pr#15268 `_, Igor Fedotov) * bluestore: os/bluestore: fix false asserts in Cache::trim_all() (`pr#15470 `_, xie xingguo) * bluestore: os/bluestore: fix fsck deferred_replay (`pr#15295 `_, Sage Weil) +* bluestore: os/bluestore: fix possible out of order shard(offset == 0); add sanity check (`pr#15658 `_, xie xingguo) +* bluestore: os/bluestore: fix potential access violation (`pr#15657 `_, xie xingguo) +* bluestore: os/bluestore: fix unsharing blob dirty_range args (`issue#20227 `_, `pr#15766 `_, Sage Weil) +* bluestore: os/bluestore: handle rounding error in cache ratios (`pr#15672 `_, Sage Weil) +* bluestore: os/bluestore: initialize finishers properly (`pr#15666 `_, xie xingguo) * bluestore: os/bluestore/KernelDevice: fix comments (`pr#15264 `_, xie xingguo) * bluestore: os/bluestore/KernelDevice: helpful warning when aio limit exhausted (`pr#15116 `_, Sage Weil) +* bluestore: os/bluestore: move sharedblob to new collection in same shard (`issue#20358 `_, `pr#15783 `_, Sage Weil) +* bluestore: os/bluestore: refactor BlueStore::_do_write; kill dead ExtentMap::find_lextent() method (`pr#15750 `_, xie xingguo) +* bluestore: os/bluestore: remove unused variables (`pr#15718 `_, zhanglei) +* bluestore: os/bluestore: stop calculating bound if we must reshard; narrow shard combination condition (`pr#15631 `_, xie xingguo) +* bluestore: os/bluestore: target_bytes should scale with meta/data ratios. (`pr#15708 `_, Mark Nelson) * bluestore,performance: os/bluestore: avoid overloading extents during reshard; atomic deferred_batch_ops (`pr#15502 `_, xie xingguo) * bluestore,performance: os/bluestore: batch throttle (`pr#15284 `_, Jianpeng Ma) * bluestore,performance: os/bluestore: keep statfs replica in RAM to avoid expensive KV retrieval (`pr#15309 `_, Igor Fedotov) @@ -133,6 +150,9 @@ Notable Changes * bluestore,performance: os/bluestore: put bluefs in the middle of the shared device (`pr#14873 `_, Sage Weil) * bluestore,performance: os/bluestore: separate kv_sync_thread into two parts (`pr#14035 `_, Jianpeng Ma, Igor Fedotov, Sage Weil) * bluestore,performance: os/bluestore: try to unshare blobs for EC overwrite workload (`pr#14239 `_, Sage Weil) +* bluestore,tests: qa/objectstore/bluestore*: fsck on mount (`pr#15785 `_, Sage Weil) +* bluestore,tests: test/unittest_bluefs: When fsync ret is less than 0, fsync can not be… (`pr#15365 `_, shiqi) +* bluestore: wrap blob id when it reaches maximum value of int16_t (`issue#19555 `_, `pr#15654 `_, Xiaoyan Li) * build/ops: 12.0.3 (`pr#15600 `_, Jenkins Build Slave User) * build/ops: build: move bash_completion.d/ceph to ceph-common (`pr#15148 `_, Leo Zhang) * build/ops: build: remove ceph-disk-udev entirely (`pr#15259 `_, Leo Zhang) @@ -142,16 +162,22 @@ Notable Changes * build/ops,common: build: Adds C++ warning flag for C Variable-Length Arrays. (`pr#15342 `_, Jesse Williamson) * build/ops,common: common/blkdev.cc: propagate get_device_by_fd to different OSes (`pr#15547 `_, Willem Jan Withagen) * build/ops: conditionalize rgw Beast frontend so it isn't built on s390x architecture (`issue#20048 `_, `pr#15225 `_, Willem Jan Withagen, Nathan Cutler, Kefu Chai, Tim Serong, Casey Bodley) +* build/ops,core: common/freebsd_errno.cc: fix missing ; (`pr#15741 `_, Willem Jan Withagen) +* build/ops,core: erasure-code: update ec_isa version + add missing AVX512 ISA-L sources (`pr#15636 `_, Ganesh Mahalingam, Tushar Gohad) * build/ops,core,tests: osd/dmclock/testing: reorganize testing, building now optional (`pr#15375 `_, J. Eric Ivancich) * build/ops: debian: ceph-mgr: fix package description (`pr#15513 `_, Fabian Grünbichler) * build/ops: debian: sync logrotate packaging with downstream (`issue#19938 `_, `pr#15567 `_, Fabian Grünbichler) * build/ops: do_cmake.sh: enable ccache if installed (`pr#15274 `_, Sage Weil) * build/ops: drop libfcgi build dependency (`pr#15285 `_, Nathan Cutler) +* build/ops: .gitignore: exclude rpm files (`pr#15745 `_, Leo Zhang) * build/ops: install-deps.sh: workaround setuptools' dependency on six (`pr#15406 `_, Kefu Chai) +* build/ops: miscellaneous cleanups and fixes (run-make-check.sh, ceph.spec.in) (`issue#20091 `_, `issue#20127 `_, `pr#15399 `_, Nathan Cutler) +* build/ops,rbd,tests: test/librbd: decouple ceph_test_librbd_api from libceph-common (`issue#20175 `_, `pr#15611 `_, Kefu Chai) * build/ops: rpm: apply epoch only if %epoch macro is defined (`pr#15286 `_, Nathan Cutler) * build/ops: rpm: make librbd1 %post scriptlet depend on coreutils (`issue#20052 `_, `pr#15231 `_, Giacomo Comes, Nathan Cutler) * build/ops: rpm: move _epoch_prefix below Epoch definition (`pr#15417 `_, Nathan Cutler) * build/ops: rpm: move RDMA and python-prettytables build dependencies to distro-conditional section (`pr#15200 `_, Nathan Cutler) +* build/ops: rpm: package COPYING, move sample ceph.conf to ceph-common (`pr#15596 `_, Nathan Cutler) * build/ops: selinux: Allow read on var_run_t (`issue#16674 `_, `pr#15523 `_, Boris Ranto) * build/ops: selinux: Do parallel relabel on package install (`issue#20077 `_, `pr#14871 `_, Boris Ranto) * build/ops: selinux: Install ceph-base before ceph-selinux (`issue#20184 `_, `pr#15490 `_, Boris Ranto) @@ -160,22 +186,36 @@ Notable Changes * build/ops: yasm-wrapper: filter -pthread (`pr#15249 `_, Alessandro Barbieri) * cephfs: #17980: MDS client blacklisting and blacklist on eviction (`issue#17980 `_, `issue#9754 `_, `pr#14610 `_, John Spray) * cephfs: ceph: simplify CInode::maybe_export_pin() (`pr#15106 `_, "Yan, Zheng") +* cephfs: client: avoid returning negative space available (`issue#20178 `_, `pr#15481 `_, John Spray) +* cephfs: client: call the lru_remove() twice,when trim cache (`pr#15662 `_, huanwen ren) +* cephfs: client: check for luminous MDS before sending FLUSH_MDLOG (`pr#15805 `_, John Spray) +* cephfs: client: fix Dentry::dump (`pr#15779 `_, huanwen ren) * cephfs: client: fix display ino in the ldout (`pr#15314 `_, huanwen ren) * cephfs: client/inode: fix the dump type of Inode::dump() (`pr#15198 `_, huanwen ren) * cephfs,common,rbd: blkin: librbd trace hooks (`pr#15053 `_, Victor Araujo, Jason Dillaman) +* cephfs: mds/MDBalancer: remove useless check_targets and hit_targets logic from MDS balancer (`issue#20131 `_, `pr#15407 `_, Zhi Zhang) +* cephfs,mgr: pybind/mgr/fsstatus: use mds_mem.dn as dentry counter (`pr#15255 `_, Zhi Zhang) * cephfs: mon/FSCommand: fix indentation (`pr#15423 `_, Sage Weil) * cephfs: mon/MDSMonitor: respect mds_standby_for_rank config (`pr#15129 `_, "Yan, Zheng") * cephfs: osdc/Journaler: avoid executing on_safe contexts prematurely (`issue#20055 `_, `pr#15240 `_, "Yan, Zheng") +* cephfs: osdc/Journaler: fix memory leak in Journaler::_issue_read() (`issue#20338 `_, `pr#15776 `_, "Yan, Zheng") +* cephfs: osdc/Objecter: fix inflight_ops update (`pr#15768 `_, "Yan, Zheng") * cephfs: qa/cephfs: disable mds_bal_frag for TestStrays.test_purge_queue_op_rate (`issue#19892 `_, `pr#15105 `_, "Yan, Zheng") * cephfs: qa/tasks/cephfs: use getattr to guarantee inode is in client cache (`issue#19912 `_, `pr#15062 `_, "Yan, Zheng") * cephfs: qa: update log whitelists for kcephfs suite (`pr#14922 `_, "Yan, Zheng") +* cephfs,tests: ceph-object-corpus: mark MMDSSlaveRequest incompat change (`pr#15730 `_, Sage Weil) * cephfs,tests: qa: fix float parse error in test_fragment (`pr#15122 `_, Patrick Donnelly) +* cephfs,tests: qa: misc cephfs test improvements (`issue#20131 `_, `pr#15411 `_, John Spray) * cephfs,tests: qa: silence upgrade test failure (`issue#19934 `_, `pr#15126 `_, Patrick Donnelly) * cephfs,tests: qa: simplify TestJournalRepair (`pr#15096 `_, John Spray) +* cephfs: tools/cephfs: remove `apply` mode of cephfs-journal-tool (`pr#15715 `_, John Spray) * cleanup: src: put-to operator function - const input cleanup (`issue#3977 `_, `pr#15364 `_, Jos Collin) * cmake: Add -finstrument-functions flag to OSD code (`pr#15055 `_, Mohamad Gebai) +* cmake: build boost as an external project (`pr#15376 `_, Kefu Chai) * cmake: check the existence of gperf before using it (`pr#15164 `_, Kefu Chai) +* cmake: do not add dependencies to INTERFACE library on cmake < 3.3 (`pr#15813 `_, Kefu Chai) * cmake: do not link libcommon against some libs (`pr#15340 `_, Willem Jan Withagen) +* cmake: exclude *.css while generating ctags (`pr#15663 `_, Leo Zhang) * cmake: fix boost components for WITH_SYSTEM_BOOST (`pr#15160 `_, Bassam Tabbara) * cmake: improved build speed by 5x when using ccache (`pr#15147 `_, Bassam Tabbara) * cmake: link against fcgi only if enabled (`pr#15425 `_, Yao Zongyou) @@ -188,13 +228,16 @@ Notable Changes * common: cmdparse: more constness (`pr#15023 `_, Kefu Chai) * common: common/ceph_context: 'config diff get' option added (`pr#10736 `_, Daniel Oliveira) * common: common/ceph_context: fix leak of registered commands on exit (`pr#15302 `_, xie xingguo) +* common: common/freebsd_errno.cc: fixed again a stupid typo (`pr#15742 `_, Willem Jan Withagen) * common: common/iso_8601.cc: Make return expression Clang compatible (`pr#15336 `_, Willem Jan Withagen) * common: common/LogEntry: include EntityName in log entries (`pr#15395 `_, Sage Weil) * common: common,osdc: remove atomic_t completely (`pr#15562 `_, Kefu Chai) * common: common/perf_counters: add average time for PERFCOUNTER_TIME (`pr#15478 `_, xie xingguo) * common: common/perf_counters: make schema more friendly and update docs (`pr#14933 `_, Sage Weil) * common: common,test: migrate atomic_t to std::atomic (`pr#14866 `_, Jesse Williamson) +* common: config_opt: use bool instead of int for the default value of filestore_debug_omap_check (`pr#15651 `_, Leo Zhang) * common,core: ceph_test_rados_api_misc: fix LibRadosMiscConnectFailure.ConnectFailure retry (`issue#19901 `_, `pr#15522 `_, Sage Weil) +* common: core/common: Fix ENODATA for FreeBSD with compat.h (`issue#19883 `_, `pr#15685 `_, Willem Jan Withagen) * common,core: osd/OSDMap: make osd_state 32 bits wide (`pr#15390 `_, Sage Weil) * common,core: osd/OSDMap: replace require_*_osds flags with a single require_osd_release field (`pr#15068 `_, Sage Weil) * common,core: osd/OSDMap: replace string-based min_compat_client with a CEPH_RELEASE_* uint8_t (`pr#15351 `_, Sage Weil) @@ -202,31 +245,41 @@ Notable Changes * common: crush/CrushWrapper: fix has_incompat_choose_args (`pr#15218 `_, Sage Weil) * common: crush/CrushWrapper: fix has_incompat_choose_args() (`pr#15244 `_, Sage Weil) * common: denc: add encode/decode for basic_sstring (`pr#15135 `_, Kefu Chai, Casey Bodley) +* common: fix typo in option of rados_mon_op_timeout's comment (`pr#15681 `_, Leo Zhang) * common: get_process_name: use getprogname on bsd systems (`pr#15338 `_, Mykola Golub) * common: Improved CRC calculation for zero buffers (`pr#11966 `_, Adam Kupczyk) * common: include/lru.h: add const to member functions (`pr#15408 `_, yonghengdexin735) * common: include/rados: Fix typo in rados_ioctx_cct() doc (`pr#15220 `_, Jos Collin) * common: include: Redo some includes for FreeBSD (`issue#19883 `_, `pr#15337 `_, Willem Jan Withagen) +* common: initialize _hash in LogEntryKey() (`pr#15615 `_, Jos Collin) * common: int_types.h: remove hacks to workaround old systems (`pr#15069 `_, Kefu Chai) * common: librados,libradosstriper,test: migrate atomic_t to std::atomic (baragon) (`pr#14658 `_, Jesse Williamson) * common: libradosstriper: Add example code (`pr#15350 `_, Logan Blyth) * common: mempool: improve dump; fix buffer accounting bugs (`pr#15403 `_, Sage Weil) +* common: mgr/PyFormatter: implement dump_format_va (`pr#15634 `_, Sage Weil) * common,mon: messenger,client,compressor: migrate atomic_t to std::atomic (`pr#14657 `_, Jesse Williamson) * common,mon: mon,crush: add 'osd crush swap-bucket' command (`pr#15072 `_, Sage Weil) +* common: msg/async: add assert of ms_async_op_threads > 0 (`pr#15629 `_, linbing) * common,performance: buffer: allow buffers to be accounted in arbitrary mempools (`pr#15352 `_, Sage Weil) * common,performance: crc32c: Add ppc64le fast zero optimized assembly. (`pr#15100 `_, Andrew Solomon) * common,performance: inline_memory: optimized mem_is_zero for non-x64 (`pr#15307 `_, Piotr Dałek) * common,performance: kv/rocksdb: supports SliceParts interface (`pr#15058 `_, Haomai Wang) * common,performance: osd/OSDMap: make pg_temp more efficient (`pr#15291 `_, Sage Weil) +* common,rdma: msg/async/rdma: automatically set RDMAV_HUGEPAGES_SAFE according to conf (`pr#15755 `_, DanielBar-On) * common: Remove redundant includes - 2 (`issue#19883 `_, `pr#15169 `_, Jos Collin) * common: Remove redundant includes - 3 (`issue#19883 `_, `pr#15204 `_, Jos Collin) +* common: Remove redundant includes - 4 (`issue#19883 `_, `pr#15251 `_, Jos Collin) * common: Remove redundant includes - 5 (`issue#19883 `_, `pr#15267 `_, Jos Collin) * common: Remove redundant includes - 6 (`issue#19883 `_, `pr#15299 `_, Jos Collin) * common: Remove redundant includes (`issue#19883 `_, `pr#15042 `_, Brad Hubbard) * common: Remove redundant includes (`issue#19883 `_, `pr#15086 `_, Jos Collin) +* common: src/common/ceph_string: stringify new osd states (`pr#15751 `_, xie xingguo) * common,tests: ceph_test_rados_api_list: more fix LibRadosListNP.ListObjectsError (`issue#19963 `_, `pr#15138 `_, Sage Weil) +* common,tests: test: Make screencandy optional for FreeBSD (`pr#15444 `_, Willem Jan Withagen) * common: xio: migrate atomic_t to std::atomic<> (`pr#15230 `_, Jesse Williamson) +* core: ceph-dencoder: Silence coverity CID 1412579 (`pr#15744 `_, Brad Hubbard) * core: ceph-disk: do not setup_statedir on trigger (`issue#19941 `_, `pr#15410 `_, Loic Dachary) +* core: ceph-objectstore-tool: do not populate snapmapper with missing clones (`issue#19943 `_, `pr#15787 `_, Sage Weil) * core: compressor: add LZ4 support (`pr#15434 `_, Haomai Wang) * core: compressor: optimize header file dependency (`pr#15187 `_, Brad Hubbard, Xiaowei Chen) * core: crush, mon: make jewel the lower bound for client/crush compat for new clusters (`pr#15370 `_, Sage Weil) @@ -235,36 +288,65 @@ Notable Changes * core: filestore: migrate atomic_t to std::atomic<> (`pr#15228 `_, Jesse Williamson) * core: include/types.h, introduce host_to_ceph_errno (`pr#15496 `_, Willem Jan Withagen) * core: Install Pecan for FreeBSD (`pr#15610 `_, Willem Jan Withagen) +* core: introduce (and fix) code to pass errno to other OSes (`pr#15495 `_, Willem Jan Withagen) +* core: messages/MOSDPing: initialize MOSDPing padding (`issue#20323 `_, `pr#15714 `_, Sage Weil) * core,mgr: mgr/DaemonServer: stop spamming log with pg stats (`pr#15487 `_, Sage Weil) * core,mgr,mon: mon/PGMap: fix osd_epoch update when removing osd_stat (`issue#20208 `_, `pr#15573 `_, Sage Weil) +* core,mgr,tests: qa/suites/rados/rest: test restful mgr module (`pr#15604 `_, Sage Weil) +* core: mon,mgr: fix "ceph osd df", add some tools to find untested commands (`issue#20256 `_, `pr#15675 `_, Greg Farnum) * core,mon: mon/LogMonitor: 'log last' command (`pr#15497 `_, Sage Weil) +* core,mon: mon/MgrStatMonitor: keep mgrstat version ahead of pgmon (`issue#20219 `_, `pr#15584 `_, Sage Weil) +* core,mon: mon,osd: add crush_version to OSDMap, and allow crush map updates to gate on crush_version (`pr#15533 `_, Sage Weil) * core,mon: mon/OSDMonitor: cancel mapping job from update_from_paxos (`issue#20067 `_, `pr#15320 `_, Sage Weil) * core,mon: mon/OSDMonitor: use up set instead of acting set in reweight_by_utilization (`pr#13802 `_, Mingxin Liu) * core,mon: mon/PGMap: call blocked requests ERR not WARN (`pr#15501 `_, Sage Weil) +* core: mon/OSDMonitor: batch noup/noin osds support (`pr#15725 `_, xie xingguo) +* core: mon/OSDMonitor: batch OSDs nodown/noout support (`pr#15381 `_, xie xingguo) * core: mon/OSDMonitor: change info in 'osd failed' messages (`pr#15321 `_, Sage Weil) * core: mon,osd/OSDMap: a couple pg-upmap fixes (`pr#15319 `_, Sage Weil) * core: msg/async: avoid requeue racing with handle_write (`issue#20093 `_, `pr#15324 `_, Haomai Wang) +* core: msg/async: fix deleted_conn is out of sync with conns (`issue#20230 `_, `pr#15645 `_, Haomai Wang) +* core: objclass-sdk: use namespace ceph for bufferlist (`pr#15581 `_, Neha Ojha) +* core: osd/ECTransaction: only read partial stripes when below *original* object size (`issue#19882 `_, `pr#15712 `_, Sage Weil) * core: osd,librados: add manifest, redirect (`pr#15325 `_, Sage Weil) +* core: osd, messages/MOSDPing: bunch of fixes related to ping inflation (`pr#15727 `_, Piotr Dałek) +* core: osd/OSD: bump up current version; conditionally encoding manifest into oi (`pr#15687 `_, xie xingguo) +* core: osd/OSDMap: Change *pg_to_* to return void (`pr#15684 `_, Brad Hubbard) * core: osd/OSDMap: improve upmap calculation (`issue#19818 `_, `pr#14902 `_, Sage Weil) * core: osd/PG: drop pre-firefly compat_mode for choose_*_acting (`pr#15057 `_, Sage Weil) * core: osd/pglog: remove loop through empty collection (`pr#15121 `_, J. Eric Ivancich) +* core: osd/PG: make non-empty PastIntervals non-fatal (`issue#20167 `_, `pr#15639 `_, Sage Weil) +* core: osd/PrimaryLogPG: clear oi from trim_object() (`issue#19947 `_, `pr#15519 `_, Sage Weil) +* core: osd/PrimaryLogPG: fix oi reset during trim_object (`issue#19947 `_, `pr#15696 `_, Sage Weil) +* core: osd/PrimaryLogPG: record prior_version for DELETE events (`issue#20274 `_, `pr#15649 `_, Sage Weil) +* core: os/filestore: when print log, use __func__ instead of hard code function name (`pr#15261 `_, mychoxin) +* core: os/filestore: zfs add get_name() (`pr#15650 `_, Yanhu Cao) * core,performance: msg/async: reduce write_lock contention (`pr#15092 `_, Haomai Wang) +* core,performance: osd/OSD.h: requeue the scrub job with higher priority to shorten the blocking time of related requests (`pr#15552 `_, Jin Cai) +* core: qa: do not restrict valgrind runs to centos (`issue#18126 `_, `pr#15389 `_, Greg Farnum) * core,rgw: qa: Removed all 'default_idle_timeout' due to chnage in rwg task (`pr#15420 `_, Yuri Weinstein) * core,rgw,tests: qa/rgw_snaps: move default_idle_timeout config under the client (`issue#20128 `_, `pr#15400 `_, Yehuda Sadeh) * core: src/ceph.in: Use env(CEPH_DEV) to suppress noise from ceph (`pr#14746 `_, Willem Jan Withagen) * core,tests: ceph-disk: sensible default for block.db (`pr#15576 `_, Loic Dachary) +* core,tests: ceph_test_rados_api_*: wait for snap trim on ENOENT during cleanup (`issue#19948 `_, `pr#15638 `_, Sage Weil) * core,tests: qa/suites/rados/*/at-end: wait for healthy before scrubbing (`pr#15245 `_, Sage Weil) * core,tests: qa/suites/rados/singleton-nomsg/health-warnings: behave on ext4 (`issue#20043 `_, `pr#15207 `_, Sage Weil) * core,tests: qa/suites/rados: temporarily remove scrub_test from basic/ until post-luminous (`issue#19935 `_, `pr#15202 `_, Sage Weil) * core,tests: qa/suites/upgrade/kraken-x: enable experimental for bluestore (`pr#15359 `_, Sage Weil) +* core,tests: qa/tasks/ceph_manager: get osds all in after thrashing (`pr#15784 `_, Sage Weil) +* core,tests: qa/tasks/ceph: osd_scrub_pgs: reissue scrub requests in loop (`issue#20326 `_, `pr#15747 `_, Sage Weil) +* core,tests: qa/tasks/ceph.py: tolerate active+clean+something (`pr#15717 `_, Sage Weil) * core,tests: qa/workunits/cephtool/test.sh: fix osd full health detail grep (`issue#20187 `_, `pr#15494 `_, Sage Weil) * core,tests: qa/workunits/rados/test_health_warning: misc fixes (`issue#19990 `_, `pr#15201 `_, Sage Weil) +* core,tests: Revert "qa: do not restrict valgrind runs to centos" (`issue#20360 `_, `pr#15791 `_, Sage Weil) * core,tests: test/osd/TestRados.cc: run set-redirect test after finishing setup (`issue#20114 `_, `pr#15385 `_, Myoungwon Oh) * core,tools: osdmaptool: require --upmap-save before modifying input osdmap (`pr#15247 `_, Sage Weil) * crush: add missing tunable in tests (`pr#15412 `_, Loic Dachary) +* crush: detect and (usually) fix ruleset != rule id (`pr#13683 `_, Sage Weil) * crush: encode can override weights with weight set (`issue#19836 `_, `pr#15002 `_, Loic Dachary) * crush: optimize header file dependency (`pr#9307 `_, Xiaowei Chen) * crush: update choose_args when items are added/removed (`pr#15311 `_, Loic Dachary) +* doc: 12.1.0/release notes 2 (`pr#15627 `_, Abhishek Lekshmanan) * doc: add descriptions for mon/mgr options (`pr#15032 `_, Kefu Chai) * doc: add FreeBSD manual install (`pr#14941 `_, Willem Jan Withagen) * doc: add new cn ceph mirror to doc and mirroring (`pr#15089 `_, Shengjing Zhu) @@ -273,18 +355,24 @@ Notable Changes * doc: AUTHORS: update with release manager, backport team (`pr#15391 `_, Sage Weil) * doc: Change the default values of some OSD options (`issue#20199 `_, `pr#15566 `_, Bara Ancincova) * doc: describe CephFS max_file_size (`pr#15287 `_, Ken Dreyer) +* doc: describe mark_events logging available via the OSD's OpTracker (`pr#15095 `_, Greg Farnum) * doc: dev improve the s3tests doc to reflect current scripts (`pr#15180 `_, Abhishek Lekshmanan) * doc: doc/cephfs: mention RADOS object size limit (`pr#15550 `_, John Spray) * doc: doc/release-notes: update which jewel version does sortbitwise warning (`pr#15209 `_, Sage Weil) +* doc: doc/release-note: update release-note (`pr#15748 `_, liuchang0812) * doc: doc/rgw: remove fastcgi page and sample configs (`pr#15133 `_, Casey Bodley) +* doc: doc/rgw: remove Federated Configuration, clean up multisite (`issue#19504 `_, `issue#18082 `_, `pr#15132 `_, Casey Bodley) * doc: Documentation Fixes for http://tracker.ceph.com/issues/19879 (`issue#20057 `_, `issue#19879 `_, `pr#15606 `_, Sameer Tiwari) * doc: document perf historgrams (`pr#15150 `_, Piotr Dałek) * doc: Document RGW quota cache options (`issue#18747 `_, `pr#13395 `_, Daniel Gryniewicz) +* doc: document the setup of restful and dashboard plugins (`issue#20239 `_, `pr#15707 `_, Kefu Chai) * doc: fix broken link in erasure-code.rst (`issue#19972 `_, `pr#15143 `_, MinSheng Lin) * doc: fix factual inaccuracy in doc/architecture.rst (`pr#15235 `_, Nathan Cutler, Sage Weil) * doc: fixing an error in 12.0.3 release notes (`pr#15195 `_, Abhishek Lekshmanan) * doc: fix syntax on code snippets in cephfs/multimds (`pr#15499 `_, John Spray) +* doc: freshen mgr docs (`pr#15690 `_, John Spray) * doc: kill some broken links (`pr#15203 `_, liuchang0812) +* doc: mailmap for v12.0.2 (`pr#14753 `_, Abhishek Lekshmanan) * doc: mailmap: Leo Zhang infomation and affiliation (`pr#15145 `_, Leo Zhang) * doc: mention certain conf vars should be in global (`pr#15119 `_, Ali Maredia) * doc: Merge pull request from stiwari/wip-19879 (`issue#19879 `_, `pr#15609 `_, Sameer Tiwari) @@ -293,15 +381,19 @@ Notable Changes * doc: PendingReleaseNotes: warning about 'osd rm ...' and #19119 (`issue#19119 `_, `pr#13731 `_, Sage Weil) * doc: release-notes clarify about rgw encryption (`pr#14800 `_, Abhishek Lekshmanan) * doc: release notes for v12.0.3 (dev) (`pr#15090 `_, Abhishek Lekshmanan) +* doc: remove some non-existent and fix the default value according to … (`pr#15664 `_, Leo Zhang) * docs document "osd recovery max single start" setting (`issue#17396 `_, `pr#15275 `_, Ken Dreyer) +* doc: tools/cephfs: fix cephfs-journal-tool --help (`pr#15614 `_, John Spray) * doc: typo fixes on hyperlink/words (`pr#15144 `_, Drunkard Zhang) * doc: update sample explaning "%" operator in test suites (`pr#15511 `_, Kefu Chai) * doc: Update some RGW documentation (`pr#15175 `_, Jens Rosenboom) * doc: update the usage of 'ceph-deploy purge' (`pr#15080 `_, Yu Shengzuo) * doc: use do_cmake.sh instead of `cmake ..` (`pr#15110 `_, Kefu Chai) +* librbd: add create timestamp metadata for image (`pr#15757 `_, runsisi) * librbd: discard related IO should skip op if object non-existent (`issue#19962 `_, `pr#15239 `_, Mykola Golub) * librbd: do not raise an error if trash list returns -ENOENT (`pr#15085 `_, runsisi) * librbd: filter expected error codes from is_exclusive_lock_owner (`issue#20182 `_, `pr#15483 `_, Jason Dillaman) +* librbd: fix issues with image removal state machine (`pr#15734 `_, Jason Dillaman) * librbd: fix valgrind errors and ensure tests detect future leaks (`pr#15415 `_, Jason Dillaman) * librbd: optimize copy-up to add hints only once to object op (`issue#19875 `_, `pr#15037 `_, Mykola Golub) * librbd: potential read IO hang when image is flattened (`issue#19832 `_, `pr#15234 `_, Jason Dillaman) @@ -310,41 +402,60 @@ Notable Changes * librbd: remove unused rbd_image_options_t ostream operator (`pr#15443 `_, Mykola Golub) * mds: change the type of data_pools (`pr#15278 `_, Vicente Cheng) * mds: check export pin during replay (`issue#20039 `_, `pr#15205 `_, Patrick Donnelly) +* mds: Client syncfs is slow (waits for next MDS tick) (`issue#20129 `_, `pr#15544 `_, dongdong tao) +* mds: explicitly output error msg for dump cache asok command (`pr#15592 `_, Zhi Zhang) * mds: fix CDir::merge() for mds_debug_auth_pins (`issue#19946 `_, `pr#15130 `_, "Yan, Zheng") * mds: fix client ID truncation (`pr#15258 `_, Henry Chang) +* mds: fix hangs involving re-entrant calls to journaler (`issue#20165 `_, `pr#15430 `_, John Spray) +* mds: improvements for stray reintegration (`pr#15548 `_, "Yan, Zheng") * mds: limit client writable range increment (`issue#19955 `_, `pr#15131 `_, "Yan, Zheng") +* mds: mds perf item 'l_mdl_expos' always behind journaler (`pr#15621 `_, redickwang) +* mds: miscellaneous multimds fixes part2 (`pr#15125 `_, "Yan, Zheng") * mds: miscellaneous multimds fixes (`pr#14550 `_, "Yan, Zheng") +* mds: multimds flock fixes (`pr#15440 `_, "Yan, Zheng") * mds: Pass empty string to clear mantle balancer (`issue#20076 `_, `pr#15282 `_, Zhi Zhang) * mds: properly create aux subtrees for pinned directory (`issue#20083 `_, `pr#15300 `_, "Yan, Zheng") +* mds: save projected path into inode_t::stray_prior_path (`issue#20340 `_, `pr#15800 `_, "Yan, Zheng") * mgr: ceph-create-keys: update client.admin if it already exists (`issue#19940 `_, `pr#15112 `_, John Spray) * mgr: ceph: introduce "tell x help" subcommand (`issue#19885 `_, `pr#15111 `_, liuchang0812) * mgr: ceph-mgr: Implement new pecan-based rest api (`pr#14457 `_, Boris Ranto) * mgr: cleanup, stop clients sending in perf counters (`pr#15578 `_, John Spray) * mgr: dashboard code cleanup (`pr#15577 `_, John Spray) * mgr: dashboard GUI module (`pr#14946 `_, John Spray, Dan Mick) +* mgr: fix MgrStandby eating messages (`pr#15716 `_, John Spray) * mgr: load modules in separate python sub-interpreters (`pr#14971 `_, Tim Serong) * mgr: Mark session connections down on shutdown (`issue#19900 `_, `pr#15192 `_, Brad Hubbard) * mgr: mgr/DaemonServer.cc: log daemon type string as well as id (`pr#15560 `_, Dan Mick) +* mgr: mgr/dashboard: fix type error in get_rate function (`issue#20276 `_, `pr#15668 `_, liuchang0812) +* mgr: mgr/dashboard: load log lines on startup, split out audit log (`pr#15709 `_, John Spray) * mgr: mgr/MgrStandby: prevent use-after-free on just-shut-down Mgr (`issue#19595 `_, `pr#15297 `_, Sage Weil) * mgr: mgr/MgrStandby: respawn when deactivated (`issue#19595 `_, `issue#19549 `_, `pr#15557 `_, Sage Weil) * mgr: mgr,osd: ceph-mgr --help, unify usage text of other daemons (`pr#15176 `_, Tim Serong) * mgr,mon: mon,mgr: extricate PGmap from monitor (`issue#20067 `_, `issue#20174 `_, `issue#20050 `_, `pr#15073 `_, Kefu Chai, Sage Weil, Greg Farnum) * mgr,mon: mon/MgrMonitor: add 'mgr dump [epoch]' command (`pr#15158 `_, Sage Weil) +* mgr,mon: mon,mgr: print pgmap reports to debug (not cluster) log (`pr#15740 `_, Sage Weil) * mgr: optimize DaemonStateIndex::cull() a little bit (`pr#14967 `_, Kefu Chai) +* mgr: print a more helpful error message for when users lack mgr ceph caps (`issue#20296 `_, `pr#15697 `_, Greg Farnum) * mgr: pybind/mgr/dashboard: monkeypatch os.exit to stop cherrypy from taking down mgr (`issue#20216 `_, `pr#15588 `_, Sage Weil) * mgr: pybind/mgr: Delete `rest` module (`pr#15429 `_, John Spray) * mgr: pybind/mgr/restful: improve cert handling; work with vstart (`pr#15405 `_, Sage Weil) +* mgr: raise python exception on failure in send_command() (`pr#15704 `_, Kefu Chai) +* mgr: remove default cert; disable both restful and dashboard by default (`pr#15601 `_, Boris Ranto, Sage Weil) +* mgr,tests: qa/upgrade/jewel-x/point-to-point: add a mgr during final upgrade (`pr#15637 `_, Sage Weil) * mon: add crush type down health warnings (`pr#14914 `_, Neha Ojha) * mon: Add override for FsNewHandler::handle() (`pr#15331 `_, yonghengdexin735) * mon: cleanups (`pr#15272 `_, Kefu Chai) * mon: delete useless function definition (`pr#15188 `_, shiqi) +* mon: DIVIDE_BY_ZERO in PGMapDigest::dump_pool_stats_full() (`pr#15622 `_, Jos Collin) * mon: don't prefix mgr summary with epoch number (`pr#15512 `_, John Spray) * mon: fix accesing pending_fsmap from peon (`issue#20040 `_, `pr#15213 `_, John Spray) * mon: fix a few bugs with the osd health reporting (`pr#15179 `_, Sage Weil) * mon: Fixed typo in function comment blocks and in other comments (`pr#15304 `_, linbing) * mon: Fixed typo in @post of _active() (`pr#15191 `_, Linbing) * mon: fix mon_keyvaluedb application (`pr#15059 `_, Sage Weil) +* mon: Incorrect expression in PGMap::get_health() (`pr#15648 `_, Jos Collin) * mon: it's no need to get pg action_primary osd twice in pg scrub (`pr#15313 `_, linbing) +* mon: Log errors at startup (`issue#14088 `_, `pr#15723 `_, Kefu Chai) * mon: mon/MonClient: cancel pending commands on shutdown (`issue#20051 `_, `pr#15227 `_, Kefu Chai, Sage Weil) * mon: {mon,osd,mds} {versions,count-metadata} (`pr#15436 `_, Sage Weil) @@ -358,6 +469,8 @@ Notable Changes * msg: do not enable client-side binding by default (`issue#20049 `_, `pr#15392 `_, Jason Dillaman) * msg: don't set msgr addr when disabing client bind (`pr#15243 `_, Haomai Wang) * msgr: msg/async: Lower down the AsyncMessenger's standby warning from debug (`pr#15242 `_, Pan Liu) +* msgr: msg/async/rdma: check if fin message completed (`pr#15624 `_, Alexander Mikheev, Adir Lev) +* msgr: msg/async/rdma: handle buffers after close msg (`pr#15749 `_, DanielBar-On, Alexander Mikheev, Adir Lev) * msgr: msg/async: remove false alert "assert" (`pr#15288 `_, Haomai Wang) * osd: don't leak pgrefs or reservations in SnapTrimmer (`issue#19931 `_, `pr#15214 `_, Greg Farnum) * osd: fix argument-dependent lookup of swap() (`pr#15124 `_, Casey Bodley) @@ -383,6 +496,7 @@ Notable Changes * pybind: pybind/rados: avoid call free() on invalid pointer (`pr#15159 `_, Mingxin Liu) * pybind,rbd: pybind/rbd: OSError should be picklable (`issue#20223 `_, `pr#15574 `_, Jason Dillaman) * pybind: support mon target in pybind (`pr#15409 `_, liuchang0812) +* qa: test/osd/osd-dup.sh: use wait_for_clean (`pr#15722 `_, Dan Mick) * rbd-mirror: coordinate image syncs with leader (`issue#18789 `_, `pr#14745 `_, Mykola Golub) * rbd-mirror: lock loss during sync should wait for in-flight copies (`pr#15532 `_, Jason Dillaman) * rbd-mirror: permit release of local image exclusive lock after force promotion (`issue#18963 `_, `pr#15140 `_, Jason Dillaman) @@ -424,6 +538,7 @@ Notable Changes * rgw: rgw_file: add lock protection for readdir against gc (`issue#20121 `_, `pr#15329 `_, Gui Hecheng) * rgw: rgw_file cleanup names (`pr#15568 `_, Gui Hecheng) * rgw: rgw_file: fix flags set on unsuccessful unlink (`pr#15222 `_, Gui Hecheng) +* rgw: rgw_file: prevent conflict of mkdir between restarts (`issue#20275 `_, `pr#15655 `_, Gui Hecheng) * rgw: rgw_file: release rgw_fh lock and ref on ENOTEMPTY (`issue#20061 `_, `pr#15246 `_, Matt Benjamin) * rgw: rgw_file: removed extra rele() on fs in rgw_umount() (`pr#15152 `_, Gui Hecheng) * rgw: rgw_file: remove hidden uxattr objects from buckets on delete (`issue#20045 `_, `pr#15210 `_, Matt Benjamin) @@ -468,15 +583,19 @@ Notable Changes * tests: test: test_denc.cc: silence warning from -Wsign-compare (`pr#15355 `_, Jos Collin) * tests: test: Test fix for SnapSet change (`pr#15161 `_, David Zafman) * tests: test/unittest_bluefs: check whether mounted success (`pr#14988 `_, shiqi) +* tools: ceph-disk: command invocation needs all fields separate (`pr#15733 `_, Willem Jan Withagen) * tools: ceph.in: adjust usage width according to user's tty (`pr#15190 `_, Kefu Chai) * tools: ceph.in: assert(state==connected) before help_for_target() (`pr#15156 `_, Kefu Chai) * tools: ceph.in: drop the compatiiblity to handle non json commands (`pr#15508 `_, Kefu Chai) * tools: ceph.in: print return code when json_command failed (`pr#15378 `_, liuchang0812) +* tools: ceph-rest-api: be more tolerant on network failure (`issue#20115 `_, `pr#15706 `_, Kefu Chai) +* tools: fio_ceph_objectstore: Print db_statistics when rocksdb_perf is enabled (`pr#15796 `_, Xiaoyan Li) * tools: tools/ceph_kvstore_tool: add "bluestore-kv" to usage (`pr#15326 `_, xie xingguo) * tools: tools/crushtool: replicated-rule API support (`pr#15011 `_, xie xingguo) * tools: vstart: "debug_ms=1" for mgr by default (`pr#15127 `_, Kefu Chai) * tools: vstart: print "start osd.$id" instead of "start osd$id" (`pr#15427 `_, Kefu Chai) + v12.0.3 Luminous (dev) ======================