From: Abhishek Lekshmanan Date: Wed, 30 May 2018 09:44:07 +0000 (+0200) Subject: doc: release/mimic: correct the changelog to the latest version X-Git-Tag: v14.0.1~1226^2 X-Git-Url: http://git.apps.os.sepia.ceph.com/?a=commitdiff_plain;h=refs%2Fpull%2F22319%2Fhead;p=ceph.git doc: release/mimic: correct the changelog to the latest version We were still having the changelog from mimic-dev1, update this to reflect the current state of mimic Signed-off-by: Abhishek Lekshmanan --- diff --git a/doc/releases/mimic.rst b/doc/releases/mimic.rst index a671eedeffc22..137d56311c24e 100644 --- a/doc/releases/mimic.rst +++ b/doc/releases/mimic.rst @@ -445,11 +445,14 @@ These changes occurred between the Luminous and Mimic releases. new build dependency. -Other Notable Changes ---------------------- +Detailed Changelog +------------------ + +* arch/arm: set ceph_arch_aarch64_crc32 only if the build host supports crc32cx (`issue#19705 `_, `pr#17420 `_, Kefu Chai) +* assert(false)->ceph_abort() (`pr#18072 `_, Li Wang) +* auth: keep /dev/urandom open for get_random_bytes (`issue#21401 `_, `pr#17972 `_, Casey Bodley) * bluestore: BlueStore::ExtentMap::dup impl (`pr#19719 `_, Shinobu Kinjo) * bluestore: bluestore/NVMEDevice: accurate the latency perf counter of queue latency (`pr#17435 `_, Ziye Yang, Pan Liu) -* bluestore: bluestore/NVMEDevice: code cleanup (`pr#17284 `_, Ziye Yang, Pan Liu) * bluestore: bluestore/NVMEDevice: convert the legacy config opt related with SPDK (`pr#18502 `_, Ziye Yang) * bluestore: bluestore/NVMEDevice: do not deference a dangling pointer (`pr#19067 `_, Kefu Chai) * bluestore: bluestore/NVMEDevice: fix the bug in write function (`pr#17086 `_, Ziye Yang, Pan Liu) @@ -462,64 +465,88 @@ Other Notable Changes * bluestore: ceph-bluestore-tool: better fsck/repair, bluefs-bdev-{expand,sizes} (`pr#17709 `_, Sage Weil) * bluestore: ceph-bluestore-tool: check if bdev is empty on 'bluefs-bdev-expand' (`pr#17874 `_, WANG Guoqin) * bluestore: ceph-bluestore-tool: link target shouldn't ending with "\n" (`pr#18585 `_, Yao Zongyou) +* bluestore,common: intarith: get rid of P2\* and ROUND_UP\* macros (`pr#21085 `_, xie xingguo) * bluestore: comp_min_blob_size init error (`pr#18318 `_, linbing) * bluestore: config: Change bluestore_cache_kv_max to type INT64 (`pr#20255 `_, Zhi Zhang) * bluestore,core: ceph-bluestore-tool: prime-osd-dir: update symlinks instead of bailing (`pr#18565 `_, Sage Weil) +* bluestore,core: common/options: bluefs_buffered_io=true by default (`pr#20542 `_, Sage Weil) * bluestore,core: os/bluestore: compensate for bad freelistmanager size/blocks metadata (`issue#21089 `_, `pr#17268 `_, Sage Weil) * bluestore,core: os/bluestore: fix data read error injection in bluestore (`pr#19866 `_, Sage Weil) +* bluestore,core: os/bluestore: kv_max -> kv_min (`pr#20544 `_, Sage Weil) * bluestore,core: os/bluestore: switch default allocator to stupid; test both bitmap and stupid in qa (`pr#16906 `_, Sage Weil) -* bluestore,core,performance: common/options: bluefs_buffered_io=true by default (`pr#20542 `_, Sage Weil) -* bluestore,core,performance: os/bluestore: kv_max -> kv_min (`pr#20544 `_, Sage Weil) * bluestore,core: src/bluestore/NVMEDevice: make all read use aio_submit (`pr#17655 `_, Ziye Yang, Pan Liu) * bluestore,core,tests: test/unittest_bluefs: check whether rmdir success (`pr#15363 `_, shiqi) * bluestore,core: tool: ceph-kvstore-tool doesn't umount BlueStore properly (`issue#21625 `_, `pr#18083 `_, Chang Liu) * bluestore: define default value of LoglevelV only once (3 templates) (`pr#20727 `_, Matt Benjamin) * bluestore: drop unused friend class in SharedDriverQueueData (`pr#16894 `_, Pan Liu) +* bluestore: fix aio_t::rval type (`issue#23527 `_, `pr#21136 `_, kungf) +* bluestore: fix build on armhf (`pr#20951 `_, Kefu Chai) * bluestore: fixed compilation error when enable spdk with gcc 4.8.5 (`pr#16945 `_, Ziye Yang, Pan Liu) * bluestore: kv/RocksDBStore: extract common code to a new function (`pr#16532 `_, Pan Liu) -* bluestore: NVMDevice: fix issued caused by #17002 (`pr#17112 `_, Ziye Yang) +* bluestore/NVMEDevice: code cleanup (`pr#17284 `_, Ziye Yang, Pan Liu) * bluestore: os/bluestore: add bluestore_prefer_deferred_size_hdd/ssd to tracked keys (`pr#17459 `_, xie xingguo) +* bluestore: os/bluestore: add discard method for ssd's performance (`pr#14727 `_, Taeksang Kim) * bluestore: os/bluestore: Add lat record of deferred_queued and deferred_aio_wait (`pr#17015 `_, lisali) * bluestore: os/bluestore: Add missing __func__ in dout (`pr#17903 `_, lisali) +* bluestore: os/bluestore: add perf counter for allocator fragmentation (`pr#21377 `_, Igor Fedotov) +* bluestore: os/bluestore: allocate entire write in one go (`pr#17698 `_, Sage Weil) * bluestore: os/bluestore: allow reconstruction of osd data dir from bluestore bdev label (`pr#18256 `_, Sage Weil) +* bluestore: os/bluestore: alter the allow_eio policy regarding kernel's error list (`issue#23333 `_, `pr#21306 `_, Radoslaw Zarzynski) +* bluestore: os/bluestore: avoid excessive ops in _txc_release_alloc (`pr#18854 `_, Igor Fedotov) * bluestore: os/bluestore: avoid omit cache for remove-collection (`pr#18785 `_, Jianpeng Ma) +* bluestore: os/bluestore: avoid overhead of std::function in blob_t (`pr#20294 `_, Radoslaw Zarzynski) * bluestore: os/bluestore: avoid unneeded BlobRefing in _do_read() (`pr#19864 `_, Radoslaw Zarzynski) +* bluestore: os/bluestore: be more verbose when hitting unloaded shard in extent map (`pr#21245 `_, Igor Fedotov) * bluestore: os/bluestore/BlueFS: compact log even when sync_metadata sees no work (`pr#17354 `_, Sage Weil) * bluestore: os/bluestore/BlueFS: Don't call debug related code under any condition (`pr#17627 `_, Jianpeng Ma) +* bluestore: os/bluestore/BlueFS: don't need wait for aio when using _sync_write (`pr#16066 `_, Haodong Tang) * bluestore: os/bluestore/BlueFS: fix race with log flush during async log compaction (`issue#21878 `_, `pr#18428 `_, Sage Weil) * bluestore: os/bluestore/BlueFS: move release unused extents work in _flush_and_syn_log (`pr#17684 `_, Jianpeng Ma) * bluestore: os/bluestore/BlueFS: prevent _compact_log_async reentry (`issue#21250 `_, `pr#17503 `_, Sage Weil) * bluestore: os/bluestore/BlueFS: Reduce unnecessary operations in collect_metadata (`pr#17995 `_, Luo Kexue) * bluestore: os/bluestore/BlueFS: sanity check that alloc->allocate() won't return 0 (`pr#18259 `_, xie xingguo) * bluestore: os/bluestore/BlueFS: several cleanups (`pr#17966 `_, xie xingguo) +* bluestore: os/bluestore/bluefs_types: make block_mask 64-bit (`pr#21629 `_, Sage Weil) +* bluestore: os/bluestore/BlueStore: ASAP wake up _kv_finalize_thread (`pr#18203 `_, Jianpeng Ma) * bluestore: os/bluestore/BlueStore: narrow deferred_lock in _deferred_submit_unlock (`pr#17628 `_, Jianpeng Ma) +* bluestore: os/bluestore: bluestore repair should use interval_set::union_insert (`pr#20900 `_, Igor Fedotov) * bluestore: os/bluestore: cleanup around ExtentList, AllocExtent and bluestore_extent_t classes (`pr#20360 `_, Igor Fedotov) +* bluestore: os/bluestore: clearer comments, not slower code (`pr#16872 `_, Mark Nelson) * bluestore: os/bluestore: correctly check all block devices to decide if journal is_rotational (`issue#23141 `_, `pr#20602 `_, Greg Farnum) * bluestore: os/bluestore: delete redundant header file in KernelDevice.cc (`pr#18631 `_, Jing Li) +* bluestore: os/bluestore: do not assert if BlueFS rebalance is unable to allocate sufficient space (`pr#18494 `_, Igor Fedotov) * bluestore: os/bluestore: do not core dump when BlueRocksEnv gets EEXIST error (`issue#20871 `_, `pr#17357 `_, liuchang0812) * bluestore: os/bluestore: do not core dump when we try to open kvstore twice (`pr#18161 `_, Chang Liu) +* bluestore: os/bluestore: do not release empty bluefs_extents_reclaiming (`pr#18671 `_, Igor Fedotov) * bluestore: os/bluestore: do not segv on kraken upgrade debug print (`issue#20977 `_, `pr#16992 `_, Sage Weil) * bluestore: os/bluestore: don't re-initialize csum-setting for existing blobs (`issue#21175 `_, `pr#17398 `_, xie xingguo) +* bluestore: os/bluestore: do SSD discard on mkfs (`pr#20897 `_, Igor Fedotov) * bluestore: os/bluestore: drop deferred_submit_lock, fix aio leak (`issue#21171 `_, `pr#17352 `_, Sage Weil) * bluestore: os/bluestore: drop unused function declaration (`pr#18075 `_, Li Wang) * bluestore: os/bluestore: drop unused param "what" in apply() (`pr#17251 `_, songweibin) * bluestore: os/bluestore: _dump_onode() don't prolongate Onode anymore (`pr#19841 `_, Radoslaw Zarzynski) +* bluestore: os/bluestore: dynamic CF configuration; put pglog omap in separate CF (`pr#18224 `_, Sage Weil) * bluestore: os/bluestore: enlarege aligned_size avoid too many vector(> IOV_MAX) (`issue#21932 `_, `pr#18828 `_, Jianpeng Ma) * bluestore: os/bluestore: ExtentMap::reshard - fix wrong shard length (`pr#17334 `_, chenliuzhong) * bluestore: os/bluestore: fail early on very large objects (`issue#20923 `_, `pr#16924 `_, Sage Weil) * bluestore: os/bluestore: fix another aio stall/deadlock (`issue#21470 `_, `pr#18118 `_, Sage Weil) +* bluestore: os/bluestore: fix broken cap in _balance_bluefs_freespace() (`pr#21097 `_, Igor Fedotov) * bluestore: os/bluestore: fix clone dirty_range again (`issue#20983 `_, `pr#16994 `_, Sage Weil) * bluestore: os/bluestore: fix dirty_shard off-by-one (`pr#16850 `_, Sage Weil) +* bluestore: os/bluestore: fix exceeding the max IO queue depth in KernelDevice (`issue#23246 `_, `pr#20996 `_, Radoslaw Zarzynski) * bluestore: os/bluestore: fix potential assert when splitting collection (`pr#19519 `_, Igor Fedotov) * bluestore: os/bluestore: fix SharedBlob unregistration (`issue#22039 `_, `pr#18805 `_, Sage Weil) +* bluestore: os/bluestore: fix some code formatting (`pr#21037 `_, Gu Zhongyan) * bluestore: os/bluestore: fix the allocate in bluefs (`pr#19030 `_, tangwenjun) * bluestore: os/bluestore: fix the demotion in StupidAllocator::init_rm_free (`pr#20430 `_, Kefu Chai) * bluestore: os/bluestore: fix the wrong usage for map_any (`pr#18939 `_, Jianpeng Ma) * bluestore: os/bluestore: fix wrong usage for BlueFS::_allocate (`pr#20708 `_, Jianpeng Ma) +* bluestore: os/bluestore: free the spdk qpair resource correctly in destructor of SharedDriverQueueData (`pr#20929 `_, Jianyu Li) * bluestore: os/bluestore: handle small main device properly (`pr#17416 `_, xie xingguo) * bluestore: os/bluestore: ignore 0x2000~2000 extent oddity from luminous upgrade (`issue#21408 `_, `pr#17845 `_, Sage Weil) * bluestore: os/bluestore: implement BlueStore repair (`pr#19843 `_, Igor Fedotov) * bluestore: os/bluestore: make bluefs behave better near enospc (`pr#18120 `_, Sage Weil) +* bluestore: os/bluestore: mark derivatives of AioContext as final (`pr#20227 `_, Radoslaw Zarzynski) * bluestore: os/bluestore: move aio_callback{,_priv} to base class BlockDevice (`pr#17002 `_, mychoxin) * bluestore: os/bluestore: move assert of read/write to base class (`pr#17033 `_, mychoxin) * bluestore: os/bluestore: move size and block_size to the base class BlockDevice (`pr#16886 `_, Pan Liu) @@ -527,64 +554,56 @@ Other Notable Changes * bluestore: os/bluestore: no trim debug noise if there is no trimming to be done (`pr#20684 `_, Sage Weil) * bluestore: os/bluestore/NVMEDevice: change write_bl to bl (`pr#17145 `_, Ziye Yang, Pan Liu) * bluestore: os/bluestore/NVMEDevice: fix the nvme queue depth issue (`pr#17200 `_, Ziye Yang, Pan Liu) +* bluestore: os/bluestore/NVMEDevice: Remove using dpdk thread (`pr#17769 `_, Ziye Yang, Pan Liu) * bluestore: os/bluestore: OpSequencer: reduce kv_submitted_waiters if _is_all_kv_submitted() return true (`pr#18622 `_, Jianpeng Ma) * bluestore: os/bluestore: optimize _collection_list (`pr#18777 `_, Jianpeng Ma) * bluestore: os/bluestore: pass strict flag to bluestore_blob_use_tracker_t::equal() (`pr#15705 `_, xie xingguo) * bluestore: os/bluestore: Prealloc memory avoid realloc in list_collection (`pr#18804 `_, Jianpeng Ma) * bluestore: os/bluestore: prevent mount if osd_max_object_size >= 4G (`pr#19043 `_, Sage Weil) +* bluestore: os/bluestore: print aio in batch (`pr#18873 `_, Kefu Chai) * bluestore: os/bluestore: print leaked extents to debug output (`pr#17225 `_, Sage Weil) * bluestore: os/bluestore: propagate read-EIO to high level callers (`pr#17744 `_, xie xingguo) * bluestore: os/bluestore: put cached attrs in correct mempool (`issue#21417 `_, `pr#18001 `_, Sage Weil) * bluestore: os/bluestore: recalc_allocated() when decoding bluefs_fnode_t (`issue#23212 `_, `pr#20701 `_, Jianpeng Ma, Kefu Chai) +* bluestore: os/bluestore: reduce meaningless flush (`pr#19027 `_, tangwenjun) * bluestore: os/bluestore: refactor FreeListManager to get clearer view on the number (`issue#22535 `_, `pr#19718 `_, Igor Fedotov) +* bluestore: os/bluestore: release disk extents in bulky manner (`pr#17913 `_, Radoslaw Zarzynski) +* bluestore: os/bluestore: remove ineffective BlueFS fnode extent calculation (`pr#18905 `_, Igor Fedotov) * bluestore: os/bluestore: remove unused parameters (`pr#18635 `_, Jianpeng Ma) +* bluestore: os/bluestore: remove unused variable (`pr#21063 `_, Gu Zhongyan) * bluestore: os/bluestore: remove useless function submit (`pr#17537 `_, mychoxin) +* bluestore: os/bluestore: reorder members of bluefs_extent_t for space efficiency (`pr#21034 `_, Radoslaw Zarzynski) * bluestore: os/bluestore: replace dout with ldout in StupidAllocator (`pr#17404 `_, Radoslaw Zarzynski) * bluestore: os/bluestore: report error and quit correctly when disk error happens (`issue#21263 `_, `pr#17522 `_, Pan Liu) * bluestore: os/bluestore: Revert "os/bluestore: allow multiple DeferredBatches in flight at once" (`issue#20925 `_, `issue#20295 `_, `pr#16900 `_, Sage Weil) +* bluestore: os/bluestore: s/bluefs_total/bluefs_free/ (`pr#21036 `_, xie xingguo) * bluestore: os/bluestore: separate finisher for deferred_try_submit (`issue#21207 `_, `pr#17409 `_, Sage Weil) +* bluestore: os/bluestore: set bitmap freelist resolution to min_alloc_size (`pr#17610 `_, Sage Weil) * bluestore: os/bluestore: shrink aio submit size to pending value (`pr#17588 `_, kungf) * bluestore: os/bluestore: silence -Wreturn-type warning (`pr#18286 `_, Kefu Chai) * bluestore: os/bluestore: support calculate cost when using spdk (`pr#17091 `_, Ziye Yang, Pan Liu) +* bluestore: os/bluestore: synchronous on_applied completions (`pr#18196 `_, Sage Weil) +* bluestore: os/bluestore: trim cache every 50ms (instead of 200ms) (`pr#20498 `_, Sage Weil) +* bluestore: os/bluestore: update description for bluestore_compression\_[min|max|_blob_size options (`pr#21244 `_, Igor Fedotov) * bluestore: os/bluestore: using macro OBJECT_MAX_SIZE to check osd_max_object_size (`pr#19622 `_, Jianpeng Ma) +* bluestore: osd/bluestore: delete unused variable in KernelDevice (`pr#20857 `_, Leo Zhang) * bluestore: osd,os/bluestore: Display current size of osd_max_object_size (`pr#19725 `_, Shinobu Kinjo) -* bluestore,performance: os/bluestore: add discard method for ssd's performance (`pr#14727 `_, Taeksang Kim) -* bluestore,performance: os/bluestore: allocate entire write in one go (`pr#17698 `_, Sage Weil) -* bluestore,performance: os/bluestore: avoid excessive ops in _txc_release_alloc (`pr#18854 `_, Igor Fedotov) -* bluestore,performance: os/bluestore: avoid overhead of std::function in blob_t (`pr#20294 `_, Radoslaw Zarzynski) -* bluestore,performance: os/bluestore/BlueFS: don't need wait for aio when using _sync_write (`pr#16066 `_, Haodong Tang) -* bluestore,performance: os/bluestore/BlueStore: ASAP wake up _kv_finalize_thread (`pr#18203 `_, Jianpeng Ma) -* bluestore,performance: os/bluestore: clearer comments, not slower code (`pr#16872 `_, Mark Nelson) -* bluestore,performance: os/bluestore: do not assert if BlueFS rebalance is unable to allocate sufficient space (`pr#18494 `_, Igor Fedotov) -* bluestore,performance: os/bluestore: do not release empty bluefs_extents_reclaiming (`pr#18671 `_, Igor Fedotov) -* bluestore,performance: os/bluestore: dynamic CF configuration; put pglog omap in separate CF (`pr#18224 `_, Sage Weil) -* bluestore,performance: os/bluestore: mark derivatives of AioContext as final (`pr#20227 `_, Radoslaw Zarzynski) -* bluestore,performance: os/bluestore/NVMEDevice: Remove using dpdk thread (`pr#17769 `_, Ziye Yang, Pan Liu) -* bluestore,performance: os/bluestore: print aio in batch (`pr#18873 `_, Kefu Chai) -* bluestore,performance: os/bluestore: reduce meaningless flush (`pr#19027 `_, tangwenjun) -* bluestore,performance: os/bluestore: release disk extents in bulky manner (`pr#17913 `_, Radoslaw Zarzynski) -* bluestore,performance: os/bluestore: remove ineffective BlueFS fnode extent calculation (`pr#18905 `_, Igor Fedotov) -* bluestore,performance: os/bluestore: set bitmap freelist resolution to min_alloc_size (`pr#17610 `_, Sage Weil) -* bluestore,performance: os/bluestore: synchronous on_applied completions (`pr#18196 `_, Sage Weil) -* bluestore,performance: os/bluestore: trim cache every 50ms (instead of 200ms) (`pr#20498 `_, Sage Weil) * bluestore: Revert "os/bluestore: pass strict flag to bluestore_blob_use_tracker_t::equal()" (`issue#21293 `_, `pr#17569 `_, Sage Weil) * bluestore,rgw: rgw,unittest_bit_alloc: silence clang analyzer warning (`pr#17294 `_, Kefu Chai) * bluestore,tests: objectstore/store_test: fix lack of flush prior to collection_empty()… (`issue#22409 `_, `pr#19764 `_, Igor Fedotov) * bluestore,tests: Revert "bluestore/fio: Fixed problem with all objects having the same hash (`pr#18352 `_, Radoslaw Zarzynski) * bluestore,tools: ceph-bluestore-tool: create out_dir before create full path of kvdb (`pr#18367 `_, Leo Zhang) * bluestore,tools: os/bluestore/bluestore_tool: add log-dump command to dump bluefs's log (`pr#18535 `_, Yang Honggang) -* build/ops: Add noreturn attribute to silence uninitialized warning (`pr#19348 `_, Adam C. Emerson) -* build/ops: arch/arm: set ceph_arch_aarch64_crc32 only if the build host supports crc32cx (`issue#19705 `_, `pr#17420 `_, Kefu Chai) -* build/ops: assert(false)->ceph_abort() (`pr#18072 `_, Li Wang) +* build: fix dpdk build error (`pr#18087 `_, chunmei) +* build mimic-dev1 with gcc 7 (`issue#22438 `_, `pr#19548 `_, Kefu Chai) * build/ops: automake: remove files required by automake (`pr#17937 `_, Kefu Chai) * build/ops: blkin: link against lttng-ust-fork (`pr#17673 `_, Mohamad Gebai) * build/ops: boost: remove boost submodule (`pr#17405 `_, Kefu Chai) * build/ops: build: do_cmake: allow ARGS to be overridden (`pr#19876 `_, Abhishek Lekshmanan) -* build/ops: build: fix dpdk build error (`pr#18087 `_, chunmei) -* build/ops: build mimic-dev1 with gcc 7 (`issue#22438 `_, `pr#19548 `_, Kefu Chai) * build/ops: build: remove PGMap.cc from libcommon (`pr#18496 `_, Sage Weil) * build/ops: ceph-disk activate unlocks bluestore data partition (`issue#20488 `_, `pr#16357 `_, Felix Winterhalter) * build/ops: ceph_disk: allow "no fsid" on activate (`pr#18991 `_, Dan Mick) +* build/ops,cephfs: ceph-object-corpus: update to fix make check (`pr#21261 `_, Patrick Donnelly) * build/ops,cephfs: cmake, test/fs, client: fix build with clang (`pr#20392 `_, Adam C. Emerson) * build/ops: ceph.spec: use devtoolset-6-gcc-c++ on aarch64 (`issue#22301 `_, `pr#19341 `_, Kefu Chai) * build/ops: ceph-volume: Require lvm2, move to osd package (`issue#22443 `_, `pr#19529 `_, Theofilos Mouratidis) @@ -593,11 +612,17 @@ Other Notable Changes * build/ops: cmake/BuildBoost: fixes to ready seastar (`pr#20616 `_, Kefu Chai, Casey Bodley) * build/ops: cmake,deb: install system units using cmake (`pr#20618 `_, Kefu Chai) * build/ops: cmake: link libcommon with libstdc++ statically if WITH_STATIC_LIBSTDCXX (`issue#22438 `_, `pr#19515 `_, Kefu Chai) +* build/ops: cmake,make-dist: bump up boost version to 1.67 (`pr#21572 `_, Kefu Chai) +* build/ops: cmake,mds: detect std::map::merge() before using it (`pr#21211 `_, Willem Jan Withagen, Kefu Chai) +* build/ops: cmake/mgr: use Python 3 virtualenv if mgr subinterpreter is Python 3 (`pr#21446 `_, Nathan Cutler) +* build/ops,common: cmake, common: silence cmake and gcc warnings (`issue#23774 `_, `pr#21484 `_, Kefu Chai) * build/ops: common/time: add time.h for Alpine build (`pr#19863 `_, huanwen ren) * build/ops,common: Update C++ standard to 14 and clean up (`pr#19490 `_, Adam C. Emerson) * build/ops,core: ceph-crush-location: remove (`pr#19881 `_, Sage Weil) * build/ops,core: ceph-volume: do not use --key during mkfs (`issue#22283 `_, `pr#19276 `_, Kefu Chai, Sage Weil) * build/ops,core: /etc/sysconfig/ceph: remove jemalloc option (`issue#20557 `_, `pr#18487 `_, Sage Weil) +* build/ops,core: mimic: cmake,common,filestore: silence gcc-8 warnings/errors (`pr#21862 `_, Kefu Chai) +* build/ops,core: mimic: cmake: do not check for aligned_alloc() anymore (`issue#23653 `_, `pr#22048 `_, Kefu Chai) * build/ops,core: msg/async: update to work with dpdk shipped with spdk v17.10 (`pr#19470 `_, Kefu Chai) * build/ops,core: zstd: Upgrade to v1.3.2 (`pr#18407 `_, Adam C. Emerson) * build/ops: debian/control: adjust ceph-{osdomap,kvstore,monstore}-tool feature move (`issue#22319 `_, `pr#19328 `_, Sage Weil) @@ -607,15 +632,19 @@ Other Notable Changes * build/ops: debian: lock ceph user during purge (`pr#15118 `_, Caleb Boylan) * build/ops: debian/rules: no more ChangeLog (`pr#18023 `_, Sage Weil) * build/ops: debian/rules: strip ceph-base libraries (`issue#22640 `_, `pr#19870 `_, Sage Weil) -* build/ops: do_{cmake,freebsd}: Don't invoke nproc(1) on FreeBSD (`pr#17949 `_, Alan Somers) +* build/ops: do\_{cmake,freebsd}: Don't invoke nproc(1) on FreeBSD (`pr#17949 `_, Alan Somers) * build/ops: dpdk: remove redundant dpdk submodule (`pr#18712 `_, chunmei) +* build/ops: EventKqueue: Clang want realloc return to be typed (`pr#21550 `_, Willem Jan Withagen) +* build/ops: filestore,rgw: fix types/casts making clang on 32-Bit working (`pr#21055 `_, Daniel Glaser) * build/ops: Fix ppc64 support for ceph (`pr#16753 `_, Boris Ranto) * build/ops: Fix two dpdk assert happened in dpdk library (`pr#18409 `_, chunmei) +* build/ops: FreeBSD: add new required packages to be installed (`pr#21349 `_, Willem Jan Withagen) * build/ops: githubmap: add some known Ceph reviewers (`pr#17507 `_, Patrick Donnelly) * build/ops: .githubmap: Add wjwithagen as a known Ceph reviewer (`pr#17518 `_, Willem Jan Withagen) * build/ops: .githubmap: Update (`pr#18230 `_, Sage Weil) * build/ops: .gitignore: allow debian .patch files (`pr#17577 `_, Ken Dreyer) * build/ops: include: compat.h, fix the return result of pthread_set_name() (`pr#20474 `_, Willem Jan Withagen) +* build/ops: install-deps: Add support for 'opensuse-tumbleweed' (`pr#21650 `_, Ricardo Marques) * build/ops: install-deps.sh: avoid re-installing g++-7 (`pr#19468 `_, Kefu Chai) * build/ops: install-deps.sh, cmake: use GCC-7 on xenial also (`pr#19418 `_, Kefu Chai) * build/ops: install-deps.sh: install new gcc as the default the right way (`pr#19417 `_, Kefu Chai) @@ -629,20 +658,28 @@ Other Notable Changes * build/ops: make-dist,cmake: avoid re-downloading boost (`pr#19124 `_, Kefu Chai) * build/ops: make-dist,cmake: move boost tarball location to download.ceph.com (`pr#17980 `_, Sage Weil) * build/ops: make-dist,cmake: Try multiple URLs to download boost before failing (`pr#18048 `_, Brad Hubbard) +* build/ops: make-dist: fall back to python3 (`pr#21127 `_, Nathan Cutler) +* build/ops,mgr: mgr/dashboard: build tweaks (`pr#20752 `_, John Spray) +* build/ops,mgr: mgr/dashboard: remove node/npm system installation (`pr#20898 `_, Tiago Melo) * build/ops,mgr: packaging: explicit jinja2 dependency for dashboard (`issue#22457 `_, `pr#19598 `_, John Spray) +* build/ops,mgr,tests: mgr/dashboard: replace dashboard with dashboard_v2 (`pr#20912 `_, Ricardo Dias) +* build/ops: mimic: cmake: use javac -h for creating JNI native headers (`issue#24012 `_, `pr#21824 `_, Kefu Chai) +* build/ops: mimic: silence various warnings to enable GCC-8 build (`pr#22081 `_, Adam C. Emerson, Kefu Chai) * build/ops: mon,osd: do not use crush_device_class file to initalize class for new osds (`pr#19939 `_, Sage Weil) * build/ops: mstart.sh: support read CLUSTERS_LIST from env var (`pr#16988 `_, Jiaying Ren) +* build/ops: os/CMakeLists: fix link errro when enable WITH_PMEM=ON (`pr#20658 `_, Jianpeng Ma) * build/ops: osdc,os,osd: fix build on osx (`pr#20029 `_, Kefu Chai) * build/ops: python-numpy-devel build dependency for SUSE (`issue#21176 `_, `pr#17366 `_, Nathan Cutler) +* build/ops: qa/tests - added for the suites with subset be able to use 'testing' … (`pr#21454 `_, Yuri Weinstein) * build/ops,rbd: ceph-dencoder: moved RBD types outside of RGW preprocessor guard (`issue#22321 `_, `pr#19343 `_, Jason Dillaman) * build/ops: rbdmap: fix umount when multiple mounts use the same RBD (`pr#17978 `_, Alexandre Marangone) * build/ops: Revert "make-dist: add OBS-specific release suffix on SUSE" (`pr#20813 `_, Nathan Cutler) -* build/ops: Revert "msg/async/rdma: fix multi cephcontext confllicting" (`pr#16980 `_, Haomai Wang) -* build/ops: Revert "os/bluestore: compensate for bad freelistmanager size/blocks metadata" (`pr#17275 `_, Xie Xingguo) * build/ops,rgw: radosgw: Make compilation with CryptoPP possible (`pr#14955 `_, Adam Kupczyk) +* build/ops: rocksdb: do not use aligned_alloc (`issue#23653 `_, `pr#21632 `_, Kefu Chai) * build/ops: rpm: adjust ceph-{osdomap,kvstore,monstore}-tool feature move (`issue#22558 `_, `pr#19777 `_, Kefu Chai) * build/ops: rpm: build-depends on "cunit-devel" for suse (`pr#18997 `_, Kefu Chai) * build/ops: rpm: conditionalize Python 2 availability to enable Ceph build on Python 3-only system (`pr#20018 `_, Nathan Cutler) +* build/ops: rpm,debian: Ensure all ceph-disk runtime dependencies are declared for ceph-base (`issue#23657 `_, `pr#21356 `_, Nathan Cutler) * build/ops: rpm,deb: package ceph-kvstore-tool man page (`pr#17387 `_, Sage Weil) * build/ops: rpm: drop legacy librbd.so.1 symlink in /usr/lib64/qemu (`pr#17324 `_, Nathan Cutler) * build/ops: rpm: fix _defined_if_python2_absent conditional (`pr#20166 `_, Nathan Cutler) @@ -651,52 +688,53 @@ Other Notable Changes * build/ops: rpm: Python 3-only ceph-disk and ceph-volume (`pr#20140 `_, Nathan Cutler) * build/ops: rpm: recommend chrony instead of ntp-daemon (`pr#20138 `_, Nathan Cutler) * build/ops: rpm: recommend python-influxdb with ceph-mgr (`pr#18511 `_, Nathan Cutler, Tim Serong) +* build/ops: rpm: Revert "ceph.spec: work around build.opensuse.org" (`pr#21716 `_, Nathan Cutler) * build/ops: rpm: rip out rcceph script (`pr#19899 `_, Nathan Cutler) -* build/ops: rpm: rm macros in comments (`issue#22250 `_, `pr#17070 `_, Ken Dreyer) * build/ops: rpm: selinux-policy fixes (`pr#19026 `_, Brad Hubbard) * build/ops: rpm: set build parallelism based on available memory (`pr#19122 `_, Nathan Cutler, Richard Brown) * build/ops: rpm: set permissions 0755 on rbd resource agent (`issue#22362 `_, `pr#19494 `_, Nathan Cutler) * build/ops: run-make-check.sh: fix SUSE support (`issue#22875 `_, `pr#20234 `_, Nathan Cutler) +* build/ops: run-make-check.sh: handle Python 2 absence (`issue#23035 `_, `pr#20480 `_, Nathan Cutler) * build/ops: run-make-check.sh: run ulimit without sudo (`pr#17361 `_, yang.wang) -* build/ops: script/build-integration-branch: check errors (`pr#17578 `_, Sage Weil) * build/ops: script/build-integration-branch: print pr url list with titles (`pr#17426 `_, Sage Weil) -* build/ops: script/build-integration-branch: python3 compatible and pep8 clean (`pr#18035 `_, Kefu Chai) -* build/ops: selinux: Allow getattr on lnk sysfs files (`pr#17891 `_, Boris Ranto) * build/ops: selinux: Allow nvme devices (`issue#19200 `_, `pr#15597 `_, Boris Ranto) +* build/ops: setup-virtualenv.sh: do not hardcode python binary (`issue#23437 `_, `pr#21002 `_, Nathan Cutler) * build/ops: spdk: update SPDK to fix the build failure on aarch64 (`pr#20134 `_, Tone Zhang, Kefu Chai) * build/ops: spdk: update SPDK to v17.10 (`pr#19208 `_, Kefu Chai) * build/ops: spdk: update submodule to more recent upstream (`pr#20077 `_, Nathan Cutler) +* build/ops: specs: require of e2fsprogs (`pr#21345 `_, Guillaume Abrioux) * build/ops: src/script/build-integration-branch (`pr#17382 `_, Sage Weil) +* build/ops: src: s/pip --use-wheel/pip/ (`pr#21159 `_, Kefu Chai) * build/ops: submodule: make libmpem as a submodule (`pr#17036 `_, Jianpeng Ma) * build/ops: sysctl.d: set kernel.pid_max=4194304 (`issue#21929 `_, `pr#18544 `_, David Disseldorp) * build/ops: systemd: rbd-mirror does not start on reboot (`pr#17969 `_, Sébastien Han) +* build/ops: test: delete asok directories correctly (`pr#21023 `_, Chang Liu) * build/ops: test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code (`pr#20535 `_, Igor Fedotov) * build/ops,tests: common,test,cmake: various changes to re-enable build on osx (`pr#18888 `_, Kefu Chai) +* build/ops,tests: qa/tests: Changed rhel7.4 to rhel7.5 (`pr#21336 `_, Yuri Weinstein) * build/ops,tests: test/fio: fix fio objectstore plugin building broken by (`pr#20514 `_, Igor Fedotov) -* build/ops: The Day Has Come! (`pr#19657 `_, Adam C. Emerson) * build/ops: udev: Fix typo in udev OSD rules file (`pr#18976 `_, Mitch Birti) * build/ops: use devtoolset-7 on centos/rhel-7 (`pr#18863 `_, Kefu Chai) -* ceph-disk: fix '--runtime' omission for ceph-osd service (`issue#21498 `_, `pr#17904 `_, Carl Xiong) -* ceph-disk: fix signed integer is greater than maximum when call major (`pr#19196 `_, Song Shun) -* ceph-disk: include output of failed command in exception (`pr#20497 `_, Kefu Chai) -* ceph-disk: more precise error message when a disk is specified (`pr#18018 `_, Kefu Chai) -* ceph-disk: retry on OSError (`issue#21728 `_, `pr#18162 `_, Kefu Chai) -* ceph-disk: unlock all partitions when activate (`pr#17363 `_, Kefu Chai) * cephfs: Client:Fix readdir bug (`pr#18784 `_, dongdong tao) * cephfs: Client: setattr should drop "Fs" rather than "As" for mtime and size (`pr#18786 `_, dongdong tao) -* cephfs,common,performance,rbd: common/common_init: disable ms subsystem log gathering for clients (`issue#21860 `_, `pr#18418 `_, Jason Dillaman) +* cephfs,common,rbd: common/common_init: disable ms subsystem log gathering for clients (`issue#21860 `_, `pr#18418 `_, Jason Dillaman) +* cephfs,common,rbd: Various fixes for SCA issues (`pr#21708 `_, Danny Al-Gaaf) +* cephfs,core: mon/OSDMonitor: set FLAG_SELFMANAGED_SNAPS on cephfs snap removal (`issue#23949 `_, `pr#21756 `_, Sage Weil) * cephfs: MDS: add null check before we push_back "onfinish" (`pr#18892 `_, dongdong tao) * cephfs: MDS: correct the error msg when init mon client (`pr#18836 `_, dongdong tao) * cephfs: MDS: make popular counter decay at proper rate (`pr#18776 `_, Jianyu Li) * cephfs: MDS: make rebalancer evaluate the overload state of each mds with the same criterion (`pr#19255 `_, Jianyu Li) * cephfs: messages: Initialization of is_primary (`pr#16897 `_, amitkuma) * cephfs: messages: Initialization of member variables (`pr#16898 `_, amitkuma) +* cephfs: mimic: MDSMonitor: clean up use of pending fsmap in uncommitted ops (`issue#23768 `_, `pr#22005 `_, Patrick Donnelly) * cephfs: mon/MDSMonitor: wait for readable OSDMap before sanitizing (`issue#21945 `_, `pr#18603 `_, Patrick Donnelly) * cephfs,mon: mon/MDSMonitor: fix a bug at preprocess_beacon (`pr#17415 `_, wangshuguang) * cephfs: osdc/Journaler: use new style options (`pr#17806 `_, Kefu Chai) * cephfs: qa: check pool full flags (`issue#22475 `_, `pr#19588 `_, Patrick Donnelly) +* cephfs: qa: fix typo in test_full (`issue#23643 `_, `pr#21334 `_, Patrick Donnelly) * cephfs: Revert "ceph_context: re-expand admin_socket metavariables in child process" (`pr#18545 `_, Patrick Donnelly) * cephfs,tests: qa/suites/powercycle/osd/whitelist_health: whitelist slow trimming (`pr#17307 `_, Sage Weil) +* cephfs,tests: qa/workunits/cephtool/test.sh: fix test_mon_mds() (`pr#21579 `_, Kefu Chai) * cephfs,tools: mount.fuse.ceph: Fix typo (`pr#19128 `_, Jos Collin) * cephfs: vstart_runner: fixes for recent cephfs changes (`pr#19533 `_, Patrick Donnelly) * ceph-volume: add ANSIBLE_SSH_RETRIES=5 to functional tests (`pr#20592 `_, Andrew Schoen) @@ -712,48 +750,70 @@ Other Notable Changes * ceph-volume: adds support to zap encrypted devices (`issue#22878 `_, `pr#20537 `_, Andrew Schoen) * ceph-volume: adds the ceph-volume lvm zap subcommand (`pr#18513 `_, Andrew Schoen) * ceph-volume allow filtering by `uuid`, do not require osd id (`pr#17606 `_, Andrew Schoen, Alfredo Deza) +* ceph-volume: allow parallel creates (`issue#23757 `_, `pr#21489 `_, Theofilos Mouratidis) +* ceph-volume: allow skipping systemd interactions on activate/create (`issue#23678 `_, `pr#21496 `_, Alfredo Deza) * ceph-volume: allow using a device or partition for `lvm --data` (`pr#18924 `_, Alfredo Deza) * ceph-volume be resilient to $PATH issues (`pr#20650 `_, Alfredo Deza) * ceph-volume consume mount/format options from ceph.conf (`pr#20408 `_, Alfredo Deza) * ceph-volume: correctly fallback to bluestore when no objectstore is specified (`pr#19213 `_, Alfredo Deza) * ceph-volume correctly normalize mount flags (`pr#20543 `_, Alfredo Deza) +* ceph-volume: create the ceph-volume and ceph-volume-systemd man pages (`issue#21030 `_, `pr#17152 `_, Alfredo Deza) * ceph-volume: dmcrypt support for lvm (`issue#22619 `_, `pr#20054 `_, Alfredo Deza) * ceph-volume dmcrypt support for simple (`issue#22620 `_, `pr#20264 `_, Andrew Schoen, Alfredo Deza) * ceph-volume/doc: add missing subcommand in examples (`pr#19381 `_, Guillaume Abrioux) * ceph-volume: ensure correct --filestore/--bluestore behavior (`pr#18518 `_, Alfredo Deza) +* ceph-volume failed ceph-osd --mkfs command doesn't halt the OSD creation process (`issue#23874 `_, `pr#21685 `_, Alfredo Deza) +* ceph-volume: fix action plugins path in tests (`pr#20910 `_, Guillaume Abrioux) * ceph-volume fix filestore OSD creation after mon-config changes (`issue#23260 `_, `pr#20787 `_, Alfredo Deza) +* ceph-volume: fix typo in ceph-volume lvm prepare help (`pr#21196 `_, Jeffrey Zhang) * ceph-volume: fix usage of the --osd-id flag (`issue#22642 `_, `issue#22836 `_, `pr#20203 `_, Andrew Schoen) * ceph-volume Format correctly when vg/lv cannot be used (`issue#22299 `_, `pr#19285 `_, Alfredo Deza) * ceph-volume handle inline comments in the ceph.conf file (`issue#22297 `_, `pr#19319 `_, Alfredo Deza) * ceph-volume: handle leading whitespace/tabs in ceph.conf (`issue#22280 `_, `pr#19259 `_, Alfredo Deza) +* ceph-volume Implement an 'activate all' to help with dense servers or migrating OSDs (`pr#21130 `_, Alfredo Deza) +* ceph-volume improve robustness when reloading vms in tests (`pr#21070 `_, Alfredo Deza) * ceph-volume: log the current running command for easier debugging (`issue#23004 `_, `pr#20594 `_, Andrew Schoen) * ceph-volume lvm api refactor/move (`pr#18110 `_, Alfredo Deza) * ceph-volume lvm list (`pr#18095 `_, Alfredo Deza) * ceph-volume lvm.prepare update to use create_osd_path (`pr#18514 `_, Alfredo Deza) * ceph-volume: lvm zap will unmount osd paths used by zapped devices (`issue#22876 `_, `pr#20265 `_, Andrew Schoen) +* ceph-volume: Nits noticed while studying code (`pr#21455 `_, Dan Mick) * ceph-volume Persist non-lv devices for journals (`pr#17403 `_, Alfredo Deza) +* ceph-volume process the abspath of the executable first (`issue#23259 `_, `pr#20824 `_, Alfredo Deza) * ceph-volume: removed the explicit use of sudo (`issue#22282 `_, `pr#19363 `_, Andrew Schoen) +* ceph-volume: remove extra space (`pr#21140 `_, Sébastien Han) * ceph-volume rollback on failed OSD prepare/create (`issue#22281 `_, `pr#19351 `_, Alfredo Deza) * ceph-volume should be able to handle multiple LVM (VG/LV) tags (`issue#22305 `_, `pr#19321 `_, Alfredo Deza) * ceph-volume: support GPT and other deployed OSDs (`pr#18823 `_, Alfredo Deza) +* ceph-volume tests add optional flags for vagrant (`pr#20849 `_, Alfredo Deza) * ceph-volume tests alleviate libvirt timeouts when reloading (`issue#23163 `_, `pr#20718 `_, Alfredo Deza) * ceph-volume tests.devices.lvm prepare isn't bluestore specific anymore (`pr#18984 `_, Alfredo Deza) * ceph-volume tests remove unused import (`pr#20459 `_, Alfredo Deza) +* ceph-volume tests use granular env vars for vagrant (`pr#20864 `_, Alfredo Deza) * ceph-volume: Try to cast OSD metadata to int while scanning directory (`pr#19477 `_, Wido den Hollander) +* ceph-volume update man page for prepare/activate flags (`pr#21570 `_, Alfredo Deza) * ceph-volume use realpath when checking mounts (`issue#22988 `_, `pr#20427 `_, Alfredo Deza) * ceph-volume: use unique logical volumes (`pr#17207 `_, Alfredo Deza) +* ceph-volume: Using --readonly for {vg|pv|lv}s commands (`pr#21409 `_, Erwan Velu) * ceph-volume: warn on missing ceph.conf file (`issue#22326 `_, `pr#19347 `_, Alfredo Deza) * ceph-volume warn on mix of filestore and bluestore flags (`issue#23003 `_, `pr#20513 `_, Alfredo Deza) +* cleanup: Replacing MIN,MAX with std::min,std::max (`pr#18124 `_, Amit Kumar) * cli: rados: support for high precision time using stat2 (`issue#21199 `_, `pr#17395 `_, Abhishek Lekshmanan) +* cls_acl/_crypto: Add modeline (`pr#19010 `_, Shinobu Kinjo) +* cmake: add chrono to BOOST_COMPONENTS (`issue#23424 `_, `pr#20977 `_, Nathan Cutler) * cmake: add cython_rbd as a dependency to vstart target (`pr#18382 `_, Ali Maredia) * cmake: bail out if GCC version is less than 5.1 (`pr#19344 `_, Kefu Chai) * cmake: BuildBoost.cmake: use specified compiler for building boost (`pr#19898 `_, Kefu Chai) +* cmake: bump target jdk to 1.7 (`issue#23458 `_, `pr#21082 `_, Shengjing Zhu) * cmake: bump up required cmake version to 2.8.12 (`pr#18285 `_, Kefu Chai) +* cmake: changes of BuildBoost.cmake to ready seastar (`pr#21404 `_, Kefu Chai) * cmake: check for aligned_alloc() instead of checking tcmalloc version (`pr#18557 `_, Kefu Chai) * cmake: check gcc version not release date for libstdc++ saneness (`pr#18938 `_, Kefu Chai) * cmake: check version of boost in src/boost (`pr#19914 `_, Kefu Chai) * cmake: cleanups (`pr#18597 `_, Kefu Chai) +* cmake,common: changes to port part of ceph to osx (`pr#17615 `_, Kefu Chai) * cmake: compile nvml as an external project (`pr#17462 `_, Jianpeng Ma) +* cmake: define HAVE_STDLIB_MAP_SPLICING for both libstdc++ and libc++ (`pr#21284 `_, Kefu Chai) * cmake: disable DOWNLOAD_NO_PROGRESS if cmake ver is lower than 3.1 (`pr#20492 `_, Kefu Chai) * cmake: disable FAIL_ON_WARNINGS for rocksdb (`pr#19426 `_, Kefu Chai) * cmake: disable VTA on options.cc (`pr#17393 `_, Kefu Chai) @@ -761,14 +821,24 @@ Other Notable Changes * cmake: do not link against librados.a (`pr#18576 `_, Kefu Chai) * cmake: do not link against unused or duplicated libraries (`pr#18092 `_, Kefu Chai) * cmake: enabled py3 only build (`pr#20064 `_, Kefu Chai) +* cmake: enable LZ4 by default (`pr#21332 `_, Grant Slater, Casey Bodley) +* cmake: enable new policies to silence cmake warnings (`pr#21662 `_, Kefu Chai) +* cmake: fix building without mgr module (`pr#21591 `_, Yuan Zhou) +* cmake: fix frontend cmake build (`pr#21449 `_, Ricardo Dias) * cmake: fix libcephfs-test.jar build failure (`issue#22828 `_, `pr#20175 `_, Tone Zhang) * cmake: fix the include dir for building boost::python (`pr#20324 `_, Kefu Chai) +* cmake: fix typo in status message (`pr#21464 `_, Lenz Grimmer) +* cmake: hide symbols import from other libraries in libcls\_\* (`issue#23517 `_, `pr#21571 `_, Kefu Chai) * cmake: identify the possible incompatibility of rocksdb and tcmalloc (`issue#21422 `_, `pr#17788 `_, Kefu Chai) * cmake: in case of bad "ALLOCATOR" selected issue warning (`pr#17422 `_, Adam Kupczyk) +* cmake: include frontend build in 'all' target (`pr#21466 `_, John Spray) +* cmake: let "tests" depend on "mgr-dashboard-frontend-build" (`pr#21468 `_, Kefu Chai) * cmake: 'make check' builds radosgw and its cls dependencies (`pr#20422 `_, Casey Bodley) * cmake: mgr: exclude .gitignore (`pr#19174 `_, Nathan Cutler) +* cmake/modules/BuildRocksDB.cmake: enable compressions for rocksdb (`issue#24025 `_, `pr#22183 `_, Kefu Chai) * cmake: only create sysctl file on linux (`pr#19029 `_, Kefu Chai) * cmake: pass static linkflags to the linker who links libcommon (`pr#19763 `_, Kefu Chai) +* cmake: s/boost_256/boost_sha256/ (`pr#21573 `_, Kefu Chai) * cmake: set supported language the right way (`pr#18216 `_, Kefu Chai) * cmake: should use the value of GPERFTOOLS_LIBRARIES as REQUIRED_VARS (`pr#18645 `_, Kefu Chai) * cmake: s/sysconf/sysconfig/ (`pr#20631 `_, Kefu Chai) @@ -781,49 +851,61 @@ Other Notable Changes * cmake: WITH_SPDK=ON by default (`pr#18944 `_, Liu-Chunmei, Kefu Chai, wanjun.lp, Ziye Yang) * common: adding line break at end of some cli results (`issue#21019 `_, `pr#16687 `_, songweibin) * common: add line break for "ceph daemon TYPE.ID version" (`pr#17146 `_, Zhu Shangzhong) +* common: Add metadata with only Ceph version number and release (`pr#21095 `_, Wido den Hollander) * common: Add min/max of ms_async_op_threads (`pr#19942 `_, Shinobu Kinjo) +* common: Add noreturn attribute to silence uninitialized warning (`pr#19348 `_, Adam C. Emerson) * common: auth: add err reason for log info in load function (`pr#17256 `_, Luo Kexue) -* common: auth: keep /dev/urandom open for get_random_bytes (`issue#21401 `_, `pr#17972 `_, Casey Bodley) * common: bench test fall into dead loop when =0 (`pr#16382 `_, PC) * common: buffer: avoid changing bufferlist ABI by removing new _mempool field (`issue#21573 `_, `pr#18408 `_, Sage Weil) * common: by default, do not assert on leaks in the shared_cache code (`issue#21737 `_, `pr#18201 `_, Greg Farnum) * common: ceph: add the right bracket to watch-channel argument in the help message (`pr#19698 `_, Chang Liu) +* common: ceph.in: execv using the same python (`pr#17713 `_, Kefu Chai) +* common: ceph_release: s/rc/stable/ (`pr#22264 `_, Sage Weil) * common: change routines to public access (`pr#20003 `_, Willem Jan Withagen) * common: Check this->data.op_size before use (`pr#18816 `_, Amit Kumar) * common: cleanup address_helper (`pr#19643 `_, Shinobu Kinjo) -* common: cleanup: Replacing MIN,MAX with std::min,std::max (`pr#18124 `_, Amit Kumar) -* common: cls_acl/_crypto: Add modeline (`pr#19010 `_, Shinobu Kinjo) -* common: cmake,common: changes to port part of ceph to osx (`pr#17615 `_, Kefu Chai) +* common: cmake,common/RWLock: check for libpthread extensions (`pr#19202 `_, Kefu Chai) * common: common: add for_each_substr() for cheap string split (`pr#18798 `_, Casey Bodley) * common: common: add streaming interfaces for json/xml escaping (`pr#19806 `_, Casey Bodley) * common: common/admin_socket: validate command json before feeding it to hook (`pr#20437 `_, Kefu Chai) * common: common/blkdev: fix build in FreeBSD environment (`pr#19316 `_, Mykola Golub) * common: common/buffer: cleanups (`pr#18312 `_, Shinobu Kinjo) +* common: common/buffer: switch crc cache to single pair instead of map (`pr#18906 `_, Piotr Dałek) * common: common/config: add units to options (`issue#22747 `_, `pr#20419 `_, Kefu Chai) +* common: common/config: limit calls to normalize_key_name (`pr#20318 `_, Piotr Dałek) * common: common/config: make internal_safe_to_start_threads internal (`pr#18884 `_, Sage Weil) +* common: common/ConfUtils: check key before actually normalizing (`pr#20370 `_, Piotr Dałek) * common: common/dns_resolv.cc: Query for AAAA-record if ms_bind_ipv6 is True (`issue#23078 `_, `pr#20530 `_, Wido den Hollander) * common: common/dns_resolve: fix memory leak (`pr#19649 `_, Yao Zongyou) * common: common/event_socket.h: include to use errno (`pr#18351 `_, Kefu Chai) +* common: common/Formatter: fix string_view usage for {json,xml}_stream_escaper (`issue#23622 `_, `pr#21317 `_, Sage Weil) +* common: common/hobject: compare two objects' key directly (`pr#21062 `_, xie xingguo) +* common: common/hobject: preserve the order of hobject (`pr#21217 `_, Kefu Chai) +* common: common/ipaddr: Do not select link-local IPv6 addresses (`issue#21813 `_, `pr#20862 `_, Wido den Hollander) * common: common/lockdep: drop hash specialization (`pr#20574 `_, Kefu Chai) * common: common/LogClient: assign seq and queue atomically (`issue#18209 `_, `pr#16828 `_, Sage Weil) -* common: common,mon,osd,pybind: silence warning and remove executable mode bit (`pr#17512 `_, Kefu Chai) -* common: common/options.cc: cleanup readable literals for default sizes (`pr#18425 `_, Enming Zhang) +* common: common/log: Speed improvement for log (`pr#19100 `_, Adam Kupczyk, Kefu Chai) +* common: common/OpHistory: move insert/cleanup into separate thread (`pr#20540 `_, Piotr Dałek) +* common: common/options: drop unused options (`pr#20895 `_, Kefu Chai) * common: common/options: long description for log_stderr_prefix (`pr#19869 `_, Sage Weil) * common: common/options: pass by reference and use user-literals for size (`pr#18034 `_, Kefu Chai) * common: common/options: use user-defined literals for default values (`pr#17180 `_, Kefu Chai) -* common: common,os: initialize commit_data,cmount,iocb (`pr#17766 `_, Amit Kumar) * common: common/perf_counters: remove unused parameter (`pr#19805 `_, Kefu Chai) * common: common/pick_address.cc: Cleanup (`pr#19707 `_, Shinobu Kinjo) * common: common/pick_address: wrong prefix_len in pick_iface() (`pr#20128 `_, Gu Zhongyan) * common: common/str_list: s/boost::string_view/std::string_view (`pr#20475 `_, Kefu Chai) +* common: common/strtol: fix strict_strtoll() so it accepts hex starting with 0x (`pr#21521 `_, Kefu Chai) +* common: common/strtoll: remove superfluous const modifier (`pr#21560 `_, Jan Fajerski) * common: common/throttle: start using 64-bit values (`issue#22539 `_, `pr#19759 `_, Igor Fedotov) * common: common/types: make numbers a bit nicer when displaying space usage (`pr#17126 `_, xie xingguo) * common: common/util: do not print error if VERSION_ID is missing (`pr#17787 `_, Kefu Chai) -* common: compressor: conditionalize on HAVE_LZ4 (`pr#17059 `_, Kefu Chai) -* common: compressor: kill AsyncCompressor which is broken (`pr#18472 `_, Haomai Wang) * common: compressor: use generate_random_number() for type="random" (`pr#18272 `_, Casey Bodley) +* common: compressor/zstd: improvements (`pr#18879 `_, Sage Weil) * common: compute SimpleLRU's size with contents.size() instead of lru.size() (`issue#22613 `_, `pr#19813 `_, Xuehan Xu) +* common: config: expand tilde for ~/.ceph/$cluster.conf (`issue#23215 `_, `pr#20774 `_, Rishabh Dave) +* common: config: notify config observers on set_mon_vals() (`pr#21161 `_, Casey Bodley) * common: config: Remove _get_val (`pr#18222 `_, Adam C. Emerson) +* common/config: use with_val() for better performance (`pr#19056 `_, Adam C. Emerson) * common: consolidate spinlocks (`pr#15816 `_, Jesse Williamson) * common,core: common, osd: various cleanups (`pr#18149 `_, Kefu Chai) * common,core: common/pick_address: add {public,cluster}_network_interface option (`pr#18028 `_, Sage Weil) @@ -831,75 +913,83 @@ Other Notable Changes * common,core: fix broken use of streamstream::rdbuf() (`issue#22715 `_, `pr#19998 `_, Sage Weil) * common,core: include/ceph_features: deprecate a bunch of features (`pr#18546 `_, Sage Weil) * common,core: include,messages,rbd: Initialize counter,group_pool (`pr#17774 `_, Amit Kumar) +* common,core: options: Do not use linked lists of pointers! (`pr#17984 `_, Adam C. Emerson) * common,core: osdc/Objecter: take budgets across a LingerOp instead of on child Ops (`issue#22882 `_, `pr#20519 `_, Greg Farnum) -* common,core,performance: options: Do not use linked lists of pointers! (`pr#17984 `_, Adam C. Emerson) -* common,core,performance: osd/OpRequest: reduce overhead when disabling tracking (`pr#18470 `_, Haomai Wang) +* common,core: osd/OpRequest: reduce overhead when disabling tracking (`pr#18470 `_, Haomai Wang) * common,core: rados: Prefer templates to macros (`pr#19913 `_, Adam C. Emerson) +* common,core,rbd,rgw: common,osd,rgw: Fixes for issues found during SCA (`pr#21419 `_, Danny Al-Gaaf) * common,core,rbd,tests,tools: common,mds,mgr,mon,osd: store event only if it's added (`pr#16312 `_, Kefu Chai) * common,core: Revert "msg/async/AsyncConnection: unregister connection when racing happened" (`issue#22231 `_, `pr#19586 `_, Sage Weil) * common,core: Revert "osd/OSDMap: allow bidirectional swap of pg-upmap-items" (`issue#21410 `_, `pr#17760 `_, Sage Weil) -* common: create the ceph-volume and ceph-volume-systemd man pages (`issue#21030 `_, `pr#17152 `_, Alfredo Deza) +* common: Coverity and SCA fixes (`pr#17431 `_, Danny Al-Gaaf) +* common/crc/aarch64: Added cpu feature pmull and make aarch64 specific… (`pr#22184 `_, Adam Kupczyk) * common: crush/CrushWrapper: fix out of bounds access (`issue#20926 `_, `pr#16869 `_, Sage Weil) * common: crypto: remove cryptopp library (`pr#20015 `_, Casey Bodley) * common: denc cleanups and other fixes (`pr#19877 `_, Adam C. Emerson) -* common: denc: should check element's type not 'size_t' (`pr#19986 `_, Kefu Chai) * common: denc: support enum with underlying type (`pr#18701 `_, Kefu Chai) -* common: denc: use constexpr-if to replace some SFINAE impls (`pr#19662 `_, Kefu Chai) * common: Destroy attr of RWLock after initialized (`pr#17103 `_, Wen Zhang) +* common: dmclock: update mClockPriorityQueue with changes in subtree (`pr#20992 `_, Casey Bodley) +* common: dout: DoutPrefixProvider operates directly on stream (`pr#21608 `_, Casey Bodley) * common: drop namespace using directives for std (`pr#19159 `_, Shinobu Kinjo) -* common: drop unused variables "bluestore_csum_block" in opts (`pr#17394 `_, songweibin) +* common: drop unused variables "bluestore_csum\_\*_block" in opts (`pr#17394 `_, songweibin) * common: encoding: reset optional<> if it is uninitialized (`pr#17599 `_, Kefu Chai) +* common: Extends random.h: numeric types relaxed to compatible types (with (`pr#20670 `_, Jesse Williamson) * common: fix BoundedKeyCounter const_pointer_iterator (`issue#22139 `_, `pr#18953 `_, Casey Bodley) * common: fix daemon abnormal exit at parsing invalid arguments (`issue#21365 `_, `issue#21338 `_, `pr#17664 `_, Yan Jun) * common: fix potential memory leak in HTMLFormatter (`pr#20699 `_, Yao Zongyou) * common: fix typo deamon in comments (`pr#17687 `_, yonghengdexin735) * common: fix typo in options.cc (`pr#20549 `_, songweibin) +* common: FreeBSD wants the correct struct selection for ipv6 (`issue#21813 `_, `pr#21143 `_, Willem Jan Withagen) +* common: global: output usage on -h, --help, or no args before contacting mons (`pr#20812 `_, Sage Weil) +* common: hint the main branch of dout() accordingly to default verbosity (`pr#21259 `_, Radoslaw Zarzynski) * common: implement random number generator (following N3551) (`issue#18873 `_, `pr#15341 `_, Jesse Williamson) -* common: include/buffer.h: fix typo in comment (`pr#17489 `_, mychoxin) +* common: Improving message sent to user when getting signals (`issue#23320 `_, `pr#21000 `_, Erwan Velu) * common: include/encoding: fix compat version error message (`pr#19660 `_, Xinying Song) * common: include/interval_set: parameterize by map type and kill btree_interval_set.h (`pr#18611 `_, Sage Weil) * common: include/rados: fix typo in librados.h (`pr#17988 `_, wumingqiao) * common: include: Remove unused header, ciso646 (`pr#18320 `_, Shinobu Kinjo) +* common: include/types: format decimal numbers with decimal factor (`issue#22095 `_, `pr#19117 `_, Jan Fajerski) * common: include: xlist: Fix Clang error for missing string (`pr#19367 `_, Willem Jan Withagen) -* common: install-deps.sh: fix an error condition expression (`pr#20819 `_, Yao Guotao) +* common: interval_set: kill subset_of() (`pr#21108 `_, xie xingguo) +* common: interval_set: optimize intersect_of insert operations (`issue#21229 `_, `pr#17265 `_, Zac Medico) * common: introduce md_config_cacher_t (`pr#20320 `_, Radoslaw Zarzynski) -* common: java/native: fix milliseconds to mtime/atime conversion (`pr#17460 `_, dengquan) -* common: java/native: s/jni: lstat/jni: stat in native_ceph_stat (`pr#20142 `_, Shangzhong Zhu) * common: kick off mimic (`pr#16993 `_, Sage Weil) -* common: kv/leveldb: fix deadlock when close db (`pr#16643 `_, Zengran) -* common: kv: unify create and open() methods (`pr#18177 `_, Kefu Chai) * common: lockdep fixes (`issue#20988 `_, `pr#17738 `_, Jeff Layton) * common: log: clear thread-local stream's ios flags on reuse (`pr#20174 `_, Casey Bodley) * common: logically dead code inside shunique_lock.h (`pr#17341 `_, Amit Kumar) * common: make ceph_clock_now() inlineable (`pr#20443 `_, Radoslaw Zarzynski) +* common: Make code to invoke assert() smaller (`pr#20445 `_, Adam Kupczyk) * common: make some message informative, instead of error (`pr#16594 `_, Willem Jan Withagen) * common: mark events of TrackedOp outside its constructor (`issue#22608 `_, `pr#19828 `_, Xuehan Xu) * common: mgr/dashboard_v2: Fix test_cluster_configuration test (`issue#23265 `_, `pr#20782 `_, Sebastian Wagner) +* common: mimic: include/types: space between number and units (`pr#22107 `_, Sage Weil) * common,mon: crush,mon: fix weight-set vs crush device classes (`issue#20939 `_, `pr#16883 `_, Sage Weil) +* common,mon,osd,pybind: silence warning and remove executable mode bit (`pr#17512 `_, Kefu Chai) * common: msg/async/AsyncConnection: less noisy debug (`pr#20600 `_, Sage Weil) * common: msg/async: execute on core specified by core_id not its index (`pr#20659 `_, Kefu Chai) * common: msg/msg_types: fix the entity_addr_t's decoder (`pr#17699 `_, Kefu Chai) * common: msg/simple: s/ceph::size/std::size/ (`pr#19896 `_, Kefu Chai) +* common/options.cc: cleanup readable literals for default sizes (`pr#18425 `_, Enming Zhang) +* common/options.cc: Set Filestore rocksdb compaction readahead option (`issue#21505 `_, `pr#17900 `_, Mark Nelson) +* common: OpTracker doesn't visit TrackedOp when nref == 0 (`issue#24037 `_, `pr#22160 `_, Radoslaw Zarzynski) +* common: osdc/Objecter: fix warning (`pr#21757 `_, Sage Weil) * common: osdc/Objecter: record correctly value for l_osdc_op_send_bytes (`issue#21982 `_, `pr#18810 `_, Jianpeng Ma) * common: osd/PrimaryLogPG: send requests to primary on cache miss (`issue#20919 `_, `pr#16884 `_, Sage Weil) -* common,performance: cmake,common/RWLock: check for libpthread extensions (`pr#19202 `_, Kefu Chai) -* common,performance: common/buffer: switch crc cache to single pair instead of map (`pr#18906 `_, Piotr Dałek) -* common,performance: common/config: limit calls to normalize_key_name (`pr#20318 `_, Piotr Dałek) -* common,performance: common/ConfUtils: check key before actually normalizing (`pr#20370 `_, Piotr Dałek) -* common,performance: common/log: Speed improvement for log (`pr#19100 `_, Adam Kupczyk, Kefu Chai) -* common,performance: common/OpHistory: move insert/cleanup into separate thread (`pr#20540 `_, Piotr Dałek) -* common,performance: compressor/zstd: improvements (`pr#18879 `_, Sage Weil) -* common,performance: interval_set: optimize intersect_of insert operations (`issue#21229 `_, `pr#17265 `_, Zac Medico) -* common,performance: osd_types: define max in eversion_t::max() to static (`pr#17453 `_, yang.wang) -* common,performance,rdma: src/msg/async/rdma: decrease cpu usage by rdtsc instruction (`pr#16965 `_, Jin Cai) +* common: osd_types: define max in eversion_t::max() to static (`pr#17453 `_, yang.wang) +* common,os: initialize commit_data,cmount,iocb (`pr#17766 `_, Amit Kumar) +* common: posix_fallocate on ZFS returns EINVAL (`pr#20398 `_, Willem Jan Withagen) * common: rados: clean up rados_getxattrs() and rados_striper_getxattrs() (`pr#20259 `_, Gu Zhongyan) * common: RAII-styled mechanism for updating PerfCounters (`pr#19149 `_, Radoslaw Zarzynski) +* common: random: revert change from boost::optional to std::optional (`issue#23778 `_, `pr#21567 `_, Casey Bodley) * common: Remove ceph_clock_gettime, extern keyword (`pr#19353 `_, Shinobu Kinjo) +* common: retry_sys_call no need take address of a function pointer (`pr#21281 `_, Leo Zhang) * common: Revert "common/config: return const reference instead of a copy" (`pr#18934 `_, Kefu Chai) * common: Revert "core: hint the dout()'s message crafting as a cold code." (`issue#23169 `_, `pr#20636 `_, Kefu Chai) * common,rgw: rgw,common,rbd: s/boost::regex/std::regex/ (`pr#19393 `_, Kefu Chai) * common,rgw: rgw,common: remove already included header files (`pr#19390 `_, Yao Zongyou) * common: silence jenkins's buiding warning in obj_bencher.cc (`pr#17272 `_, Luo Kexue) +* common: src/common: update some ms\_\* options to be more consistent (`pr#20652 `_, shangfufei) +* common: src/msg/async/rdma: decrease cpu usage by rdtsc instruction (`pr#16965 `_, Jin Cai) * common: Static Pointer (`pr#19079 `_, Adam C. Emerson) * common: strict_strtol INT_MAX and INT_MIN is valid (`pr#18574 `_, Shasha Lu) * common: s/unique_lock/lock_guard/, if manual lock/unlock are not necessary (`pr#19770 `_, Shinobu Kinjo) @@ -910,6 +1000,10 @@ Other Notable Changes * common: tools/crushtool: skip device id if no name exists (`issue#22117 `_, `pr#18901 `_, Jan Fajerski) * common: use mono clock for HeartbeatMap (`pr#17827 `_, Xinze Chi, Kefu Chai) * common: use move instead of copy in build_options() (`pr#18003 `_, Casey Bodley) +* common: utime： fix __32u sec time overflow (`pr#21113 `_, kungf) +* compressor: add zstd back (`pr#21106 `_, Kefu Chai) +* compressor: conditionalize on HAVE_LZ4 (`pr#17059 `_, Kefu Chai) +* compressor: kill AsyncCompressor which is broken (`pr#18472 `_, Haomai Wang) * core: blkin: Fix unconditional tracing in OSD (`pr#19156 `_, Yingxin) * core: ceph-debug-docker.sh: add ceph-osd-dbg package (`pr#17947 `_, Patrick Donnelly) * core: ceph.in: Add blocking mode for scrub and deep-scrub (`pr#19793 `_, Brad Hubbard) @@ -917,19 +1011,24 @@ Other Notable Changes * core: ceph.in: print all matched commands if arg missing (`issue#22344 `_, `pr#19547 `_, Kefu Chai) * core: ceph.in: use a different variable for holding thrown exception (`pr#20663 `_, Kefu Chai) * core: ceph-kvstore-tool: copy to different store type and cleanup properly (`pr#18029 `_, Josh Durgin) +* core: ceph-mgr: exit after usage (`issue#23482 `_, `pr#21401 `_, Sage Weil) * core: ceph_osd.cc: Drop legacy or redundant code (`pr#18718 `_, Shinobu Kinjo) * core: ceph-osd: some flags are not documented in the help output (`issue#20057 `_, `pr#15565 `_, Yanhu Cao) +* core: ceph: print output of "status" as string not as bytes (`pr#21297 `_, Kefu Chai) * core: ceph-rest-api: when port=0 use the DEFAULT_PORT instead (`pr#17443 `_, You Ji) * core: ceph_test_objectstore: disable filestore_fiemap for tests (`issue#21880 `_, `pr#18452 `_, Sage Weil) * core: ceph_test_objectstore: do not change model for 0-length zero (`issue#21712 `_, `pr#18519 `_, Sage Weil) +* core: ceph_test_rados_api_aio: fix race with full pool and osdmap (`issue#23916 `_, `issue#23917 `_, `pr#21709 `_, Sage Weil) * core: ceph_test_rados_api_tier: add ListSnap test (`pr#17706 `_, Xuehan Xu) * core: client,osd,test: Initialize fuse_req_key,snap,who,seq (`pr#17772 `_, Amit Kumar) * core: common/admin_socket: various cleanups (`pr#20274 `_, Adam C. Emerson) * core: common/config: cleanup remove some unused macros (`pr#19599 `_, Yao Zongyou) * core: common,mds,osd: Explicitly delete copy ctor if noncopyable (`pr#19465 `_, Shinobu Kinjo) +* core: common/options: enable multiple rocksdb compaction threads for filestore (`pr#18232 `_, Josh Durgin) +* core: common, osd: duplicated "start" event in OpTracker, improve OpTracker::dump_ops (`pr#21119 `_, Chang Liu) * core: compressor: Add Brotli Compressor (`pr#19549 `_, BI SHUN KE) * core: config: lower default omap entries recovered at once (`issue#21897 `_, `pr#19910 `_, Josh Durgin) -* core: Coverity and SCA fixes (`pr#17431 `_, Danny Al-Gaaf) +* core: crush/CrushWrapper: fix potential invalid use of iterator (`pr#21325 `_, xie xingguo) * core: dmclock: Delivery of the dmclock delta, rho and phase parameter + Enabling the client service tracker (`pr#16369 `_, Byungsu Park, Taewoong Kim) * core: erasure-code: refactor the interfaces to hide internals from public (`pr#18683 `_, Kefu Chai) * core: erasure-code: use jerasure_free_schedule to properly free a schedule (`pr#19650 `_, Yao Zongyou) @@ -938,13 +1037,16 @@ Other Notable Changes * core: filestore: include (`pr#20415 `_, wumingqiao) * core: Fix a dead lock when doing rdma performance test by fio (`pr#17016 `_, Wang Chuanhong) * core: Fix asserts caused by DNE pgs left behind after lots of OSD restarts (`issue#21833 `_, `pr#20571 `_, David Zafman) +* core: include: kill MIN and MAX macros (`pr#20886 `_, Sage Weil) +* core: interval_set: optimize intersection_of (`pr#17088 `_, Zac Medico) +* core: kv/KeyValueDB: add column family (`pr#18049 `_, Jianjian Huo, Adam C. Emerson, Sage Weil) * core: kv/RocksDB: get index and filter blocks memory usage (`pr#19934 `_, Zhi Zhang) +* core: kv/RocksDBStore: fix rocksdb error when block cache is disabled (`issue#23816 `_, `pr#21583 `_, Yang Honggang) +* core: librados: add OPERATION_ORDERSNAP flag and yet another aio_operate method (`pr#20343 `_, Mykola Golub) * core: librados.h: add LIBRADOS_SUPPORTS_APP_METADATA (`pr#16542 `_, Matt Benjamin) * core: libradosstriper: fix the function declaration of rados_striper_trunc (`pr#20301 `_, yuelongguang) * core: libradosstriper: silence warning from -Wreorder (`pr#16890 `_, songweibin) -* core: librbd,librados: do not include stdbool.h in C++ headers (`pr#19945 `_, Kefu Chai) * core: make the main dout() paths faster and more maintanable (`pr#20290 `_, Radoslaw Zarzynski) -* core: mempool: fix lack of pool names in mempool:dump output for JSON format (`pr#18329 `_, Igor Fedotov) * core: messages: Initialization of variable beat (`pr#17641 `_, Amit Kumar) * core: messages: Initialize member variables (`pr#16846 `_, amitkuma) * core: messages: initialize variable tid in MMDSFindIno (`pr#16793 `_, amitkuma) @@ -955,14 +1057,25 @@ Other Notable Changes * core: messages/MOSDOp: a fixes of encode_payload (`pr#16836 `_, Ying He) * core: messages: Silence uninitialized member warnings (`pr#17596 `_, Amit Kumar) * core: mgr/DaemonServer.cc: add 'is_valid=false' when decode caps error (`issue#20990 `_, `pr#16978 `_, Yanhu Cao) +* core,mgr: mgr/balancer: improve error message (`issue#22814 `_, `pr#21427 `_, Sage Weil) * core,mgr: osd,mgrclient: pass daemon_status by rvalue ref and other cleanups (`pr#18509 `_, Kefu Chai) * core,mgr: osd,mgr: report slow requests and pending creating pgs to mgr (`pr#18614 `_, Kefu Chai) +* core: mimic: crush: update choose_args on bucket removal (`issue#24167 `_, `pr#22120 `_, Sage Weil) +* core: mimic: osdc: guard op->on_notify_finish with lock (`issue#23966 `_, `pr#21834 `_, Kefu Chai) +* core: mimic: osd: clean up smart probe (`issue#23899 `_, `issue#24104 `_, `pr#21959 `_, Sage Weil, Gu Zhongyan) +* core: mimic: osd: Don't evict even when preemption has restarted with smaller chunk (`pr#22041 `_, David Zafman) +* core: mimic: osd/PrimaryLogPG: fix try_flush_mark_clean write contention case (`issue#24200 `_, `issue#24174 `_, `pr#22113 `_, Sage Weil) +* core: mon/ConfigKeyService: dump: print placeholder value for binary blobs (`issue#23622 `_, `pr#21329 `_, Sage Weil) * core,mon: crush, mon: bump up map version only if we truly created a weight-set (`pr#20178 `_, xie xingguo) +* core: mon/LogMonitor: separate out summary by channel (`pr#21395 `_, Sage Weil) +* core,mon: mon/AuthMonitor: create bootstrap keys on create_initial() (`pr#21236 `_, Joao Eduardo Luis) +* core,mon: mon/LogMonitor: do not crash on log sub w/ no messages (`pr#21469 `_, Sage Weil) * core,mon: mon,osd,crush: misc cleanup (`pr#20687 `_, songweibin) * core,mon: mon/OSDMonitor: Comment out unused function (`pr#20275 `_, Brad Hubbard) * core,mon: mon/OSDMonitor: don't create pgs if pool was deleted (`issue#21309 `_, `pr#17600 `_, Joao Eduardo Luis) * core,mon: mon/OSDMonitor: implement cluster pg limit (`pr#17427 `_, Sage Weil) * core,mon: mon/OSDMonitor: list osd tree in named bucket (`pr#19564 `_, kungf) +* core: mon, osd: add create-time for pool (`pr#21690 `_, xie xingguo) * core: mon, osd: fix potential collided \*Up Set\* after PG remapping (`issue#23118 `_, `pr#20653 `_, xie xingguo) * core,mon: osd,mon: add max-pg-per-osd limit (`pr#18358 `_, Kefu Chai) * core: mon/OSDMonitor: filter out pgs that shouldn't exist from force-create-pg (`pr#20267 `_, Sage Weil) @@ -970,11 +1083,18 @@ Other Notable Changes * core: mon/OSDMonitor: Fix OSDMonitor error message outputs (`issue#22351 `_, `pr#20022 `_, Brad Hubbard) * core: mon/OSDMonitor: make 'osd crush class rename' idempotent (`pr#17330 `_, xie xingguo) * core: mon/OSDMonitor: rename outer name declaration to avoid shadowing (`pr#20032 `_, Sage Weil) +* core: mon/OSDMonitor: require --yes-i-really-mean-it for force-create-pg (`pr#21619 `_, Sage Weil) +* core: mon,osd,osdc: refactor snap trimming (phase 1) (`pr#18276 `_, Sage Weil) * core: mon, osd: per pool space-full flag support (`pr#17371 `_, xie xingguo) * core: mon, osd: turn down non-error scrub message severity (`issue#20947 `_, `pr#16916 `_, John Spray) +* core: mon/PGMap: fix PGMapDigest decode (`pr#22099 `_, Sage Weil) * core: mon/PGMap: Fix %USED calculation bug (`issue#22247 `_, `pr#19165 `_, Xiaoxi Chen) +* core: mon/PGMap: remove or narrow columns 'pg ls' output (`pr#20945 `_, Sage Weil) * core: mon/PGMap: 'unclean' does not imply damaged (`pr#18493 `_, Sage Weil) * core: MOSDPGRecoveryDelete[Reply]: bump header version (`pr#17585 `_, Josh Durgin) +* core: msg/asyc/rmda: fix the bug of assert when Infiniband::recv_msg receives disconnect message (`pr#17688 `_, Jin Cai) +* core: msg/async/AsyncConnection: combine multi alloc into one (`pr#18833 `_, Haomai Wang) +* core: msg/async/AsyncConnection: Fix FPE in process_connection (`issue#23618 `_, `pr#21314 `_, Brad Hubbard) * core: msg/async/AsyncConnection: state will be NONE if replacing by another one (`issue#21883 `_, `pr#18467 `_, Haomai Wang) * core: msg/async/AsyncConnection: unregister connection when racing happened (`pr#19013 `_, Haomai Wang) * core: msg/async: batch handle numevents (`pr#18321 `_, Jianpeng Ma) @@ -982,46 +1102,66 @@ Other Notable Changes * core: msg/async: don't stuck into resetsession/retrysession loop (`pr#17276 `_, Haomai Wang) * core: msg/async: fix bug of data type conversion when uint64_t -> int -> uint64_t (`pr#18210 `_, shangfufei) * core: msg/async: print error log if add_event fail (`pr#17102 `_, mychoxin) +* core: msg/async/rdma: fix multi cephcontext confllicting (`pr#16893 `_, Haomai Wang) * core: msg/async/rdma: fix the bug that rdma polling thread uses the same thread name with msg worker (`pr#16936 `_, Jin Cai) * core: msg/async/rdma: improves RX buffer management (`pr#16693 `_, Alex Mikheev) * core: msg/async/rdma: uninitialized variable fix (`pr#18091 `_, Vasily Philipov) * core: msg/DispatchQueue: clear queue after wait() (`issue#18351 `_, `pr#20374 `_, Sage Weil) -* core: Optimization for connection establishment (`pr#16006 `_, shangfufei) -* core: os/bluestore/bluestore_tool: Move redundant code into one method (`pr#19160 `_, Shinobu Kinjo) +* core: msgr/simple: set Pipe::out_seq to in_seq of the connecting side (`issue#23807 `_, `pr#21585 `_, Xuehan Xu) * core: os/bluestore: debug bluestore cache shutdown (`issue#21259 `_, `pr#17844 `_, Sage Weil) * core: os/bluestore: disable on_applied sync_complete (`issue#22668 `_, `pr#20169 `_, Sage Weil) -* core: os/bluestore: implement BlueRocksEnv::AreFilesSame() (`issue#21842 `_, `pr#18392 `_, Kefu Chai) * core: os/bluestore: make bdev label parsing error more meaningful and less noisy (`pr#20090 `_, Sage Weil) * core: os/bluestore: make BlueStore opened by start_kv_only umountable (`issue#21624 `_, `pr#18082 `_, Chang Liu) +* core: os/bluestore: use db->rm_range_keys to delete range of keys (`pr#18279 `_, Xiaoyan Li) * core: OSD/admin_socket: add get_mapped_pools command (`pr#19112 `_, Xiaoxi Chen) * core: osdc, class_api: kill implicit string conversions (`pr#16648 `_, Piotr Dałek) +* core: osdc: dec num_in_flight for pool_dne case (`pr#21110 `_, Jianpeng Ma) * core: osdc: Do not use lock_guard as unique_lock (`pr#19756 `_, Shinobu Kinjo) +* core: osdc: invoke notify finish context on linger commit failure (`issue#23966 `_, `pr#21786 `_, Jason Dillaman) +* core: osdc/Objecter: add ignore overlay flag if got redirect reply (`pr#21275 `_, Ting Yi Lin) +* core: osdc/Objecter: delay initialization of hobject_t in _send_op (`issue#21845 `_, `pr#18427 `_, Jason Dillaman) +* core: osdc/Objecter: fix recursive locking in _finish_command (`issue#23940 `_, `pr#21742 `_, Sage Weil) * core: osdc/Objecter: misc cleanups (`pr#18476 `_, Jianpeng Ma) +* core: osdc/Objecter: prevent double-invocation of linger op callback (`issue#23872 `_, `pr#21649 `_, Jason Dillaman) * core: osdc/Objecter: skip sparse-read result decode if bufferlist is empty (`issue#21844 `_, `pr#18400 `_, Jason Dillaman) -* core: osdc: Remove a bit too redundant public label (`pr#19466 `_, Shinobu Kinjo) +* core: osd,compressor: Expose compression algorithms via MOSDBoot (`issue#22420 `_, `pr#20558 `_, Jesse Williamson) +* core: osdc: remove unused function (`pr#21081 `_, Jianpeng Ma) +* core: osd,dmclock: use pointer to ClientInfo instead of a copy of it (`pr#18387 `_, Kefu Chai) * core: osd: do not forget pg_stat acks which failed to send (`pr#16702 `_, huangjun) * core: OSD: drop unsed parameter passed to check_osdmap_features (`pr#18466 `_, Leo Zhang) * core: osd/ECBackend: inject sleep during deep scrub (`pr#20531 `_, xie xingguo) +* core: osd/ECBackend: only check required shards when finishing recovery reads (`issue#23195 `_, `pr#21273 `_, Josh Durgin) +* core: osd/ECBackend: update misleading comment about EIO handling (`pr#21686 `_, Josh Durgin) +* core: osd/ECBackend: wait for apply for luminous peers (`pr#21604 `_, Sage Weil) * core: osd/ECMsgTypes: fix ECSubRead compat decode (`pr#20948 `_, Sage Weil) * core: osd, librados: add a rados op (TIER_PROMOTE) (`pr#19362 `_, Myoungwon Oh) * core: osd,librados: add manifest, operations for chunked object (`pr#15482 `_, Myoungwon Oh) * core: osd,messages: Initialize read_length,options,send_reply (`pr#17799 `_, Amit Kumar) +* core: osd/OSD: batch-list objects to reduce memory consumption (`pr#20767 `_, xie xingguo) * core: osd/OSD.cc: add 'isvalid=false' when failed to parse caps (`pr#16888 `_, Yanhu Cao) * core: osd/OSD.cc: use option 'osd_scrub_cost' instead (`pr#18479 `_, Liao Weizhong) * core: osd/OSDMap: add osdmap epoch info when printing info summary (`pr#20184 `_, shun-s) +* core: osd/OSDMap: fix HAVE_FEATURE logic in encode() (`pr#20922 `_, Ilya Dryomov) * core: osd/OSDMap: ignore PGs from pools of failure-domain OSD (`pr#20703 `_, xie xingguo) * core: osd/OSDMap: misleading message in print_oneline_summary() (`issue#22350 `_, `pr#20313 `_, Gu Zhongyan) +* core: osd/OSDMap: more pg upmap fixes (`issue#23878 `_, `pr#21670 `_, xiexingguo) * core: osd/OSDMap: remove the unnecessary checks for null (`pr#18636 `_, Kefu Chai) * core: osd/OSDMap: skip out/crush-out osds (`pr#20655 `_, xie xingguo) * core: osd/OSDMap: upmap should respect the osd reweights (`issue#21538 `_, `pr#17944 `_, Theofilos Mouratidis) +* core: osd/osd_type: get_clone_bytes - inline size() for overlapping size (`pr#17823 `_, xie xingguo) * core: osd/osd_types.cc: copy extents map too while making clone (`pr#18396 `_, xie xingguo) +* core: osd/osd_types: fix ideal lower bound object-id of pg (`pr#21235 `_, xie xingguo) * core: osd/osd_types: fix object_stat_sum_t decode (`pr#18551 `_, Sage Weil) +* core: osd/osd_types: fix pg_pool_t encoding for hammer (`pr#21282 `_, Sage Weil) * core: osd/osd_types: kill preferred field in pg_t (`pr#20567 `_, Sage Weil) * core: osd/osd_types: object_info_t: remove unused function (`pr#17905 `_, Kefu Chai) * core: osd/osd_types: pg_pool_t: remove crash_replay_interval member (`pr#18379 `_, Sage Weil) +* core: osd/osd_types: remove backlog type for pg_log_entry_t (`pr#20887 `_, Sage Weil) * core: osd/OSD: Using Wait rather than WaitInterval to wait queue.is_empty() (`pr#17659 `_, Jianpeng Ma) -* core: osd,os,io: Initializing C_ProxyChunkRead members,queue,request (`pr#19336 `_, amitkuma) +* core: osd/PG: allow scrub preemption (`pr#18971 `_, Sage Weil) * core: osd/PGBackend: delete reply if fails to complete delete request (`issue#20913 `_, `pr#17183 `_, Kefu Chai) +* core: osd/PGBackend: drop input "snapid_t" from objects_list_range() (`pr#21151 `_, xie xingguo) +* core: osd/PGBackend: fix large_omap_objects checking (`pr#21150 `_, xie xingguo) * core: osd/PGBackend: release a msg using msg->put() not delete (`issue#20913 `_, `pr#17246 `_, Kefu Chai) * core: osd/PG: const cleanup for recoverable/readable predicates (`pr#18982 `_, Neha Ojha) * core: osd/PG: decay scrub_chunk_max too if scrub is preempted (`pr#20552 `_, xie xingguo) @@ -1029,36 +1169,55 @@ Other Notable Changes * core: osd/PG: drop unused variable "oldest_update" in PG.h (`pr#17142 `_, songweibin) * core: osd/PG: extend pg state bits to fix pg ls commands error (`issue#21609 `_, `pr#18058 `_, Yan Jun) * core: osd/PG: fix calc of misplaced objects (`pr#18528 `_, Kefu Chai) -* core: osd/PG: fix clang build vs private state events (`pr#18217 `_, Sage Weil) +* core: osd/PG: fix DeferRecovery vs AllReplicasRecovered race (`issue#23860 `_, `pr#21706 `_, Sage Weil) * core: osd/PG: fix objects degraded higher than 100% (`issue#21803 `_, `issue#21898 `_, `pr#18297 `_, Sage Weil, David Zafman) +* core: osd/PG: fix out of order priority for PG deletion (`pr#21613 `_, xie xingguo) * core: osd/PG: fix recovery op leak (`pr#18524 `_, Sage Weil) +* core: osd/PG: fix uninit read in Incomplete::react(AdvMap&) (`issue#23980 `_, `pr#21798 `_, Sage Weil) * core: osd/PG: force rebuild of missing set on jewel upgrade (`issue#20958 `_, `pr#16950 `_, Sage Weil) -* core: osd/PG: handle flushed event directly (`pr#19441 `_, wumingqiao) * core: osd/PG: include primary in PG operator<< for ec pools (`pr#19453 `_, Sage Weil) +* core: osd/PGLog: assert out on performing overflowed log trimming (`pr#21580 `_, xie xingguo) * core: osd/PGLog: cleanup unused function revise_have (`pr#19329 `_, Enming Zhang) +* core: osd/PGLog: fix sanity check against \*\*complete-to\*\* iter (`pr#21612 `_, songweibin) +* core: osd/PGLog: get rid of ineffective container operations (`pr#19161 `_, xie xingguo) +* core: osd/PGLog: write only changed dup entries (`issue#21026 `_, `pr#17245 `_, Josh Durgin) * core: osd, pg, mgr: make snap trim queue problems visible (`issue#22448 `_, `pr#19520 `_, Piotr Dałek) * core: osd/PG: misc cleanups (`pr#18340 `_, Yan Jun) * core: osd/PG: miscellaneous choose acting changes and cleanups (`pr#18481 `_, xie xingguo) * core: osd/PG: pass scrub priority to replica (`pr#20317 `_, Sage Weil) +* core: osd/PG: perfer async_recovery_targets in reverse order of cost (`pr#21578 `_, xie xingguo) +* core: osd/PG: perfer EC async_recovery_targets in reverse order of cost (`pr#21588 `_, xie xingguo) +* core: osd/PG: PGPool::update: avoid expensive union_of (`pr#17239 `_, Zac Medico) +* core: osd/PGPool::update: optimize with subset_of (`pr#17820 `_, Zac Medico) +* core: osd/PG: reduce some overhead on operating MissingLoc (`pr#18186 `_, xie xingguo) * core: osd/PG: remote recovery preemption, and new feature bit to condition it on (`pr#18553 `_, Sage Weil) * core: osd/PG: remove unused parameter in calc_ec_acting (`pr#17304 `_, yang.wang) * core: osd/PG: restart recovery if NotRecovering and unfound found (`issue#22145 `_, `pr#18974 `_, Sage Weil) +* core: osd/PG: revert approx size (`issue#22654 `_, `pr#18755 `_, Adam Kupczyk) * core: osd/PG: re-write of _update_calc_stats and improve pg degraded state (`issue#20059 `_, `pr#19850 `_, David Zafman) * core: osd/PG: some cleanups && add should_gather filter for loop logging (`pr#19546 `_, Enming Zhang) * core: osd/PG: two cleanups (`pr#17171 `_, xie xingguo) * core: osd/PG: use osd_backfill_retry_interval for schedule_backfill_retry() (`pr#18686 `_, xie xingguo) * core: osd/PrimaryLogPG: add condition "is_chunky_scrub_active" to check object in chunky_scrub (`pr#18506 `_, Jianpeng Ma) +* core: osd/PrimaryLogPG: arrange recovery order by number of missing objects (`pr#18292 `_, xie xingguo) +* core: osd/PrimaryLogPG: avoid infinite loop when flush collides with write lock (`pr#21653 `_, Sage Weil) +* core: osd/PrimaryLogPG: calc clone_overlap size in a more efficient and concise way (`pr#17928 `_, xie xingguo) * core: osd/PrimaryLogPG: cleanup do_sub_op && do_sub_op_reply and define soid in prepare_transaction more appropriate (`pr#19495 `_, Enming Zhang) +* core: osd/PrimaryLogPG: clear data digest on WRITEFULL if skip_data_digest (`pr#21676 `_, Sage Weil) * core: osd/PrimaryLogPG: clear pin_stats_invalid bit properly on scrub-repair completion (`pr#18052 `_, xie xingguo) -* core: osd/PrimaryLogPG: derr when object size becomes over osd_max_object_size (`pr#19049 `_, Shinobu Kinjo) +* core: osd/PrimaryLogPG: defer evict if head \*or\* object intersect scrub interval (`issue#23646 `_, `pr#21628 `_, Sage Weil) * core: osd/PrimaryLogPG: do not pull-up snapc to snapset (`pr#18713 `_, Sage Weil) +* core: osd/PrimaryLogPG: do not set data digest for bluestore (`pr#17515 `_, xie xingguo) * core: osd/PrimaryLogPG: do not set data/omap digest blindly (`pr#18061 `_, xie xingguo) * core: osd/PrimaryLogPG: do not use approx_size() for log trimming (`pr#18338 `_, xie xingguo) * core: osd/PrimaryLogPG: do_osd_ops - propagate EAGAIN/EINPROGRESS on failok (`pr#17222 `_, xie xingguo) * core: osd/PrimaryLogPG: drop unused parameters (`pr#18581 `_, Liao Weizhong) * core: osd/PrimaryLogPG: fix dup stat for async read (`pr#18693 `_, Xinze Chi) +* core: osd/PrimaryLogPG: Fix log messages (`pr#21639 `_, Gu Zhongyan) * core: osd/PrimaryLogPG: fix sparse read won't trigger repair correctly (`pr#17221 `_, xie xingguo) +* core: osd/PrimaryLogPG: fix the oi size mismatch with real object size (`issue#23701 `_, `pr#21408 `_, Peng Xie) * core: osd/PrimaryLogPG: kick off recovery on backoffing a degraded object (`pr#17987 `_, xie xingguo) +* core: osd/PrimaryLogPG: kill add_interval_usage (`pr#17807 `_, xie xingguo) * core: osd/PrimaryLogPG: maybe_handle_manifest_detail - sanity check obc existence (`pr#17298 `_, xie xingguo) * core: osd/PrimaryLogPG: misc cleanups (`pr#17830 `_, Yan Jun) * core: osd/PrimaryLogPG: more oi.extents fixes (`pr#18616 `_, xie xingguo) @@ -1069,57 +1228,38 @@ Other Notable Changes * core: osd/ReplicatedBackend: clean up code (`pr#20127 `_, Jianpeng Ma) * core: osd/ReplicatedBackend: 'osd_deep_scrub_keys' doesn't work (`pr#20221 `_, fang yuxiang) * core: osd/ReplicatedPG: add omap write bytes to delta_stats (`pr#18471 `_, Haomai Wang) -* core: osd,rgw,librbd: SCA fixes (`pr#18495 `_, Danny Al-Gaaf) * core: osd_types.cc: reorder fields in serialized pg_stat_t (`pr#19965 `_, Piotr Dałek) +* core: os/filestore: disable rocksdb compression (`pr#18707 `_, Sage Weil) * core: os/filestore/FileStore: Initialized by nullptr, NULL or 0 instead (`pr#18980 `_, Shinobu Kinjo) * core: os/filestore: fix device/partition metadata detection (`issue#20944 `_, `pr#16913 `_, Sage Weil) * core: os/filestore: fix do_copy_range replay bug (`issue#23298 `_, `pr#20832 `_, Sage Weil) * core: os/Filestore: fix wbthrottle assert (`pr#14213 `_, Xiaoxi Chen) -* core: os/filestore/HashIndex.h: fixed a typo in comment (`pr#17685 `_, yaoguotao) * core: os/filestore: print out the error if do_read_entry() fails (`pr#18346 `_, Kefu Chai) * core: os: FileStore, Using stl min | max, MIN | MAX macros instead (`pr#19832 `_, Shinobu Kinjo) * core: os: fix 0-length zero semantics, add tests (`issue#21712 `_, `pr#18159 `_, Sage Weil) * core: os/FuseStore: fix incorrect used space statistics for fuse's statfs interface (`pr#19033 `_, Zhi Zhang) -* core: os: Initializing uninitialized members aio_info (`pr#17066 `_, amitkuma) * core: os/kstore: Drop unused function declaration (`pr#18077 `_, Jos Collin) +* core: os/kstore: fix statfs problem and add vstart.sh support (`issue#23590 `_, `pr#21287 `_, Yang Honggang) +* core: os/memstore: Fix wrong use of lock_guard (`pr#20914 `_, Shen-Ta Hsieh) * core: os/ObjectStore: fix get_data_alignment return -1 causing problem in msg header (`pr#18475 `_, Haomai Wang) * core: os/ObjectStore.h: fix mistake in comment TRANSACTION ISOLATION (`pr#16840 `_, mychoxin) * core: os,osd: initial work to drop onreadable/onapplied callbacks (`issue#23029 `_, `pr#20177 `_, Sage Weil) -* core: os: Removing dead code from LFNIndex.cc (`pr#17297 `_, Amit Kumar) * core: os: unify Sequencer and CollectionHandle (`pr#20173 `_, Sage Weil) -* core,performance: common/options: enable multiple rocksdb compaction threads for filestore (`pr#18232 `_, Josh Durgin) -* core,performance: interval_set: optimize intersection_of (`pr#17088 `_, Zac Medico) -* core,performance: kv/KeyValueDB: add column family (`pr#18049 `_, Jianjian Huo, Adam C. Emerson, Sage Weil) -* core,performance: mon,osd,osdc: refactor snap trimming (phase 1) (`pr#18276 `_, Sage Weil) -* core,performance: msg/async/AsyncConnection: combine multi alloc into one (`pr#18833 `_, Haomai Wang) -* core,performance: os/bluestore: use db->rm_range_keys to delete range of keys (`pr#18279 `_, Xiaoyan Li) -* core,performance: osdc/Objecter: delay initialization of hobject_t in _send_op (`issue#21845 `_, `pr#18427 `_, Jason Dillaman) -* core,performance: osd,dmclock: use pointer to ClientInfo instead of a copy of it (`pr#18387 `_, Kefu Chai) -* core,performance: osd/OSD: batch-list objects to reduce memory consumption (`pr#20767 `_, xie xingguo) -* core,performance: osd/osd_type: get_clone_bytes - inline size() for overlapping size (`pr#17823 `_, xie xingguo) -* core,performance: osd/PG: allow scrub preemption (`pr#18971 `_, Sage Weil) -* core,performance: osd/PGLog: get rid of ineffective container operations (`pr#19161 `_, xie xingguo) -* core,performance: osd/PGLog: write only changed dup entries (`issue#21026 `_, `pr#17245 `_, Josh Durgin) -* core,performance: osd/PG: PGPool::update: avoid expensive union_of (`pr#17239 `_, Zac Medico) -* core,performance: osd/PGPool::update: optimize with subset_of (`pr#17820 `_, Zac Medico) -* core,performance: osd/PG: reduce some overhead on operating MissingLoc (`pr#18186 `_, xie xingguo) -* core,performance: osd/PG: revert approx size (`issue#22654 `_, `pr#18755 `_, Adam Kupczyk) -* core,performance: osd/PrimaryLogPG: arrange recovery order by number of missing objects (`pr#18292 `_, xie xingguo) -* core,performance: osd/PrimaryLogPG: calc clone_overlap size in a more efficient and concise way (`pr#17928 `_, xie xingguo) -* core,performance: osd/PrimaryLogPG: do not set data digest for bluestore (`pr#17515 `_, xie xingguo) -* core,performance: osd/PrimaryLogPG: kill add_interval_usage (`pr#17807 `_, xie xingguo) -* core,performance: os/filestore: disable rocksdb compression (`pr#18707 `_, Sage Weil) -* core,performance: PGPool::update: optimize removed_snaps comparison when possible (`pr#17410 `_, Zac Medico) -* core,performance: PGPool::update: optimize with interval_set.swap (`pr#17121 `_, Zac Medico) * core: PG: fix name of WaitActingChange (`pr#18768 `_, wumingqiao) +* core: pg: handle MNotifyRec event in down state (`pr#20959 `_, Mingxin Liu) +* core: PGPool::update: optimize removed_snaps comparison when possible (`pr#17410 `_, Zac Medico) +* core: PGPool::update: optimize with interval_set.swap (`pr#17121 `_, Zac Medico) * core: PG: primary should not be in the peer_info, skip if it is (`pr#20189 `_, Neha Ojha) * core: ptl-tool: checkout branch after creation (`pr#18157 `_, Patrick Donnelly) * core: ptl-tool: load labeled PRs (`pr#18231 `_, Patrick Donnelly) * core: ptl-tool: make branch name configurable (`pr#18499 `_, Patrick Donnelly) * core: ptl-tool: print bzs/tickets cited in commit msgs/comments (`pr#18547 `_, Patrick Donnelly) * core: pybind/ceph_argparse: fix cli output info (`pr#17667 `_, Luo Kexue) +* core: pybind/ceph_argparse: Fix UnboundLocalError if command doesn't validate (`pr#21342 `_, Tim Serong) +* core: pybind/ceph_argparse.py:'timeout' must in kwargs when call run_in_thread (`pr#21659 `_, yangdeliu) * core,pybind: pybind/ceph_argparse: accept flexible req (`pr#20791 `_, Gu Zhongyan) -* core,pybind: pybind/rados: add rados_service*() (`pr#18812 `_, Kefu Chai) +* core,pybind: pybind/rados: add alignment getter to IoCtx (`pr#21222 `_, Bruce Flynn) +* core,pybind: pybind/rados: add rados_service\_\*() (`pr#18812 `_, Kefu Chai) * core,pybind: pybind/rados: add support open_ioctx2 API (`pr#17710 `_, songweibin) * core,pybind: rados: support python API of "set_osdmap_full_try" (`pr#17418 `_, songweibin) * core: qa: fix the potential delay of pg state change (`pr#17253 `_, huangjun) @@ -1131,22 +1271,28 @@ Other Notable Changes * core: qa/suites/rados/rest/mgr: provision openstack volumes (`pr#20573 `_, Sage Weil) * core: qa/suites/rados/singleton/all/mon-seesaw: whitelist MON_DOWN (`pr#18246 `_, Sage Weil) * core: qa/suites/rados/singleton/all/recover-preemption: handle slow starting osd (`pr#18078 `_, Sage Weil) +* core: qa/suites/rados/singleton/all/recovery_preemption: whitelist SLOW_OPS (`pr#21250 `_, Sage Weil) * core: qa/suites/rados/singleton/diverget_priors\*: broaden whitelist (`pr#17379 `_, Sage Weil) * core: qa/suites/rados/thrash: extend mgr beacon grace when many msgr failures injected (`issue#21147 `_, `pr#19242 `_, Sage Weil) +* core: qa/suites/rados/verify/tasks/rados_api_tests: whitelist OBJECT_MISPLACED (`pr#21646 `_, Sage Weil) * core: qa/workunits/rest/test.py: stop trying to test obsolte cluster_up/down (`pr#18552 `_, Sage Weil) +* core: rados/objclass.h: fix build define CEPH_CLS_API in all cases (`pr#21606 `_, Danny Al-Gaaf) * core: rados: use WaitInterval()'s return value instead of manual timing (`pr#20028 `_, Mohamad Gebai) * core,rbd: common,rbd-nbd: fix up prefork behavior vs AsyncMessenger singletons (`issue#23143 `_, `pr#20681 `_, Sage Weil) * core,rbd: librbd,os: address coverity false positives (`pr#17793 `_, Amit Kumar) * core,rbd: mgr,osd,kv: Fix various warnings for Clang and GCC7 (`pr#17976 `_, Adam C. Emerson) * core,rbd: vstart.sh: fix mstart variables (`pr#20826 `_, Sage Weil) * core: rdma: Assign instead of compare (`pr#16664 `_, amitkuma) -* core,rdma: msg/asyc/rmda: fix the bug of assert when Infiniband::recv_msg receives disconnect message (`pr#17688 `_, Jin Cai) -* core,rdma: msg/async/rdma: fix multi cephcontext confllicting (`pr#16893 `_, Haomai Wang) * core: remove startsync (`issue#20604 `_, `pr#16396 `_, Amit Kumar) * core: rocksdb: sync with upstream (`issue#20529 `_, `pr#17388 `_, Kefu Chai) +* core: rocksdb: sync with upstream (`pr#21320 `_, Kefu Chai) +* core: scrub errors not cleared on replicas can cause inconsistent pg state when replica takes over primary (`issue#23267 `_, `pr#21101 `_, David Zafman) * core: Snapset inconsistency is detected with its own error (`issue#22996 `_, `pr#20450 `_, David Zafman) * core: src/messages/MOSDMap: reencode OSDMap for older clients (`issue#21660 `_, `pr#18134 `_, Sage Weil) +* core: src/osd/PG.cc: 6455: FAILED assert(0 == "we got a bad state machine event") (`pr#20933 `_, David Zafman) * core: src/test/osd: add two pool test for manifest objects (`pr#20096 `_, Myoungwon Oh) +* core: test/cli/osdmaptool/test-map-pgs.t: remove nondetermimistic test (`pr#20872 `_, Sage Weil) +* core: test/objectstore_bench: Don't forget judging whether call usage (`pr#21369 `_, Jianpeng Ma) * core,tests: ceph_test_filestore_idempotent_sequence: many fixes (`issue#22920 `_, `pr#20279 `_, Sage Weil) * core,tests: ceph_test_objectstore: drop expect regex (`pr#16968 `_, Sage Weil) * core,tests: Erasure code read test and code cleanup (`issue#14513 `_, `pr#17703 `_, David Zafman) @@ -1156,18 +1302,25 @@ Other Notable Changes * core,tests: os/memstore: memstore_page_set=false (`issue#20738 `_, `pr#16995 `_, Sage Weil) * core,tests: qa/ceph_manager: check pg state again before timedout (`issue#21294 `_, `pr#17810 `_, huangjun) * core,tests: qa/clusters/fixed-[23]: 4 osds per node, not 3 (`pr#16799 `_, Sage Weil) +* core,tests: qa: modify rgw default pool names (`pr#21630 `_, Neha Ojha) * core,tests: qa/objectstore/bluestore\*: less debug output (`issue#20910 `_, `pr#17505 `_, Sage Weil) * core,tests: qa/overrides/2-size-2-min-size: whitelist REQUEST_STUCK (`pr#17243 `_, Sage Weil) * core,tests: qa/standalone/ceph-helpers: pass --verbose to ceph-disk (`pr#19456 `_, Sage Weil) * core,tests: qa/standalone/scrub/osd-scrub-repair: fix grep pattern (`issue#21127 `_, `pr#17258 `_, Sage Weil) * core,tests: qa/standalone/scrub/osd-scrub-snaps: adjust test for lack of snapdir objects (`pr#17927 `_, Sage Weil) * core,tests: qa/suites/rados/monthrash: tolerate PG_AVAILABILITY during mon thrashing (`pr#18122 `_, Sage Weil) +* core,tests: qa/suites/rados/monthrash: whitelist SLOW_OPS (`pr#21331 `_, Sage Weil) * core,tests: qa/suites/rados/objectstore: logs (`issue#20738 `_, `pr#16923 `_, Sage Weil) * core,tests: qa/suites/rados/perf: create pool with lower pg_num (`pr#17819 `_, Neha Ojha) * core,tests: qa/suites/rados/rest/mgr-restful: whitelist more health (`pr#18457 `_, Sage Weil) * core,tests: qa/suites/rados/rest: move rest_test from qa/suites/rest/ (`pr#19175 `_, Sage Weil) * core,tests: qa/suites/rados/thrash: fix thrashing with ec vs map discon (`pr#16842 `_, Sage Weil) +* core,tests: qa/suites/rados/thrash-old-clients: add hammer clients (`pr#21703 `_, Sage Weil) +* core,tests: qa/suites/rados/thrash-old-clients: add rbd tests (`pr#21704 `_, Sage Weil) +* core,tests: qa/suites/rados/thrash-old-clients: do some thrashing with jewel and luminous clients (`pr#21679 `_, Sage Weil) +* core,tests: qa/suites/rados/thrash-old-clients: only centos and 16.04 (`pr#22125 `_, Sage Weil) * core,tests: qa/suites/upgrade/jewel-x/stress-split: tolerate sloppy past_intervals (`pr#17226 `_, Sage Weil) +* core,tests: qa/suites/upgrade/luminous-x/stress-split: avoid enospc (`pr#21753 `_, Sage Weil) * core,tests: qa/tasks/ceph_manager: revive osds before doing final rerr reset (`issue#21206 `_, `pr#17406 `_, Sage Weil) * core,tests: qa/tasks/ceph_manager: tolerate tell osd.\* error (`pr#19365 `_, Sage Weil) * core,tests: qa/tasks/ceph.py: tolerate flush pg stats exception (`pr#16905 `_, Sage Weil) @@ -1175,7 +1328,8 @@ Other Notable Changes * core,tests: qa/tasks/thrashosds: set min_in default to 4 (`issue#21997 `_, `pr#18670 `_, Sage Weil) * core,tests: qa/tests: run ceph-ansible task on installer.0 role/node (`pr#19605 `_, Yuri Weinstein) * core,tests: qa: tolerate failure to force backfill (`issue#22614 `_, `pr#19765 `_, Sage Weil) -* core,tests: qa/workunits/rados/test_rados_tool: fix stray `|`, race (`issue#22676 `_, `pr#19946 `_, Sage Weil) +* core,tests: qa/workunits/rados/test_rados_tool: fix stray ``|``, race (`issue#22676 `_, `pr#19946 `_, Sage Weil) +* core,tests: qa/workunits/rados/test.sh: ensure tee output is valid filename (`pr#21507 `_, Sage Weil) * core,tests: rados: Initialization of alignment (`pr#17723 `_, Amit Kumar) * core,tests: rados: Initializing members of librados/TestCase.h (`pr#16896 `_, amitkuma) * core,tests: test: Checking fd for negative before closing (`pr#17190 `_, amitkuma) @@ -1192,14 +1346,19 @@ Other Notable Changes * core: vstart.sh: drop .ceph_port and use randomly selected available port (`pr#19268 `_, Shinobu Kinjo) * core: vstart.sh: drop --{mon,osd,mds,rgw,mgr}_num options (`pr#18648 `_, Kefu Chai) * core: vstart.sh: Remove duplicate global section (`pr#17543 `_, iliul) +* crush: cleanup update_device_class() log messages (`pr#21174 `_, Gu Zhongyan) * crush: fix CrushCompiler won't compile maps with empty shadow tree (`pr#17058 `_, xie xingguo) +* crush: fix device_class_clone for unpopulated/empty weight-sets (`issue#23386 `_, `pr#22169 `_, Sage Weil) * crush: fix fast rule lookup when uniform (`pr#17510 `_, Sage Weil) * crush: force rebuilding shadow hierarchy after swapping buckets (`pr#17083 `_, xie xingguo) * crush: improve straw2 algorithm's readability (`pr#20196 `_, Yao Zongyou) * crush: "osd crush class rename" support (`pr#16961 `_, xie xingguo) +* crush, osd: handle multiple parents properly when applying pg upmaps (`issue#23921 `_, `pr#21835 `_, xiexingguo) * crush: safe check for 'ceph osd crush swap-bucket' (`pr#17335 `_, Carudy) * crush: various CrushWrapper cleanups (`pr#17360 `_, Kefu Chai) * crush: various weight-set fixes (`pr#17014 `_, xie xingguo) +* denc: should check element's type not 'size_t' (`pr#19986 `_, Kefu Chai) +* denc: use constexpr-if to replace some SFINAE impls (`pr#19662 `_, Kefu Chai) * doc: 12.1.3 release notes (`pr#16975 `_, Abhishek Lekshmanan) * doc: 12.2.0 major release announcements (`pr#16915 `_, Abhishek Lekshmanan) * doc: 12.2.1 release notes (`pr#18014 `_, Abhishek Lekshmanan) @@ -1207,7 +1366,6 @@ Other Notable Changes * doc: add 12.2.2 release notes (`pr#19264 `_, Abhishek Lekshmanan) * doc: add allow_multimds and fs_name parameter (`pr#15847 `_, Jan Fajerski) * doc: add ceph-kvstore-tool's man (`pr#17092 `_, liuchang0812) -* doc: Add cephmetrics suite (`pr#18451 `_, Zack Cerza) * doc: add changelog for 12.2.1 (`pr#18020 `_, Abhishek Lekshmanan) * doc: add changelog for v11.2.1 (`pr#16956 `_, Abhishek Lekshmanan) * doc: add changelog for v12.2.2 (`pr#19284 `_, Abhishek Lekshmanan) @@ -1223,18 +1381,25 @@ Other Notable Changes * doc: add osd_max_object_size in osd configuration (`pr#18115 `_, Mohamad Gebai) * doc: build-doc: Upgrade ceph python libraries (`pr#20726 `_, Boris Ranto) * doc: ceph-disk: create deprecation warnings (`issue#22154 `_, `pr#18988 `_, Alfredo Deza) +* doc: ceph-volume: automatic VDO detection (`issue#23581 `_, `pr#21451 `_, Alfredo Deza) * doc: ceph-volume docs (`pr#17068 `_, Alfredo Deza) +* doc: ceph-volume document multipath support (`pr#20878 `_, Alfredo Deza) +* doc: ceph-volume doc updates (`pr#20758 `_, Alfredo Deza) +* doc: ceph-volume include physical devices associated with an LV when listing (`pr#21645 `_, Alfredo Deza) * doc: ceph-volume lvm bluestore support (`pr#18448 `_, Alfredo Deza) +* doc/ceph-volume OSD use the fsid file, not the osd_fsid (`issue#22427 `_, `pr#20059 `_, Alfredo Deza) * doc: change boolean option default value from zero to false (`pr#17733 `_, Yao Zongyou) * doc: change cn mirror to ustc domain (`pr#18081 `_, Shengjing Zhu) * doc: changelog for v12.2.3 (`pr#20503 `_, Abhishek Lekshmanan) * doc: cleanup erasure coded pool doc on cephfs use (`pr#20572 `_, Patrick Donnelly) +* doc: CodingStyle: add python and javascript/typescript (`pr#20186 `_, Joao Eduardo Luis) * doc: common/options: document filestore and filejournal options (`pr#17739 `_, Sage Weil) * doc: common/options: document objecter, filer, and journal options (`pr#17740 `_, Sage Weil) * doc: complete and update the subsystem logging level info table (`pr#18500 `_, Luo Kexue) * doc: correcting typos in bluestore-config-ref and bluestore-migration (`pr#19154 `_, Katie Holly) * doc: correct wrong bluestore config types (`pr#18205 `_, Yao Zongyou) * doc: delete duplicate words (`pr#17104 `_, iliul) +* doc: dev description of async recovery (`pr#21051 `_, Neha Ojha, Josh Durgin) * doc: doc/bluestore: add SPDK usage for bluestore (`pr#17654 `_, Haomai Wang) * doc: doc/cephfs/experimental-features: kernel client snapshots limit (`pr#18579 `_, Ilya Dryomov) * doc: doc/cephfs/posix: remove stale information for seekdir (`pr#17658 `_, "Yan, Zheng") @@ -1248,8 +1413,11 @@ Other Notable Changes * doc: doc/dev/perf: add doc on disabling -fomit-frame-pointer (`pr#17358 `_, Kefu Chai) * doc: doc for mount.fuse.ceph (`issue#21539 `_, `pr#19172 `_, Jos Collin) * doc: doc/man remove deprecation of ceph-disk man page title (`pr#19325 `_, Alfredo Deza) +* doc: doc/mgr: Add limitations section to plugin guide (`pr#21347 `_, Tim Serong) * doc: doc/mgr: add "local pool" plugin to toc (`pr#17961 `_, Kefu Chai) +* doc: doc/mgr/balancer: document (`issue#22789 `_, `pr#21421 `_, Sage Weil) * doc: doc/mgr: document facilities methods using `automethod` directive (`pr#18680 `_, Kefu Chai) +* doc: doc/mgr/plugins: add note about distinction between config and kv store (`pr#21671 `_, Jan Fajerski) * doc: doc/mgr: remove non user-facing code from doc (`pr#20372 `_, Kefu Chai) * doc: doc,os,osdc: drop and modify comments (`pr#17732 `_, songweibin) * doc: doc/rados: Add explanation of straw2 (`pr#19247 `_, Shinobu Kinjo) @@ -1275,15 +1443,20 @@ Other Notable Changes * doc: doc/scripts: py3 compatible (`pr#17640 `_, Kefu Chai) * doc: docs: Do not use "min size = 1" as an example (`pr#17912 `_, Alfredo Deza) * doc: docs fix ceph-volume missing sub-commands (`issue#23148 `_, `pr#20673 `_, Alfredo Deza) +* doc: doc/start/os-recommendations.rst: bump krbd kernels (`pr#21478 `_, Ilya Dryomov) * doc: docs update ceph-deploy reference to reflect ceph-volume API (`pr#20510 `_, Alfredo Deza) * doc: document ceph-disk prepare class hierarchy (`pr#17019 `_, Loic Dachary) * doc: document include/ipaddr.h (`issue#12056 `_, `pr#17613 `_, Nathan Cutler) * doc: drop duplicate line in ceph-bluestore-tool man page (`pr#19169 `_, Xiaojun Liao) * doc: eliminate useless cat statement (`pr#17154 `_, Ken Dreyer) * doc: examples: add new librbd example (`pr#18314 `_, Mahati Chamarthy) +* doc: expand developer documentation of unit tests (`pr#19594 `_, Nathan Cutler) +* doc: Fix a grammar error in rbd-snapshot.rst (`pr#21470 `_, Zeqing Tyler Qi) * doc: fix CFLAGS in doc/dev/cpu-profiler.rst (`pr#19752 `_, Chang Liu) * doc: fix desc of option "mon cluster log file" (`pr#18770 `_, Kefu Chai) * doc: fix doc/radosgw/admin.rst typos (`pr#17397 `_, Enming Zhang) +* doc: Fix dynamic resharding doc formatting (`pr#20970 `_, Ashish Singh) +* doc: fix error in osd scrub load threshold (`pr#21678 `_, Dirk Sarpe) * doc: Fixes a spelling error and a broken hyperlink (`pr#20442 `_, Jordan Hus) * doc: Fixes rbd snapshot flatten example (`issue#17723 `_, `pr#17436 `_, Ashish Singh) * doc: fixes syntax in osd-config-ref (`issue#21733 `_, `pr#18188 `_, Joshua Schmid) @@ -1307,6 +1480,7 @@ Other Notable Changes * doc: githubmap: add map for GitHub contributor lookup (`pr#17457 `_, Patrick Donnelly) * doc: .githubmap, .mailmap, .organizationmap, .peoplemap: update Igor (`pr#19314 `_, Igor Fedotov) * doc: globally change CRUSH ruleset to CRUSH rule (`issue#20559 `_, `pr#19435 `_, Nathan Cutler) +* doc: Improved dashboard documentation (`pr#21443 `_, Lenz Grimmer) * doc: Improved hitset parameters description (`pr#19691 `_, Alexey Stupnikov) * doc: improve links in doc/releases.rst (`pr#18155 `_, Nathan Cutler) * doc: Improve mgr/restful module documentation (`pr#20717 `_, Boris Ranto) @@ -1314,6 +1488,8 @@ Other Notable Changes * doc: include ceph-disk and ceph-disk-volume man pages in index (`pr#17168 `_, Alfredo Deza) * doc: init flags to 0 in rados example (`pr#20671 `_, Patrick Donnelly) * doc: Kube + Helm installation (`pr#18520 `_, Alexandre Marangone) +* doc: legal: remove doc license ambiguity (`issue#23336 `_, `pr#20876 `_, Nathan Cutler) +* doc: lock_timeout is a per mapping option (`pr#21563 `_, Ilya Dryomov) * doc: log-and-debug: fix default value of "log max recent" (`pr#20316 `_, Nathan Cutler) * doc: mailmap: Add Sibei, XueYu Affiliation (`pr#18395 `_, Sibei Gao) * doc: mailmap: Fixed maintenance guide URL (`pr#18076 `_, Jos Collin) @@ -1325,27 +1501,42 @@ Other Notable Changes * doc: mailmap: update affiliation for Mykola Golub (`pr#19667 `_, Mykola Golub) * doc: mailmap: Update umcloud affiliation (`pr#17441 `_, Yixing Yan) * doc: make the commands in README.md properly aligned (`pr#18639 `_, Yao Zongyou) +* doc/man: add "ls" to "ceph osd" command's subcommands list (`pr#19382 `_, Rishabh Dave) * doc: "mds blacklist interval" vs manually blacklisting (`pr#18195 `_, Ken Dreyer) +* doc: mgr/dashboard.rst: mention ceph.conf and ceph mgr services (`pr#20961 `_, Nathan Cutler) +* doc/mgr/plugins: mgr accessor during init causes exception (`pr#16973 `_, Jan Fajerski) +* doc: mimic: doc: Updated dashboard documentation (features, SSL config) (`pr#22079 `_, Lenz Grimmer) * doc: misc fix spell errors in osd/OSD and doc (`pr#17107 `_, songweibin) +* doc: misc: fix various spelling errors (`pr#20831 `_, Shengjing Zhu) * doc: Misc iSCSI doc updates (`pr#19931 `_, Mike Christie) * doc: move glance_api_version option to the right place (`pr#17337 `_, Luo Kexue) * doc: options.cc: document rgw config options (`pr#18007 `_, Yehuda Sadeh) * doc: organizationmap: Add Adam Wolfe Gordon's affiliation (`pr#18295 `_, Adam Wolfe Gordon) * doc: organizationmap: Add Ashish Singh affiliation (`pr#17109 `_, Ashish Singh) +* doc: .organizationmap: add Xin Yuan and Yichao Li's affiliation (`pr#21170 `_, Li Wang) +* doc: PendingReleaseNotes: Added note about Dashboard v2, fixed typo (`pr#21597 `_, Lenz Grimmer) * doc: PendingReleaseNotes:Announce FreeBSD availability (`pr#16782 `_, Willem Jan Withagen) +* doc: PendingReleaseNotes: mention some monitor changes (`pr#21474 `_, Joao Eduardo Luis) * doc: PendingReleaseNotes: note about upmap mapping change in luminous release notes (`pr#17813 `_, Sage Weil) * doc: qa,doc: drop support of ubuntu trusty (`pr#19307 `_, Kefu Chai) +* doc/rados/operations/bluestore-migration: typos and whitespace (`pr#16991 `_, Sage Weil) +* doc/rados/operations/bluestore-migration: typos (`pr#17581 `_, Sage Weil) * doc: README: Improve vstart.sh usage (`pr#17644 `_, Fabian Vogt) * doc: README.md: bump up cmake to 2.8.12 (`pr#18348 `_, Yan Jun) * doc: redundant "cephfs" when set the "allow_multimds" (`pr#20045 `_, Shangzhong Zhu) * doc: release notes: fix grammar/style nits (`pr#18876 `_, Nathan Cutler) * doc: release notes for 12.2.3 (`pr#20500 `_, Abhishek Lekshmanan) * doc: release notes for v12.1.4 Luminous (`pr#17037 `_, Abhishek Lekshmanan) +* doc/release-notes: remove mention of crush weight optimization (`pr#16974 `_, Sage Weil) * doc: release-notes.rst: add Kraken v11.2.1 and update releases.rst (`pr#16879 `_, Nathan Cutler) * doc: release notes update for 10.2.10 (`pr#18148 `_, Abhishek Lekshmanan) +* doc/releases: drop LTS/stable line from second chart (`pr#18153 `_, Sage Weil) * doc: Remove additional arguments when replacing OSD (`pr#18345 `_, Wido den Hollander) * doc: remove duplicated --max-buckets option desc (`pr#19737 `_, Kefu Chai) +* doc: remove references to unversioned repository addresses (`pr#21357 `_, Greg Farnum) +* doc: remove unused config: "osd op threads" (`pr#21319 `_, Jianpeng Ma) * doc: rename changelog with a .txt extension (`pr#18156 `_, Abhishek Lekshmanan) +* doc: reorganize releases (`pr#20784 `_, Abhishek Lekshmanan) * doc: replace injectargs usage with "config set" (`pr#18789 `_, John Spray) * doc: replace region with zonegroup in configure bucket sharding section (`issue#21610 `_, `pr#18063 `_, Orit Wasserman) * doc: restructure bluestore migration insructions (`pr#17603 `_, Sage Weil) @@ -1356,32 +1547,30 @@ Other Notable Changes * doc: rgw mention about tagging & bucket policies in s3api (`pr#16907 `_, Abhishek Lekshmanan) * doc: rgw: mention the civetweb support for binding to multiple ports (`issue#20942 `_, `pr#17141 `_, Abhishek Lekshmanan) * doc: rm stray ")" character from mds config ref (`pr#18228 `_, Ken Dreyer) -* docs: doc/ceph-volume OSD use the fsid file, not the osd_fsid (`issue#22427 `_, `pr#20059 `_, Alfredo Deza) -* docs: doc/man: add "ls" to "ceph osd" command's subcommands list (`pr#19382 `_, Rishabh Dave) -* docs: doc/mgr/plugins: mgr accessor during init causes exception (`pr#16973 `_, Jan Fajerski) -* docs: doc/rados/operations/bluestore-migration: typos and whitespace (`pr#16991 `_, Sage Weil) -* docs: doc/rados/operations/bluestore-migration: typos (`pr#17581 `_, Sage Weil) -* docs: doc/release-notes: remove mention of crush weight optimization (`pr#16974 `_, Sage Weil) -* docs: doc/releases: drop LTS/stable line from second chart (`pr#18153 `_, Sage Weil) -* docs: docs: ceph-volume CLI updates (`pr#17425 `_, Alfredo Deza) -* docs: doc/templates update toctree call to include hidden entries (`pr#17076 `_, Alfredo Deza) -* docs: examples: fix link order in librados example Makefile (`pr#17842 `_, Mahati Chamarthy) -* docs: follow-up fixups for atomic_t spinlocks (`pr#17611 `_, Jesse Williamson) -* docs: githubmap: Add ktdreyer (`pr#19209 `_, Jos Collin) +* docs: ceph-volume CLI updates (`pr#17425 `_, Alfredo Deza) +* doc: s/deamon/daemon/ (`pr#20931 `_, ashitakasam) +* doc: some improvements to ceph-conf.rst (`pr#21268 `_, Nathan Cutler) * doc: Specify mount details in ceph-fuse (`pr#20071 `_, Jos Collin) * doc: SubmittingPatches: clarify PR title section (`pr#17143 `_, Nathan Cutler) +* doc/templates update toctree call to include hidden entries (`pr#17076 `_, Alfredo Deza) * doc: the client inputs the pool name instead of pool ID (`pr#17672 `_, Frank Yu) +* doc: typo fix (`pr#21077 `_, Ashita Dashottar) * doc: update Blacklisting and OSD epoch barrier (`issue#22542 `_, `pr#19701 `_, Jos Collin) * doc: update ceph-disk with a state-transition diagram (`pr#17639 `_, Kefu Chai) * doc: update ceph iscsi kernel and package info (`pr#20020 `_, Mike Christie) * doc: Update commands and options in radosgw-admin (`pr#18267 `_, Jos Collin) * doc: update Component Technical Leads and maintainers to canonical location (`pr#18376 `_, Patrick McGarry) * doc: Update config file search paths to reflect reality (`pr#19882 `_, Adam Wolfe Gordon) +* doc: updated add primary storage documentation for latest CloudStack release (4.11) (`pr#21050 `_, James McClune, John Wilkins) +* doc: Update dashboard feature list (added RGW management) (`pr#21781 `_, Lenz Grimmer) +* doc: updated dashboard feature list (added new RGW details, Pools) (`pr#21562 `_, Lenz Grimmer) +* doc: Updated dashboard feature list (`pr#21693 `_, Lenz Grimmer) * doc: Updated dashboard v2 feature list (`pr#20755 `_, Lenz Grimmer) * doc: Updated documentation for Zabbix Mgr module (`pr#18356 `_, Wido den Hollander) * doc: update default value of option mon_sync_timeout (`pr#17802 `_, Yao Guotao) * doc: update default value of parameter mon_subscribe_interval (`pr#17669 `_, yaoguotao) * doc: Update docs to remove gitbuilder and add shaman references (`pr#17022 `_, Alfredo Deza) +* doc: updated the dashboard feature list (`pr#21531 `_, Lenz Grimmer) * doc: Updated the get-packages.rst to luminous (`pr#20815 `_, Kai Wagner) * doc: update firewall doc to mention ceph-mgr (`pr#17974 `_, John Spray) * doc: update iSCSI upstream kernel to 4.16 (`pr#20695 `_, Mike Christie) @@ -1397,14 +1586,31 @@ Other Notable Changes * doc: update references to use ceph-volume (`pr#19241 `_, Alfredo Deza) * doc: update releases to the current state (`pr#17364 `_, Abhishek Lekshmanan) * doc: Updates to bluestore migration doc (`pr#17602 `_, David Galloway) +* doc: v12.2.5 luminous release notes (`pr#21621 `_, Abhishek Lekshmanan) * doc: various cleanups (`pr#18480 `_, Kefu Chai) +* examples: fix link order in librados example Makefile (`pr#17842 `_, Mahati Chamarthy) +* Fix ceph-mgr restarts (`pr#22051 `_, Boris Ranto) +* follow-up fixups for atomic_t spinlocks (`pr#17611 `_, Jesse Williamson) +* githubmap: Add ktdreyer (`pr#19209 `_, Jos Collin) +* include/buffer.h: fix typo in comment (`pr#17489 `_, mychoxin) +* include/ceph_features: fix OS_PERF_STAT_NS's incarnation (`pr#21467 `_, Kefu Chai) +* install-deps.sh: fix an error condition expression (`pr#20819 `_, Yao Guotao) +* java/native: fix milliseconds to mtime/atime conversion (`pr#17460 `_, dengquan) +* java/native: s/jni: lstat/jni: stat in native_ceph_stat (`pr#20142 `_, Shangzhong Zhu) +* KStore: statfs needs extra includes on FreeBSD (`pr#21429 `_, Willem Jan Withagen) +* kv/leveldb: fix deadlock when close db (`pr#16643 `_, Zengran) +* kv: unify {create_and\_,}open() methods (`pr#18177 `_, Kefu Chai) * librados: add async interfaces for use with Networking TS (`pr#19054 `_, Casey Bodley) +* librados: block MgrClient::start_command until mgrmap (`pr#21832 `_, John Spray, Kefu Chai) +* librados: extend C API for so it accepts keys with NUL chars (`pr#20314 `_, Piotr Dałek) * librados: Fix a potential risk of buffer::list::claim_prepend(list& b… (`issue#21338 `_, `pr#17661 `_, Guan yunfei) +* librados: fix potential race condition if notify immediately fails (`issue#23966 `_, `pr#21859 `_, Jason Dillaman) * librados: getter for min compatible client versions (`pr#20080 `_, Jason Dillaman) * librados: invalid free() in rados_getxattrs_next() (`issue#22042 `_, `pr#20260 `_, Gu Zhongyan) * librados: make OPERATION_FULL_FORCE the default for rados_remove() (`issue#22413 `_, `pr#20534 `_, Kefu Chai) * librbd: abstract hard-coded journal and cache hooks on IO path (`pr#20682 `_, Jason Dillaman) * librbd: Add a function to list image watchers (`pr#19188 `_, Adam Wolfe Gordon) +* librbd: add API function to get image name (`pr#20935 `_, Mykola Golub) * librbd: added preprocessor macro for detecting compare-and-write support (`issue#22036 `_, `pr#18708 `_, Jason Dillaman) * librbd: add eventtrace support (`pr#19251 `_, Mahati Chamarthy) * librbd: add preliminary support for new operation feature bit (`pr#19903 `_, Jason Dillaman) @@ -1420,6 +1626,8 @@ Other Notable Changes * librbd: deep_copy: don't create snapshots above snap_id_end (`pr#19383 `_, Mykola Golub) * librbd: default localize parent reads to false (`issue#20941 `_, `pr#16882 `_, Jason Dillaman) * librbd: default to sparse-reads for any IO operation over 64K (`issue#21849 `_, `pr#18405 `_, Jason Dillaman) +* librbd: disable ENOENT tracking within the object cacher (`issue#23597 `_, `pr#21308 `_, Jason Dillaman) +* librbd: disallow creation of v1 image format (`pr#20460 `_, Julien COLLET, Julien Collet) * librbd: don't read metadata twice on image open (`pr#18542 `_, Mykola Golub) * librbd: drop redundant check for null ImageCtx (`pr#18265 `_, Jianpeng Ma) * librbd: filter out potential race with image rename (`issue#18435 `_, `pr#19618 `_, Jason Dillaman) @@ -1428,6 +1636,8 @@ Other Notable Changes * librbd: fix don't send get_stripe_unit_count if striping is not enabled (`issue#21360 `_, `pr#17660 `_, Yanhu Cao) * librbd: fix issues discovered in clone v2 during upgrade tests (`issue#22979 `_, `pr#20406 `_, Jason Dillaman) * librbd: fix missing return in NotifyMessage::get_notify_op (`pr#20656 `_, Yao Zongyou) +* librbd: fix rbd close race with rewatch (`pr#21141 `_, Song Shun) +* librbd: fix refuse to release lock when cookie is the same at rewatch (`pr#20868 `_, Song Shun) * librbd: fix structure size check in rbd_mirror_image_get_info/status (`pr#20478 `_, Mykola Golub) * librbd: force removal of a snapshot cannot ignore dependent children (`issue#22791 `_, `pr#20105 `_, Jason Dillaman) * librbd: generalized deep copy function (`pr#16238 `_, Mykola Golub) @@ -1440,16 +1650,22 @@ Other Notable Changes * librbd: Initializing members image,operation,journal (`pr#16934 `_, amitkuma) * librbd: Initializing member variables (`pr#16867 `_, amitkuma) * librbd: journal should ignore -EILSEQ errors from compare-and-write (`issue#21628 `_, `pr#18099 `_, Jason Dillaman) +* librbd,librados: do not include stdbool.h in C++ headers (`pr#19945 `_, Kefu Chai) * librbd: list_children should not attempt to refresh image (`issue#21670 `_, `pr#18114 `_, Jason Dillaman) * librbd: minor cleanup of the IO pathway (`pr#20560 `_, Jason Dillaman) +* librbd: minor code cleanup (`pr#21165 `_, songweibin) +* librbd: missing 'return' in deep_copy::ObjectCopyRequest::send_read_object (`pr#21493 `_, Mykola Golub) * librbd: new tag should use on-disk committed position (`issue#22945 `_, `pr#20423 `_, Jason Dillaman) * librbd: object map batch update might cause OSD suicide timeout (`issue#21797 `_, `pr#18315 `_, Jason Dillaman) * librbd: possible deadlock with synchronous maintenance operations (`issue#22120 `_, `pr#18909 `_, Jason Dillaman) * librbd: potential crash if object map check encounters error (`issue#22819 `_, `pr#20214 `_, Jason Dillaman) +* librbd: potential race between discard and writeback (`pr#21248 `_, Jason Dillaman) * librbd: potential race in RewatchRequest when resetting watch_handle (`pr#20420 `_, Mykola Golub) * librbd: prefer templates to macros (`pr#19912 `_, Adam C. Emerson) * librbd: prevent overflow of discard API result code (`issue#21966 `_, `pr#18923 `_, Jason Dillaman) +* librbd: prevent watcher from unregistering with in-flight actions (`issue#23955 `_, `pr#21763 `_, Jason Dillaman) * librbd: refresh image after applying new metadata (`issue#21711 `_, `pr#18158 `_, Jason Dillaman) +* librbd: release lock executing deep copy progress callback (`issue#23929 `_, `pr#21727 `_, Mykola Golub) * librbd: remove unused member in FlattenRequest (`pr#19416 `_, Mykola Golub) * librbd: remove unused variables from ReadResult refactor (`pr#18277 `_, Jason Dillaman) * librbd: rename of non-existent image results in seg fault (`issue#21248 `_, `pr#17502 `_, Jason Dillaman) @@ -1462,39 +1678,138 @@ Other Notable Changes * librbd: update mirror::EnableRequest diagram according to code (`pr#19130 `_, Mykola Golub) * librbd: use steady clock to measure elapsed time in AioCompletion (`pr#20007 `_, Mohamad Gebai) * librbd: validate if dst group snap name is the same with src (`pr#20395 `_, songweibin) +* log: Fix AddressSanitizer: new-delete-type-mismatch (`issue#23324 `_, `pr#20930 `_, Brad Hubbard) * log: fix build on osx (`pr#18213 `_, Kefu Chai) * log: silence warning from -Wsign-compare (`pr#18326 `_, Jos Collin) * log: Use the coarse real time clock in log timestamps (`pr#18141 `_, Adam C. Emerson) * mds: check metadata pool not cluster is full (`issue#22483 `_, `pr#19602 `_, Patrick Donnelly) +* mds: fix CEPH_STAT_RSTAT definition (`pr#21633 `_, "Yan, Zheng") * mds: get rid of the "if" check which is unnecessary inside a loop (`pr#18904 `_, dongdong tao) * mds: Remove redundant null pointer check (`pr#19750 `_, Brad Hubbard) * mds: simplify the code logic in replay_alloc_ids (`pr#18893 `_, dongdong tao) +* mempool: fix lack of pool names in mempool:dump output for JSON format (`pr#18329 `_, Igor Fedotov) +* messages: Initialization of uninitialized members various classes (`pr#16848 `_, amitkuma) +* messages/MDentryLink: add const to member function (`pr#15479 `_, yonghengdexin735) +* messages,test,msg: initialize h,reply_type,owner (`pr#17767 `_, Amit Kumar) +* mgr: add mgr daemon to DaemonStateIndex with metadata (hostname) (`issue#23286 `_, `pr#20875 `_, Jan Fajerski) * mgr: add missing call to pick_addresses (`issue#20955 `_, `pr#16940 `_, John Spray) * mgr: add the ip addr of standbys (`pr#16476 `_, huanwen ren) * mgr: add units to performance counters (`issue#22747 `_, `pr#20152 `_, Rubab Syed) +* mgr: allow service daemons to unregister from ServiceMap (`pr#20761 `_, Sage Weil) * mgr: apply a threshold to perf counter prios (`pr#16699 `_, John Spray) * mgr: balancer: fixed mistype "AttributeError: 'Logger' object has no attribute 'err'" (`pr#20130 `_, Konstantin Shalygin) +* mgr: centralized setting/getting of mgr configs (`pr#21442 `_, John Spray, Rubab Syed) * mgr: ceph-mgr: can not change prometheus port for mgr (`pr#17746 `_, wujian) * mgr: common interface for TSDB modules (`pr#17735 `_, Jan Fajerski, John Spray, My Do) +* mgr/dashboard: Adapt help text if server_addr is not set (`pr#21640 `_, Volker Theile) +* mgr/dashboard: Adapt RBD form to new application_metadata type (`pr#21602 `_, Volker Theile) +* mgr/dashboard: Add Api module (`pr#21126 `_, Tiago Melo) +* mgr/dashboard: Add 'autofocus' directive (`pr#21559 `_, Volker Theile) +* mgr/dashboard: Add CdDatePipe (`pr#21087 `_, Ricardo Marques) +* mgr/dashboard: Add 'cd-error-panel' component to display error messages (`pr#21558 `_, Volker Theile) +* mgr/dashboard: Add 'cd-loading-panel' component (`pr#21618 `_, Volker Theile) +* mgr/dashboard: Add custom validators (`pr#21041 `_, Volker Theile) +* mgr/dashboard: Add DimlessBinaryDirective (`pr#20972 `_, Ricardo Marques) +* mgr/dashboard: Add ErasureCodeProfile controller (`issue#23345 `_, `pr#20920 `_, Sebastian Wagner, Stephan Müller) +* mgr/dashboard: Add 'forceIdentifier' attribute to datatable (`pr#21497 `_, Volker Theile) +* mgr/dashboard: Add helper component (`pr#20971 `_, Ricardo Marques) +* mgr/dashboard: additional fixes to block pages (`pr#20941 `_, Jason Dillaman) +* mgr/dashboard: Add minimalistic browsable API (`pr#20873 `_, Sebastian Wagner) +* mgr/dashboard: Add notification service/component (`pr#21078 `_, Tiago Melo) +* mgr/dashboard: Add Pool-create to the backend (`issue#23345 `_, `pr#20865 `_, Sebastian Wagner) +* mgr/dashboard: Add RGW user and bucket management features (`pr#21351 `_, Volker Theile) +* mgr/dashboard: Adds reusable deletion dialog (`pr#20899 `_, Stephan Müller, Tiago Melo) +* mgr/dashboard: Add submit button component (`pr#21011 `_, Tiago Melo) +* mgr/dashboard: Add usage bar component (`pr#21128 `_, Ricardo Marques) +* mgr/dashboard: Angular modules cleanup (`pr#21402 `_, Tiago Melo) +* mgr/dashboard: Asynchronous tasks (frontend) (`pr#20962 `_, Ricardo Marques) +* mgr/dashboard: awsauth: fix python3 string decode problem (`pr#21875 `_, Ricardo Dias) +* mgr/dashboard: Change font-family of checkbox (`pr#21787 `_, Tiago Melo) +* mgr/dashboard: Clean up Pylint warnings (`pr#21694 `_, Sebastian Wagner) +* mgr/dashboard: Convert floating values to bytes (`pr#21677 `_, Stephan Müller) +* mgr/dashboard: Convert the RBD feature names to a list of strings (`pr#21024 `_, Tatjana Dehler) +* mgr/dashboard: Deletion dialog falsely executes deletion when pressing 'Cancel' (`pr#22032 `_, Volker Theile) +* mgr/dashboard: Display notification if RGW is not configured (`pr#21977 `_, Volker Theile) +* mgr/dashboard: Display RBD form errors on submission (`pr#21529 `_, Ricardo Marques) +* mgr/dashboard: Enable object rendering in KV-table (`pr#21701 `_, Stephan Müller) +* mgr/dashboard: fix 500 error on block device iSCSI status page (`pr#20928 `_, Jason Dillaman) +* mgr/dashboard: fix dashboard python 3 support (`pr#21007 `_, Ricardo Dias) +* mgr/dashboard: Fix data race and use-before-assignment (`pr#21590 `_, Sebastian Wagner) +* mgr/dashboard: fixed password generation in Auth controller (`issue#23404 `_, `pr#21006 `_, Ricardo Dias) +* mgr/dashboard: Fixes documentation link- to open in new tab (`pr#22262 `_, Kanika Murarka) +* mgr/dashboard: Fixes type error in RBD form (`pr#21681 `_, Stephan Müller) +* mgr/dashboard: fix frontend e2e tests (`pr#20943 `_, Tiago Melo) +* mgr/dashboard: fix FS status on old MDS daemons (`issue#20692 `_, `pr#16960 `_, John Spray) +* mgr/dashboard: fix linting problem (`pr#22277 `_, Tiago Melo) +* mgr/dashboard: Fix missing $event on deletion modal (`pr#21667 `_, Ricardo Marques) +* mgr/dashboard: Fix moment.js deprecation warning (`pr#22052 `_, Tiago Melo) +* mgr/dashboard: Fix objects named `default` are inaccessible (`pr#20976 `_, Sebastian Wagner) +* mgr/dashboard: Fix RBD task metadata (`pr#22152 `_, Tiago Melo) +* mgr/dashboard: Fix table without fetchData (`pr#21086 `_, Ricardo Marques) +* mgr/dashboard: Fix the data table action selector (`pr#21270 `_, Stephan Müller) +* mgr/dashboard: fix two type errors found by mypy (`pr#21774 `_, Sebastian Wagner) +* mgr/dashboard: Handle errors during deletion (`pr#22029 `_, Volker Theile) +* mgr/dashboard: Implement a RGW proxy (`pr#21258 `_, Volker Theile, Patrick Nawracay) +* mgr/dashboard: Improve background tasks style (`pr#21462 `_, Ricardo Marques) +* mgr/dashboard: improve error handling (`pr#18182 `_, Nick Erdmann) +* mgr/dashboard: Improve error panel (`pr#21978 `_, Volker Theile) +* mgr/dashboard: Improve `npm start` script (`pr#20989 `_, Ricardo Marques) +* mgr/dashboard: Improve table search (`pr#20807 `_, Stephan Müller) +* mgr/dashboard: Load the datatable content on component initialization (`pr#21595 `_, Volker Theile) +* mgr/dashboard: Navbar dropdown button does not respond for mobile browsers (`pr#21979 `_, Volker Theile) +* mgr/dashboard: Notification improvements (`pr#21350 `_, Tiago Melo) +* mgr/dashboard: pool: fix python3 dict_keys error (`pr#21636 `_, Ricardo Dias) +* mgr/dashboard: Pool listing (`pr#21353 `_, Stephan Müller) +* mgr/dashboard: rbd: add @AuthRequired to snapshots controller (`pr#21517 `_, Ricardo Dias) +* mgr/dashboard: RBD copy, RBD flatten and snapshot clone (frontend) (`pr#21526 `_, Ricardo Marques, Ricardo Dias) +* mgr/dashboard: RBD management (frontend) (`pr#21385 `_, Ricardo Marques) +* mgr/dashboard: Refactor multiple duplicates of `get_rate()` (`pr#21022 `_, Sebastian Wagner) +* mgr/dashboard: Refactor RGW backend (`pr#21855 `_, Volker Theile) +* mgr/dashboard: Rename and refactor ApiInterceptorService class (`pr#21386 `_, Volker Theile) +* mgr/dashboard: Replace font-awesome with fork-awesome (`pr#21327 `_, Lenz Grimmer) +* mgr/dashboard: restcontroller: fix detection of id args in element requests (`pr#21290 `_, Ricardo Dias) +* mgr/dashboard: RESTController improvements (`pr#21516 `_, Ricardo Dias) +* mgr/dashboard: run-tox: pass CEPH_BUILD_DIR value into tox script (`pr#21445 `_, Ricardo Dias) * mgr: dashboard: show per pool IOPS on health page (#22495) (`issue#22495 `_, `pr#19981 `_, Konstantin Shalygin) +* mgr/dashboard: Support aditional info on 'cd-view-cache' (`pr#21060 `_, Ricardo Marques) +* mgr/dashboard: TaskManager bug fixes (`pr#21240 `_, Ricardo Dias) +* mgr/dashboard: Update selected items on table refresh (`pr#21099 `_, Ricardo Marques) +* mgr/dashboard: Use Bootstrap CSS (`pr#21780 `_, Volker Theile) +* mgr/dashboard: using RoutesDispatcher as HTTP request dispatcher (`pr#21239 `_, Ricardo Dias) +* mgr/dashboard_v2: add mgr to the list of perf counters (`pr#20783 `_, Tiago Melo) +* mgr/dashboard_v2: add mocked service provider for TcmuIscsiService (`pr#20775 `_, Tiago Melo) +* mgr/dashboard_v2: Add toggle able columns (`pr#20806 `_, Stephan Müller) +* mgr/dashboard_v2: Configuration settings support (`pr#20743 `_, Ricardo Dias) +* mgr/dashboard_v2: fix and improve table details (`pr#20811 `_, Tiago Melo) +* mgr/dashboard_v2: Fix cephfs template table usage (`pr#20804 `_, Stephan Müller) +* mgr/dashboard_v2: fix cluster configuration page (`pr#20821 `_, Tiago Melo) +* mgr/dashboard_v2: Improve charts tooltips (`pr#20757 `_, Tiago Melo) +* mgr/dashboard_v2: Pool controller (`pr#20823 `_, Ricardo Dias) +* mgr/dashboard_v2: Rotate the refresh icon on load (`pr#20805 `_, Stephan Müller) * mgr: die on bind() failure (`pr#20595 `_, John Spray) * mgr: disconnect unregistered service daemon when report received (`issue#22286 `_, `pr#19261 `_, Jason Dillaman) * mgr: emit cluster log message on serve() exception (`issue#21999 `_, `pr#18672 `_, John Spray) +* mgr: Expose rgw perf counters (`pr#21269 `_, Boris Ranto) * mgr: fix "access denied" message (`pr#19518 `_, John Spray) * mgr: fix crashable DaemonStateIndex::get calls (`issue#17737 `_, `pr#17933 `_, John Spray) * mgr: fix crash in MonCommandCompletion (`issue#21157 `_, `pr#17308 `_, John Spray) +* mgr: fixes python error handling (`issue#23406 `_, `pr#21005 `_, Ricardo Dias) +* mgr: fix MSG_MGR_MAP handling (`pr#20892 `_, Gu Zhongyan) * mgr: fix "osd status" command exception if OSD not in pgmap stats (`issue#21707 `_, `pr#18173 `_, Yanhu Cao) * mgr: fix py3 support (`issue#22880 `_, `pr#20362 `_, Kefu Chai) * mgr: fix py calls for dne service perf counters (`issue#21253 `_, `pr#17605 `_, John Spray) * mgr: implement completion of osd MetadataUpdate (`issue#21159 `_, `pr#16925 `_, Yanhu Cao) * mgr: implement 'osd safe-to-destroy' and 'osd ok-to-stop' commands (`pr#16976 `_, Sage Weil) * mgr: improved module loading for error reporting etc (`issue#21999 `_, `issue#21683 `_, `issue#21502 `_, `pr#19235 `_, John Spray) +* mgr: improve reporting on unloadable modules (`issue#23358 `_, `pr#20921 `_, John Spray) * mgr: increase time resolution of Commit/Apply OSD latencies (`pr#19232 `_, Коренберг Марк) +* mgr: initialize PyModuleRegistry sooner (`issue#22918 `_, `pr#20321 `_, John Spray) * mgr: In plugins 'module' classes need not to be called "Module" anymore (`issue#17454 `_, `pr#18526 `_, Kefu Chai, bhavishyagopesh) * mgr: locking fixes (`issue#21158 `_, `pr#17309 `_, John Spray) * mgr: mgr/balancer: cast config vals to int or float (`issue#22429 `_, `pr#19493 `_, Dan van der Ster) * mgr: mgr/balancer: don't use 'foo' tags on commands (`issue#22361 `_, `pr#19482 `_, John Spray) * mgr: mgr/balancer: fix KeyError in balancer rm (`issue#22470 `_, `pr#19578 `_, Dan van der Ster) +* mgr: mgr/balancer: fix OPTIONS definition (`pr#21620 `_, John Spray) * mgr: mgr/balancer: fix upmap; default balancer module enabled (`pr#18691 `_, Sage Weil) * mgr: mgr/balancer: make crush-compat mode work (`pr#17983 `_, Sage Weil) * mgr: mgr/balancer: mgr module to automatically balance PGs across OSDs (`pr#16272 `_, Spandan Kumar Sahu, Sage Weil) @@ -1503,63 +1818,83 @@ Other Notable Changes * mgr: mgr/balancer: replace magic value of -1 for DEFAULT_CHOOSE_ARGS (`pr#20258 `_, Kefu Chai) * mgr: mgr/balancer: skip CRUSH_ITEM_NONE (`pr#18894 `_, Sage Weil) * mgr: mgr/balancer: two more fixes (`pr#20180 `_, xie xingguo) +* mgr: mgrc: free MMgrClose in handle_mgr_close (`issue#23846 `_, `pr#21626 `_, Casey Bodley) +* mgr: mgr/DaemonServer: add overrides value to 'config show' (`pr#21093 `_, Gu Zhongyan) * mgr: mgr/DaemonServer.cc: [Cleanup] Change to using get_val template function (`pr#18717 `_, Shinobu Kinjo) * mgr: mgr/DaemonServer: [Cleanup] Remove redundant code (`pr#18716 `_, Shinobu Kinjo) * mgr: mgr/dashboard: add configuration setting browser (`issue#22522 `_, `pr#20043 `_, Rubab Syed) +* mgr: mgr/dashboard: add image id to mgr rbd info instead of block_name_prefix (`pr#20884 `_, zouaiguo) * mgr: mgr/dashboard: Add monitor list (`pr#19632 `_, Rubab Syed) +* mgr: mgr/dashboard: Add RGW user and bucket lists (read-only) (`pr#20869 `_, Volker Theile) +* mgr: mgr/dashboard: add TLS (`pr#21627 `_, John Spray) +* mgr: mgr/dashboard: Add toBytes() method to FormatterService (`pr#20978 `_, Volker Theile) +* mgr: mgr/dashboard: asynchronous task support (`pr#20870 `_, Ricardo Dias) * mgr: mgr/dashboard: change raw usage chart's color depending on usage (`pr#17421 `_, Nick Erdmann) * mgr: mgr/dashboard: fix audit log loading (`pr#18848 `_, John Spray) * mgr: mgr/dashboard: Fix backend tests for newer CherryPy versions (`pr#20778 `_, Patrick Nawracay) -* mgr: mgr/dashboard: fix FS status on old MDS daemons (`issue#20692 `_, `pr#16960 `_, John Spray) * mgr: mgr/dashboard: Fix PG status coloring (`pr#19431 `_, Wido den Hollander) * mgr: mgr/dashboard: format tooltip's label as user friendly string (`pr#18769 `_, Yao Zongyou) * mgr: mgr/dashboard: handle null in format_number (`issue#21570 `_, `pr#17991 `_, John Spray) * mgr: mgr/dashboard: HTTP request logging (`pr#20797 `_, Ricardo Dias) -* mgr: mgr/dashboard: improve error handling (`pr#18182 `_, Nick Erdmann) +* mgr: mgr/dashboard: Improve auth interceptor (`pr#20847 `_, Volker Theile) * mgr: mgr/dashboard: performance counter browsers (`issue#22521 `_, `pr#19922 `_, Rubab-Syed) +* mgr: mgr/dashboard: RBD management (backend) (`pr#21360 `_, Ricardo Dias) +* mgr: mgr/dashboard: Remove unused code (`pr#21045 `_, Volker Theile) +* mgr: mgr/dashboard: Remove useless code (`pr#20958 `_, Volker Theile) * mgr: mgr/dashboard: show warnings if data is out of date or mons are down (`pr#18847 `_, John Spray) * mgr: mgr/dashboard: sort servers and OSDs in OSD list (`issue#21572 `_, `pr#17993 `_, John Spray) * mgr: mgr/dashboard: use rel="icon" for favicon (`pr#18013 `_, Kefu Chai) * mgr: mgr/dashboard v2: Add CSS class for required form fields (`pr#20747 `_, Volker Theile) -* mgr: mgr/dashboard_v2: add mocked service provider for TcmuIscsiService (`pr#20775 `_, Tiago Melo) -* mgr: mgr/dashboard_v2: Add toggle able columns (`pr#20806 `_, Stephan Müller) +* mgr: mgr/dashboard_v2: Add RBD create functionality to the Python backend (`pr#20751 `_, Tatjana Dehler) * mgr: mgr/dashboard v2: Add units to performance counters (`pr#20742 `_, Volker Theile) -* mgr: mgr/dashboard_v2: Configuration settings support (`pr#20743 `_, Ricardo Dias) * mgr: mgr/dashboard v2: Display loading indicator in datatables during first load (`pr#20744 `_, Volker Theile) * mgr: mgr/dashboard v2: Don't show details if multiple OSDs are selected (`pr#20772 `_, Volker Theile) -* mgr: mgr/dashboard_v2: fix cluster configuration page (`pr#20821 `_, Tiago Melo) +* mgr: mgr/dashboard v2: implement can_run method (`pr#20728 `_, John Spray) * mgr: mgr/dashboard_v2: Initial submission of a web-based management UI (replacement for the existing dashboard) (`pr#20103 `_, Stephan Müller, Lenz Grimmer, Tiago Melo, Ricardo Marques, Sebastian Wagner, Patrick Nawracay, Ricardo Dias, Volker Theile, Kai Wagner, Tatjana Dehler) * mgr: mgr/dashboard v2: Introduce CdTableSelection model (`pr#20746 `_, Volker Theile) -* mgr: mgr/dashboard_v2: Pool controller (`pr#20823 `_, Ricardo Dias) * mgr: mgr/dashboard_v2: Removed unused `tools.detail_route()` (`pr#20765 `_, Sebastian Wagner) -* mgr: mgr/dashboard_v2: Rotate the refresh icon on load (`pr#20805 `_, Stephan Müller) +* mgr: mgr/influx: Added Additional Stats (`pr#21424 `_, mhdo2) * mgr: mgr/influx: Add InfluxDB SSL Option (`pr#19374 `_, Tobias Gall) +* mgr: mgr/influx: Only split string on first occurence of dot (.) (`issue#23996 `_, `pr#21795 `_, Wido den Hollander) * mgr: mgr/influx: PEP-8 and other fixes to Influx module (`pr#19229 `_, Wido den Hollander) * mgr: mgr/influx: Various fixes and improvements (`pr#20187 `_, Wido den Hollander) +* mgr: mgr/influx: Various time fixes (`pr#20494 `_, Wido den Hollander) * mgr: mgr/localpool: default to 3x; allow min_size adjustment (`pr#18089 `_, Sage Weil) -* mgr: mgr/PGMap: drop REQUEST_{SLOW,STUCK} HEALTH_WARNs (`pr#19114 `_, Kefu Chai) +* mgr: mgr/MgrClient: guard send_pgstats() with lock (`issue#23370 `_, `pr#20909 `_, Kefu Chai) +* mgr: mgr/MgrClient: service registration filtered by service name instead of daemon name (`pr#21459 `_, runsisi) +* mgr: mgr/PGMap: drop REQUEST\_{SLOW,STUCK} HEALTH_WARNs (`pr#19114 `_, Kefu Chai) * mgr: mgr/prometheus: add ceph_disk_occupation series (`issue#21594 `_, `pr#18021 `_, John Spray) * mgr: mgr/prometheus: add missing 'deep' state to PG_STATES in ceph-mgr prometheus plugin (`issue#22116 `_, `pr#18890 `_, Peter Woodman) * mgr: mgr/prometheus: Fix for MDS metrics (`issue#20899 `_, `pr#17318 `_, John Spray, Jeremy H Austin) +* mgr: mgr/prometheus: fix PG state names (`pr#21288 `_, John Spray) * mgr: mgr/prometheus: Skip bogus entries (`pr#20456 `_, Boris Ranto) * mgr: mgr/prometheus: skip OSD output if missing from CRUSH devices (`pr#20644 `_, John Spray) * mgr: mgr/restful: A couple of restful fixes (`pr#18649 `_, Boris Ranto) * mgr: mgr/restful: cleaner message when not configured (`issue#21292 `_, `pr#17573 `_, John Spray) +* mgr: mgr/smart: fix python3 module loading (`pr#21047 `_, Ricardo Dias) * mgr: mgr/status: fix ceph fs status returns error (`issue#21752 `_, `pr#18233 `_, Yanhu Cao) * mgr: mgr/status: format byte quantities in base 2 multiples (`issue#21189 `_, `pr#17380 `_, John Spray) +* mgr: mgr/telemetry: Add Ceph Telemetry module to send reports back to project (`pr#21970 `_, Wido den Hollander) * mgr: mgr/zabbix: fix div by zero (`issue#21518 `_, `pr#17931 `_, John Spray) * mgr: mgr/zabbix: ignore osd with 0 kb capacity (`issue#21904 `_, `pr#18809 `_, Ilja Slepnev) * mgr: mgr/zabbix: Implement health checks (`pr#20198 `_, Wido den Hollander) -* mgr: mgr/zabbix: monitoring template improvements (`pr#19901 `_, Marc Schoechlin) +* mgr: mgr/zabbix: Send max, min and avg PGs of OSDs to Zabbix (`pr#21043 `_, Wido den Hollander) * mgr: mgr/Zabbix: Various fixes to Zabbix module (`pr#19452 `_, Wido den Hollander) +* mgr: mimic: mgr/telegraf: Telegraf module for Ceph Mgr (`pr#22013 `_, Wido den Hollander) * mgr: Modify mgr-influx module database check to not require admin privileges (`pr#18102 `_, Benjeman Meekhof) +* mgr: mon,mgr: improve 'mgr module disable' cmd (`pr#21188 `_, Gu Zhongyan) +* mgr: mon, mgr: move "osd pool stats" command to mgr and mgr python module (`pr#19985 `_, Chang Liu) * mgr: mon/MgrStatMonitor: fix formatting of pending_digest (`issue#22991 `_, `pr#20426 `_, Patrick Donnelly) * mgr,mon: mon/MgrMonitor: read cmd descs if empty on update_from_paxos() (`issue#21300 `_, `pr#17846 `_, Joao Eduardo Luis) +* mgr,mon: mon,mgr: remove single wildcard '\*' from ceph comand line description (`pr#21139 `_, Gu Zhongyan) * mgr,mon: mon/mgr: sync "mgr_command_descs","osd_metadata" and "mgr_metadata" prefixes to new mons (`issue#21527 `_, `pr#17929 `_, huanwen ren) +* mgr,mon: mon/MonCommands: mgr metadata - improve parameter naming consistency (`issue#23330 `_, `pr#20866 `_, Jan Fajerski) * mgr: preventing blank hostname in DaemonState (`issue#20887 `_, `issue#21060 `_, `pr#17138 `_, liuchang0812) * mgr: prometheus: added osd commit/apply latency metrics (#22718) (`issue#22718 `_, `pr#19980 `_, Konstantin Shalygin) * mgr: prometheus: Don't crash on OSDs without metadata (`pr#20539 `_, Christopher Blum) +* mgr: prometheus fix metadata labels (`pr#21557 `_, Jan Fajerski) * mgr: prometheus: set metadata metrics value to '1' (#22717) (`issue#22717 `_, `pr#19979 `_, Konstantin Shalygin) +* mgr: pybind/mgr/balancer: add sanity check against empty adjusted_map (`pr#20836 `_, xie xingguo) * mgr: pybind/mgr/balancer: fix pool-deletion vs auto-optimization race (`pr#20706 `_, xie xingguo) * mgr: pybind/mgr/balancer: fix sanity check against empty weight-set (`pr#20278 `_, xie xingguo) * mgr: pybind/mgr/balancer: increase bad_steps properly (`pr#20194 `_, xie xingguo) @@ -1571,52 +1906,70 @@ Other Notable Changes * mgr: pybind/mgr/dashboard: fix duplicated slash in html href (`issue#22851 `_, `pr#20229 `_, Shengjing Zhu) * mgr,pybind: mgr/dashboard: fix pool size base conversion (`pr#16771 `_, Yixing Yan) * mgr: pybind/mgr/dashboard: fix reverse proxy support (`issue#22557 `_, `pr#19758 `_, Nick Erdmann) +* mgr,pybind: mgr/iostat: print output as a table (`pr#21338 `_, Mohamad Gebai) * mgr: pybind/mgr/localpool: module to automagically create localized pools (`pr#17528 `_, Sage Weil) * mgr: pybind/mgr/mgr_module: add default param for MgrStandbyModule.get_con… (`pr#19948 `_, Kefu Chai) * mgr: pybind/mgr/mgr_module: make rados handle available to all modules (`pr#19972 `_, Sage Weil) -* mgr: pybind/mgr_module: move PRIO_* and PERFCOUNTER_* to MgrModule class (`pr#18251 `_, Jan Fajerski) +* mgr: pybind/mgr_module: move PRIO\_\* and PERFCOUNTER\_\* to MgrModule class (`pr#18251 `_, Jan Fajerski) +* mgr: pybind/mgr: new 'hello world' mgr module skeleton (`pr#19491 `_, Yaarit Hatuka) +* mgr: pybind/mgr/prometheus: add file_sd_config command (`pr#21061 `_, Jan Fajerski) * mgr: pybind/mgr/prometheus: add osd_in/out metric; make osd_weight a metric (`pr#18243 `_, Jan Fajerski) * mgr: pybind/mgr/prometheus: add StandbyModule and handle failed MON cluster (`pr#19744 `_, Jan Fajerski) * mgr: pybind/mgr/prometheus: don't crash when encountering an unknown PG state (`pr#18903 `_, Jan Fajerski) * mgr: pybind/mgr/prometheus: don't export metrics for dead daemon; new metrics (`pr#20506 `_, Jan Fajerski) +* mgr: pybind/mgr/prometheus: fix creation of osd_metadata metric (`pr#21530 `_, Jan Fajerski) * mgr: pybind/mgr/prometheus: fix metric type undef -> untyped (`issue#22313 `_, `pr#19524 `_, Ilya Margolin) * mgr: pybind/mgr/prometheus: fix metric type undef -> untyped (`pr#18208 `_, Jan Fajerski) +* mgr,pybind: pybing/mgr/prometheus: return default port if config-key get returns … (`pr#21696 `_, Jan Fajerski) * mgr: python interface rework + enable modules to run in standby mode (`issue#21593 `_, `issue#17460 `_, `pr#16651 `_, John Spray, Sage Weil) * mgr: quieten logging on missing OSD stats (`pr#20485 `_, John Spray) * mgr,rbd: mgr/dashboard: added iSCSI IOPS/throughput metrics (`issue#21391 `_, `pr#18653 `_, Jason Dillaman) * mgr,rbd: mgr/dashboard: fix duplicate images listed on iSCSI status page (`issue#21017 `_, `pr#17055 `_, Jason Dillaman) +* mgr: reconcile can_run checks and selftest (`pr#21607 `_, John Spray, Kefu Chai) * mgr: remove a few junk lines (`pr#20005 `_, John Spray) * mgr: remove unused static files from dashboard module (`pr#16762 `_, John Spray) * mgr: request daemon's metadata when receiving a report from an unknown server (`issue#21687 `_, `pr#18484 `_, Chang Liu) * mgr,rgw: mgr/dashboard: RGW page (`pr#19512 `_, Chang Liu) +* mgr,rgw: prometheus: Implement rgw_metadata metric (`pr#21383 `_, Boris Ranto) * mgr: safety checks on pyThreadState usage (`pr#18093 `_, John Spray) * mgr: set explicit thread name (`issue#21404 `_, `pr#17756 `_, John Spray) * mgr: silence warning from -Wsign-compare (`pr#17881 `_, Jos Collin) * mgr: skip first non-zero incremental in PGMap::apply_incremental() (`issue#21773 `_, `pr#18347 `_, Aleksei Gutikov) +* mgr/status: output to stdout, not stderr (`issue#24175 `_, `pr#22135 `_, John Spray) * mgr: store declared_types in MgrSession (`issue#21197 `_, `pr#17932 `_, John Spray) * mgr: systemd: Wait 10 seconds before restarting ceph-mgr (`issue#23083 `_, `pr#20533 `_, Wido den Hollander) +* mgr,tests: mgr/dashboard: skip data pool testcase for none-bluestore clusters (`pr#21004 `_, Tatjana Dehler) +* mgr,tests: mgr/dashboard_v2: fix test_perf_counters_mgr_get (`pr#20916 `_, Tiago Melo) * mgr,tests: qa: add new prometheus test to rados/mgr suite (`pr#20047 `_, John Spray) * mgr,tests: qa: configure zabbix properly before selftest (`issue#22514 `_, `pr#19634 `_, John Spray) * mgr,tests: qa: fix mgr _load_module helper (`pr#18685 `_, John Spray) +* mgr,tools: mgr/iostat: implement 'ceph iostat' as a mgr plugin (`pr#20100 `_, Mohamad Gebai) * mgr: use new style config opts + add metadata (`pr#17374 `_, John Spray) +* mgr/zabbix: Fix wrong log message (`pr#21237 `_, Gu Zhongyan) +* mgr/zabbix: monitoring template improvements (`pr#19901 `_, Marc Schoechlin) * mon: Add `ceph osd get-require-min-compat-client` command (`pr#19015 `_, hansbogert) +* mon: add 'ceph osd pool get erasure allow_ec_overwrites' command (`pr#21102 `_, Mykola Golub) * mon: add MMonHealth back (`issue#22462 `_, `pr#20528 `_, Kefu Chai) * mon: add mon_health_preluminous_compat_warning (`pr#16902 `_, Sage Weil) * mon: a few conversions to monotonic clock (`pr#18595 `_, Patrick Donnelly) * mon: align lspools output (`pr#19597 `_, Jos Collin) * mon: allow cluter and debug logs to go to stderr, with appropriate prefix (`pr#19385 `_, Sage Weil) +* mon: cache reencoded osdmaps (`issue#23713 `_, `pr#21605 `_, Sage Weil, Xiaoxi CHEN) * mon: centralized config (`pr#20172 `_, Sage Weil) * mon: "ceph osd crush rule rename" support (`pr#17029 `_, xie xingguo) * mon: check monitor address configuration (`pr#18073 `_, Li Wang) * mon: clean up cluster logging on mon events (`issue#22082 `_, `pr#18822 `_, John Spray) +* mon: cleanups to optracker code (`pr#21371 `_, John Spray) * mon: cleanup unused option mon_health_data_update_interval (`pr#17728 `_, Yao Guotao) * mon: common/options: set max_background_jobs instead of max_background_compactions (`pr#18397 `_, Kefu Chai) * mon: Compress the warnings of pgs not scrubbed or deep-scrubbed (`pr#17295 `_, Zhi Zhang) * mon: do not use per_pool_sum_delta to show recovery summary (`issue#22727 `_, `pr#20009 `_, Chang Liu) * mon: don't blow away bootstrap-mgr on upgrades (`issue#20950 `_, `pr#18399 `_, John Spray) +* mon: double mon_mgr_mkfs_grace from 60s -> 120s (`pr#20955 `_, Sage Weil) * mon: Drop redundant access specifier, etc (cleanup) (`pr#19028 `_, Shinobu Kinjo) * mon: dump percent_used PGMap field as float (`pr#20439 `_, John Spray) * mon: dump servicemap along with MgrStatMonitor dump info (`pr#18760 `_, Zhi Zhang) +* mon: expand cap validity check for mgr, osd, mds (`issue#22525 `_, `pr#21311 `_, Jing Li, Sage Weil) * mon: final luminous compatset feature and osdmap flag (`pr#17333 `_, Sage Weil) * mon: fix commands advertised during mon cluster upgrade (`pr#16871 `_, Sage Weil) * mon: fix dropping mgr metadata for active mgr (#21260) (`issue#21260 `_, `pr#17571 `_, John Spray) @@ -1624,17 +1977,25 @@ Other Notable Changes * mon: fix legacy health checks in 'ceph status' during upgrade; fix jewel-x upgrade combo (`pr#16967 `_, Sage Weil) * mon: fix mgr using auth_client_required policy (`pr#20048 `_, John Spray) * mon: fix `osd out` clog message (`issue#21249 `_, `pr#17525 `_, John Spray) +* mon: fix slow op warning on mon, improve slow op warnings (`issue#23769 `_, `pr#21684 `_, Sage Weil) * mon: fix structure of 'features' command (`pr#20115 `_, Sage Weil) * mon: fix two stray legacy get_health() callers (`pr#17269 `_, Sage Weil) * mon: fix wrong mon-num counting logic of 'ceph features' command (`pr#16887 `_, xie xingguo) +* mon: handle bad snapshot removal reqs gracefully (`issue#18746 `_, `pr#20835 `_, Paul Emmerich) * mon: handle monitor lag when killing mgrs (`issue#20629 `_, `pr#18268 `_, John Spray) * mon: incorrect MAX AVAIL in "ceph df" (`issue#21243 `_, `pr#17513 `_, liuchang0812) * mon: invalid JSON returned when querying pool parameters (`issue#23200 `_, `pr#20745 `_, Chang Liu) +* mon/LogMonitor: call no_reply() on ignored log message (`pr#22104 `_, Sage Weil) +* mon: mark mgr reports as no_reply (`issue#22114 `_, `pr#21057 `_, Kefu Chai) * mon: mark mon_allow_pool_delete as observed (`pr#18125 `_, Dan van der Ster) * mon: mark OSD beacons and pg_create messages as no_reply (`issue#22114 `_, `pr#20517 `_, Greg Farnum) +* mon: mon/AuthMonitor: don't validate `fs authorize` caps with `valid_caps()` (`pr#21418 `_, Joao Eduardo Luis) +* mon: mon/ConfigMonitor: clean up prepare_command() (`pr#20911 `_, Gu Zhongyan) * mon: mon/Elector: force election epoch bump on start (`issue#20949 `_, `pr#16944 `_, Sage Weil) +* mon: mon/Elector: remove unused member fields start_stamp and ack_stamp (`pr#21091 `_, runsisi) * mon: mon/LogMonitor: "log last" should return up to n entries (`pr#18759 `_, Kefu Chai) * mon: mon/MDSMonitor: fix clang build failure (`pr#20637 `_, Willem Jan Withagen) +* mon: mon,mgr: make osd_metric more popular and report slow ops to mgr (`issue#23045 `_, `pr#20660 `_, lvshanchun) * mon: mon/MgrMonitor: limit mgrmap history (`issue#22257 `_, `pr#19185 `_, Sage Weil) * mon: mon/MonCommands: fix copy-and-paste error (`pr#17271 `_, xie xingguo) * mon: mon,option: set default value for mon_dns_srv_name (`issue#21204 `_, `pr#17539 `_, Kefu Chai) @@ -1646,31 +2007,37 @@ Other Notable Changes * mon: mon/OSDMonitor.cc: fix expected_num_objects interpret error (`issue#22530 `_, `pr#19651 `_, Yang Honggang) * mon: mon/OSDMonitor.cc : set erasure-code-profile to "" when create replicated pools (`pr#19673 `_, zouaiguo) * mon: mon/OSDMonitor: check last_scan_epoch instead when sending creates (`issue#20785 `_, `pr#17248 `_, Kefu Chai) -* mon: mon/OSDMonitor: cleanup: move bufferlist before use (`pr#18258 `_, Shinobu Kinjo) +* mon: mon/OSDMonitor: clean up cmd 'osd tree-from' (`pr#20839 `_, Gu Zhongyan) * mon: mon/OSDMonitor: do not send_pg_creates with stale info (`issue#20785 `_, `pr#17065 `_, Kefu Chai) * mon: mon/OSDMonitor: error out if setting ruleset-\* ec profile property (`pr#17848 `_, Sage Weil) * mon: mon/OSDMonitor: fix improper input/testing range of crush somke testing (`pr#17179 `_, xie xingguo) * mon: mon/OSDMonitor: fix 'osd pg temp' unable to cleanup pg-temp (`pr#16892 `_, xie xingguo) * mon: mon/OSDMonitor: implement 'osd crush ls ' (`pr#16920 `_, Sage Weil) * mon: mon/OSDMonitor: kill pending upmap changes too if pool is gone (`pr#20704 `_, xie xingguo) +* mon: mon/OSDMonitor: logging non-active osd id when handling osd beacon (`pr#21092 `_, runsisi) * mon: mon/OSDMonitor: make 'osd crush rule rename' idempotent (`issue#21162 `_, `pr#17329 `_, xie xingguo) * mon: mon/OSDMonitor: "osd pool application get" support (`issue#20976 `_, `pr#16955 `_, xie xingguo) -* mon: mon/OSDMonitor: use new style options (`pr#18209 `_, Kefu Chai) +* mon: mon/OSDMonitor: txsize should be greater or eq to prune_interval - 1 (`pr#21430 `_, Kefu Chai) * mon: mon/PGMap: drop DISK LOG column (`pr#17617 `_, Sage Weil) * mon: mon/PGMap: fix "0 stuck requests are blocked > 4096 sec" warn (`pr#17099 `_, xie xingguo) -* mon: mon/PGMap: let pg_string_state() return boost::optional<> (`issue#21609 `_, `pr#18218 `_, Kefu Chai) * mon: mon/PGMap: nice numbers for 'data' section of 'ceph df' command (`pr#17368 `_, xie xingguo) * mon: mon/PGMap: Remove unnecessary header (`pr#18343 `_, Shinobu Kinjo) * mon: mon/PGMap: reweight::by_utilization - skip DNE osds (`issue#20970 `_, `pr#17064 `_, xie xingguo) * mon: mon/pgmap: update pool nearfull display (`pr#17043 `_, huanwen ren) -* mon: mon/PGMap: use new-style options and cleanup (`pr#18647 `_, Kefu Chai) * mon: more aggressively convert crush rulesets -> distinct rules (`pr#17508 `_, John Spray, Sage Weil) * mon: more constness (`pr#17748 `_, Kefu Chai) +* mon: node ls improvement (`pr#20820 `_, Gu Zhongyan) * mon: 'node ls' mgr support (`pr#20711 `_, Gu Zhongyan) * mon: NULL check of logger before use (`pr#18788 `_, Amit Kumar) +* mon,osd: dump "compression_algorithms" in "mon metadata" (`issue#24135 `_, `issue#22420 `_, `pr#22004 `_, Kefu Chai, Casey Bodley) * mon: osd feature checks with 0 up osds (`issue#21471 `_, `issue#20751 `_, `pr#17831 `_, Brad Hubbard, Sage Weil) +* mon: osdmap prune (`pr#19331 `_, Joao Eduardo Luis) +* mon/OSDMonitor: cleanup: move bufferlist before use (`pr#18258 `_, Shinobu Kinjo) +* mon/OSDMonitor: use new style options (`pr#18209 `_, Kefu Chai) * mon: osd/OSDMap.h: toss osd out if it has no more pending states (`pr#19642 `_, xie xingguo) * mon: paxos cleanup (`pr#20078 `_, huanwen ren) +* mon/PGMap: let pg_string_state() return boost::optional<> (`issue#21609 `_, `pr#18218 `_, Kefu Chai) +* mon/PGMap: use new-style options and cleanup (`pr#18647 `_, Kefu Chai) * mon: post-luminous cleanup (part 3 of ?) (`pr#17607 `_, Sage Weil) * mon: rate limit on health check update logging (`issue#20888 `_, `pr#16942 `_, John Spray) * mon: reenable timer to send digest when paxos is temporarily inactive (`issue#22142 `_, `pr#19404 `_, Jan Fajerski) @@ -1682,75 +2049,105 @@ Other Notable Changes * mon: revert mds metadata argument name change (`issue#22527 `_, `pr#19926 `_, Patrick Donnelly) * mon: show feature flags when printing MonSession (`pr#17535 `_, Paul Emmerich) * mon: some cleanup (`pr#17067 `_, huanwen ren) +* mon,tests: vstart: set osd_pool_default_erasure_code_profile in initial ceph.conf (`pr#21008 `_, Mykola Golub) * mon: update get_store_prefixes implementations (`issue#21534 `_, `pr#17940 `_, John Spray, huanwen ren) * mon: update PaxosService::cached_first_committed in PaxosService::maybe_trim() (`issue#11332 `_, `pr#19397 `_, Xuehan Xu, yupeng chen) * mon: use ceph_clock_now if message is self-generated (`pr#17311 `_, huangjun) * mon: warn about using osd new instead of osd create (`issue#21023 `_, `pr#17242 `_, Neha Ojha) +* msg/async/AsyncConnection: remove legacy feature case handle (`pr#18469 `_, Haomai Wang) +* msg/async: avoid referencing the temporary string (`pr#20640 `_, Kefu Chai) +* msg/async: batch handle msg_iovlen (`pr#18415 `_, Jianpeng Ma) +* msg/async/dpdk: remove xsky copyright and LGPL copying (`pr#21121 `_, Kefu Chai) +* msg/async/EventKqueue: assert on OOM (`pr#21488 `_, Kefu Chai) +* msg/async: fix ms_dpdk_coremask and ms_dpdk_coremask conflict (`pr#18678 `_, chunmei) +* msg/async:fix the incoming parameter type of EventCenter::process_events() (`pr#20607 `_, shangfufei) +* msg/async misc cleanup (`pr#18531 `_, Jianpeng Ma) +* msg/async: misc cleanup (`pr#18575 `_, Jianpeng Ma) +* msg/async/rdma: a tiny typo fix (`pr#18660 `_, Yan Lei) +* msg/async/rdma: fix a coredump introduced by PR #18053 (`pr#18204 `_, Yan Lei) +* msg/async/rdma: fix a potential coredump when handling tx_buffers under heavy RDMA (`pr#18036 `_, Yan Lei) +* msg/async/rdma: fixes crash for multi rados client within one process (`pr#16981 `_, Alex Mikheev, Haomai Wang, Adir Lev) +* msg/async/rdma: fix Tx buffer leakage that can introduce "heartbeat no reply" (`pr#18053 `_, Yan Lei) +* msg/async/rdma: refactor rx buffer pool allocator (`pr#17018 `_, Alex Mikheev) +* msg/async/rdma: unnecessary reinitiliazation of an iterator (`pr#18190 `_, JustL) +* msg/async: size of EventCenter::file_events should be greater than fd (`issue#23253 `_, `pr#20764 `_, Yupeng Chen) +* msg/async: use bitset<> to do the popcnt (`pr#18681 `_, Kefu Chai) +* msg/async: use device before checking (`pr#19738 `_, Xiaoyan Li) * msg: drop duplicate include (`pr#19623 `_, /bin/bash) * msg: drop the unnecessary polling_stop() (`pr#17079 `_, Jos Collin) * msg: Initialize lkey,bound,port_cnt,num_chunk,gid_idx (`pr#17797 `_, Amit Kumar) * msg: Initializing class members in module msg (`pr#17568 `_, Amit Kumar) -* msg: messages: Initialization of uninitialized members various classes (`pr#16848 `_, amitkuma) -* msg: messages/MDentryLink: add const to member function (`pr#15479 `_, yonghengdexin735) -* msg: messages,test,msg: initialize h,reply_type,owner (`pr#17767 `_, Amit Kumar) -* msg: msg/async/AsyncConnection: remove legacy feature case handle (`pr#18469 `_, Haomai Wang) -* msg: msg/async: avoid referencing the temporary string (`pr#20640 `_, Kefu Chai) -* msg: msg/async: batch handle msg_iovlen (`pr#18415 `_, Jianpeng Ma) -* msg: msg/async: fix ms_dpdk_coremask and ms_dpdk_coremask conflict (`pr#18678 `_, chunmei) -* msg: msg/async:fix the incoming parameter type of EventCenter::process_events() (`pr#20607 `_, shangfufei) -* msg: msg/async misc cleanup (`pr#18531 `_, Jianpeng Ma) -* msg: msg/async: misc cleanup (`pr#18575 `_, Jianpeng Ma) -* msg: msg/async/rdma: a tiny typo fix (`pr#18660 `_, Yan Lei) -* msg: msg/async/rdma: fixes crash for multi rados client within one process (`pr#16981 `_, Alex Mikheev, Haomai Wang, Adir Lev) -* msg: msg/async: size of EventCenter::file_events should be greater than fd (`issue#23253 `_, `pr#20764 `_, Yupeng Chen) -* msg: msg/async: use bitset<> to do the popcnt (`pr#18681 `_, Kefu Chai) -* msg: msg/async: use device before checking (`pr#19738 `_, Xiaoyan Li) -* msg: msg/simple: pass a char for reading from shutdown_rd_fd (`pr#19094 `_, Kefu Chai) * msg: reimplement sigpipe blocking (`pr#18105 `_, Greg Farnum) * msg: remove the ),it's redundant (`pr#17544 `_, linxuhua) * msg: resurrect support for !CEPH_FEATURE_MSG_AUTH (`pr#19044 `_, Ilya Dryomov) +* msgr: Optimization for connection establishment (`pr#16006 `_, shangfufei) +* msg/simple: pass a char for reading from shutdown_rd_fd (`pr#19094 `_, Kefu Chai) +* NVMDevice: fix issued caused by #17002 (`pr#17112 `_, Ziye Yang) +* objclass-sdk: expose __cls_init() to the world (`pr#21581 `_, Kefu Chai) * objecter: minor cleanups (`pr#19994 `_, runsisi) +* os/bluestore/bluestore_tool: Move redundant code into one method (`pr#19160 `_, Shinobu Kinjo) +* os/bluestore: implement BlueRocksEnv::AreFilesSame() (`issue#21842 `_, `pr#18392 `_, Kefu Chai) +* os/bluestore: simplify and fix SharedBlob::put() (`issue#24211 `_, `pr#22170 `_, Sage Weil) * osd: additional protection for out-of-bounds EC reads (`issue#21629 `_, `pr#18088 `_, Jason Dillaman) * osd: add multiple objecter finishers (`pr#16521 `_, Myoungwon Oh) +* osd: add num_object_manifest (`pr#20690 `_, Myoungwon Oh) * osd: add numpg_removing metric (`pr#18450 `_, Sage Weil) * osd: add processed_subop_count for cls_cxx_subop_version() (`issue#21964 `_, `pr#18610 `_, Casey Bodley) * osd: add scrub week day constraint (`pr#18368 `_, kungf) * osd: adjust osd_min_pg_log_entries (`issue#21026 `_, `pr#17075 `_, J. Eric Ivancich) * osd: allow FULL_TRY after failsafe (`pr#17177 `_, Pan Liu) * osd: allow PG recovery scheduling preemption (`pr#17839 `_, Sage Weil) +* osd: async recovery (`pr#19811 `_, Neha Ojha) * osd: avoid encoding the same log entries repeatedly for different peers (`pr#20201 `_, Jianpeng Ma) * osd: avoid the config's get_val() overhead on the read path (`pr#20217 `_, Radoslaw Zarzynski) +* osd: avoid unnecessary ref-counting across PrimaryLogPG::get_rw_locks (`pr#21028 `_, Radoslaw Zarzynski) * osd: be more precise about our asserts and cases when rebuilding missing sets (`issue#20985 `_, `pr#17000 `_, Greg Farnum) * osd: bring in dmclock library changes (`pr#16755 `_, J. Eric Ivancich) * osd: bring in latest dmclock library updates (`pr#17997 `_, J. Eric Ivancich) +* osd: cap snaptrimq_len at 2^32 (`pr#21107 `_, Kefu Chai) * osd: change log level when withholding pg creation (`issue#22440 `_, `pr#20167 `_, Dan van der Ster) * osd: change op delayed state to 'waiting for scrub' (`pr#19295 `_, kungf) +* osd: Change shard digests to hex like object info digests (`pr#21362 `_, David Zafman) * osd: change the conditional in _update_calc_stats (`pr#13383 `_, Zhiqiang Wang) * osd: check feature bits when encoding objectstore_perf_stat_t (`pr#20378 `_, Kefu Chai) * osd: clean up dup index logic; maintain index flag logic in fewer places (`pr#16829 `_, J. Eric Ivancich) * osd: clean up pre-luminous compat cruft (part 1 of many) (`pr#17247 `_, Sage Weil) * osd: cleanups (`pr#17753 `_, Kefu Chai) +* osdc/Objecter: using coarse_mono instead (`pr#18473 `_, Haomai Wang) +* osdc/Objector: use std::shared_mutex instead of boost::shared_mutex (`issue#23910 `_, `pr#21702 `_, Abhishek Lekshmanan) +* osd: correct several spell errors in comments (`pr#21064 `_, songweibin) +* osdc: Remove a bit too redundant public label (`pr#19466 `_, Shinobu Kinjo) +* osdc: self-managed snapshot helper should catch decode exception (`issue#24103 `_, `issue#24000 `_, `pr#21958 `_, Jason Dillaman) * osd: debug dispatch_context cases where queries not sent (`pr#20917 `_, Sage Weil) * osd: Deleting dead code PrimaryLogPG.cc (`pr#17339 `_, Amit Kumar) +* osd: don't crash on empty snapset (`issue#23851 `_, `pr#21058 `_, Mykola Golub, Igor Fedotov) * osd: Don't include same header twice (`pr#18319 `_, Shinobu Kinjo) * osd: Don't initialze pointers by NULL or 0 (`pr#18311 `_, Shinobu Kinjo) +* osd: don't memcpy hobject_t in PrimaryLogPG::close_op_ctx() (`pr#21029 `_, Radoslaw Zarzynski) +* osd: don't process ostream strings when not debugging (`pr#20298 `_, Mark Nelson) * osd: drop redundant comment (`pr#20347 `_, songweibin) * osd: Drop the unused code in OSD::_collect_metadata (`pr#17131 `_, Luo Kexue) +* osd: drop unused osd_disk_tp related options (`pr#21339 `_, Gu Zhongyan) * osd: eliminate ineffective container operations (`pr#19099 `_, Igor Fedotov) * osd: enumerate device names in a simple way (`pr#18453 `_, Sage Weil) * osd: exit(1) directly without lock if init fails (`pr#16647 `_, Kefu Chai) +* osd: fast dispatch of peering events and pg_map + osd sharded wq refactor (`pr#19973 `_, Sage Weil) * osd: fine-grained statistics of logical object space usage (`pr#15199 `_, xie xingguo) * osd: Fix assert when checking missing version (`issue#21218 `_, `pr#20410 `_, David Zafman) * osd: fix a valgrind issue (conditional jump depends on uninitialized value) (`issue#22641 `_, `pr#19874 `_, Myoungwon Oh) +* osd: fix bug which cause can't erase OSDShardPGSlot (`pr#21771 `_, Jianpeng Ma) * osd: fix build_initial_pg_history (`issue#21203 `_, `pr#17423 `_, w11979, Sage Weil) +* osd: fix crash caused by divide by zero in heartbeat code (`pr#21373 `_, Piotr Dałek) * osd: fix dpdk memzon mz_name setting issue (`pr#19809 `_, chunmei Liu) * osd: fix dpdk runtime issue based on spdk/dpdk libarary (`pr#19559 `_, chunmei Liu) * osd: fix dpdk worker references issue (`pr#19886 `_, chunmei Liu) * osd: Fixes for osd_scrub_during_recovery handling (`issue#18206 `_, `pr#17039 `_, David Zafman) * osd: fix out of order caused by letting old msg from down osd be processed (`issue#22570 `_, `pr#19796 `_, Mingxin Liu) +* osd: fix _process handling for pg vs slot race (`pr#21745 `_, Sage Weil) * osd: fix recovery reservation bugs, and implement remote reservation preemption (`pr#18485 `_, Sage Weil) * osd: fix replica/backfill target handling of REJECT (`issue#21613 `_, `pr#18070 `_, Sage Weil) * osd: fix reqid assignment for reply messages in OpRequest (`pr#17060 `_, Yingxin Cheng) +* osd: fix s390x build failure (`issue#23238 `_, `pr#20969 `_, Nathan Cutler) * osd: fix typos and some cleanups (`pr#19211 `_, Enming Zhang) * osd: fix unordered read bug (for chunked object) (`issue#22369 `_, `pr#19464 `_, Myoungwon Oh) * osd: fix waiting_for_peered vs flushing (`issue#21407 `_, `pr#17759 `_, Sage Weil) @@ -1759,6 +2156,7 @@ Other Notable Changes * osd: get loadavg per cpu for scrub load threshold check (`pr#17718 `_, kungf) * osd: get rid off extent map in object_info (`pr#19616 `_, Igor Fedotov) * osd: hold lock while accessing recovery_needs_sleep (`issue#21566 `_, `pr#18022 `_, Neha Ojha) +* osd: Improve recovery stat handling by using peer_missing and missing_loc info (`issue#22837 `_, `pr#20220 `_, Sage Weil, David Zafman) * osd: Improve size scrub error handling and ignore system attrs in xattr checking (`issue#20243 `_, `issue#18836 `_, `pr#16407 `_, David Zafman) * osd: include front_iface+back_iface in metadata (`issue#20956 `_, `pr#16941 `_, John Spray) * osd: Initialization of data members (`pr#17691 `_, Amit Kumar) @@ -1769,27 +2167,39 @@ Other Notable Changes * osd: kill snapdirs (`pr#17579 `_, Sage Weil) * osd: Make dmclock's anticipation timeout be configurable (`pr#18827 `_, Taewoong Kim) * osd: make operations on ReplicatedBackend::in_progress_ops more effective (`pr#19035 `_, Igor Fedotov) +* osd: make PG::\*Force\* event structs public (`pr#21312 `_, Willem Jan Withagen) * osd: make scrub no deadline when max interval is zero (`pr#18354 `_, kungf) * osd: make scrub right now when pg stats_invalid is true (`pr#17884 `_, kungf) * osd: make scrub wait for ec read/modify/writes to apply (`issue#23339 `_, `pr#20944 `_, Sage Weil) +* osd: make snapmapper warn+clean up instead of assert (`issue#22752 `_, `pr#20040 `_, Sage Weil) * osd: make stat_bytes and stat_bytes_used counters PRIO_USEFUL (`issue#21981 `_, `pr#18637 `_, Yao Zongyou) * osd: make the PG's SORTBITWISE assert a more generous shutdown (`issue#20416 `_, `pr#18047 `_, Greg Farnum) * osd: Making use of find to reduce computational complexity (`pr#19732 `_, Shinobu Kinjo) -* osd: migrate PGLOG macros to constexpr (`issue#20811 `_, `pr#19352 `_, Jesse Williamson) +* osd: migrate PGLOG\_\* macros to constexpr (`issue#20811 `_, `pr#19352 `_, Jesse Williamson) * osd: minor optimizations for op wq (`pr#17704 `_, Sage Weil, J. Eric Ivancich) * osd: min_pg_log_entries == max == pg_log_dups_tracked (`pr#20394 `_, Sage Weil) * osd: misc cleanups (`pr#17430 `_, songweibin) +* osd: miscellaneous cleanups (`pr#21431 `_, songweibin) * osd: more debugging for snapmapper bug (`issue#21557 `_, `pr#19366 `_, Sage Weil) * osd: object added to missing set for backfill, but is not in recovering, error! (`issue#18162 `_, `pr#18145 `_, David Zafman) * osd: only exit if \*latest\* map(s) say we are destroyed (`issue#22673 `_, `pr#19988 `_, Sage Weil) * osd: Only scan for omap corruption once (`issue#21328 `_, `pr#17705 `_, David Zafman) +* osd,os,io: Initializing C_ProxyChunkRead members,queue,request (`pr#19336 `_, amitkuma) * osd: pass ops_blocked_by_scrub() to requeue_scrub() (`pr#20319 `_, Kefu Chai) * osd: pass pool options to ObjectStore on pg create (`issue#22419 `_, `pr#19486 `_, Sage Weil) +* osd/PG: fix clang build vs private state events (`pr#18217 `_, Sage Weil) +* osd/PG: handle flushed event directly (`pr#19441 `_, wumingqiao) +* osd/PrimaryLogPG: derr when object size becomes over osd_max_object_size (`pr#19049 `_, Shinobu Kinjo) +* osd: process _scan_snaps() with all snapshots with head (`issue#22881 `_, `issue#23909 `_, `pr#21546 `_, David Zafman) +* osd: publish osdmap to OSDService before starting wq threads (`issue#21977 `_, `pr#21623 `_, Sage Weil) * osd: pull latest dmclock subtree (`pr#19345 `_, J. Eric Ivancich) * osd: put peering events in main sharded wq (`pr#18752 `_, Sage Weil) * osd: put pg removal in op_wq (`pr#19433 `_, Sage Weil) * osd: reduce all_info map find to get primary (`pr#19425 `_, kungf) +* osd: refcount for manifest object (redirect, chunked) (`pr#19935 `_, Myoungwon Oh) +* osd: remove cost from mclock op queues; cost not handled well in dmclock (`pr#21428 `_, J. Eric Ivancich) * osd: Remove double space (`pr#19296 `_, Shinobu Kinjo) +* osd: remove duplicated "commit_queued_for_journal_write" in OpTracker (`issue#23440 `_, `pr#21018 `_, ashitakasam) * osd: remove duplicated function ec_pool in pg_pool_t (`pr#18059 `_, Chang Liu) * osd: Remove redundant local variable declaration (`pr#19812 `_, Shinobu Kinjo) * osd: Remove unnecessary headers (`pr#19735 `_, Shinobu Kinjo) @@ -1798,6 +2208,9 @@ Other Notable Changes * osd: replace mclock subop opclass w/ rep_op opclass; combine duplicated code (`pr#18194 `_, J. Eric Ivancich) * osd: replace vectors_equal() with operator==(vector<>, vector<>) (`pr#18064 `_, Kefu Chai) * osd: request new map from PG when needed (`issue#21428 `_, `pr#17795 `_, Josh Durgin) +* osd: resend osd_pgtemp if it's not acked (`issue#23610 `_, `pr#21310 `_, Kefu Chai) +* osd: Revert use of dmclock message feature bit since not yet finalized (`pr#21398 `_, J. Eric Ivancich) +* osd,rgw,librbd: SCA fixes (`pr#18495 `_, Danny Al-Gaaf) * osd: set min_version to newest version in maybe_force_recovery (`pr#17752 `_, Xinze Chi) * osd: Sign in early SIGHUP signal (`issue#22746 `_, `pr#19958 `_, huanwen ren) * osd: silence maybe-uninitialized false positives (`pr#19820 `_, Yao Zongyou) @@ -1808,6 +2221,8 @@ Other Notable Changes * osd: subscribe osdmaps if any pending pgs (`issue#22113 `_, `pr#18916 `_, Kefu Chai) * osd: subscribe to new osdmap while waiting_for_healthy (`issue#21121 `_, `pr#17244 `_, Sage Weil) * osd: support class method whitelisting within caps (`pr#19786 `_, Jason Dillaman) +* osd: treat successful and erroroneous writes the same for log trimming (`issue#22050 `_, `pr#20827 `_, Josh Durgin) +* osd: two cleanups (`pr#20830 `_, songweibin) * osd: update dmclock library w git subtree pull (`pr#17737 `_, J. Eric Ivancich) * osd: update info only if new_interval (`pr#17437 `_, Kefu Chai) * osd: update store with options after pg is created (`issue#22419 `_, `pr#20044 `_, Kefu Chai) @@ -1816,31 +2231,59 @@ Other Notable Changes * osd: use prefix increment for non trivial iterator (`pr#19097 `_, Kefu Chai) * osd: Use specializations, typedefs instead (`pr#19354 `_, Shinobu Kinjo) * osd: Warn about objects with too many omap entries (`pr#16332 `_, Brad Hubbard) -* performance: common/config: use with_val() for better performance (`pr#19056 `_, Adam C. Emerson) -* performance: common/options.cc: Set Filestore rocksdb compaction readahead option (`issue#21505 `_, `pr#17900 `_, Mark Nelson) -* performance: osdc/Objecter: using coarse_mono instead (`pr#18473 `_, Haomai Wang) -* performance,rgw: Cache on the barrelhead (`issue#22517 `_, `pr#19581 `_, Adam C. Emerson) -* performance: rocksdb: sync with upstream (`issue#21603 `_, `pr#18262 `_, Kefu Chai) -* performance,tests: qa: add cbt task for performance testing (`pr#17583 `_, Neha Ojha) -* performance,tools: ceph-objectstore-tool: "$OBJ get-omaphdr" and "$OBJ list-omap" scan all pgs instead of using specific pg (`issue#21327 `_, `pr#17985 `_, David Zafman) -* pybind: ceph.in: execv using the same python (`pr#17713 `_, Kefu Chai) +* os/filestore/HashIndex.h: fixed a typo in comment (`pr#17685 `_, yaoguotao) +* os: Initializing uninitialized members aio_info (`pr#17066 `_, amitkuma) +* os: Removing dead code from LFNIndex.cc (`pr#17297 `_, Amit Kumar) +* prometheus: Handle the TIME perf counter type metrics (`pr#21749 `_, Boris Ranto) +* pybind: add return note in rbd.pyx (`pr#21768 `_, Zheng Yin) +* pybind/ceph_daemon: expand the order of magnitude of (`issue#23962 `_, `pr#21836 `_, Guan yunfei) * pybind: fix chart size become bigger when refresh (`issue#20746 `_, `pr#16857 `_, Yixing Yan) * pybind: mgr/dashboard: fix rbd's pool sub menu (`pr#16774 `_, yanyx) * pybind,rbd: pybind/rbd: support open the image by image_id (`pr#19361 `_, songweibin) * pybind: remove unused get_ceph_version() (`pr#17727 `_, Kefu Chai) +* qa: add cbt repo parameter (`pr#18543 `_, Neha Ojha) +* qa: Add cephmetrics suite (`pr#18451 `_, Zack Cerza) +* qa: add upgrade/luminous-x suite (`pr#17160 `_, Yuri Weinstein) +* qa/crontab: run the perf-basic suite every day (`pr#21252 `_, Neha Ojha) +* qa: Decreased amount of jobs on master, kraken, luminous runs (`pr#17069 `_, Yuri Weinstein) +* qa: install collectl with cbt task (`pr#19324 `_, Neha Ojha) +* qa: mimic-dev1 backports to avoid trusty nodes (`pr#19600 `_, Kefu Chai) +* qa: preserve cbt task results (`pr#19364 `_, Neha Ojha) +* qa: qa/ceph-disk: enlarge the simulated SCSI disk (`issue#22136 `_, `pr#19199 `_, Kefu Chai) +* qa/suites/perf-basic: add desc regarding test machines (`pr#21183 `_, Neha Ojha) +* qa/suites/rados/multimon/tasks/mon_lock_with_skew: whitelist PG (`pr#17004 `_, Sage Weil) +* qa/suites/rados/perf: add optimized settings (`pr#17786 `_, Neha Ojha) +* qa/suites/rados/perf: add workloads (`pr#18573 `_, Neha Ojha) +* qa/suites/rados/verify/validater/valgrind: whitelist PG (`pr#17005 `_, Sage Weil) +* qa/suites/upgrade/jewel-x/parallel: tolerate laggy mgr (`pr#17227 `_, Sage Weil) +* qa/suites/upgrade/kraken-x: fixes (`pr#16881 `_, Sage Weil) +* qa/suites/upgrade/luminous-x fixes (`pr#22101 `_, Sage Weil) +* qa/tests - Added options to use both cases: mon.a and installer.0 (`pr#19745 `_, Yuri Weinstein) +* qa/tests - Fixed typo in crontab entry (`pr#21482 `_, Yuri Weinstein) +* qa/tests: fixed typo (`pr#21728 `_, Yuri Weinstein) +* qa/tests - minor clean ups and made perf-suite run 3 times, so we can… (`pr#21309 `_, Yuri Weinstein) +* qa/tests - one more typo fixed :( (`pr#21483 `_, Yuri Weinstein) +* qa/tests: removed rest suite from the mix (`pr#21743 `_, Yuri Weinstein) +* qa: wait longer for osd to flush pg stats (`issue#24321 `_, `pr#22288 `_, Kefu Chai) +* qa/workunits/ceph-disk: --no-mon-config (`pr#21956 `_, Kefu Chai) * rados: make ceph_perf_msgr_client work for multiple jobs (`issue#22103 `_, `pr#18877 `_, Jeegn Chen) * rbd: add deep cp CLI method (`pr#19996 `_, songweibin) * rbd: add group rename methods (`issue#22981 `_, `pr#20577 `_, songweibin) +* rbd: add notrim option to rbd map (`pr#21056 `_, Hitoshi Kamei) * rbd: add parent info when moving child into trash bin (`pr#19280 `_, songweibin) * rbd: adjusted "lock list" JSON and XML formatted output (`pr#19900 `_, Jason Dillaman) * rbd: adjusted "showmapped" JSON and XML formatted output (`pr#19937 `_, Mykola Golub) * rbd: allow remove all unprotected snapshots (`issue#23126 `_, `pr#20608 `_, songweibin) * rbd: allow trash rm/purge when pool quota is full used (`pr#20697 `_, songweibin) +* rbd: backport of mimic bug fixes (`issue#24009 `_, `issue#24008 `_, `pr#21930 `_, Jason Dillaman) * rbd: check if an image is already mapped before rbd map (`issue#20580 `_, `pr#16517 `_, Jing Li) +* rbd: children list should support snapshot id optional (`issue#23399 `_, `pr#20966 `_, Jason Dillaman) +* rbd: cleanup handling of IEC byte units (`pr#21564 `_, Jason Dillaman) * rbd: clean up warnings when mirror commands used on non-setup pool (`issue#21319 `_, `pr#17636 `_, Jason Dillaman) * rbd: cls/journal: ensure tags are properly expired (`issue#21960 `_, `pr#18604 `_, Jason Dillaman) * rbd: cls/journal: fixed possible infinite loop in expire_tags (`issue#21956 `_, `pr#18592 `_, Jason Dillaman) * rbd: cls/journal: possible infinite loop within tag_list class method (`issue#21771 `_, `pr#18270 `_, Jason Dillaman) +* rbd: cls/rbd: group_image_list incorrectly flagged as RW (`issue#23388 `_, `pr#20939 `_, Jason Dillaman) * rbd: cls/rbd: metadata_list not honoring max_return parameter (`issue#21247 `_, `pr#17499 `_, Jason Dillaman) * rbd: cls/rbd: Silence gcc7 maybe-uninitialized warning (`pr#18504 `_, Brad Hubbard) * rbd: common/options,librbd/Utils: refactor RBD feature validation (`pr#20014 `_, Sage Weil) @@ -1848,6 +2291,7 @@ Other Notable Changes * rbd: do not show title if there is no group snapshot (`pr#20311 `_, songweibin) * rbd: don't overwrite the error code from the remove action (`pr#20481 `_, Jason Dillaman) * rbd: drop unnecessary using declaration, etc (`pr#19005 `_, Shinobu Kinjo) +* rbd: eager-thick provisioning support (`pr#18317 `_, Hitoshi Kamei) * rbd: export/import image-meta when we export/import an image (`pr#17134 `_, PCzhangPC) * rbd: filter out UserSnapshotNamespace in do_disk_usage (`pr#20532 `_, songweibin) * rbd: fix crash during map when "rw" option is specified (`issue#21808 `_, `pr#18313 `_, Peter Keresztes Schmidt) @@ -1858,8 +2302,10 @@ Other Notable Changes * rbd: group snapshot rename (`pr#12431 `_, Victor Denisov) * rbd: implement image qos in tokenbucket algorithm (`pr#17032 `_, Dongsheng Yang) * rbd: import with option --export-format 2 fails to protect snapshot (`issue#23038 `_, `pr#20613 `_, songweibin) +* rbd: improve 'import-diff' corrupt input error messages (`issue#18844 `_, `pr#21249 `_, Jason Dillaman) * rbd: Initializing m_finalize_ctx (`pr#17563 `_, Amit Kumar) * rbd: introduce commands of "image-meta ls/rm" (`pr#16591 `_, PCzhangPC) +* rbd: journal: limit number of appends sent in one librados op (`issue#23526 `_, `pr#21157 `_, Mykola Golub) * rbd: journal: trivial cleanup (`pr#19317 `_, Shinobu Kinjo) * rbd: krbd: include sys/sysmacros.h for major, minor and makedev (`pr#20773 `_, Ilya Dryomov) * rbd: krbd: rewrite "already mapped" code (`pr#17638 `_, Ilya Dryomov) @@ -1867,8 +2313,13 @@ Other Notable Changes * rbd: librbd: create+truncate for whole-object layered discards (`issue#23285 `_, `pr#20809 `_, Ilya Dryomov) * rbd: librbd: make rename request complete with filtered code (`issue#23068 `_, `pr#20507 `_, Mykola Golub) * rbd: librbd misc cleanup (`pr#18419 `_, Jianpeng Ma) +* rbd: librbd: skip head object map update when deep copying object beyond image size (`pr#21586 `_, Mykola Golub) * rbd: librbd: sync flush should re-use existing async flush logic (`pr#18403 `_, Jason Dillaman) * rbd: librbd,test: address coverity false positives (`pr#17825 `_, Amit Kumar) +* rbd: mimic: librbd: deep copy optionally support flattening cloned image (`issue#22787 `_, `pr#22038 `_, Mykola Golub) +* rbd: mimic: rbd-mirror: optionally support active/active replication (`pr#22105 `_, Jason Dillaman) +* rbd: mimic: rbd-mirror: potential deadlock when running asok 'flush' command (`issue#24141 `_, `pr#22039 `_, Mykola Golub) +* rbd-mirror: additional thrasher testing (`pr#21697 `_, Jason Dillaman) * rbd-mirror: clean up spurious error log messages (`issue#21961 `_, `pr#18601 `_, Jason Dillaman) * rbd-mirror: cluster watcher should ensure it has latest OSD map (`issue#22461 `_, `pr#19550 `_, Jason Dillaman) * rbd-mirror: ensure unique service daemon name is utilized (`pr#19492 `_, Jason Dillaman) @@ -1876,11 +2327,14 @@ Other Notable Changes * rbd-mirror: forced promotion can result in incorrect status (`issue#21559 `_, `pr#17979 `_, Jason Dillaman) * rbd-mirror: ImageMap memory leak fixes (`pr#19163 `_, Venky Shankar) * rbd-mirror: Improve data pool selection when creating images (`pr#18006 `_, Adam Wolfe Gordon) +* rbd-mirror: integrate image map policy as incremental step to active-active (`pr#21300 `_, Jason Dillaman) * rbd-mirror: introduce basic image mapping policy (`issue#18786 `_, `pr#15691 `_, Venky Shankar) * rbd-mirror: missing lock when re-sending update_sync_point (`pr#19011 `_, Mykola Golub) * rbd-mirror: persist image map timestamp (`pr#19338 `_, Venky Shankar) * rbd-mirror: primary image should register in remote, non-primary image's journal (`issue#21561 `_, `pr#18136 `_, Jason Dillaman) +* rbd-mirror: properly translate remote tag mirror uuid for local mirror (`issue#23876 `_, `pr#21657 `_, Jason Dillaman) * rbd-mirror: removed dedicated thread from image deleter (`issue#15322 `_, `pr#19000 `_, Jason Dillaman) +* rbd-mirror: rename asok hook to match image name when not replaying (`issue#23888 `_, `pr#21682 `_, Jason Dillaman) * rbd-mirror: rollback state transitions in image policy (`pr#19577 `_, Venky Shankar) * rbd-mirror: Set the data pool correctly when creating images (`issue#20567 `_, `pr#17073 `_, Adam Wolfe Gordon) * rbd-mirror: simplify notifications for image assignment (`issue#15764 `_, `pr#16642 `_, Jason Dillaman) @@ -1894,6 +2348,7 @@ Other Notable Changes * rbd-nbd: bug fix when running in container (`issue#22012 `_, `issue#22011 `_, `pr#18663 `_, Li Wang) * rbd-nbd: certain kernels may not discover resized block devices (`issue#22131 `_, `pr#18947 `_, Jason Dillaman) * rbd-nbd: cleanup for NBDServer shut down (`pr#17283 `_, Pan Liu) +* rbd-nbd: fix ebusy when do map (`issue#23528 `_, `pr#21142 `_, Li Wang) * rbd-nbd: fix generic option issue (`issue#20426 `_, `pr#17375 `_, Pan Liu) * rbd-nbd: output format support for list-mapped command (`pr#19704 `_, Mykola Golub) * rbd-nbd: support optionally setting device timeout (`issue#22333 `_, `pr#19436 `_, Mykola Golub) @@ -1904,6 +2359,7 @@ Other Notable Changes * rbd: properly pass ceph global command line args to subprocess (`pr#19821 `_, Mykola Golub) * rbd: pybind/rbd: add deep_copy method (`pr#19406 `_, Mykola Golub) * rbd: pybind/rbd: fix metadata functions error handling (`issue#22306 `_, `pr#19337 `_, Mykola Golub) +* rbd: python bindings fixes and improvements (`issue#23609 `_, `pr#21304 `_, Ricardo Dias) * rbd: rbd-ggate: fix parsing ceph global options (`pr#19822 `_, Mykola Golub) * rbd: rbd-ggate: fix syntax error (`pr#19919 `_, Willem Jan Withagen) * rbd: rbd-ggate: make list command produce valid xml format output (`pr#19823 `_, Mykola Golub) @@ -1919,10 +2375,14 @@ Other Notable Changes * rbd: snap limit should't be set smaller than the number of existing snaps (`pr#16597 `_, PCzhangPC) * rbd: support cloning an image from a non-primary snapshot (`issue#18480 `_, `pr#19724 `_, Jason Dillaman) * rbd: support iterating over metadata items when listing (`issue#21179 `_, `pr#17532 `_, Jason Dillaman) +* rbd: support lock_timeout in rbd mapping (`pr#21344 `_, Dongsheng Yang) * rbd: support osd_request_timeout in rbd map command (`issue#23073 `_, `pr#20792 `_, Dongsheng Yang) * rbd: switched from legacy to new-style configuration options (`issue#20737 `_, `pr#16737 `_, Jason Dillaman) * rbd,tests: qa: additional krbd discard test cases (`pr#20499 `_, Ilya Dryomov) * rbd,tests: qa: fix POOL_APP_NOT_ENABLED warning in krbd:unmap suite (`pr#16966 `_, Ilya Dryomov) +* rbd,tests: qa: introduce rbd-mirror thrasher to existing tests (`issue#18753 `_, `pr#21541 `_, Jason Dillaman) +* rbd,tests: qa: krbd_exclusive_option.sh: add lock_timeout test case (`pr#21522 `_, Ilya Dryomov) +* rbd,tests: qa: krbd_fallocate.sh: add notrim test case (`pr#21513 `_, Ilya Dryomov) * rbd,tests: qa: krbd huge-image test (`pr#20692 `_, Ilya Dryomov) * rbd,tests: qa: krbd latest-osdmap-on-map test (`pr#20591 `_, Ilya Dryomov) * rbd,tests: qa: krbd msgr-segments test (`pr#20714 `_, Ilya Dryomov) @@ -1936,14 +2396,17 @@ Other Notable Changes * rbd,tests: qa/suites/rbd: fewer socket failures (`pr#19617 `_, Sage Weil) * rbd,tests: qa/suites/rbd: miscellaneous test fixes (`issue#21251 `_, `pr#17504 `_, Jason Dillaman) * rbd,tests: qa/suites/rbd: segregated v1 image format tests (`issue#22738 `_, `pr#20729 `_, Jason Dillaman) +* rbd,tests: qa/suites/rbd: set qemu task time_wait param (`pr#21131 `_, Mykola Golub) * rbd,tests: qa/tasks/cram: include /usr/sbin in the PATH for all commands (`pr#18793 `_, Ilya Dryomov) * rbd,tests: qa/tasks/rbd: run all xfstests runs to completion (`pr#18583 `_, Ilya Dryomov) * rbd,tests: qa/workunits/rbd: fix cli_generic test_purge for rbd default format 1 (`pr#20389 `_, Mykola Golub) * rbd,tests: qa/workunits/rbd: fixed variable name for resync image id (`issue#21663 `_, `pr#18097 `_, Jason Dillaman) * rbd,tests: qa/workunits/rbd: fix issues within permissions test (`issue#23043 `_, `pr#20491 `_, Jason Dillaman) * rbd,tests: qa/workunits/rbd: pool create may fail for small cluster (`pr#18067 `_, Mykola Golub) +* rbd,tests: qa/workunits/rbd: potential race in mirror disconnect test (`issue#23938 `_, `pr#21733 `_, Mykola Golub) * rbd,tests: qa/workunits/rbd: relax greps to support upgrade formatting change (`issue#21181 `_, `pr#17559 `_, Jason Dillaman) * rbd,tests: qa/workunits/rbd: remove sanity check in journal.sh test (`pr#20490 `_, Jason Dillaman) +* rbd,tests: qa/workunits/rbd: remove sanity check in test_admin_socket.sh (`pr#21116 `_, Mykola Golub) * rbd,tests: qa/workunits/rbd: remove "trash purge --threshold" test (`issue#22803 `_, `pr#20170 `_, Mykola Golub) * rbd,tests: qa/workunits/rbd: simplify split-brain test to avoid potential race (`issue#22485 `_, `pr#19604 `_, Jason Dillaman) * rbd,tests: qa/workunits/rbd: switch devstack tempest to 17.2.0 tag (`issue#22961 `_, `pr#20599 `_, Jason Dillaman) @@ -1954,11 +2417,14 @@ Other Notable Changes * rbd,tests: qa/workunits/rbd: wait for demote status is propagated (`pr#19073 `_, Mykola Golub) * rbd,tests: qa/workunits/rbd: wait for status propagated only if daemon started (`pr#19082 `_, Mykola Golub) * rbd,tests: rbd/test: add snap protection test for ex/import (`pr#20689 `_, songweibin) +* rbd,tests: stop.sh: use --no-mon-config when trying to unmap rbd devices (`pr#21020 `_, Mykola Golub) * rbd,tests: test: address coverity false positives (`pr#17803 `_, Amit Kumar) * rbd,tests: test/cls_rbd: mask newer feature bits to support upgrade tests (`issue#21217 `_, `pr#17509 `_, Jason Dillaman) +* rbd,tests: test/librados_test_stub: always create copy of buffers passed to operation (`pr#21074 `_, Mykola Golub) * rbd,tests: test/librbd: added update_features RPC message to test_notify (`issue#21936 `_, `pr#18561 `_, Jason Dillaman) * rbd,tests: test/librbd: clean up for several mock function tests (`pr#18952 `_, Jason Dillaman) * rbd,tests: test/librbd: Do not instantiate TrimRequest template class (`pr#19402 `_, Boris Ranto) +* rbd,tests: test/librbd: ensure OutOfOrder test has enough concurrent management ops (`pr#21436 `_, Mykola Golub) * rbd,tests: test/librbd: fix mock method macro of set_journal_policy (`pr#17216 `_, Yan Jun) * rbd,tests: test/librbd: fix race condition with OSD map refresh (`issue#20918 `_, `pr#16877 `_, Jason Dillaman) * rbd,tests: test/librbd: fix valgrind memory leak warning (`pr#17187 `_, Mykola Golub) @@ -1974,22 +2440,26 @@ Other Notable Changes * rbd,tests: test/rbd: cli_generic fails if v1 image format or deep-flatten disabled (`issue#22950 `_, `pr#20364 `_, songweibin) * rbd,tests: test/rbd_mirror: fix valgrind warnings in unittest (`pr#19016 `_, Mykola Golub) * rbd,tests: test/rbd-mirror: image map policy test (`pr#19320 `_, Venky Shankar) +* rbd,tests: test/rbd-mirror: improve coverage for dead instance handling (`pr#21403 `_, Jason Dillaman) * rbd,tests: test/rbd_mirror: "use of uninitialised value" valgrind warning (`pr#19437 `_, Mykola Golub) * rbd,tools: rbd-fuse: make sure PATH_MAX is defined (`pr#18615 `_, Roberto Oliveira) +* rbd,tools: rbd-replay: remove boost dependency (`pr#21202 `_, Kefu Chai) * rbd: tools/rbd: use steady clock in bencher (`pr#20008 `_, Mohamad Gebai) * rbd: 'trash list --long' will return a failure on non-cloned images (`pr#19540 `_, Jason Dillaman) +* rbd: 'trash ls -l' will display column titles if existed non-USER trash image only (`pr#21343 `_, songweibin) * rbd: unified way to map images using different drivers (`pr#19711 `_, Mykola Golub) * rbd: use different logic to disturb thread's offset in bench seq test (`pr#17218 `_, PCzhangPC) -* rdma: msg/async/rdma: fix a coredump introduced by PR #18053 (`pr#18204 `_, Yan Lei) -* rdma: msg/async/rdma: fix a potential coredump when handling tx_buffers under heavy RDMA (`pr#18036 `_, Yan Lei) -* rdma: msg/async/rdma: fix Tx buffer leakage that can introduce "heartbeat no reply" (`pr#18053 `_, Yan Lei) -* rdma: msg/async/rdma: refactor rx buffer pool allocator (`pr#17018 `_, Alex Mikheev) -* rdma: msg/async/rdma: unnecessary reinitiliazation of an iterator (`pr#18190 `_, JustL) +* Revert "ceph-fuse: Delete inode's bufferhead was in Tx state would le… (`pr#21976 `_, "Yan, Zheng") +* Revert "msg/async/rdma: fix multi cephcontext confllicting" (`pr#16980 `_, Haomai Wang) +* Revert "os/bluestore: compensate for bad freelistmanager size/blocks metadata" (`pr#17275 `_, Xie Xingguo) +* rgw: ability to list bucket contents in unsorted order for efficiency (`pr#21026 `_, J. Eric Ivancich) * rgw: abort multipart if upload meta object doesn't exist (`pr#19918 `_, fang yuxiang) * rgw: Access RGWConf through RGWEnv (`pr#17432 `_, Jos Collin) * rgw: add "Accept-Ranges" to response header of Swift API (`issue#21554 `_, `pr#17967 `_, Tone Zhang) * rgw: add a default redirect field for zones (`pr#9571 `_, Yehuda Sadeh) * rgw: add an option to clear all usage entries (`pr#19322 `_, Abhishek Lekshmanan) +* rgw: add an option to recalculate user stats (`issue#23335 `_, `pr#20853 `_, Abhishek Lekshmanan) +* rgw: add buffering filter to compression for fetch_remote_obj (`issue#23547 `_, `pr#21479 `_, Casey Bodley) * rgw: add cors header rule check in cors option request (`issue#22002 `_, `pr#18556 `_, yuliyang) * rgw: Add dynamic resharding documentation (`issue#21553 `_, `pr#15941 `_, Orit Wasserman) * rgw: add logs if get_data returns error in RGWPutObj::execute (`pr#18642 `_, Zhang Shaowen) @@ -2000,6 +2470,7 @@ Other Notable Changes * rgw: address warnings due to incorrect format code (`pr#18796 `_, J. Eric Ivancich) rgw: Add retry_raced_bucket_write * rgw: add rewrite cmd and options into radosgw-admin usage and doc (`pr#18918 `_, Enming Zhang) +* rgw: add ssl support to beast frontend (`issue#22832 `_, `pr#20464 `_, Casey Bodley) * rgw: add support for Swift's per storage policy statistics (`issue#17932 `_, `pr#12704 `_, Radoslaw Zarzynski) * rgw: add support for Swift's reversed account listings (`issue#21148 `_, `pr#17320 `_, Radoslaw Zarzynski) * rgw: add support for tagging and other conditionals in policy (`pr#17094 `_, Abhishek Lekshmanan) @@ -2011,16 +2482,21 @@ Other Notable Changes * rgw: adjust log format for lifecycle (`pr#19576 `_, Bingyin Zhang) * rgw: admin api - add ability to sync user stats from admin api (`issue#21301 `_, `pr#17589 `_, Nathan Johnson) * rgw: Admin API Support for bucket quota change (`issue#21811 `_, `pr#18324 `_, Jeegn Chen) +* rgw: admin rest api shouldn't return error when getting user's stats if the user hasn't create any bucket (`pr#21551 `_, Zhang Shaowen) * rgw: allow beast frontend to listen on specific IP address (`issue#22778 `_, `pr#20000 `_, Yuan Zhou) * rgw: Allow swift acls to be deleted (`issue#22897 `_, `pr#20471 `_, Marcus Watts) * rgw: avoid logging keystone revocation messages when not configured (`issue#21400 `_, `pr#17775 `_, Abhishek Lekshmanan) +* rgw: aws4 auth supports PutBucketRequestPayment (`issue#23803 `_, `pr#21569 `_, Casey Bodley) * rgw: AWS v4 authorization work when INIT_MULTIPART is chunked (`issue#22129 `_, `pr#18956 `_, Jeegn Chen) * rgw: beast frontend can listen on multiple endpoints (`issue#22779 `_, `pr#20188 `_, Casey Bodley) +* rgw: beast frontend no longer experimental (`pr#21272 `_, Casey Bodley) * rgw: Better ERANGE error message (`issue#22351 `_, `pr#20023 `_, Brad Hubbard) * rgw: break sending data-log list infinitely (`issue#20951 `_, `pr#16926 `_, fang.yuxiang) * rgw: bucket resharding should not update bucket ACL or user stats (`issue#22742 `_, `issue#22124 `_, `pr#20038 `_, Orit Wasserman) +* rgw: Cache on the barrelhead (`issue#22517 `_, `pr#19581 `_, Adam C. Emerson) * rgw: Cache Register! (`issue#22604 `_, `issue#22603 `_, `pr#20144 `_, Adam C. Emerson) * rgw: can't download object with range when compression enabled (`issue#22852 `_, `pr#20226 `_, fang yuxiang) +* rgw: ceph-dencoder: add missing begin_iter & end_iter item for RGWObjManifest (`pr#19509 `_, wangsongbo) * rgw: ceph-dencoder: add support for cls_rgw_lc_obj_head (`pr#18920 `_, Yao Zongyou) * rgw: ceph-dencoder: add support for RGWLifecycleConfiguration (`pr#18959 `_, wangsongbo) * rgw: change ObjectCache::lru from deque back to list (`issue#22560 `_, `pr#19768 `_, Casey Bodley) @@ -2031,10 +2507,12 @@ Other Notable Changes * rgw: check going_down() when lifecycle processing (`issue#22099 `_, `pr#18846 `_, Yao Zongyou) * rgw: Check payment operations in policy (`issue#21389 `_, `pr#17742 `_, Adam C. Emerson) * rgw: check read_op.read return value in RGWRados::copy_obj_data (`pr#18962 `_, Enming Zhang) +* rgw: civetweb fixes for v1.1 upgrade (`pr#21123 `_, Abhishek Lekshmanan) * rgw: clean code with helper function dump_header_if_nonempty (`pr#18979 `_, Xinying Song) * rgw: clean up and fix some bugs for encryption (`issue#21581 `_, `pr#17882 `_, Enming Zhang) * rgw: cleanup MIN macro with std::min (`pr#17546 `_, Jiaying Ren) * rgw: cleanup unused parameters in RGWRados::copy_obj_data (`pr#18917 `_, Enming Zhang) +* rgw: cloud sync fixes (`pr#21648 `_, Yehuda Sadeh) * rgw: cls/log: cls_log_list always returns next marker (`issue#20906 `_, `pr#17024 `_, Casey Bodley) * rgw: cls/rgw: fix bi_log_iterate_entries return wrong truncated (`issue#22737 `_, `pr#20021 `_, Tianshan Qu) * rgw: cls/rgw: Initialization of uninitialized members (`pr#16932 `_, amitkuma) @@ -2042,6 +2520,7 @@ Other Notable Changes * rgw: cls/rgw: remove unused variable bl (`pr#19570 `_, Yao Zongyou) * rgw: cls/rgw: trim all usage entries in cls_rgw (`issue#22234 `_, `pr#19131 `_, Abhishek Lekshmanan) * rgw: cls_rgw: use more effective container operations in get_obj_vals (`pr#19272 `_, Xinying Song) +* rgw: comparison between signed and unsigned integer expressions (`pr#21105 `_, ashitakasam) * rgw: consolidate code that implements hashing algorithms (`pr#18248 `_, J. Eric Ivancich) * rgw: copy object add response error messages (`pr#18291 `_, Enming Zhang) * rgw: correct comment in function parse_credentials (`pr#19275 `_, Bingyin Zhang) @@ -2057,6 +2536,7 @@ Other Notable Changes * rgw: data sync: set num_shards when building full maps (`issue#22083 `_, `pr#18852 `_, Abhishek Lekshmanan) * rgw: Delete to_string functions. stringify defined in include/stringify.h can provide the same feature (`pr#18522 `_, zhangwen) * rgw: disable dynamic resharding in multisite environment (`issue#21725 `_, `pr#18184 `_, Orit Wasserman) +* rgw: do not reflect period if not current (`issue#22844 `_, `pr#20212 `_, Tianshan Qu) * rgw: do not update all gateway caches upon creation of system obj w/ exclusive flag (`pr#19384 `_, J. Eric Ivancich) * rgw: don't change rados object's mtime when update olh (`issue#21743 `_, `pr#18214 `_, Shasha Lu) * rgw: don't hold data_lock over frontend io (`pr#20621 `_, Casey Bodley) @@ -2082,16 +2562,20 @@ Other Notable Changes * rgw: drop unused variable bucket_instance_ids (`pr#19708 `_, Bingyin Zhang) * rgw: drop unused variable in copy_obj_data() (`pr#18477 `_, Enming Zhang) * rgw: drop unused vector elements (`pr#19815 `_, Bingyin Zhang) -* rgw: drop useless includes in rgw_{main.cc, common.h} (`pr#19109 `_, Jiaying Ren) +* rgw: drop useless includes in rgw\_{main.cc, common.h} (`pr#19109 `_, Jiaying Ren) * rgw: drop useless lines (`pr#19817 `_, Bingyin Zhang) * rgw: drop useless type conversion (`pr#19824 `_, Bingyin Zhang) * rgw: drop variable bl in rgw_op.cc (`pr#19584 `_, Bingyin Zhang) * rgw: Drop #warning TODO (`issue#19851 `_, `pr#17012 `_, Jos Collin) * rgw: dump Last-Modified in Swift's responses for GET/HEAD on container (`issue#20883 `_, `pr#16757 `_, Radoslaw Zarzynski) +* rgw: enable 'qlen' & 'qactive' performance counters (`pr#20842 `_, Mark Kogan) +* rgw: encoding fixes (`issue#23779 `_, `pr#21500 `_, Yehuda Sadeh) * rgw: Error check on return of read_line() (`pr#17880 `_, Amit Kumar) +* rgw: es module: set compression type correctly (`issue#22758 `_, `pr#20796 `_, Abhishek Lekshmanan) * rgw: evaluate the correct bucket action for GetACL bucket operation (`issue#21013 `_, `pr#17050 `_, Abhishek Lekshmanan) * rgw: exit early if rgw_bucket_set_attrs() fails (`pr#17041 `_, dengxiafubi) rgw: Expire entries in bucket info cache +* rgw_file: fix write error when the write offset overlaps (`issue#21455 `_, `pr#17809 `_, Yao Zongyou) * rgw: fix a bug in rgw cache in delete_system_obj and get_system_obj (`pr#10992 `_, zhangshaowen) * rgw: fix accessing expired memory in PrefixableSignatureHelper (`issue#21085 `_, `pr#17206 `_, Radoslaw Zarzynski) * rgw: fix a typo in comment (`pr#19608 `_, luomuyao) @@ -2109,18 +2593,24 @@ Other Notable Changes * rgw: fix 'copy part' without 'x-amz-copy-source-range' when compression enabled (`issue#23196 `_, `pr#20686 `_, fang yuxiang) * rgw: fix crash with rgw_run_sync_thread false (`issue#20448 `_, `pr#20769 `_, Orit Wasserman) * rgw: Fix dereference of empty optional (`issue#21962 `_, `pr#18602 `_, Adam C. Emerson) +* rgw: fix error handling for GET with ?torrent (`issue#23506 `_, `pr#21576 `_, Casey Bodley) * rgw: fix error handling in Browser Uploads (`pr#15054 `_, Radoslaw Zarzynski) * rgw: fix error handling in ListBucketIndexesCR (`issue#21735 `_, `pr#18198 `_, Casey Bodley) * rgw: fixes for multisite replication of encrypted objects (`issue#20668 `_, `issue#20671 `_, `pr#16612 `_, Casey Bodley) * rgw: fix extra_data_len handling in PutObj filters (`issue#21895 `_, `pr#18489 `_, Casey Bodley) * rgw: fix for empty query string in beast frontend (`issue#22797 `_, `pr#20120 `_, Casey Bodley) +* rgw: fix for issue #21647 (`issue#23859 `_, `pr#21647 `_, Yehuda Sadeh) * rgw: fix for pause in beast frontend (`issue#21831 `_, `pr#18402 `_, Casey Bodley) +* rgw: fix for usage truncated flag (`pr#20926 `_, Yehuda Sadeh, Greg Farnum, Robin H. Johnson) * rgw: Fix getter function names in RGWEnv (`pr#18377 `_, Jos Collin) * rgw: fix GET website response error code (`issue#22272 `_, `pr#19236 `_, Dmitry Plyakin) * rgw: fix handling of ENOENT in RGWRadosGetOmapKeysCR (`pr#19878 `_, Casey Bodley) * rgw: fix index cancel op miss update header (`pr#20396 `_, Tianshan Qu) +* rgw: Fix infinite call for bi list when resharding a bucket (`issue#22721 `_, `pr#21584 `_, Orit Wasserman) * rgw: fix lc process only schdule the first item of lc objects (`issue#21022 `_, `pr#17061 `_, Shasha Lu) * rgw:fix list objects with marker wrong result when bucket is enable versioning (`issue#21500 `_, `pr#17934 `_, yuliyang) +* rgw: fix memory fragmentation problem reading data from client (`pr#20724 `_, Marcus Watts) +* rgw: Fix multisite Synchronization failed when read and write delete … (`issue#22804 `_, `pr#20814 `_, Niu Pengju) * rgw: fix not responding when receiving SIGHUP signal (`pr#16854 `_, Yao Zongyou) * rgw: fix null pointer crush (`pr#18861 `_, Sibei Gao) * rgw: fix obj copied from remote gateway acl full_control issue (`issue#20658 `_, `pr#16127 `_, Enming Zhang) @@ -2129,6 +2619,7 @@ Other Notable Changes * rgw: fix opslog uri as per Amazon s3 (`issue#20971 `_, `pr#16958 `_, Jiaying Ren) * rgw: fix radosgw-admin bucket rm with --purge-objects and --bypass-gc (`issue#22122 `_, `issue#19959 `_, `pr#18922 `_, Aleksei Gutikov) * rgw: fix radosgw-admin quota enable return value bug (`issue#21608 `_, `pr#18057 `_, baixueyu) +* rgw: fix radosgw linkage with WITH_RADOSGW_BEAST_FRONTEND=OFF (`issue#23680 `_, `pr#21380 `_, Casey Bodley) * rgw: fix recursive lock (`pr#19430 `_, Tianshan Qu) * rgw: fix resource leak in rgw_bucket.cc and rgw_user.cc (`issue#21214 `_, `pr#17353 `_, Luo Kexue) * rgw: fix return value of auth v2/v4 (`issue#22439 `_, `pr#19310 `_, Bingyin Zhang) @@ -2140,16 +2631,19 @@ Other Notable Changes * rgw: fix s3website redirect location string length (`pr#19826 `_, yuliyang) * rgw: fix Swift container naming rules (`issue#19264 `_, `pr#13992 `_, Robin H. Johnson) * rgw: Fix swift object expiry not deleting objects (`issue#22084 `_, `pr#18821 `_, Pavan Rallabhandi) +* rgw: fix sync status conflict with cloud sync (`pr#21425 `_, Casey Bodley) * rgw: fix the bug of radowgw-admin zonegroup set requires realm (`issue#21583 `_, `pr#19061 `_, lvshanchun) * rgw: fix the max-uploads parameter not work (`issue#22825 `_, `pr#20158 `_, Xin Liao) * rgw: fix the return type is wrong (`pr#19773 `_, hechuang) * rgw: fix total_time to msec as per AWS S3 (`pr#17541 `_, Jiaying Ren) * rgw: fix typo anynoymous to anonymous (`pr#19281 `_, Bingyin Zhang) * rgw: fix typo compete to complete (`pr#19675 `_, Bingyin Zhang) +* rgw: Fix typo in comment (`pr#21032 `_, Simran Singhal) * rgw: fix typo in GetOmapKeysCR (`pr#19713 `_, lvshanchun) * rgw: fix typo signle to single (`pr#19517 `_, Bingyin Zhang) * rgw: fix typo woild to would (`pr#19472 `_, Bingyin Zhang) * rgw: Fix use after free in IAM policy parser (`pr#16823 `_, Adam C. Emerson) +* rgw: fix use of libcurl with empty header values (`issue#23663 `_, `pr#21358 `_, Casey Bodley) * rgw: format logs in file rgw_lc.cc (`pr#19615 `_, Bingyin Zhang) * rgw: format rgw_bucket_dir_header in ceph-dencoder (`pr#19753 `_, Bingyin Zhang) * rgw: gc use aio (`pr#20546 `_, Yehuda Sadeh) @@ -2164,6 +2658,7 @@ Other Notable Changes * rgw: Ignoring the returned error (`pr#17907 `_, Amit Kumar) * rgw: implement ipv4 aws:SourceIp condition for bucket policy (`pr#19167 `_, yuliyang) * rgw: improve handling of Swift's error messages and limits (`issue#17938 `_, `issue#21169 `_, `issue#17935 `_, `issue#17934 `_, `issue#17936 `_, `pr#15369 `_, Radoslaw Zarzynski) +* rgw: improve sync status: display behind bucket shards (`pr#20027 `_, lvshanchun) * rgw: improve sync status (`pr#19573 `_, lvshanchun) * rgw: include SSE-KMS headers in encrypted upload response (`issue#21576 `_, `pr#17999 `_, Casey Bodley) * rgw: incorporate the Transfer-Encoding fix for CivetWeb (`issue#21015 `_, `pr#17072 `_, Radoslaw Zarzynski) @@ -2183,9 +2678,13 @@ Other Notable Changes * rgw: loadgen fix generate random object name rgw crash issue (`issue#22006 `_, `pr#18536 `_, Enming Zhang) * rgw: log the right http status code in civetweb frontend's access log (`issue#22538 `_, `pr#19678 `_, Yao Zongyou) * rgw: log unlink_instance mtime as object's mtime (`issue#18885 `_, `pr#20016 `_, Yehuda Sadeh) +* rgw: lttng: Trace rgw data transfer, bi entry and object header update processes (`pr#20556 `_, Yang Honggang) * rgw: make init env methods return an error (`issue#23039 `_, `pr#20488 `_, Abhishek Lekshmanan) * rgw: make radosgw object stat RGW_ATTR_COMPRESSION dump readable (`pr#19846 `_, fang yuxiang) +* rgw: mfa support (`pr#19283 `_, Yehuda Sadeh) +* rgw: mimic: rgw: policy: modify s3:ListBucketMultiPartUploads to s3:ListBucketMul (`issue#24062 `_, `pr#21916 `_, xiangxiang) * rgw: modify s3 type subuser access permissions fail through admin rest api (`issue#21983 `_, `pr#18641 `_, yuliyang) +* rgw: move all pool creation into rgw_init_ioctx (`issue#23480 `_, `pr#21534 `_, Casey Bodley) * rgw: mrgw.sh uses instance name 'client.rgw' (`pr#18404 `_, Casey Bodley) * rgw: multisite log tracing (`pr#16492 `_, Yehuda Sadeh, Casey Bodley) * rgw,nfs: Add hint to use -o sync when mouting (`pr#16210 `_, Adam Kupczyk) @@ -2205,10 +2704,14 @@ Other Notable Changes * rgw: proper error message when tier_type does not exist (`issue#22469 `_, `pr#19575 `_, lvshanchun, Chang Liu) * rgw: pull up beast submodule and update frontend (`pr#17923 `_, Casey Bodley) * rgw: put bucket policy panics RGW process (`issue#22541 `_, `pr#19687 `_, Bingyin Zhang) +* rgw: radosgw-admin abort early for user stats for empty uids (`issue#23322 `_, `pr#20846 `_, Abhishek Lekshmanan) +* rgw: radosgw-admin should not use metadata cache for readonly commands (`issue#23468 `_, `pr#21129 `_, Orit Wasserman) * rgw: radosgw-admin zonegroup get and zone get return defaults when there is no realm (`issue#21615 `_, `pr#18667 `_, lvshanchun) * rgw: radosgw: fix awsv4 header line sort order (`issue#21607 `_, `pr#18046 `_, Marcus Watts) * rgw: radosgw: usage: fix bytes_sent bug (`issue#19870 `_, `pr#16834 `_, Marcus Watts) +* rgw: raise log level on coroutine shutdown errors (`issue#23974 `_, `pr#21791 `_, Casey Bodley) * rgw: Reinstating error codes mapping for Roles (`pr#20309 `_, Pritha Srivastava) +* rgw: reject encrypted object COPY before supported (`issue#23232 `_, `pr#20739 `_, Jeegn Chen) * rgw: release cls lock if taken in RGWCompleteMultipart (`issue#21596 `_, `pr#18104 `_, Matt Benjamin) * rgw: Remove assertions in IAM Policy (`pr#18225 `_, Adam C. Emerson) * rgw: remove get_system_obj_attrs in function RGWDeleteLC::execute and RGWDeleteCORS::execute (`pr#19582 `_, Bingyin Zhang) @@ -2225,11 +2728,13 @@ Other Notable Changes * rgw: remove unused param in get_bucket_instance_policy_from_attr (`pr#19129 `_, Bingyin Zhang) * rgw: remove unused variables (`pr#16649 `_, Zhang Lei) * rgw: remove useless lines in RGWDeleteBucket::execute (`pr#19699 `_, Bingyin Zhang) +* rgw: reshard cancel command should clear bucket resharding flag (`issue#21619 `_, `pr#21120 `_, Orit Wasserman) * rgw: reshard should not update stats when linking new bucket instance (`issue#22124 `_, `pr#19253 `_, Orit Wasserman) * rgw: retry CORS put/delete operations on ECANCELLED (`issue#22517 `_, `pr#19601 `_, Adam C. Emerson) * rgw: return 'Access-Control-Allow-Origin' header when the set and delete bucket website through XMLHttpRequest (`pr#17632 `_, yuliyang) * rgw: return 'Access-Control-Allow-Origin' header when the set bucket versioning through XMLHttpRequest (`pr#17631 `_, yuliyang) * rgw: return bucket's location no matter which zonegroup it located in (`issue#21125 `_, `pr#17250 `_, Shasha Lu) +* rgw: return EINVAL if max_keys can not convert correctly (`issue#23586 `_, `pr#21285 `_, yuliyang) * rgw: Return Error if Bucket Policy Contians Undefined Action (`pr#17433 `_, zhangwen) * rgw: Returning when dst_ioctx.operate() returns error (`pr#17873 `_, Amit Kumar) * rgw: return valid Location element, CompleteMultipartUpload (`pr#19902 `_, Matt Benjamin) @@ -2244,15 +2749,17 @@ Other Notable Changes * rgw: rgw-admin: require --yes-i-really-mean-it when using --inconsistent_index (`issue#20777 `_, `pr#17185 `_, Orit Wasserman) * rgw: rgw-admin: support for processing all gc objects including unexpired (`pr#17482 `_, Yao Zongyou) * rgw: RGW: change function parameters from value to refrence (`pr#18355 `_, Sibei Gao) +* rgw: RGWCivetWeb::read_data: fix arguments to mg_read() call (`issue#23596 `_, `pr#21291 `_, Nathan Cutler) * rgw: rgw clean-up: remove unreferenced pure virtual class StreamObjData (`pr#18799 `_, J. Eric Ivancich) * rgw: rgw clean-up: remove unused var & func in RGWRados::SystemObject (`pr#18987 `_, J. Eric Ivancich) * rgw: rgw cleanup: some unnecessary function called and repeated assignment (`pr#18817 `_, Enming Zhang) +* rgw: rgw cloud sync (`issue#21802 `_, `pr#18932 `_, lvshanchun, Yehuda Sadeh, Chang Liu, Abhishek Lekshmanan) * rgw: RGWEnv::set() takes std::string (`issue#22101 `_, `pr#18866 `_, Casey Bodley) * rgw: rgw_file: alternate fix deadlock on lru eviction (`pr#20034 `_, Matt Benjamin) * rgw: rgw_file: avoid evaluating nullptr for readdir offset (`pr#20145 `_, Matt Benjamin) +* rgw: rgw_file: conditionally unlink handles when direct deleted (`issue#23299 `_, `pr#20834 `_, Matt Benjamin) * rgw: rgw_file: explicit NFSv3 open() emulation (`pr#18365 `_, Matt Benjamin) * rgw: rgw_file: fix LRU lane lock in evict_block() (`issue#21141 `_, `pr#17267 `_, Matt Benjamin) -* rgw: rgw_file: fix write error when the write offset overlaps (`issue#21455 `_, `pr#17809 `_, Yao Zongyou) * rgw: rgw_file: implement variant offset readdir processing (`pr#18335 `_, Matt Benjamin) * rgw: rgw_file: introduce new fsid and rgw_mount (`pr#15330 `_, Gui Hecheng) * rgw: rgw_file: set s->obj_size from bytes_written (`issue#21940 `_, `pr#18571 `_, Matt Benjamin) @@ -2263,6 +2770,7 @@ Other Notable Changes * rgw: rgw_lc: support for AWSv4 authentication (`pr#16734 `_, Abhishek Lekshmanan) * rgw: rgw_log, rgw_file: account for new required envvars (`issue#21942 `_, `pr#18572 `_, Matt Benjamin) * rgw: Rgw master fix plus (`issue#21000 `_, `issue#21003 `_, `issue#20501 `_, `pr#17040 `_, Zhang Shaowen, Marcus Watts) +* rgw: rgw, mon: normalize delete/remove in admin console (cleanup 22444) (`issue#14363 `_, `issue#22444 `_, `pr#19439 `_, Jesse Williamson) * rgw: RGW: Multipart upload may double the quota (`issue#21586 `_, `pr#17959 `_, Sibei Gao) * rgw: rgw multisite: automated trimming for bucket index logs (`issue#18229 `_, `pr#17761 `_, Casey Bodley) * rgw: RGW NFS: mount cmdline example missing -osync (`pr#15855 `_, Matt Benjamin) @@ -2291,12 +2799,15 @@ Other Notable Changes * rgw: silence not allow register storage class specifier warning (`pr#19859 `_, Yao Zongyou) * rgw: simplify use of map::emplace in iam (`pr#18706 `_, Casey Bodley) * rgw: Small refactor and two bug fixes (`issue#21901 `_, `issue#21896 `_, `pr#18606 `_, Adam C. Emerson) +* rgw: some cleanup for sync status (`pr#20894 `_, Enming Zhang) * rgw: stop/join TokenCache revoke thread only if started (`issue#21666 `_, `pr#18106 `_, Karol Mroz) * rgw: stream metadata full sync init (`issue#18079 `_, `pr#12429 `_, Yehuda Sadeh) * rgw: submodule: update Beast to ceph/ceph-master branch (`pr#19182 `_, Casey Bodley) * rgw: switch beast frontend back to stackful coroutine (`issue#20048 `_, `pr#20449 `_, Casey Bodley) * rgw: sync tracing fixes (`issue#22833 `_, `pr#20191 `_, Yehuda Sadeh) * rgw: tenant fixes for dynamic resharding (`issue#22046 `_, `pr#18811 `_, Orit Wasserman) +* rgw,tests: fix s3atests that are failing for sometime (`pr#20678 `_, Vasu Kulkarni) +* rgw,tests: qa: fix overrides for openssl_keys task (`pr#20981 `_, Casey Bodley) * rgw,tests: qa: re enable LC tests (`pr#17020 `_, Abhishek Lekshmanan) * rgw,tests: qa/rgw: add beast frontend to some rgw suites (`pr#17977 `_, Casey Bodley) * rgw,tests: qa/rgw: combine swift, s3tests, ragweed into single verify task (`pr#20756 `_, Casey Bodley) @@ -2304,14 +2815,17 @@ Other Notable Changes * rgw,tests: qa/rgw: hadoop-s3a suite targets centos_latest (`pr#17777 `_, Casey Bodley) * rgw,tests: qa/rgw: ignore errors from 'pool application enable' (`issue#21715 `_, `pr#18193 `_, Casey Bodley) * rgw,tests: qa/rgw: remove some civetweb overrides for beast testing (`issue#23002 `_, `pr#20440 `_, Casey Bodley) +* rgw,tests: qa/rgw: renamed ssl task to openssl_keys (`pr#20863 `_, Ricardo Dias) * rgw,tests: qa/rgw: use 'ceph osd pool application enable' on created pools (`pr#17162 `_, Casey Bodley) * rgw,tests: qa/rgw: verify suite tests civetweb with ssl (`pr#20444 `_, Casey Bodley) * rgw,tests: qa/smoke: add rgw crypto config for s3tests (`pr#17700 `_, Casey Bodley) +* rgw,tests: qa/tasks/swift: add support for the "force-branch" configurable (`pr#21027 `_, Radoslaw Zarzynski) * rgw,tests: rgw, qa: integrate Tempest to verify RadosGW's compliance with Swift API (`pr#16344 `_, Radoslaw Zarzynski) * rgw,tests: test/rgw: fix test_encrypted_object_sync for 3+ zones (`pr#17377 `_, Casey Bodley) * rgw: the metavariables in frontends-related config won't be expanded (`pr#19689 `_, root) * rgw,tools: tools/rgw: add script to inspect admin socket "cr dump" (`pr#15554 `_, Casey Bodley) * rgw: Torrents are not supported for objects encrypted using SSE-C (`issue#21720 `_, `pr#17956 `_, Zhang Shaowen) +* rgw: trim all spaces inside a metadata value (`issue#23301 `_, `pr#20841 `_, Orit Wasserman) * rgw: udpate radosgw-admin usage with bi purge (`pr#18245 `_, Yao Zongyou) * rgw: unlink deleted bucket from bucket's owner (`issue#22248 `_, `pr#20017 `_, Casey Bodley) * rgw: unreachable return in RGWRados::trim_bi_log_entries (`pr#17367 `_, Amit Kumar) @@ -2329,57 +2843,81 @@ Other Notable Changes * rgw: Various cleanups and options update in rgw_admin.cc (`pr#18302 `_, Jos Collin) * rgw: vstart.sh: fix mstop.sh can not stop rgw (`pr#17438 `_, Jiaying Ren) * rgw: 'zone placement' commands validate compression type (`issue#21775 `_, `pr#18273 `_, Casey Bodley) +* rocksdb: sync with upstream (`issue#21603 `_, `pr#18262 `_, Kefu Chai) +* rpm: rm macros in comments (`issue#22250 `_, `pr#17070 `_, Ken Dreyer) +* script/build-integration-branch: check errors (`pr#17578 `_, Sage Weil) +* script/build-integration-branch: python3 compatible and pep8 clean (`pr#18035 `_, Kefu Chai) +* scripts: new backport-create-issue script (`pr#21480 `_, Nathan Cutler) +* selinux: Allow ceph to execute ldconfig (`pr#21974 `_, Boris Ranto) +* selinux: Allow getattr on lnk sysfs files (`pr#17891 `_, Boris Ranto) * spdk: advance to upstream dc82989d (`pr#20713 `_, Nathan Cutler) +* src: fix various log messages (`pr#21112 `_, Gu Zhongyan) * src/msg/rdma: fixes failure on assert in notify() (`pr#17007 `_, Alex Mikheev) -* tests: assert check for negative returns (`pr#17296 `_, Amit Kumar) +* suites/cephmetrics: Add Centos 7 (`pr#18594 `_, Zack Cerza) +* test: assert check for negative returns (`pr#17296 `_, Amit Kumar) +* test/fio: generate db histogram to help debug rocksdb performance (`pr#16808 `_, Pan Liu, Xiaoyan Li) +* test: fix bash path in shebangs (part 2) (`pr#17955 `_, Alan Somers) +* test: fix CLI unit formatting tests (`pr#22260 `_, Jason Dillaman) +* test: Incorrect conversion to double (`pr#18963 `_, Amit Kumar) +* test/librados: reorder ASSERT_EQ() arguments (`pr#16625 `_, Yan Jun) +* test,osd,kvstore_tool: silence warnings and prepare test buffer in the right way (`pr#18406 `_, Adam C. Emerson) * tests: bluestore/fio: Fixed problem with all objects having the same hash (`pr#17770 `_, Adam Kupczyk) * tests: CentOS 7.4 is now the latest (`pr#17776 `_, Nathan Cutler) -* tests: ceph-ansible vars additions (`issue#21822 `_, `pr#18378 `_, Yuri Weinstein) +* tests - ceph-ansible vars additions (`issue#21822 `_, `pr#18378 `_, Yuri Weinstein) * tests: ceph-disk: ignore E722 in flake8 test (`issue#22207 `_, `pr#19072 `_, Nathan Cutler) * tests: ceph-disk: mock get fsid (`pr#19254 `_, Kefu Chai) * tests: ceph-disk: Remove sitepackages=True (`issue#22823 `_, `pr#20151 `_, Brad Hubbard) * tests: ceph-objectstore-tool: don't destroy SnapMapper until the txn is completed (`issue#23121 `_, `pr#20593 `_, Kefu Chai) * tests: Changes required for teuthology's systemd support (`pr#18380 `_, Zack Cerza) +* tests: Check for empty output in test_dump_pgstate_history (`pr#20838 `_, Brad Hubbard) +* tests: cleanup: drop calamari tasks (`pr#17531 `_, Nathan Cutler) * tests: cleanup: drop upgrade/jewel-x/point-to-point-x (`issue#22888 `_, `pr#20245 `_, Nathan Cutler) * tests: cmake,test/mgr: restructure dashboard tests and cmake related fixes (`pr#20768 `_, Kefu Chai) * tests: common/obj_bencher: set {min,max}_iops if runtime < 1 sec (`pr#17182 `_, Kefu Chai) * tests: c_read_operations.cc: Silence tautological-compare compiler warning (`pr#19953 `_, Brad Hubbard) -* tests: Decreased amount of jobs on master, kraken, luminous runs (`pr#17069 `_, Yuri Weinstein) -* tests: fix bash path in shebangs (part 2) (`pr#17955 `_, Alan Somers) * tests: fix uninitialized value found by coverity scan (`pr#17895 `_, J. Eric Ivancich) -* tests: Incorrect conversion to double (`pr#18963 `_, Amit Kumar) * tests: Increase sleep in test_pidfile.sh (`pr#17052 `_, David Zafman) * tests: librgw_file: remove unused `using` statement (`pr#17085 `_, Yao Zongyou) * tests: mark_unfound_lost fix and some other minor changes (`issue#21907 `_, `pr#18449 `_, David Zafman) +* tests: mgr/dashboard: Allow sourcing `run-backend-api-tests.sh` (`pr#20874 `_, Sebastian Wagner) +* tests: mgr/dashboard: create venv for running tox (`pr#21490 `_, Kefu Chai) +* tests: mgr/dashboard: notification queue: fix priority tests (`pr#21147 `_, Ricardo Dias) +* tests: mimic: qa: fix test on "ceph fs set cephfs allow_new_snaps" (`pr#21830 `_, Kefu Chai) +* tests: mimic: qa/workunits/rados/test_envlibrados_for_rocksdb: install g++ not g++-4.7 (`pr#22117 `_, Kefu Chai) +* tests: mimic: test: Need to escape parens in log-whitelist for grep (`pr#22075 `_, David Zafman) +* tests: mimic: test: wait_for_pg_stats() should do another check after last 13 secon… (`pr#22199 `_, David Zafman) * tests: misc: Fix bash path in shebangs (`pr#16494 `_, Alan Somers) * tests: mstart.sh: remove bashizm in /bin/sh script (`pr#18541 `_, Mykola Golub) * tests: point-to-point-x: upgrade client.1 to -x along with cluster nodes (`issue#21499 `_, `pr#17910 `_, Nathan Cutler) -* tests: qa: add cbt repo parameter (`pr#18543 `_, Neha Ojha) +* tests: qa: add cbt task for performance testing (`pr#17583 `_, Neha Ojha) +* tests: qa: add cosbench workloads and override teuthology default settings (`pr#21710 `_, Neha Ojha) +* tests/qa: Adding $ distro mix - rgw (`pr#22070 `_, Yuri Weinstein) +* tests/qa: adding rados/.. dirs (`pr#22068 `_, Yuri Weinstein) * tests: qa: add "restful" to ceph_mgr_modules in ceph-ansible suite (`pr#18634 `_, Kefu Chai) * tests: qa: add simple and dirty script to find ports being used (`pr#19102 `_, Joao Eduardo Luis) -* tests: qa: add upgrade/luminous-x suite (`pr#17160 `_, Yuri Weinstein) * tests: qa: big: add openstack.yaml (`pr#16864 `_, Nathan Cutler) -* tests: qa/ceph-disk: enlarge the simulated SCSI disk (`issue#22136 `_, `pr#19199 `_, Kefu Chai) * tests: qa: clean up dnsmasq task and fix EPERM error (`pr#20680 `_, Casey Bodley) * tests: qa: create_cache_pool no longer runs 'pool application enable' (`issue#21155 `_, `pr#17312 `_, Casey Bodley) * tests: qa: decrease the msg_inject_socket_failures from 1/500 to 1/1000 (`issue#22093 `_, `pr#19542 `_, Kefu Chai) * tests: qa: disable mon-health-to-clog in upgrade test (`pr#19233 `_, Kefu Chai) +* tests: qa: disable -Werror when compiling env_librados_test (`pr#21433 `_, Kefu Chai) * tests: qa: do not "ceph fs get cephfs" w/o a cephfs fs (`pr#18533 `_, Kefu Chai) * tests: qa: do not wait for down/out osd for pg convergence (`pr#18808 `_, Kefu Chai) +* tests/qa - enabled `ceph-deploy` runs on `mira` nodes (`pr#21253 `_, Yuri Weinstein) * tests: qa: fix pool-quota related tests (`issue#21409 `_, `pr#17763 `_, xie xingguo) * tests: qa: Fix shebangs on openstack scripts (`pr#16546 `_, Alan Somers) -* tests: qa: install collectl with cbt task (`pr#19324 `_, Neha Ojha) -* tests: qa: mimic-dev1 backports to avoid trusty nodes (`pr#19600 `_, Kefu Chai) -* tests: qa: preserve cbt task results (`pr#19364 `_, Neha Ojha) +* tests: qa: reduce "mon client hunt interval max multiple" to 2 for all clients (`pr#21658 `_, Kefu Chai) * tests: qa: reduce mon-client-hunt-interval-max-multiple to 2 (`pr#18283 `_, Kefu Chai) * tests: qa: revert "qa: use config_path property instead of literal" (`pr#17850 `_, Patrick Donnelly) * tests: qa/run-standalone.sh: set PYTHONPATH for FreeBSD also (`pr#20646 `_, Kefu Chai) * tests: qa: s/backfill/backfilling/ (`pr#18235 `_, Kefu Chai) * tests: qa/stanalone: pass options using --= (`pr#19544 `_, Kefu Chai) * tests: qa/standalone: Add trap for signals to restore the kernel core pattern (`pr#17026 `_, David Zafman) +* tests: qa/standalone/ceph-helpers.sh: provide argument to dirname (`issue#23805 `_, `pr#21552 `_, Nathan Cutler) * tests: qa/standalone/ceph-helpers.sh: silence ceph-disk DEPRECATION_WARNING (`pr#19478 `_, Kefu Chai) * tests: qa/standalone: extract delete_pool() (`pr#20634 `_, Kefu Chai) * tests: qa/standalone: misc fixes (`issue#20465 `_, `issue#20921 `_, `pr#16709 `_, David Zafman) +* tests: qa/standalone/mon/misc.sh: Add osdmap-prune tests (`issue#23621 `_, `pr#21318 `_, Brad Hubbard) * tests: qa/standalone/osd/osd-mark-down: create pool to get updated osdmap faster (`pr#18191 `_, huangjun) * tests: qa/standalone: remove osd-map-max-advance related tests (`issue#22596 `_, `pr#19816 `_, Kefu Chai) * tests: qa/standalone: respect $TEMPDIR (`pr#17747 `_, Kefu Chai) @@ -2387,52 +2925,60 @@ Other Notable Changes * tests: qa/standalone/scrub/osd-scrub-repair.sh: drop omap_digest flag (`pr#18150 `_, xie xingguo, Sage Weil) * tests: qa/standalone: s/delete_erasure_pool/delete_erasure_coded_pool/ (`pr#20667 `_, Kefu Chai) * tests: qa: stop testing deprecated "ceph osd create" (`issue#21993 `_, `pr#18659 `_, Kefu Chai) +* tests: qa/suites: add minimal performance suite (`pr#21104 `_, Neha Ojha) +* tests: qa/suites/cephmetrics: Updates for new version (`pr#21146 `_, Zack Cerza) * tests: qa/suites: change fixed-2.yaml users to get 4 openstack disks (`pr#16873 `_, Sage Weil) * tests: qa/suites: mds.0 -> mds.a (`pr#20848 `_, Sage Weil) -* tests: qa/suites/rados/multimon/tasks/mon_lock_with_skew: whitelist PG (`pr#17004 `_, Sage Weil) +* tests: qa/suites/rados: Disable scrub backoff (`issue#23578 `_, `pr#21295 `_, Brad Hubbard) +* tests: qa/suites/rados/mgr/tasks/dashboard: add MDS_ALL_DOWN to whitelist (`pr#21549 `_, Ricardo Dias) +* tests: qa/suites/rados/mgr/tasks/dashboard_v2: add fail_on_skip = false (`pr#20925 `_, Ricardo Dias) * tests: qa/suites/rados/multimon: whitelist mgr down vs clock skew test (`pr#16996 `_, Sage Weil) -* tests: qa/suites/rados/perf: add optimized settings (`pr#17786 `_, Neha Ojha) -* tests: qa/suites/rados/perf: add workloads (`pr#18573 `_, Neha Ojha) * tests: qa/suites/rados/singleton: more whitelist (`pr#19225 `_, Kefu Chai) +* tests: qa/suites/rados/thrash-old-clients: ms_type=simple (`issue#23922 `_, `pr#21739 `_, Kefu Chai) * tests: qa/suites/rados/upgrade/jewel-x-singleton: tolerate sloppy past_intervals (`pr#17293 `_, Kefu Chai) -* tests: qa/suites/rados/verify/validater/valgrind: whitelist PG (`pr#17005 `_, Sage Weil) * tests: qa/suites/rest/basic/tasks/rest_test: more whitelisting (`issue#21425 `_, `pr#17794 `_, huangjun) * tests: qa/suites/rest/basic/tasks/rest_test: whiltelist OSD_DOWN (`issue#21425 `_, `pr#18144 `_, huangjun) * tests: qa/suites/upgarde/jewel-x/parallel: tolerate mgr warning (`pr#17203 `_, Sage Weil) * tests: qa/suites/upgarde/jewel-x/point-to-point-x: disable app warnings (`pr#16947 `_, Sage Weil) -* tests: qa/suites/upgrade/jewel-x/parallel: tolerate laggy mgr (`pr#17227 `_, Sage Weil) -* tests: qa/suites/upgrade/kraken-x: fixes (`pr#16881 `_, Sage Weil) +* tests: qa/suites: whitelist SLOW_OPS (`issue#23495 `_, `pr#21324 `_, Kefu Chai) +* tests: qa/tasks: Add default timeout for wait for pg clean task (`pr#21313 `_, Vasu Kulkarni) * tests: qa/tasks/ceph_deploy: gatherkeys before mgr deploy (`pr#17224 `_, Sage Weil) +* tests: qa/tasks/ceph_manager: use set_config on revived osd (`pr#20901 `_, Neha Ojha) +* tests: qa/tasks/mgr/dashboard: Fix login expires too soon (`pr#21021 `_, Sebastian Wagner) * tests: qa/tasks: prolong revive_osd() timeout to 6 min (`issue#21474 `_, `pr#17902 `_, Kefu Chai) * tests: qa/tasks: prolong revive_osd() timeout to 6 min (`issue#21474 `_, `pr#19024 `_, Kefu Chai) +* tests: qa/tasks: run cosbench using the CBT task (`pr#21656 `_, Neha Ojha) * tests: qa/tasks: update ceph-deploy task to use newer ceph-volume syntax (`pr#19244 `_, Vasu Kulkarni) * tests: qa/tests: Add additional required ceph-ansible vars due to upstream changes (`pr#17757 `_, Vasu Kulkarni) * tests: qa/tests: add ceph-deploy upgrade tests (`issue#20950 `_, `pr#16826 `_, Vasu Kulkarni) -* tests: qa/tests - Added options to use both cases: mon.a and installer.0 (`pr#19745 `_, Yuri Weinstein) * tests: qa/tests: add openstack volume info + lvs for ceph-volume (`pr#20243 `_, Vasu Kulkarni) * tests: qa/tests: Fix get_system_type failure due to invalid remote name (`pr#17650 `_, Vasu Kulkarni) * tests: qa/tests: fix rbd pool creation for systemd tests (`pr#17536 `_, Vasu Kulkarni) * tests: [qa/tests]: misc ceph-ansible fixes and udpate (`pr#17096 `_, Vasu Kulkarni) * tests: qa/tests/rados: Remove unsupported 2-size-1-min-size config (`pr#17576 `_, Vasu Kulkarni) +* tests: qa/tests: use ceph-deploy stable branch for single node tests (`pr#20979 `_, Vasu Kulkarni) * tests: qa/tests: Various whitelists for smoke suite (`issue#21376 `_, `pr#17680 `_, Vasu Kulkarni) * tests: qa/tests: Wip ceph deploy upgrade (`pr#17651 `_, Vasu Kulkarni) +* tests: qa/workunits/rados/test_large_omap_detection: Scrub pgs instead of OSDs (`pr#21410 `_, Brad Hubbard) * tests: qa/workunits: silence py warnings for ceph-disk tests (`issue#22154 `_, `pr#19075 `_, Kefu Chai) * tests: rados: Copy payload in ceph_perf_msgr_client (`issue#22100 `_, `pr#18862 `_, Jeegn Chen) * tests: rados: Intializing members class StriperTest (`pr#16843 `_, amitkuma) +* tests: remove TestPGLog ASSERT_DEATH test (`issue#23504 `_, `pr#21117 `_, Nathan Cutler) * tests: run-standalone.sh improve error message (`pr#17093 `_, David Zafman) * tests: run-standalone.sh skip core_pattern if already set (`pr#17098 `_, David Zafman) -* tests: suites/cephmetrics: Add Centos 7 (`pr#18594 `_, Zack Cerza) * tests: test/admin_socket_output: add --vstart=path/to/asok option (`pr#20371 `_, Kefu Chai) * tests: test/admin_socket_output: better error reporting (`pr#20409 `_, Brad Hubbard) * tests: test/admin_socket_output: switch to std::experimental::filesystem (`pr#20307 `_, Kefu Chai) +* tests: test/ceph_test_objectstore: make settings update and restore less error prone (`pr#21145 `_, Igor Fedotov) * tests: test: checking negative returns from creat() (`pr#18090 `_, amitkuma) * tests: test/CMakeLists: disable test_pidfile.sh (`issue#20975 `_, `pr#16977 `_, Sage Weil) * tests: test/CMakeLists: disable test-pidfile.sh (`pr#17401 `_, Sage Weil) * tests: test/coredumpctl: support freebsd (`pr#17447 `_, Kefu Chai) +* tests: test/dashboard: hardcode .coverage path to workaround tox bugs (`pr#21485 `_, Kefu Chai) +* tests: test/dashboard: specify workdir using tox.ini (`issue#23709 `_, `pr#21416 `_, Kefu Chai) * tests: test: Don't dump core when using EXPECT_DEATH (`pr#17390 `_, Kefu Chai) * tests: test/fio: extend fio objectstore plugin to better simulate OSD behavior (`pr#19101 `_, Igor Fedotov) * tests: test/fio: fix building of the fio_ceph_objectstore plugin (`pr#18332 `_, Radoslaw Zarzynski) -* tests: test/fio: generate db histogram to help debug rocksdb performance (`pr#16808 `_, Pan Liu, Xiaoyan Li) * tests: test: Fix and enable test_pidfile.sh (`issue#20770 `_, `pr#16987 `_, David Zafman) * tests: test: Fix ceph-objectstore-tool usage check (`pr#17785 `_, David Zafman) * tests: test: fix misc fiemap testing (`issue#21716 `_, `pr#18240 `_, Kefu Chai, Ning Yao) @@ -2440,18 +2986,23 @@ Other Notable Changes * tests: test: Initializing ChunkReadOp members (`pr#19334 `_, amitkuma) * tests: test/journal: Initialize member variable m_work_queue (`pr#17089 `_, amitkuma) * tests: test/librados: be more tolerant with timed lock tests (`issue#20086 `_, `pr#20161 `_, Kefu Chai) -* tests: test/librados: reorder ASSERT_EQ() arguments (`pr#16625 `_, Yan Jun) +* tests: test/librados: increase pgp_num along with pg_num (`issue#23763 `_, `pr#21555 `_, Kefu Chai) * tests: test/librados: s/invoke_result_t/result_of_t/ (`pr#20379 `_, Kefu Chai) * tests: test/librados_test_stub: pass snap context to zero op (`pr#17186 `_, Mykola Golub) * tests: test/log: fix for crash with libc++ (`pr#20233 `_, Casey Bodley) * tests: test: Make clearer by moving code out of loop (`pr#20759 `_, David Zafman) * tests: test/objectstore/test_bluefs: cleanups (`pr#17909 `_, Kefu Chai) +* tests: test: only test dashboard_v2 when it is enabled (`pr#20777 `_, Willem Jan Withagen) +* tests: test: only test enabled python bindings (`pr#21293 `_, Kefu Chai) * tests: test/osd: initialize Non-static class members in WeightedTestGenerator (`pr#15922 `_, Jos Collin) -* tests: test,osd,kvstore_tool: silence warnings and prepare test buffer in the right way (`pr#18406 `_, Adam C. Emerson) * tests: test/osd: Non-static class members not initialized in UnsetRedirectOp (`pr#15921 `_, Jos Collin) * tests: test/osd: silence warnings from -Wsign-compare (`pr#17027 `_, Jos Collin) * tests: test: put new BlueStore tests un ifdef WITH_BLUESTORE (`pr#20576 `_, Willem Jan Withagen) +* tests: test:qa:infra - Run update daily and use bash (`pr#21218 `_, David Galloway) +* tests: test:qa:infra - teuthology crontab items as of 3/27/18 (`pr#21075 `_, Yuri Weinstein) * tests: test: reduce the chance to have degraded PGs (`issue#22711 `_, `pr#20046 `_, Kefu Chai) +* tests: test: remove distro_version assert in distro detect test (`pr#21052 `_, Shengjing Zhu) +* tests: test: Replace bc command with printf command (`pr#21013 `_, David Zafman) * tests: test: silence warning from -Wsign-compare (`pr#17790 `_, Jos Collin) * tests: test: silence warnings from -Wsign-compare (`pr#17962 `_, Jos Collin) * tests: tests - Replaced requests for "centos 7.3" to centos_latest (`pr#19262 `_, Yuri Weinstein) @@ -2460,40 +3011,61 @@ Other Notable Changes * tests: test/throttle: kill tests exercising dtor of Throttle classes (`pr#17442 `_, Kefu Chai) * tests: test/unittest_bufferlist: check retvals of syscalls (`pr#18238 `_, Kefu Chai) * tests: test/unittest_pg_log: silence gcc warning (`pr#17328 `_, Kefu Chai) +* tests: test: Use jq in a compatible way and for easier diff analysis (`pr#21450 `_, David Zafman) +* tests: test: Whitelist corrections (`pr#22167 `_, David Zafman) * tests,tools: crushtool: print error message to stderr not dout(1) (`issue#21758 `_, `pr#18242 `_, Kefu Chai) * tests: unittest_crypto: Don't exceed limit for getentropy (`pr#18505 `_, Brad Hubbard) +* tests: vstart: fix initial start when there is no ceph.conf (`pr#21019 `_, Jianpeng Ma) +* The Day Has Come! (`pr#19657 `_, Adam C. Emerson) * tools: Align use of uint64_t in service_daemon::AttributeType (`pr#16938 `_, James Page) * tools: ceph-disk: erase 110MB for nuking existing bluestore (`issue#22354 `_, `pr#20400 `_, Kefu Chai) +* tools: ceph-disk: fix '--runtime' omission for ceph-osd service (`issue#21498 `_, `pr#17904 `_, Carl Xiong) +* tools: ceph-disk: fix signed integer is greater than maximum when call major (`pr#19196 `_, Song Shun) +* tools: ceph-disk: include output of failed command in exception (`pr#20497 `_, Kefu Chai) +* tools: ceph-disk: more precise error message when a disk is specified (`pr#18018 `_, Kefu Chai) * tools: ceph-disk: reduce the scope of activate_lock (`pr#20114 `_, zhaokun) +* tools: ceph-disk: retry on OSError (`issue#21728 `_, `pr#18162 `_, Kefu Chai) +* tools: ceph-disk: unlock all partitions when activate (`pr#17363 `_, Kefu Chai) * tools: ceph-disk: write log to /var/log/ceph not to /var/run/ceph (`pr#18375 `_, Kefu Chai) * tools: ceph: fixes for "tell .\*" command (`issue#21230 `_, `pr#17463 `_, Kefu Chai) +* tools: ceph-kvstore-tool: make it a bit more friendly (`pr#21477 `_, Sage Weil) * tools: ceph-kvstore-tool: use unique_ptr<> to manage the lifecycle of bluestore (`pr#18221 `_, Kefu Chai) * tools: ceph-objectstore-tool: Add option "dump-import" to examine an export (`issue#22086 `_, `pr#19368 `_, David Zafman) * tools: ceph-objectstore-tool: Fix set-size to clear data_digest if changing … (`pr#18885 `_, David Zafman) +* tools: ceph-objectstore-tool: "$OBJ get-omaphdr" and "$OBJ list-omap" scan all pgs instead of using specific pg (`issue#21327 `_, `pr#17985 `_, David Zafman) * tools: ceph-objectstore-tool: skip object with generated version (`pr#18507 `_, huangjun) * tools: ceph-syn: silence clang analyzer warning (`pr#18577 `_, Kefu Chai) * tools: ceph-volume: Use a delimited CLI output parser instead of JSON (`pr#17097 `_, Alfredo Deza) * tools: cleanup: rip out ceph-rest-api (`issue#21264 `_, `issue#22457 `_, `pr#17530 `_, Nathan Cutler) +* tools: correct total size formatting (`pr#21641 `_, Zheng Yin) * tools: crushtool: add --add-bucket and --move options (`pr#20183 `_, Kefu Chai) +* tools: FreeBSD basic getopt does not use --options (`pr#21148 `_, Willem Jan Withagen) * tools: Initialization of \*server, command variables (`pr#17135 `_, amitkuma) * tools: make rados get/put/append command help txt clear (`issue#22958 `_, `pr#20363 `_, lvshuhua) * tools: Modify "rados df" header's alignment (`pr#17549 `_, iliul) * tools: rados add a cli option to clear omap keys (`issue#22255 `_, `pr#19180 `_, Abhishek Lekshmanan) * tools: rados/tool: fixup rados stat command hint (`pr#16983 `_, huanwen ren) +* tools: script: build-integration-branch: avoid Unicode error (`issue#24003 `_, `pr#21918 `_, Nathan Cutler) * tools: script: ceph-release-notes: minor fixes for split_component (`pr#16605 `_, Abhishek Lekshmanan) +* tools: Special scrub handling of hinfo_key errors (`issue#23428 `_, `issue#23364 `_, `pr#20947 `_, David Zafman) * tools: src/vstart.sh: default os to filestore for FreeBSD (`pr#17454 `_, xie xingguo) +* tools: stop.sh: add ceph configure file location (`pr#20888 `_, Jianpeng Ma) * tools: tools/ceph-conf: dump parsed config in plain text or as json (`issue#21862 `_, `pr#18350 `_, Piotr Dałek) * tools: tools/ceph_monstore_tool: include mgrmap in initial paxos epoch (`issue#22266 `_, `pr#19780 `_, Kefu Chai) * tools: tools/ceph_monstore_tool: rebuild initial mgrmap also (`issue#22266 `_, `pr#19238 `_, Kefu Chai) * tools: tools/ceph-objectstore-tool: command to trim the pg log (`issue#23242 `_, `pr#20786 `_, Josh Durgin, David Zafman) * tools: tools/ceph_objectstore_tool: fix 'dup' unable to duplicate meta PG (`pr#17572 `_, xie xingguo) +* tools: tools/rados: improve the ls command usage (`pr#21553 `_, Li Wang) * tools: tools: rados: make -f be --format for consistency with ceph tool (`issue#15904 `_, `pr#20147 `_, Nathan Cutler) * tools: tools/rados: use the monotonic clock in rados bench (`issue#21375 `_, `pr#18588 `_, Mohamad Gebai) * tools: update monstore tool for fsmap, mgrmap (`issue#21577 `_, `pr#18005 `_, John Spray) -* tools: vstart.sh: fix a typo (`pr#18729 `_, iliul) +* tools: Use --no-mon-config so ceph_objectstore_tool.py test doesn't hang (`pr#21274 `_, David Zafman) * tools: vstart.sh: move rgw configuration to client.rgw section (`pr#18331 `_, Yan Jun) -* tools: vstart.sh: quote cmd params when display executing cmd (`pr#17057 `_, Jiaying Ren) -* tools: vstart.sh: quote command only when necessary (`pr#18181 `_, Kefu Chai) -* tools: vstart.sh: should quote the parameters to get them quoted (`pr#18523 `_, Kefu Chai) -* tools: vstart.sh: simplify the objectstore related logic (`pr#17749 `_, Kefu Chai) * tools: vstart.sh: use bluestore as default osd objectstore backend (`pr#17100 `_, mychoxin) +* vstart: fix option (due to quotes) and allow disabling dashboard (`issue#23345 `_, `pr#20986 `_, Joao Eduardo Luis) +* vstart.sh: fix a typo (`pr#18729 `_, iliul) +* vstart.sh: Fix help text in vstart.sh (`pr#21071 `_, Marc Koderer) +* vstart.sh: quote cmd params when display executing cmd (`pr#17057 `_, Jiaying Ren) +* vstart.sh: quote command only when necessary (`pr#18181 `_, Kefu Chai) +* vstart.sh: should quote the parameters to get them quoted (`pr#18523 `_, Kefu Chai) +* vstart.sh: simplify the objectstore related logic (`pr#17749 `_, Kefu Chai)