crimson/os/seastore/cache: fine-grained lru cache control with GC
GC transaction is not sourced by user behaviors, so the extent read
operations from GC transaction don’t satisfy the time locality
principle. These extents should not be added to LRU cache.
Image contexts are reopen even though we pass the context as an
argument. This commit changes that so you can forget about reopening
a rbd image context again.
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
mgr/dashboard: add rbd list search and disable sorting
- Disable sorting in each column because it will not be possible to
sort with this pagination implementation.
- Add search capabilities to the rbd list pagination endpoint.
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
Yin Congmin [Fri, 24 Jun 2022 16:11:18 +0000 (00:11 +0800)]
cmake: add ndctl and daxctl to build pmdk
In order to enable the pmem character device, add ndctl=y to the parameter
of compiling the pmdk library when WITH_BLUESTORE_PMEM is ON. Because
after find_ndctl and find_daxctl are added, the dependency packages
required by WITH_BLUESTORE_PMEM and WITH_RBD_RWL become more different.
So separated these two. libpmem has no version required. libpmemobj
required version >=1.8.
Yin Congmin [Fri, 13 May 2022 12:47:07 +0000 (20:47 +0800)]
cmake: add findndctl and finddaxctl function
In order to support the character device of pmem usage in bluestore via
libpmem built by Ceph itself, we need to enable daxctl and ndctl
dependency. add the installation of ndctl and find it. the version of
ndctl and daxctl library requires >63. "apt-get install" meet the version
under ubuntu focal.
the installation of ndctl-devel in ceph.spec.in has not been verified.
This PR adds a section to the Developer Guide chapter
"Essentials" that explains what Dependabot is. This
section is adapted from an email from Ernesto Puerta
to the CLT that was sent on 08 Jul 2022.
luo rixin [Mon, 11 Jul 2022 02:28:58 +0000 (10:28 +0800)]
tools/crimson/perf_crimson_msgr: init ConfigPorxy when pref_crimson_msgr start running
While constructing socket connections, there are some config items needed(
`ceph::msgr::v2::FrameAssembler` need the value of `common::local_conf()->ms_crc_data`,
Access `common::local_conf()->ms_crc_data` will cause SIGSEGV without the ConfigProxy init).
So adding ConfigPorxy init before pref_crimson_msgr start running.
Fixes: https://tracker.ceph.com/issues/56520 Signed-off-by: luo rixin <luorixin@huawei.com>
TransactionManager::get_extents_if_live should return a list of
extents that are located in range paddr~len. When SegmentCleaner
invokes get_extents_if_live, the target extent may have been split into
multiple pieces by other transaction, so only search the paddr as key
will lose other pieces need to be rewritten.
Signed-off-by: Zhang Song <zhangsong325@gmail.com>
cmake: link librados applications against ceph-common
to address link failures like:
[100%] Linking CXX executable ../../../bin/unittest_global_doublefree
/opt/rh/gcc-toolset-12/root/usr/bin/ld: /opt/rh/gcc-toolset-12/root/usr/lib/gcc/x86_64-redhat-linux/12/libstdc++_nonshared.a(sstream-inst80.o): undefined reference to symbol '_ZTVNSt7__cxx1119basic_ostringstreamIwSt11char_traitsIwESaIwEEE@@GLIBCXX_3.4.21'
/opt/rh/gcc-toolset-12/root/usr/bin/ld: /usr/lib64/libstdc++.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
this happens when using gcc-toolset to build the tree.
because neither librados.so nor libcephfs exposes libstdc++ symbols
to executable linking against it. while CMake uses "c++" to link
C++ executables. the "c++" executable comes from GTS links the C++
executables agaist
/opt/rh/gcc-toolset-12/root/usr/lib/gcc/x86_64-redhat-linux/12/libstdc++.so,
which in turn is a ld script:
```
$ cat /opt/rh/gcc-toolset-12/root/usr/lib/gcc/x86_64-redhat-linux/12/libstdc++.so
/* GNU ld script
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf64-x86-64)
INPUT ( /usr/lib64/libstdc++.so.6 -lstdc++_nonshared )
```
but the thing is, stdc++_nonshared references some symbols
provided by libstdc++.so.6, and it is listed before it. that's
why "ld" is not able to resolve the referenced symbols used by
the executable, despite that they are provided by libstdc++ in
this case.
in this change, ceph-common is added to the linkage of executables
linked against librados and/or libcephfs, even the executables
in question does not reference ceph-common symbols. unlike librados,
libcephfs and librgw, ceph-common is an internal library, which does
not hide *any* symbols from its consumer, it is also able to provide
symbols from C++ standard library linked by it. so, in our case,
we can link the C++ executables against ceph-common for accessing
the C++ standard library. the reason why we don't link aginst libstdc++
explictly is that, we should leave this to the C++ compiler instead of
referencing a specific C++ standard library explictly by its name.
what if user wants to link against libc++ instead of libstdc++?
another fix could be to remove '-Wl,--as-needed' linker options
from the command line linking the librados applications, so the linker
does not ignore the symbols from libstdc++ when resolving the ones
referenced by stdc++_nonshared, but that would be complicated.
please note, linking against ceph-common does not change the linkage
of
* Ceph executables compiled using non-gcc-toolset toolchain, because we
always pass '-Wl,--as-needed' to "c++" when linking executables,
so "ld" should be able to drop ceph-common even we instruct it
to link against ceph-common. so it would be a no-op in this case.
* 3rd party librados executables compiled using non-gcc-toolset toolchain,
but linked against librados compiled using gcc-toolset toolchain.
because they still link against the /usr/lib64/libstdc++.so.6, when
these executables are compiled and linked. and librados is always
able to access libceph-common. so librados is safe.
Even though box is a rootless podman container, there are some
capabilities that are need like SYS_ADMIN, NET_ADMIN and SYS_TIME.
In rootless podman chronyd is not capabable of using adjtimex syscall
so we had to add -x flag to chronyd entrypoint which states it will
not try to call that function.
To test run `./box.py -v cluster start`.
To fix: There are some problems with /dev permissions so osds cannot be
deployed still. `./box.py cluster start --expanded` won't fully work.
Details:
- Dockerfile:
- Fedora image instead of centos.
- Tons of basic packages aren't installed, therefore, those are
explicitly installed.
- Some container folders are created. It isn't clear/tested whether
those are needed.
- .box_container is an empty file to identify if a command is ran on
containers (see box.py for usage).
- box.py
- podman-compose is missing some features but it compensates with args
as --podman-run-args so you can add some extra commands whenever
podman-compose calls podman run.
- sudo ./box.py cluster start will exit since it can be dangerous.
- Added some sed trickeroo to modify chronyd options. (I don't know
how can I pass OPTIONS to it :P)
- host.py
- setup_ssh now is need on every box container.
- util.py
- rice command printing :3.
- run multiple commands with a multiline string.
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
`lsblk_all()` should return an empty dict `{}` if nothing was found.
If we raise `RuntimeError()` then the loop in `scan.Scan.main` will stop
and make ceph-volume fails because we don't try to catch this exception.
`scan.Scan.main()` has its own logic in order to detect the given path
is a ceph-disk created OSD anyway.
Laura Flores [Thu, 7 Jul 2022 17:15:10 +0000 (12:15 -0500)]
.github/workflows: run the stale bot every hour
Currently, the stale bot runs once a day at 1:30 UTC.
This will only process a few PRs per day, which is not
enough to handle the volume of PRs that come into the
Ceph repository. Running the bot once every hour with
a limit of 30 operations will keep within the rate
limit, but it will also process more PRs per day.
`cephadm` started passing this argv which caused the
problem reported by Li, Jianxin <jianxin1.li@intel.com>.
See:
* https://gist.github.com/rzarzynski/4d1225971b6c28758cb2b68fbda3bf5f?permalink_comment_id=4223998#gistcomment-4223998
* https://docs.ceph.com/en/octopus/cephadm/drivegroups/
doc/cephfs/dirfrags: clarify the unit of threshold limits
Rationale: There are many threshold limits for split and
merge in this doc that just says like:
"A directory fragment is eligible for splitting
when its size exceeds `mds_bal_split_size`
(default 10000)". Need to clarify what 10000 actually
means. This applies to all other such entries in this
doc.
projects / ceph.git / log