Sage Weil [Mon, 7 Oct 2019 20:31:14 +0000 (15:31 -0500)]
Merge PR #30603 into master
* refs/pull/30603/head:
ceph-daemon: -n type.id instead of -i id
ceph-daemon: drop unused VERSION
ceph-daemon: clean up dir helpers, tighten up permissions
ceph-daemon: fchmod before writing to keyring file
test_ceph_daemon.sh: skip ssh until container image has remoto
ceph-daemon: decode utf-8 in run() helper
mgr/ssh: clean up debug cruft
mgr/ssh: clean up bare except: block
ceph-daemon: clean up bare except: blocks
ceph-daemon: all imports to top
ceph-volume: no_tmpfs -> tmpfs
doc/bootstrap: add new bootstrap documentation
ceph-daemon: add --output-pub-ssh-key for bootstrap
ceph-daemon: make 'shell' easier to use
ceph-daemon: support docker; prefer podman
qa: add ceph-daemon
debian: ceph-daemon package, required by ceph-mgr-ssh
ceph.spec.in: ceph-daemon package, required by ceph-mgr
common/options: cleanup whitespace
mgr/ssh: simplify getting the cluster fsid
mgr/ssh: pipe ceph-daemon script to stdin of python3
ceph-daemon: add support for args and/or stdin from top of script
ceph-daemon: make ceph-volume use get_config_and_keyring
ceph-daemon: ls: behave if /var/log/ceph doesn't exist
ceph-daemon: implement 'adopt' for legacy style daemons
ceph-daemon: fix fsid detection for legacy osds
ceph-daemon: make rm-cluster clean up system-ceph*.slice too
ceph-daemon: configure ssh orchestrator
ceph-daemon: be more restrictive with file permissions
mgr/ssh: create osd with ceph-daemon
mgr/ssh: pass daemon id separately to _create_daemon
ceph-daemon: add --config-and-keyring to ceph-volume command
ceph-daemon: create log path for shell (if needed)
mgr/ssh: use _run_ceph_daemon for _create_daemon
mgr/ssh: factor _run_ceph_daemon out of _get_device_inventory
mon/ConfigMonitor: allow entity type only for 'config get'
ceph-daemon: add ceph-volume subcommand
ceph-daemon: remove unused CephContainer dname property
ceph-daemon: drop useless uid/gid checks
mgr/ssh: deploy new mgrs with ceph-daemon
mgr/ssh: factor _create_daemon out of create_mon
mon/MonCap: allow mgr to create new auth keys
mgr/ssh: run c-v with podman when getting inventory
mgr/ssh: simplify ssh connection management
mgr/ssh: use ceph-daemon for deploying mon
ceph-daemon: allow --mon-network for deploying new mon (vs specifying IP)
ceph-daemon: --config-and-keyring (not key)
common/options: add 'image' config option
test_ceph_daemon: specify image name
vstart.sh: add --ssh to enable+configure ssh orchestrator
mgr/ssh: use ssh identity from config-key, if present
mgr/ssh: hardcode default ssh_config
ceph-daemon: store ssh identity in mon config-key store
ceph-daemon: --privileged arg for 'exec'
ceph-daemon: make deploy work for osd (do a c-v prepare)
ceph-daemon: make shell privileged
ceph-daemon: move get_container_mounts to a helper
ceph-daemon: pass full path for entrypoint
ceph-daemon: make id portion of 'shell' optional
ceph-volume: accept --no-tmpfs argument for bluestore
ceph-daemon: 'unit' command
ceph-daemon: fix run command to use call(), not check_output()
src/ceph-daemon: whitespace
ceph-daemon: add 'enter', 'exec' commands
ceph-daemon: bind config to default location
test_ceph_daemon.sh: test deploy mds too
ceph-daemon: generate ssh keys
ceph-daemon: --config, not --conf
ceph-daemon: long lines
ceph-daemon: add --config to bootstrap
ceph-daemon: add 'shell' command
ceph-daemon: do not import subprocess symbols directly
ceph-daemon: add mons with 'deploy mon.x ...'
ceph-daemon: add 'ls'
ceph-daemon: simplify uid/gid a bit
ceph-daemon: fix libudev
ceph-daemon: autodetect uid/gid from container image
ceph-daemon: default to empty log files, log to stderr (systemd journal)
ceph-daemon: rm-{daemon,cluster}
ceph-daemon: fix bootstrap config
ceph-daemon: fix args.fsid usage
ceph-daemon: be careful overwriting live files
ceph-daemon: slurp some options over from the standard systemd unit
ceph-daemon: add ceph.target and ceph-$fsid.target units
test_ceph_daemon.sh: stupid test script
ceph-daemon: bootstrap and deploy (mgr) work
ceph-daemon: initial checkin
ceph-mon: fix debug print of public_addr
Sage Weil [Sun, 6 Oct 2019 14:08:28 +0000 (09:08 -0500)]
Merge PR #30217 into master
* refs/pull/30217/head:
crimson: common/admin_socket kludge so that it builds
mon/MonClient: fix sending mon command to a specific rank
src/.gitignore: ignore .tox
mon/MonClient: interpret numeric mon target name as rank
mgr,mgr/MgrClient: use fsid to signal mon-mgr vs cli MCommands
qa/workunits/cephtool: fix errpr checks for 'ceph daemon' commands
common/ceph_context: make 'config unset' idempotent
qa/tasks/dump_stuck: mon.a, not mon.0
qa/suites/rados/singleton/all/admin-socket: fix test
common/config: EPERM setting config option after startup
qa/workunits/cephtool/test.sh: fix tell output error check
common/admin_socket: pass Formatter from generic infrastructure
common/admin_socket: pass ostream to call() for error output
os/bluestore: fix asok hook return value
rgw: fix asok return value
common/ceph_context: return error code from asok commands
test/pybind/test_rados: fix accidental mon tell test
mon: print entity_name along with caps to debug log
PendingReleaseNotes: notes about asok changes
mgr/MgrClient: empty target string for 'tell' means active mgr
common/admin_socket: report error code as part of output string
osd: change trigger_[deep_]scrub tommands to a pg tell command
osd: remove old command workqueue, threadpool
osd: drop MMonCommand handling
osdc/Objecter: resend OSD tell commands on EAGAIN
osd: route tell commands to asok; migrate commands
osd: use unique_ptr<Formatter> for asok_command
common/ceph_context: add generic asok 'injectargs'
common/admin_socket: allow dup prefixes
common/admin_socket: refactor with sync and async execute_command variants
common/admin_socket: pass input bufferlist
osd: transition to call_async() for asok
common/admin_socket: support alternative call_async()
mon/MonClient: send tell commands out of band via MCommand
mon: accept tell commands via MCommand and send them to asok handler
common/admin_socket: return int from hook call()
mgr/DaemonServer: route MCommand (for octopus+) to asok commands
do not use 'ceph tell mgr'
pybind/ceph_argparse: disambiguate mgr tell and CLI commands
ceph: make 'ceph tell mgr.*' send to the active mgr
ceph: send 'ceph tell mgr.X' to the right mgr
librados: add rados_mgr_command_target
mgr/MgrClient: add start_command variant that takes a target
common/admin_socket: drop unregister_command(); use per-hook variant
common/admin_socket: drop explicit prefix arg to register_command
common/admin_socket: simplify command routing
common/admin_socket: add ability to process MCommand via asok queue
common/admin_socket: pass cmdvec to execute_command
common/admin_socket: use pipe for general wakeup
include/compat: add flags arg to pipe_cloexec
common/admin_socket: drop unused args
Sage Weil [Fri, 4 Oct 2019 21:59:24 +0000 (16:59 -0500)]
ceph-daemon: clean up dir helpers, tighten up permissions
- don't pass args.{data,log}_dir to get_{data,log}_dir
- pass uid, gid, and mode to makedirs
- add make_data_dir and make_log_dir helpers, that optionally take uid/gid
- add make_data_dir_base that requires uid/gid
- use standard data and log dir modes, defined at top of file (0o700 for
data, 0o770 for logs). The data dir mode applies both to the fsid
directory for the whole cluster and to each daemon's subdirectory; the
log mode applies only to the fsid dir for the whole cluster, where
all daemon logs are combined together in one directory.
Sage Weil [Sat, 5 Oct 2019 01:42:07 +0000 (20:42 -0500)]
Merge PR #29035 into master
* refs/pull/29035/head:
mgr/pg_autoscaler: changes made reflect jdurgin's request
mgr/pg_autoscaler: current pg_num compared to distance between the last initial_pg_num
mgr/progress & mgr/pg_autoscaler: changes reflect liewegas' comment
mgr/pg_autoscaler: get rid of white space
mgr/progress: change threshold value to origin
mgr/progress: cleaning up for pg_autoscaler
mgr/progress: Added Pg Autoscaler Event
Sage Weil [Thu, 3 Oct 2019 21:13:47 +0000 (16:13 -0500)]
mgr/ssh: pipe ceph-daemon script to stdin of python3
This avoids any need for the script to be present on the remote host.
We introduce a config option to indicate where the script should be
read from, since the location varies between a vstart environment (source
dir) and a real install (/usr/sbin).
Sage Weil [Wed, 2 Oct 2019 21:19:14 +0000 (16:19 -0500)]
ceph-daemon: implement 'adopt' for legacy style daemons
Caveats:
- this doesn't clean out /etc/ceph/*, since we don't know which is the
last daemon to go, and/or whether the user wants to keep it around
for using the ceph CLI on this host
- leaves behind /var/lib/ceph/bootstrap-* keys, even after all daemons
have been converted.
Sage Weil [Tue, 1 Oct 2019 16:34:22 +0000 (11:34 -0500)]
mon/ConfigMonitor: allow entity type only for 'config get'
e.g., 'ceph config get osd debug_osd' to return the config value that
would apply to a generic OSD (either from the osd or global section of
the config).
Sage Weil [Tue, 1 Oct 2019 15:29:53 +0000 (10:29 -0500)]
mgr/ssh: simplify ssh connection management
- Use a single instance of the config and identity files for the whole
module. There's no need to create these for *every* connection--it just
pollutes /tmp.
- Drop the SSHConnection wrapper, since the temp files are tied to the
daemon lifecycle now.
- Prefix the tmp files so I can tell wtf is going on.
- Always connect to root@host, to avoid remoto's localhost detection
feature. This ensures we have a consistent connection model and user.
(The daemon might be running as user ceph and try to connect to localhost,
but end up running the command as the wrong user and/or inside the
ceph-mgr container.)
Sage Weil [Fri, 4 Oct 2019 14:49:36 +0000 (09:49 -0500)]
osd/PeeringState: base lease support checks on features, not require_osd_release
The require_osd_release value will change at an arbitrary point in time,
midway through a peering interval. That can leave us in a state where
we are midway through an interval and suddenly enforcing checks on leases
we don't have.
In contrast, the upacting features change at an interval boundary, which
lets us adjust our behavior for the whole interval.
Sage Weil [Mon, 30 Sep 2019 15:58:40 +0000 (10:58 -0500)]
mon/MonClient: interpret numeric mon target name as rank
This allows us to behave when a rank is passed to mon_command(..., target=),
which will call rados_mon_command_target() -> MonClient::start_mon_command
with a (string) target name.
We could make an integer variant of rados_mon_command_target, and do the
int vs string differentiation in python, but this is much easier.
Sage Weil [Fri, 27 Sep 2019 21:01:44 +0000 (16:01 -0500)]
mgr,mgr/MgrClient: use fsid to signal mon-mgr vs cli MCommands
We can't use the feature bit for the MCommand connection to tell whether
it is a tell or CLI command because new clients may have to send CLI
commands via MCommand for old clusters, and they don't always know whether
this mgr is new or old yet.
Prior to octopus, MCommand contained a mon/mgr CLI command, and did not
have the fsid field set. Start populating the fsid field, and use this
to signal whether a client is a new MgrClient that knows MCommand vs
MMgrCommand. If we get an MCommand with the fsid set, that means it is
a tell command; otherwise, it's an old client sending a CLI command.
projects / ceph.git / log