]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agoqa: kernel.sh: unlock before rolling back 32838/head
Ilya Dryomov [Wed, 4 Dec 2019 18:08:46 +0000 (19:08 +0100)]
qa: kernel.sh: unlock before rolling back

"rbd snap rollback" expects an unlocked image, but we may get there
locked if object map is enabled (or if lock_on_read is specified in
rbd_default_map_options).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 3aae140353a4b27dfe6e2d2ac7a49cdcdbbe5ffe)

5 years agoqa: krbd_exclusive_option.sh: update for recent kernel changes
Ilya Dryomov [Wed, 4 Dec 2019 14:26:54 +0000 (15:26 +0100)]
qa: krbd_exclusive_option.sh: update for recent kernel changes

Since 5.3:
- a plain "rbd map" acquires the lock, so it's not different from
  "rbd map -o exclusive" in this regard
- if the lock is held by the exclusive peer, I/O is failed right away
  instead of blocking
- lock_timeout option is respected only by "rbd map" and not by I/O

Since 5.5:
- if the mapping is read-only, the lock isn't acquired

Added blacklisting test case, dropped lock_timeout test case.

Fixes: https://tracker.ceph.com/issues/43127
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 50933b863a1d239bbca0a25663c0dbe7ab8963ce)

5 years agoMerge pull request #32245 from smithfarm/wip-41106-nautilus
Yuri Weinstein [Fri, 24 Jan 2020 16:06:55 +0000 (08:06 -0800)]
Merge pull request #32245 from smithfarm/wip-41106-nautilus

nautilus: mds: add command that config individual client session

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #32746 from jan--f/wip-43026-nautilus
Nathan Cutler [Fri, 24 Jan 2020 15:51:07 +0000 (16:51 +0100)]
Merge pull request #32746 from jan--f/wip-43026-nautilus

nautilus: doc/ceph-volume: initial docs for zfs/inventory and zfs/api

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
5 years agoMerge pull request #32469 from zhengchengyao/diff_nautilus
Yuri Weinstein [Fri, 24 Jan 2020 13:53:27 +0000 (05:53 -0800)]
Merge pull request #32469 from zhengchengyao/diff_nautilus

nautilus: librbd: diff iterate with fast-diff now correctly includes parent

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #32299 from callithea/wip-42139-nautilus
Yuri Weinstein [Fri, 24 Jan 2020 13:52:23 +0000 (05:52 -0800)]
Merge pull request #32299 from callithea/wip-42139-nautilus

nautilus: mgr/dashboard: Standby Dashboards don't handle all requests properly

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoMerge pull request #31980 from epuertat/wip-43103-nautilus
Yuri Weinstein [Fri, 24 Jan 2020 13:51:15 +0000 (05:51 -0800)]
Merge pull request #31980 from epuertat/wip-43103-nautilus

nautilus: mgr/dashboard,grafana: remove shortcut menu

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #31942 from ricardoasmarques/wip-43083-nautilus
Yuri Weinstein [Fri, 24 Jan 2020 13:50:14 +0000 (05:50 -0800)]
Merge pull request #31942 from ricardoasmarques/wip-43083-nautilus

nautilus: mgr/dashboard: Trim IQN on iSCSI target form

Reviewed-by:  Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoMerge pull request #31941 from ricardoasmarques/wip-42955-nautilus
Yuri Weinstein [Fri, 24 Jan 2020 13:49:18 +0000 (05:49 -0800)]
Merge pull request #31941 from ricardoasmarques/wip-42955-nautilus

nautilus: mgr/dashboard: Unable to set boolean values to false when default is true

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #32123 from dvanders/dvanders_40891
Yuri Weinstein [Wed, 22 Jan 2020 16:35:06 +0000 (08:35 -0800)]
Merge pull request #32123 from dvanders/dvanders_40891

nautilus: osd: set collection pool opts on collection create, pg load

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #32070 from smithfarm/wip-42993-nautilus
Yuri Weinstein [Wed, 22 Jan 2020 16:34:31 +0000 (08:34 -0800)]
Merge pull request #32070 from smithfarm/wip-42993-nautilus

nautilus: monitoring: wait before firing osd full alert

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
5 years agoMerge pull request #32069 from smithfarm/wip-42999-nautilus
Yuri Weinstein [Wed, 22 Jan 2020 16:33:27 +0000 (08:33 -0800)]
Merge pull request #32069 from smithfarm/wip-42999-nautilus

DNM: nautilus: mgr/pg_autoscaler: default to pg_num[_min] = 16

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #32068 from smithfarm/wip-42695-nautilus
Yuri Weinstein [Wed, 22 Jan 2020 16:32:57 +0000 (08:32 -0800)]
Merge pull request #32068 from smithfarm/wip-42695-nautilus

nautilus: mgr/pg_autoscaler: implement shutdown method

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #32733 from leseb/cv-raw-bp
Jan Fajerski [Wed, 22 Jan 2020 13:14:13 +0000 (14:14 +0100)]
Merge pull request #32733 from leseb/cv-raw-bp

nautilus: ceph-volume: add raw (--bluestore) mode

5 years agoMerge pull request #31956 from dzafman/wip-balancer3-nautilus
Yuri Weinstein [Tue, 21 Jan 2020 23:10:20 +0000 (15:10 -0800)]
Merge pull request #31956 from dzafman/wip-balancer3-nautilus

nautilus: mgr: Balancer fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #31556 from jan--f/wip-42766-nautilus
Yuri Weinstein [Tue, 21 Jan 2020 23:08:34 +0000 (15:08 -0800)]
Merge pull request #31556 from jan--f/wip-42766-nautilus

nautilus: mgr/prometheus: assign a value to osd_dev_node when obj_store is not filestore or bluestore

Reviewed-by: Nathan Cutler <ncutler@suse.com>
5 years agoceph-volume: fix device unittest, mock has_bluestore_label, lint 32733/head
Jan Fajerski [Wed, 15 Jan 2020 14:55:45 +0000 (15:55 +0100)]
ceph-volume: fix device unittest, mock has_bluestore_label, lint

This also adds a test case for has_bluestore_label.

Fixes: https://tracker.ceph.com/issues/43611
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 91c7f5189d163ca517c643b8548a2e16176392cc)

5 years agoqa/cephfs: test case for timeout config of individual session 32245/head
Yan, Zheng [Tue, 23 Jul 2019 12:21:07 +0000 (20:21 +0800)]
qa/cephfs: test case for timeout config of individual session

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 4b28a4a7569ab516d75752c97226f598e6229ca2)

5 years agomds: add command that config individual client session
Yan, Zheng [Thu, 18 Jul 2019 07:31:15 +0000 (15:31 +0800)]
mds: add command that config individual client session

For now, the command only can change session's timeout config

Fixes: http://tracker.ceph.com/issues/40811
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit a9c21bdaef3473d99ac362122d441b9d869230dd)

5 years agoMerge pull request #32743 from leseb/fast-shutdown-bkp
Sébastien Han [Tue, 21 Jan 2020 16:34:20 +0000 (17:34 +0100)]
Merge pull request #32743 from leseb/fast-shutdown-bkp

nautilus: osd: add osd_fast_shutdown option (default true)

5 years agodoc/ceph-volume: docs for zfs/inventory 32746/head
Willem Jan Withagen [Wed, 30 Oct 2019 12:17:08 +0000 (13:17 +0100)]
doc/ceph-volume: docs for zfs/inventory

fixes: https://tracker.ceph.com/issues/42722
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
(cherry picked from commit 799673a580da20ffffdc1f2922fa5f08123a86cb)

5 years agoMerge pull request #31554 from jan--f/wip-42703-nautilus
Jan Fajerski [Tue, 21 Jan 2020 13:28:45 +0000 (14:28 +0100)]
Merge pull request #31554 from jan--f/wip-42703-nautilus

nautilus: ceph-volume: reject disks smaller then 5GB in inventory

5 years agoMerge pull request #31553 from jan--f/wip-42741-nautilus
Jan Fajerski [Tue, 21 Jan 2020 13:27:44 +0000 (14:27 +0100)]
Merge pull request #31553 from jan--f/wip-42741-nautilus

nautilus: ceph-volume: use fsync for dd command

5 years agoqa/standalone/ceph-helpers.sh: remove osd down check 32743/head
Sage Weil [Sat, 23 Nov 2019 14:55:00 +0000 (08:55 -0600)]
qa/standalone/ceph-helpers.sh: remove osd down check

A kill doesn't induce a mark-down of the OSD with osd_fast_shutdown=true.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3a62d166a7af5a62d5210417a5c10f101ce75229)

5 years agoqa/standalone/ceph-helpers.sh: destroy_osd: mark osd down
Sage Weil [Wed, 20 Nov 2019 02:03:44 +0000 (20:03 -0600)]
qa/standalone/ceph-helpers.sh: destroy_osd: mark osd down

Stopping the OSD doesn't guarantee that it will be marked down.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ede1d3677310bd152b3493e8a84da1e648fd3ea9)

5 years agoosd: add osd_fast_shutdown option (default true)
Sage Weil [Fri, 15 Nov 2019 15:31:50 +0000 (09:31 -0600)]
osd: add osd_fast_shutdown option (default true)

If we get a SIGINT or SIGTERM or are deleted from the OSDMap, do a fast
shutdown by exiting immediately.  This has a few important benefits:

 - We immediately stop responding (binding) to any sockets, which means
   other OSDs will immediately decide we are down (and dead!).  This
   minimizes IO interruption.
 - We avoid the complex "clean" shutdown process, which is historically a
   source of bugs.

In reality, the only purpose of the "clean" shutdown is to try to tear down
everything in memory so we can do memory leak checking with valgrind.  Set
this option to false for valgrind QA runs so we can still do that.

Not that with the new read leases in octopus, we rely on the default
behavior that a ECONNREFUSED is taken to mean that the OSD is fully dead,
so that we don't have to wait for any leases to time out.  This works in
sane environments with normal IP networks, but that behavior could
conceivably be a bad idea if there are some weird network shenanigans
going on.  If osd_fast_fail_on_connection_refused were disabled, then this
fast shutdown procedure might be *worse* than the clean shutdown because
we would have to wait for the heartbeat timeout.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit cf352c3ac0bd87d8b7e0c52ac724f94576ae5aa7)

5 years agoMerge pull request #32078 from smithfarm/wip-43141-nautilus
Yuri Weinstein [Mon, 20 Jan 2020 21:17:52 +0000 (13:17 -0800)]
Merge pull request #32078 from smithfarm/wip-43141-nautilus

nautilus: tools/cephfs: make 'cephfs-data-scan scan_links' fix dentry's first

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #32077 from smithfarm/wip-43138-nautilus
Yuri Weinstein [Mon, 20 Jan 2020 19:57:40 +0000 (11:57 -0800)]
Merge pull request #32077 from smithfarm/wip-43138-nautilus

nautilus: mds: remove unnecessary debug warning

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #32075 from smithfarm/wip-43001-nautilus
Yuri Weinstein [Mon, 20 Jan 2020 19:57:00 +0000 (11:57 -0800)]
Merge pull request #32075 from smithfarm/wip-43001-nautilus

nautilus: tests/cephfs: Ignore "ceph.dir.pin: No such attribute errors" in getfattr tests for old kernel client

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #32073 from smithfarm/wip-42949-nautilus
Yuri Weinstein [Mon, 20 Jan 2020 19:56:30 +0000 (11:56 -0800)]
Merge pull request #32073 from smithfarm/wip-42949-nautilus

nautilus: mds: properly evaluate unstable locks when evicting client

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #32072 from smithfarm/wip-43170-nautilus
Yuri Weinstein [Mon, 20 Jan 2020 19:55:55 +0000 (11:55 -0800)]
Merge pull request #32072 from smithfarm/wip-43170-nautilus

nautilus: tests/multimds: ignore RECENT_CRASH for multimds snapshot testing

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #32233 from s0nea/wip-43159-nautilus
Laura Paduano [Mon, 20 Jan 2020 14:41:41 +0000 (15:41 +0100)]
Merge pull request #32233 from s0nea/wip-43159-nautilus

nautilus: mgr/dashboard: KeyError on dashboard reload

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoceph-volume: raw: activate: drop --all, --osd-id and --osd-fsid args
Sage Weil [Mon, 13 Jan 2020 23:51:19 +0000 (17:51 -0600)]
ceph-volume: raw: activate: drop --all, --osd-id and --osd-fsid args

We don't need these currently.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e7b8b5c46550ee707b09e35015fabea8c6a0c918)

5 years agoraw: fix activate args
Sage Weil [Wed, 18 Dec 2019 14:27:47 +0000 (08:27 -0600)]
raw: fix activate args

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5e9d9179b15808a26e6421d2a1772bbbadc3d408)

5 years agoraw: --osd-fsid throughout
Sage Weil [Wed, 18 Dec 2019 14:13:55 +0000 (08:13 -0600)]
raw: --osd-fsid throughout

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4e7e694fa53c29ff1552656c892bee592d5db871)

5 years agoceph-volume: add raw mode
Sage Weil [Fri, 6 Dec 2019 20:08:12 +0000 (14:08 -0600)]
ceph-volume: add raw mode

list, prepare, activate, create (prepare+activate) basic bluestore OSDs
on a single device.

No support (or plans to ever support) dmcrypt.

I think support for db/wal could be added later without much trouble.

For now, we require --no-systemd and --bluestore flags because we haven't
implemented support for anything else but suspect we will in the future,
and would like to ensure users are being explicit about what they're
asking for.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1e9de77add67fa94204c450c52d22fc3a33d7b22)

5 years agoceph-volume: show devices with bluestore labels and unavailable
Sage Weil [Mon, 9 Dec 2019 13:38:13 +0000 (07:38 -0600)]
ceph-volume: show devices with bluestore labels and unavailable

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4b81bc8b358390f37eb5bcc3e2764ee6ede0ceda)

5 years agoMerge pull request #31789 from rhcs-dashboard/wip-42808-nautilus
Laura Paduano [Mon, 20 Jan 2020 09:50:20 +0000 (10:50 +0100)]
Merge pull request #31789 from rhcs-dashboard/wip-42808-nautilus

nautilus:  mgr/dashboard: key-value-table doesn't render booleans

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
5 years agoMerge pull request #31741 from joscollin/wip-42886-nautilus
Yuri Weinstein [Sat, 18 Jan 2020 16:27:07 +0000 (08:27 -0800)]
Merge pull request #31741 from joscollin/wip-42886-nautilus

nautilus: mgr/volumes: allow setting uid, gid of subvolume and subvolume group during creation

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #32071 from smithfarm/wip-42650-nautilus
Yuri Weinstein [Sat, 18 Jan 2020 16:26:16 +0000 (08:26 -0800)]
Merge pull request #32071 from smithfarm/wip-42650-nautilus

nautilus: mds: no assert on frozen dir when scrub path

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #31802 from batrick/i42943
Yuri Weinstein [Fri, 17 Jan 2020 21:49:16 +0000 (13:49 -0800)]
Merge pull request #31802 from batrick/i42943

nautilus: mds: free heap memory may grow too large for some workloads

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
5 years agoMerge pull request #32065 from smithfarm/wip-42631-nautilus
Yuri Weinstein [Fri, 17 Jan 2020 21:48:09 +0000 (13:48 -0800)]
Merge pull request #32065 from smithfarm/wip-42631-nautilus

nautilus: cephfs: client: add warning when cap != in->auth_cap.

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #32012 from rhcs-dashboard/wip-43123-nautilus
Yuri Weinstein [Fri, 17 Jan 2020 21:46:35 +0000 (13:46 -0800)]
Merge pull request #32012 from rhcs-dashboard/wip-43123-nautilus

nautilus: mgr/dashboard: Cross sign button not working for some modals

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
5 years agoMerge pull request #32128 from votdev/issue_43079
Yuri Weinstein [Fri, 17 Jan 2020 21:45:14 +0000 (13:45 -0800)]
Merge pull request #32128 from votdev/issue_43079

nautilus: mgr/dashboard: Hardening accessing the metadata

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
5 years agoMerge pull request #31784 from rhcs-dashboard/wip-42773-nautilus
Yuri Weinstein [Fri, 17 Jan 2020 20:27:25 +0000 (12:27 -0800)]
Merge pull request #31784 from rhcs-dashboard/wip-42773-nautilus

nautilus: mgr/dashboard: Remove compression mode unset in pool from

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #31999 from rhcs-dashboard/wip-43115-nautilus
Yuri Weinstein [Fri, 17 Jan 2020 20:26:22 +0000 (12:26 -0800)]
Merge pull request #31999 from rhcs-dashboard/wip-43115-nautilus

nautilus: mgr/dashboard: grafana charts match time picker selection.

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
5 years agoMerge pull request #32658 from tchaikov/nautilus-remove-seastar-test
Kefu Chai [Thu, 16 Jan 2020 09:52:15 +0000 (17:52 +0800)]
Merge pull request #32658 from tchaikov/nautilus-remove-seastar-test

nautilus: cmake: remove seastar tests from "make check"

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agomgr/dashboard: Standby Dashboards don't handle all requests properly 32299/head
Volker Theile [Thu, 19 Sep 2019 14:30:15 +0000 (16:30 +0200)]
mgr/dashboard: Standby Dashboards don't handle all requests properly

Fixes: https://tracker.ceph.com/issues/41930
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 29cc91f92d766e478251f301dc8e3e1e76f39c57)

5 years agotest: Sort pool list because the order isn't guaranteed from "balancer pool ls" 31956/head
David Zafman [Fri, 20 Dec 2019 21:46:34 +0000 (13:46 -0800)]
test: Sort pool list because the order isn't guaranteed from "balancer pool ls"

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit c65d5c8d1421438da999698f1f734d6a61adb3bb)

5 years agomgr: Change default upmap_max_deviation to 5
David Zafman [Sat, 14 Dec 2019 01:43:44 +0000 (17:43 -0800)]
mgr: Change default upmap_max_deviation to 5

Fixes: https://tracker.ceph.com/issues/43312
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b0a1b758d012dfea40db3feca1a841c96f79defe)

5 years agoosdmaptool: Add --upmap-active to simulate active upmap balancing
David Zafman [Wed, 18 Dec 2019 19:27:02 +0000 (11:27 -0800)]
osdmaptool: Add --upmap-active to simulate active upmap balancing

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 621acf8ce7f48253e9d2189a9a2ee432fa1d3ba1)

5 years agodoc: Add upmap options to osdmaptool man page and give example
David Zafman [Wed, 18 Dec 2019 03:38:51 +0000 (19:38 -0800)]
doc: Add upmap options to osdmaptool man page and give example

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 184e9d1ae3b5bcc332d5fe3330d46a5cb8fcacd6)

5 years agotools: osdmaptool document non-upmap options that were missing
David Zafman [Wed, 18 Dec 2019 01:35:14 +0000 (17:35 -0800)]
tools: osdmaptool document non-upmap options that were missing

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit e42a6ccb1819be4988e3ed7bd78fcf513f8d1589)

5 years agotest: Fix test case for pool based balancing instead of rule batched
David Zafman [Sat, 14 Dec 2019 01:14:51 +0000 (17:14 -0800)]
test: Fix test case for pool based balancing instead of rule batched

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 8e46bbbf36e3d3795c1d3ba50c431ac80467da20)

5 years agoPartially revert "mgr/balancer: balance pools with same crush_rule in batch"
David Zafman [Fri, 13 Dec 2019 03:42:10 +0000 (19:42 -0800)]
Partially revert "mgr/balancer: balance pools with same crush_rule in batch"

This partially reverts commit 3a730d751deff892e7a0bddba87eba3dbb829c3e

Fixes: https://tracker.ceph.com/issues/43307
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 7a1d6872c74a03b2493ecef0f2c4a5f9656a7c5e)

Conflicts:
src/pybind/mgr/balancer/module.py (No active+clean handling)

5 years agoRevert "tools: osdmaptool sync with balancer module behavior"
David Zafman [Tue, 10 Dec 2019 16:55:46 +0000 (08:55 -0800)]
Revert "tools: osdmaptool sync with balancer module behavior"

This was the rules batching version of osdmaptool

This reverts commit f165d4ca39b1edef4bf22a7597af79339c95026f.

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a92f48773e9c59d49ee248de1d7951e7c9d3ac25)

5 years agorelease note: Add pending release notes for already merged code
David Zafman [Thu, 5 Dec 2019 17:52:04 +0000 (09:52 -0800)]
release note: Add pending release notes for already merged code

Follow on to https://github.com/ceph/ceph/pull/31774

Fix incorrect version number for Nautilus

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 65d03bae8b4f50cc3cbaa50640eaeab4cabd711f)

5 years agotools/osdmaptool.cc: do not use deprecated std::random_shuffle()
Kefu Chai [Wed, 4 Dec 2019 00:06:40 +0000 (16:06 -0800)]
tools/osdmaptool.cc: do not use deprecated std::random_shuffle()
the use of `std::random_shuffle()` was introduced by
b946308 .

in this change, it is replaced using `std::shuffle()`.

Fixes: https://tracker.ceph.com/issues/43084
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 30dfef208d859a67c1f1b13c314735861c536e85)

5 years agotest: Add test case based on Xie script in commit comment
David Zafman [Wed, 27 Nov 2019 18:50:01 +0000 (10:50 -0800)]
test: Add test case based on Xie script in commit comment

Other test fixes to reflect changes

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9d2e0267e1f025a634ee4925c919008cacdd3d38)

5 years agoosd: ceph_pg_upmaps() use any overfull when there are still underfull
David Zafman [Wed, 27 Nov 2019 18:48:30 +0000 (10:48 -0800)]
osd: ceph_pg_upmaps() use any overfull when there are still underfull

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f38d38398f33f64bd49f017b1c8e737452157c53)

5 years agoosd: Create more_underfull with below target that aren't in underfull
David Zafman [Tue, 26 Nov 2019 23:19:01 +0000 (15:19 -0800)]
osd: Create more_underfull with below target that aren't in underfull

Use more_underfull for finding remaps for overfull OSDs

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit fe03383e6691f6bd95189d3ba4d91fbb2ea9e9c8)

5 years agoosd/OSDMap.cc: don't output over/underfull messages to lderr
Neha Ojha [Mon, 11 Nov 2019 21:32:15 +0000 (13:32 -0800)]
osd/OSDMap.cc: don't output over/underfull messages to lderr

There can be cases where overfull and underfull(see example in
https://tracker.ceph.com/issues/42756) will be empty, which is not
necessarily an error. These error messages can end up spamming
the ceph-mgr log.

Fixes: https://tracker.ceph.com/issues/42756
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 3cf28f09a2ebdd922fc5c1d29c65469f8331e741)

5 years agoosd: calc_pg_upmaps() pick most overfull remap from try_pg_upmap()
David Zafman [Tue, 26 Nov 2019 05:38:16 +0000 (21:38 -0800)]
osd: calc_pg_upmaps() pick most overfull remap from try_pg_upmap()

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3f132061fd33c82dc131e18ae80cbec067b53cae)

5 years agoosd: ceph_pg_upmaps() use max_deviation to determine perfect distribution
David Zafman [Fri, 15 Nov 2019 05:10:28 +0000 (21:10 -0800)]
osd: ceph_pg_upmaps() use max_deviation to determine perfect distribution

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 7f4ae9312314c329b63c850e09b403a95ad18bed)

5 years agotools: osdmaptool sync with balancer module behavior
David Zafman [Sat, 9 Nov 2019 00:23:14 +0000 (16:23 -0800)]
tools: osdmaptool sync with balancer module behavior

This can be backported only to nautilus

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f165d4ca39b1edef4bf22a7597af79339c95026f)

5 years agotools: osdmaptool: Perform upmap calculation as ceph-mgr does
David Zafman [Thu, 7 Nov 2019 04:32:48 +0000 (20:32 -0800)]
tools: osdmaptool: Perform upmap calculation as ceph-mgr does

This is the backportable commit that works with older balancer module.py

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b946308f03f1798915fafe7878f34b4a234c2ae4)

5 years agoosd: For balancer crush needs the rule passed to get_parent_of_type()
David Zafman [Wed, 20 Nov 2019 06:21:00 +0000 (22:21 -0800)]
osd: For balancer crush needs the rule passed to get_parent_of_type()

Fixes: https://tracker.ceph.com/issues/42718
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3eb69fd0cb15974e33e675babdd5d232d69c3c6d)

5 years agoosdmaptool: Match default max value of 10
David Zafman [Thu, 7 Nov 2019 04:31:21 +0000 (20:31 -0800)]
osdmaptool: Match default max value of 10

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 91bab14ea0e2ec7c94eb758cd76f9a620688725f)

5 years agotools: odsmaptool truncate target upmap file
David Zafman [Wed, 20 Nov 2019 19:55:15 +0000 (11:55 -0800)]
tools: odsmaptool truncate target upmap file

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a49324ceb9c322f792ea00720f438ea9ea09555f)

5 years agomgr: Fix balancer print
David Zafman [Wed, 20 Nov 2019 22:08:24 +0000 (14:08 -0800)]
mgr: Fix balancer print

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 0af7e25620ae823d46eb842992570ddc1607cc4f)

5 years agoMerge pull request #32063 from smithfarm/wip-42878-nautilus
Yuri Weinstein [Wed, 15 Jan 2020 16:54:46 +0000 (08:54 -0800)]
Merge pull request #32063 from smithfarm/wip-42878-nautilus

nautilus: common/admin_socket: Increase socket timeouts

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #31850 from smithfarm/wip-43007-nautilus
Yuri Weinstein [Wed, 15 Jan 2020 16:42:52 +0000 (08:42 -0800)]
Merge pull request #31850 from smithfarm/wip-43007-nautilus

nautilus: common/options: remove unused ms_msgr2_{sign,encrypt}

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #32059 from smithfarm/wip-41954-nautilus
Yuri Weinstein [Wed, 15 Jan 2020 16:42:03 +0000 (08:42 -0800)]
Merge pull request #32059 from smithfarm/wip-41954-nautilus

nautilus: tests/krbd: avoid hexdump skip and length options

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
5 years agoMerge pull request #31848 from smithfarm/wip-42989-nautilus
Yuri Weinstein [Wed, 15 Jan 2020 16:41:34 +0000 (08:41 -0800)]
Merge pull request #31848 from smithfarm/wip-42989-nautilus

nautilus: tests: kernel.sh: update for read-only changes

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
5 years agoMerge pull request #31782 from rhcs-dashboard/wip-42809-nautilus
Yuri Weinstein [Wed, 15 Jan 2020 16:41:01 +0000 (08:41 -0800)]
Merge pull request #31782 from rhcs-dashboard/wip-42809-nautilus

nautilus: mgr/dashboard: Handle always-on Ceph Manager modules correctly

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
5 years agoMerge pull request #31697 from k0ste/wip-42850-nautilus
Yuri Weinstein [Wed, 15 Jan 2020 16:39:17 +0000 (08:39 -0800)]
Merge pull request #31697 from k0ste/wip-42850-nautilus

nautilus: pybind/mgr: Cancel output color control

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agocmake: remove seastar tests from "make check" 32658/head
Kefu Chai [Wed, 15 Jan 2020 16:20:49 +0000 (00:20 +0800)]
cmake: remove seastar tests from "make check"

because crimson is released yet, it does not make sense to run seaetar
based tests in LTS branches.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts: this change is not cherry-picked from master. as explained
  in the commit message above. we need to use these seastar based
  tests to verify that the crimson facilities are not broken in master.
  but it does not help to run them in nautilus.

5 years agomgr/dashboard: KeyError on dashboard reload 32233/head
Patrick Seidensal [Thu, 7 Nov 2019 15:57:26 +0000 (15:57 +0000)]
mgr/dashboard: KeyError on dashboard reload

When reloading the dashboard using `ceph mgr module disable/enable
dashbaord`, the error is thrown shortly after the dashboard module has
been reactivated, because `dashboard` is not yet available in
`services`. A second attempt works.

Fixes: https://tracker.ceph.com/issues/42684
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 808f0bf5e0adf4abda192758edbccdee3d4113d2)

5 years agoMerge pull request #32651 from tchaikov/nautilus-dts-8-on-arm64
Kefu Chai [Wed, 15 Jan 2020 15:37:19 +0000 (23:37 +0800)]
Merge pull request #32651 from tchaikov/nautilus-dts-8-on-arm64

nautilus: build/ops: install-deps,rpm: enable devtoolset-8 on aarch64 also

Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
5 years agoMerge pull request #31791 from rhcs-dashboard/wip-42935-nautilus
Laura Paduano [Wed, 15 Jan 2020 15:29:05 +0000 (16:29 +0100)]
Merge pull request #31791 from rhcs-dashboard/wip-42935-nautilus

nautilus: mgr/dashboard: sort monitors by open sessions correctly.

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoinstall-deps,rpm: enable devtoolset-8 on aarch64 also 32651/head
Kefu Chai [Wed, 3 Apr 2019 03:08:04 +0000 (11:08 +0800)]
install-deps,rpm: enable devtoolset-8 on aarch64 also

in 5ae3b06e, we left aarch64 with devtoolset-7, but aarch64 builders are
also suffering from http://tracker.ceph.com/issues/38892. so i installed
devtoolset-8 on all aarch64 builders manually from the rpm packages
downloaded from koji builder:

- https://cbs.centos.org/koji/buildinfo?buildID=24923
- https://cbs.centos.org/koji/buildinfo?buildID=24931

libasan5, libubsan1 are not installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 00f19923b4225c25480ddb3c09a898a2444fbeb9)

5 years agoMerge pull request #31646 from batrick/i42818
Yuri Weinstein [Tue, 14 Jan 2020 20:35:30 +0000 (12:35 -0800)]
Merge pull request #31646 from batrick/i42818

nautilus: global: disable THP for Ceph daemons

5 years agoMerge pull request #31082 from smithfarm/wip-42279-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 20:34:39 +0000 (12:34 -0800)]
Merge pull request #31082 from smithfarm/wip-42279-nautilus

nautilus: tests/ceph.py: tolerate ECONNRESET errcode during logrotate

5 years agoMerge pull request #30820 from liewegas/wip-alerts-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 20:33:59 +0000 (12:33 -0800)]
Merge pull request #30820 from liewegas/wip-alerts-nautilus

nautilus: mgr/alerts: simple health alerts

5 years agoMerge pull request #30765 from smithfarm/wip-42129-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 20:33:17 +0000 (12:33 -0800)]
Merge pull request #30765 from smithfarm/wip-42129-nautilus

nautilus: doc/ceph-fuse: mention -k option in ceph-fuse man page

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
5 years agoMerge pull request #30739 from smithfarm/wip-41849-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 20:32:24 +0000 (12:32 -0800)]
Merge pull request #30739 from smithfarm/wip-41849-nautilus

nautilus: tools/setup-virtualenv.sh: do not default to python2.7

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #30009 from smithfarm/wip-40126-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 20:14:40 +0000 (12:14 -0800)]
Merge pull request #30009 from smithfarm/wip-40126-nautilus

nautilus: mgr/zabbix: Adds possibility to send data to multiple zabbix servers.

Reviewed-by: Wido den Hollander <wido@widodh.nl>
5 years agomake-dist: drop Python 2/3 autoselect 30739/head
Nathan Cutler [Fri, 27 Dec 2019 09:55:21 +0000 (10:55 +0100)]
make-dist: drop Python 2/3 autoselect

virtualenv will prefer Python 3 over Python 2 if both are available.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 249bf81a32ff6d7046e525b947718d49b2314c13)

5 years agotools/setup-virtualenv.sh: do not default to python2.7
Nathan Cutler [Fri, 13 Sep 2019 14:12:52 +0000 (16:12 +0200)]
tools/setup-virtualenv.sh: do not default to python2.7

This commit changes the script's behavior when run without a --python option.
Before, we were running virtualenv with "--python python2.7" in that case.

That is not a reasonable default in September 2019, given that Python 2 EOL is
coming on January 1, 2020. Also, it breaks setup-virtualenv.sh in py3-only
environments.

With this patch, the script will not pass any "--python" option to virtualenv
(i.e., it will let virtualenv do whatever it thinks is best) in this case.

Fixes: https://tracker.ceph.com/issues/41821
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 78468e49ab2110d69bc34333c0ce90acb832de1a)

5 years agoMerge pull request #32593 from trociny/wip-43500-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 01:09:14 +0000 (17:09 -0800)]
Merge pull request #32593 from trociny/wip-43500-nautilus

nautilus: rbd-mirror: make logrotate work

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #32447 from trociny/wip-43428-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 01:08:33 +0000 (17:08 -0800)]
Merge pull request #32447 from trociny/wip-43428-nautilus

nautilus: rbd-mirror: fix 'rbd mirror status' asok command output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #32086 from dillaman/wip-42630-nautilus
Yuri Weinstein [Tue, 14 Jan 2020 01:07:51 +0000 (17:07 -0800)]
Merge pull request #32086 from dillaman/wip-42630-nautilus

nautilus: mgr: add 'rbd' profiles to support 'rbd_support' module commands

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
5 years agoMerge pull request #32167 from rzarzynski/wip-fips-zeroize-memset_bzero_nautilus
Neha Ojha [Mon, 13 Jan 2020 17:45:45 +0000 (09:45 -0800)]
Merge pull request #32167 from rzarzynski/wip-fips-zeroize-memset_bzero_nautilus

nautilus: common: FIPS: audit and switch some memset & bzero users

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #31332 from joscollin/wip-41182-nautilus
Yuri Weinstein [Mon, 13 Jan 2020 16:29:07 +0000 (08:29 -0800)]
Merge pull request #31332 from joscollin/wip-41182-nautilus

nautilus: mgr/volumes: fs subvolume resize command

5 years agoMerge pull request #31302 from smithfarm/wip-42142-nautilus
Yuri Weinstein [Mon, 13 Jan 2020 16:28:18 +0000 (08:28 -0800)]
Merge pull request #31302 from smithfarm/wip-42142-nautilus

nautilus: mds: split the dir if the op makes it oversized, because some ops maybe in flight

5 years agoMerge pull request #31084 from smithfarm/wip-42424-nautilus
Yuri Weinstein [Mon, 13 Jan 2020 16:27:47 +0000 (08:27 -0800)]
Merge pull request #31084 from smithfarm/wip-42424-nautilus

nautilus: tests: whitelist "Error recovering journal" for cephfs-data-scan

5 years agoMerge pull request #31083 from smithfarm/wip-42422-nautilus
Yuri Weinstein [Mon, 13 Jan 2020 16:27:10 +0000 (08:27 -0800)]
Merge pull request #31083 from smithfarm/wip-42422-nautilus

nautilus: tests/cephfs.py: allow client mount to reset fully

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #31081 from smithfarm/wip-42158-nautilus
Yuri Weinstein [Mon, 13 Jan 2020 16:25:59 +0000 (08:25 -0800)]
Merge pull request #31081 from smithfarm/wip-42158-nautilus

nautilus: cephfs: osdc/objecter: Fix last_sent in scientific format and add age to ops

Reviewed-by: Ramana Raja <rraja@redhat.com>
5 years agoMerge pull request #30769 from smithfarm/wip-41888-nautilus
Yuri Weinstein [Fri, 10 Jan 2020 20:19:41 +0000 (12:19 -0800)]
Merge pull request #30769 from smithfarm/wip-41888-nautilus

nautilus: cephfs: "client: fix lazyio_synchronize() to update file size" and "libcephfs: Add Tests for LazyIO"

5 years agoMerge pull request #30767 from smithfarm/wip-42147-nautilus
Yuri Weinstein [Fri, 10 Jan 2020 20:18:54 +0000 (12:18 -0800)]
Merge pull request #30767 from smithfarm/wip-42147-nautilus

nautilus: cephfs: mds returns -5(EIO) error when the deleted file does not exist