mgr/nfs: add --port to 'nfs cluster create' and port to 'nfs cluster info'

Fixes: https://tracker.ceph.com/issues/51787
Signed-off-by: Sage Weil <sage@newdream.net>

qa/suites/orch/cephadm/smoke-roleless: test taking ganeshas offline

Signed-off-by: Sage Weil <sage@newdream.net>

qa/tasks/vip: exec with bash -ex

Signed-off-by: Sage Weil <sage@newdream.net>

qa/suites/orch/cephadm: separate test_nfs from test_orch_cli

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #42242 from sebastian-philipp/cephadm-container-name-dashes

cephadm: use dashes for container names

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #42462 from sebastian-philipp/mon-ip-change

doc/cephadm: Move MONs to a different network

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #42619 from jdurgin/wip-release-notes-script

script/ceph-release-notes: improve output for markdown and limit verbosity

Reviewed-by: David Galloway <dgallowa@redhat.com>

script/ceph-release-notes: don't include PR body by default, just the title

This avoids overly long output for merges involving lots of PRs,
e.g. cephadm batch backports.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

script/ceph-release-notes: infer actual title from ptl-tool merges

This helps the notes stay accurate when the commit title has changed,
but the PR title is out of date.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #42636 from zdover23/wip-doc-rados-config-storage-devices-bluestore-2021-08-03

doc/rados: rewrite BlueStore material

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #41897 from ivancich/wip-versioned-deletes

rgw: when deleted obj removed in versioned bucket, extra del-marker added

Reviewed-by: Casey Bodley <cbodley@redhat.com>

cephadm: Introduce unit.stop

Reason is we now have to stop two containers named differently. This is possible
with `bash -c ... echo %i | tr . -`. But we're not gainig anything readability
compared to putting it into a unit.stop script.

As not all daemons have this stop script, we still have to call podman for old daemons.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

cephadm: don't log on exit.

At this point, the garbage collector already colleceted the `open` function

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

cephadm: `stats` might return `--` for containers

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

cephadm: exec: also search for old cname

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

cephadm: dashes: use both cnames for `inspect`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

cephadm: Add CephContainer.for_daemon()

To avoid calling get_container()

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

cephadm: use dashes for container names

podman adds the current container name to the /etc/hosts
file. Turns out, python's `socket.getfqdn()` differs from
`hostname -f`, when we have the container names containing
dots in it.:

[root@sebastians-laptop /]# cat /etc/hosts
127.0.0.1   localhost
::1         localhost
127.0.1.1   sebastians-laptop foo.bar.baz.com
[root@sebastians-laptop /]# hostname -f
sebastians-laptop
[root@sebastians-laptop /]# python3 -c 'import socket; print(socket.getfqdn())'
foo.bar.baz.com

Fascinatingly, this doesn't happen when using dashes.

Fixes: https://tracker.ceph.com/issues/51590
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42395 from sebastian-philipp/cephadm-improve-no-enigne-msg

cephadm: Clarify no container engine message

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/cephadm: MON IP change

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42587 from liewegas/rgw-mon-secure

rgw: default ms_mon_client_mode = secure

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #42610 from rhcs-dashboard/52022-fix-rgw-perf-counters

mgr/dashboard: show perf. counters for rgw svc. on Cluster > Hosts

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #42554 from rhcs-dashboard/fix-rgw-ssl-cert-validation

mgr/dashboard: fix ssl cert validation for rgw service creation

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #42575 from tchaikov/wip-venv

*: s/virtualenv/python -m venv/

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42594 from sebastian-philipp/mypy-ini-disallow-untyped-root

mypy.ini: Globally enable `disallow_untyped_defs = True`

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42571 from tchaikov/wip-disable-kvs

cmake: disable kvs rados cls by default

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42614 from soumyakoduri/wip-skoduri-sqlite3

cmake: Remove duplicate SQLITE module

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: show perf. counters for rgw svc. on Cluster > Hosts

- Use service_map_id to retrieve perf. counters for rgw services.
- Move logic from controller to CephService.

Fixes: https://tracker.ceph.com/issues/52022
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

cmake: disable kvs rados cls by default

libcls_kvs was introduced back in
73d016fdb304ad19bba8aed3f2877b4bdb6ed32e, but we don't have an internal
user so far. to reduce the build time. let's disable the build of it by
default.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42567 from dsavineau/issue_51973

cephadm: set global default container for ingress

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42559 from tchaikov/wip-rpm-dwz

rpm: enable dwz

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

script/ceph-release-notes: sort PRs case-insensitively

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

mgr/dashboard: fix ssl cert validation for rgw service creation

Fixes: https://tracker.ceph.com/issues/51965
Signed-off-by: Avan Thakkar <athakkar@redhat.com>

script/ceph-release-notes: adapt markdown output for new website

* escape underscores
* avoid automatic links from PR titles containing .

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

cmake: Remove duplicate SQLite module

Signed-off-by: Soumya Koduri <skoduri@redhat.com>

doc/rados: rewrite BlueStore material

This PR improves the English text
explaining the BlueStore filesystem
backend.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #42605 from aaryanporwal/visual-tests

mgr/dashboard: Jenkins Applitools plugin: add batchId variable

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge PR #42463 into master

* refs/pull/42463/head:
src/pybind/mgr/test_orchestrator: remove nfs pool and namespace
src/vstart: update nfs-ganesha pool name

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #42518 into master

* refs/pull/42518/head:
doc/cephadm: update about disabling logging to journald for quincy
doc/cephadm: remove repeated section on disabling logging to stderr

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #42588 from liewegas/doc-foundation-didi

doc/foundation: drop didi

cephadm: Clarify no container engine message

Multiple people didn't understand the message. Let's try to impove it.

(Also make mypy happy)

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

mgr/dashboard: Cluster > Hosts > Performance Counters: do not show context bar.

Do not show the context bar (Object Gateway selection).

Fixes: https://tracker.ceph.com/issues/52022
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

cephadm: Add missing type annotations

Turnd out `command_maintenance` missing an if-branch that returned
None instead of a str

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

doc/cephadm: update about disabling logging to journald for quincy

Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/dashboard: Jenkins Applitools plugin: add batchId variable

batchId variable is mandatory for the applitools test runner to show up in jenkins build reports, for more details, check out: https://plugins.jenkins.io/applitools-eyes/

Signed-off-by: Aaryan Porwal <aaryanporwal2233@gmail.com>

Merge pull request #42601 from idryomov/wip-rbd-qemu-iotests-8stream

qa/workunits/rbd: use xenial version of qemu-iotests for centos stream 8

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #42446 from zdover23/wip-doc-cephadm-troubleshooting-logs-2021-07-20

doc/cephadm: linking to log material

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/cephadm: remove repeated section on disabling logging to stderr

Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #42328 from aaryanporwal/visual-tests

mgr/dashboard: Visual regression tests for ceph dashboard

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #42514 from liu-chunmei/seastore-onode-interruptible-future

crimson/onode-staged-tree: integrate interruptible future

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

qa/workunits/rbd: use xenial version of qemu-iotests for centos stream 8

It is already used for centos 8(.3) and rhel 8(.4).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

crimson/onode-staged-tree: misc fixes to integrate interruptive-future

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

doc/cephadm: linking to log material

This PR rewrites a section in the Troubleshooting
chapter of the Cephadm Guide. The material that this
section discusses has been covered already in the
Cephadm Guide in the Cephadm Operations chapter.
There's no reason to repeat this information twice,
unless adding technical debt to the documentation
is our goal (which of course it is not, and the
opposite of adding technical debt to the documentation
has been the aim that has guided my work these past
six months).

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #42471 from Dorthu/fix-pybind-rbd-mirror-image-get-status

pybind/rbd: fix mirror_image_get_status

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #42570 from tchaikov/wip-libcryptsetup

librbd/crypto/luks: require libcryptsetup v2.0.5

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

pybind/mgr: enable `disallow_untyped_defs` for mgr_module

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

mypy.ini: Globally enable `disallow_untyped_defs = True`

Plus introduce an ignorelist of all modules that are not compatible yet.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

rgw: default ms_mon_client_mode = secure

Require a secure connection to the monitor.  This is important because
we may store SSL certs on the mon.

Note that secure mode was introduce before Nautilus, so this can be
backported at least to Pacific.

Fixes: https://tracker.ceph.com/issues/52000
Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #42551 from adk3798/maint-message

mgr/cephadm: make return message for entering maintenance mode more explicit

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

doc/foundation: drop didi

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #42382 from xrmeng8756/master

rgw: avoid occuring radosgw daemon crash when access a conditionally …

Merge pull request #42387 from wzbxqt327/master

rgw:add lock to copy object

Merge pull request #42404 from ivancich/wip-broken-list-plain-entries

 rgw: bucket index list can produce I/O errors

Merge pull request #42582 from tchaikov/wip-doc-health-report

doc/dev: add health-reports.rst

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #42577 from tchaikov/wip-rtd

.readthedocs.yml: use python3.8 and native ditaa

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

doc/dev: add health-reports.rst

to explain the data flow of health metrics.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42263 from rhcs-dashboard/51612-cephadm-e2e-improvements

mgr/dashboard: cephadm-e2e job script: improvements

Merge pull request #42562 from tchaikov/wip-bluestore-cleanups

os/bluestore: use scope_guard do to cleanups

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #41880 from david-caro/fix_cluster_grafana_dashboard

monitoring/grafana/cluster: use per-unit max and limit values

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: p-se <NOT@FOUND>

admin, doc: introduce sphinxcontrib.seqdiag

for rendering sequence-diagram. unlike ditaa, seqdiag allows us to
create sequence-diagram without worrying about the layout. and the
syntax is quite like that of dot.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42572 from tchaikov/wip-cmake-mgr-cleanup

cmake: initialize dpdk_LIBRARIES with empty list

Reviewed-by: Xiubo Li <xiubli@redhat.com>

doc/conf.py: run ditaa with java

just in case, otherwise we could have

  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/42577/lib/python3.8/site-packages/sphinxcontrib/ditaa.py", line 200, in html_visit_ditaa
    render_ditaa_html(self, node, node['code'], node['options'])
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/42577/lib/python3.8/site-packages/sphinxcontrib/ditaa.py", line 177, in render_ditaa_html
    fname, outfn = render_ditaa(self, code, options, prefix)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/42577/lib/python3.8/site-packages/sphinxcontrib/ditaa.py", line 141, in render_ditaa
    p = Popen(ditaa_args, stdout=PIPE, stdin=PIPE, stderr=PIPE)
  File "/home/docs/.pyenv/versions/3.8.6/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/docs/.pyenv/versions/3.8.6/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/usr/bin/ditaa'

Signed-off-by: Kefu Chai <kchai@redhat.com>

.readthedocs.yml: use ditaa instead of plantweb

use ditaa to reader ditaa images instead of relying plantweb service.
more stable this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>

.readthedocs.yml: use python3.8

to prepare the python3.8 migration

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42129 from tchaikov/wip-cmake-build-type

cmake: set CMAKE_BUILD_TYPE only if .git exists

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42573 from tchaikov/wip-crimson-header

cmake: let crimson-admin depend on legacy-option-headers

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #42445 from LiumxNL/ups-eliminate-rollfwd

osd/PGLog: set acceptable rollback_info_trimmed_to for pg of replicated pool

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #40511 into master

* refs/pull/40511/head:
qa: update mds_pre_upgrade to no longer stop standbys
qa: update mds_pre_upgrade to disable standby-replay
qa: add tests for compat manipulation and upgrade
doc: remove deprecated compat commands
doc: update MDS upgrade procedure
mon,mds: use per-MDS compat to inform replacement
mon: do not update inline incompat except via mds
mds: add MDSMap method for creating null MDSMap
mds: only update beacon epoch if newer
mds: harden standby_mds lookup
mon/FSCommands: accept generic ostream rather than stringstream
include: add less verbose CompatSet dump
include: add dump operator for Feature
include: add const qualifier to appropriate CompatSet methods

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

rpm, debian: drop virtualenv from build deps

since we've replaced "virtualenv" with "python3 -m venv", there is no
need to have it in the build deps list.

since, on ubuntu, venv modules is not available by default, we need to
install python3-venv.

Signed-off-by: Kefu Chai <kchai@redhat.com>

script/ceph-release-notes: s/virtualenv/python3 -m venv/

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/rgw/bucket_notification: s/virtualenv/python3 -m venv/

so we don't need to use virtualenv python package for creating a
virtualenv, the "venv" module in Python3 would suffice.

see also https://docs.python.org/3/library/venv.html

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks: s/virtualenv/python3 -m venv/

so we don't need to use virtualenv python package for creating a
virtualenv, the "venv" module in Python3 would suffice.

see also https://docs.python.org/3/library/venv.html

Signed-off-by: Kefu Chai <kchai@redhat.com>

admin/build-doc: s/virtualenv/python3 -m venv/

so we don't need to use virtualenv python package for creating a
virtualenv, the "venv" module in Python3 would suffice.

see also https://docs.python.org/3/library/venv.html

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/man/8/cephfs-shell: s/virtualenv/python3 -m venv/

so user does not have to use virtualenv python package for creating a
virtualenv, the "venv" module in Python3 would suffice.

see also https://docs.python.org/3/library/venv.html

Signed-off-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: s/virtualenv/python -m venv/

so we don't need to use virtualenv python package for creating a
virtualenv, the "venv" module in Python3 would suffice.

see also https://docs.python.org/3/library/venv.html

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/dashboard: s/virtualenv/python -m venv/

so we don't need to use virtualenv python package for creating a
virtualenv, the "venv" module in Python3 would suffice.

see also https://docs.python.org/3/library/venv.html

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/run-cli-tests: s/virtualenv/python -m venv/

so we don't need to use virtualenv python package for creating a
virtualenv, the "venv" module in Python3 would suffice.

see also https://docs.python.org/3/library/venv.html

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/setup-virtualenv: s/virtualenv/python -m venv/

so we don't need to use virtualenv python package for creating a
virtualenv, the "venv" module in Python3 would suffice.

see also https://docs.python.org/3/library/venv.html

Signed-off-by: Kefu Chai <kchai@redhat.com>

script/run_tox.sh: drop support of python2

since we don't support python2 anymore, there is not need to mention or
support python2 when performing tests

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: let crimson-admin depend on legacy-option-headers

legacy-option-headers provides global_legacy_options.h, which is in turn
required for building crimson-admin.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42400 from adk3798/offline-daemon-removal

mgr/cephadm: don't remove daemons from offline hosts

Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>

cmake: prefer static library when finding DPDK

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: initialize dpdk_LIBRARIES with empty list

set(dpdk_LIBRARIES) does not reset this variable, it leaves it
unchanged.

if pkg-config manages to find DPDK libraries, dpdk_LIBRARIES would be
set with a string like "rte_node;rte_graph;..." by
pkg_check_modules(dpdk QUIET libdpdk).

but we would want to set this variable to the import paths of the
required libraries. so reset it before appending them to this variable.

this change helps to address the build failure when building Ceph with
DPDK installed into system along with its .pc file.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/ServiceMap: do not include unused headers

<experimental/iterator> was included for std::experimental::make_ostream_joiner
in a968f65d784b3d6c6a172929aa293f09e6917fa6. but the code using
std::experimental::make_ostream_joiner was later rewritten in
ab0d8f2ae9f551e15a4c7bacbf69161e91263785, in which
std::experimental::make_ostream_joiner is not used anymore.

so let's drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42553 from cfsnyder/wip_51961

mgr/cephadm: fix exceptions causing stuck progress indicators

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #42506 from p-se/mgr-prom-counter-type

mgr/prometheus: Fix metric types from gauge to counter

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd/crypto/luks: require libcryptsetup v2.0.5

- ubuntu focal ships libcryptsetup-dev (2:2.2.2),
- centos 8 app stream comes with cryptsetup-devel-2.3.3.
- openSUSE Leap 15.3 packages libcryptsetup-devel-2.3.4
- openSUSE Leap 15.2 packages libcryptsetup-devel-2.0.5

so we can drop the support for libcryptsetup < 2.0.5

see also ea3c1bfb9ef2edcdf572df0cb143c463b7551905

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa: update mds_pre_upgrade to no longer stop standbys

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: update mds_pre_upgrade to disable standby-replay

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add tests for compat manipulation and upgrade

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>