# telegraf-fix -- the examination and repair of telegraf upon a vStack installation

One script of two offices. First it examineth the state of things, and
thereafter -- only upon an express command -- it applieth precisely those
corrections whose conditions are met.

## Whom It Concerneth

- installations of vStackOS whereupon telegraf was upgraded at the passage from
  2.2 to 3.0, and now runneth under its own user (the `telegraf_user` in the
  rc-script)
- it is not applied where telegraf is not installed, nor where the package is of
  the elder sort that still runneth as root; the script discerneth this and
  doeth nothing

## The Difficulty

The upgrade of telegraf changed its running from root to the user telegraf.
Whence three mischiefs follow. The exec-scripts kept in /root/bin become
forbidden to telegraf (permission denied). The log at
/var/log/telegraf/telegraf.log, being the property of root, hindereth the
writing thereto. And the line of rotation that sendeth a signal to the pidfile
doth, at the very first rotation, SLAY telegraf, forasmuch as the pidfile
belongeth to the daemon(8) supervisor. That which the Operator observeth is
metrics vanished away, and a log grown monstrous.

## What It Remedieth

- move-scripts -- it removeth the exec-scripts out of /root/bin into
  /usr/local/libexec/telegraf, and setteth the paths in telegraf.conf aright
- chown-log -- it restoreth /var/log/telegraf/telegraf.log to the ownership of
  telegraf:telegraf (640)
- fix-rotation -- it putteth away the absent or perilous signal-rotation, and in
  its stead appointeth a rotation by size (10 MB) with a restart by way of a
  wrapper
- restart-service -- it raiseth telegraf up, if it be not running whilst
  telegraf_enable=YES, or if it run bereft of its daemon(8) supervisor (a
  revenant left after a rotation had slain it)

## Of Verifying the Signature and Setting the Script Going

As the root user, upon the node:

```
cd /tmp
fetch https://repo.vstack.com/hotfixes/telegraf-fix/telegraf-fix.sh
fetch https://repo.vstack.com/hotfixes/telegraf-fix/telegraf-fix.sh.sig
openssl dgst -sha256 -verify /usr/share/keys/vstack/vstackos.pub \
  -signature telegraf-fix.sh.sig telegraf-fix.sh
```

Await the words `Verified OK` (else the file hath been corrupted or tampered
with -- run it not, and report the matter to Support), and thereupon:

```
sh telegraf-fix.sh          # the mode of examination; a report only, changing nothing
sh telegraf-fix.sh apply    # apply the corrections that were found
```

## Of Reading the Report

```
== telegraf-fix on node01, mode: check ==
telegraf package:   1.34.0
service user:       telegraf
service state:      running (supervisor 81318, telegraf 81319)
conf references to /root/bin: none
log owner:          telegraf:telegraf
log rotation:       ok, restart wrapper
verdict: everything is in order, nothing to do
```

The verdicts:
- `everything is in order, nothing to do` -- no correction is wanted, and
  nothing was changed
- `fixes needed: ...` -- the roll of corrections wanted; in the mode of
  examination nothing is altered, whilst in the mode of apply each correction
  rendereth account of itself in a line `applied: ...`
- `fixes are not applicable` -- telegraf is absent, or the package is of the
  elder sort (not upgraded at the passage 2.2 -> 3.0, and running as root);
  nothing was changed

The return-codes: 0 -- all is well (nothing to mend, or not applicable, or the
corrections applied); 3 -- the examination found somewhat to mend; 1 -- an error,
or an apply that did not succeed (whereupon a reverting was performed).

## Of Undoing the Work

Before telegraf.conf is touched, a backup `telegraf.conf.bak-<date>` is laid by.
The service is restarted only if a correction be applied that requireth it; if
the rotation alone was changed, the service is left untouched.

After the restart the script waiteth twenty seconds, and proveth that telegraf
runneth (supervisor and process both) and that in the fresh log there be no fatal
errors. Should telegraf not rise again, a reverting is done of itself: the config
is brought back from the backup, the scripts return unto /root/bin, the service
is restarted with the former config, and the return-code is 1.

Upon such a failure, send to Support: the whole of the script's output, the
output of `telegraf --test --config /usr/local/etc/telegraf.conf`, and the tail
of /var/log/telegraf/telegraf.log.

## Known Limitations

- Scripts to which the config referreth, yet which are found neither in
  /root/bin nor in /usr/local/libexec/telegraf -- these are a stoppage, to be
  mended by hand (the report will show MISSING)
- A perilous rotation-line dwelling outside
  /usr/local/etc/newsyslog.conf.d/telegraf.conf (as in /etc/newsyslog.conf) --
  a stoppage; remove it by hand and restart