434 lines
13 KiB
Markdown
434 lines
13 KiB
Markdown
|
<h2><img align="middle" src="https://raw.githubusercontent.com/odb/official-bash-logo/master/assets/Logos/Icons/PNG/64x64.png" >
|
||
|
Octokuma - Easy Monitor and Reboot Lightsail Instances
|
||
|
</h2>
|
||
|
|
||
|
**octokuma** is a script designed to monitor specific Uptime Kuma monitors and automatically reboot corresponding Amazon Lightsail instances if any of them go down. It leverages the `octosail` script to manage the Lightsail instances and can be scheduled to run at regular intervals using systemd timers.
|
||
|
|
||
|
## Table of Contents
|
||
|
|
||
|
- [Features](#features)
|
||
|
- [How It Works](#how-it-works)
|
||
|
- [Prerequisites](#prerequisites)
|
||
|
- [Installation](#installation)
|
||
|
- [Configuration](#configuration)
|
||
|
- [Automatic Configuration](#automatic-configuration)
|
||
|
- [Manual Configuration](#manual-configuration)
|
||
|
- [Running the Script](#running-the-script)
|
||
|
- [Setting Up Systemd Timer](#setting-up-systemd-timer)
|
||
|
- [Create the Service File](#create-the-service-file)
|
||
|
- [Create the Timer File](#create-the-timer-file)
|
||
|
- [Enable and Start the Timer](#enable-and-start-the-timer)
|
||
|
- [Logging](#logging)
|
||
|
- [Troubleshooting](#troubleshooting)
|
||
|
- [Additional Notes](#additional-notes)
|
||
|
- [Running the Script as Current User](#running-the-script-as-current-user)
|
||
|
- [Environment Variables](#environment-variables)
|
||
|
- [Ensuring AWS CLI Access](#ensuring-aws-cli-access)
|
||
|
- [Permissions for octosail Script](#permissions-for-octosail-script)
|
||
|
- [Testing the Setup](#testing-the-setup)
|
||
|
- [Monitoring the Service](#monitoring-the-service)
|
||
|
- [Stopping the Timer](#stopping-the-timer)
|
||
|
- [License](#free-software-license)
|
||
|
|
||
|
## Features
|
||
|
|
||
|
- Monitors specified Uptime Kuma monitors.
|
||
|
- Automatically reboots corresponding Amazon Lightsail instances if a monitor is down.
|
||
|
- Interactive configuration setup using `whiptail`.
|
||
|
- Dependency checks and automatic installation prompts.
|
||
|
- Designed to run as a systemd service for regular monitoring.
|
||
|
- **Runs as the current user** to access user-specific configurations.
|
||
|
- **Logs are stored in `~/.local/log/`** to keep your home directory organized.
|
||
|
|
||
|
## How It Works
|
||
|
|
||
|
1. **Dependency Checks:**
|
||
|
- Ensures that `jq`, `whiptail`, and `octosail` are installed.
|
||
|
- Prompts the user to install any missing dependencies.
|
||
|
|
||
|
2. **Configuration Loading:**
|
||
|
- Reads configuration from a JSON file located in the user's home directory.
|
||
|
- If the configuration file is not found, prompts the user to create it interactively.
|
||
|
|
||
|
3. **Monitoring Logic:**
|
||
|
- Iterates over each monitor specified in the configuration.
|
||
|
- Checks the status of each monitor via Uptime Kuma's API.
|
||
|
- If a monitor is down, uses `octosail` to reboot the associated Lightsail instance.
|
||
|
|
||
|
## Prerequisites
|
||
|
|
||
|
- **AWS CLI:** The script requires the AWS Command Line Interface (CLI) to interact with your AWS account.
|
||
|
- **AWS Credentials:** You need to have your AWS credentials configured. The `octosail` script will help you set this up if not already configured.
|
||
|
- **Dependencies:**
|
||
|
- `jq`: For JSON parsing.
|
||
|
- `whiptail`: For interactive prompts.
|
||
|
- `octosail`: Script to manage Lightsail instances.
|
||
|
|
||
|
## Installation
|
||
|
|
||
|
1. **Download the Script:**
|
||
|
|
||
|
```bash
|
||
|
sudo curl -L "https://git.vdm.dev/api/v1/repos/octoleo/octokuma/raw/src/octokuma" -o /usr/local/bin/octokuma
|
||
|
sudo chmod +x /usr/local/bin/octokuma
|
||
|
```
|
||
|
|
||
|
2. **Ensure Dependencies are Installed:**
|
||
|
|
||
|
The script will check for dependencies and prompt you to install any that are missing when you run it for the first time.
|
||
|
|
||
|
## Configuration
|
||
|
|
||
|
The script uses a JSON configuration file to store settings, including the Uptime Kuma URL and the list of monitors to check.
|
||
|
|
||
|
### Automatic Configuration
|
||
|
|
||
|
When you run the script for the first time, it will check for the configuration file. If not found, it will prompt you to create it interactively using `whiptail`.
|
||
|
|
||
|
**Steps:**
|
||
|
|
||
|
1. **Run the Script:**
|
||
|
|
||
|
```bash
|
||
|
octokuma
|
||
|
```
|
||
|
|
||
|
2. **Follow the Prompts:**
|
||
|
|
||
|
- **Uptime Kuma URL:**
|
||
|
- You will be asked to enter your Uptime Kuma URL.
|
||
|
- **Add Monitors:**
|
||
|
- The script will ask if you want to add a monitor.
|
||
|
- If you select "Yes," you will be prompted to enter the Monitor ID and the corresponding Lightsail Instance Name.
|
||
|
- Repeat the process to add multiple monitors.
|
||
|
- **Save Configuration:**
|
||
|
- Once you have added all monitors, the configuration will be saved to the default path (`$HOME/.config/octokuma/config.json`).
|
||
|
|
||
|
### Manual Configuration
|
||
|
|
||
|
If you prefer to create or edit the configuration file manually, follow these steps:
|
||
|
|
||
|
1. **Create the Configuration Directory:**
|
||
|
|
||
|
```bash
|
||
|
mkdir -p $HOME/.config/octokuma
|
||
|
```
|
||
|
|
||
|
2. **Create the Configuration File:**
|
||
|
|
||
|
```bash
|
||
|
nano $HOME/.config/octokuma/config.json
|
||
|
```
|
||
|
|
||
|
3. **Add the Following Content:**
|
||
|
|
||
|
```json
|
||
|
{
|
||
|
"uptime_kuma_url": "https://your-uptime-kuma-url",
|
||
|
"monitors": [
|
||
|
{
|
||
|
"monitor_id": 24,
|
||
|
"instance_name": "YourInstanceName1"
|
||
|
},
|
||
|
{
|
||
|
"monitor_id": 25,
|
||
|
"instance_name": "YourInstanceName2"
|
||
|
}
|
||
|
]
|
||
|
}
|
||
|
```
|
||
|
|
||
|
- Replace `"https://your-uptime-kuma-url"` with your actual Uptime Kuma URL.
|
||
|
- Replace `monitor_id` and `instance_name` with your actual monitor IDs and corresponding Lightsail instance names.
|
||
|
|
||
|
4. **Save and Close the File:**
|
||
|
|
||
|
Press `Ctrl+O` to save and `Ctrl+X` to exit.
|
||
|
|
||
|
**Note:** You can also specify a custom configuration file path using the `--config` option:
|
||
|
|
||
|
```bash
|
||
|
octokuma --config /path/to/your/config.json
|
||
|
```
|
||
|
|
||
|
## Running the Script
|
||
|
|
||
|
To run the script manually, simply execute:
|
||
|
|
||
|
```bash
|
||
|
octokuma
|
||
|
```
|
||
|
|
||
|
- If you have specified a custom configuration file, use:
|
||
|
|
||
|
```bash
|
||
|
octokuma --config /path/to/your/config.json
|
||
|
```
|
||
|
|
||
|
- Running the script manually allows you to ensure that it functions correctly before automating it with systemd.
|
||
|
|
||
|
## Setting Up Systemd Timer
|
||
|
|
||
|
To run `octokuma` at regular intervals (e.g., every 5 minutes), you can set up a systemd service and timer. We'll configure the service to run as the current user to ensure that the script can access user-specific configurations.
|
||
|
|
||
|
### Create the Service File
|
||
|
|
||
|
Create a file named `octokuma.service` in your user's systemd directory:
|
||
|
|
||
|
```bash
|
||
|
mkdir -p ~/.config/systemd/user
|
||
|
nano ~/.config/systemd/user/octokuma.service
|
||
|
```
|
||
|
|
||
|
Add the following content:
|
||
|
|
||
|
```ini
|
||
|
[Unit]
|
||
|
Description=OctoKuma Monitoring Service
|
||
|
|
||
|
[Service]
|
||
|
Type=oneshot
|
||
|
ExecStart=/usr/local/bin/octokuma
|
||
|
User=%U
|
||
|
Group=%G
|
||
|
Environment="PATH=$HOME/.local/bin:/usr/local/bin:/usr/bin:/bin"
|
||
|
StandardOutput=append:%h/.local/log/octokuma.log
|
||
|
StandardError=append:%h/.local/log/octokuma.log
|
||
|
```
|
||
|
|
||
|
**Explanation:**
|
||
|
|
||
|
- **User and Group:**
|
||
|
- `User=%U` and `Group=%G` ensure that the service runs as the current user and group.
|
||
|
- **Environment:**
|
||
|
- Sets the `PATH` environment variable to include common binary directories. Adjust as needed.
|
||
|
- **StandardOutput and StandardError:**
|
||
|
- Redirects output and errors to a log file at `~/.local/log/octokuma.log`.
|
||
|
- **User Service Directory:**
|
||
|
- By placing the service file in `~/.config/systemd/user/`, we create a user-level systemd service.
|
||
|
|
||
|
### Create the Timer File
|
||
|
|
||
|
Create a file named `octokuma.timer` in the same directory:
|
||
|
|
||
|
```bash
|
||
|
nano ~/.config/systemd/user/octokuma.timer
|
||
|
```
|
||
|
|
||
|
Add the following content:
|
||
|
|
||
|
```ini
|
||
|
[Unit]
|
||
|
Description=Run OctoKuma Monitoring Service every 5 minutes
|
||
|
|
||
|
[Timer]
|
||
|
OnBootSec=1min
|
||
|
OnUnitActiveSec=5min
|
||
|
Unit=octokuma.service
|
||
|
|
||
|
[Install]
|
||
|
WantedBy=timers.target
|
||
|
```
|
||
|
|
||
|
### Enable and Start the Timer
|
||
|
|
||
|
Reload the user systemd daemon to recognize the new unit files:
|
||
|
|
||
|
```bash
|
||
|
systemctl --user daemon-reload
|
||
|
```
|
||
|
|
||
|
Enable and start the timer:
|
||
|
|
||
|
```bash
|
||
|
systemctl --user enable --now octokuma.timer
|
||
|
```
|
||
|
|
||
|
**Note:** If you get an error about `Failed to connect to bus: No such file or directory`, you may need to enable lingering for your user:
|
||
|
|
||
|
```bash
|
||
|
sudo loginctl enable-linger $USER
|
||
|
```
|
||
|
|
||
|
### Verify the Timer
|
||
|
|
||
|
Check the status of the timer:
|
||
|
|
||
|
```bash
|
||
|
systemctl --user list-timers --all | grep octokuma
|
||
|
```
|
||
|
|
||
|
You should see something like:
|
||
|
|
||
|
```
|
||
|
Tue YYYY-MM-DD HH:MM:SS UTC 5min left n/a n/a octokuma.timer octokuma.service
|
||
|
```
|
||
|
|
||
|
## Logging
|
||
|
|
||
|
By configuring the service to run as the current user and redirecting output to a log file in `~/.local/log/`, you can easily monitor the script's activity without cluttering your home directory.
|
||
|
|
||
|
- **Log File Location:** The log file will be located at `~/.local/log/octokuma.log`.
|
||
|
|
||
|
- **Create Log Directory:** Ensure the log directory exists and set appropriate permissions:
|
||
|
|
||
|
```bash
|
||
|
mkdir -p ~/.local/log
|
||
|
```
|
||
|
|
||
|
- **Permissions:** Since the log file is in your home directory, no special permissions are needed.
|
||
|
|
||
|
Reload the user systemd daemon to apply changes:
|
||
|
|
||
|
```bash
|
||
|
systemctl --user daemon-reload
|
||
|
```
|
||
|
|
||
|
## Troubleshooting
|
||
|
|
||
|
- **Dependencies Not Installed:**
|
||
|
|
||
|
If the script reports that `jq`, `whiptail`, or `octosail` are not installed, follow the prompts to install them. You can also install them manually:
|
||
|
|
||
|
```bash
|
||
|
# Install jq
|
||
|
sudo apt-get install -y jq # Debian/Ubuntu
|
||
|
sudo yum install -y jq # CentOS/RHEL
|
||
|
brew install jq # macOS
|
||
|
|
||
|
# Install whiptail
|
||
|
sudo apt-get install -y whiptail # Debian/Ubuntu
|
||
|
sudo yum install -y newt # CentOS/RHEL
|
||
|
brew install newt # macOS
|
||
|
|
||
|
# Install octosail
|
||
|
sudo curl -L "https://git.vdm.dev/api/v1/repos/octoleo/octosail/raw/src/octosail" -o /usr/local/bin/octosail
|
||
|
sudo chmod +x /usr/local/bin/octosail
|
||
|
```
|
||
|
|
||
|
- **Configuration File Issues:**
|
||
|
|
||
|
Ensure that the JSON configuration file is correctly formatted. You can validate the JSON file using `jq`:
|
||
|
|
||
|
```bash
|
||
|
jq '.' /path/to/config.json
|
||
|
```
|
||
|
|
||
|
If there are errors, fix them in the configuration file.
|
||
|
|
||
|
- **Script Not Executing:**
|
||
|
|
||
|
Ensure that the script has execute permissions:
|
||
|
|
||
|
```bash
|
||
|
sudo chmod +x /usr/local/bin/octokuma
|
||
|
```
|
||
|
|
||
|
- **Permissions Issues:**
|
||
|
|
||
|
Since the script runs as your user, ensure that your user has the necessary permissions to:
|
||
|
|
||
|
- Access the AWS CLI and manage Lightsail instances.
|
||
|
- Read and write to the configuration and log files in your home directory.
|
||
|
|
||
|
- **AWS CLI Configuration:**
|
||
|
|
||
|
Ensure that the AWS CLI is installed and configured with the necessary credentials. Run `aws configure` as your user:
|
||
|
|
||
|
```bash
|
||
|
aws configure
|
||
|
```
|
||
|
|
||
|
- **Systemd User Services Not Working:**
|
||
|
|
||
|
Ensure that your system supports user-level systemd services. If you encounter issues, you may need to enable lingering for your user:
|
||
|
|
||
|
```bash
|
||
|
sudo loginctl enable-linger $USER
|
||
|
```
|
||
|
|
||
|
This allows your user-level services to run even when you're not logged in.
|
||
|
|
||
|
## Additional Notes
|
||
|
|
||
|
### Running the Script as Current User
|
||
|
|
||
|
By configuring the systemd service to run as the current user, the script can access user-specific configurations and environments. This approach ensures that:
|
||
|
|
||
|
- The script uses the AWS credentials configured for your user.
|
||
|
- The script reads the configuration file from your home directory.
|
||
|
- There are no permission issues with accessing files in your home directory.
|
||
|
|
||
|
### Environment Variables
|
||
|
|
||
|
Systemd services have a limited environment. To ensure the script has access to the necessary environment variables (like `PATH`), we've added the `Environment` directive in the service file.
|
||
|
|
||
|
```ini
|
||
|
Environment="PATH=$HOME/.local/bin:/usr/local/bin:/usr/bin:/bin"
|
||
|
```
|
||
|
|
||
|
- Adjust the `PATH` as needed based on where your commands are located.
|
||
|
|
||
|
### Ensuring AWS CLI Access
|
||
|
|
||
|
Make sure that your user has the AWS CLI configured in their home directory (`~/.aws/credentials` and `~/.aws/config`).
|
||
|
|
||
|
- Run `aws configure` as your user if you haven't already.
|
||
|
- Verify that you can run AWS CLI commands without issues.
|
||
|
|
||
|
### Permissions for octosail Script
|
||
|
|
||
|
Ensure that the `octosail` script is executable and accessible to your user.
|
||
|
|
||
|
- Install `octosail` in a directory accessible to your user (e.g., `/usr/local/bin`).
|
||
|
- Set the execute permission:
|
||
|
|
||
|
```bash
|
||
|
sudo chmod +x /usr/local/bin/octosail
|
||
|
```
|
||
|
|
||
|
### Testing the Setup
|
||
|
|
||
|
Before relying on the automated systemd timer, test the entire setup manually:
|
||
|
|
||
|
1. **Run `octokuma` Manually:**
|
||
|
|
||
|
```bash
|
||
|
octokuma
|
||
|
```
|
||
|
|
||
|
2. **Check for Errors:**
|
||
|
|
||
|
- Resolve any issues that arise.
|
||
|
- Ensure that the script can access the configuration file and AWS CLI.
|
||
|
|
||
|
3. **Check Logs:**
|
||
|
|
||
|
- If you have set up logging, check the log file (`~/.local/log/octokuma.log`) for any errors or warnings.
|
||
|
|
||
|
### Monitoring the Service
|
||
|
|
||
|
You can check the status of the service and timer using:
|
||
|
|
||
|
```bash
|
||
|
systemctl --user status octokuma.service
|
||
|
systemctl --user status octokuma.timer
|
||
|
```
|
||
|
|
||
|
### Stopping the Timer
|
||
|
|
||
|
If you need to stop the timer, you can disable it:
|
||
|
|
||
|
```bash
|
||
|
systemctl --user disable --now octokuma.timer
|
||
|
```
|
||
|
|
||
|
---
|
||
|
# Free Software License
|
||
|
```txt
|
||
|
@copyright Copyright (C) 2021 Llewellyn van der Merwe. All rights reserved.
|
||
|
@license GNU General Public License version 2; see LICENSE
|
||
|
```
|
||
|
|
||
|
**Disclaimer:** Be cautious when using scripts that can reboot servers. Ensure that you have proper access controls and safeguards in place to prevent unintended actions. Test the script thoroughly in a safe environment before deploying it to production.
|