mirror of
https://github.com/frappe/bench.git
synced 2025-01-09 08:30:39 +00:00
Merge branch 'staging' into v5.x
This commit is contained in:
Ankush Menat
commit
c886d74a03
15
.editorconfig
Normal file
15
.editorconfig
Normal file
@ -0,0 +1,15 @@
|
||||
# Root editor config file
|
||||
root = true
|
||||
|
||||
# Common settings
|
||||
[*]
|
||||
end_of_line = lf
|
||||
insert_final_newline = true
|
||||
trim_trailing_whitespace = true
|
||||
charset = utf-8
|
||||
|
||||
# python, js indentation settings
|
||||
[{*.py,*.js,*.vue,*.css,*.scss,*.html}]
|
||||
indent_style = tab
|
||||
indent_size = 4
|
||||
max_line_length = 99
|
||||
@ -29,8 +29,8 @@ repos:
|
||||
- id: black
|
||||
additional_dependencies: ['click==8.0.4']
|
||||
|
||||
- repo: https://gitlab.com/pycqa/flake8
|
||||
rev: 3.9.2
|
||||
- repo: https://github.com/pycqa/flake8
|
||||
rev: 5.0.4
|
||||
hooks:
|
||||
- id: flake8
|
||||
additional_dependencies: ['flake8-bugbear',]
|
||||
|
||||
74
README.md
74
README.md
@ -30,7 +30,6 @@ Bench is a command-line utility that helps you to install, update, and manage mu
|
||||
|
||||
- [Installation](#installation)
|
||||
- [Containerized Installation](#containerized-installation)
|
||||
- [Easy Install Script](#easy-install-script)
|
||||
- [Manual Installation](#manual-installation)
|
||||
- [Usage](#basic-usage)
|
||||
- [Custom Bench commands](#custom-bench-commands)
|
||||
@ -49,12 +48,11 @@ A typical bench setup provides two types of environments — Development and
|
||||
The setup for each of these installations can be achieved in multiple ways:
|
||||
|
||||
- [Containerized Installation](#containerized-installation)
|
||||
- [Easy Install Script](#easy-install-script)
|
||||
- [Manual Installation](#manual-installation)
|
||||
|
||||
We recommend using either the Docker Installation or the Easy Install Script to setup a Production Environment. For Development, you may choose either of the three methods to setup an instance.
|
||||
We recommend using either the Docker Installation to setup a Production Environment. For Development, you may choose either of the two methods to setup an instance.
|
||||
|
||||
Otherwise, if you are looking to evaluate ERPNext, you can register for [a free trial on erpnext.com](https://erpnext.com/pricing).
|
||||
Otherwise, if you are looking to evaluate Frappe apps without hassle of hosting, you can try them [on frappecloud.com](https://frappecloud.com/).
|
||||
|
||||
|
||||
### Containerized Installation
|
||||
@ -70,53 +68,6 @@ $ cd frappe_docker
|
||||
|
||||
A quick setup guide for both the environments can be found below. For more details, check out the [Frappe/ERPNext Docker Repository](https://github.com/frappe/frappe_docker).
|
||||
|
||||
### Easy Install Script
|
||||
|
||||
The Easy Install script should get you going with a Frappe/ERPNext setup with minimal manual intervention and effort. Since there are a lot of configurations being automatically setup, we recommend executing this script on a fresh server.
|
||||
|
||||
**Note:** This script works only on GNU/Linux based server distributions, and has been designed and tested to work on Ubuntu 16.04+, CentOS 7+, and Debian-based systems.
|
||||
|
||||
> This script installs Version 12 by default. It is untested with Version 13 and above. Containerized or manual installs are recommended for newer setups.
|
||||
|
||||
#### Prerequisites
|
||||
|
||||
You need to install the following packages for the script to run:
|
||||
|
||||
- ##### Ubuntu and Debian-based Distributions:
|
||||
|
||||
```sh
|
||||
$ apt install python3-minimal build-essential python3-setuptools
|
||||
```
|
||||
|
||||
- ##### CentOS and other RPM Distributions:
|
||||
|
||||
```sh
|
||||
$ dnf groupinstall "Development Tools"
|
||||
$ dnf install python3
|
||||
```
|
||||
|
||||
#### Setup
|
||||
|
||||
Download the Easy Install script and execute it:
|
||||
|
||||
```sh
|
||||
$ wget https://raw.githubusercontent.com/frappe/bench/develop/install.py
|
||||
$ python3 install.py --production
|
||||
```
|
||||
|
||||
The script should then prompt you for the MySQL root password and an Administrator password for the Frappe/ERPNext instance, which will then be saved under `$HOME/passwords.txt` of the user used to setup the instance. This script will then install the required stack, setup bench and a default ERPNext instance.
|
||||
|
||||
When the setup is complete, you will be able to access the system at `http://<your-server-ip>`, wherein you can use the administrator password to login.
|
||||
|
||||
#### Troubleshooting
|
||||
|
||||
In case the setup fails, the log file is saved under `/tmp/logs/install_bench.log`. You may then:
|
||||
|
||||
- Create an Issue in this repository with the log file attached.
|
||||
- Search for an existing issue or post the log file on the [Frappe/ERPNext Discuss Forum](https://discuss.erpnext.com/c/bench) with the tag `installation_problem` under "Install/Update" category.
|
||||
|
||||
For more information and advanced setup instructions, check out the [Easy Install Documentation](https://github.com/frappe/bench/blob/develop/docs/easy_install.md).
|
||||
|
||||
|
||||
### Manual Installation
|
||||
|
||||
@ -129,11 +80,6 @@ You'll have to set up the system dependencies required for setting up a Frappe E
|
||||
$ pip install frappe-bench
|
||||
```
|
||||
|
||||
For more extensive distribution-dependent documentation, check out the following guides:
|
||||
|
||||
- [Hitchhiker's Guide to Installing Frappe on Linux](https://github.com/frappe/frappe/wiki/The-Hitchhiker%27s-Guide-to-Installing-Frappe-on-Linux)
|
||||
- [Hitchhiker's Guide to Installing Frappe on MacOS](https://github.com/frappe/bench/wiki/Setting-up-a-Mac-for-Frappe-ERPNext-Development)
|
||||
|
||||
|
||||
## Basic Usage
|
||||
|
||||
@ -189,22 +135,6 @@ For more in-depth information on commands and their usage, follow [Commands and
|
||||
If you wish to extend the capabilities of bench with your own custom Frappe Application, you may follow [Adding Custom Bench Commands](https://github.com/frappe/bench/blob/develop/docs/bench_custom_cmd.md).
|
||||
|
||||
|
||||
## Bench Manager
|
||||
|
||||
[Bench Manager](https://github.com/frappe/bench_manager) is a GUI frontend for Bench with the same functionalties. You can install it by executing the following command:
|
||||
|
||||
```sh
|
||||
$ bench setup manager
|
||||
```
|
||||
|
||||
- **Note:** This will create a new site to setup Bench Manager, if you want to set it up on an existing site, run the following commands:
|
||||
|
||||
```sh
|
||||
$ bench get-app https://github.com/frappe/bench_manager.git
|
||||
$ bench --site <sitename> install-app bench_manager
|
||||
```
|
||||
|
||||
|
||||
## Guides
|
||||
|
||||
- [Configuring HTTPS](https://frappe.io/docs/user/en/bench/guides/configuring-https.html)
|
||||
|
||||
@ -198,7 +198,7 @@ class App(AppMeta):
|
||||
|
||||
@step(title="Archiving App {repo}", success="App {repo} Archived")
|
||||
def remove(self, no_backup: bool = False):
|
||||
active_app_path = os.path.join("apps", self.name)
|
||||
active_app_path = os.path.join("apps", self.repo)
|
||||
|
||||
if no_backup:
|
||||
if not os.path.islink(active_app_path):
|
||||
|
||||
@ -147,7 +147,7 @@ class Bench(Base, Validator):
|
||||
|
||||
if conf.get("developer_mode"):
|
||||
restart_process_manager(bench_path=self.name, web_workers=web)
|
||||
if supervisor and conf.get("restart_supervisor_on_update"):
|
||||
if supervisor or conf.get("restart_supervisor_on_update"):
|
||||
restart_supervisor_processes(bench_path=self.name, web_workers=web)
|
||||
if systemd and conf.get("restart_systemd_on_update"):
|
||||
restart_systemd_processes(bench_path=self.name, web_workers=web)
|
||||
|
||||
@ -15,6 +15,8 @@ default_config = {
|
||||
"live_reload": True,
|
||||
}
|
||||
|
||||
DEFAULT_MAX_REQUESTS = 5000
|
||||
|
||||
|
||||
def setup_config(bench_path):
|
||||
make_pid_folder(bench_path)
|
||||
@ -62,6 +64,20 @@ def get_gunicorn_workers():
|
||||
return {"gunicorn_workers": multiprocessing.cpu_count() * 2 + 1}
|
||||
|
||||
|
||||
def compute_max_requests_jitter(max_requests: int) -> int:
|
||||
return int(max_requests * 0.1)
|
||||
|
||||
|
||||
def get_default_max_requests(worker_count: int):
|
||||
"""Get max requests and jitter config based on number of available workers."""
|
||||
|
||||
if worker_count <= 1:
|
||||
# If there's only one worker then random restart can cause spikes in response times and
|
||||
# can be annoying. Hence not enabled by default.
|
||||
return 0
|
||||
return DEFAULT_MAX_REQUESTS
|
||||
|
||||
|
||||
def update_config_for_frappe(config, bench_path):
|
||||
ports = make_ports(bench_path)
|
||||
|
||||
|
||||
@ -8,7 +8,12 @@ import bench
|
||||
from bench.app import use_rq
|
||||
from bench.utils import get_bench_name, which
|
||||
from bench.bench import Bench
|
||||
from bench.config.common_site_config import update_config, get_gunicorn_workers
|
||||
from bench.config.common_site_config import (
|
||||
update_config,
|
||||
get_gunicorn_workers,
|
||||
get_default_max_requests,
|
||||
compute_max_requests_jitter,
|
||||
)
|
||||
|
||||
# imports - third party imports
|
||||
import click
|
||||
@ -26,6 +31,13 @@ def generate_supervisor_config(bench_path, user=None, yes=False, skip_redis=Fals
|
||||
template = bench.config.env().get_template("supervisor.conf")
|
||||
bench_dir = os.path.abspath(bench_path)
|
||||
|
||||
web_worker_count = config.get(
|
||||
"gunicorn_workers", get_gunicorn_workers()["gunicorn_workers"]
|
||||
)
|
||||
max_requests = config.get(
|
||||
"gunicorn_max_requests", get_default_max_requests(web_worker_count)
|
||||
)
|
||||
|
||||
config = template.render(
|
||||
**{
|
||||
"bench_dir": bench_dir,
|
||||
@ -39,9 +51,9 @@ def generate_supervisor_config(bench_path, user=None, yes=False, skip_redis=Fals
|
||||
"redis_socketio_config": os.path.join(bench_dir, "config", "redis_socketio.conf"),
|
||||
"redis_queue_config": os.path.join(bench_dir, "config", "redis_queue.conf"),
|
||||
"webserver_port": config.get("webserver_port", 8000),
|
||||
"gunicorn_workers": config.get(
|
||||
"gunicorn_workers", get_gunicorn_workers()["gunicorn_workers"]
|
||||
),
|
||||
"gunicorn_workers": web_worker_count,
|
||||
"gunicorn_max_requests": max_requests,
|
||||
"gunicorn_max_requests_jitter": compute_max_requests_jitter(max_requests),
|
||||
"bench_name": get_bench_name(bench_path),
|
||||
"background_workers": config.get("background_workers") or 1,
|
||||
"bench_cmd": which("bench"),
|
||||
|
||||
@ -9,7 +9,12 @@ import click
|
||||
import bench
|
||||
from bench.app import use_rq
|
||||
from bench.bench import Bench
|
||||
from bench.config.common_site_config import get_gunicorn_workers, update_config
|
||||
from bench.config.common_site_config import (
|
||||
get_gunicorn_workers,
|
||||
update_config,
|
||||
get_default_max_requests,
|
||||
compute_max_requests_jitter,
|
||||
)
|
||||
from bench.utils import exec_cmd, which, get_bench_name
|
||||
|
||||
|
||||
@ -61,6 +66,13 @@ def generate_systemd_config(
|
||||
get_bench_name(bench_path) + "-frappe-long-worker@" + str(i + 1) + ".service"
|
||||
)
|
||||
|
||||
web_worker_count = config.get(
|
||||
"gunicorn_workers", get_gunicorn_workers()["gunicorn_workers"]
|
||||
)
|
||||
max_requests = config.get(
|
||||
"gunicorn_max_requests", get_default_max_requests(web_worker_count)
|
||||
)
|
||||
|
||||
bench_info = {
|
||||
"bench_dir": bench_dir,
|
||||
"sites_dir": os.path.join(bench_dir, "sites"),
|
||||
@ -73,9 +85,9 @@ def generate_systemd_config(
|
||||
"redis_socketio_config": os.path.join(bench_dir, "config", "redis_socketio.conf"),
|
||||
"redis_queue_config": os.path.join(bench_dir, "config", "redis_queue.conf"),
|
||||
"webserver_port": config.get("webserver_port", 8000),
|
||||
"gunicorn_workers": config.get(
|
||||
"gunicorn_workers", get_gunicorn_workers()["gunicorn_workers"]
|
||||
),
|
||||
"gunicorn_workers": web_worker_count,
|
||||
"gunicorn_max_requests": max_requests,
|
||||
"gunicorn_max_requests_jitter": compute_max_requests_jitter(max_requests),
|
||||
"bench_name": get_bench_name(bench_path),
|
||||
"worker_target_wants": " ".join(background_workers),
|
||||
"bench_cmd": which("bench"),
|
||||
@ -93,7 +105,7 @@ def generate_systemd_config(
|
||||
setup_web_config(bench_info, bench_path)
|
||||
setup_redis_config(bench_info, bench_path)
|
||||
|
||||
update_config({"restart_systemd_on_update": True}, bench_path=bench_path)
|
||||
update_config({"restart_systemd_on_update": False}, bench_path=bench_path)
|
||||
update_config({"restart_supervisor_on_update": False}, bench_path=bench_path)
|
||||
|
||||
|
||||
|
||||
@ -11,9 +11,7 @@ watch: bench watch
|
||||
{% endif %}
|
||||
{% if use_rq -%}
|
||||
schedule: bench schedule
|
||||
worker_short: bench worker --queue short 1>> logs/worker.log 2>> logs/worker.error.log
|
||||
worker_long: bench worker --queue long 1>> logs/worker.log 2>> logs/worker.error.log
|
||||
worker_default: bench worker --queue default 1>> logs/worker.log 2>> logs/worker.error.log
|
||||
worker: bench worker 1>> logs/worker.log 2>> logs/worker.error.log
|
||||
{% for worker_name, worker_details in workers.items() %}
|
||||
worker_{{ worker_name }}: bench worker --queue {{ worker_name }} 1>> logs/worker.log 2>> logs/worker.error.log
|
||||
{% endfor %}
|
||||
|
||||
@ -3,7 +3,7 @@
|
||||
; killasgroup=true --> send kill signal to child processes too
|
||||
|
||||
[program:{{ bench_name }}-frappe-web]
|
||||
command={{ bench_dir }}/env/bin/gunicorn -b 127.0.0.1:{{ webserver_port }} -w {{ gunicorn_workers }} -t {{ http_timeout }} frappe.app:application --preload
|
||||
command={{ bench_dir }}/env/bin/gunicorn -b 127.0.0.1:{{ webserver_port }} -w {{ gunicorn_workers }} --max-requests {{ gunicorn_max_requests }} --max-requests-jitter {{ gunicorn_max_requests_jitter }} -t {{ http_timeout }} frappe.app:application --preload
|
||||
priority=4
|
||||
autostart=true
|
||||
autorestart=true
|
||||
|
||||
@ -6,7 +6,7 @@ PartOf={{ bench_name }}-web.target
|
||||
User={{ user }}
|
||||
Group={{ user }}
|
||||
Restart=always
|
||||
ExecStart={{ bench_dir }}/env/bin/gunicorn -b 127.0.0.1:{{ webserver_port }} -w {{ gunicorn_workers }} -t {{ http_timeout }} frappe.app:application --preload
|
||||
ExecStart={{ bench_dir }}/env/bin/gunicorn -b 127.0.0.1:{{ webserver_port }} -w {{ gunicorn_workers }} -t {{ http_timeout }} --max-requests {{ gunicorn_max_requests }} --max-requests-jitter {{ gunicorn_max_requests_jitter }} frappe.app:application --preload
|
||||
StandardOutput=file:{{ bench_dir }}/logs/web.log
|
||||
StandardError=file:{{ bench_dir }}/logs/web.error.log
|
||||
WorkingDirectory={{ sites_dir }}
|
||||
|
||||
@ -125,7 +125,10 @@ def check_latest_version():
|
||||
local_version = Version(VERSION)
|
||||
|
||||
if pypi_version > local_version:
|
||||
log(f"A newer version of bench is available: {local_version} → {pypi_version}", stderr=True)
|
||||
log(
|
||||
f"A newer version of bench is available: {local_version} → {pypi_version}",
|
||||
stderr=True,
|
||||
)
|
||||
|
||||
|
||||
def pause_exec(seconds=10):
|
||||
|
||||
@ -185,7 +185,9 @@ def get_required_deps(org, name, branch, deps="hooks.py"):
|
||||
res = requests.get(url=git_api_url, params=params).json()
|
||||
|
||||
if "message" in res:
|
||||
git_url = f"https://raw.githubusercontent.com/{org}/{name}/{params['ref']}/{name}/{deps}"
|
||||
git_url = (
|
||||
f"https://raw.githubusercontent.com/{org}/{name}/{params['ref']}/{name}/{deps}"
|
||||
)
|
||||
return requests.get(git_url).text
|
||||
|
||||
return base64.decodebytes(res["content"].encode()).decode()
|
||||
|
||||
@ -1,93 +0,0 @@
|
||||
# Easy Install Script
|
||||
|
||||
- This script will install the pre-requisites, install bench and setup an ERPNext site `(site1.local under frappe-bench)`
|
||||
- Passwords for Frappe Administrator and MariaDB (root) will be asked and saved under `~/passwords.txt`
|
||||
- MariaDB (root) password may be `password` on a fresh server
|
||||
- You can then login as **Administrator** with the Administrator password
|
||||
- The log file is saved under `/tmp/logs/install_bench.log` in case you run into any issues during the install.
|
||||
- If you find any problems, post them on the forum: [https://discuss.erpnext.com](https://discuss.erpnext.com/tags/installation_problem) under the "Install / Update" category.
|
||||
|
||||
---
|
||||
|
||||
## What will this script do?
|
||||
|
||||
- Install all the pre-requisites
|
||||
- Install the command line `bench` (under ~/.bench)
|
||||
- Create a new bench (a folder that will contain your entire frappe/erpnext setup at ~/frappe-bench)
|
||||
- Create a new ERPNext site on the bench (site1.local)
|
||||
|
||||
---
|
||||
|
||||
## Getting started with easy install...
|
||||
|
||||
Open your Terminal and enter:
|
||||
|
||||
#### 0. Setup user & Download the install script
|
||||
|
||||
If you are on a fresh server and logged in as root, at first create a dedicated user for frappe
|
||||
& equip this user with sudo privileges
|
||||
|
||||
```
|
||||
adduser [frappe-user]
|
||||
usermod -aG sudo [frappe-user]
|
||||
```
|
||||
|
||||
*(it is very common to use "frappe" as frappe-username, but this comes with the security flaw of ["frappe" ranking very high](https://www.reddit.com/r/dataisbeautiful/comments/b3sirt/i_deployed_over_a_dozen_cyber_honeypots_all_over/?st=JTJ0SC0Q&sh=76e05240) in as a username challenged in hacking attempts. So, for production sites it is highly recommended to use a custom username harder to guess)*
|
||||
|
||||
*(you can specify the flag --home to specify a directory for your [frappe-user]. Bench will follow the home directory specified by the user's home directory e.g. /data/[frappe-user]/frappe-bench)*
|
||||
|
||||
Switch to `[frappe-user]` (using `su [frappe-user]`) and start the setup
|
||||
|
||||
wget https://raw.githubusercontent.com/frappe/bench/develop/install.py
|
||||
|
||||
|
||||
#### 1. Run the install script
|
||||
|
||||
sudo python3 install.py
|
||||
|
||||
*Note: `user` flag to create a user and install using that user (By default, the script will create a user with the username `frappe` if the --user flag is not used)*
|
||||
|
||||
For production or development, append the `--production` or `--develop` flag to the command respectively.
|
||||
|
||||
sudo python3 install.py --production --user [frappe-user]
|
||||
|
||||
or
|
||||
|
||||
sudo python3 install.py --develop
|
||||
sudo python3 install.py --develop --user [frappe-user]
|
||||
|
||||
sudo python3 install.py --production --user [frappe-user] --container
|
||||
|
||||
*Note: `container` flag to install inside a container (this will prevent the `/proc/sys/vm/swappiness: Read-only` file system error)*
|
||||
|
||||
|
||||
python3 install.py --production --version 11 --user [frappe-user]
|
||||
|
||||
use --version flag to install specific version
|
||||
|
||||
python3 install.py --production --version 11 --python python2.7 --user [frappe-user]
|
||||
|
||||
use --python flag to specify virtual environments python version, by default script setup python3
|
||||
|
||||
---
|
||||
|
||||
## How do I start ERPNext
|
||||
|
||||
1. For development: Go to your bench folder (`~[frappe-user]/frappe-bench` by default) and start the bench with `bench start`
|
||||
2. For production: Your process will be setup and managed by `nginx` and `supervisor`. Checkout [Setup Production](https://frappe.io/docs/user/en/bench/guides/setup-production.html) for more information.
|
||||
|
||||
---
|
||||
|
||||
## An error occured mid installation?
|
||||
|
||||
TLDR; Save the logs!
|
||||
|
||||
1. The easy install script starts multiple processes to install prerequisites, system dependencies, requirements, sets up locales, configuration files, etc.
|
||||
|
||||
2. The script pipes all these process outputs and saves it under `/tmp/log/{easy-install-filename}.log` as prompted by the script in the beginning of the script or/and if something went wrong again.
|
||||
|
||||
3. Retain this log file and share it in case you need help with proceeding with the install. Since, the file's saved under `/tmp` it'll be cleared by the system after a reboot. Be careful to save it elsewhere if needed!
|
||||
|
||||
3. A lot of things can go wrong in setting up the environment due to prior settings, company protocols or even breaking changes in system packages and their dependencies.
|
||||
|
||||
4. Sharing your logfile in any issues opened related to this can help us find solutions to it faster and make the script better!
|
||||
499
install.py
499
install.py
@ -1,499 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import os
|
||||
import sys
|
||||
import subprocess
|
||||
import getpass
|
||||
import json
|
||||
import multiprocessing
|
||||
import shutil
|
||||
import platform
|
||||
import warnings
|
||||
import datetime
|
||||
|
||||
|
||||
tmp_bench_repo = os.path.join('/', 'tmp', '.bench')
|
||||
tmp_log_folder = os.path.join('/', 'tmp', 'logs')
|
||||
execution_timestamp = datetime.datetime.utcnow()
|
||||
execution_day = "{:%Y-%m-%d}".format(execution_timestamp)
|
||||
execution_time = "{:%H:%M}".format(execution_timestamp)
|
||||
log_file_name = "easy-install__{0}__{1}.log".format(execution_day, execution_time.replace(':', '-'))
|
||||
log_path = os.path.join(tmp_log_folder, log_file_name)
|
||||
log_stream = sys.stdout
|
||||
distro_required = not ((sys.version_info.major < 3) or (sys.version_info.major == 3 and sys.version_info.minor < 5))
|
||||
|
||||
|
||||
def log(message, level=0):
|
||||
levels = {
|
||||
0: '\033[94m', # normal
|
||||
1: '\033[92m', # success
|
||||
2: '\033[91m', # fail
|
||||
3: '\033[93m' # warn/suggest
|
||||
}
|
||||
start = levels.get(level) or ''
|
||||
end = '\033[0m'
|
||||
print(start + message + end)
|
||||
|
||||
|
||||
def setup_log_stream(args):
|
||||
global log_stream
|
||||
sys.stderr = sys.stdout
|
||||
|
||||
if not args.verbose:
|
||||
if not os.path.exists(tmp_log_folder):
|
||||
os.makedirs(tmp_log_folder)
|
||||
log_stream = open(log_path, 'w')
|
||||
log("Logs are saved under {0}".format(log_path), level=3)
|
||||
print("Install script run at {0} on {1}\n\n".format(execution_time, execution_day), file=log_stream)
|
||||
|
||||
|
||||
def check_environment():
|
||||
needed_environ_vars = ['LANG', 'LC_ALL']
|
||||
message = ''
|
||||
|
||||
for var in needed_environ_vars:
|
||||
if var not in os.environ:
|
||||
message += "\nexport {0}=C.UTF-8".format(var)
|
||||
|
||||
if message:
|
||||
log("Bench's CLI needs these to be defined!", level=3)
|
||||
log("Run the following commands in shell: {0}".format(message), level=2)
|
||||
sys.exit()
|
||||
|
||||
|
||||
def check_system_package_managers():
|
||||
if 'Darwin' in os.uname():
|
||||
if not shutil.which('brew'):
|
||||
raise Exception('''
|
||||
Please install brew package manager before proceeding with bench setup. Please run following
|
||||
to install brew package manager on your machine,
|
||||
|
||||
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
|
||||
''')
|
||||
if 'Linux' in os.uname():
|
||||
if not any([shutil.which(x) for x in ['apt-get', 'yum']]):
|
||||
raise Exception('Cannot find any compatible package manager!')
|
||||
|
||||
|
||||
def check_distribution_compatibility():
|
||||
dist_name, dist_version = get_distribution_info()
|
||||
supported_dists = {
|
||||
'macos': [10.9, 10.10, 10.11, 10.12],
|
||||
'ubuntu': [14, 15, 16, 18, 19, 20],
|
||||
'debian': [8, 9, 10],
|
||||
'centos': [7]
|
||||
}
|
||||
|
||||
log("Checking System Compatibility...")
|
||||
if dist_name in supported_dists:
|
||||
if float(dist_version) in supported_dists[dist_name]:
|
||||
log("{0} {1} is compatible!".format(dist_name, dist_version), level=1)
|
||||
else:
|
||||
log("{0} {1} is detected".format(dist_name, dist_version), level=1)
|
||||
log("Install on {0} {1} instead".format(dist_name, supported_dists[dist_name][-1]), level=3)
|
||||
else:
|
||||
log("Sorry, the installer doesn't support {0}. Aborting installation!".format(dist_name), level=2)
|
||||
|
||||
|
||||
def import_with_install(package):
|
||||
# copied from https://discuss.erpnext.com/u/nikunj_patel
|
||||
# https://discuss.erpnext.com/t/easy-install-setup-guide-for-erpnext-installation-on-ubuntu-20-04-lts-with-some-modification-of-course/62375/5
|
||||
# need to move to top said v13 for fully python3 era
|
||||
import importlib
|
||||
|
||||
try:
|
||||
importlib.import_module(package)
|
||||
except ImportError:
|
||||
# caveat : pip3 must be installed
|
||||
|
||||
import pip
|
||||
|
||||
pip.main(['install', package])
|
||||
finally:
|
||||
globals()[package] = importlib.import_module(package)
|
||||
|
||||
|
||||
def get_distribution_info():
|
||||
# return distribution name and major version
|
||||
if platform.system() == "Linux":
|
||||
if distro_required:
|
||||
current_dist = distro.linux_distribution(full_distribution_name=True)
|
||||
else:
|
||||
current_dist = platform.dist()
|
||||
|
||||
return current_dist[0].lower(), current_dist[1].rsplit('.')[0]
|
||||
|
||||
elif platform.system() == "Darwin":
|
||||
current_dist = platform.mac_ver()
|
||||
return "macos", current_dist[0].rsplit('.', 1)[0]
|
||||
|
||||
|
||||
def run_os_command(command_map):
|
||||
'''command_map is a dictionary of {'executable': command}. For ex. {'apt-get': 'sudo apt-get install -y python2.7'}'''
|
||||
success = True
|
||||
|
||||
for executable, commands in command_map.items():
|
||||
if shutil.which(executable):
|
||||
if isinstance(commands, str):
|
||||
commands = [commands]
|
||||
|
||||
for command in commands:
|
||||
returncode = subprocess.check_call(command, shell=True, stdout=log_stream, stderr=sys.stderr)
|
||||
success = success and (returncode == 0)
|
||||
|
||||
return success
|
||||
|
||||
|
||||
def install_prerequisites():
|
||||
# pre-requisites for bench repo cloning
|
||||
run_os_command({
|
||||
'apt-get': [
|
||||
'sudo apt-get update',
|
||||
'sudo apt-get install -y git build-essential python3-setuptools python3-dev libffi-dev'
|
||||
],
|
||||
'yum': [
|
||||
'sudo yum groupinstall -y "Development tools"',
|
||||
'sudo yum install -y epel-release redhat-lsb-core git python-setuptools python-devel openssl-devel libffi-devel'
|
||||
]
|
||||
})
|
||||
|
||||
# until psycopg2-binary is available for aarch64 (Arm 64-bit), we'll need libpq and libssl dev packages to build psycopg2 from source
|
||||
if platform.machine() == 'aarch64':
|
||||
log("Installing libpq and libssl dev packages to build psycopg2 for aarch64...")
|
||||
run_os_command({
|
||||
'apt-get': ['sudo apt-get install -y libpq-dev libssl-dev'],
|
||||
'yum': ['sudo yum install -y libpq-devel openssl-devel']
|
||||
})
|
||||
|
||||
install_package('curl')
|
||||
install_package('wget')
|
||||
install_package('git')
|
||||
install_package('pip3', 'python3-pip')
|
||||
|
||||
run_os_command({
|
||||
'python3': "sudo -H python3 -m pip install --upgrade pip setuptools-rust"
|
||||
})
|
||||
success = run_os_command({
|
||||
'python3': "sudo -H python3 -m pip install --upgrade setuptools wheel cryptography ansible~=2.8.15"
|
||||
})
|
||||
|
||||
if not (success or shutil.which('ansible')):
|
||||
could_not_install('Ansible')
|
||||
|
||||
|
||||
def could_not_install(package):
|
||||
raise Exception('Could not install {0}. Please install it manually.'.format(package))
|
||||
|
||||
|
||||
def is_sudo_user():
|
||||
return os.geteuid() == 0
|
||||
|
||||
|
||||
def install_package(package, package_name=None):
|
||||
if shutil.which(package):
|
||||
log("{0} already installed!".format(package), level=1)
|
||||
else:
|
||||
log("Installing {0}...".format(package))
|
||||
package_name = package_name or package
|
||||
success = run_os_command({
|
||||
'apt-get': ['sudo apt-get install -y {0}'.format(package_name)],
|
||||
'yum': ['sudo yum install -y {0}'.format(package_name)],
|
||||
'brew': ['brew install {0}'.format(package_name)]
|
||||
})
|
||||
if success:
|
||||
log("{0} installed!".format(package), level=1)
|
||||
return success
|
||||
could_not_install(package)
|
||||
|
||||
|
||||
def install_bench(args):
|
||||
# clone bench repo
|
||||
if not args.run_travis:
|
||||
clone_bench_repo(args)
|
||||
|
||||
if not args.user:
|
||||
if args.production:
|
||||
args.user = 'frappe'
|
||||
|
||||
elif 'SUDO_USER' in os.environ:
|
||||
args.user = os.environ['SUDO_USER']
|
||||
|
||||
else:
|
||||
args.user = getpass.getuser()
|
||||
|
||||
if args.user == 'root':
|
||||
raise Exception('Please run this script as a non-root user with sudo privileges, but without using sudo or pass --user=USER')
|
||||
|
||||
# Python executable
|
||||
dist_name, dist_version = get_distribution_info()
|
||||
if dist_name=='centos':
|
||||
args.python = 'python3.6'
|
||||
else:
|
||||
args.python = 'python3'
|
||||
|
||||
# create user if not exists
|
||||
extra_vars = vars(args)
|
||||
extra_vars.update(frappe_user=args.user)
|
||||
|
||||
extra_vars.update(user_directory=get_user_home_directory(args.user))
|
||||
|
||||
if os.path.exists(tmp_bench_repo):
|
||||
repo_path = tmp_bench_repo
|
||||
else:
|
||||
repo_path = os.path.join(os.path.expanduser('~'), 'bench')
|
||||
|
||||
extra_vars.update(repo_path=repo_path)
|
||||
run_playbook('create_user.yml', extra_vars=extra_vars)
|
||||
|
||||
extra_vars.update(get_passwords(args))
|
||||
if args.production:
|
||||
extra_vars.update(max_worker_connections=multiprocessing.cpu_count() * 1024)
|
||||
|
||||
if args.version <= 10:
|
||||
frappe_branch = "{0}.x.x".format(args.version)
|
||||
erpnext_branch = "{0}.x.x".format(args.version)
|
||||
else:
|
||||
frappe_branch = "version-{0}".format(args.version)
|
||||
erpnext_branch = "version-{0}".format(args.version)
|
||||
|
||||
# Allow override of frappe_branch and erpnext_branch, regardless of args.version (which always has a default set)
|
||||
if args.frappe_branch:
|
||||
frappe_branch = args.frappe_branch
|
||||
if args.erpnext_branch:
|
||||
erpnext_branch = args.erpnext_branch
|
||||
|
||||
extra_vars.update(frappe_branch=frappe_branch)
|
||||
extra_vars.update(erpnext_branch=erpnext_branch)
|
||||
|
||||
bench_name = 'frappe-bench' if not args.bench_name else args.bench_name
|
||||
extra_vars.update(bench_name=bench_name)
|
||||
|
||||
# Will install ERPNext production setup by default
|
||||
if args.without_erpnext:
|
||||
log("Initializing bench {bench_name}:\n\tFrappe Branch: {frappe_branch}\n\tERPNext will not be installed due to --without-erpnext".format(bench_name=bench_name, frappe_branch=frappe_branch))
|
||||
else:
|
||||
log("Initializing bench {bench_name}:\n\tFrappe Branch: {frappe_branch}\n\tERPNext Branch: {erpnext_branch}".format(bench_name=bench_name, frappe_branch=frappe_branch, erpnext_branch=erpnext_branch))
|
||||
run_playbook('site.yml', sudo=True, extra_vars=extra_vars)
|
||||
|
||||
if os.path.exists(tmp_bench_repo):
|
||||
shutil.rmtree(tmp_bench_repo)
|
||||
|
||||
|
||||
def clone_bench_repo(args):
|
||||
'''Clones the bench repository in the user folder'''
|
||||
branch = args.bench_branch or 'develop'
|
||||
repo_url = args.repo_url or 'https://github.com/frappe/bench'
|
||||
|
||||
if os.path.exists(tmp_bench_repo):
|
||||
log('Not cloning already existing Bench repository at {tmp_bench_repo}'.format(tmp_bench_repo=tmp_bench_repo))
|
||||
return 0
|
||||
elif args.without_bench_setup:
|
||||
clone_path = os.path.join(os.path.expanduser('~'), 'bench')
|
||||
log('--without-bench-setup specified, clone path is: {clone_path}'.format(clone_path=clone_path))
|
||||
else:
|
||||
clone_path = tmp_bench_repo
|
||||
# Not logging repo_url to avoid accidental credential leak in case credential is embedded in URL
|
||||
log('Cloning bench repository branch {branch} into {clone_path}'.format(branch=branch, clone_path=clone_path))
|
||||
|
||||
success = run_os_command(
|
||||
{'git': 'git clone --quiet {repo_url} {bench_repo} --depth 1 --branch {branch}'.format(
|
||||
repo_url=repo_url, bench_repo=clone_path, branch=branch)}
|
||||
)
|
||||
|
||||
return success
|
||||
|
||||
|
||||
def passwords_didnt_match(context=''):
|
||||
log("{} passwords did not match!".format(context), level=3)
|
||||
|
||||
|
||||
def get_passwords(args):
|
||||
"""
|
||||
Returns a dict of passwords for further use
|
||||
and creates passwords.txt in the bench user's home directory
|
||||
"""
|
||||
log("Input MySQL and Frappe Administrator passwords:")
|
||||
ignore_prompt = args.run_travis or args.without_bench_setup
|
||||
mysql_root_password, admin_password = '', ''
|
||||
passwords_file_path = os.path.join(os.path.expanduser('~' + args.user), 'passwords.txt')
|
||||
|
||||
if not ignore_prompt:
|
||||
# set passwords from existing passwords.txt
|
||||
if os.path.isfile(passwords_file_path):
|
||||
with open(passwords_file_path, 'r') as f:
|
||||
passwords = json.load(f)
|
||||
mysql_root_password, admin_password = passwords['mysql_root_password'], passwords['admin_password']
|
||||
|
||||
# set passwords from cli args
|
||||
if args.mysql_root_password:
|
||||
mysql_root_password = args.mysql_root_password
|
||||
if args.admin_password:
|
||||
admin_password = args.admin_password
|
||||
|
||||
# prompt for passwords
|
||||
pass_set = True
|
||||
while pass_set:
|
||||
# mysql root password
|
||||
if not mysql_root_password:
|
||||
mysql_root_password = getpass.unix_getpass(prompt='Please enter mysql root password: ')
|
||||
conf_mysql_passwd = getpass.unix_getpass(prompt='Re-enter mysql root password: ')
|
||||
|
||||
if mysql_root_password != conf_mysql_passwd or mysql_root_password == '':
|
||||
passwords_didnt_match("MySQL")
|
||||
mysql_root_password = ''
|
||||
continue
|
||||
|
||||
# admin password, only needed if we're also creating a site
|
||||
if not admin_password and not args.without_site:
|
||||
admin_password = getpass.unix_getpass(prompt='Please enter the default Administrator user password: ')
|
||||
conf_admin_passswd = getpass.unix_getpass(prompt='Re-enter Administrator password: ')
|
||||
|
||||
if admin_password != conf_admin_passswd or admin_password == '':
|
||||
passwords_didnt_match("Administrator")
|
||||
admin_password = ''
|
||||
continue
|
||||
elif args.without_site:
|
||||
log("Not creating a new site due to --without-site")
|
||||
|
||||
pass_set = False
|
||||
else:
|
||||
mysql_root_password = admin_password = 'travis'
|
||||
|
||||
passwords = {
|
||||
'mysql_root_password': mysql_root_password,
|
||||
'admin_password': admin_password
|
||||
}
|
||||
|
||||
if not ignore_prompt:
|
||||
with open(passwords_file_path, 'w') as f:
|
||||
json.dump(passwords, f, indent=1)
|
||||
|
||||
log('Passwords saved at ~/passwords.txt')
|
||||
|
||||
return passwords
|
||||
|
||||
|
||||
def get_extra_vars_json(extra_args):
|
||||
# We need to pass production as extra_vars to the playbook to execute conditionals in the
|
||||
# playbook. Extra variables can passed as json or key=value pair. Here, we will use JSON.
|
||||
json_path = os.path.join('/', 'tmp', 'extra_vars.json')
|
||||
extra_vars = dict(list(extra_args.items()))
|
||||
|
||||
with open(json_path, mode='w') as j:
|
||||
json.dump(extra_vars, j, indent=1, sort_keys=True)
|
||||
|
||||
return ('@' + json_path)
|
||||
|
||||
def get_user_home_directory(user):
|
||||
# Return home directory /home/USERNAME or anything else defined as home directory in
|
||||
# passwd for user.
|
||||
return os.path.expanduser('~'+user)
|
||||
|
||||
|
||||
def run_playbook(playbook_name, sudo=False, extra_vars=None):
|
||||
args = ['ansible-playbook', '-c', 'local', playbook_name , '-vvvv']
|
||||
|
||||
if extra_vars:
|
||||
args.extend(['-e', get_extra_vars_json(extra_vars)])
|
||||
|
||||
if sudo:
|
||||
user = extra_vars.get('user') or getpass.getuser()
|
||||
args.extend(['--become', '--become-user={0}'.format(user)])
|
||||
|
||||
if os.path.exists(tmp_bench_repo):
|
||||
cwd = tmp_bench_repo
|
||||
else:
|
||||
cwd = os.path.join(os.path.expanduser('~'), 'bench')
|
||||
|
||||
playbooks_locations = [os.path.join(cwd, 'bench', 'playbooks'), os.path.join(cwd, 'playbooks')]
|
||||
playbooks_folder = [x for x in playbooks_locations if os.path.exists(x)][0]
|
||||
|
||||
success = subprocess.check_call(args, cwd=playbooks_folder, stdout=log_stream, stderr=sys.stderr)
|
||||
return success
|
||||
|
||||
|
||||
def setup_script_requirements():
|
||||
if distro_required:
|
||||
install_package('pip3', 'python3-pip')
|
||||
import_with_install('distro')
|
||||
|
||||
|
||||
def parse_commandline_args():
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(description='Frappe Installer')
|
||||
# Arguments develop and production are mutually exclusive both can't be specified together.
|
||||
# Hence, we need to create a group for discouraging use of both options at the same time.
|
||||
args_group = parser.add_mutually_exclusive_group()
|
||||
|
||||
args_group.add_argument('--develop', dest='develop', action='store_true', default=False, help='Install developer setup')
|
||||
args_group.add_argument('--production', dest='production', action='store_true', default=False, help='Setup Production environment for bench')
|
||||
parser.add_argument('--site', dest='site', action='store', default='site1.local', help='Specify name for your first ERPNext site')
|
||||
parser.add_argument('--without-site', dest='without_site', action='store_true', default=False, help='Do not create a new site')
|
||||
parser.add_argument('--verbose', dest='verbose', action='store_true', default=False, help='Run the script in verbose mode')
|
||||
parser.add_argument('--user', dest='user', help='Install frappe-bench for this user')
|
||||
parser.add_argument('--bench-branch', dest='bench_branch', help='Clone a particular branch of bench repository')
|
||||
parser.add_argument('--repo-url', dest='repo_url', help='Clone bench from the given url')
|
||||
parser.add_argument('--frappe-repo-url', dest='frappe_repo_url', action='store', default='https://github.com/frappe/frappe', help='Clone frappe from the given url')
|
||||
parser.add_argument('--frappe-branch', dest='frappe_branch', action='store', help='Clone a particular branch of frappe')
|
||||
parser.add_argument('--erpnext-repo-url', dest='erpnext_repo_url', action='store', default='https://github.com/frappe/erpnext', help='Clone erpnext from the given url')
|
||||
parser.add_argument('--erpnext-branch', dest='erpnext_branch', action='store', help='Clone a particular branch of erpnext')
|
||||
parser.add_argument('--without-erpnext', dest='without_erpnext', action='store_true', default=False, help='Prevent fetching ERPNext')
|
||||
# direct provision to install versions
|
||||
parser.add_argument('--version', dest='version', action='store', default=13, type=int, help='Clone particular version of frappe and erpnext')
|
||||
# To enable testing of script using Travis, this should skip the prompt
|
||||
parser.add_argument('--run-travis', dest='run_travis', action='store_true', default=False, help=argparse.SUPPRESS)
|
||||
parser.add_argument('--without-bench-setup', dest='without_bench_setup', action='store_true', default=False, help=argparse.SUPPRESS)
|
||||
# whether to overwrite an existing bench
|
||||
parser.add_argument('--overwrite', dest='overwrite', action='store_true', default=False, help='Whether to overwrite an existing bench')
|
||||
# set passwords
|
||||
parser.add_argument('--mysql-root-password', dest='mysql_root_password', help='Set mysql root password')
|
||||
parser.add_argument('--mariadb-version', dest='mariadb_version', default='10.4', help='Specify mariadb version')
|
||||
parser.add_argument('--admin-password', dest='admin_password', help='Set admin password')
|
||||
parser.add_argument('--bench-name', dest='bench_name', help='Create bench with specified name. Default name is frappe-bench')
|
||||
# Python interpreter to be used
|
||||
parser.add_argument('--python', dest='python', default='python3', help=argparse.SUPPRESS)
|
||||
# LXC Support
|
||||
parser.add_argument('--container', dest='container', default=False, action='store_true', help='Use if you\'re creating inside LXC')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
return args
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
if sys.version[0] == '2':
|
||||
if not os.environ.get('CI'):
|
||||
if not raw_input("It is recommended to run this script with Python 3\nDo you still wish to continue? [Y/n]: ").lower() == "y":
|
||||
sys.exit()
|
||||
|
||||
try:
|
||||
from distutils.spawn import find_executable
|
||||
except ImportError:
|
||||
try:
|
||||
subprocess.check_call('pip install --upgrade setuptools')
|
||||
except subprocess.CalledProcessError:
|
||||
print("Install distutils or use Python3 to run the script")
|
||||
sys.exit(1)
|
||||
|
||||
shutil.which = find_executable
|
||||
|
||||
if not is_sudo_user():
|
||||
log("Please run this script as a non-root user with sudo privileges", level=3)
|
||||
sys.exit()
|
||||
|
||||
args = parse_commandline_args()
|
||||
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("ignore")
|
||||
setup_log_stream(args)
|
||||
install_prerequisites()
|
||||
setup_script_requirements()
|
||||
check_distribution_compatibility()
|
||||
check_system_package_managers()
|
||||
check_environment()
|
||||
install_bench(args)
|
||||
|
||||
log("Bench + Frappe + ERPNext has been successfully installed!")
|
||||
Loading…
Reference in New Issue
Block a user