mirror of
https://github.com/frappe/bench.git
synced 2024-11-13 16:56:33 +00:00
Merge pull request #208 from frappe/fix/6-7-compatibility
v6–v7 compatibility
This commit is contained in:
Anand Doshi
commit
eb8fe5635f
@ -17,5 +17,5 @@ script:
|
||||
- cd ~
|
||||
- sudo pip install --upgrade pip
|
||||
- sudo pip install -e bench-repo
|
||||
# - sudo python -m unittest bench.tests.test_setup_production.TestSetupProduction.test_new_site
|
||||
# - sudo python -m unittest bench.tests.test_setup_production.TestSetupProduction.test_setup_production_v6
|
||||
- sudo python -m unittest -v bench.tests.test_setup_production
|
||||
|
||||
265
README.md
Normal file → Executable file
265
README.md
Normal file → Executable file
@ -1,47 +1,76 @@
|
||||
Bench
|
||||
=====
|
||||
|
||||
The bench allows you to setup Frappe / ERPNext apps on your local Linux (CentOS 6, Debian 7 or Ubuntu) machine or a production server. You can use the bench to serve multiple frappe sites.
|
||||
The bench allows you to setup Frappe / ERPNext apps on your local Linux (CentOS 6, Debian 7, Ubuntu, etc) machine or a production server. You can use the bench to serve multiple frappe sites. If you are using a DigitalOcean droplet or any other VPS / Dedicated Server, make sure it has >= 1Gb of ram or has swap setup properly.
|
||||
|
||||
To do this install, you must have basic information on how Linux works and should be able to use the command-line. If you are looking easier ways to get started and evaluate ERPNext, [download the Virtual Machine or take a free trial on ERPNext.com](https://erpnext.com/pricing).
|
||||
To do this install, you must have basic information on how Linux works and should be able to use the command-line. If you are looking easier ways to get started and evaluate ERPNext, [download the Virtual Machine](https://erpnext.com/download) or take [a free trial on erpnext.com](https://erpnext.com/pricing).
|
||||
|
||||
If you have questions, please ask them on our [forum](https://discuss.erpnext.com/).
|
||||
|
||||
Installation
|
||||
============
|
||||
|
||||
Easy Production Setup
|
||||
Production vs Development
|
||||
--------------------
|
||||
|
||||
| Production | Development |
|
||||
|--------------------------------------------------------------------------|-------------------------------------------------------------------|
|
||||
| The Production setup uses Nginx and Supervisor | The development setup uses Socketio. |
|
||||
| This setup isn't meant for instant updates in code. | Any code changes will be reflected instantly. |
|
||||
| Background services handle all the work, and they start with the system. | You need to explicitly start your server by running `bench start` |
|
||||
| Uses Celery for job queuing | Uses RQ for queuing |
|
||||
| Installs with master branch | Installs with develop branch |
|
||||
|
||||
|
||||
Easy Setup
|
||||
---------------------
|
||||
> For production use and which also installs ERPNext. Supported for CentOS 6, CentOS 7, Debian 7 and Ubuntu 12.04+
|
||||
> This is an opinionated setup with logging and SE Linux. So, it is best to setup on a blank server.
|
||||
- This is an opinionated setup with logging and SE Linux. So, it is best to setup on a blank server.
|
||||
- Supported for CentOS 6, CentOS 7, Debian 7 and Ubuntu 12.04+
|
||||
- This script will install the pre-requisites, install bench and setup an ERPNext site
|
||||
- Passwords for Frappe, Frappe Administrator and MariaDB (root) will be generated
|
||||
- You can then login as **Administrator** with the Administrator password printed
|
||||
|
||||
Open your Terminal and enter:
|
||||
|
||||
|
||||
####For Production:
|
||||
```
|
||||
wget https://raw.githubusercontent.com/frappe/bench/master/install_scripts/setup_frappe.sh
|
||||
sudo bash setup_frappe.sh --setup-production
|
||||
```
|
||||
This will install Frappe and ERPNext with Supervisor, Nginx and Celery. Supervisor will keep all services working in the background and make sure they all run.
|
||||
|
||||
- This script will install the pre-requisites, install bench and setup an ERPNext site
|
||||
- This will setup ERPNext with nginx, with Supervisor enabled and checkout the master branch of the ERPNext repo
|
||||
- Passwords for Frappe, Frappe Administrator and MariaDB (root) will be generated
|
||||
- You can then login as **Administrator** with the Administrator password printed
|
||||
####For Development:
|
||||
> We recommend using the [Beta Development Setup](#beta-development-setup) if it supports your OS
|
||||
|
||||
If you want to develop ERPNext or any Frappe App, you can omit the "--setup-production" part from the command. This will setup ERPNext as well. Use ```bench start``` to run the server.
|
||||
```
|
||||
wget https://raw.githubusercontent.com/frappe/bench/master/install_scripts/setup_frappe.sh
|
||||
sudo bash setup_frappe.sh --bench-branch develop
|
||||
```
|
||||
This will install with Socketio and Redis Queue. You have to explicitly start services by running `bench start`.
|
||||
|
||||
> Note: If you are using a DigitalOcean droplet or any other cloud provider's vps, make sure it has >= 1gb of ram or has swap setup properly.
|
||||
####Script Options:
|
||||
```
|
||||
-h | --help
|
||||
-v | --verbose
|
||||
--mysql-root-password
|
||||
--frappe-user
|
||||
--setup-production
|
||||
--skip-setup-bench
|
||||
--skip-install-bench
|
||||
```
|
||||
|
||||
Development Setup (Beta)
|
||||
|
||||
Beta Development Setup
|
||||
------------------------
|
||||
|
||||
Tested on Ubuntu 14.04+ and MacOS X. If you find any problems, post them on our forum: [https://discuss.erpnext.com](https://discuss.erpnext.com)
|
||||
Tested on Ubuntu 14.04+, Debian 7+, CentOS 7+, and MacOS X. If you find any problems, post them on our forum: [https://discuss.erpnext.com](https://discuss.erpnext.com)
|
||||
|
||||
```
|
||||
wget https://raw.githubusercontent.com/frappe/bench/develop/playbooks/install.py
|
||||
python install.py --develop
|
||||
```
|
||||
|
||||
This script requires Python2.7+ installed on your machine. You need to run this with a user that is **not** `root`, but can `sudo`. If you don't have such a user, you can search the web for *How to add a new user in { your OS }* and *How to add an existing user to sudoers in { your OS }*.
|
||||
This will install with Socketio and Redis Queue. You have to explicitly start services by running `bench start`. This script requires Python2.7+ installed on your machine. You need to run this with a user that is **not** `root`, but can `sudo`. If you don't have such a user, you can search the web for *How to add a new user in { your OS }* and *How to add an existing user to sudoers in { your OS }*.
|
||||
|
||||
This script will:
|
||||
|
||||
@ -57,97 +86,6 @@ This script will:
|
||||
|
||||
You will have to manually create a new site (`bench new-site`) and get apps that you need (`bench get-app`, `bench install-app`).
|
||||
|
||||
Manual Install
|
||||
--------------
|
||||
|
||||
Install pre-requisites,
|
||||
|
||||
* [Python 2.7](https://www.python.org/download/releases/2.7/)
|
||||
* [MariaDB](https://mariadb.org/)
|
||||
* [Redis](http://redis.io/topics/quickstart)
|
||||
* [WKHTMLtoPDF with patched QT](http://wkhtmltopdf.org/downloads.html) (required for pdf generation)
|
||||
|
||||
[Installing pre-requisites on OSX](https://github.com/frappe/bench/wiki/Installing-Bench-Pre-requisites-on-MacOSX)
|
||||
|
||||
Install bench as a *non root* user,
|
||||
|
||||
git clone https://github.com/frappe/bench bench-repo
|
||||
sudo pip install -e bench-repo
|
||||
|
||||
Note: Please do not remove the bench directory the above commands will create
|
||||
|
||||
Initialize Bench using: `bench init frappe-bench`. Here you can replace `frappe-bench` with a name of your choice for your bench.
|
||||
|
||||
|
||||
Installing ERPNext
|
||||
------------------
|
||||
|
||||
If you're here to setup ERPNext, continue with [ERPNext setup](https://github.com/frappe/bench#setting-up-erpnext)
|
||||
|
||||
|
||||
Migrating from existing installation
|
||||
------------------------------------
|
||||
|
||||
If want to migrate from ERPNext version 3, follow the instructions here, https://github.com/frappe/bench/wiki/Migrating-from-ERPNext-version-3
|
||||
|
||||
If want to migrate from the old bench, follow the instructions here, https://github.com/frappe/bench/wiki/Migrating-from-old-bench
|
||||
|
||||
|
||||
Basic Usage
|
||||
===========
|
||||
|
||||
* Create a new bench
|
||||
|
||||
The init command will create a bench directory with frappe framework
|
||||
installed. It will be setup for periodic backups and auto updates once
|
||||
a day.
|
||||
|
||||
bench init frappe-bench && cd frappe-bench
|
||||
|
||||
* Add apps
|
||||
|
||||
The get-app command gets and installs frappe apps. Examples include
|
||||
|
||||
- [erpnext](https://github.com/frappe/erpnext)
|
||||
- [erpnext_shopify](https://github.com/frappe/erpnext_shopify)
|
||||
- [paypal_integration](https://github.com/frappe/paypal_integration)
|
||||
|
||||
bench get-app erpnext https://github.com/frappe/erpnext
|
||||
|
||||
* Add site
|
||||
|
||||
Frappe apps are run by frappe sites and you will have to create at least one
|
||||
site. The new-site command allows you to do that.
|
||||
|
||||
bench new-site site1.local
|
||||
|
||||
* Start bench
|
||||
|
||||
To start using the bench, use the `bench start` command
|
||||
|
||||
bench start
|
||||
|
||||
To login to Frappe / ERPNext, open your browser and go to `localhost:8000`
|
||||
|
||||
The default user name is "Administrator" and password is what you set when you created the new site.
|
||||
|
||||
|
||||
Setting Up ERPNext
|
||||
==================
|
||||
|
||||
To setup a bench that runs ERPNext, run the following commands
|
||||
|
||||
```
|
||||
cd ~
|
||||
bench init frappe-bench
|
||||
cd frappe-bench
|
||||
bench get-app erpnext https://github.com/frappe/erpnext # Add ERPNext to your bench apps
|
||||
bench new-site site1.local # Create a new site
|
||||
bench install-app erpnext # Install ERPNext for the site
|
||||
```
|
||||
|
||||
You can now either use `bench start` or setup the bench for production use.
|
||||
|
||||
|
||||
Updating
|
||||
========
|
||||
@ -165,107 +103,22 @@ You can also run the parts of the bench selectively.
|
||||
|
||||
`bench update --bench` will only update the bench utility (this project)
|
||||
|
||||
Running the bench
|
||||
==================
|
||||
|
||||
To run the bench,
|
||||
|
||||
*For development*: `bench start`
|
||||
|
||||
*For production*: Configure supervisor and nginx
|
||||
|
||||
To run the bench, a few services need to be running apart from the processes.
|
||||
|
||||
External services
|
||||
-----------------
|
||||
|
||||
* MariaDB (Datastore for frappe)
|
||||
* Redis (Queue for frappe background workers and caching)
|
||||
* nginx (for production deployment)
|
||||
* supervisor (for production deployment)
|
||||
|
||||
Frappe Processes
|
||||
----------------
|
||||
|
||||
* WSGI Server
|
||||
|
||||
* The WSGI server is responsible for responding to the HTTP requests to
|
||||
frappe. In development scenario (`bench serve` or `bench start`), the
|
||||
Werkzeug WSGI server is used and in production, gunicorn (automatically
|
||||
configured in supervisor) is used.
|
||||
|
||||
* Celery Worker Processes
|
||||
|
||||
* The Celery worker processes execute background jobs in the Frappe system.
|
||||
These processes are automatically started when `bench start` is run and
|
||||
for production are configured in supervisor configuration.
|
||||
|
||||
* Celery Worker Beat Process
|
||||
|
||||
* The Celery worker beat process schedules enqeueing of scheduled jobs in the
|
||||
Frappe system. This process is automatically started when `bench start` is
|
||||
run and for production are configured in supervisor configuration.
|
||||
Guides
|
||||
=======
|
||||
- [Configuring HTTPS](https://frappe.github.io/frappe/user/en/bench/guides/configuring-https.html)
|
||||
- [Diagnosing the Scheduler](https://frappe.github.io/frappe/user/en/bench/guides/diagnosing-the-scheduler.html)
|
||||
- [Change Hostname](https://frappe.github.io/frappe/user/en/bench/guides/how-to-change-host-name-from-localhost.html)
|
||||
- [Manual Setup](https://frappe.github.io/frappe/user/en/bench/guides/manual-setup.html)
|
||||
- [Setup Production](https://frappe.github.io/frappe/user/en/bench/guides/setup-production.html)
|
||||
- [Setup Multitenancy](https://frappe.github.io/frappe/user/en/bench/guides/setup-multitenancy.html)
|
||||
- [Setup SSL](https://frappe.github.io/frappe/user/en/bench/guides/setup-ssl.html)
|
||||
- [Stopping Production](https://frappe.github.io/frappe/user/en/bench/guides/stop-production-and-start-development.html)
|
||||
|
||||
|
||||
Production Deployment
|
||||
=====================
|
||||
Resources
|
||||
=======
|
||||
|
||||
|
||||
You can setup the bench for production use by configuring two programs, Supervisor and nginx.
|
||||
|
||||
> These steps are automated if you pass `--setup-production` to the easy install script
|
||||
> or run `sudo bench setup production`
|
||||
|
||||
Supervisor
|
||||
----------
|
||||
|
||||
Supervisor makes sure that the process that power the Frappe system keep running
|
||||
and it restarts them if they happen to crash. You can generate the required
|
||||
configuration for supervisor using the command `bench setup supervisor`. The
|
||||
configuration will be available in `config/supervisor.conf` directory. You can
|
||||
then copy/link this file to the supervisor config directory and reload it for it to
|
||||
take effect.
|
||||
|
||||
eg,
|
||||
|
||||
```
|
||||
bench setup supervisor
|
||||
sudo ln -s `pwd`/config/supervisor.conf /etc/supervisor/conf.d/frappe-bench.conf
|
||||
```
|
||||
|
||||
Note: For CentOS 7, the extension should be `ini`, thus the command becomes
|
||||
```
|
||||
bench setup supervisor
|
||||
sudo ln -s `pwd`/config/supervisor.conf /etc/supervisor/conf.d/frappe-bench.ini #for CentOS 7 only
|
||||
```
|
||||
|
||||
The bench will also need to restart the processes managed by supervisor when you
|
||||
update the apps. To automate this, you will have to setup sudoers using the
|
||||
command, `sudo bench setup sudoers $(whoami)`.
|
||||
|
||||
Nginx
|
||||
-----
|
||||
|
||||
Nginx is a web server and we use it to serve static files and proxy rest of the
|
||||
requests to frappe. You can generate the required configuration for nginx using
|
||||
the command `bench setup nginx`. The configuration will be available in
|
||||
`config/nginx.conf` file. You can then copy/link this file to the nginx config
|
||||
directory and reload it for it to take effect.
|
||||
|
||||
eg,
|
||||
|
||||
```
|
||||
bench setup nginx
|
||||
sudo ln -s `pwd`/config/nginx.conf /etc/nginx/conf.d/frappe-bench.conf
|
||||
```
|
||||
|
||||
Note: When you restart nginx after the configuration change, it might fail if
|
||||
you have another configuration with server block as default for port 80 (in most
|
||||
cases for the nginx welcome page). You will have to disable this config. Most
|
||||
probable places for it to exist are `/etc/nginx/conf.d/default.conf` and
|
||||
`/etc/nginx/conf.d/default`.
|
||||
|
||||
Multitenant setup
|
||||
=================
|
||||
|
||||
Follow https://github.com/frappe/bench/wiki/Multitenant-Setup
|
||||
- [Background Services](https://frappe.github.io/frappe/user/en/bench/resources/background-services.html)
|
||||
- [Bench Commands Cheat Sheet](https://frappe.github.io/frappe/user/en/bench/resources/bench-commands-cheatsheet.html)
|
||||
- [Bench Procfile](https://frappe.github.io/frappe/user/en/bench/resources/bench-procfile.html)
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
import os
|
||||
from .utils import (exec_cmd, get_frappe, check_git_for_shallow_clone, build_assets,
|
||||
from .utils import (exec_cmd, get_frappe, check_git_for_shallow_clone, build_assets,
|
||||
restart_supervisor_processes, get_cmd_output, run_frappe_cmd)
|
||||
from .config.common_site_config import get_config
|
||||
|
||||
@ -46,6 +46,7 @@ def get_app(app, git_url, branch=None, bench='.', build_asset_files=True, verbos
|
||||
logger.info('getting app {}'.format(app))
|
||||
shallow_clone = '--depth 1' if check_git_for_shallow_clone() else ''
|
||||
branch = '--branch {branch}'.format(branch=branch) if branch else ''
|
||||
|
||||
exec_cmd("git clone {git_url} {branch} {shallow_clone} --origin upstream {app}".format(
|
||||
git_url=git_url,
|
||||
app=app,
|
||||
@ -110,7 +111,7 @@ def get_current_frappe_version(bench='.'):
|
||||
try:
|
||||
return get_major_version(get_current_version('frappe', bench=bench))
|
||||
except IOError:
|
||||
return ''
|
||||
return 0
|
||||
|
||||
def get_current_branch(app, bench='.'):
|
||||
repo_dir = get_repo_dir(app, bench=bench)
|
||||
|
||||
@ -1,14 +1,20 @@
|
||||
import bench, os, click
|
||||
from bench.utils import find_executable
|
||||
from bench.app import get_current_frappe_version, get_current_branch
|
||||
from bench.config.common_site_config import get_config
|
||||
|
||||
def setup_procfile(bench_path, force=False):
|
||||
config = get_config(bench=bench_path)
|
||||
procfile_path = os.path.join(bench_path, 'Procfile')
|
||||
if not force and os.path.exists(procfile_path):
|
||||
click.confirm('A Procfile already exists and this will overwrite it. Do you want to continue?',
|
||||
abort=True)
|
||||
|
||||
procfile = bench.env.get_template('Procfile').render(node=find_executable("node") \
|
||||
or find_executable("nodejs"))
|
||||
procfile = bench.env.get_template('Procfile').render(
|
||||
node=find_executable("node") or find_executable("nodejs"),
|
||||
frappe_version=get_current_frappe_version(bench_path),
|
||||
frappe_branch=get_current_branch('frappe', bench_path),
|
||||
webserver_port=config.get('webserver_port'))
|
||||
|
||||
with open(procfile_path, 'w') as f:
|
||||
f.write(procfile)
|
||||
|
||||
@ -16,9 +16,12 @@ def setup_production(user, bench='.'):
|
||||
supervisor_conf = os.path.join(get_supervisor_confdir(), '{bench_name}.{extn}'.format(
|
||||
bench_name=bench_name, extn=supervisor_conf_extn))
|
||||
|
||||
# Check if symlink exists, If not then create it.
|
||||
if not os.path.islink(supervisor_conf):
|
||||
os.symlink(os.path.abspath(os.path.join(bench, 'config', 'supervisor.conf')), supervisor_conf)
|
||||
|
||||
os.symlink(os.path.abspath(os.path.join(bench, 'config', 'supervisor.conf')), supervisor_conf)
|
||||
os.symlink(os.path.abspath(os.path.join(bench, 'config', 'nginx.conf')), nginx_conf)
|
||||
if not os.path.islink(nginx_conf):
|
||||
os.symlink(os.path.abspath(os.path.join(bench, 'config', 'nginx.conf')), nginx_conf)
|
||||
|
||||
exec_cmd('supervisorctl reload')
|
||||
if os.environ.get('NO_SERVICE_RESTART'):
|
||||
|
||||
@ -2,7 +2,7 @@ import os, getpass, click
|
||||
import bench
|
||||
|
||||
def generate_supervisor_config(bench_path, user=None, force=False):
|
||||
from bench.app import get_current_frappe_version
|
||||
from bench.app import get_current_frappe_version, get_current_branch
|
||||
from bench.utils import get_bench_name, find_executable
|
||||
from bench.config.common_site_config import get_config, update_config, get_gunicorn_workers
|
||||
|
||||
@ -18,13 +18,14 @@ def generate_supervisor_config(bench_path, user=None, force=False):
|
||||
"bench_dir": bench_dir,
|
||||
"sites_dir": os.path.join(bench_dir, 'sites'),
|
||||
"user": user,
|
||||
"frappe_version": get_current_frappe_version(bench_path),
|
||||
"frappe_branch": get_current_branch('frappe', bench_path),
|
||||
"http_timeout": config.get("http_timeout", 120),
|
||||
"redis_server": find_executable('redis-server'),
|
||||
"node": find_executable('node') or find_executable('nodejs'),
|
||||
"redis_cache_config": os.path.join(bench_dir, 'config', 'redis_cache.conf'),
|
||||
"redis_socketio_config": os.path.join(bench_dir, 'config', 'redis_socketio.conf'),
|
||||
"redis_queue_config": os.path.join(bench_dir, 'config', 'redis_queue.conf'),
|
||||
"frappe_version": get_current_frappe_version(),
|
||||
"webserver_port": config.get('webserver_port', 8000),
|
||||
"gunicorn_workers": config.get('gunicorn_workers', get_gunicorn_workers()["gunicorn_workers"]),
|
||||
"bench_name": get_bench_name(bench_path),
|
||||
|
||||
@ -1,10 +1,19 @@
|
||||
{%- set use_rq = (frappe_branch=='develop' or frappe_version >= 7) -%}
|
||||
redis_cache: redis-server config/redis_cache.conf
|
||||
redis_socketio: redis-server config/redis_socketio.conf
|
||||
redis_queue: redis-server config/redis_queue.conf
|
||||
web: bench serve
|
||||
web: bench serve {% if webserver_port -%} --port {{ webserver_port }} {%- endif %}
|
||||
|
||||
socketio: {{ node }} apps/frappe/socketio.js
|
||||
watch: bench watch
|
||||
{% if use_rq -%}
|
||||
schedule: bench schedule
|
||||
worker_short: bench worker --queue short
|
||||
worker_long: bench worker --queue long
|
||||
worker_default: bench worker --queue default
|
||||
{% else %}
|
||||
workerbeat: sh -c 'cd sites && exec ../env/bin/python -m frappe.celery_app beat -s scheduler.schedule'
|
||||
worker: sh -c 'cd sites && exec ../env/bin/python -m frappe.celery_app worker -n jobs@%h -Ofair --soft-time-limit 360 --time-limit 390'
|
||||
longjob_worker: sh -c 'cd sites && exec ../env/bin/python -m frappe.celery_app worker -n longjobs@%h -Ofair --soft-time-limit 1500 --time-limit 1530'
|
||||
async_worker: sh -c 'cd sites && exec ../env/bin/python -m frappe.celery_app worker -n async@%h -Ofair --soft-time-limit 1500 --time-limit 1530'
|
||||
{%- endif %}
|
||||
|
||||
@ -1,3 +1,4 @@
|
||||
{%- set use_rq = (frappe_branch=='develop' or frappe_version >= 7) -%}
|
||||
; Notes:
|
||||
; priority=1 --> Lower priorities indicate programs that start first and shut down last
|
||||
; killasgroup=true --> send kill signal to child processes too
|
||||
@ -12,6 +13,7 @@ stderr_logfile={{ bench_dir }}/logs/web.error.log
|
||||
user={{ user }}
|
||||
directory={{ sites_dir }}
|
||||
|
||||
{% if use_rq %}
|
||||
[program:{{ bench_name }}-frappe-schedule]
|
||||
command=bench schedule
|
||||
priority=3
|
||||
@ -64,6 +66,55 @@ killasgroup=true
|
||||
numprocs={{ background_workers }}
|
||||
process_name=%(program_name)s-%(process_num)d
|
||||
|
||||
{% else %}
|
||||
[program:{{ bench_name }}-frappe-workerbeat]
|
||||
command={{ bench_dir }}/env/bin/python -m frappe.celery_app beat -s beat.schedule
|
||||
priority=3
|
||||
autostart=true
|
||||
autorestart=true
|
||||
stdout_logfile={{ bench_dir }}/logs/workerbeat.log
|
||||
stderr_logfile={{ bench_dir }}/logs/workerbeat.error.log
|
||||
user={{ user }}
|
||||
directory={{ sites_dir }}
|
||||
|
||||
[program:{{ bench_name }}-frappe-worker]
|
||||
command={{ bench_dir }}/env/bin/python -m frappe.celery_app worker -n jobs@%%h -Ofair --soft-time-limit 360 --time-limit 390 --loglevel INFO
|
||||
priority=4
|
||||
autostart=true
|
||||
autorestart=true
|
||||
stdout_logfile={{ bench_dir }}/logs/worker.log
|
||||
stderr_logfile={{ bench_dir }}/logs/worker.error.log
|
||||
user={{ user }}
|
||||
stopwaitsecs=400
|
||||
directory={{ sites_dir }}
|
||||
killasgroup=true
|
||||
|
||||
[program:{{ bench_name }}-frappe-longjob-worker]
|
||||
command={{ bench_dir }}/env/bin/python -m frappe.celery_app worker -n longjobs@%%h -Ofair --soft-time-limit 1500 --time-limit 1530 --loglevel INFO
|
||||
priority=2
|
||||
autostart=true
|
||||
autorestart=true
|
||||
stdout_logfile={{ bench_dir }}/logs/worker.log
|
||||
stderr_logfile={{ bench_dir }}/logs/worker.error.log
|
||||
user={{ user }}
|
||||
stopwaitsecs=1540
|
||||
directory={{ sites_dir }}
|
||||
killasgroup=true
|
||||
|
||||
[program:{{ bench_name }}-frappe-async-worker]
|
||||
command={{ bench_dir }}/env/bin/python -m frappe.celery_app worker -n async@%%h -Ofair --soft-time-limit 1500 --time-limit 1530 --loglevel INFO
|
||||
priority=2
|
||||
autostart=true
|
||||
autorestart=true
|
||||
stdout_logfile={{ bench_dir }}/logs/worker.log
|
||||
stderr_logfile={{ bench_dir }}/logs/worker.error.log
|
||||
user={{ user }}
|
||||
stopwaitsecs=1540
|
||||
directory={{ sites_dir }}
|
||||
killasgroup=true
|
||||
|
||||
{% endif %}
|
||||
|
||||
[program:{{ bench_name }}-redis-cache]
|
||||
command={{ redis_server }} {{ redis_cache_config }}
|
||||
priority=1
|
||||
@ -112,8 +163,17 @@ directory={{ bench_dir }}
|
||||
[group:{{ bench_name }}-web]
|
||||
programs={{ bench_name }}-frappe-web {%- if node -%} ,{{ bench_name }}-node-socketio {%- endif%}
|
||||
|
||||
{% if use_rq %}
|
||||
|
||||
[group:{{ bench_name }}-workers]
|
||||
programs={{ bench_name }}-frappe-schedule,{{ bench_name }}-frappe-default-worker,{{ bench_name }}-frappe-short-worker,{{ bench_name }}-frappe-long-worker
|
||||
|
||||
{% else %}
|
||||
|
||||
[group:{{ bench_name }}-workers]
|
||||
programs={{ bench_name }}-frappe-workerbeat,{{ bench_name }}-frappe-worker,{{ bench_name }}-frappe-longjob-worker,{{ bench_name }}-frappe-async-worker
|
||||
|
||||
{% endif %}
|
||||
|
||||
[group:{{ bench_name }}-redis]
|
||||
programs={{ bench_name }}-redis-cache,{{ bench_name }}-redis-queue {%- if frappe_version > 5 -%} ,{{ bench_name }}-redis-socketio {%- endif %}
|
||||
|
||||
@ -18,8 +18,10 @@ def run(bench_path):
|
||||
if patch not in executed_patches:
|
||||
module = importlib.import_module(patch.split()[0])
|
||||
execute = getattr(module, 'execute')
|
||||
execute(bench_path)
|
||||
executed_patches.append(patch)
|
||||
result = execute(bench_path)
|
||||
|
||||
if result != False:
|
||||
executed_patches.append(patch)
|
||||
|
||||
finally:
|
||||
with open(target_patch_file, 'w') as f:
|
||||
|
||||
@ -1,8 +1,17 @@
|
||||
import click, os
|
||||
from bench.config.procfile import setup_procfile
|
||||
from bench.config.supervisor import generate_supervisor_config
|
||||
from bench.app import get_current_frappe_version, get_current_branch
|
||||
|
||||
def execute(bench_path):
|
||||
frappe_branch = get_current_branch('frappe', bench_path)
|
||||
frappe_version = get_current_frappe_version(bench_path)
|
||||
|
||||
if not (frappe_branch=='develop' or frappe_version >= 7):
|
||||
# not version 7+
|
||||
# prevent running this patch
|
||||
return False
|
||||
|
||||
click.confirm('\nThis update will remove Celery config and prepare the bench to use Python RQ.\n'
|
||||
'And it will overwrite Procfile and supervisor.conf.\n'
|
||||
'If you don\'t know what this means, type Y ;)\n\n'
|
||||
|
||||
@ -20,8 +20,8 @@ class TestBenchInit(unittest.TestCase):
|
||||
if os.path.exists(bench_path):
|
||||
shutil.rmtree(bench_path, ignore_errors=True)
|
||||
|
||||
def test_init(self, bench_name="test-bench"):
|
||||
self.init_bench(bench_name)
|
||||
def test_init(self, bench_name="test-bench", **kwargs):
|
||||
self.init_bench(bench_name, **kwargs)
|
||||
|
||||
self.assert_folders(bench_name)
|
||||
|
||||
@ -107,9 +107,9 @@ class TestBenchInit(unittest.TestCase):
|
||||
out = subprocess.check_output(["bench", "--site", site_name, "list-apps"], cwd=bench_path)
|
||||
self.assertTrue("erpnext" in out)
|
||||
|
||||
def init_bench(self, bench_name):
|
||||
def init_bench(self, bench_name, **kwargs):
|
||||
self.benches.append(bench_name)
|
||||
bench.utils.init(bench_name)
|
||||
bench.utils.init(bench_name, **kwargs)
|
||||
|
||||
def test_drop_site(self):
|
||||
self.init_bench('test-bench')
|
||||
|
||||
@ -28,6 +28,21 @@ class TestSetupProduction(test_init.TestBenchInit):
|
||||
|
||||
self.assert_nginx_process()
|
||||
|
||||
def test_setup_production_v6(self):
|
||||
bench_name = 'test-bench-v6'
|
||||
self.test_init(bench_name, frappe_branch='master')
|
||||
|
||||
user = getpass.getuser()
|
||||
|
||||
bench_path = os.path.join(os.path.abspath(self.benches_path), bench_name)
|
||||
setup_production(user, bench_path)
|
||||
|
||||
self.assert_nginx_config(bench_name)
|
||||
self.assert_nginx_process()
|
||||
|
||||
self.assert_supervisor_config(bench_name, use_rq=False)
|
||||
self.assert_supervisor_process(bench_name, use_rq=False)
|
||||
|
||||
def assert_nginx_config(self, bench_name):
|
||||
conf_src = os.path.join(os.path.abspath(self.benches_path), bench_name, 'config', 'nginx.conf')
|
||||
conf_dest = "/etc/nginx/conf.d/{bench_name}.conf".format(bench_name=bench_name)
|
||||
@ -48,7 +63,11 @@ class TestSetupProduction(test_init.TestBenchInit):
|
||||
):
|
||||
self.assertTrue(key.format(bench_name=bench_name) in f)
|
||||
|
||||
def assert_supervisor_config(self, bench_name):
|
||||
def assert_nginx_process(self):
|
||||
out = bench.utils.get_cmd_output("sudo nginx -t 2>&1")
|
||||
self.assertTrue("nginx: configuration file /etc/nginx/nginx.conf test is successful" in out)
|
||||
|
||||
def assert_supervisor_config(self, bench_name, use_rq=True):
|
||||
conf_src = os.path.join(os.path.abspath(self.benches_path), bench_name, 'config', 'supervisor.conf')
|
||||
|
||||
supervisor_conf_dir = get_supervisor_confdir()
|
||||
@ -64,44 +83,69 @@ class TestSetupProduction(test_init.TestBenchInit):
|
||||
with open(conf_src, "r") as f:
|
||||
f = f.read().decode("utf-8")
|
||||
|
||||
for key in (
|
||||
"program:{bench_name}-frappe-web",
|
||||
tests = [
|
||||
"program:{bench_name}-frappe-web",
|
||||
"program:{bench_name}-redis-cache",
|
||||
"program:{bench_name}-redis-queue",
|
||||
"program:{bench_name}-redis-socketio",
|
||||
"program:{bench_name}-node-socketio",
|
||||
"group:{bench_name}-web",
|
||||
"group:{bench_name}-workers",
|
||||
"group:{bench_name}-redis"
|
||||
]
|
||||
|
||||
if use_rq:
|
||||
tests.extend([
|
||||
"program:{bench_name}-frappe-schedule",
|
||||
"program:{bench_name}-frappe-default-worker",
|
||||
"program:{bench_name}-frappe-short-worker",
|
||||
"program:{bench_name}-frappe-long-worker",
|
||||
"program:{bench_name}-frappe-schedule",
|
||||
"program:{bench_name}-redis-cache",
|
||||
"program:{bench_name}-redis-queue",
|
||||
"program:{bench_name}-redis-socketio",
|
||||
"program:{bench_name}-node-socketio",
|
||||
"group:{bench_name}-web",
|
||||
"group:{bench_name}-workers",
|
||||
"group:{bench_name}-redis"
|
||||
):
|
||||
"program:{bench_name}-frappe-long-worker"
|
||||
])
|
||||
|
||||
else:
|
||||
tests.extend([
|
||||
"program:{bench_name}-frappe-workerbeat",
|
||||
"program:{bench_name}-frappe-worker",
|
||||
"program:{bench_name}-frappe-longjob-worker",
|
||||
"program:{bench_name}-frappe-async-worker"
|
||||
])
|
||||
|
||||
for key in tests:
|
||||
self.assertTrue(key.format(bench_name=bench_name) in f)
|
||||
|
||||
def assert_supervisor_process(self, bench_name):
|
||||
def assert_supervisor_process(self, bench_name, use_rq=True):
|
||||
out = bench.utils.get_cmd_output("sudo supervisorctl status")
|
||||
|
||||
if "STARTING" in out:
|
||||
time.sleep(10)
|
||||
out = bench.utils.get_cmd_output("sudo supervisorctl status")
|
||||
|
||||
for key in (
|
||||
"{bench_name}-web:{bench_name}-frappe-web[\s]+RUNNING",
|
||||
"{bench_name}-web:{bench_name}-node-socketio[\s]+RUNNING",
|
||||
tests = [
|
||||
"{bench_name}-web:{bench_name}-frappe-web[\s]+RUNNING",
|
||||
"{bench_name}-web:{bench_name}-node-socketio[\s]+RUNNING",
|
||||
"{bench_name}-redis:{bench_name}-redis-cache[\s]+RUNNING",
|
||||
"{bench_name}-redis:{bench_name}-redis-queue[\s]+RUNNING",
|
||||
"{bench_name}-redis:{bench_name}-redis-socketio[\s]+RUNNING"
|
||||
]
|
||||
|
||||
if use_rq:
|
||||
tests.extend([
|
||||
"{bench_name}-workers:{bench_name}-frappe-schedule[\s]+RUNNING",
|
||||
"{bench_name}-workers:{bench_name}-frappe-default-worker-0[\s]+RUNNING",
|
||||
"{bench_name}-workers:{bench_name}-frappe-short-worker-0[\s]+RUNNING",
|
||||
"{bench_name}-workers:{bench_name}-frappe-long-worker-0[\s]+RUNNING",
|
||||
"{bench_name}-workers:{bench_name}-frappe-schedule[\s]+RUNNING",
|
||||
"{bench_name}-redis:{bench_name}-redis-cache[\s]+RUNNING",
|
||||
"{bench_name}-redis:{bench_name}-redis-queue[\s]+RUNNING",
|
||||
"{bench_name}-redis:{bench_name}-redis-socketio[\s]+RUNNING",
|
||||
):
|
||||
"{bench_name}-workers:{bench_name}-frappe-long-worker-0[\s]+RUNNING"
|
||||
])
|
||||
|
||||
else:
|
||||
tests.extend([
|
||||
"{bench_name}-workers:{bench_name}-frappe-workerbeat[\s]+RUNNING",
|
||||
"{bench_name}-workers:{bench_name}-frappe-worker[\s]+RUNNING",
|
||||
"{bench_name}-workers:{bench_name}-frappe-longjob-worker[\s]+RUNNING",
|
||||
"{bench_name}-workers:{bench_name}-frappe-async-worker[\s]+RUNNING"
|
||||
])
|
||||
|
||||
for key in tests:
|
||||
self.assertTrue(re.search(key.format(bench_name=bench_name), out))
|
||||
|
||||
def assert_nginx_process(self):
|
||||
out = bench.utils.get_cmd_output("sudo nginx -t 2>&1")
|
||||
self.assertTrue("nginx: configuration file /etc/nginx/nginx.conf test is successful" in out)
|
||||
|
||||
|
||||
|
||||
@ -271,18 +271,26 @@ def restart_supervisor_processes(bench='.'):
|
||||
from .config.common_site_config import get_config
|
||||
conf = get_config(bench=bench)
|
||||
bench_name = get_bench_name(bench)
|
||||
cmd = conf.get('supervisor_restart_cmd',
|
||||
'sudo supervisorctl restart {bench_name}-web: {bench_name}-workers:'.format(bench_name=bench_name))
|
||||
|
||||
try:
|
||||
cmd = conf.get('supervisor_restart_cmd')
|
||||
if cmd:
|
||||
exec_cmd(cmd, cwd=bench)
|
||||
|
||||
except CommandFailedError:
|
||||
if '{bench_name}-workers:'.format(bench_name=bench_name) in cmd:
|
||||
# backward compatibility
|
||||
exec_cmd('sudo supervisorctl restart frappe:', cwd=bench)
|
||||
else:
|
||||
supervisor_status = subprocess.check_output(['sudo', 'supervisorctl', 'status'], cwd=bench)
|
||||
|
||||
if '{bench_name}-workers:'.format(bench_name=bench_name) in supervisor_status:
|
||||
group = '{bench_name}-web: {bench_name}-workers:'.format(bench_name=bench_name)
|
||||
|
||||
# backward compatibility
|
||||
elif '{bench_name}-processes:'.format(bench_name=bench_name) in supervisor_status:
|
||||
group = '{bench_name}-processes:'.format(bench_name=bench_name)
|
||||
|
||||
# backward compatibility
|
||||
else:
|
||||
raise
|
||||
group = 'frappe:'
|
||||
|
||||
exec_cmd('sudo supervisorctl restart {group}'.format(group=group), cwd=bench)
|
||||
|
||||
def get_site_config(site, bench='.'):
|
||||
config_path = os.path.join(bench, 'sites', site, 'site_config.json')
|
||||
|
||||
Loading…
Reference in New Issue
Block a user