accounting/bench
2
0
Fork 0
mirror of https://github.com/frappe/bench.git synced 2025-02-02 10:58:24 +00:00
Code Issues Releases Activity

Merge pull request #208 from frappe/fix/6-7-compatibility

Browse Source 
v6–v7 compatibility
This commit is contained in:
Anand Doshi 2016-04-27 22:50:27 +05:30
commit eb8fe5635f
13 changed files with 252 additions and 256 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.travis.yml
View File

@ -17,5 +17,5 @@ script:
- cd ~ - cd ~
- sudo pip install --upgrade pip - sudo pip install --upgrade pip
- sudo pip install -e bench-repo - sudo pip install -e bench-repo
# - sudo python -m unittest bench.tests.test_setup_production.TestSetupProduction.test_new_site # - sudo python -m unittest bench.tests.test_setup_production.TestSetupProduction.test_setup_production_v6
- sudo python -m unittest -v bench.tests.test_setup_production - sudo python -m unittest -v bench.tests.test_setup_production

265
README.md Normal file → Executable file
View File

@ -1,47 +1,76 @@
Bench Bench
===== =====
The bench allows you to setup Frappe / ERPNext apps on your local Linux (CentOS 6, Debian 7 or Ubuntu) machine or a production server. You can use the bench to serve multiple frappe sites. The bench allows you to setup Frappe / ERPNext apps on your local Linux (CentOS 6, Debian 7, Ubuntu, etc) machine or a production server. You can use the bench to serve multiple frappe sites. If you are using a DigitalOcean droplet or any other VPS / Dedicated Server, make sure it has >= 1Gb of ram or has swap setup properly.
To do this install, you must have basic information on how Linux works and should be able to use the command-line. If you are looking easier ways to get started and evaluate ERPNext, [download the Virtual Machine or take a free trial on ERPNext.com](https://erpnext.com/pricing). To do this install, you must have basic information on how Linux works and should be able to use the command-line. If you are looking easier ways to get started and evaluate ERPNext, [download the Virtual Machine](https://erpnext.com/download) or take [a free trial on erpnext.com](https://erpnext.com/pricing).
If you have questions, please ask them on our [forum](https://discuss.erpnext.com/). If you have questions, please ask them on our [forum](https://discuss.erpnext.com/).
Installation Installation
============ ============
Easy Production Setup Production vs Development
--------------------
| Production | Development |
|--------------------------------------------------------------------------|-------------------------------------------------------------------|
| The Production setup uses Nginx and Supervisor | The development setup uses Socketio. |
| This setup isn't meant for instant updates in code. | Any code changes will be reflected instantly. |
| Background services handle all the work, and they start with the system. | You need to explicitly start your server by running `bench start` |
| Uses Celery for job queuing | Uses RQ for queuing |
| Installs with master branch | Installs with develop branch |
Easy Setup
--------------------- ---------------------
> For production use and which also installs ERPNext. Supported for CentOS 6, CentOS 7, Debian 7 and Ubuntu 12.04+ - This is an opinionated setup with logging and SE Linux. So, it is best to setup on a blank server.
> This is an opinionated setup with logging and SE Linux. So, it is best to setup on a blank server. - Supported for CentOS 6, CentOS 7, Debian 7 and Ubuntu 12.04+
- This script will install the pre-requisites, install bench and setup an ERPNext site
- Passwords for Frappe, Frappe Administrator and MariaDB (root) will be generated
- You can then login as **Administrator** with the Administrator password printed
Open your Terminal and enter: Open your Terminal and enter:
####For Production:
``` ```
wget https://raw.githubusercontent.com/frappe/bench/master/install_scripts/setup_frappe.sh wget https://raw.githubusercontent.com/frappe/bench/master/install_scripts/setup_frappe.sh
sudo bash setup_frappe.sh --setup-production sudo bash setup_frappe.sh --setup-production
``` ```
This will install Frappe and ERPNext with Supervisor, Nginx and Celery. Supervisor will keep all services working in the background and make sure they all run.
- This script will install the pre-requisites, install bench and setup an ERPNext site ####For Development:
- This will setup ERPNext with nginx, with Supervisor enabled and checkout the master branch of the ERPNext repo > We recommend using the [Beta Development Setup](#beta-development-setup) if it supports your OS
- Passwords for Frappe, Frappe Administrator and MariaDB (root) will be generated
- You can then login as **Administrator** with the Administrator password printed
If you want to develop ERPNext or any Frappe App, you can omit the "--setup-production" part from the command. This will setup ERPNext as well. Use ```bench start``` to run the server. ```
wget https://raw.githubusercontent.com/frappe/bench/master/install_scripts/setup_frappe.sh
sudo bash setup_frappe.sh --bench-branch develop
```
This will install with Socketio and Redis Queue. You have to explicitly start services by running `bench start`.
> Note: If you are using a DigitalOcean droplet or any other cloud provider's vps, make sure it has >= 1gb of ram or has swap setup properly. ####Script Options:
```
-h | --help
-v | --verbose
--mysql-root-password
--frappe-user
--setup-production
--skip-setup-bench
--skip-install-bench
```
Development Setup (Beta)
Beta Development Setup
------------------------ ------------------------
Tested on Ubuntu 14.04+ and MacOS X. If you find any problems, post them on our forum: [https://discuss.erpnext.com](https://discuss.erpnext.com) Tested on Ubuntu 14.04+, Debian 7+, CentOS 7+, and MacOS X. If you find any problems, post them on our forum: [https://discuss.erpnext.com](https://discuss.erpnext.com)
``` ```
wget https://raw.githubusercontent.com/frappe/bench/develop/playbooks/install.py wget https://raw.githubusercontent.com/frappe/bench/develop/playbooks/install.py
python install.py --develop python install.py --develop
``` ```
This will install with Socketio and Redis Queue. You have to explicitly start services by running `bench start`. This script requires Python2.7+ installed on your machine. You need to run this with a user that is **not** `root`, but can `sudo`. If you don't have such a user, you can search the web for *How to add a new user in { your OS }* and *How to add an existing user to sudoers in { your OS }*.
This script requires Python2.7+ installed on your machine. You need to run this with a user that is **not** `root`, but can `sudo`. If you don't have such a user, you can search the web for *How to add a new user in { your OS }* and *How to add an existing user to sudoers in { your OS }*.
This script will: This script will:
@ -57,97 +86,6 @@ This script will:
You will have to manually create a new site (`bench new-site`) and get apps that you need (`bench get-app`, `bench install-app`). You will have to manually create a new site (`bench new-site`) and get apps that you need (`bench get-app`, `bench install-app`).
Manual Install
--------------
Install pre-requisites,
* [Python 2.7](https://www.python.org/download/releases/2.7/)
* [MariaDB](https://mariadb.org/)
* [Redis](http://redis.io/topics/quickstart)
* [WKHTMLtoPDF with patched QT](http://wkhtmltopdf.org/downloads.html) (required for pdf generation)
[Installing pre-requisites on OSX](https://github.com/frappe/bench/wiki/Installing-Bench-Pre-requisites-on-MacOSX)
Install bench as a *non root* user,
git clone https://github.com/frappe/bench bench-repo
sudo pip install -e bench-repo
Note: Please do not remove the bench directory the above commands will create
Initialize Bench using: `bench init frappe-bench`. Here you can replace `frappe-bench` with a name of your choice for your bench.
Installing ERPNext
------------------
If you're here to setup ERPNext, continue with [ERPNext setup](https://github.com/frappe/bench#setting-up-erpnext)
Migrating from existing installation
------------------------------------
If want to migrate from ERPNext version 3, follow the instructions here, https://github.com/frappe/bench/wiki/Migrating-from-ERPNext-version-3
If want to migrate from the old bench, follow the instructions here, https://github.com/frappe/bench/wiki/Migrating-from-old-bench
Basic Usage
===========
* Create a new bench
The init command will create a bench directory with frappe framework
installed. It will be setup for periodic backups and auto updates once
a day.
bench init frappe-bench && cd frappe-bench
* Add apps
The get-app command gets and installs frappe apps. Examples include
- [erpnext](https://github.com/frappe/erpnext)
- [erpnext_shopify](https://github.com/frappe/erpnext_shopify)
- [paypal_integration](https://github.com/frappe/paypal_integration)
bench get-app erpnext https://github.com/frappe/erpnext
* Add site
Frappe apps are run by frappe sites and you will have to create at least one
site. The new-site command allows you to do that.
bench new-site site1.local
* Start bench
To start using the bench, use the `bench start` command
bench start
To login to Frappe / ERPNext, open your browser and go to `localhost:8000`
The default user name is "Administrator" and password is what you set when you created the new site.
Setting Up ERPNext
==================
To setup a bench that runs ERPNext, run the following commands
```
cd ~
bench init frappe-bench
cd frappe-bench
bench get-app erpnext https://github.com/frappe/erpnext # Add ERPNext to your bench apps
bench new-site site1.local # Create a new site
bench install-app erpnext # Install ERPNext for the site
```
You can now either use `bench start` or setup the bench for production use.
Updating Updating
======== ========
@ -165,107 +103,22 @@ You can also run the parts of the bench selectively.
`bench update --bench` will only update the bench utility (this project) `bench update --bench` will only update the bench utility (this project)
Running the bench
==================
To run the bench, Guides
=======
*For development*: `bench start` - [Configuring HTTPS](https://frappe.github.io/frappe/user/en/bench/guides/configuring-https.html)
- [Diagnosing the Scheduler](https://frappe.github.io/frappe/user/en/bench/guides/diagnosing-the-scheduler.html)
*For production*: Configure supervisor and nginx - [Change Hostname](https://frappe.github.io/frappe/user/en/bench/guides/how-to-change-host-name-from-localhost.html)
- [Manual Setup](https://frappe.github.io/frappe/user/en/bench/guides/manual-setup.html)
To run the bench, a few services need to be running apart from the processes. - [Setup Production](https://frappe.github.io/frappe/user/en/bench/guides/setup-production.html)
- [Setup Multitenancy](https://frappe.github.io/frappe/user/en/bench/guides/setup-multitenancy.html)
External services - [Setup SSL](https://frappe.github.io/frappe/user/en/bench/guides/setup-ssl.html)
----------------- - [Stopping Production](https://frappe.github.io/frappe/user/en/bench/guides/stop-production-and-start-development.html)
* MariaDB (Datastore for frappe)
* Redis (Queue for frappe background workers and caching)
* nginx (for production deployment)
* supervisor (for production deployment)
Frappe Processes
----------------
* WSGI Server
* The WSGI server is responsible for responding to the HTTP requests to
frappe. In development scenario (`bench serve` or `bench start`), the
Werkzeug WSGI server is used and in production, gunicorn (automatically
configured in supervisor) is used.
* Celery Worker Processes
* The Celery worker processes execute background jobs in the Frappe system.
These processes are automatically started when `bench start` is run and
for production are configured in supervisor configuration.
* Celery Worker Beat Process
* The Celery worker beat process schedules enqeueing of scheduled jobs in the
Frappe system. This process is automatically started when `bench start` is
run and for production are configured in supervisor configuration.
Production Deployment Resources
===================== =======
- [Background Services](https://frappe.github.io/frappe/user/en/bench/resources/background-services.html)
You can setup the bench for production use by configuring two programs, Supervisor and nginx. - [Bench Commands Cheat Sheet](https://frappe.github.io/frappe/user/en/bench/resources/bench-commands-cheatsheet.html)
- [Bench Procfile](https://frappe.github.io/frappe/user/en/bench/resources/bench-procfile.html)
> These steps are automated if you pass `--setup-production` to the easy install script
> or run `sudo bench setup production`
Supervisor
----------
Supervisor makes sure that the process that power the Frappe system keep running
and it restarts them if they happen to crash. You can generate the required
configuration for supervisor using the command `bench setup supervisor`. The
configuration will be available in `config/supervisor.conf` directory. You can
then copy/link this file to the supervisor config directory and reload it for it to
take effect.
eg,
```
bench setup supervisor
sudo ln -s `pwd`/config/supervisor.conf /etc/supervisor/conf.d/frappe-bench.conf
```
Note: For CentOS 7, the extension should be `ini`, thus the command becomes
```
bench setup supervisor
sudo ln -s `pwd`/config/supervisor.conf /etc/supervisor/conf.d/frappe-bench.ini #for CentOS 7 only
```
The bench will also need to restart the processes managed by supervisor when you
update the apps. To automate this, you will have to setup sudoers using the
command, `sudo bench setup sudoers $(whoami)`.
Nginx
-----
Nginx is a web server and we use it to serve static files and proxy rest of the
requests to frappe. You can generate the required configuration for nginx using
the command `bench setup nginx`. The configuration will be available in
`config/nginx.conf` file. You can then copy/link this file to the nginx config
directory and reload it for it to take effect.
eg,
```
bench setup nginx
sudo ln -s `pwd`/config/nginx.conf /etc/nginx/conf.d/frappe-bench.conf
```
Note: When you restart nginx after the configuration change, it might fail if
you have another configuration with server block as default for port 80 (in most
cases for the nginx welcome page). You will have to disable this config. Most
probable places for it to exist are `/etc/nginx/conf.d/default.conf` and
`/etc/nginx/conf.d/default`.
Multitenant setup
=================
Follow https://github.com/frappe/bench/wiki/Multitenant-Setup

5
bench/app.py
View File

@ -1,5 +1,5 @@
import os import os
from .utils import (exec_cmd, get_frappe, check_git_for_shallow_clone, build_assets, from .utils import (exec_cmd, get_frappe, check_git_for_shallow_clone, build_assets,
restart_supervisor_processes, get_cmd_output, run_frappe_cmd) restart_supervisor_processes, get_cmd_output, run_frappe_cmd)
from .config.common_site_config import get_config from .config.common_site_config import get_config
@ -46,6 +46,7 @@ def get_app(app, git_url, branch=None, bench='.', build_asset_files=True, verbos
logger.info('getting app {}'.format(app)) logger.info('getting app {}'.format(app))
shallow_clone = '--depth 1' if check_git_for_shallow_clone() else '' shallow_clone = '--depth 1' if check_git_for_shallow_clone() else ''
branch = '--branch {branch}'.format(branch=branch) if branch else '' branch = '--branch {branch}'.format(branch=branch) if branch else ''
exec_cmd("git clone {git_url} {branch} {shallow_clone} --origin upstream {app}".format( exec_cmd("git clone {git_url} {branch} {shallow_clone} --origin upstream {app}".format(
git_url=git_url, git_url=git_url,
app=app, app=app,
@ -110,7 +111,7 @@ def get_current_frappe_version(bench='.'):
try: try:
return get_major_version(get_current_version('frappe', bench=bench)) return get_major_version(get_current_version('frappe', bench=bench))
except IOError: except IOError:
return '' return 0
def get_current_branch(app, bench='.'): def get_current_branch(app, bench='.'):
repo_dir = get_repo_dir(app, bench=bench) repo_dir = get_repo_dir(app, bench=bench)

10
bench/config/procfile.py
View File

@ -1,14 +1,20 @@
import bench, os, click import bench, os, click
from bench.utils import find_executable from bench.utils import find_executable
from bench.app import get_current_frappe_version, get_current_branch
from bench.config.common_site_config import get_config
def setup_procfile(bench_path, force=False): def setup_procfile(bench_path, force=False):
config = get_config(bench=bench_path)
procfile_path = os.path.join(bench_path, 'Procfile') procfile_path = os.path.join(bench_path, 'Procfile')
if not force and os.path.exists(procfile_path): if not force and os.path.exists(procfile_path):
click.confirm('A Procfile already exists and this will overwrite it. Do you want to continue?', click.confirm('A Procfile already exists and this will overwrite it. Do you want to continue?',
abort=True) abort=True)
procfile = bench.env.get_template('Procfile').render(node=find_executable("node") \ procfile = bench.env.get_template('Procfile').render(
or find_executable("nodejs")) node=find_executable("node") or find_executable("nodejs"),
frappe_version=get_current_frappe_version(bench_path),
frappe_branch=get_current_branch('frappe', bench_path),
webserver_port=config.get('webserver_port'))
with open(procfile_path, 'w') as f: with open(procfile_path, 'w') as f:
f.write(procfile) f.write(procfile)

7
bench/config/production_setup.py
View File

@ -16,9 +16,12 @@ def setup_production(user, bench='.'):
supervisor_conf = os.path.join(get_supervisor_confdir(), '{bench_name}.{extn}'.format( supervisor_conf = os.path.join(get_supervisor_confdir(), '{bench_name}.{extn}'.format(
bench_name=bench_name, extn=supervisor_conf_extn)) bench_name=bench_name, extn=supervisor_conf_extn))
# Check if symlink exists, If not then create it.
if not os.path.islink(supervisor_conf):
os.symlink(os.path.abspath(os.path.join(bench, 'config', 'supervisor.conf')), supervisor_conf)
os.symlink(os.path.abspath(os.path.join(bench, 'config', 'supervisor.conf')), supervisor_conf) if not os.path.islink(nginx_conf):
os.symlink(os.path.abspath(os.path.join(bench, 'config', 'nginx.conf')), nginx_conf) os.symlink(os.path.abspath(os.path.join(bench, 'config', 'nginx.conf')), nginx_conf)
exec_cmd('supervisorctl reload') exec_cmd('supervisorctl reload')
if os.environ.get('NO_SERVICE_RESTART'): if os.environ.get('NO_SERVICE_RESTART'):

5
bench/config/supervisor.py
View File

@ -2,7 +2,7 @@ import os, getpass, click
import bench import bench
def generate_supervisor_config(bench_path, user=None, force=False): def generate_supervisor_config(bench_path, user=None, force=False):
from bench.app import get_current_frappe_version from bench.app import get_current_frappe_version, get_current_branch
from bench.utils import get_bench_name, find_executable from bench.utils import get_bench_name, find_executable
from bench.config.common_site_config import get_config, update_config, get_gunicorn_workers from bench.config.common_site_config import get_config, update_config, get_gunicorn_workers
@ -18,13 +18,14 @@ def generate_supervisor_config(bench_path, user=None, force=False):
"bench_dir": bench_dir, "bench_dir": bench_dir,
"sites_dir": os.path.join(bench_dir, 'sites'), "sites_dir": os.path.join(bench_dir, 'sites'),
"user": user, "user": user,
"frappe_version": get_current_frappe_version(bench_path),
"frappe_branch": get_current_branch('frappe', bench_path),
"http_timeout": config.get("http_timeout", 120), "http_timeout": config.get("http_timeout", 120),
"redis_server": find_executable('redis-server'), "redis_server": find_executable('redis-server'),
"node": find_executable('node') or find_executable('nodejs'), "node": find_executable('node') or find_executable('nodejs'),
"redis_cache_config": os.path.join(bench_dir, 'config', 'redis_cache.conf'), "redis_cache_config": os.path.join(bench_dir, 'config', 'redis_cache.conf'),
"redis_socketio_config": os.path.join(bench_dir, 'config', 'redis_socketio.conf'), "redis_socketio_config": os.path.join(bench_dir, 'config', 'redis_socketio.conf'),
"redis_queue_config": os.path.join(bench_dir, 'config', 'redis_queue.conf'), "redis_queue_config": os.path.join(bench_dir, 'config', 'redis_queue.conf'),
"frappe_version": get_current_frappe_version(),
"webserver_port": config.get('webserver_port', 8000), "webserver_port": config.get('webserver_port', 8000),
"gunicorn_workers": config.get('gunicorn_workers', get_gunicorn_workers()["gunicorn_workers"]), "gunicorn_workers": config.get('gunicorn_workers', get_gunicorn_workers()["gunicorn_workers"]),
"bench_name": get_bench_name(bench_path), "bench_name": get_bench_name(bench_path),

11
bench/config/templates/Procfile
View File

@ -1,10 +1,19 @@
{%- set use_rq = (frappe_branch=='develop' or frappe_version >= 7) -%}
redis_cache: redis-server config/redis_cache.conf redis_cache: redis-server config/redis_cache.conf
redis_socketio: redis-server config/redis_socketio.conf redis_socketio: redis-server config/redis_socketio.conf
redis_queue: redis-server config/redis_queue.conf redis_queue: redis-server config/redis_queue.conf
web: bench serve web: bench serve {% if webserver_port -%} --port {{ webserver_port }} {%- endif %}
socketio: {{ node }} apps/frappe/socketio.js socketio: {{ node }} apps/frappe/socketio.js
watch: bench watch watch: bench watch
{% if use_rq -%}
schedule: bench schedule schedule: bench schedule
worker_short: bench worker --queue short worker_short: bench worker --queue short
worker_long: bench worker --queue long worker_long: bench worker --queue long
worker_default: bench worker --queue default worker_default: bench worker --queue default
{% else %}
workerbeat: sh -c 'cd sites && exec ../env/bin/python -m frappe.celery_app beat -s scheduler.schedule'
worker: sh -c 'cd sites && exec ../env/bin/python -m frappe.celery_app worker -n jobs@%h -Ofair --soft-time-limit 360 --time-limit 390'
longjob_worker: sh -c 'cd sites && exec ../env/bin/python -m frappe.celery_app worker -n longjobs@%h -Ofair --soft-time-limit 1500 --time-limit 1530'
async_worker: sh -c 'cd sites && exec ../env/bin/python -m frappe.celery_app worker -n async@%h -Ofair --soft-time-limit 1500 --time-limit 1530'
{%- endif %}

60
bench/config/templates/supervisor.conf
View File

@ -1,3 +1,4 @@
{%- set use_rq = (frappe_branch=='develop' or frappe_version >= 7) -%}
; Notes: ; Notes:
; priority=1 --> Lower priorities indicate programs that start first and shut down last ; priority=1 --> Lower priorities indicate programs that start first and shut down last
; killasgroup=true --> send kill signal to child processes too ; killasgroup=true --> send kill signal to child processes too
@ -12,6 +13,7 @@ stderr_logfile={{ bench_dir }}/logs/web.error.log
user={{ user }} user={{ user }}
directory={{ sites_dir }} directory={{ sites_dir }}
{% if use_rq %}
[program:{{ bench_name }}-frappe-schedule] [program:{{ bench_name }}-frappe-schedule]
command=bench schedule command=bench schedule
priority=3 priority=3
@ -64,6 +66,55 @@ killasgroup=true
numprocs={{ background_workers }} numprocs={{ background_workers }}
process_name=%(program_name)s-%(process_num)d process_name=%(program_name)s-%(process_num)d
{% else %}
[program:{{ bench_name }}-frappe-workerbeat]
command={{ bench_dir }}/env/bin/python -m frappe.celery_app beat -s beat.schedule
priority=3
autostart=true
autorestart=true
stdout_logfile={{ bench_dir }}/logs/workerbeat.log
stderr_logfile={{ bench_dir }}/logs/workerbeat.error.log
user={{ user }}
directory={{ sites_dir }}
[program:{{ bench_name }}-frappe-worker]
command={{ bench_dir }}/env/bin/python -m frappe.celery_app worker -n jobs@%%h -Ofair --soft-time-limit 360 --time-limit 390 --loglevel INFO
priority=4
autostart=true
autorestart=true
stdout_logfile={{ bench_dir }}/logs/worker.log
stderr_logfile={{ bench_dir }}/logs/worker.error.log
user={{ user }}
stopwaitsecs=400
directory={{ sites_dir }}
killasgroup=true
[program:{{ bench_name }}-frappe-longjob-worker]
command={{ bench_dir }}/env/bin/python -m frappe.celery_app worker -n longjobs@%%h -Ofair --soft-time-limit 1500 --time-limit 1530 --loglevel INFO
priority=2
autostart=true
autorestart=true
stdout_logfile={{ bench_dir }}/logs/worker.log
stderr_logfile={{ bench_dir }}/logs/worker.error.log
user={{ user }}
stopwaitsecs=1540
directory={{ sites_dir }}
killasgroup=true
[program:{{ bench_name }}-frappe-async-worker]
command={{ bench_dir }}/env/bin/python -m frappe.celery_app worker -n async@%%h -Ofair --soft-time-limit 1500 --time-limit 1530 --loglevel INFO
priority=2
autostart=true
autorestart=true
stdout_logfile={{ bench_dir }}/logs/worker.log
stderr_logfile={{ bench_dir }}/logs/worker.error.log
user={{ user }}
stopwaitsecs=1540
directory={{ sites_dir }}
killasgroup=true
{% endif %}
[program:{{ bench_name }}-redis-cache] [program:{{ bench_name }}-redis-cache]
command={{ redis_server }} {{ redis_cache_config }} command={{ redis_server }} {{ redis_cache_config }}
priority=1 priority=1
@ -112,8 +163,17 @@ directory={{ bench_dir }}
[group:{{ bench_name }}-web] [group:{{ bench_name }}-web]
programs={{ bench_name }}-frappe-web {%- if node -%} ,{{ bench_name }}-node-socketio {%- endif%} programs={{ bench_name }}-frappe-web {%- if node -%} ,{{ bench_name }}-node-socketio {%- endif%}
{% if use_rq %}
[group:{{ bench_name }}-workers] [group:{{ bench_name }}-workers]
programs={{ bench_name }}-frappe-schedule,{{ bench_name }}-frappe-default-worker,{{ bench_name }}-frappe-short-worker,{{ bench_name }}-frappe-long-worker programs={{ bench_name }}-frappe-schedule,{{ bench_name }}-frappe-default-worker,{{ bench_name }}-frappe-short-worker,{{ bench_name }}-frappe-long-worker
{% else %}
[group:{{ bench_name }}-workers]
programs={{ bench_name }}-frappe-workerbeat,{{ bench_name }}-frappe-worker,{{ bench_name }}-frappe-longjob-worker,{{ bench_name }}-frappe-async-worker
{% endif %}
[group:{{ bench_name }}-redis] [group:{{ bench_name }}-redis]
programs={{ bench_name }}-redis-cache,{{ bench_name }}-redis-queue {%- if frappe_version > 5 -%} ,{{ bench_name }}-redis-socketio {%- endif %} programs={{ bench_name }}-redis-cache,{{ bench_name }}-redis-queue {%- if frappe_version > 5 -%} ,{{ bench_name }}-redis-socketio {%- endif %}

6
bench/patches/__init__.py
View File

@ -18,8 +18,10 @@ def run(bench_path):
if patch not in executed_patches: if patch not in executed_patches:
module = importlib.import_module(patch.split()[0]) module = importlib.import_module(patch.split()[0])
execute = getattr(module, 'execute') execute = getattr(module, 'execute')
execute(bench_path) result = execute(bench_path)
executed_patches.append(patch)
if result != False:
executed_patches.append(patch)
finally: finally:
with open(target_patch_file, 'w') as f: with open(target_patch_file, 'w') as f:

9
bench/patches/v3/celery_to_rq.py
View File

@ -1,8 +1,17 @@
import click, os import click, os
from bench.config.procfile import setup_procfile from bench.config.procfile import setup_procfile
from bench.config.supervisor import generate_supervisor_config from bench.config.supervisor import generate_supervisor_config
from bench.app import get_current_frappe_version, get_current_branch
def execute(bench_path): def execute(bench_path):
frappe_branch = get_current_branch('frappe', bench_path)
frappe_version = get_current_frappe_version(bench_path)
if not (frappe_branch=='develop' or frappe_version >= 7):
# not version 7+
# prevent running this patch
return False
click.confirm('\nThis update will remove Celery config and prepare the bench to use Python RQ.\n' click.confirm('\nThis update will remove Celery config and prepare the bench to use Python RQ.\n'
'And it will overwrite Procfile and supervisor.conf.\n' 'And it will overwrite Procfile and supervisor.conf.\n'
'If you don\'t know what this means, type Y ;)\n\n' 'If you don\'t know what this means, type Y ;)\n\n'

8
bench/tests/test_init.py
View File

@ -20,8 +20,8 @@ class TestBenchInit(unittest.TestCase):
if os.path.exists(bench_path): if os.path.exists(bench_path):
shutil.rmtree(bench_path, ignore_errors=True) shutil.rmtree(bench_path, ignore_errors=True)
def test_init(self, bench_name="test-bench"): def test_init(self, bench_name="test-bench", **kwargs):
self.init_bench(bench_name) self.init_bench(bench_name, **kwargs)
self.assert_folders(bench_name) self.assert_folders(bench_name)
@ -107,9 +107,9 @@ class TestBenchInit(unittest.TestCase):
out = subprocess.check_output(["bench", "--site", site_name, "list-apps"], cwd=bench_path) out = subprocess.check_output(["bench", "--site", site_name, "list-apps"], cwd=bench_path)
self.assertTrue("erpnext" in out) self.assertTrue("erpnext" in out)
def init_bench(self, bench_name): def init_bench(self, bench_name, **kwargs):
self.benches.append(bench_name) self.benches.append(bench_name)
bench.utils.init(bench_name) bench.utils.init(bench_name, **kwargs)
def test_drop_site(self): def test_drop_site(self):
self.init_bench('test-bench') self.init_bench('test-bench')

96
bench/tests/test_setup_production.py
View File

@ -28,6 +28,21 @@ class TestSetupProduction(test_init.TestBenchInit):
self.assert_nginx_process() self.assert_nginx_process()
def test_setup_production_v6(self):
bench_name = 'test-bench-v6'
self.test_init(bench_name, frappe_branch='master')
user = getpass.getuser()
bench_path = os.path.join(os.path.abspath(self.benches_path), bench_name)
setup_production(user, bench_path)
self.assert_nginx_config(bench_name)
self.assert_nginx_process()
self.assert_supervisor_config(bench_name, use_rq=False)
self.assert_supervisor_process(bench_name, use_rq=False)
def assert_nginx_config(self, bench_name): def assert_nginx_config(self, bench_name):
conf_src = os.path.join(os.path.abspath(self.benches_path), bench_name, 'config', 'nginx.conf') conf_src = os.path.join(os.path.abspath(self.benches_path), bench_name, 'config', 'nginx.conf')
conf_dest = "/etc/nginx/conf.d/{bench_name}.conf".format(bench_name=bench_name) conf_dest = "/etc/nginx/conf.d/{bench_name}.conf".format(bench_name=bench_name)
@ -48,7 +63,11 @@ class TestSetupProduction(test_init.TestBenchInit):
): ):
self.assertTrue(key.format(bench_name=bench_name) in f) self.assertTrue(key.format(bench_name=bench_name) in f)
def assert_supervisor_config(self, bench_name): def assert_nginx_process(self):
out = bench.utils.get_cmd_output("sudo nginx -t 2>&1")
self.assertTrue("nginx: configuration file /etc/nginx/nginx.conf test is successful" in out)
def assert_supervisor_config(self, bench_name, use_rq=True):
conf_src = os.path.join(os.path.abspath(self.benches_path), bench_name, 'config', 'supervisor.conf') conf_src = os.path.join(os.path.abspath(self.benches_path), bench_name, 'config', 'supervisor.conf')
supervisor_conf_dir = get_supervisor_confdir() supervisor_conf_dir = get_supervisor_confdir()
@ -64,44 +83,69 @@ class TestSetupProduction(test_init.TestBenchInit):
with open(conf_src, "r") as f: with open(conf_src, "r") as f:
f = f.read().decode("utf-8") f = f.read().decode("utf-8")
for key in ( tests = [
"program:{bench_name}-frappe-web", "program:{bench_name}-frappe-web",
"program:{bench_name}-redis-cache",
"program:{bench_name}-redis-queue",
"program:{bench_name}-redis-socketio",
"program:{bench_name}-node-socketio",
"group:{bench_name}-web",
"group:{bench_name}-workers",
"group:{bench_name}-redis"
]
if use_rq:
tests.extend([
"program:{bench_name}-frappe-schedule",
"program:{bench_name}-frappe-default-worker", "program:{bench_name}-frappe-default-worker",
"program:{bench_name}-frappe-short-worker", "program:{bench_name}-frappe-short-worker",
"program:{bench_name}-frappe-long-worker", "program:{bench_name}-frappe-long-worker"
"program:{bench_name}-frappe-schedule", ])
"program:{bench_name}-redis-cache",
"program:{bench_name}-redis-queue", else:
"program:{bench_name}-redis-socketio", tests.extend([
"program:{bench_name}-node-socketio", "program:{bench_name}-frappe-workerbeat",
"group:{bench_name}-web", "program:{bench_name}-frappe-worker",
"group:{bench_name}-workers", "program:{bench_name}-frappe-longjob-worker",
"group:{bench_name}-redis" "program:{bench_name}-frappe-async-worker"
): ])
for key in tests:
self.assertTrue(key.format(bench_name=bench_name) in f) self.assertTrue(key.format(bench_name=bench_name) in f)
def assert_supervisor_process(self, bench_name): def assert_supervisor_process(self, bench_name, use_rq=True):
out = bench.utils.get_cmd_output("sudo supervisorctl status") out = bench.utils.get_cmd_output("sudo supervisorctl status")
if "STARTING" in out: if "STARTING" in out:
time.sleep(10) time.sleep(10)
out = bench.utils.get_cmd_output("sudo supervisorctl status") out = bench.utils.get_cmd_output("sudo supervisorctl status")
for key in ( tests = [
"{bench_name}-web:{bench_name}-frappe-web[\s]+RUNNING", "{bench_name}-web:{bench_name}-frappe-web[\s]+RUNNING",
"{bench_name}-web:{bench_name}-node-socketio[\s]+RUNNING", "{bench_name}-web:{bench_name}-node-socketio[\s]+RUNNING",
"{bench_name}-redis:{bench_name}-redis-cache[\s]+RUNNING",
"{bench_name}-redis:{bench_name}-redis-queue[\s]+RUNNING",
"{bench_name}-redis:{bench_name}-redis-socketio[\s]+RUNNING"
]
if use_rq:
tests.extend([
"{bench_name}-workers:{bench_name}-frappe-schedule[\s]+RUNNING",
"{bench_name}-workers:{bench_name}-frappe-default-worker-0[\s]+RUNNING", "{bench_name}-workers:{bench_name}-frappe-default-worker-0[\s]+RUNNING",
"{bench_name}-workers:{bench_name}-frappe-short-worker-0[\s]+RUNNING", "{bench_name}-workers:{bench_name}-frappe-short-worker-0[\s]+RUNNING",
"{bench_name}-workers:{bench_name}-frappe-long-worker-0[\s]+RUNNING", "{bench_name}-workers:{bench_name}-frappe-long-worker-0[\s]+RUNNING"
"{bench_name}-workers:{bench_name}-frappe-schedule[\s]+RUNNING", ])
"{bench_name}-redis:{bench_name}-redis-cache[\s]+RUNNING",
"{bench_name}-redis:{bench_name}-redis-queue[\s]+RUNNING", else:
"{bench_name}-redis:{bench_name}-redis-socketio[\s]+RUNNING", tests.extend([
): "{bench_name}-workers:{bench_name}-frappe-workerbeat[\s]+RUNNING",
"{bench_name}-workers:{bench_name}-frappe-worker[\s]+RUNNING",
"{bench_name}-workers:{bench_name}-frappe-longjob-worker[\s]+RUNNING",
"{bench_name}-workers:{bench_name}-frappe-async-worker[\s]+RUNNING"
])
for key in tests:
self.assertTrue(re.search(key.format(bench_name=bench_name), out)) self.assertTrue(re.search(key.format(bench_name=bench_name), out))
def assert_nginx_process(self):
out = bench.utils.get_cmd_output("sudo nginx -t 2>&1")
self.assertTrue("nginx: configuration file /etc/nginx/nginx.conf test is successful" in out)

24
bench/utils.py
View File

@ -271,18 +271,26 @@ def restart_supervisor_processes(bench='.'):
from .config.common_site_config import get_config from .config.common_site_config import get_config
conf = get_config(bench=bench) conf = get_config(bench=bench)
bench_name = get_bench_name(bench) bench_name = get_bench_name(bench)
cmd = conf.get('supervisor_restart_cmd',
'sudo supervisorctl restart {bench_name}-web: {bench_name}-workers:'.format(bench_name=bench_name))
try: cmd = conf.get('supervisor_restart_cmd')
if cmd:
exec_cmd(cmd, cwd=bench) exec_cmd(cmd, cwd=bench)
except CommandFailedError: else:
if '{bench_name}-workers:'.format(bench_name=bench_name) in cmd: supervisor_status = subprocess.check_output(['sudo', 'supervisorctl', 'status'], cwd=bench)
# backward compatibility
exec_cmd('sudo supervisorctl restart frappe:', cwd=bench) if '{bench_name}-workers:'.format(bench_name=bench_name) in supervisor_status:
group = '{bench_name}-web: {bench_name}-workers:'.format(bench_name=bench_name)
# backward compatibility
elif '{bench_name}-processes:'.format(bench_name=bench_name) in supervisor_status:
group = '{bench_name}-processes:'.format(bench_name=bench_name)
# backward compatibility
else: else:
raise group = 'frappe:'
exec_cmd('sudo supervisorctl restart {group}'.format(group=group), cwd=bench)
def get_site_config(site, bench='.'): def get_site_config(site, bench='.'):
config_path = os.path.join(bench, 'sites', site, 'site_config.json') config_path = os.path.join(bench, 'sites', site, 'site_config.json')