Merge branch 'develop' into declarative-setup-bench

.github/ISSUE_TEMPLATE/bug-report.md

@@ -6,8 +6,14 @@ labels: bug
 
 Issue: Bug report
 
+Please make sure your issue is reproducible on the latest bench version. The currently supported branches are:
+ - PyPI [latest](https://pypi.org/project/frappe-bench/) 
+ - master         (minor bug fixes)
+ - v5.x           (Merged with develop on every release)
+ - develop        (all updates)
+
 **Do the checklist before filing an issue:**
- - [ ] Can you replicate the issue?
+ - [ ] Can you replicate the issue on the supported bench versions?
  - [ ] Is this something you can debug and fix? Send a pull request! Bug fixes and documentation fixes are welcome
 
 **Describe the bug** :chart_with_downwards_trend:
@@ -34,9 +40,8 @@ If applicable, add screenshots to help explain your problem.
 
 **Version Information**
 
-Can be found out by running `bench version` in your respective bench folder
-
-- Bench Branch: `master` _(Only master is supported)_
+Can be found out by running `bench version` in your respective bench folder.
+- Bench Branch:
 - Frappe Version:
 - ERPNext Version:
 

.travis.yml

@@ -18,45 +18,55 @@ matrix:
   - name: "Python 2.7 Basic Setup"
     python: 2.7
     env: TEST=bench
-    script: python -m unittest -v bench.tests.test_init
+    script: python bench/tests/test_init.py TestBenchInit.basic
 
   - name: "Python 3.6 Basic Setup"
     python: 3.6
     env: TEST=bench
-    script: python -m unittest -v bench.tests.test_init
+    script: python bench/tests/test_init.py TestBenchInit.basic
 
   - name: "Python 3.7 Basic Setup"
     python: 3.7
     env: TEST=bench
-    script: python -m unittest -v bench.tests.test_init
+    script: python bench/tests/test_init.py TestBenchInit.basic
 
   - name: "Python 3.8 Production Setup"
     python: 3.8
     env: TEST=bench
-    script: python -m unittest -v bench.tests.test_setup_production
+    script: python bench/tests/test_setup_production.py TestSetupProduction.production
 
   - name: "Python 2.7 Production Setup"
     python: 2.7
     env: TEST=bench
-    script: python -m unittest -v bench.tests.test_setup_production
+    script: python bench/tests/test_setup_production.py TestSetupProduction.production
 
   - name: "Python 3.6 Production Setup"
     python: 3.6
     env: TEST=bench
-    script: python -m unittest -v bench.tests.test_setup_production
+    script: python bench/tests/test_setup_production.py TestSetupProduction.production
 
   - name: "Python 3.7 Production Setup"
     python: 3.7
     env: TEST=bench
-    script: python -m unittest -v bench.tests.test_setup_production
+    script: python bench/tests/test_setup_production.py TestSetupProduction.production
 
   - name: "Python 3.8 Production Setup"
     python: 3.8
     env: TEST=bench
-    script: python -m unittest -v bench.tests.test_setup_production
+    script: python bench/tests/test_setup_production.py TestSetupProduction.production
 
-  - name: "Python 3.6 Easy Install"
-    python: 3.6
+  - name: "Python 2.7 Tests"
+    python: 2.7
+    env: TEST=bench
+    script: python -m unittest -v bench.tests.test_init
+
+  - name: "Python 3.7 Tests"
+    python: 3.7
+    env: TEST=bench
+    script: python -m unittest -v bench.tests.test_init
+
+  - name: "Python 3.5 Easy Install"
+    python: 3.5
     env: TEST=easy_install
     script: sudo python $TRAVIS_BUILD_DIR/install.py --user travis --run-travis --production --verbose
 

README.md

@@ -18,6 +18,7 @@ Bench is a command-line utility that helps you to install, update, and manage mu
  - [Bench Manager](#bench-manager)
  - [Guides](#guides)
  - [Resources](#resources)
+ - [Development](#development)
  - [License](#license)
 
 
@@ -158,7 +159,7 @@ $ pip install frappe-bench
 For more extensive distribution-dependent documentation, check out the following guides:
 
  - [Hitchhiker's Guide to Installing Frappe on Linux](https://github.com/frappe/frappe/wiki/The-Hitchhiker%27s-Guide-to-Installing-Frappe-on-Linux)
- - [Hitchhiker's Guide to Installing Frappe on MacOS](https://github.com/frappe/frappe/wiki/The-Hitchhiker%27s-Guide-to-Installing-Frappe-on-Mac-OS-X)
+ - [Hitchhiker's Guide to Installing Frappe on MacOS](https://github.com/frappe/bench/wiki/Setting-up-a-Mac-for-Frappe-ERPNext-Development)
 
 
 ## Basic Usage
@@ -254,6 +255,36 @@ For an exhaustive list of guides, check out [Bench Guides](https://frappe.io/doc
 For an exhaustive list of resources, check out [Bench Resources](https://frappe.io/docs/user/en/bench/resources).
 
 
+## Development
+
+To contribute and develop on the bench CLI tool, clone this repo and create an editable install. In editable mode, you may get the following warning everytime you run a bench command:
+
+	WARN: bench is installed in editable mode!
+
+	This is not the recommended mode of installation for production. Instead, install the package from PyPI with: `pip install frappe-bench`
+
+
+```sh
+$ git clone https://github.com/frappe/bench ~/bench-repo
+$ pip3 install -e ~/bench-repo
+$ bench src
+/Users/frappe/bench-repo
+```
+
+To clear up the editable install and switch to a stable version of bench, uninstall via pip and delete the corresponding egg file from the python path.
+
+
+```sh
+# Delete bench installed in editable install
+$ rm -r $(find ~ -name '*.egg-info')
+$ pip3 uninstall frappe-bench
+
+# Install latest released version of bench
+$ pip3 install -U frappe-bench
+```
+
+To confirm the switch, check the output of `bench src`. It should change from something like `$HOME/bench-repo` to `/usr/local/lib/python3.6/dist-packages` and stop the editable install warnings from getting triggered at every command.
+
 ## License
 
 This repository has been released under the [GNU GPLv3 License](LICENSE).

bench/__init__.py

@@ -1,4 +1,4 @@
-VERSION = "5.1.0"
+VERSION = "5.2.1"
 PROJECT_NAME = "frappe-bench"
 FRAPPE_VERSION = None
 

bench/cli.py

@@ -18,6 +18,7 @@ from bench.utils import PatchError, bench_cache_file, check_latest_version, drop
 
 from_command_line = False
 change_uid_msg = "You should not run this command as root"
+src = os.path.dirname(__file__)
 
 
 def cli():
@@ -26,11 +27,14 @@ def cli():
 	command = " ".join(sys.argv)
 
 	change_working_directory()
-	logger = setup_logging() or logging.getLogger(bench.PROJECT_NAME)
+	logger = setup_logging()
 	logger.info(command)
-	check_uid()
+
+	if sys.argv[1] not in ("src", ):
+		check_uid()
+		change_uid()
+
 	change_dir()
-	change_uid()
 
 	if is_dist_editable(bench.PROJECT_NAME) and len(sys.argv) > 1 and sys.argv[1] != "src" and not get_config(".").get("developer_mode"):
 		log("bench is installed in editable mode!\n\nThis is not the recommended mode of installation for production. Instead, install the package from PyPI with: `pip install frappe-bench`\n", level=3)
@@ -72,7 +76,7 @@ def check_uid():
 
 
 def cmd_requires_root():
-	if len(sys.argv) > 2 and sys.argv[2] in ('production', 'sudoers', 'supervisor', 'lets-encrypt', 'fonts',
+	if len(sys.argv) > 2 and sys.argv[2] in ('production', 'sudoers', 'lets-encrypt', 'fonts',
 		'print', 'firewall', 'ssh-port', 'role', 'fail2ban', 'wildcard-ssl'):
 		return True
 	if len(sys.argv) >= 2 and sys.argv[1] in ('patch', 'renew-lets-encrypt', 'disable-production'):

bench/commands/setup.py

@@ -43,6 +43,7 @@ def reload_nginx():
 @click.option("--user", help="optional user argument")
 @click.option("--yes", help="Yes to regeneration of supervisor config", is_flag=True, default=False)
 def setup_supervisor(user=None, yes=False):
+	bench.config.supervisor.update_supervisord_config(user=user, yes=yes)
 	bench.config.supervisor.generate_supervisor_config(bench_path=".", user=user, yes=yes)
 
 

bench/commands/update.py

@@ -6,7 +6,7 @@ from bench.app import pull_apps
 from bench.utils import post_upgrade, patch_sites, build_assets
 
 
-@click.command('update', help="Updates bench tool and if executed in a bench directory, without any flags will backup, pull, setup requirements, build, run patches and restart bench. Using specific flags will only do certain tasks instead of all")
+@click.command('update', help="Performs an update operation on current bench. Without any flags will backup, pull, setup requirements, build, run patches and restart bench. Using specific flags will only do certain tasks instead of all")
 @click.option('--pull', is_flag=True, help="Pull updates for all the apps in bench")
 @click.option('--apps', type=str)
 @click.option('--patch', is_flag=True, help="Run migrations for all sites in the bench")

bench/commands/utils.py

@@ -8,11 +8,12 @@ import click
 
 @click.command('start', help="Start Frappe development processes")
 @click.option('--no-dev', is_flag=True, default=False)
+@click.option('--no-prefix', is_flag=True, default=False, help="Hide process name from bench start log")
 @click.option('--concurrency', '-c', type=str)
 @click.option('--procfile', '-p', type=str)
-def start(no_dev, concurrency, procfile):
+def start(no_dev, concurrency, procfile, no_prefix):
 	from bench.utils import start
-	start(no_dev=no_dev, concurrency=concurrency, procfile=procfile)
+	start(no_dev=no_dev, concurrency=concurrency, procfile=procfile, no_prefix=no_prefix)
 
 
 @click.command('restart', help="Restart supervisor processes or systemd units")
@@ -110,7 +111,7 @@ def download_translations():
 	download_translations_p()
 
 
-@click.command('renew-lets-encrypt', help="Renew Let's Encrypt certificate")
+@click.command('renew-lets-encrypt', help="Sets Up latest cron and Renew Let's Encrypt certificate")
 def renew_lets_encrypt():
 	from bench.config.lets_encrypt import renew_certs
 	renew_certs()
@@ -163,8 +164,8 @@ def disable_production():
 
 @click.command('src', help="Prints bench source folder path, which can be used as: cd `bench src`")
 def bench_src():
-	import bench
-	print(os.path.dirname(bench.__path__[0]))
+	from bench.cli import src
+	print(os.path.dirname(src))
 
 
 @click.command('find', help="Finds benches recursively from location")

bench/config/lets_encrypt.py

@@ -87,11 +87,17 @@ def run_certbot_and_setup_ssl(site, custom_domain, bench_path, interactive=True)
 
 def setup_crontab():
 	job_command = '/opt/certbot-auto renew -a nginx --post-hook "systemctl reload nginx"'
+	job_comment = 'Renew lets-encrypt every month'
+	print("Setting Up cron job to {0}".format(job_comment))
+
 	system_crontab = CronTab(user='root')
-	if job_command not in str(system_crontab):
-		job = system_crontab.new(command=job_command, comment="Renew lets-encrypt every month")
-		job.day.on(1)
-		system_crontab.write()
+
+	for job in system_crontab.find_comment(comment=job_comment): # Removes older entries
+		system_crontab.remove(job)
+
+	job = system_crontab.new(command=job_command, comment=job_comment)
+	job.setall('0 0 */1 * *') # Run at 00:00 every day-of-month
+	system_crontab.write()
 
 
 def create_dir_if_missing(path):
@@ -113,10 +119,13 @@ def get_certbot_path():
 
 
 def renew_certs():
+	# Needs to be run with sudo
 	click.confirm('Running this will stop the nginx service temporarily causing your sites to go offline\n'
 		'Do you want to continue?',
 		abort=True)
 
+	setup_crontab()
+
 	service('nginx', 'stop')
 	exec_cmd("{path} renew".format(path=get_certbot_path()))
 	service('nginx', 'start')

bench/config/production_setup.py

@@ -1,13 +1,18 @@
 # imports - standard imports
 import os
+import logging
 import sys
 
 # imports - module imports
+import bench
 from bench.config.common_site_config import get_config
 from bench.config.nginx import make_nginx_conf
-from bench.config.supervisor import generate_supervisor_config
+from bench.config.supervisor import generate_supervisor_config, update_supervisord_config
 from bench.config.systemd import generate_systemd_config
-from bench.utils import CommandFailedError, exec_cmd, find_executable, fix_prod_setup_perms, get_bench_name, get_cmd_output
+from bench.utils import CommandFailedError, exec_cmd, find_executable, fix_prod_setup_perms, get_bench_name, get_cmd_output, log
+
+
+logger = logging.getLogger(bench.PROJECT_NAME)
 
 
 def setup_production_prerequisites():
@@ -23,14 +28,20 @@ def setup_production_prerequisites():
 
 
 def setup_production(user, bench_path='.', yes=False):
+	print("Setting Up prerequisites...")
 	setup_production_prerequisites()
 	if get_config(bench_path).get('restart_supervisor_on_update') and get_config(bench_path).get('restart_systemd_on_update'):
 		raise Exception("You cannot use supervisor and systemd at the same time. Modify your common_site_config accordingly." )
 
 	if get_config(bench_path).get('restart_systemd_on_update'):
+		print("Setting Up systemd...")
 		generate_systemd_config(bench_path=bench_path, user=user, yes=yes)
 	else:
+		print("Setting Up supervisor...")
+		update_supervisord_config(user=user, yes=yes)
 		generate_supervisor_config(bench_path=bench_path, user=user, yes=yes)
+
+	print("Setting Up NGINX...")
 	make_nginx_conf(bench_path=bench_path, yes=yes)
 	fix_prod_setup_perms(bench_path, frappe_user=user)
 	remove_default_nginx_configs()
@@ -38,6 +49,7 @@ def setup_production(user, bench_path='.', yes=False):
 	bench_name = get_bench_name(bench_path)
 	nginx_conf = '/etc/nginx/conf.d/{bench_name}.conf'.format(bench_name=bench_name)
 
+	print("Setting Up symlinks and reloading services...")
 	if get_config(bench_path).get('restart_supervisor_on_update'):
 		supervisor_conf_extn = "ini" if is_centos7() else "conf"
 		supervisor_conf = os.path.join(get_supervisor_confdir(), '{bench_name}.{extn}'.format(
@@ -100,7 +112,7 @@ def service(service_name, service_option):
 			exec_cmd(service_manager_command)
 
 		else:
-			raise Exception('No service manager found')
+			log("No service manager found: '{0} {1}' failed to execute".format(service_name, service_option), level=2)
 
 
 def get_supervisor_confdir():

bench/config/supervisor.py

@@ -1,5 +1,6 @@
 # imports - standard imports
 import getpass
+import logging
 import os
 
 # imports - module imports
@@ -13,13 +14,14 @@ import click
 from six.moves import configparser
 
 
+logger = logging.getLogger(bench.PROJECT_NAME)
+
+
 def generate_supervisor_config(bench_path, user=None, yes=False):
 	"""Generate supervisor config for respective bench path"""
 	if not user:
 		user = getpass.getuser()
 
-	update_supervisord_conf(user=user)
-
 	template = bench.config.env.get_template('supervisor.conf')
 	config = get_config(bench_path=bench_path)
 	bench_dir = os.path.abspath(bench_path)
@@ -64,14 +66,23 @@ def get_supervisord_conf():
 			return possibility
 
 
-def update_supervisord_conf(user):
-	"""From bench v5.0, we're moving to supervisor running as user"""
+def update_supervisord_config(user=None, yes=False):
+	"""From bench v5.x, we're moving to supervisor running as user"""
 	from bench.config.production_setup import service
 
+	if not user:
+		user = getpass.getuser()
+
 	supervisord_conf = get_supervisord_conf()
 	section = "unix_http_server"
+	updated_values = {
+		"chmod": "0760",
+		"chown": "{user}:{user}".format(user=user)
+	}
+	supervisord_conf_changes = ""
 
 	if not supervisord_conf:
+		logger.log("supervisord.conf not found")
 		return
 
 	config = configparser.ConfigParser()
@@ -79,12 +90,35 @@ def update_supervisord_conf(user):
 
 	if section not in config.sections():
 		config.add_section(section)
+		action = "Section {0} Added".format(section)
+		logger.log(action)
+		supervisord_conf_changes += '\n' + action
 
-	config.set(section, "chmod", "0760")
-	config.set(section, "chown", "{user}:{user}".format(user=user))
+	for key, value in updated_values.items():
+		try:
+			current_value = config.get(section, key)
+		except configparser.NoOptionError:
+			current_value = ""
 
-	with open(supervisord_conf, "w") as f:
-		config.write(f)
+		if current_value.strip() != value:
+			config.set(section, key, value)
+			action = "Updated supervisord.conf: '{0}' changed from '{1}' to '{2}'".format(key, current_value, value)
+			logger.log(action)
+			supervisord_conf_changes += '\n' + action
 
-	# restart supervisor to take new changes into effect
-	service('supervisor', 'restart')
+	if not supervisord_conf_changes:
+		logger.log("supervisord.conf not updated")
+		return
+
+	if not yes:
+		click.confirm("{0} will be updated with the following values:\n{1}\nDo you want to continue?".format(supervisord_conf, supervisord_conf_changes), abort=True)
+
+	try:
+		with open(supervisord_conf, "w") as f:
+			config.write(f)
+			logger.log("Updated supervisord.conf at '{0}'".format(supervisord_conf))
+	except Exception as e:
+		logger.log("Updating supervisord.conf failed due to '{0}'".format(e))
+
+	# Reread supervisor configuration, reload supervisord and supervisorctl, restart services that were started
+	service('supervisor', 'reload')

bench/config/templates/nginx.conf

@@ -85,7 +85,7 @@ server {
 	}
 
 	location @webserver {
-		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+		proxy_set_header X-Forwarded-For $remote_addr;
 		proxy_set_header X-Forwarded-Proto $scheme;
 		proxy_set_header X-Frappe-Site-Name {{ site_name }};
 		proxy_set_header Host $host;

bench/patches/patches.txt

@@ -5,3 +5,4 @@ bench.patches.v4.update_node
 bench.patches.v4.update_socketio
 bench.patches.v4.install_yarn #2
 bench.patches.v5.fix_user_permissions
+bench.patches.v5.fix_backup_cronjob

bench/patches/v5/fix_backup_cronjob.py

@@ -0,0 +1,15 @@
+from bench.config.common_site_config import get_config
+from crontab import CronTab
+
+
+def execute(bench_path):
+	"""
+		This patch fixes a cron job that would backup sites every minute per 6 hours
+	"""
+
+	user = get_config(bench_path=bench_path).get('frappe_user')
+	user_crontab = CronTab(user=user)
+
+	for job in user_crontab.find_comment("bench auto backups set for every 6 hours"):
+		job.every(6).hours()
+		user_crontab.write()

bench/playbooks/roles/dns_caching/handlers/main.yml

@@ -1,4 +1,4 @@
 ---
-- name: Restart network manager
+- name: restart network manager
   service: name=NetworkManager state=restarted
 ...

bench/playbooks/roles/nodejs/defaults/main.yml

@@ -1,3 +1,3 @@
 ---
-node_version: 8
+node_version: 12
 ...

bench/tests/test_base.py

@@ -1,19 +1,19 @@
 # imports - standard imports
+import getpass
 import json
 import os
 import shutil
 import subprocess
 import sys
+import traceback
 import unittest
-import getpass
-
-# imports - module imports
-import bench
-import bench.utils
 
 # imports - third party imports
 from six import PY2
 
+# imports - module imports
+import bench
+import bench.utils
 
 if PY2:
 	FRAPPE_BRANCH = "version-12"
@@ -102,3 +102,9 @@ class TestBenchBase(unittest.TestCase):
 		if os.environ.get("CI"):
 			return not subprocess.call(["sudo", "test", "-f", path])
 		return os.path.isfile(path)
+
+	def get_traceback(self):
+		exc_type, exc_value, exc_tb = sys.exc_info()
+		trace_list = traceback.format_exception(exc_type, exc_value, exc_tb)
+		body = "".join(str(t) for t in trace_list)
+		return body

bench/tests/test_init.py

@@ -11,7 +11,7 @@ import git
 import bench
 import bench.utils
 from bench.release import get_bumped_version
-from bench.tests.test_base import TestBenchBase, FRAPPE_BRANCH
+from bench.tests.test_base import FRAPPE_BRANCH, TestBenchBase
 
 
 class TestBenchInit(TestBenchBase):
@@ -34,6 +34,13 @@ class TestBenchInit(TestBenchBase):
 		self.assert_config(bench_name)
 
 
+	def basic(self):
+		try:
+			self.test_init()
+		except Exception:
+			print(self.get_traceback())
+
+
 	def test_multiple_benches(self):
 		for bench_name in ("test-bench-1", "test-bench-2"):
 			self.init_bench(bench_name)

bench/tests/test_setup_production.py

@@ -19,7 +19,7 @@ class TestSetupProduction(TestBenchBase):
 		for bench_name in ("test-bench-1", "test-bench-2"):
 			bench_path = os.path.join(os.path.abspath(self.benches_path), bench_name)
 			self.init_bench(bench_name)
-			bench.utils.exec_cmd("sudo bench setup production {0}".format(user), cwd=bench_path)
+			bench.utils.exec_cmd("sudo bench setup production {0} --yes".format(user), cwd=bench_path)
 			self.assert_nginx_config(bench_name)
 			self.assert_supervisor_config(bench_name)
 			self.assert_supervisor_process(bench_name)
@@ -33,6 +33,13 @@ class TestSetupProduction(TestBenchBase):
 			bench.utils.exec_cmd("sudo bench disable-production", cwd=bench_path)
 
 
+	def production(self):
+		try:
+			self.test_setup_production()
+		except Exception:
+			print(self.get_traceback())
+
+
 	def assert_nginx_config(self, bench_name):
 		conf_src = os.path.join(os.path.abspath(self.benches_path), bench_name, 'config', 'nginx.conf')
 		conf_dest = "/etc/nginx/conf.d/{bench_name}.conf".format(bench_name=bench_name)

bench/utils.py

@@ -400,7 +400,7 @@ def setup_backups(bench_path='.'):
 
 	if job_command not in str(system_crontab):
 		job = system_crontab.new(command=job_command, comment="bench auto backups set for every 6 hours")
-		job.hour.every(6)
+		job.every(6).hours()
 		system_crontab.write()
 
 
@@ -443,15 +443,18 @@ def setup_logging(bench_path='.'):
 	logging.Logger.log = logv
 
 	if os.path.exists(os.path.join(bench_path, 'logs')):
-		logger = logging.getLogger(bench.PROJECT_NAME)
 		log_file = os.path.join(bench_path, 'logs', 'bench.log')
-		formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
 		hdlr = logging.FileHandler(log_file)
-		hdlr.setFormatter(formatter)
-		logger.addHandler(hdlr)
-		logger.setLevel(logging.DEBUG)
+	else:
+		hdlr = logging.NullHandler()
 
-		return logger
+	logger = logging.getLogger(bench.PROJECT_NAME)
+	formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
+	hdlr.setFormatter(formatter)
+	logger.addHandler(hdlr)
+	logger.setLevel(logging.DEBUG)
+
+	return logger
 
 
 def get_process_manager():
@@ -461,7 +464,7 @@ def get_process_manager():
 			return proc_man_path
 
 
-def start(no_dev=False, concurrency=None, procfile=None):
+def start(no_dev=False, concurrency=None, procfile=None, no_prefix=False):
 	program = get_process_manager()
 	if not program:
 		raise Exception("No process manager found")
@@ -476,6 +479,9 @@ def start(no_dev=False, concurrency=None, procfile=None):
 	if procfile:
 		command.extend(['-f', procfile])
 
+	if no_prefix:
+		command.extend(['--no-prefix'])
+		
 	os.execv(program, command)
 
 

docs/bench_usage.md

@@ -72,7 +72,7 @@ These commands belong directly to the bench group so they can be invoked directl
 
  - **init**: Initialize a new bench instance in the specified path. This sets up a complete bench folder with an `apps` folder which contains all the Frappe apps available in the current bench, `sites` folder that stores all site data seperated by individual site folders, `config` folder that contains your redis, NGINX and supervisor configuration files. The `env` folder consists of all python dependencies the current bench and installed Frappe applications have.
  - **restart**: Restart web, supervisor, systemd processes units. Used in production setup.
- - **update**: Updates bench tool and if executed in a bench directory, without any flags will backup, pull, setup requirements, build, run patches and restart bench. Using specific flags will only do certain tasks instead of all.
+ - **update**: If executed in a bench directory, without any flags will backup, pull, setup requirements, build, run patches and restart bench. Using specific flags will only do certain tasks instead of all.
  - **migrate-env**: Migrate Virtual Environment to desired Python version. This regenerates the `env` folder with the specified Python version.
  - **retry-upgrade**: Retry a failed upgrade
  - **disable-production**: Disables production environment for the bench.

docs/commands_and_usage.md

@@ -2,9 +2,14 @@
 
 * Updating
 
-	Currently, `bench update` can be run from any directory however the context of the command changes. If run from a bench directory, the vanilla command itself updates all apps, runs migrations and backs up all sites.
+To update the bench CLI tool, depending on your method of installation, you may use 
 
-		bench update
+	pip3 install -U frappe-bench
+
+
+To backup, update all apps and sites on your bench, you may use
+
+	bench update
 
 
 To manually update the bench, run `bench update` to update all the apps, run

install.py

@@ -20,6 +20,7 @@ execution_time = "{:%H:%M}".format(execution_timestamp)
 log_file_name = "easy-install__{0}__{1}.log".format(execution_day, execution_time.replace(':', '-'))
 log_path = os.path.join(tmp_log_folder, log_file_name)
 log_stream = sys.stdout
+distro_required = not ((sys.version_info.major < 3) or (sys.version_info.major == 3 and sys.version_info.minor < 5))
 
 
 def log(message, level=0):
@@ -78,8 +79,8 @@ def check_distribution_compatibility():
 	dist_name, dist_version = get_distribution_info()
 	supported_dists = {
 		'macos': [10.9, 10.10, 10.11, 10.12],
-		'ubuntu': [14, 15, 16, 18, 19],
-		'debian': [8, 9],
+		'ubuntu': [14, 15, 16, 18, 19, 20],
+		'debian': [8, 9, 10],
 		'centos': [7]
 	}
 
@@ -94,10 +95,32 @@ def check_distribution_compatibility():
 		log("Sorry, the installer doesn't support {0}. Aborting installation!".format(dist_name), level=2)
 
 
+def import_with_install(package):
+	# copied from https://discuss.erpnext.com/u/nikunj_patel
+	# https://discuss.erpnext.com/t/easy-install-setup-guide-for-erpnext-installation-on-ubuntu-20-04-lts-with-some-modification-of-course/62375/5
+	# need to move to top said v13 for fully python3 era
+	import importlib
+
+	try:
+		importlib.import_module(package)
+	except ImportError:
+		# caveat : pip3 must be installed
+
+		import pip
+
+		pip.main(['install', package])
+	finally:
+		globals()[package] = importlib.import_module(package)
+
+
 def get_distribution_info():
 	# return distribution name and major version
 	if platform.system() == "Linux":
-		current_dist = platform.dist()
+		if distro_required:
+			current_dist = distro.linux_distribution(full_distribution_name=True)
+		else:
+			current_dist = platform.dist()
+
 		return current_dist[0].lower(), current_dist[1].rsplit('.')[0]
 
 	elif platform.system() == "Darwin":
@@ -140,7 +163,7 @@ def install_prerequisites():
 	install_package('pip3', 'python3-pip')
 
 	success = run_os_command({
-		'python3': "sudo -H python3 -m pip install --upgrade setuptools cryptography ansible==2.8.5 pip"
+		'python3': "sudo -H python3 -m pip install --upgrade setuptools wheel cryptography ansible==2.8.5 pip"
 	})
 
 	if not (success or shutil.which('ansible')):
@@ -246,7 +269,7 @@ def install_bench(args):
 
 def clone_bench_repo(args):
 	'''Clones the bench repository in the user folder'''
-	branch = args.bench_branch or 'master'
+	branch = args.bench_branch or 'develop'
 	repo_url = args.repo_url or 'https://github.com/frappe/bench'
 
 	if os.path.exists(tmp_bench_repo):
@@ -366,6 +389,12 @@ def run_playbook(playbook_name, sudo=False, extra_vars=None):
 	return success
 
 
+def setup_script_requirements():
+	if distro_required:
+		install_package('pip3', 'python3-pip')
+		import_with_install('distro')
+
+
 def parse_commandline_args():
 	import argparse
 
@@ -396,17 +425,19 @@ def parse_commandline_args():
 	parser.add_argument('--overwrite', dest='overwrite', action='store_true', default=False, help='Whether to overwrite an existing bench')
 	# set passwords
 	parser.add_argument('--mysql-root-password', dest='mysql_root_password', help='Set mysql root password')
-	parser.add_argument('--mariadb-version', dest='mariadb_version', default='10.2', help='Specify mariadb version')
+	parser.add_argument('--mariadb-version', dest='mariadb_version', default='10.4', help='Specify mariadb version')
 	parser.add_argument('--admin-password', dest='admin_password', help='Set admin password')
 	parser.add_argument('--bench-name', dest='bench_name', help='Create bench with specified name. Default name is frappe-bench')
 	# Python interpreter to be used
 	parser.add_argument('--python', dest='python', default='python3', help=argparse.SUPPRESS)
 	# LXC Support
 	parser.add_argument('--container', dest='container', default=False, action='store_true', help='Use if you\'re creating inside LXC')
+
 	args = parser.parse_args()
 
 	return args
 
+
 if __name__ == '__main__':
 	if sys.version[0] == '2':
 		if not os.environ.get('CI'):
@@ -433,10 +464,11 @@ if __name__ == '__main__':
 	with warnings.catch_warnings():
 		warnings.simplefilter("ignore")
 		setup_log_stream(args)
+		install_prerequisites()
+		setup_script_requirements()
 		check_distribution_compatibility()
 		check_system_package_managers()
 		check_environment()
-		install_prerequisites()
 		install_bench(args)
 
 	log("Bench + Frappe + ERPNext has been successfully installed!")

requirements.txt

@@ -5,6 +5,6 @@ Jinja2==2.10.3
 python-crontab==2.4.0
 requests==2.22.0
 semantic-version==2.8.2
-setuptools==40.8.0
-six==1.12.0
-virtualenv==16.6.0
+setuptools
+six
+virtualenv

vm/scripts/set_message_production.sh

@@ -6,7 +6,7 @@ message="
  Please access ERPNext by going to http://localhost:8080 on the host system.
  The username is \"Administrator\" and password is \"admin\"
 
- Do consider donating at https://frappe.io/buy
+ Consider buying professional support from us at https://erpnext.com/support 
 
  To update, login as
  username: frappe