Merge branch 'master' into develop

README.md

@@ -1,43 +1,103 @@
-Bench
+# Bench
-=====
 
-The bench allows you to setup Frappe / ERPNext apps on your local Linux (CentOS 6, Debian 7, Ubuntu, etc) machine or a production server. You can use the bench to serve multiple frappe sites. If you are using a DigitalOcean droplet or any other VPS / Dedicated Server, make sure it has >= 1Gb of ram or has swap setup properly.
+[![Build Status](https://travis-ci.org/frappe/bench.svg?branch=master)](https://travis-ci.org/frappe/bench)
+
+The bench is a command-line utility that helps you to install apps, manage multiple sites and update Frappe / ERPNext apps on */nix (CentOS 6, Debian 7, Ubuntu, etc) for development and production. Bench will also create nginx and supervisor config files, setup backups and much more.
+
+If you are using on a VPS make sure it has >= 1Gb of RAM or has swap setup properly.
 
 To do this install, you must have basic information on how Linux works and should be able to use the command-line. If you are looking easier ways to get started and evaluate ERPNext, [download the Virtual Machine](https://erpnext.com/download) or take [a free trial on erpnext.com](https://erpnext.com/pricing).
 
-If you have questions, please ask them on our [forum](https://discuss.erpnext.com/).
+If you have questions, please ask them on the [forum](https://discuss.erpnext.com/).
 
-Installation
+## Installation
-============
 
-Easy Setup
+## Manual Install
-----------
 
-- This is an opinionated setup with logging and SE Linux. So, it is best to setup on a blank server.
+To manually install frappe/erpnext here are the steps
+
+#### 1. Install Pre-requisites
+
+- Python 2.7
+- MariaDB 10+
+- Nginx (for production)
+- Nodejs
+- Redis
+- cron (crontab is required)
+- wkhtmltopdf with patched Qt (for pdf generation)
+
+#### 2. Install Bench
+
+Install bench as a *non root* user,
+
+	git clone https://github.com/frappe/bench bench-repo
+	sudo pip install -e bench-repo
+
+Note: Please do not remove the bench directory the above commands will create
+
+#### Basic Usage
+
+* Create a new bench
+
+	The init command will create a bench directory with frappe framework
+	installed. It will be setup for periodic backups and auto updates once
+	a day.
+
+		bench init frappe-bench && cd frappe-bench
+
+* Add a site
+
+	Frappe apps are run by frappe sites and you will have to create at least one
+	site. The new-site command allows you to do that.
+
+		bench new-site site1.local
+
+* Add apps
+
+	The get-app command gets remote frappe apps from a remote git repository and installs them. Example: [erpnext](https://github.com/frappe/erpnext)
+
+		bench get-app erpnext https://github.com/frappe/erpnext
+
+* Install apps
+
+	To install an app on your new site, use the bench `install-app` command.
+
+		bench --site site1.local install-app erpnext
+		
+* Start bench
+
+	To start using the bench, use the `bench start` command
+
+		bench start
+
+	To login to Frappe / ERPNext, open your browser and go to `[your-external-ip]:8000`, probably `localhost:8000`
+
+	The default username is "Administrator" and password is what you set when you created the new site.
+
+
+---
+
+## Easy Install
+
+- This is an opinionated setup so it is best to setup on a blank server.
 - Works on Ubuntu 14.04 to 16.04, CentOS 7+, Debian 7 to 8 and MacOS X.
 - You may have to install Python 2.7 (eg on Ubuntu 16.04+) by running `apt-get install python-minimal`
 - This script will install the pre-requisites, install bench and setup an ERPNext site
 - Passwords for Frappe Administrator and MariaDB (root) will be asked
 - You can then login as **Administrator** with the Administrator password
-- If you find any problems, post them on our forum: [https://discuss.erpnext.com](https://discuss.erpnext.com)
+- If you find any problems, post them on the forum: [https://discuss.erpnext.com](https://discuss.erpnext.com)
-
-Production vs Develop
----------------------
-
-*Production* setup should be run on a new box and installs nginx and supervisor to manage the processes. *Develop* setup uses `honcho` to manage the processes and uses the built-in web server (`bench start`)
-
-Steps
------
 
 Open your Terminal and enter:
 
-#### Linux:
+#### 1. Download the install script
+
+For Linux:
 
 	wget https://raw.githubusercontent.com/frappe/bench/master/playbooks/install.py
 
-#### MacOS:
+For Mac OSX:
 
-Install X Code (from App store) and HomeBrew (http://brew.sh/)
+Install X Code (from App store) and HomeBrew (http://brew.sh/) first
 
 	brew install python
 	brew install git
@@ -46,49 +106,40 @@ Download the Script
 
 	curl "https://raw.githubusercontent.com/frappe/bench/master/playbooks/install.py" -o install.py
 
-#### Run the Script
+#### 2. Run the install script
 
-	# For development
+If you are on a fresh server and logged in as root, use --user flag to create a user and install using that user
-	sudo python install.py --develop
 
-	# For production
-	sudo python install.py --production
-
-	# If you're logged in as root, use --user flag to create a user and install using that user
 	python install.py --develop --user frappe
 
+For developer setup:
 
-For development, you have to explicitly start services by running `bench start`. This script requires Python2.7+ installed on your machine. You will have to manually create a new site (`bench new-site`) and get apps that you need (`bench get-app`, `bench install-app`).
+	sudo python install.py --develop
 
-For production, you will have a preinstalled site with ERPNext installed in it.
+For production:
 
-You need to run this with a user that is **not** `root`, but can `sudo`. If you don't have such a user, you can search the web for *How to add a new user in { your OS }* and *How to add an existing user to sudoers in { your OS }*.
+	sudo python install.py --production
 
-On Mac OS X, you will have to create a group with the same name as *{ your User }*. On creating this group, you have to assign *{ your User }* to it. You can do this by going to "System preferences" -> "Users & Groups" -> "+" (as if you were adding new account) -> Under "New account" select "Group" -> Type in group name -> "Create group"
+#### What will this script do?
 
-This script will:
+- Install all the pre-requisites
+- Install the command line `bench`
+- Create a new bench (a folder that will contain your entire frappe/erpnext setup)
+- Create a new site on the bench
 
-- Install pre-requisites like git and ansible
+#### How do I start ERPNext
-- Shallow clones this bench repository under `/usr/local/frappe/bench-repo`
-- Runs the Ansible playbook 'playbooks/develop/install.yml', which:
-	- Installs
-		- MariaDB and its config
-		- Redis
-		- NodeJS
-		- WKHTMLtoPDF with patched QT
-	- Initializes a new Bench at `~/frappe/frappe-bench` with `frappe` framework already installed under `apps`.
 
-####Script Options:
+1. For development: Go to your bench folder (`frappe-bench` by default) and start the bench with `bench start`
-```
+2. For production: Your process will be setup and managed by `nginx` and `supervisor`. [Setup Production](https://frappe.github.io/frappe/user/en/bench/guides/setup-production.html)
-	--help
+
-	--verbose
+---
-	--develop
+
-	--production
+Help
-	--site
+====
-	--user
+
-	--bench-branch
+For bench help, you can type
-	--repo-url
+
-```
+	bench --help
 
 Updating
 ========
@@ -108,17 +159,16 @@ You can also run the parts of the bench selectively.
 
 `bench update --requirements` will only update dependencies (python packages) for the apps installed
 
-
 Guides
 =======
 - [Configuring HTTPS](https://frappe.github.io/frappe/user/en/bench/guides/configuring-https.html)
 - [Using Let's Encrypt to setup HTTPS](https://frappe.github.io/frappe/user/en/bench/guides/lets-encrypt-ssl-setup.html)
 - [Diagnosing the Scheduler](https://frappe.github.io/frappe/user/en/bench/guides/diagnosing-the-scheduler.html)
-- [Change Hostname](https://frappe.github.io/frappe/user/en/bench/guides/how-to-change-host-name-from-localhost.html)
+- [Change Hostname](https://frappe.github.io/frappe/user/en/bench/guides/adding-custom-domains)
 - [Manual Setup](https://frappe.github.io/frappe/user/en/bench/guides/manual-setup.html)
 - [Setup Production](https://frappe.github.io/frappe/user/en/bench/guides/setup-production.html)
 - [Setup Multitenancy](https://frappe.github.io/frappe/user/en/bench/guides/setup-multitenancy.html)
-- [Stopping Production](https://frappe.github.io/frappe/user/en/bench/guides/stop-production-and-start-development.html)
+- [Stopping Production](https://github.com/frappe/bench/wiki/Stopping-Production-and-starting-Development)
 
 
 Resources

bench/app.py

@@ -49,6 +49,9 @@ def write_appstxt(apps, bench_path='.'):
 		return f.write('\n'.join(apps))
 
 def get_app(git_url, branch=None, bench_path='.', build_asset_files=True, verbose=False):
+	#less verbose app install
+	if '/' not in git_url:
+		git_url = 'https://github.com/frappe/' + git_url
 	#Gets repo name from URL
 	repo_name = git_url.rsplit('/', 1)[1].rsplit('.', 1)[0]
 	logger.info('getting app {}'.format(repo_name))
@@ -69,7 +72,7 @@ def get_app(git_url, branch=None, bench_path='.', build_asset_files=True, verbos
 			apps_path = os.path.join(os.path.abspath(bench_path), 'apps')
 			os.rename(os.path.join(apps_path, repo_name), os.path.join(apps_path, app_name))
 
-	print 'installing', app_name
+	print('installing', app_name)
 	install_app(app=app_name, bench_path=bench_path, verbose=verbose)
 
 	if build_asset_files:
@@ -92,20 +95,21 @@ def new_app(app, bench_path='.'):
 		run_frappe_cmd('make-app', apps, app, bench_path=bench_path)
 	install_app(app, bench_path=bench_path)
 
-def install_app(app, bench_path='.', verbose=False):
+def install_app(app, bench_path='.', verbose=False, no_cache=False):
 	logger.info('installing {}'.format(app))
 	# find_links = '--find-links={}'.format(conf.get('wheel_cache_dir')) if conf.get('wheel_cache_dir') else ''
 	find_links = ''
-	exec_cmd("{pip} install {quiet} {find_links} -e {app}".format(
+	exec_cmd("{pip} install {quiet} {find_links} -e {app} {no_cache}".format(
 				pip=os.path.join(bench_path, 'env', 'bin', 'pip'),
 				quiet="-q" if not verbose else "",
+				no_cache='--no-cache-dir' if not no_cache else '',
 				app=os.path.join(bench_path, 'apps', app),
 				find_links=find_links))
 	add_to_appstxt(app, bench_path=bench_path)
 
 def remove_app(app, bench_path='.'):
 	if not app in get_apps(bench_path):
-		print "No app named {0}".format(app)
+		print("No app named {0}".format(app))
 		sys.exit(1)
 
 	app_path = os.path.join(bench_path, 'apps', app)
@@ -117,7 +121,7 @@ def remove_app(app, bench_path='.'):
 		if os.path.exists(req_file):
 			out = subprocess.check_output(["bench", "--site", site, "list-apps"], cwd=bench_path)
 			if re.search(r'\b' + app + r'\b', out):
-				print "Cannot remove, app is installed on site: {0}".format(site)
+				print("Cannot remove, app is installed on site: {0}".format(site))
 				sys.exit(1)
 
 	exec_cmd(["{0} uninstall -y {1}".format(pip, app_path)])
@@ -128,14 +132,40 @@ def remove_app(app, bench_path='.'):
 		restart_supervisor_processes(bench_path=bench_path)
 
 
-def pull_all_apps(bench_path='.'):
+def pull_all_apps(bench_path='.', reset=False):
+	'''Check all apps if there no local changes, pull'''
 	rebase = '--rebase' if get_config(bench_path).get('rebase_on_pull') else ''
 	
+	# chech for local changes
+	if not reset:
+		for app in get_apps(bench_path=bench_path):
+			app_dir = get_repo_dir(app, bench_path=bench_path)
+			if os.path.exists(os.path.join(app_dir, '.git')):
+				out = subprocess.check_output(["git", "status"], cwd=app_dir)
+				if not re.search(r'nothing to commit, working (directory|tree) clean', out):
+					print('''
+
+Cannot proceed with update: You have local changes in app "{0}" that are not committed.
+
+Here are your choices:
+
+1. Merge the {0} app manually with "git pull" / "git pull --rebase" and fix conflicts.
+1. Temporarily remove your changes with "git stash" or discard them completely
+	with "bench update --reset" or for individual repositries "git reset --hard"
+2. If your changes are helpful for others, send in a pull request via GitHub and
+	wait for them to be merged in the core.'''.format(app))
+					sys.exit(1)
+
 	for app in get_apps(bench_path=bench_path):
 		app_dir = get_repo_dir(app, bench_path=bench_path)
 		if os.path.exists(os.path.join(app_dir, '.git')):
 			remote = get_remote(app)
 			logger.info('pulling {0}'.format(app))
+			if reset:
+				exec_cmd("git fetch --all", cwd=app_dir)
+				exec_cmd("git reset --hard {remote}/{branch}".format(
+					remote=remote, branch=get_current_branch(app,bench_path=bench_path)), cwd=app_dir)
+			else:
 				exec_cmd("git pull {rebase} {remote} {branch}".format(rebase=rebase,
 					remote=remote, branch=get_current_branch(app, bench_path=bench_path)), cwd=app_dir)
 			exec_cmd('find . -name "*.pyc" -delete', cwd=app_dir)
@@ -208,7 +238,7 @@ def get_upstream_version(app, branch=None, bench_path='.'):
 		branch = get_current_branch(app, bench_path=bench_path)
 	try:
 		contents = subprocess.check_output(['git', 'show', 'upstream/{branch}:{app}/__init__.py'.format(branch=branch, app=app)], cwd=repo_dir, stderr=subprocess.STDOUT)
-	except subprocess.CalledProcessError, e:
+	except subprocess.CalledProcessError as e:
 		if "Invalid object" in e.output:
 			return None
 		else:
@@ -224,7 +254,7 @@ def get_repo_dir(app, bench_path='.'):
 
 def switch_branch(branch, apps=None, bench_path='.', upgrade=False, check_upgrade=True):
 	from .utils import update_requirements, backup_all_sites, patch_sites, build_assets, pre_upgrade, post_upgrade
-	import utils
+	from . import utils
 	apps_dir = os.path.join(bench_path, 'apps')
 	version_upgrade = (False,)
 	switched_apps = []
@@ -243,7 +273,7 @@ def switch_branch(branch, apps=None, bench_path='.', upgrade=False, check_upgrad
 					version_upgrade = is_version_upgrade(app=app, bench_path=bench_path, branch=branch)
 					if version_upgrade[0] and not upgrade:
 						raise MajorVersionUpgradeException("Switching to {0} will cause upgrade from {1} to {2}. Pass --upgrade to confirm".format(branch, version_upgrade[1], version_upgrade[2]), version_upgrade[1], version_upgrade[2])
-				print "Switching for "+app
+				print("Switching for "+app)
 				unshallow = "--unshallow" if os.path.exists(os.path.join(app_dir, ".git", "shallow")) else ""
 				exec_cmd("git config --unset-all remote.upstream.fetch", cwd=app_dir)
 				exec_cmd("git config --add remote.upstream.fetch '+refs/heads/*:refs/remotes/upstream/*'", cwd=app_dir)
@@ -252,14 +282,14 @@ def switch_branch(branch, apps=None, bench_path='.', upgrade=False, check_upgrad
 				exec_cmd("git merge upstream/{branch}".format(branch=branch), cwd=app_dir)
 				switched_apps.append(app)
 			except CommandFailedError:
-				print "Error switching to branch {0} for {1}".format(branch, app)
+				print("Error switching to branch {0} for {1}".format(branch, app))
 			except InvalidRemoteException:
-				print "Remote does not exist for app "+app
+				print("Remote does not exist for app "+app)
 			except InvalidBranchException:
-				print "Branch {0} does not exist in Upstream for {1}".format(branch, app)
+				print("Branch {0} does not exist in Upstream for {1}".format(branch, app))
 
 	if switched_apps:
-		print "Successfully switched branches for:\n" + "\n".join(switched_apps)
+		print("Successfully switched branches for:\n" + "\n".join(switched_apps))
 
 	if version_upgrade[0] and upgrade:
 		update_requirements()

bench/cli.py

@@ -26,9 +26,9 @@ def cli():
 		return frappe_cmd()
 
 	elif len(sys.argv) > 1 and sys.argv[1]=="--help":
-		print click.Context(bench_command).get_help()
+		print(click.Context(bench_command).get_help())
-		print
+		print()
-		print get_frappe_help()
+		print(get_frappe_help())
 		return
 
 	elif len(sys.argv) > 1 and sys.argv[1] in get_apps():
@@ -43,11 +43,11 @@ def cli():
 
 def check_uid():
 	if cmd_requires_root() and not is_root():
-		print 'superuser privileges required for this command'
+		print('superuser privileges required for this command')
 		sys.exit(1)
 
 def cmd_requires_root():
-	if len(sys.argv) > 2 and sys.argv[2] in ('production', 'sudoers', 'lets-encrypt', 'fonts', 'reload-nginx'):
+	if len(sys.argv) > 2 and sys.argv[2] in ('production', 'sudoers', 'lets-encrypt', 'fonts', 'reload-nginx', 'firewall', 'ssh-port'):
 	    return True
 	if len(sys.argv) >= 2 and sys.argv[1] in ('patch', 'renew-lets-encrypt', 'disable-production'):
 	    return True
@@ -69,7 +69,7 @@ def change_uid():
 			drop_privileges(uid_name=frappe_user, gid_name=frappe_user)
 			os.environ['HOME'] = pwd.getpwnam(frappe_user).pw_dir
 		else:
-			print 'You should not run this command as root'
+			print('You should not run this command as root')
 			sys.exit(1)
 
 def old_frappe_cli(bench_path='.'):

bench/commands/config.py

@@ -64,7 +64,7 @@ def config_http_timeout(seconds):
 
 
 @click.command('set-common-config')
-@click.option('configs', '-c', '--config', multiple=True, type=(unicode, unicode))
+@click.option('configs', '-c', '--config', multiple=True, type=(str, str))
 def set_common_config(configs):
 	import ast
 	from bench.config.common_site_config import update_config

bench/commands/git.py

@@ -29,5 +29,5 @@ def remote_urls():
 		if os.path.exists(os.path.join(repo_dir, '.git')):
 			remote = get_remote(app)
 			remote_url = subprocess.check_output(['git', 'config', '--get', 'remote.{}.url'.format(remote)], cwd=repo_dir).strip()
-			print "{app}	{remote_url}".format(app=app, remote_url=remote_url)
+			print("{app}	{remote_url}".format(app=app, remote_url=remote_url))
 

bench/commands/setup.py

@@ -75,13 +75,33 @@ def setup_env():
 	from bench.utils import setup_env
 	setup_env()
 
+@click.command('firewall')
+def setup_firewall():
+	"Setup firewall"
+	from bench.utils import run_playbook
+	click.confirm('Setting up the firewall will block all ports except 80, 443 and 22\n'
+		'Do you want to continue?',
+		abort=True)
+	run_playbook('production/setup_firewall.yml')
+
+@click.command('ssh-port')
+@click.argument('port')
+def set_ssh_port(port):
+	"Set SSH Port"
+	from bench.utils import run_playbook
+	click.confirm('This will change your SSH Port to {}\n'
+		'Do you want to continue?'.format(port),
+		abort=True)
+	run_playbook('production/change_ssh_port.yml', {"ssh_port": port})
+
 
 @click.command('lets-encrypt')
 @click.argument('site')
-def setup_letsencrypt(site):
+@click.option('--custom-domain')
+def setup_letsencrypt(site, custom_domain):
 	"Setup lets-encrypt for site"
 	from bench.config.lets_encrypt import setup_letsencrypt
-	setup_letsencrypt(site, bench_path='.')
+	setup_letsencrypt(site, custom_domain, bench_path='.')
 
 
 @click.command('procfile')
@@ -115,7 +135,7 @@ def add_domain(domain, site=None, ssl_certificate=None, ssl_certificate_key=None
 	from bench.config.site_config import add_domain
 
 	if not site:
-		print "Please specify site"
+		print("Please specify site")
 		sys.exit(1)
 
 	add_domain(site, domain, ssl_certificate, ssl_certificate_key, bench_path='.')
@@ -128,7 +148,7 @@ def remove_domain(domain, site=None):
 	from bench.config.site_config import remove_domain
 
 	if not site:
-		print "Please specify site"
+		print("Please specify site")
 		sys.exit(1)
 
 	remove_domain(site, domain, bench_path='.')
@@ -140,12 +160,12 @@ def sync_domains(domains, site=None):
 	from bench.config.site_config import sync_domains
 
 	if not site:
-		print "Please specify site"
+		print("Please specify site")
 		sys.exit(1)
 
 	domains = json.loads(domains)
 	if not isinstance(domains, list):
-		print "Domains should be a json list of strings or dictionaries"
+		print("Domains should be a json list of strings or dictionaries")
 		sys.exit(1)
 
 	changed = sync_domains(site, domains, bench_path='.')
@@ -170,3 +190,5 @@ setup.add_command(setup_fonts)
 setup.add_command(add_domain)
 setup.add_command(remove_domain)
 setup.add_command(sync_domains)
+setup.add_command(setup_firewall)
+setup.add_command(set_ssh_port)

bench/commands/update.py

@@ -15,10 +15,11 @@ from bench import patches
 @click.option('--requirements',is_flag=True, help="Update requirements")
 @click.option('--restart-supervisor',is_flag=True, help="restart supervisor processes after update")
 @click.option('--auto',is_flag=True)
-@click.option('--upgrade',is_flag=True)
+@click.option('--upgrade',is_flag=True, help="Required for major version updates")
 @click.option('--no-backup',is_flag=True)
 @click.option('--force',is_flag=True)
-def update(pull=False, patch=False, build=False, bench=False, auto=False, restart_supervisor=False, requirements=False, no_backup=False, upgrade=False, force=False):
+@click.option('--reset', is_flag=True, help="Hard resets git branch's to their new states overriding any changes and overriding rebase on pull")
+def update(pull=False, patch=False, build=False, bench=False, auto=False, restart_supervisor=False, requirements=False, no_backup=False, upgrade=False, force=False, reset=False):
 	"Update bench"
 
 	if not (pull or patch or build or bench or requirements):
@@ -39,28 +40,29 @@ def update(pull=False, patch=False, build=False, bench=False, auto=False, restar
 				'requirements': requirements,
 				'no-backup': no_backup,
 				'restart-supervisor': restart_supervisor,
-				'upgrade': upgrade
+				'upgrade': upgrade,
+				'reset':reset
 		})
 
 	if conf.get('release_bench'):
-		print 'Release bench, cannot update'
+		print('Release bench, cannot update')
 		sys.exit(1)
 
 	version_upgrade = is_version_upgrade()
 
 	if version_upgrade[0] and not upgrade:
-		print
+		print()
-		print
+		print()
-		print "This update will cause a major version change in Frappe/ERPNext from {0} to {1}.".format(*version_upgrade[1:])
+		print("This update will cause a major version change in Frappe/ERPNext from {0} to {1}.".format(*version_upgrade[1:]))
-		print "This would take significant time to migrate and might break custom apps. Please run `bench update --upgrade` to confirm."
+		print("This would take significant time to migrate and might break custom apps. Please run `bench update --upgrade` to confirm.")
-		print
+		print()
-		print "You can stay on the latest stable release by running `bench switch-to-master` or pin your bench to {0}".format(version_upgrade[1])
+		print("You can stay on the latest stable release by running `bench switch-to-master` or pin your bench to {0} by running `bench switch-to-v{0}`".format(version_upgrade[1]))
 		sys.exit(1)
 
-	_update(pull, patch, build, bench, auto, restart_supervisor, requirements, no_backup, upgrade, force=force)
+	_update(pull, patch, build, bench, auto, restart_supervisor, requirements, no_backup, upgrade, force=force, reset=reset)
 
 
-def _update(pull=False, patch=False, build=False, update_bench=False, auto=False, restart_supervisor=False, requirements=False, no_backup=False, upgrade=False, bench_path='.', force=False):
+def _update(pull=False, patch=False, build=False, update_bench=False, auto=False, restart_supervisor=False, requirements=False, no_backup=False, upgrade=False, bench_path='.', force=False, reset=False):
 	conf = get_config(bench_path=bench_path)
 	version_upgrade = is_version_upgrade(bench_path=bench_path)
 
@@ -73,20 +75,25 @@ def _update(pull=False, patch=False, build=False, update_bench=False, auto=False
 	before_update(bench_path=bench_path, requirements=requirements)
 
 	if pull:
-		pull_all_apps(bench_path=bench_path)
+		pull_all_apps(bench_path=bench_path, reset=reset)
 
 	if requirements:
+		print('Updating Python libraries...')
 		update_requirements(bench_path=bench_path)
 
 	if upgrade and (version_upgrade[0] or (not version_upgrade[0] and force)):
 		pre_upgrade(version_upgrade[1], version_upgrade[2], bench_path=bench_path)
 		import bench.utils, bench.app
+		print('Reloading bench...')
 		reload(bench.utils)
 		reload(bench.app)
 
 	if patch:
 		if not no_backup:
+			print('Backing up sites...')
 			backup_all_sites(bench_path=bench_path)
+
+		print('Patching sites...')
 		patch_sites(bench_path=bench_path)
 	if build:
 		build_assets(bench_path=bench_path)
@@ -95,9 +102,10 @@ def _update(pull=False, patch=False, build=False, update_bench=False, auto=False
 	if restart_supervisor or conf.get('restart_supervisor_on_update'):
 		restart_supervisor_processes(bench_path=bench_path)
 
-	print "_"*80
+	print("_"*80)
-	print "Bench: Open source installer + admin for Frappe and ERPNext (https://erpnext.com)"
+	print("Bench: Deployment tool for Frappe and ERPNext (https://erpnext.org).")
-	print
+	print("Open source depends on your contributions, so please contribute bug reports, patches, fixes or cash and be a part of the community")
+	print()
 
 
 @click.command('retry-upgrade')
@@ -110,7 +118,7 @@ def retry_upgrade(version):
 
 
 def restart_update(kwargs):
-	args = ['--'+k for k, v in kwargs.items() if v]
+	args = ['--'+k for k, v in list(kwargs.items()) if v]
 	os.execv(sys.argv[0], sys.argv[:2] + args)
 
 
@@ -122,8 +130,8 @@ def switch_to_branch(branch, apps, upgrade=False):
 	"Switch all apps to specified branch, or specify apps separated by space"
 	from bench.app import switch_to_branch
 	switch_to_branch(branch=branch, apps=list(apps), upgrade=upgrade)
-	print 'Switched to ' + branch 
+	print('Switched to ' + branch)
-	print 'Please run `bench update --patch` to be safe from any differences in database schema'
+	print('Please run `bench update --patch` to be safe from any differences in database schema')
 
 
 @click.command('switch-to-master')
@@ -132,9 +140,9 @@ def switch_to_master(upgrade=False):
 	"Switch frappe and erpnext to master branch"
 	from bench.app import switch_to_master
 	switch_to_master(upgrade=upgrade, apps=['frappe', 'erpnext'])
-	print
+	print()
-	print 'Switched to master'
+	print('Switched to master')
-	print 'Please run `bench update --patch` to be safe from any differences in database schema'
+	print('Please run `bench update --patch` to be safe from any differences in database schema')
 
 
 @click.command('switch-to-develop')
@@ -143,6 +151,8 @@ def switch_to_develop(upgrade=False):
 	"Switch frappe and erpnext to develop branch"
 	from bench.app import switch_to_develop
 	switch_to_develop(upgrade=upgrade, apps=['frappe', 'erpnext'])
-	print
+	print()
-	print 'Switched to develop'
+	print('Switched to develop')
-	print 'Please run `bench update --patch` to be safe from any differences in database schema'
+	print('Please run `bench update --patch` to be safe from any differences in database schema')
+
+

bench/commands/utils.py

@@ -12,11 +12,11 @@ def start(no_dev, concurrency):
 
 
 @click.command('restart')
-@click.option('--web-workers', is_flag=True, default=False)
+@click.option('--web', is_flag=True, default=False)
-def restart(web_workers):
+def restart(web):
 	"Restart supervisor processes"
 	from bench.utils import restart_supervisor_processes
-	restart_supervisor_processes(bench_path='.', web_workers=web_workers)
+	restart_supervisor_processes(bench_path='.', web_workers=web)
 
 
 @click.command('set-nginx-port')
@@ -86,10 +86,10 @@ def renew_lets_encrypt():
 @click.command()
 def shell(bench_path='.'):
 	if not os.environ.get('SHELL'):
-		print "Cannot get shell"
+		print("Cannot get shell")
 		sys.exit(1)
 	if not os.path.exists('sites'):
-		print "sites dir doesn't exist"
+		print("sites dir doesn't exist")
 		sys.exit(1)
 	env = copy.copy(os.environ)
 	env['PS1'] = '(' + os.path.basename(os.path.dirname(os.path.abspath(__file__))) + ')' + env.get('PS1', '')
@@ -104,7 +104,7 @@ def backup_site(site):
 	"backup site"
 	from bench.utils import get_sites, backup_site
 	if not site in get_sites(bench_path='.'):
-		print 'site not found'
+		print('site not found')
 		sys.exit(1)
 	backup_site(site, bench_path='.')
 
@@ -142,4 +142,4 @@ def disable_production():
 def bench_src():
 	"""Prints bench source folder path, which can be used as: cd `bench src` """
 	import bench
-	print os.path.dirname(bench.__path__[0])
+	print(os.path.dirname(bench.__path__[0]))

bench/config/common_site_config.py

@@ -1,4 +1,9 @@
-import os, multiprocessing, getpass, json, urlparse
+import os, multiprocessing, getpass, json
+
+try:
+	from urllib.parse import urlparse
+except ImportError:
+	from urlparse import urlparse
 
 default_config = {
 	'restart_supervisor_on_update': False,
@@ -81,19 +86,19 @@ def make_ports(bench_path):
 		bench_path = os.path.join(benches_path, folder)
 		if os.path.isdir(bench_path):
 			bench_config = get_config(bench_path)
-			for key in default_ports.keys():
+			for key in list(default_ports.keys()):
 				value = bench_config.get(key)
 
 				# extract port from redis url
 				if value and (key in ('redis_cache', 'redis_queue', 'redis_socketio')):
-					value = urlparse.urlparse(value).port
+					value = urlparse(value).port
 
 				if value:
 					existing_ports.setdefault(key, []).append(value)
 
 	# new port value = max of existing port value + 1
 	ports = {}
-	for key, value in default_ports.items():
+	for key, value in list(default_ports.items()):
 		existing_value = existing_ports.get(key, [])
 		if existing_value:
 			value = max(existing_value) + 1

bench/config/lets_encrypt.py

@@ -1,16 +1,32 @@
-import bench, os, click, errno, urllib
+import bench, os, click, errno
 from bench.utils import exec_cmd, CommandFailedError
-from bench.config.site_config import update_site_config
+from bench.config.site_config import update_site_config, remove_domain, get_domains
 from bench.config.nginx import make_nginx_conf
 from bench.config.production_setup import service
 from bench.config.common_site_config import get_config
 from crontab import CronTab
 
-def setup_letsencrypt(site, bench_path):
+try:
+	from urllib.request import urlretrieve
+except ImportError:
+	from urllib import urlretrieve
+
+def setup_letsencrypt(site, custom_domain, bench_path):
 
 	site_path = os.path.join(bench_path, "sites", site, "site_config.json")
 	if not os.path.exists(os.path.dirname(site_path)):
-		print "No site named "+site
+		print("No site named "+site)
+		return
+
+	if custom_domain:
+		domains = get_domains(site, bench_path)
+		for d in domains:
+			if (isinstance(d, dict) and d['domain']==custom_domain):
+				print("SSL for Domain {0} already exists".format(custom_domain))
+				return
+
+		if not custom_domain in domains:
+			print("No custom domain named {0} set for site".format(custom_domain))
 			return
 
 	click.confirm('Running this will stop the nginx service temporarily causing your sites to go offline\n'
@@ -18,48 +34,54 @@ def setup_letsencrypt(site, bench_path):
 		abort=True)
 
 	if not get_config(bench_path).get("dns_multitenant"):
-		print "You cannot setup SSL without DNS Multitenancy"
+		print("You cannot setup SSL without DNS Multitenancy")
 		return
 
-	create_config(site)
+	create_config(site, custom_domain)
-	run_certbot_and_setup_ssl(site, bench_path)
+	run_certbot_and_setup_ssl(site, custom_domain, bench_path)
 	setup_crontab()
 
 
-def create_config(site):
+def create_config(site, custom_domain):
-	config = bench.env.get_template('letsencrypt.cfg').render(domain=site)
+	config = bench.env.get_template('letsencrypt.cfg').render(domain=custom_domain or site)
-	config_path = '/etc/letsencrypt/configs/{site}.cfg'.format(site=site)
+	config_path = '/etc/letsencrypt/configs/{site}.cfg'.format(site=custom_domain or site)
 	create_dir_if_missing(config_path)
 
 	with open(config_path, 'w') as f:
 		f.write(config)
 
 
-def run_certbot_and_setup_ssl(site, bench_path):
+def run_certbot_and_setup_ssl(site, custom_domain, bench_path):
 	service('nginx', 'stop')
 	get_certbot()
 
 	try:
-		exec_cmd("{path} --config /etc/letsencrypt/configs/{site}.cfg certonly".format(path=get_certbot_path(), site=site))
+		exec_cmd("{path} --config /etc/letsencrypt/configs/{site}.cfg certonly".format(path=get_certbot_path(), site=custom_domain or site))
 	except CommandFailedError:
 		service('nginx', 'start')
-		print "There was a problem trying to setup SSL for your site"
+		print("There was a problem trying to setup SSL for your site")
 		return
 
-	ssl_path = "/etc/letsencrypt/live/{site}/".format(site=site)
+	ssl_path = "/etc/letsencrypt/live/{site}/".format(site=custom_domain or site)
-
 	ssl_config = { "ssl_certificate": os.path.join(ssl_path, "fullchain.pem"),
 					"ssl_certificate_key": os.path.join(ssl_path, "privkey.pem") }
 
+	if custom_domain:
+		remove_domain(site, custom_domain, bench_path)
+		domains = get_domains(site, bench_path)
+		ssl_config['domain'] = custom_domain
+		domains.append(ssl_config)
+		update_site_config(site, { "domains": domains }, bench_path=bench_path)
+	else:
 		update_site_config(site, ssl_config, bench_path=bench_path)
-	make_nginx_conf(bench_path)
 
+	make_nginx_conf(bench_path)
 	service('nginx', 'start')
 
 
 def setup_crontab():
 	job_command = 'sudo service nginx stop && /opt/certbot-auto renew && sudo service nginx start'
-	user_crontab = CronTab(user=True)
+	user_crontab = CronTab()
 	if job_command not in str(user_crontab):
 		job  = user_crontab.new(command=job_command, comment="Renew lets-encrypt every month")
 		job.every().month()
@@ -77,8 +99,8 @@ def get_certbot():
 	create_dir_if_missing(certbot_path)
 
 	if not os.path.isfile(certbot_path):
-		urllib.urlretrieve ("https://dl.eff.org/certbot-auto", certbot_path)
+		urlretrieve ("https://dl.eff.org/certbot-auto", certbot_path)
-		os.chmod(certbot_path, 0744)
+		os.chmod(certbot_path, 0o744)
 
 
 def get_certbot_path():

bench/config/nginx.py

@@ -25,7 +25,7 @@ def make_nginx_conf(bench_path, yes=False):
 		"error_pages": get_error_pages(),
 		"allow_rate_limiting": allow_rate_limiting,
 		# for nginx map variable
-		"random_string": "".join(random.choice(string.ascii_lowercase) for i in xrange(7))
+		"random_string": "".join(random.choice(string.ascii_lowercase) for i in range(7))
 	}
 
 	if allow_rate_limiting:
@@ -159,7 +159,7 @@ def get_sites_with_config(bench_path):
 		if dns_multitenant and site_config.get('domains'):
 			for domain in site_config.get('domains'):
 				# domain can be a string or a dict with 'domain', 'ssl_certificate', 'ssl_certificate_key'
-				if isinstance(domain, basestring):
+				if isinstance(domain, str) or isinstance(domain, unicode):
 					domain = { 'domain': domain }
 
 				domain['name'] = site

bench/config/production_setup.py

@@ -66,7 +66,7 @@ def service(service, option):
 			exec_cmd(service_manager_command)
 
 		else:
-			raise Exception, 'No service manager found'
+			raise Exception('No service manager found')
 
 def get_supervisor_confdir():
 	possiblities = ('/etc/supervisor/conf.d', '/etc/supervisor.d/', '/etc/supervisord/conf.d', '/etc/supervisord.d')

bench/config/redis.py

@@ -1,13 +1,18 @@
 from .common_site_config import get_config
-import re, os, subprocess, urlparse, semantic_version
+import re, os, subprocess, semantic_version
 import bench
 
+try:
+	from urllib.parse import urlparse
+except ImportError:
+	from urlparse import urlparse
+
 def generate_config(bench_path):
 	config = get_config(bench_path)
 
 	ports = {}
 	for key in ('redis_cache', 'redis_queue', 'redis_socketio'):
-		ports[key] = urlparse.urlparse(config[key]).port
+		ports[key] = urlparse(config[key]).port
 
 	write_redis_config(
 		template_name='redis_queue.conf',
@@ -29,7 +34,7 @@ def generate_config(bench_path):
 	write_redis_config(
 		template_name='redis_cache.conf',
 		context={
-			"maxmemory": config.get('cache_maxmemory', '50'),
+			"maxmemory": config.get('cache_maxmemory', get_max_redis_memory()),
 			"port": ports['redis_cache'],
 			"redis_version": get_redis_version(),
 		},
@@ -60,3 +65,14 @@ def get_redis_version():
 
 	version = semantic_version.Version(version[0], partial=True)
 	return float('{major}.{minor}'.format(major=version.major, minor=version.minor))
+
+def get_max_redis_memory():
+	import psutil
+	
+	total_virtual_mem = psutil.virtual_memory().total/(pow(1024, 2))
+	max_memory = int(total_virtual_mem * 0.05) # Max memory for redis is 5% of virtual memory
+
+	if max_memory < 50:
+		return 50
+	else:
+		return max_memory

bench/config/site_config.py

@@ -43,7 +43,7 @@ def add_domain(site, domain, ssl_certificate, ssl_certificate_key, bench_path='.
 	domains = get_domains(site, bench_path)
 	for d in domains:
 		if (isinstance(d, dict) and d['domain']==domain) or d==domain:
-			print "Domain {0} already exists".format(domain)
+			print("Domain {0} already exists".format(domain))
 			return
 
 	if ssl_certificate_key and ssl_certificate:
@@ -75,7 +75,7 @@ def sync_domains(site, domains, bench_path='.'):
 		changed = True
 
 	else:
-		for d in existing_domains.values():
+		for d in list(existing_domains.values()):
 			if d != new_domains.get(d['domain']):
 				changed = True
 				break
@@ -92,7 +92,7 @@ def get_domains(site, bench_path='.'):
 def get_domains_dict(domains):
 	domains_dict = defaultdict(dict)
 	for d in domains:
-		if isinstance(d, basestring):
+		if isinstance(d, str):
 			domains_dict[d] = { 'domain': d }
 
 		elif isinstance(d, dict):

bench/patches/patches.txt

@@ -2,3 +2,5 @@ bench.patches.v3.deprecate_old_config
 bench.patches.v3.celery_to_rq
 bench.patches.v3.redis_bind_ip
 bench.patches.v4.update_node
+bench.patches.v4.update_socketio
+

bench/patches/v4/update_node.py

@@ -12,9 +12,10 @@ def execute(bench_path):
 	else:
 		click.echo('''
 		No node executable was found on your machine.
-		Please install node 5.x before running "bench update".
+		Please install latest node version before running "bench update". For installation instructions
-		Installation instructions for CentOS and Ubuntu can be found on the following link,
+		please refer "Debian and Ubuntu based Linux distributions" section or "Enterprise Linux and
-		"https://www.metachris.com/2015/10/how-to-install-nodejs-5-on-centos-and-ubuntu/"
+		Fedora" section depending upon your OS on the following link,
+		"https://nodejs.org/en/download/package-manager/"
 		''')
 		sys.exit(1)
 
@@ -22,8 +23,10 @@ def execute(bench_path):
 
 	if node_ver < expected_node_ver:
 		click.echo('''
-		Please update node version to 5.x before running "bench update".
+		Please update node to latest version before running "bench update".
-		Installation instructions for CentOS and Ubuntu can be found on the following link,
+		Please install latest node version before running "bench update". For installation instructions
-		"https://www.metachris.com/2015/10/how-to-install-nodejs-5-on-centos-and-ubuntu/"
+		please refer "Debian and Ubuntu based Linux distributions" section or "Enterprise Linux and
+		Fedora" section depending upon your OS on the following link,
+		"https://nodejs.org/en/download/package-manager/"
 		''')
 		sys.exit(1)

bench/patches/v4/update_socketio.py

@@ -0,0 +1,4 @@
+import subprocess
+
+def execute(bench_path):
+	subprocess.check_output(['npm', 'install', 'socket.io'])

bench/release.py

@@ -29,7 +29,7 @@ def release(bench_path, app, bump_type, develop='develop', master='master',
 def validate(bench_path):
 	config = get_config(bench_path)
 	if not config.get('release_bench'):
-		print 'bench not configured to release'
+		print('bench not configured to release')
 		sys.exit(1)
 
 	global github_username, github_password
@@ -38,7 +38,7 @@ def validate(bench_path):
 	github_password = config.get('github_password')
 
 	if not github_username:
-		github_username = raw_input('Username: ')
+		github_username = input('Username: ')
 
 	if not github_password:
 		github_password = getpass.getpass()
@@ -54,12 +54,12 @@ def bump(bench_path, app, bump_type, develop, master, remote, owner, repo_name=N
 	message = get_release_message(repo_path, develop=develop, master=master, remote=remote)
 
 	if not message:
-		print 'No commits to release'
+		print('No commits to release')
 		return
 
-	print
+	print()
-	print message
+	print(message)
-	print
+	print()
 
 	click.confirm('Do you want to continue?', abort=True)
 
@@ -68,7 +68,7 @@ def bump(bench_path, app, bump_type, develop, master, remote, owner, repo_name=N
 	tag_name = create_release(repo_path, new_version, develop=develop, master=master)
 	push_release(repo_path, develop=develop, master=master, remote=remote)
 	create_github_release(repo_path, tag_name, message, remote=remote, owner=owner, repo_name=repo_name)
-	print 'Released {tag} for {repo_path}'.format(tag=tag_name, repo_path=repo_path)
+	print('Released {tag} for {repo_path}'.format(tag=tag_name, repo_path=repo_path))
 
 def update_branches_and_check_for_changelog(repo_path, develop='develop', master='master', remote='upstream'):
 
@@ -81,7 +81,7 @@ def update_branches_and_check_for_changelog(repo_path, develop='develop', master
 	check_for_unmerged_changelog(repo_path)
 
 def update_branch(repo_path, branch, remote):
-	print "updating local branch of", repo_path, 'using', remote + '/' + branch
+	print("updating local branch of", repo_path, 'using', remote + '/' + branch)
 
 	repo = git.Repo(repo_path)
 	g = repo.git
@@ -95,7 +95,7 @@ def check_for_unmerged_changelog(repo_path):
 		raise Exception("Unmerged change log! in " + repo_path)
 
 def get_release_message(repo_path, develop='develop', master='master', remote='upstream'):
-	print 'getting release message for', repo_path, 'comparing', master, '...', develop
+	print('getting release message for', repo_path, 'comparing', master, '...', develop)
 
 	repo = git.Repo(repo_path)
 	g = repo.git
@@ -109,7 +109,7 @@ def bump_repo(repo_path, bump_type, develop='develop', master='master'):
 	current_version = get_current_version(repo_path)
 	new_version = get_bumped_version(current_version, bump_type)
 
-	print 'bumping version from', current_version, 'to', new_version
+	print('bumping version from', current_version, 'to', new_version)
 
 	set_version(repo_path, new_version)
 	return new_version
@@ -154,7 +154,7 @@ def get_bumped_version(version, bump_type):
 		else:
 			v.prerelease[1] = str(int(v.prerelease[1]) + 1)
 
-	return unicode(v)
+	return str(v)
 
 def set_version(repo_path, version):
 	set_filename_version(os.path.join(repo_path, os.path.basename(repo_path),'__init__.py'), version, '__version__')
@@ -192,7 +192,7 @@ def set_filename_version(filename, version_number, pattern):
 		f.write(contents)
 
 def commit_changes(repo_path, new_version):
-	print 'committing version change to', repo_path
+	print('committing version change to', repo_path)
 
 	repo = git.Repo(repo_path)
 	app_name = os.path.basename(repo_path)
@@ -200,13 +200,13 @@ def commit_changes(repo_path, new_version):
 	repo.index.commit('bumped to version {}'.format(new_version))
 
 def create_release(repo_path, new_version, develop='develop', master='master'):
-	print 'creating release for version', new_version
+	print('creating release for version', new_version)
 	repo = git.Repo(repo_path)
 	g = repo.git
 	g.checkout(master)
 	try:
 		g.merge(develop, '--no-ff')
-	except git.exc.GitCommandError, e:
+	except git.exc.GitCommandError as e:
 		handle_merge_error(e, source=develop, target=master)
 
 	tag_name = 'v' + new_version
@@ -215,29 +215,29 @@ def create_release(repo_path, new_version, develop='develop', master='master'):
 
 	try:
 		g.merge(master)
-	except git.exc.GitCommandError, e:
+	except git.exc.GitCommandError as e:
 		handle_merge_error(e, source=master, target=develop)
 
 	if develop != 'develop':
-		print 'merging master into develop'
+		print('merging master into develop')
 		g.checkout('develop')
 		try:
 			g.merge(master)
-		except git.exc.GitCommandError, e:
+		except git.exc.GitCommandError as e:
 			handle_merge_error(e, source=master, target='develop')
 
 	return tag_name
 
 def handle_merge_error(e, source, target):
-	print '-'*80
+	print('-'*80)
-	print 'Error when merging {source} into {target}'.format(source=source, target=target)
+	print('Error when merging {source} into {target}'.format(source=source, target=target))
-	print e
+	print(e)
-	print 'You can open a new terminal, try to manually resolve the conflict/error and continue'
+	print('You can open a new terminal, try to manually resolve the conflict/error and continue')
-	print '-'*80
+	print('-'*80)
 	click.confirm('Have you manually resolved the error?', abort=True)
 
 def push_release(repo_path, develop='develop', master='master', remote='upstream'):
-	print 'pushing branches', master, develop, 'of', repo_path
+	print('pushing branches', master, develop, 'of', repo_path)
 	repo = git.Repo(repo_path)
 	g = repo.git
 	args = [
@@ -246,22 +246,22 @@ def push_release(repo_path, develop='develop', master='master', remote='upstream
 	]
 
 	if develop != 'develop':
-		print 'pushing develop branch of', repo_path
+		print('pushing develop branch of', repo_path)
 		args.append('develop:develop')
 
 	args.append('--tags')
 
-	print g.push(remote, *args)
+	print(g.push(remote, *args))
 
 def create_github_release(repo_path, tag_name, message, remote='upstream', owner='frappe', repo_name=None,
 		gh_username=None, gh_password=None):
 
-	print 'creating release on github'
+	print('creating release on github')
 
 	global github_username, github_password
 	if not (gh_username and gh_password):
 		if not (github_username and github_password):
-			raise Exception, "No credentials"
+			raise Exception("No credentials")
 		gh_username = github_username
 		gh_password = github_password
 
@@ -274,7 +274,7 @@ def create_github_release(repo_path, tag_name, message, remote='upstream', owner
 		'draft': False,
 		'prerelease': False
 	}
-	for i in xrange(3):
+	for i in range(3):
 		try:
 			r = requests.post('https://api.github.com/repos/{owner}/{repo_name}/releases'.format(
 				owner=owner, repo_name=repo_name),
@@ -282,12 +282,12 @@ def create_github_release(repo_path, tag_name, message, remote='upstream', owner
 			r.raise_for_status()
 			break
 		except requests.exceptions.HTTPError:
-			print 'request failed, retrying....'
+			print('request failed, retrying....')
 			sleep(3*i + 1)
 			if i !=2:
 				continue
 			else:
-				print r.json()
+				print(r.json())
 				raise
 	return r
 

bench/tests/test_init.py

@@ -1,4 +1,4 @@
-from __future__ import unicode_literals
+
 import unittest
 import json, os, shutil, subprocess
 import bench
@@ -178,7 +178,7 @@ class TestBenchInit(unittest.TestCase):
 		try:
 			subprocess.check_output(drop_site_cmd, cwd=bench_path)
 		except subprocess.CalledProcessError as err:
-			print err.output
+			print(err.output)
 
 		if not archived_sites_path:
 			archived_sites_path = os.path.join(bench_path, 'archived_sites')
@@ -234,8 +234,8 @@ class TestBenchInit(unittest.TestCase):
 
 		config = self.load_json(common_site_config_path)
 
-		for key, value in expected_config.items():
+		for key, value in list(expected_config.items()):
-			self.assertEquals(config.get(key), value)
+			self.assertEqual(config.get(key), value)
 
 	def assert_exists(self, *args):
 		self.assertTrue(os.path.exists(os.path.join(*args)))

bench/tests/test_setup_production.py

@@ -1,4 +1,4 @@
-from __future__ import unicode_literals
+
 from bench.tests import test_init
 from bench.config.production_setup import setup_production, get_supervisor_confdir, disable_production
 import bench.utils
@@ -36,23 +36,6 @@ class TestSetupProduction(test_init.TestBenchInit):
 			bench_path = os.path.join(os.path.abspath(self.benches_path), bench_name)
 			disable_production(bench_path)
 
-	def test_setup_production_v6(self):
-		bench_name = 'test-bench-v6'
-		self.test_init(bench_name, frappe_branch='v6.x.x')
-
-		user = getpass.getuser()
-
-		bench_path = os.path.join(os.path.abspath(self.benches_path), bench_name)
-		setup_production(user, bench_path)
-
-		self.assert_nginx_config(bench_name)
-		self.assert_nginx_process()
-
-		self.assert_supervisor_config(bench_name, use_rq=False)
-		self.assert_supervisor_process(bench_name, use_rq=False)
-
-		disable_production(bench_path)
-
 	def test_disable_production(self):
 		bench_name = 'test-disable-prod'
 		self.test_init(bench_name, frappe_branch='master')
@@ -76,7 +59,7 @@ class TestSetupProduction(test_init.TestBenchInit):
 		self.assertTrue(os.path.exists(conf_dest))
 
 		# symlink matches
-		self.assertEquals(os.path.realpath(conf_dest), conf_src)
+		self.assertEqual(os.path.realpath(conf_dest), conf_src)
 
 		# file content
 		with open(conf_src, "r") as f:
@@ -113,7 +96,7 @@ class TestSetupProduction(test_init.TestBenchInit):
 		self.assertTrue(os.path.exists(conf_dest))
 
 		# symlink matches
-		self.assertEquals(os.path.realpath(conf_dest), conf_src)
+		self.assertEqual(os.path.realpath(conf_dest), conf_src)
 
 		# file content
 		with open(conf_src, "r") as f:

bench/utils.py

@@ -16,8 +16,8 @@ folders_in_bench = ('apps', 'sites', 'config', 'logs', 'config/pids')
 def get_frappe(bench_path='.'):
 	frappe = get_env_cmd('frappe', bench_path=bench_path)
 	if not os.path.exists(frappe):
-		print 'frappe app is not installed. Run the following command to install frappe'
+		print('frappe app is not installed. Run the following command to install frappe')
-		print 'bench get-app https://github.com/frappe/frappe.git'
+		print('bench get-app https://github.com/frappe/frappe.git')
 	return frappe
 
 def get_env_cmd(cmd, bench_path='.'):
@@ -33,7 +33,7 @@ def init(path, apps_path=None, no_procfile=False, no_backups=False,
 	from bench.patches import set_all_patches_executed
 
 	if os.path.exists(path):
-		print 'Directory {} already exists!'.format(path)
+		print('Directory {} already exists!'.format(path))
 		raise Exception("Site directory already exists")
 		# sys.exit(1)
 
@@ -76,17 +76,17 @@ def init(path, apps_path=None, no_procfile=False, no_backups=False,
 
 def clone_apps_from(bench_path, clone_from):
 	from .app import install_app
-	print 'Copying apps from {0}...'.format(clone_from)
+	print('Copying apps from {0}...'.format(clone_from))
 	subprocess.check_output(['cp', '-R', os.path.join(clone_from, 'apps'), bench_path])
 
-	print 'Copying node_modules from {0}...'.format(clone_from)
+	print('Copying node_modules from {0}...'.format(clone_from))
 	subprocess.check_output(['cp', '-R', os.path.join(clone_from, 'node_modules'), bench_path])
 
 	def setup_app(app):
 		# run git reset --hard in each branch, pull latest updates and install_app
 		app_path = os.path.join(bench_path, 'apps', app)
 		if os.path.exists(os.path.join(app_path, '.git')):
-			print 'Cleaning up {0}'.format(app)
+			print('Cleaning up {0}'.format(app))
 
 			# remove .egg-ino
 			subprocess.check_output(['rm', '-rf', app + '.egg-info'], cwd=app_path)
@@ -116,6 +116,9 @@ def exec_cmd(cmd, cwd='.'):
 		stderr = stdout = subprocess.PIPE
 	else:
 		stderr = stdout = None
+
+	logger.info(cmd)
+
 	p = subprocess.Popen(cmd, cwd=cwd, shell=True, stdout=stdout, stderr=stderr)
 
 	if async:
@@ -137,7 +140,6 @@ def setup_socketio(bench_path='.'):
 	exec_cmd("npm install socket.io redis express superagent cookie", cwd=bench_path)
 
 def new_site(site, mariadb_root_password=None, admin_password=None, bench_path='.'):
-	import hashlib
 	logger.info('creating new site {}'.format(site))
 	mariadb_root_password_fragment = '--root_password {}'.format(mariadb_root_password) if mariadb_root_password else ''
 	admin_password_fragment = '--admin_password {}'.format(admin_password) if admin_password else ''
@@ -238,7 +240,7 @@ def setup_sudoers(user):
 			f.write('\n#includedir /etc/sudoers.d\n')
 
 		if set_permissions:
-			os.chmod('/etc/sudoers', 0440)
+			os.chmod('/etc/sudoers', 0o440)
 
 	sudoers_file = '/etc/sudoers.d/frappe'
 
@@ -255,7 +257,7 @@ def setup_sudoers(user):
 	with open(sudoers_file, 'w') as f:
 		f.write(frappe_sudoers.encode('utf-8'))
 
-	os.chmod(sudoers_file, 0440)
+	os.chmod(sudoers_file, 0o440)
 
 def setup_logging(bench_path='.'):
 	if os.path.exists(os.path.join(bench_path, 'logs')):
@@ -308,7 +310,12 @@ def get_git_version():
 
 def check_git_for_shallow_clone():
 	from .config.common_site_config import get_config
-	if get_config(".").get('release_bench'):
+	config = get_config('.')
+
+	if config.get('release_bench'):
+		return False
+
+	if not config.get('shallow_clone'):
 		return False
 
 	git_version = get_git_version()
@@ -319,9 +326,9 @@ def check_git_for_shallow_clone():
 def get_cmd_output(cmd, cwd='.'):
 	try:
 		return subprocess.check_output(cmd, cwd=cwd, shell=True, stderr=open(os.devnull, 'wb')).strip()
-	except subprocess.CalledProcessError, e:
+	except subprocess.CalledProcessError as e:
 		if e.output:
-			print e.output
+			print(e.output)
 		raise
 
 def restart_supervisor_processes(bench_path='.', web_workers=False):
@@ -432,7 +439,7 @@ def drop_privileges(uid_name='nobody', gid_name='nogroup'):
 	os.setuid(running_uid)
 
 	# Ensure a very conservative umask
-	os.umask(022)
+	os.umask(0o22)
 
 def fix_prod_setup_perms(bench_path='.', frappe_user=None):
 	from .config.common_site_config import get_config
@@ -451,7 +458,7 @@ def fix_prod_setup_perms(bench_path='.', frappe_user=None):
 		frappe_user = get_config(bench_path).get('frappe_user')
 
 	if not frappe_user:
-		print "frappe user not set"
+		print("frappe user not set")
 		sys.exit(1)
 
 	for path in files:
@@ -463,14 +470,14 @@ def fix_prod_setup_perms(bench_path='.', frappe_user=None):
 def fix_file_perms():
 	for dir_path, dirs, files in os.walk('.'):
 		for _dir in dirs:
-			os.chmod(os.path.join(dir_path, _dir), 0755)
+			os.chmod(os.path.join(dir_path, _dir), 0o755)
 		for _file in files:
-			os.chmod(os.path.join(dir_path, _file), 0644)
+			os.chmod(os.path.join(dir_path, _file), 0o644)
 	bin_dir = './env/bin'
 	if os.path.exists(bin_dir):
 		for _file in os.listdir(bin_dir):
 			if not _file.startswith('activate'):
-				os.chmod(os.path.join(bin_dir, _file), 0755)
+				os.chmod(os.path.join(bin_dir, _file), 0o755)
 
 def get_current_frappe_version(bench_path='.'):
 	from .app import get_current_frappe_version as fv
@@ -498,7 +505,8 @@ def run_frappe_cmd(*args, **kwargs):
 		return_code = p.wait()
 
 	if return_code > 0:
-		raise CommandFailedError(args)
+		sys.exit(return_code)
+		#raise CommandFailedError(args)
 
 def get_frappe_cmd_output(*args, **kwargs):
 	bench_path = kwargs.get('bench_path', '.')
@@ -531,8 +539,8 @@ def post_upgrade(from_ver, to_ver, bench_path='.'):
 	from .config.supervisor import generate_supervisor_config
 	from .config.nginx import make_nginx_conf
 	conf = get_config(bench_path=bench_path)
-	print "-"*80
+	print("-"*80)
-	print "Your bench was upgraded to version {0}".format(to_ver)
+	print("Your bench was upgraded to version {0}".format(to_ver))
 
 	if conf.get('restart_supervisor_on_update'):
 		redis.generate_config(bench_path=bench_path)
@@ -545,17 +553,17 @@ def post_upgrade(from_ver, to_ver, bench_path='.'):
 		if from_ver <= 5 and to_ver == 6:
 			setup_socketio(bench_path=bench_path)
 
-		print "As you have setup your bench for production, you will have to reload configuration for nginx and supervisor"
+		print("As you have setup your bench for production, you will have to reload configuration for nginx and supervisor")
-		print "To complete the migration, please run the following commands"
+		print("To complete the migration, please run the following commands")
-		print
+		print()
-		print "sudo service nginx restart"
+		print("sudo service nginx restart")
-		print "sudo supervisorctl reload"
+		print("sudo supervisorctl reload")
 
 def update_translations_p(args):
 	try:
 		update_translations(*args)
 	except requests.exceptions.HTTPError:
-		print 'Download failed for', args[0], args[1]
+		print('Download failed for', args[0], args[1])
 
 def download_translations_p():
 	pool = multiprocessing.Pool(4)
@@ -592,7 +600,7 @@ def update_translations(app, lang):
 				f.write(chunk)
 				f.flush()
 
-	print 'downloaded for', app, lang
+	print('downloaded for', app, lang)
 	
 def download_chart_of_accounts():
 	charts_dir = os.path.join('apps', "erpnext", "erpnext", 'accounts', 'chart_of_accounts', "submitted")
@@ -649,18 +657,18 @@ def validate_pillow_dependencies(bench_path, requirements):
 		distro = platform.linux_distribution()
 		distro_name = distro[0].lower()
 		if "centos" in distro_name or "fedora" in distro_name:
-			print "Please install these dependencies using the command:"
+			print("Please install these dependencies using the command:")
-			print "sudo yum install libtiff-devel libjpeg-devel libzip-devel freetype-devel lcms2-devel libwebp-devel tcl-devel tk-devel"
+			print("sudo yum install libtiff-devel libjpeg-devel libzip-devel freetype-devel lcms2-devel libwebp-devel tcl-devel tk-devel")
 
 			raise
 
 		elif "ubuntu" in distro_name or "elementary os" in distro_name or "debian" in distro_name:
-			print "Please install these dependencies using the command:"
+			print("Please install these dependencies using the command:")
 
 			if "ubuntu" in distro_name and distro[1]=="12.04":
-				print "sudo apt-get install -y libtiff4-dev libjpeg8-dev zlib1g-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.5-dev tk8.5-dev python-tk"
+				print("sudo apt-get install -y libtiff4-dev libjpeg8-dev zlib1g-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.5-dev tk8.5-dev python-tk")
 			else:
-				print "sudo apt-get install -y libtiff5-dev libjpeg8-dev zlib1g-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python-tk"
+				print("sudo apt-get install -y libtiff5-dev libjpeg8-dev zlib1g-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python-tk")
 
 			raise
 
@@ -686,9 +694,18 @@ def set_git_remote_url(git_url, bench_path='.'):
 	app = git_url.rsplit('/', 1)[1].rsplit('.', 1)[0]
 
 	if app not in bench.app.get_apps(bench_path):
-		print "No app named {0}".format(app)
+		print("No app named {0}".format(app))
 		sys.exit(1)
 
 	app_dir = bench.app.get_repo_dir(app, bench_path=bench_path)
 	if os.path.exists(os.path.join(app_dir, '.git')):
 		exec_cmd("git remote set-url upstream {}".format(git_url), cwd=app_dir)
+
+def run_playbook(playbook_name, extra_vars=None):
+	if not find_executable('ansible'):
+		print("Ansible is needed to run this command, please install it using 'pip install ansible'")
+		sys.exit(1)
+	args = ['ansible-playbook', '-c', 'local', playbook_name]
+	if extra_vars:
+		args.extend(['-e', json.dumps(extra_vars)])
+	subprocess.check_call(args, cwd=os.path.join(os.path.dirname(bench.__path__[0]), 'playbooks'))

playbooks/develop/centos.yml

@@ -5,7 +5,6 @@
     bench_path: "/home/{{ ansible_user_id }}/frappe-bench"
     mysql_config_template: "templates/simple_mariadb_config.cnf"
     mysql_conf_dir: /etc/my.cnf.d/
-    wkhtmltopdf_version: 0.12.2.1
 
   tasks:
 
@@ -41,19 +40,24 @@
       - libtiff-devel
       - tcl-devel
       - tk-devel
+
+      # Python LDAP
+      - openldap-devel
+
     become: yes
     become_user: root
 
-  - name: Get nodejs 6.x bash script
+  - name: Import Node source RPM key
-    get_url:
+    rpm_key:
-      url: 'https://rpm.nodesource.com/setup_6.x'
+      key: https://rpm.nodesource.com/pub/el/NODESOURCE-GPG-SIGNING-KEY-EL
-      dest: '/tmp/setup_6.x'
+      state: present
-      mode: 0644
     become: yes
     become_user: root
 
-  - name: Run nodejs 6.x bash script
+  - name: Add Node Repo
-    command: /bin/bash /tmp/setup_6.x
+    yum:
+      name: 'https://rpm.nodesource.com/pub_6.x/el/{{ ansible_distribution_major_version }}/{{ ansible_architecture }}/nodesource-release-el{{ ansible_distribution_major_version }}-1.noarch.rpm'
+      state: present
     become: yes
     become_user: root
 
@@ -68,7 +72,7 @@
   - include: includes/mariadb_centos.yml
 
   # install WKHTMLtoPDF
-  - include: includes/wkhtmltopdf_centos.yml
+  - include: includes/wkhtmltopdf.yml
 
   # setup MariaDB
   - include: includes/setup_mariadb.yml

playbooks/develop/debian.yml

@@ -5,7 +5,6 @@
     bench_path: "/home/{{ ansible_user_id }}/frappe-bench"
     mysql_config_template: "templates/simple_mariadb_config.cnf"
     mysql_conf_dir: /etc/mysql/conf.d/
-    wkhtmltopdf_version: 0.12.2.1
 
   tasks:
 
@@ -49,6 +48,13 @@
       - libwebp-dev
       - python-tk
 
+      # Ensure apt-transport-https
+      - apt-transport-https
+
+      # Python LDAP
+      - libsasl2-dev
+      - libldap2-dev
+
     become: yes
     become_user: root
 
@@ -74,16 +80,19 @@
     become: yes
     become_user: root
 
-  - name: Get nodejs 6.x bash script
+  - name: Add apt key for node repo
-    get_url:
+    apt_key:
-      url: 'https://deb.nodesource.com/setup_6.x'
+      url: https://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0x1655A0AB68576280
-      dest: '/tmp/setup_6.x'
+      id: "68576280"
-      mode: 0644
+      state: present
     become: yes
     become_user: root
 
-  - name: Run nodejs bash script
+  - name: Add repo
-    command: /bin/bash /tmp/setup_6.x
+    apt_repository:
+      repo: "deb [arch=amd64,i386] https://deb.nodesource.com/node_6.x {{ ansible_distribution_release }} main"
+      state: present
+      update_cache: yes
     become: yes
     become_user: root
 
@@ -100,7 +109,7 @@
   - include: includes/mariadb_debian.yml
 
   # install WKHTMLtoPDF
-  - include: includes/wkhtmltopdf_ubuntu_debian.yml
+  - include: includes/wkhtmltopdf.yml
 
   # setup MariaDB
   - include: includes/setup_mariadb.yml

playbooks/develop/includes/wkhtmltopdf.yml

@@ -0,0 +1,19 @@
+---
+  - name: download wkthmltox linux
+    get_url: url=http://download.gna.org/wkhtmltopdf/0.12/0.12.3/wkhtmltox-0.12.3_linux-generic-{{ "amd64" if ansible_architecture == "x86_64" else "i386"}}.tar.xz dest=/tmp/wkhtmltox.tar.xz
+
+  - name: Creates directory
+    file: path=/tmp/wkhtmltox state=directory
+
+  - name: unarchive wkhtmltopdf
+    unarchive: src=/tmp/wkhtmltox.tar.xz dest=/tmp
+
+  - name: copy to /usr/local/bin
+    copy: src="/tmp/wkhtmltox/bin/wkhtmltopdf" dest="/usr/local/bin/wkhtmltopdf"
+    become: true
+    become_user: root
+
+  - name: make wkhtmltopdf executable
+    file: path=/usr/local/bin/wkhtmltopdf mode="o+x"
+    become: true
+    become_user: root

playbooks/develop/includes/wkhtmltopdf_centos.yml

@@ -1,12 +0,0 @@
----
-  - name: Download wkhtmltopdf
-    get_url:
-      url=http://download.gna.org/wkhtmltopdf/0.12/{{ wkhtmltopdf_version }}/wkhtmltox-{{ wkhtmltopdf_version }}_linux-centos{{ ansible_lsb.major_release }}-{{ "amd64" if ansible_architecture == "x86_64" else "i386"}}.rpm
-      dest="/tmp/"
-    become: yes
-    become_user: root
-
-  - name: Install wkhtmltopdf deb
-    yum: name=/tmp/wkhtmltox-{{ wkhtmltopdf_version }}_linux-centos{{ ansible_lsb.major_release }}-{{ "amd64" if ansible_architecture == "x86_64" else "i386"}}.rpm state=present
-    become: yes
-    become_user: root

playbooks/develop/includes/wkhtmltopdf_ubuntu_debian.yml

@@ -1,28 +0,0 @@
----
-  - name: Download wkhtmltopdf
-    get_url:
-      url=http://download.gna.org/wkhtmltopdf/0.12/{{ wkhtmltopdf_version }}/wkhtmltox-{{ wkhtmltopdf_version }}_linux-{{ ansible_distribution_release }}-{{ "amd64" if ansible_architecture == "x86_64" else "i386"}}.deb
-      dest="/tmp/"
-    become: yes
-    become_user: root
-    when: ansible_distribution_release in ("jessie", "precise", "trusty", "wheezy")
-
-  - name: Install wkhtmltopdf deb
-    apt: deb=/tmp/wkhtmltox-{{ wkhtmltopdf_version }}_linux-{{ ansible_distribution_release }}-{{ "amd64" if ansible_architecture == "x86_64" else "i386"}}.deb
-    become: yes
-    become_user: root
-    when: ansible_distribution_release in ("jessie", "precise", "trusty", "wheezy")
-
-  - name: Download wkhtmltopdf for Ubuntu Wily
-    get_url:
-      url=http://download.gna.org/wkhtmltopdf/0.12/{{ wkhtmltopdf_version }}/wkhtmltox-{{ wkhtmltopdf_version }}_linux-trusty-{{ "amd64" if ansible_architecture == "x86_64" else "i386"}}.deb
-      dest="/tmp/"
-    become: yes
-    become_user: root
-    when: ansible_distribution_release=="wily"
-
-  - name: Install wkhtmltopdf deb for Ubuntu Wily
-    apt: deb=/tmp/wkhtmltox-{{ wkhtmltopdf_version }}_linux-trusty-{{ "amd64" if ansible_architecture == "x86_64" else "i386"}}.deb
-    become: yes
-    become_user: root
-    when: ansible_distribution_release=="wily"

playbooks/develop/ubuntu.yml

@@ -5,7 +5,6 @@
     bench_path: "/home/{{ ansible_user_id }}/frappe-bench"
     mysql_config_template: "templates/simple_mariadb_config.cnf"
     mysql_conf_dir: /etc/mysql/conf.d/
-    wkhtmltopdf_version: 0.12.2.1
 
   tasks:
 
@@ -34,6 +33,13 @@
       - libwebp-dev
       - python-tk
 
+      # Ensure apt-transport-https
+      - apt-transport-https
+
+      # Python LDAP
+      - libsasl2-dev
+      - libldap2-dev
+      
     become: yes
     become_user: root
 
@@ -57,16 +63,19 @@
     become: yes
     become_user: root
 
-  - name: Get nodejs 6.x bash script
+  - name: Add apt key for node repo
-    get_url:
+    apt_key:
-      url: 'https://deb.nodesource.com/setup_6.x'
+      url: https://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0x1655A0AB68576280
-      dest: '/tmp/setup_6.x'
+      id: "68576280"
-      mode: 0644
+      state: present
     become: yes
     become_user: root
 
-  - name: Run nodejs bash script
+  - name: Add repo
-    command: /bin/bash /tmp/setup_6.x
+    apt_repository:
+      repo: "deb [arch=amd64,i386] https://deb.nodesource.com/node_6.x {{ ansible_distribution_release }} main"
+      state: present
+    register: node_repo
     become: yes
     become_user: root
 
@@ -83,7 +92,7 @@
   - include: includes/mariadb_ubuntu.yml
 
   # install WKHTMLtoPDF
-  - include: includes/wkhtmltopdf_ubuntu_debian.yml
+  - include: includes/wkhtmltopdf.yml
 
   # setup MariaDB
   - include: includes/setup_mariadb.yml

playbooks/install.py

@@ -30,7 +30,7 @@ def install_bench(args):
 		})
 
 	if not success:
-		print 'Could not install pre-requisites. Please check for errors or install them manually.'
+		print('Could not install pre-requisites. Please check for errors or install them manually.')
 		return
 
 	# secure pip installation
@@ -75,7 +75,7 @@ def install_bench(args):
 		if args.production:
 			args.user = 'frappe'
 
-		elif os.environ.has_key('SUDO_USER'):
+		elif 'SUDO_USER' in os.environ:
 			args.user = os.environ['SUDO_USER']
 
 		else:
@@ -122,9 +122,9 @@ def check_distribution_compatibility():
 		if float(dist_version) in supported_dists[dist_name]:
 			return
 
-	print "Sorry, the installer doesn't support {0} {1}. Aborting installation!".format(dist_name, dist_version)
+	print("Sorry, the installer doesn't support {0} {1}. Aborting installation!".format(dist_name, dist_version))
 	if dist_name in supported_dists:
-		print "Install on {0} {1} instead".format(dist_name, supported_dists[dist_name][-1])
+		print("Install on {0} {1} instead".format(dist_name, supported_dists[dist_name][-1]))
 	sys.exit(1)
 
 def get_distribution_info():
@@ -142,7 +142,7 @@ def install_python27():
 	if version == (2, 7):
 		return
 
-	print 'Installing Python 2.7'
+	print('Installing Python 2.7')
 
 	# install python 2.7
 	success = run_os_command({
@@ -210,9 +210,9 @@ def clone_bench_repo(args):
 def run_os_command(command_map):
 	'''command_map is a dictionary of {'executable': command}. For ex. {'apt-get': 'sudo apt-get install -y python2.7'} '''
 	success = True
-	for executable, commands in command_map.items():
+	for executable, commands in list(command_map.items()):
 		if find_executable(executable):
-			if isinstance(commands, basestring):
+			if isinstance(commands, str):
 				commands = [commands]
 
 			for command in commands:
@@ -245,7 +245,7 @@ def get_passwords(ignore_prompt=False):
 
 			# admin password
 			if not admin_password:
-				admin_password = getpass.unix_getpass(prompt='Please enter Administrator password: ')
+				admin_password = getpass.unix_getpass(prompt='Please enter the default Administrator user password: ')
 				conf_admin_passswd = getpass.unix_getpass(prompt='Re-enter Administrator password: ')
 
 				if admin_password != conf_admin_passswd:
@@ -256,16 +256,25 @@ def get_passwords(ignore_prompt=False):
 	else:
 		mysql_root_password = admin_password = 'travis'
 
-	return {
+	passwords = {
 		'mysql_root_password': mysql_root_password,
 		'admin_password': admin_password
 	}
 
+	if not ignore_prompt:
+		passwords_file_path = os.path.join(os.path.expanduser('~'), 'passwords.txt')
+		with open(passwords_file_path, 'w') as f:
+			json.dump(passwords, f, indent=1)
+
+		print('Passwords saved at ~/passwords.txt')
+
+	return passwords
+
 def get_extra_vars_json(extra_args):
 	# We need to pass production as extra_vars to the playbook to execute conditionals in the
 	# playbook. Extra variables can passed as json or key=value pair. Here, we will use JSON.
 	json_path = os.path.join('/tmp', 'extra_vars.json')
-	extra_vars = dict(extra_args.items())
+	extra_vars = dict(list(extra_args.items()))
 	with open(json_path, mode='w') as j:
 		json.dump(extra_vars, j, indent=1, sort_keys=True)
 
@@ -340,3 +349,5 @@ if __name__ == '__main__':
 	args = parse_commandline_args()
 
 	install_bench(args)
+
+	print('''Frappe/ERPNext has been successfully installed!''')

playbooks/production/change_ssh_port.yml

@@ -0,0 +1,14 @@
+- name: Change ssh port
+  gather_facts: false
+  hosts: localhost
+  user: root
+  tasks:
+    - name: change sshd config
+      lineinfile: >
+        dest=/etc/ssh/sshd_config
+        regexp="^Port"
+        line="Port {{ ssh_port }}"
+        state=present
+
+    - name: restart ssh
+      service: name=sshd state=reloaded

playbooks/production/includes/setup_prod_env.yml

@@ -78,6 +78,28 @@
     become: yes
     become_user: root
 
+  ######################################################
+  # Set InnoDB Buffer Pool size to half of total RAM
+  - name: Set InnoDB buffer pool
+    lineinfile: >
+      dest=/etc/my.cnf.d/frappe.cnf
+      regexp="^\[mysqld\]$"
+      line="[mysqld]\ninnodb_buffer_pool_size={{ (ansible_memtotal_mb/2)|round|int }}M"
+      state=present
+    when: ansible_distribution == 'CentOS'
+    become: yes
+    become_user: root
+
+  - name: Set InnoDB buffer pool
+    lineinfile: >
+      dest=/etc/mysql/conf.d/frappe.cnf
+      regexp="^\[mysqld\]$"
+      line="[mysqld]\ninnodb_buffer_pool_size={{ (ansible_memtotal_mb/2)|round|int }}M"
+      state=present
+    when: ansible_distribution == 'Ubuntu' or ansible_distribution == 'Debian'
+    become: yes
+    become_user: root
+
   ####################################################
   # Enable nginx, mysql, redis and supevisord services
   - name: Enable nginx, mysql, and redis

playbooks/production/setup_firewall.yml

@@ -0,0 +1,43 @@
+- name: Setup Firewall
+  user: root
+  hosts: localhost
+
+  tasks:
+      # For CentOS
+    - name: Install firewalld
+      yum: name=firewalld state=present
+      when: ansible_distribution == 'CentOS'
+
+    - name: Enable Firewall
+      service: name=firewalld state=started enabled=yes
+      when: ansible_distribution == 'CentOS'
+
+    - name: Add firewall rules
+      firewalld: port={{ item }}/tcp permanent=true state=enabled
+      with_items:
+        - 80
+        - 443
+        - 22
+      when: ansible_distribution == 'CentOS'
+
+    - name: Restart Firewall
+      service: name=firewalld state=restarted enabled=yes
+      when: ansible_distribution == 'CentOS'
+
+      # For Ubuntu / Debian
+    - name: Install ufw
+      apt: name=ufw state=present
+      when: ansible_distribution == 'Ubuntu' or ansible_distribution == 'Debian'
+
+    - name: Enable Firewall
+      ufw: state=enabled policy=deny
+      when: ansible_distribution == 'Ubuntu' or ansible_distribution == 'Debian'
+
+    - name: Add firewall rules
+      ufw: rule=allow proto=tcp port={{ item }}
+      with_items:
+        - 80
+        - 443
+        - 22
+      when: ansible_distribution == 'Ubuntu' or ansible_distribution == 'Debian'
+

vm/ansible/roles/wkhtmltopdf/defaults/main.yml

@@ -1 +1 @@
-wkhtmltopdf_version: 0.12.2.1
+wkhtmltopdf_version: 0.12.3

vm/ansible/roles/wkhtmltopdf/tasks/main.yml

@@ -8,10 +8,6 @@
     - xorg-x11-fonts-Type1
   when: ansible_os_family == 'RedHat'
 
-- name: Install wkhtmltopdf rpm
-  yum: name=http://download.gna.org/wkhtmltopdf/0.12/{{ wkhtmltopdf_version }}/wkhtmltox-{{ wkhtmltopdf_version }}_linux-centos{{ ansible_distribution_major_version  }}-{{ "amd64" if ansible_architecture == "x86_64" else "i386"}}.rpm
-  when: ansible_os_family == 'RedHat'
-
 - name: install base fonts
   apt: name={{ item }} state=present
   with_items:
@@ -21,12 +17,11 @@
     - xfonts-base
   when: ansible_os_family == 'Debian'
 
-- name: Download wkhtmltopdf
+- name: download wkthmltox linux
-  get_url:
+  get_url: url=http://download.gna.org/wkhtmltopdf/0.12/0.12.3/wkhtmltox-0.12.3_linux-generic-{{ "amd64" if ansible_architecture == "x86_64" else "i386"}}.tar.xz dest=/tmp/wkhtmltox.tar.xz
-    url=http://download.gna.org/wkhtmltopdf/0.12/{{ wkhtmltopdf_version }}/wkhtmltox-{{ wkhtmltopdf_version }}_linux-{{ ansible_distribution_release }}-{{ "amd64" if ansible_architecture == "x86_64" else "i386"}}.deb
-    dest="/tmp/"
-  when: ansible_os_family == 'Debian'
 
-- name: Install wkhtmltopdf deb
+- name: unarchive wkhtmltopdf
-  apt: deb=/tmp/wkhtmltox-{{ wkhtmltopdf_version }}_linux-{{ ansible_distribution_release }}-{{ "amd64" if ansible_architecture == "x86_64" else "i386"}}.deb
+  unarchive: src=/tmp/wkhtmltox.tar.xz dest=/tmp/wkhtmltox
-  when: ansible_os_family == 'Debian'
+
+- name: move to /usr/local/bin
+  command: creates="/usr/local/bin/wkhtmltopdf" mv /tmp/wkhtmltox/bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf