accounting/frappe_docker
2
0
Fork 0
mirror of https://github.com/frappe/frappe_docker.git synced 2024-11-08 22:31:07 +00:00
Code Issues Releases Activity

Merge commit '3e1f44b4b019af9152697e754ea62cbc890f2246' into feature-enumerate-oackages

Browse Source
This commit is contained in:
Davide Bortolami 2020-03-09 23:31:55 +00:00
commit 03a060f757
4 changed files with 193 additions and 125 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -7,3 +7,4 @@
sites
development
!development/README.md

14
.vscode/extensions.json vendored Normal file
View File

@ -0,0 +1,14 @@
{
// See https://go.microsoft.com/fwlink/?LinkId=827846 to learn about workspace recommendations.
// Extension identifier format: ${publisher}.${name}. Example: vscode.csharp
// List of extensions which should be recommended for users of this workspace.
"recommendations": [
"ms-vscode-remote.remote-containers",
"ms-python.python"
],
// List of extensions recommended by VS Code that should not be recommended for users of this workspace.
"unwantedRecommendations": [
]
}

159
README.md
View File

@ -8,7 +8,7 @@ This docker installation takes care of the following:
* Setting up all the system requirements: eg. MariaDB, Node, Redis.
* [OPTIONAL] Configuring networking for remote access and setting up LetsEncrypt
For docker based development refer this [README](development/README.md)
For docker based development refer to this [README](development/README.md)
## Deployment
@ -44,23 +44,22 @@ To get started, copy the existing `env-example` file to `.env` inside the `insta
- `VERSION=edge`
- In this case, `edge` corresponds to `develop`. To setup any other version, you may use the branch name or version specific tags. (eg. version-12, v11.1.15, v11)
- `MYSQL_ROOT_PASSWORD=admin`
- Bootstraps a MariaDB container with this value set as the root password. If a managed MariaDB instance is to be used, there is no need to set the password here.
- Bootstraps a MariaDB container with this value set as the root password. If a managed MariaDB instance is used, there is no need to set the password here.
- `MARIADB_HOST=mariadb`
- Sets the hostname to `mariadb`. This is required if the database is managed with the containerized MariaDB instance.
- In case of a separately managed database setup, set the value to the database's hostname/IP/domain.
- Sets the hostname to `mariadb`. This is required if the database is managed by the containerized MariaDB instance.
- In case of a separately managed database setups, set the value to the database's hostname/IP/domain.
- `SITES=site1.domain.com,site2.domain.com`
- List of sites that are part of the deployment "bench". Each site is separated by a comma(,).
- If LetsEncrypt is being setup, make sure that the DNS for all the site's domains are pointing to the current instance.
- List of sites that are part of the deployment "bench" Each site is separated by a comma(,).
- If LetsEncrypt is being setup, make sure that the DNS for all the site's domains correctly point to the current instance.
- `LETSENCRYPT_EMAIL=your.email@your.domain.com`
- Email for LetsEncrypt expiry notification. This is only required if you are setting up LetsEncrypt.
### Local deployment
### Local deployment for testing
For trying out locally or to develop apps using ERPNext ReST API port 80 must be published.
First start the containers and then run an additional command to publish port of *-nginx container.
For trying out locally or to develop apps using ERPNext REST API port 80 must be published.
The first command will start the containers; the second command will publish the port of the *-nginx container.
To start and publish Frappe/ERPNext services as local api, run the following commands:
For Erpnext:
@ -98,13 +97,14 @@ docker-compose \
--project-directory installation run --publish 80:80 -d frappe-nginx
```
Make sure to replace `<project-name>` with any desired name you wish to set for the project.
Make sure to replace `<project-name>` with the desired name you wish to set for the project.
Note:
- This command adds an additional container for frappe-nginx with published ports.
- The local deployment is for testing and REST API development purpose only.
- The site names are limited to patterns matching \*.localhost by default
- Additional site name patterns can be added to /etc/hosts of desired container or host
Notes:
- The local deployment is for testing and REST API development purpose only
- A complete development environment is available [here](Development/README.md)
- The site names are limited to patterns matching \*.localhost by default
- Additional site name patterns can be added by editing /etc/hosts of your host machine
### Deployment for production
@ -112,7 +112,7 @@ Note:
Letsencrypt Nginx Proxy Companion can optionally be setup to provide SSL. This is recommended for instances accessed over the internet.
Your DNS will need to be configured correctly in order for Letsencrypt to verify your domain.
Your DNS will need to be configured correctly for Letsencrypt to verify your domain.
To setup the proxy companion, run the following commands:
@ -124,8 +124,7 @@ cp .env.sample .env
./start.sh
```
For more details, see: https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion
Letsencrypt Nginx Proxy Companion works by automatically proxying to containers with the `VIRTUAL_HOST` environmental variable.
For more details, see the [Letsencrypt Nginx Proxy Companion github repo](https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion). Letsencrypt Nginx Proxy Companion github repo works by automatically proxying to containers with the `VIRTUAL_HOST` environmental variable.
#### Start Frappe/ERPNext Services
@ -145,7 +144,7 @@ Note: use `docker-compose-frappe.yml` in case you need only Frappe without ERPNe
### Docker containers
This repository contains the following docker-compose files each one containing the described images:
This repository contains the following docker-compose files, each one containing the described images:
* docker-compose-common.yml
* redis-cache
* volume: redis-cache-vol
@ -175,7 +174,7 @@ This repository contains the following docker-compose files each one containing
* frappe-worker-long: background runner for long-running jobs
* frappe-schedule
* docker-compose-networks.yml: this yml define the network to communicate with *Letsencrypt Nginx Proxy Companion*.
* docker-compose-networks.yml: this yaml define the network to communicate with *Letsencrypt Nginx Proxy Companion*.
### Site operations
@ -184,8 +183,8 @@ This repository contains the following docker-compose files each one containing
Note:
- Wait for the mariadb service to start before trying to create a new site.
- If new site creation fails, retry after the mariadb container is up and running.
- Wait for the MariaDB service to start before trying to create a new site.
- If new site creation fails, retry after the MariaDB container is up and running.
- If you're using a managed database instance, make sure that the database is running before setting up a new site.
- Use `.env` file or environment variables instead of passing secrets as command arguments.
@ -196,25 +195,25 @@ docker exec -it \
-e "DB_ROOT_USER=$DB_ROOT_USER" \
-e "MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD" \
-e "ADMIN_PASSWORD=$ADMIN_PASSWORD" \
-e "INSTALL_APPS='erpnext'" \
-e "INSTALL_APPS=erpnext" \
<project-name>_erpnext-python_1 docker-entrypoint.sh new
```
Environment Variables needed:
- `SITE_NAME`: name of the new site to create.
- `DB_ROOT_USER`: MariaDB Root user. The user that can create databases.
- `MYSQL_ROOT_PASSWORD`: In case of mariadb docker container use the one set in `MYSQL_ROOT_PASSWORD` in previous steps. In case of managed database use appropriate password.
- `ADMIN_PASSWORD`: set the administrator password for new site.
- `INSTALL_APPS='erpnext'`: available only in erpnext-worker and erpnext containers (or other containers with custom apps). Installs ERPNext (and/or the specified apps, comma-delinieated) on this new site.
- `FORCE=1`: is optional variable which force installs the same site.
- `DB_ROOT_USER`: MariaDB Root user.
- `MYSQL_ROOT_PASSWORD`: In case of the MariaDB docker container use the one set in `MYSQL_ROOT_PASSWORD` in previous steps. In case of a managed database use the appropriate password.
- `ADMIN_PASSWORD`: set the administrator password for the new site.
- `INSTALL_APPS=erpnext`: available only in erpnext-worker and erpnext containers (or other containers with custom apps). Installs ERPNext (and/or the specified apps, comma-delinieated) on this new site.
- `FORCE=1`: optional variable which force installation of the same site.
#### Backup Sites
Environment Variables
- `SITES` is list of sites separated by (:) colon to migrate. e.g. `SITES=site1.domain.com` or `SITES=site1.domain.com:site2.domain.com` By default all sites in bench will be backed up.
- `WITH_FILES` if set to 1, it will backup user uploaded files for the sites.
- `SITES` is list of sites separated by `:` colon to migrate. e.g. `SITES=site1.domain.com` or `SITES=site1.domain.com:site2.domain.com` By default all sites in bench will be backed up.
- `WITH_FILES` if set to 1, it will backup user-uploaded files.
```sh
docker exec -it \
@ -223,7 +222,7 @@ docker exec -it \
<project-name>_erpnext-python_1 docker-entrypoint.sh backup
```
Backup will be available in the `sites` mounted volume.
The backup will be available in the `sites` mounted volume.
#### Updating and Migrating Sites
@ -254,68 +253,69 @@ docker exec -it \
### Custom apps
> For the sake of example, we'll be using a place holder called `[custom]`, and we'll be building off the edge image.
To add your own Frappe/ERPNext apps to the image, we'll need to create a custom image with the help of a unique wrapper script
To add your own apps to the image, we'll need to create a custom image with the help of a special wrapper script
> For the sake of simplicity, in this example, we'll be using a place holder called `[custom]`, and we'll be building off the edge image.
1. Create two folders called `[custom]-worker` and `[custom]-nginx` in the `build` folder.
Create two directories called `[custom]-worker` and `[custom]-nginx` in the `build` directory.
```bash
cd frappe_docker
mkdir ./build/[custom]-worker ./build/[custom]-nginx
```
```shell
cd frappe_docker
mkdir ./build/[custom]-worker ./build/[custom]-nginx
```
2. Create a `Dockerfile` in `./build/[custom]-worker` with the following content:
Create a `Dockerfile` in `./build/[custom]-worker` with the following content:
```Dockerfile
FROM frappe/erpnext-worker:edge
```Dockerfile
FROM frappe/erpnext-worker:edge
RUN install_app [custom] https://github.com/[username]/[custom] [branch]
# Only add the branch if you are using a specific tag or branch.
```
RUN install_app [custom] https://github.com/[username]/[custom] [branch]
# Only add the branch if you are using a specific tag or branch.
```
3. Create a `Dockerfile` in `./build/[custom]-nginx` with the following content:
Create a `Dockerfile` in `./build/[custom]-nginx` with the following content:
```Dockerfile
FROM bitnami/node:12-prod
```Dockerfile
FROM bitnami/node:12-prod
COPY build/[custom]-nginx/install_app.sh /install_app
COPY build/[custom]-nginx/install_app.sh /install_app
RUN /install_app [custom] https://github.com/[username]/[custom]
RUN /install_app [custom] https://github.com/[username]/[custom]
FROM frappe/erpnext-nginx:edge
FROM frappe/erpnext-nginx:edge
COPY --from=0 /home/frappe/frappe-bench/sites/ /var/www/html/
COPY --from=0 /rsync /rsync
RUN echo -n "\n[custom]" >> /var/www/html/apps.txt
COPY --from=0 /home/frappe/frappe-bench/sites/ /var/www/html/
COPY --from=0 /rsync /rsync
RUN echo -n "\n[custom]" >> /var/www/html/apps.txt
VOLUME [ "/assets" ]
VOLUME [ "/assets" ]
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["nginx", "-g", "daemon off;"]
```
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["nginx", "-g", "daemon off;"]
```
4. Copy over the `install_app.sh` file from `./build/erpnext-nginx`
Copy over the `install_app.sh` file from `./build/erpnext-nginx`
```bash
cp ./build/erpnext-nginx/install.sh ./build/[custom]-nginx
```
```shell
cp ./build/erpnext-nginx/install.sh ./build/[custom]-nginx
```
5. Open up `./installation/docker-compose-custom.yml` and replace all instances of `[app]` with the name of your app.
Open up `./installation/docker-compose-custom.yml` and replace all instances of `[app]` with the name of your app.
```bash
sed -i "s#\[app\]#[custom]#" ./installation/docker-compose-custom.yml
```
```shell
sed -i "s#\[app\]#[custom]#" ./installation/docker-compose-custom.yml
```
6. Install like usuall, except that when you set the `INSTALL_APPS` variable set it to `erpnext,[custom]`.
Install like usual, except that when you set the `INSTALL_APPS` variable to `erpnext,[custom]`.
## Troubleshoot
1. Remove containers and volumes, and clear redis cache:
### Failed migration after image upgrade
This can be used when existing images are upgraded and migration fails.
Issue: After upgrade of the containers, the automatic migration fails.
Solution: Remove containers and volumes, and clear redis cache:
```
```shell
# change to repo root
cd $HOME/frappe_docker
@ -350,9 +350,10 @@ docker-compose \
--project-directory installation up -d
```
2. Clear redis cache using `docker exec` command:
### ValueError: There exists an active worker named XXX already
Issue: You have the following error during container restart
In case of following error during container restarts:
```
frappe-worker-short_1 | Traceback (most recent call last):
@ -367,7 +368,7 @@ frappe-worker-short_1 | raise ValueError(msg.format(self.name))
frappe-worker-short_1 | ValueError: There exists an active worker named '8dfe5c234085.10.short' already
```
Use commands :
Solution: Clear redis cache using `docker exec` command (take care of replacing `<project-name>` accordingly):
```sh
# Clear the cache which is causing problem.
@ -377,4 +378,14 @@ docker exec -it <project-name>_redis-queue_1 redis-cli FLUSHALL
docker exec -it <project-name>_redis-socketio_1 redis-cli FLUSHALL
```
Note: Environment variables from `.env` file located at current working directory will be used.
Note: Environment variables from `.env` file located at the current working directory will be used.
## Development
This repository includes a complete setup to develop with Frappe/ERPNext and Bench, Including the following features:
- VSCode containers integration
- VSCode Python debugger
- Pre-configured Docker containers for an easy start
A complete Readme is available in [development/README.md](development/README.md)

144
development/README.md
View File

@ -6,54 +6,53 @@
- docker-compose
- user added to docker group
### Bootstrap Containers for development
## Bootstrap Containers for development
Clone and change to frappe_docker directory
Clone and change directory to frappe_docker directory
```shell
git clone https://github.com/frappe/frappe_docker.git
cd frappe_docker
```
#### Manually start containers
## Use VSCode Remote Containers extension
In case VS Code is not used follow these steps.
For most people getting started with Frappe development, the best solution is to use [ VSCode Remote - Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers).
```shell
docker-compose -f .devcontainer/docker-compose.yml up -d
```
VSCode should automatically inquiry you to install the required extensions, that can also be installed manually as follows:
Enter the bench container at location `/workspace/development`
- Install Remote - Containers for VSCode
- through command line `code --install-extension ms-vscode-remote.remote-containers`
- clicking on the button at the following link: [Remote - Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)
- searching for extension `ms-vscode-remote.remote-containers`
- Install Python for VSCode
- through command line `code --install-extension ms-python.python`
- clicking on the button at the following link: [install](https://marketplace.visualstudio.com/items?itemName=ms-python.python)
- searching for extension `ms-python.python`
```shell
docker exec -e "TERM=xterm-256color" -w /workspace/development -it devcontainer_frappe_1 bash
```
After the extensions are installed, you can:
#### Use VS Code Remote Containers extension
Follow these in case VS Code is used. Do not start containers manually.
- Install Remote Development Pack / Remote Containers extension.
- Install VS Code Python extension.
- Open frappe_docker in VS Code.
- From Command Palette (Ctrl + Shift + P) Execute Remote Containers : Reopen in Container.
### Setup Docker specific bench environment
- Open frappe_docker folder in VS Code.
- `code .`
- Launch the command, from Command Palette (Ctrl + Shift + P) `Execute Remote Containers : Reopen in Container`. You can also click in the bottom left corner to access the remote container menu.
Notes:
- `development` directory is ignored by git. It is mounted and available in container. Create all your benches inside this directory.
- Execute these commands from container.
- nvm with node v12 and v10 is installed. Check with `nvm ls`. Node v12 is default.
- The `development` directory is ignored by git. It is mounted and available inside the container. Create all your benches (installations of bench, the tool that manages frappe) inside this directory.
- nvm with node v12 and v10 is installed. Check with `nvm ls`. Node v12 is used by default.
#### Setup first bench
### Setup first bench
Run the following commands in the terminal inside the container. You might need to create a new terminal in VSCode.
```shell
bench init --skip-redis-config-generation --frappe-branch version-12 frappe-bench
cd frappe-bench
```
#### Set hosts
### Setup hosts
We need to tell bench to use the right containers instead of localhost. Run the following commands inside the container:
```shell
bench set-mariadb-host mariadb
@ -62,52 +61,95 @@ bench set-redis-queue-host redis-queue:6379
bench set-redis-socketio-host redis-socketio:6379
```
#### Changes related to bench start / honcho / Procfile
### Edit Honcho's Procfile
- honcho/Procfile starts processes required for development.
- By default Procfile has 3 redis processes that it starts. Comment (`#`) or remove these lines and then run `bench start`.
- Another option is to run following command :
Honcho is the tool used by Bench to manage all the processes Frappe requires. Usually, these all run in localhost, but in this case, we have external containers for Redis. For this reason, we have to stop Honcho from trying to start Redis processes.
Open the Procfile file and remove the three lines containing the configuration from Redis, either by editing manually the file.
```shell
honcho start \
web \
socketio \
watch \
schedule \
worker_short \
worker_long \
worker_default
code Procfile
```
#### Changes related to MariaDB
or running the following command:
```shell
sed -i '/redis/d' ./Procfile
```
Notes:
### Create a new site with bench
- `bench new-site` command creates a user in mariadb with container IP as host
- After rebuilding container there is a chance that new bench container will not be able to access mariadb
- `'db_name'@'%'` needs to be set in mariadb and permission to the site database be given to the user
- Replace `db_name` and `db_password` from site's `site_config.json`
- MariaDB root password is 123
You can create a new site with the following command:
Enter mariadb shell
```shell
bench new-site sitename
```
for example:
```shell
bench new-site localhost
```
The command will ask the MariaDB root password. The default root password is `123`.
This will create a new site and a `localhost` directory under `frappe-bench/sites`.
Your website will now be accessible on [localhost on port 8000](http://locahost:8000)
### Fixing MariaDB issues after rebuilding the container
The `bench new-site` command creates a user in MariaDB with container IP as host, for this reason after rebuilding the container there is a chance that you will not be able to access MariaDB correctly with the previous configuration
The parameter `'db_name'@'%'` needs to be set in MariaDB and permission to the site database suitably assigned to the user.
This step has to be repeated for all sites available under the current bench.
Example shows the queries to be executed for site `localhost`
Open sites/localhost/site_config.json:
```shell
code sites/localhost/site_config.json
```
and take note of the parameters `db_name` and `db_password`.
Enter MariaDB Interactive shell:
```shell
mysql -uroot -p123 -hmariadb
```
Execute following queries
Execute following queries replacing `db_name` and `db_password` with the values found in site_config.json.
```sql
UPDATE mysql.user SET Host = '%' where User = 'db_name'; FLUSH PRIVILEGES;
SET PASSWORD FOR 'db_name'@'%' = PASSWORD('db_password'); FLUSH PRIVILEGES;
GRANT ALL PRIVILEGES ON `db_name`.* TO 'db_name'@'%'; FLUSH PRIVILEGES;
EXIT;
```
## Manually start containers
In case you don't use VSCode, you may start the containers manually with the following command:
```shell
docker-compose -f .devcontainer/docker-compose.yml up -d
```
And enter the interactive shell for the development container with the following command:
```shell
docker exec -e "TERM=xterm-256color" -w /workspace/development -it devcontainer_frappe_1 bash
```
### Visual Studio Code Python Debugging
- Install VS Code Python Extension once in remote container
- Reload VS Code
- Do not start `web` process with honcho
To enable Python debugging inside Visual Studio Code, you must first install the `ms-python.python` extension inside the container.
- Click on the extension icon inside VSCode
- Search `ms-python.python`
- Click on `Install on Dev Container: Frappe Bench`
- Click on 'Reload'
We need to start bench separately through the VSCode debugger. For this reason, **instead** of running `bench start` you should run the following command inside the frappe-bench directory:
```shell
honcho start \
@ -119,4 +161,4 @@ honcho start \
worker_default
```
- On debugger tab, Connect debugger. This will start the web process with debugger connected
This command starts all processes with the exception of Redis (which is already running in separate container) and the `web` process. The latter can can finally be started from the debugger tab of VSCode.