accounting/frappe_docker
2
0
Fork 0
mirror of https://github.com/frappe/frappe_docker.git synced 2024-11-09 23:00:56 +00:00
Code Issues Releases Activity

refactor(frappe-docker): code cleanup and fixes (#63)

Browse Source 
* testing replacement for dbench

* fixed perm issue and added docker control

* fixes and tests

fixed travis test, and perm issue

* auto add permissions

* bench setup anything-else should work now

* removed adding site names to host (its pointless)

* moved redis configuration folder into conf/

* added a docker down feature to dbench and changed some of the docs

* hopefully should fix the args issue

* Condensed Dockerfile

Updated to latest node, and uses python-pip, also condensed a lot the Dockerfile.

* Removed extra line

* Removed MAINTAINER in favor of LABEL

* all installs are in one apt command

* Switch bench to dbench, added -s feature of dbench to bench. updated
Docker file with sudo and vim, and setup travis to run with said changes

* dockerfile: clean up (#52)

* Condensed Dockerfile

Updated to latest node, and uses python-pip, also condensed a lot the Dockerfile.

* fixed travis, and cleaned up dbench a little

* Oops...

* oops, but only for travis, this runs fine

* Travis!

* added help to dbench, and updated readme

* changes some of the wording of the help message

* dbench: fix init issues (#55)

./dbench init would lead to 'missing argument: PATH' issue (#54), and could not find Procfile_docker and site-config.

Signed-off-by: Chinmay Pai <chinmaydpai@gmail.com>

* bench setup anything-else should work now

* Clean cache to get a smaller image (#57)

Please include this commit to decrease the size of the docker image, the "rm -rf /var/lib/apt/lists/*" it is not usefull if it is not used in the same stage of the apt-get install RUN, similar with pip cache.

References:
 - https://linux.die.net/man/8/apt-get
 - https://stackoverflow.com/questions/9510474/removing-pips-cache
 - https://semaphoreci.com/blog/2016/12/13/lightweight-docker-images-in-5-steps.html

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file;

(b) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

Signed-off-by: William Moreno Reyes <williamjmorenor@gmail.com>

* removed adding site names to host (its pointless)

* moved redis configuration folder into conf/

* added a docker down feature to dbench and changed some of the docs

* Changed the readme significantly

 Please enter the commit message for your changes. Lines starting

* changed test.py to avoid naming errors

* moving some things around for less lines

* some more slight name changes

* added a .gitignore

* added the frappe folder again

* added a .dockerignore and fixed the dockerfile up a bit

* Updating travis' docker and compose

 This is the commit message #3:

* bringing chown in dbench back

* some refractoring

* implementing codacy sugestions

* moving some things around

* Updating dbench in some areas

* some random changes and fixes

* dockerfile refactoring

* Refractoring, reducing layers

* removing the py script that took too many lines to do one simple thing

* Fixed up the readme

* some refractoring

* moving to debian slim and refractoring

* adding some logging

* Updated travis to ubuntu xenial, should fix the curl issue

* manually building curl, cause we cant use  the latest version...

* maybe we dont need it

* changing the url to curl

* trying this instead

* lets just let it pollute the stream

* please pollute!
This commit is contained in:
chabad360 2019-04-05 15:50:30 -04:00 committed by Chinmay Pai
parent 687ef7cecc
commit 33ad0c4fe9
19 changed files with 173 additions and 213 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
.dockerignore Normal file
View File

@ -0,0 +1,10 @@
# frappe_docker .dockerignore file
conf/*
.travis.yml
test.sh
README.md
LICENSE.md
.gitignore
dbench
docker-*.yml

1
.gitignore vendored Normal file
View File

@ -0,0 +1 @@
*.code-workspace

29
.travis.yml
View File

@ -1,22 +1,29 @@
language: python
python:
- "2.7"
dist: xenial
env:
- DOCKER_COMPOSE_VERSION: 1.22.0
- DOCKER_COMPOSE_VERSION: 1.23.1
services:
- docker
before_install:
- curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
- sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
- sudo apt-get update
- sudo apt-get -y -o Dpkg::Options::="--force-confnew" install docker-ce
- sudo rm /usr/local/bin/docker-compose
- curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
- chmod +x docker-compose
- sudo mv docker-compose /usr/local/bin
install:
- chmod ugo+x ./dbench
- chmod ugo+x ./test.sh
- ./dbench setup docker
- ./dbench init frappe-bench
- ./dbench new-site site1.local
- sudo ./dbench setup hosts
- pip install --upgrade virtualenv
- virtualenv -p python3 testenv
- source testenv/bin/activate
- pip install requests
- ./dbench setup hosts
script:
- docker-compose ps | grep -i frappe
@ -24,5 +31,5 @@ script:
- docker-compose ps | grep -i redis-queue
- docker-compose ps | grep -i redis-socketio
- docker-compose ps | grep -i mariadb
- python test.py
- docker-compose stop
- ./test.sh
- ./dbench setup docker stop

50
Dockerfile
View File

@ -1,34 +1,36 @@
# Frappe Bench Dockerfile
#bench Dockerfile
FROM debian:9.6-slim
LABEL author=frappé
FROM ubuntu:16.04
LABEL MAINTAINER frappé
USER root
# Generate locale C.UTF-8 for mariadb and general locale data
# Set locale C.UTF-8 for mariadb and general locale data
ENV LANG C.UTF-8
RUN apt-get update && apt-get install -y iputils-ping git build-essential python-setuptools python-dev libffi-dev libssl-dev libjpeg8-dev \
redis-tools redis-server software-properties-common libxrender1 libxext6 xfonts-75dpi xfonts-base zlib1g-dev libfreetype6-dev \
liblcms2-dev libwebp-dev python-tk apt-transport-https libsasl2-dev libldap2-dev libtiff5-dev tcl8.6-dev tk8.6-dev \
wget libmysqlclient-dev mariadb-client mariadb-common curl rlwrap redis-tools nano wkhtmltopdf python-pip vim sudo && apt-get clean \
&& rm -rf /var/lib/apt/lists/*
RUN pip install --upgrade setuptools pip && rm -rf ~/.cache/pip
RUN useradd -ms /bin/bash -G sudo frappe && printf '# User rules for frappe\nfrappe ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers.d/frappe
# Install all neccesary packages
RUN apt-get update && apt-get install -y --no-install-suggests --no-install-recommends build-essential cron curl git iputils-ping libffi-dev \
liblcms2-dev libldap2-dev libmariadbclient-dev libsasl2-dev libssl-dev libtiff5-dev libwebp-dev mariadb-client \
python-dev python-pip python-setuptools python-tk redis-tools rlwrap software-properties-common sudo tk8.6-dev \
vim xfonts-75dpi xfonts-base wget wkhtmltopdf \
&& apt-get clean && rm -rf /var/lib/apt/lists/* \
&& pip install --upgrade setuptools pip --no-cache \
&& curl https://deb.nodesource.com/node_10.x/pool/main/n/nodejs/nodejs_10.10.0-1nodesource1_amd64.deb > node.deb \
&& dpkg -i node.deb \
&& rm node.deb \
&& npm install -g yarn
#nodejs
RUN curl https://deb.nodesource.com/node_10.x/pool/main/n/nodejs/nodejs_10.10.0-1nodesource1_amd64.deb > node.deb \
&& dpkg -i node.deb \
&& rm node.deb
# Add frappe user and setup sudo
RUN useradd -ms /bin/bash -G sudo frappe \
&& printf '# Sudo rules for frappe\nfrappe ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers.d/frappe
USER frappe
WORKDIR /home/frappe
RUN git clone -b master https://github.com/frappe/bench.git bench-repo
USER root
RUN pip install -e bench-repo && rm -rf ~/.cache/pip \
&& npm install -g yarn \
&& chown -R frappe:frappe /home/frappe/*
# Install bench
RUN pip install -e git+https://github.com/frappe/bench.git#egg=bench --no-cache
USER frappe
# Add some bench files
COPY --chown=frappe:frappe ./frappe-bench /home/frappe/frappe-bench
WORKDIR /home/frappe/frappe-bench
EXPOSE 8000 9000 6787
VOLUME [ "/home/frappe/frappe-bench" ]

142
README.md
View File

@ -1,134 +1,72 @@
# frappe_docker
[![Build Status](https://travis-ci.org/frappe/frappe_docker.svg?branch=master)](https://travis-ci.org/frappe/frappe_docker)
# Frappe on Docker
- [Docker](https://docker.com/) is an open source project to pack, ship and run any Linux application in a lighter weight, faster container than a traditional virtual machine.
[![Build Status](https://travis-ci.org/frappe/frappe_docker.svg?branch=develop)](https://travis-ci.org/frappe/frappe_docker)
- Docker makes it much easier to deploy [frappe](https://github.com/frappe/frappe) on your servers.
- This container uses [bench](https://github.com/frappe/bench) to install frappe.
This is a repo designed to aide setting up frappe/ERPNext on docker.
## Getting Started
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
### Prerequisites
Unfortunetly, this container is not curently suited for a production enviorment (but we're working towards that goal!).
[Docker](https://www.docker.com/)
### Build the container and initialize the bench
[Docker Compose](https://docs.docker.com/compose/overview/)
**Note:** These instructions assume you have both [Docker](https://docs.docker.com/engine/installation) and [Docker Compose](https://docs.docker.com/compose/install/) installed on your system.
### Container Configuration
1. Clone this repo and change your working directory to it:
#### ports:
```bash
git clone https://github.com/frappe/frappe_docker.git
cd frappe_docker/
```
```
ports:
- "3307:3307" mariadb-port
- "8000:8000" webserver-port
- "11000:11000" redis-cache
- "12000:12000" redis-queue
- "13000:13000" redis-socketio
- "9000:9000" socketio-port
- "6787:6787" file-watcher-port
```
2. Build and start the container, and initialize the bench:
Expose port 3307 inside the container on port 3307 on ALL local host interfaces. In order to bind to only one interface, you may specify the host's IP address as `([<host_interface>:[host_port]])|(<host_port>):<container_port>[/udp]` as defined in the [docker port binding documentation](http://docs.docker.com/userguide/dockerlinks/). The port 3307 of the mariadb container and port 8000 of the frappe container is exposed to the host machine and other containers.
```bash
./dbench setup docker
./dbench init
```
#### volumes:
**Note:** This will take a while, as docker will now build the container.
```
volumes:
- ./frappe-bench:/home/frappe/frappe-bench
- ./conf/mariadb-conf.d:/etc/mysql/conf.d
- ./redis-conf/redis_socketio.conf:/etc/conf.d/redis.conf
- ./redis-conf/redis_queue.conf:/etc/conf.d/redis.conf
- ./redis-conf/redis_cache.conf:/etc/conf.d/redis.conf
```
Exposes a directory inside the host to the container.
3. Add a new site and start Frappe:
#### links:
```bash
./dbench new-site site1.local
./dbench setup hosts
./dbench start
```
```
links:
- redis-cache
- redis-queue
- redis-socketio
- mariadb
```
4. Use Frappe:
Open your browser to `localhost:8000/login`. Then login using the username `Administrator` and the password `admin`.
Links allow you to define extra aliases by which a service is reachable from another service.
### Basic Usage of `./dbench`
#### depends_on:
**IMPORTANT: Always make sure that your current directory is the root directory of the repo (i.e. `frappe_docker/`)**
```
depends_on:
- mariadb
- redis-cache
- redis-queue
- redis-socketio
```
Express dependency between services, which has two effects:
- `./dbench`: Launches you into an interactive shell in the container as the user `frappe`.
1. docker-compose up will start services in dependency order. In the following example, mariadb and redis will be started before frappe.
- `./dbench setup docker [ stop | down ]`: Starts and builds the docker containers using `docker-compose up -d`.
- `stop`: Stops the containers with `docker-compose stop`.
- `down`: Deletes the containers and the coresponding volumes with `docker-compose down`.
2. docker-compose up SERVICE will automatically include SERVICE’s dependencies. In the following example, docker-compose up docker_frappe will also create and start mariadb and redis.
- `./dbench setup hosts`: Adds all sites to the containers hosts file.
**Note:** Run this after you've added a new site to avoid errors.
### Installation
- `./dbench -c frappe | root <command to run>`: Runs a command in the container, as the selected user.
#### 1. Installation Pre-requisites
- `./dbench -h`: Shows this help message.
- Install [Docker](https://docs.docker.com/engine/installation) Community Edition
- `./dbench <bench command>`: Runs a command in bench (i.e. Running `./dbench new-site site1.local`, will run `bench new-site site1.local` in the container).
- Install [Docker Compose](https://docs.docker.com/compose/install/) (only for Linux users). Docker for Mac, Docker for Windows, and Docker Toolbox include Docker Compose
## For More Info
#### 2. Build the container and install bench
* Clone this repo and change your working directory to frappe_docker
git clone --depth 1 https://github.com/frappe/frappe_docker.git
cd frappe_docker
* Build the container and install bench inside the container.
1.Build the 5 linked containers frappe, mariadb, redis-cache, redis-queue and redis-socketio using this command. Make sure your current working directory is frappe_docker which contains the docker-compose.yml and Dockerfile.
It creates a user, frappe inside the frappe container, whose working directory is /home/frappe. It also clones
the bench-repo from [here](https://github.com/frappe/bench)
./dbench setup docker
Note: Please do not remove the bench-repo directory the above commands will create
#### Basic Usage
##### Make sure your current directory is frappe_docker
1. First time setup
./dbench init
2. Command to be executed everytime after starting your containers
./dbench setup hosts
3. Command to enter your container
./dbench
4. All bench commands can also be directly run from the host machine by using dbench. For instance ```bench start``` can be executed by running ```./dbench start```. Just preface the option with **`./dbench`**. For more information on dbench run the command ```./dbench -h```.
For more info on how to build this docker container refer to this [Wiki](https://github.com/frappe/frappe_docker/wiki/Hitchhiker's-guide-to-building-this-frappe_docker-image)
To login to Frappe / ERPNext, open your browser and go to `[your-external-ip]:8000`, probably `localhost:8000`
The default username is "Administrator" and password is what you set when you created the new site. The default admin password is set in common_site_config.json, and is set to 'admin' in this docker image.
## Built With
* [Docker](https://www.docker.com/)
For more info on building this docker container refer to this [Wiki](https://github.com/frappe/frappe_docker/wiki/Hitchhiker's-guide-to-building-this-frappe_docker-image)
## Contributing
Feel free to contribute to this project and make the container better
Feel free to contribute to this project and make it better.
## License

11
conf/mariadb-conf.d/my.cnf
View File

@ -15,8 +15,8 @@
#collation-server = utf8_general_ci
#character_set_server = utf8
#collation_server = utf8_general_ci
# Import all .cnf files from configuration directory
bind-address = 0.0.0.0
# Import all .cnf files from configuration directory
!includedir /etc/mysql/mariadb.conf.d/
@ -27,3 +27,12 @@ collation-server = utf8mb4_unicode_ci
[mysql]
default-character-set = utf8mb4
[mysqld_safe]
skip_log_error
syslog
[mysqldump]
quick
quote-names
max_allowed_packet = 16M

2
conf/mariadb-conf.d/mysql.cnf
View File

@ -1,2 +0,0 @@
[mysql]

4
conf/mariadb-conf.d/mysqld_safe_syslog.cnf
View File

@ -1,4 +0,0 @@
[mysqld_safe]
skip_log_error
syslog

5
conf/mariadb-conf.d/mysqldump.cnf
View File

@ -1,5 +0,0 @@
[mysqldump]
quick
quote-names
max_allowed_packet = 16M

0
redis-conf/redis_cache.conf → conf/redis-conf.d/redis_cache.conf
View File

0
redis-conf/redis_queue.conf → conf/redis-conf.d/redis_queue.conf
View File

0
redis-conf/redis_socketio.conf → conf/redis-conf.d/redis_socketio.conf
View File

48
dbench
View File

@ -1,43 +1,51 @@
#!/bin/bash
function run () {
user=$1
shift
docker exec -itu "${user}" frappe bash -c "$@"
}
if [[ $# -eq 0 ]]; then
docker exec -it frappe bash
elif [ "$1" == 'init' ]; then
docker exec -itu root frappe bash -c "cd /home/frappe && chown -R frappe:frappe ./*"
docker exec -i frappe bash -c "cd .. && bench init frappe-bench --ignore-exist --skip-redis-config-generation && cd frappe-bench"
docker exec -i frappe bash -c "mv Procfile_docker Procfile && mv sites/common_site_config_docker.json sites/common_site_config.json && bench set-mariadb-host mariadb"
elif [ "$1" == 'setup' ] && [ "$2" == 'docker' ]; then
if [ "$3" == '--swarm-mode' ]; then
elif [[ "$1" == 'init' ]]; then
run root "chown -R frappe:frappe /home/frappe"
run frappe "cd .. && bench init frappe-bench --ignore-exist --skip-redis-config-generation && cd frappe-bench"
run frappe "mv Procfile_docker Procfile && mv sites/common_site_config_docker.json sites/common_site_config.json && bench set-mariadb-host mariadb"
elif [[ "$1" == 'setup' ]]; then
if [[ "$2" == 'docker' ]]; then
if [[ "$3" == '--swarm-mode' ]]; then
echo "Docker swarm mode is not currently supported"
elif [[ "$3" == 'down' ]]; then
docker-compose down
elif [[ "$3" == 'stop' ]]; then
docker-compose stop
else
docker-compose up -d
fi
elif [ "$1" == 'setup' ] && [ "$2" == 'hosts' ]; then
a=$(cd frappe-bench && ls sites/*/site_config.json | grep -o '/.\+/')
elif [[ "$2" == 'hosts' ]]; then
a=$(run frappe "ls sites/*/site_config.json | grep -o '/.\+/'") \
a="${a//$'\n'/ }"
a=$(echo $a | tr -d / )
a=$(echo "$a" | tr -d / )
result="127.0.0.1 ${a}"
echo $result | sudo tee -a /etc/hosts
docker exec -iu root frappe bash -c "echo ${result} | tee --append /etc/hosts"
docker exec -itu root frappe bash -c "printf '# User rules for frappe\nfrappe ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers.d/frappe"
elif [ "$1" == '-c' ]; then
run root "echo ${result} | tee --append /etc/hosts"
fi
elif [[ "$1" == '-c' ]]; then
shift
user=$1
shift
docker exec -iu "$user" frappe bash -c "$@"
elif [ "$1" == '-h' ]; then
run "$user" "$@"
elif [[ "$1" == '-h' ]]; then
echo "$0 [-h] | [ -c frappe | root <command to run> ] [ <command to send to bench> ]"
echo ""
echo "$0 extends the Frappe Bench tool, and is used like it. However, it extends the tool in a few places."
echo "Usage:"
echo " $0"
echo " Launches you into an interactive shell in the container as user frappe"
echo " $0 init bench-folder-name"
echo " Runs \"bench init\" along with some other commands"
echo " $0 setup docker"
echo " $0 setup docker [ stop | down ]"
echo " Starts and builds the docker containers using \"docker-compose up -d\""
echo " $0 setup hosts"
echo " Adds all sites to the hosts file, run with sudo"
echo " Adds all sites to the containers hosts file"
echo " $0 -c frappe | root <command to run>"
echo " Runs a command in the container, as the selected user"
echo " $0 -h"
@ -46,5 +54,5 @@ elif [ "$1" == '-h' ]; then
echo " Runs a command in bench, i.e. $0 new-site site1.local = bench new-site site1.local"
else
IFS=" "
docker exec -it frappe bash -c "bench $*"
run frappe "bench $*"
fi

35
docker-compose.yml
View File

@ -1,22 +1,26 @@
version: '2'
version: '3.7'
services:
mariadb:
image: mariadb
environment:
- MYSQL_ROOT_PASSWORD=123
- MYSQL_USER=root
volumes:
- ./conf/mariadb-conf.d:/etc/mysql/conf.d
- /var/lib/mysql
ports:
- "3307:3306" #mariadb-port
- "3307:3306" # MariaDB Port
container_name: mariadb
redis-cache:
image: redis:alpine
volumes:
- ./redis-conf:/etc/conf.d
- ./conf/redis-conf.d:/etc/conf.d
command: ["redis-server","/etc/conf.d/redis_cache.conf"]
@ -25,8 +29,9 @@ services:
redis-queue:
image: redis:alpine
volumes:
- ./redis-conf:/etc/conf.d
- ./conf/redis-conf.d:/etc/conf.d
command: ["redis-server","/etc/conf.d/redis_queue.conf"]
@ -36,7 +41,7 @@ services:
image: redis:alpine
volumes:
- ./redis-conf:/etc/conf.d
- ./conf/redis-conf.d:/etc/conf.d
command: ["redis-server","/etc/conf.d/redis_socketio.conf"]
@ -44,26 +49,32 @@ services:
frappe:
build: .
volumes:
- ./frappe-bench:/home/frappe/frappe-bench
- ./redis-conf/redis_cache.conf:/home/frappe/frappe-bench/config/redis_cache.conf
- ./redis-conf/redis_queue.conf:/home/frappe/frappe-bench/config/redis_queue.conf
- ./redis-conf/redis_socketio.conf:/home/frappe/frappe-bench/config/redis_socketio.conf
build: .
- ./conf/redis-conf.d/redis_cache.conf:/home/frappe/frappe-bench/config/redis_cache.conf
- ./conf/redis-conf.d/redis_queue.conf:/home/frappe/frappe-bench/config/redis_queue.conf
- ./conf/redis-conf.d/redis_socketio.conf:/home/frappe/frappe-bench/config/redis_socketio.conf
ports:
- "8000:8000" #webserver_port
- "9000:9000" #socketio_port
- "6787:6787" #file_watcher_port
- "8000:8000" # Webserver Port
- "9000:9000" # Socketio Port
- "6787:6787" # File Watcher Port
stdin_open: true
tty: true
links:
- redis-cache
- redis-queue
- redis-socketio
- mariadb
depends_on:
- mariadb
- redis-cache
- redis-queue
- redis-socketio
container_name: frappe

1
redis-conf/pids/redis_cache.pid
View File

@ -1 +0,0 @@
20

1
redis-conf/pids/redis_queue.pid
View File

@ -1 +0,0 @@
24

1
redis-conf/pids/redis_socketio.pid
View File

@ -1 +0,0 @@
29

29
test.py
View File

@ -1,29 +0,0 @@
import subprocess, requests, datetime, _thread, time, os, signal
start_time = datetime.datetime.now().time()
bench_start = 'docker exec -i frappe bash -c "bench start"'
process = subprocess.Popen(bench_start, stdout=subprocess.PIPE , shell=True)
def print_out(val,delay):
while 1:
time.sleep(delay)
if val == 1:
result,error = process.communicate()
print(result)
elif val == 2:
try:
global r
r = requests.get("http://site1.local:8000")
print(r)
except requests.exceptions.ConnectionError as e:
print(e)
_thread.start_new_thread(print_out, (1, 1))
#result, error = process.communicate()
_thread.start_new_thread(print_out, (2, 1))
time.sleep(30)
process.kill()
assert '<title> Login </title>' in str(r.content)

17
test.sh Normal file
View File

@ -0,0 +1,17 @@
#!/bin/bash
cat <(./dbench start) &
sleep 5
while ! [[ $i == 20 ]]
do
output=$( curl "http://localhost:8000" )
{ echo "Exit status of curl: $?"
} 1>&2
sleep 2
i=$((i + 1))
done
echo "${output}" | grep '<title> Login </title>' || exit 1