Commit Graph

1016 Commits

Author SHA1 Message Date
Shlomi Noach
c2186db527 Merge branch 'master' into touch-postpone-flag-file 2017-10-02 15:55:21 +03:00
Shlomi Noach
5db266e4fa tests expecting ROW binlog format, gracefully waiting for replication to catch up 2017-10-02 15:52:31 +03:00
Shlomi Noach
9890e66755 Merge pull request #491 from zmoazeni/enable-extra-port
Allow gh-ost to modify the server using extra port
2017-10-02 14:26:59 +03:00
Shlomi Noach
dfc9f41342 Merge pull request #498 from dcadevil/master
Thank you!
2017-09-29 09:25:36 +01:00
Nikhil Mathew
84bdfdb1ad Cache DB connection pools on the migrationContext where applicable 2017-09-22 16:06:06 -07:00
Nikhil Mathew
5788ab5347 CR Revisions: Make concurrent operations safe 2017-09-22 15:17:58 -07:00
dcadevil
351bfe1a24 Correct spelling for initiateStreaming() comment. 2017-09-21 11:36:27 +08:00
Zach Moazeni
df27c5b76f Allow gh-ost to modify the server using extra port
Both Percona and Maria allow MySQL to be configured to listen on an extra port when their thread pool is enable.

* https://www.percona.com/doc/percona-server/5.7/performance/threadpool.html
* https://mariadb.com/kb/en/the-mariadb-library/thread-pool-in-mariadb-51-53/

This is valuable because if the table has a lot of traffic (read or write load), gh-ost can end up starving the thread pool as incomming connections are immediately blocked.

By using gh-ost on the extra port, MySQL locking will still behave the same, but MySQL will keep a dedicated thread for each gh-ost connection.

When doing this, it's important to inspect the extra-max-connections variable. Both Percona and Maria default to 1, so gh-ost may easily exceed with its threads.

An example local run using this

```
$ mysql -S /tmp/mysql_sandbox20393.sock -e "select @@global.port, @@global.extra_port"
+---------------+---------------------+
| @@global.port | @@global.extra_port |
+---------------+---------------------+
|         20393 |               30393 |
+---------------+---------------------+

./bin/gh-ost \
--initially-drop-ghost-table \
--initially-drop-old-table \
--assume-rbr \
--port="20395" \
--assume-master-host="127.0.0.1:30393" \
--max-load=Threads_running=25 \
--critical-load=Threads_running=1000 \
--chunk-size=1000 \
--max-lag-millis=1500 \
--user="gh-ost" \
--password="gh-ost" \
--database="test" \
--table="mytable" \
--verbose \
--alter="ADD mynewcol decimal(11,2) DEFAULT 0.0 NOT NULL" \
--exact-rowcount \
--concurrent-rowcount \
--default-retries=120 \
--panic-flag-file=/tmp/ghost.panic.flag \
--postpone-cut-over-flag-file=/tmp/ghost.postpone.flag \
--execute
```
2017-09-20 16:05:20 -04:00
Shlomi Noach
52f2dd7a20 release v1.0.42 2017-09-14 09:51:12 +03:00
Shlomi Noach
a46d5b54ea Merge pull request #494 from github/cfg-read-error
Clearer error message on config file read error
2017-09-10 19:54:13 +03:00
Shlomi Noach
6362dbe51a Clearer error message on config file read error 2017-09-10 08:20:01 +03:00
Shlomi Noach
a2847015d6 Merge branch 'master' into nm-refactor-migration-context 2017-09-10 08:08:39 +03:00
Shlomi Noach
3fc3446561 Merge pull request #489 from github/checking-thresholds
Validating password length
2017-09-06 15:36:46 +03:00
Shlomi Noach
bdae67f839 Merge branch 'master' into checking-thresholds 2017-09-06 15:24:04 +03:00
Shlomi Noach
8940649a32 Merge branch 'master' into checking-thresholds 2017-09-06 15:14:17 +03:00
Shlomi Noach
1ada323c03 Merge pull request #490 from boomballa/master
Update the main.go word spelling error
2017-09-06 10:19:29 +03:00
Peng Sung
113e115c49 Update the main.go word spelling error 2017-09-06 15:12:09 +08:00
Shlomi Noach
78c1e0432f Merge branch 'master' into checking-thresholds 2017-09-06 08:07:35 +03:00
Shlomi Noach
0eac2f8da5 Merge pull request #488 from github/check-unique-key-types
Check unique key types
2017-09-06 08:05:23 +03:00
Shlomi Noach
e464c83e13 added test 2017-09-05 06:56:19 +03:00
Shlomi Noach
7c47c722e3 tests belonged in another branch 2017-09-05 06:38:55 +03:00
Shlomi Noach
227f1de193 adding tests 2017-09-05 06:35:56 +03:00
Shlomi Noach
0924a459f2 adding tests 2017-09-05 06:34:26 +03:00
Shlomi Noach
e2171e0162 Validating password length 2017-09-03 10:27:04 +03:00
Shlomi Noach
3437cf44c4 Validating shared key column types 2017-09-03 09:57:26 +03:00
Shlomi Noach
933901e21e Validating shared key column types 2017-09-03 09:57:24 +03:00
Shlomi Noach
ed0fc1c7b9 Merge pull request #481 from github/autocommit
gh-ost uses autocommit=1 for all connections
2017-09-03 07:58:57 +03:00
Shlomi Noach
f2cd598376 Merge branch 'master' into autocommit 2017-09-03 07:55:13 +03:00
Shlomi Noach
b9050222e1 Merge pull request #487 from cjoudrey/rm-dead-code-in-migrator
Thank you
2017-09-03 07:48:50 +03:00
Christian Joudrey
2646a0a997
Remove unused acceptSignals in migrator 2017-09-02 21:59:09 -04:00
Shlomi Noach
a3ee81e116 Merge branch 'master' into autocommit 2017-08-30 11:40:56 +03:00
Shlomi Noach
3dd0c50b6d Merge pull request #473 from github/json-57-test
Supporting JSON columns
2017-08-30 11:40:45 +03:00
Shlomi Noach
10cafb71c0 gh-ost uses autocommit=1 for all connections 2017-08-30 11:38:22 +03:00
Nikhil Mathew
0a7e713e9f Ensure cleanup happens, even on error 2017-08-28 15:53:47 -07:00
Nikhil Mathew
e4bb70df43 Use change from outbrain/golib PR 2017-08-28 15:19:05 -07:00
Nikhil Mathew
3b21f4db37 Close remaining DB connections 2017-08-28 14:05:15 -07:00
Shlomi Noach
c9f2da82f0 Merge branch 'master' into json-57-test 2017-08-27 08:01:02 +03:00
Shlomi Noach
7fa16df6aa Merge pull request #471 from github/range-query-offset
Range query offset
2017-08-27 08:00:40 +03:00
Shlomi Noach
4148a32c77 Merge branch 'master' into range-query-offset 2017-08-27 07:59:26 +03:00
Shlomi Noach
7c38a50b66 POINT limitation 2017-08-22 12:12:27 +03:00
Shlomi Noach
5a8ed7b6e9 clarifying JSON limitation in docs 2017-08-22 12:10:47 +03:00
Shlomi Noach
6a5ace1842 a slighly more interesting test 2017-08-22 12:02:57 +03:00
Shlomi Noach
2174588ebf Supporting JSON types 2017-08-22 11:51:20 +03:00
Shlomi Noach
a43796d5c5 added JSON UPDATE and DELETE tests 2017-08-22 11:50:50 +03:00
Shlomi Noach
88d6163ce2 adding json 5.7 tests 2017-08-22 11:20:58 +03:00
Shlomi Noach
a5081da328 Merge pull request #472 from github/tests-57
tests compatible with 5.7
2017-08-22 11:15:10 +03:00
Shlomi Noach
17c424f671 tests adaptation for 5.7 2017-08-22 11:08:02 +03:00
Shlomi Noach
7e4d0d5721 Merge branch 'master' into json-57-tests 2017-08-21 08:44:14 +03:00
Shlomi Noach
e10add08a8 smaller chunk size for tests 2017-08-21 08:20:06 +03:00
Shlomi Noach
79e4f1cdbe detect range end based on OFFSET 2017-08-21 08:12:41 +03:00