octoleo/gh-ost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,396 Commits 51 Branches 51 Tags 24 MiB
wagnerm/docs
Commit Graph

391 Commits

Author SHA1 Message Date
Shlomi Noach
04c0be6137
Merge branch 'master' into ipv6 2018-09-17 09:35:18 +03:00
Shlomi Noach
97c1e72dbe
Merge branch 'master' into minor_changes 2018-09-17 08:58:04 +03:00
Shlomi Noach
695e32d41b
Merge branch 'master' into dev_copyrow_hangs 2018-09-17 08:42:03 +03:00
Shlomi Noach
49b80df27b Parsing ipv6 addresses 2018-09-16 11:44:52 +03:00
Shlomi Noach
32f1cf1dbe Support for GCP (Google Cloud Platform) 2018-09-02 09:51:40 +03:00
钟越
ef686a4e7b Support database with underscore 2018-08-31 09:31:58 +08:00
wangfei
c7dff99a19 fix copyRowsFunc hangs bug 2018-07-26 03:02:12 +08:00
MOON_CLJ
14eda7efe0 fix GetReplicationLag not used args 2018-05-23 13:13:25 +08:00
MOON_CLJ
21d455013e minor changes and typo correction 2018-05-23 13:13:25 +08:00
Shlomi Noach
db871b42c4 Support for GENERATED (aka virtual) columns 2018-05-22 12:36:52 +03:00
Shlomi Noach
dfeefed5dc avoid prepared statements for changelog table 2018-05-20 13:46:33 +03:00
Shlomi Noach
fb00a13871 Rejecting RENAME TO|AS 2018-05-06 11:19:03 +03:00
Shlomi Noach
160e7a11f5 Enforcing binlog_row_image=FULL 2018-04-24 05:56:03 -07:00
Shlomi Noach
3dc4930ae3
Merge branch 'master' into support-aliyun-rds 2018-04-16 14:17:03 +03:00
zj118228
d3b98d9e74 Refactor the code and remove the global variable of 
support-aliyun-rds branch.
 2018-03-29 12:20:41 +08:00
Kurt Kotzur
15e7417fc5
update comment 2018-03-19 12:29:49 -07:00
Kurt Kotzur
117b197b2a
Obey defaultNumRetries in retryOperationWithExponentialBackoff; name max interval flag more generically (#2) 
* rename flags, obey defaultNumRetries

* capitalization fixes

* fix flag description typo

* fix sleep algorithm
 2018-03-19 12:26:46 -07:00
Kurt Kotzur
628983f78f
🎨 2018-03-16 12:42:38 -07:00
Kurt Kotzur
64f66c4abb
add back nonFatalHint, fix type mismatches 2018-03-16 10:54:37 -07:00
Kurt Kotzur
bd62b19b0b
remove nonFatalHint argument from retryOperation; add retryOperationWithExponentialBackoff and conditionally use it 2018-03-15 17:41:15 -07:00
Shlomi Noach
bf01b31795
Merge branch 'master' into support-aliyun-rds 2018-03-07 16:39:55 +02:00
Shlomi Noach
9ae9cef17a support quoted values in interactive commands 2018-02-25 19:17:37 +02:00
zj118228
6f3d54a136 Aliyun RDS hide some mysql variables for secrity so gh-ost will get 
ilegal values. Add --aliyun-rds flag to avoid getting them.
 2018-01-23 14:44:22 +08:00
Shlomi Noach
02579075d0 Call the throttler.Teardown 2018-01-11 08:03:09 +02:00
Nikhil Mathew
ec6ceffbcb Pass in a migrationContext UUID for a migration specific connections cache 2017-12-14 15:53:29 -08:00
Nikhil Mathew
fac1ba7026 Throttler teardown 2017-12-14 15:14:13 -08:00
Shlomi Noach
0530b09700
Merge branch 'master' into nm-refactor-migration-context 2017-12-07 13:59:12 +02:00
Shlomi Noach
58c381f0a0 some cleanup 2017-11-21 09:14:04 +02:00
Shlomi Noach
203ea6c2cd refactor, simplifid code 2017-11-20 10:24:28 +02:00
Shlomi Noach
3898d49f6c supporting update to columns of migration key 2017-11-20 08:17:20 +02:00
Nikhil Mathew
538833ea84
Merge branch 'master' into nm-refactor-migration-context 2017-11-14 10:29:49 -08:00
Nikhil Mathew
2ad65ee1d7 Refactor usages of GetDB to use mysql 2017-11-10 15:44:22 -08:00
Shlomi Noach
8a59d7e823 failing on PK value change 2017-11-08 11:11:17 +02:00
Josh Soref
6bc04eee62 spelling: thing 2017-11-08 01:02:32 +00:00
Josh Soref
3196c890f5 spelling: running 2017-11-08 01:01:39 +00:00
Josh Soref
4c93b7ac6d spelling: response 2017-11-08 00:55:09 +00:00
Josh Soref
961e2409f6 spelling: proceeding 2017-11-08 00:50:56 +00:00
Josh Soref
6a8d97c13f spelling: progress 2017-11-08 00:50:48 +00:00
Josh Soref
867e6a28b3 spelling: merely 2017-11-08 00:49:51 +00:00
Josh Soref
e2df0b80eb spelling: iterating 2017-11-08 00:49:23 +00:00
Josh Soref
2b430e3cb6 spelling: ignoring 2017-11-08 00:49:06 +00:00
Josh Soref
e608c16567 spelling: explicitly 2017-11-08 00:47:54 +00:00
Josh Soref
291c6d3594 spelling: entirely 2017-11-08 00:47:36 +00:00
Josh Soref
418462a6ca spelling: dropping 2017-11-08 00:47:15 +00:00
Josh Soref
8901e6eec2 spelling: decision 2017-11-08 00:46:04 +00:00
Josh Soref
d29993ffda spelling: changelog 2017-11-08 00:45:19 +00:00
Josh Soref
4dda9cf930 spelling: aggressive 2017-11-08 00:44:30 +00:00
Shlomi Noach
087a6cc0cb Merge branch 'master' into touch-postpone-flag-file 2017-10-17 16:02:11 +03:00
Nikhil Mathew
f06e23282e Merge branch 'master' into nm-refactor-migration-context 2017-10-02 10:31:57 -07:00
Shlomi Noach
aff7413624 clearer error message when not able to create postpone-cut-over-flag-file 2017-10-02 16:00:58 +03:00
Shlomi Noach
c2186db527 Merge branch 'master' into touch-postpone-flag-file 2017-10-02 15:55:21 +03:00
Shlomi Noach
9890e66755 Merge pull request #491 from zmoazeni/enable-extra-port 
Allow gh-ost to modify the server using extra port
 2017-10-02 14:26:59 +03:00
Nikhil Mathew
84bdfdb1ad Cache DB connection pools on the migrationContext where applicable 2017-09-22 16:06:06 -07:00
Nikhil Mathew
5788ab5347 CR Revisions: Make concurrent operations safe 2017-09-22 15:17:58 -07:00
dcadevil
351bfe1a24 Correct spelling for initiateStreaming() comment. 2017-09-21 11:36:27 +08:00
Zach Moazeni
df27c5b76f Allow gh-ost to modify the server using extra port 
Both Percona and Maria allow MySQL to be configured to listen on an extra port when their thread pool is enable.

* https://www.percona.com/doc/percona-server/5.7/performance/threadpool.html
* https://mariadb.com/kb/en/the-mariadb-library/thread-pool-in-mariadb-51-53/

This is valuable because if the table has a lot of traffic (read or write load), gh-ost can end up starving the thread pool as incomming connections are immediately blocked.

By using gh-ost on the extra port, MySQL locking will still behave the same, but MySQL will keep a dedicated thread for each gh-ost connection.

When doing this, it's important to inspect the extra-max-connections variable. Both Percona and Maria default to 1, so gh-ost may easily exceed with its threads.

An example local run using this

```
$ mysql -S /tmp/mysql_sandbox20393.sock -e "select @@global.port, @@global.extra_port"
+---------------+---------------------+
| @@global.port | @@global.extra_port |
+---------------+---------------------+
|         20393 |               30393 |
+---------------+---------------------+

./bin/gh-ost \
--initially-drop-ghost-table \
--initially-drop-old-table \
--assume-rbr \
--port="20395" \
--assume-master-host="127.0.0.1:30393" \
--max-load=Threads_running=25 \
--critical-load=Threads_running=1000 \
--chunk-size=1000 \
--max-lag-millis=1500 \
--user="gh-ost" \
--password="gh-ost" \
--database="test" \
--table="mytable" \
--verbose \
--alter="ADD mynewcol decimal(11,2) DEFAULT 0.0 NOT NULL" \
--exact-rowcount \
--concurrent-rowcount \
--default-retries=120 \
--panic-flag-file=/tmp/ghost.panic.flag \
--postpone-cut-over-flag-file=/tmp/ghost.postpone.flag \
--execute
```
 2017-09-20 16:05:20 -04:00
Shlomi Noach
a2847015d6 Merge branch 'master' into nm-refactor-migration-context 2017-09-10 08:08:39 +03:00
Shlomi Noach
78c1e0432f Merge branch 'master' into checking-thresholds 2017-09-06 08:07:35 +03:00
Shlomi Noach
e2171e0162 Validating password length 2017-09-03 10:27:04 +03:00
Shlomi Noach
3437cf44c4 Validating shared key column types 2017-09-03 09:57:26 +03:00
Christian Joudrey
2646a0a997
Remove unused acceptSignals in migrator 2017-09-02 21:59:09 -04:00
Nikhil Mathew
0a7e713e9f Ensure cleanup happens, even on error 2017-08-28 15:53:47 -07:00
Nikhil Mathew
e4bb70df43 Use change from outbrain/golib PR 2017-08-28 15:19:05 -07:00
Nikhil Mathew
3b21f4db37 Close remaining DB connections 2017-08-28 14:05:15 -07:00
Shlomi Noach
c9f2da82f0 Merge branch 'master' into json-57-test 2017-08-27 08:01:02 +03:00
Shlomi Noach
2174588ebf Supporting JSON types 2017-08-22 11:51:20 +03:00
Shlomi Noach
79e4f1cdbe detect range end based on OFFSET 2017-08-21 08:12:41 +03:00
Nikhil Mathew
a1473dfb58 Stop infinite goroutines 2017-08-08 15:31:25 -07:00
Nikhil Mathew
7412f42371 Remove no longer needed acceptSignals func 2017-08-08 13:44:53 -07:00
Nikhil Mathew
b399533d4f Simplify GoMySQLReader 2017-08-08 13:41:46 -07:00
Nikhil Mathew
982b8eede9 Refactor global migration context 2017-08-08 13:36:54 -07:00
Shlomi Noach
de0ebcbbb0 documentation 2017-07-19 16:48:22 +03:00
Shlomi Noach
6da0b8af3d Dynamic DML batch size; apadting buffer size to match 2017-07-19 16:44:18 +03:00
Shlomi Noach
4a3a5a1ce1 Merge branch 'master' into case-insensitive-columns 2017-07-16 06:51:39 +03:00
Shlomi Noach
effae08fbf supporting change of column name-case 2017-07-12 12:33:32 +03:00
Shlomi Noach
e6dd2c90d2 responding to errors in rowcopy 2017-07-11 13:39:11 +03:00
Shlomi Noach
a8932ee1b8 freno-aware http-throttler 2017-07-04 10:28:09 +03:00
Shlomi Noach
36bcc031ce Merge branch 'master' into fix-variables-spelling 2017-06-29 20:53:49 +03:00
Shlomi Noach
cb3c7f3e20 Merge branch 'master' into touch-postpone-flag-file 2017-06-11 08:57:45 +03:00
Shlomi Noach
8da0f60582 fixed critical-load check for hibernation 2017-05-24 10:53:00 +03:00
Shlomi Noach
ad47f7c147 throttling just prior to leaving hibernation, so as to allow re-throttle checks to apply 2017-05-24 10:42:47 +03:00
Shlomi Noach
3955a6d67f hibernate on critical-load 2017-05-24 08:32:13 +03:00
Shlomi Noach
aca288e88d -postpone-cut-over-flag-file implies touching indicated file 2017-05-07 14:58:18 +03:00
Shlomi Noach
eaa3489fe0 Merge branch 'master' into server-report-coordinates 2017-05-01 07:50:13 +03:00
Shlomi Noach
79cee99f57 support for 'coordinates' command 2017-04-28 15:50:51 -07:00
Jacob Bednarz
0243e76b33
s/applyEnvironmentVairables/applyEnvironmentVariables 2017-04-24 07:45:21 +10:00
Shlomi Noach
e2a6b857d0 Merge branch 'master' into drop-add-same-column 2017-04-23 20:26:31 +03:00
Shlomi Noach
8a0f1413eb dropped columns are not 'shared' and no data copy attempted for such columns 2017-04-23 08:38:35 +03:00
Shlomi Noach
3092e9c5c0 error and nil checks on socket connection 2017-04-13 08:27:42 +03:00
Shlomi Noach
c413d508cc HEAD instead of GET 2017-03-26 13:12:56 +03:00
Shlomi Noach
23ce390d69 supporting throttle-http 2017-03-26 13:10:34 +03:00
Shlomi Noach
7517238bd3 Merge branch 'master' into fix-infinite-cutover-loop 2017-03-06 21:50:26 +02:00
Shlomi Noach
91521f9357 Merge branch 'master' into old-table-name-timestamp 2017-03-05 18:10:14 +02:00
Shlomi Noach
f11f6f978f mitigating cut-over/write race condition 2017-02-24 14:48:10 -07:00
Shlomi Noach
06c909bd10 Validating table name length 2017-02-21 17:34:49 -07:00
Shlomi Noach
289ce46a2b Merge branch 'master' into fix-infinite-cutover-loop 2017-02-16 14:16:32 +02:00
Shlomi Noach
b289041ccb Merge branch 'master' into fix-reappearing-throttled-reasons 2017-02-08 13:11:34 +02:00
Shlomi Noach
6ecf7cc0ee Auto-merged master into interactive-command-question on deployment 2017-02-08 13:03:45 +02:00
Shlomi Noach
10edf3c063 Migration only starting after first replication lag metric collected 2017-02-07 12:13:19 +02:00
Shlomi Noach
91a3fd4a84 Merge branch 'master' into which-mysql-versions 2017-02-07 11:12:48 +02:00