Shlomi Noach
fb00a13871
Rejecting RENAME TO|AS
2018-05-06 11:19:03 +03:00
Cyrill Schumacher
c6a6a533a1
go/base: Do not ignore f.Close error
2018-05-04 08:22:54 +02:00
Atsushi Ishibashi
99fcbc35f3
Fix custom type assignment
2018-05-04 01:45:14 +09:00
Shlomi Noach
160e7a11f5
Enforcing binlog_row_image=FULL
2018-04-24 05:56:03 -07:00
Shlomi Noach
3dc4930ae3
Merge branch 'master' into support-aliyun-rds
2018-04-16 14:17:03 +03:00
zj118228
d3b98d9e74
Refactor the code and remove the global variable of
...
support-aliyun-rds branch.
2018-03-29 12:20:41 +08:00
Shlomi Noach
b3f599acd8
Merge branch 'master' into support-aliyun-rds
2018-03-27 14:23:51 +03:00
Kurt Kotzur
15e7417fc5
update comment
2018-03-19 12:29:49 -07:00
Kurt Kotzur
117b197b2a
Obey defaultNumRetries in retryOperationWithExponentialBackoff; name max interval flag more generically ( #2 )
...
* rename flags, obey defaultNumRetries
* capitalization fixes
* fix flag description typo
* fix sleep algorithm
2018-03-19 12:26:46 -07:00
Kurt Kotzur
628983f78f
🎨
2018-03-16 12:42:38 -07:00
Kurt Kotzur
64f66c4abb
add back nonFatalHint, fix type mismatches
2018-03-16 10:54:37 -07:00
Kurt Kotzur
bd62b19b0b
remove nonFatalHint argument from retryOperation; add retryOperationWithExponentialBackoff and conditionally use it
2018-03-15 17:41:15 -07:00
Kurt Kotzur
3f0443833d
add flags for enabling exponential backoff and maximum backoff interval
2018-03-15 17:40:15 -07:00
Shlomi Noach
710abd5a5b
Fixed test
2018-03-12 16:20:46 +02:00
Shlomi Noach
2311df412d
gbk support, simplified
2018-03-12 16:17:53 +02:00
Shlomi Noach
bf01b31795
Merge branch 'master' into support-aliyun-rds
2018-03-07 16:39:55 +02:00
Shlomi Noach
2209136258
rename parameters
2018-03-07 16:30:02 +02:00
Shlomi Noach
1a882c6eb5
Merge branch 'master' into gbk_encoding
2018-02-27 14:06:59 +02:00
Shlomi Noach
e2f6f4bc36
Merge branch 'master' into gh-ost-ci-env
2018-02-27 13:26:56 +02:00
Shlomi Noach
9ae9cef17a
support quoted values in interactive commands
2018-02-25 19:17:37 +02:00
Shlomi Noach
9b0efd0d93
delegating to 127.0.0.1
2018-02-11 14:58:16 +02:00
zj118228
6f3d54a136
Aliyun RDS hide some mysql variables for secrity so gh-ost will get
...
ilegal values. Add --aliyun-rds flag to avoid getting them.
2018-01-23 14:44:22 +08:00
ceshihao
330650baa9
rename includeMultibyteCharset -> includeRiskCharset, multibyteCharset -> riskCharset
2018-01-17 00:15:50 +08:00
ceshihao
14315ee2cc
resolve merge conflict
2018-01-16 23:43:42 +08:00
ceshihao
346138772c
add an option --include-multibyte-charset
2018-01-16 23:43:33 +08:00
ceshihao
35becabf3c
support gbk encoding
2018-01-16 23:43:09 +08:00
Shlomi Noach
a01e3639a2
Merge branch 'master' into go-mysql-ignore-pass
2018-01-14 10:08:58 +02:00
Shlomi Noach
fcda14634d
go-mysql library to avoid printing password in cleartext
2018-01-11 14:51:14 +02:00
Shlomi Noach
02579075d0
Call the throttler.Teardown
2018-01-11 08:03:09 +02:00
Nikhil Mathew
e82d5633f9
Close the binlog syncer
2017-12-18 11:03:37 -08:00
Nikhil Mathew
ec6ceffbcb
Pass in a migrationContext UUID for a migration specific connections cache
2017-12-14 15:53:29 -08:00
Nikhil Mathew
fac1ba7026
Throttler teardown
2017-12-14 15:14:13 -08:00
Shlomi Noach
7ef284a3ad
removed redundant CanStopStreaming func
2017-12-11 08:02:32 +02:00
Shlomi Noach
0530b09700
Merge branch 'master' into nm-refactor-migration-context
2017-12-07 13:59:12 +02:00
Shlomi Noach
58c381f0a0
some cleanup
2017-11-21 09:14:04 +02:00
Shlomi Noach
203ea6c2cd
refactor, simplifid code
2017-11-20 10:24:28 +02:00
Shlomi Noach
3898d49f6c
supporting update to columns of migration key
2017-11-20 08:17:20 +02:00
Nikhil Mathew
538833ea84
Merge branch 'master' into nm-refactor-migration-context
2017-11-14 10:29:49 -08:00
Nikhil Mathew
2ad65ee1d7
Refactor usages of GetDB to use mysql
2017-11-10 15:44:22 -08:00
Nikhil Mathew
bb4a0be4c6
Add GetDB to mysql.GetDB
2017-11-10 15:41:37 -08:00
Nikhil Mathew
338a439c91
Remove GetDB from MigrationContext
2017-11-10 15:40:50 -08:00
Shlomi Noach
8a59d7e823
failing on PK value change
2017-11-08 11:11:17 +02:00
Josh Soref
15b8d5769b
spelling: unable
2017-11-08 01:02:47 +00:00
Josh Soref
6bc04eee62
spelling: thing
2017-11-08 01:02:32 +00:00
Josh Soref
cb97d6854c
spelling: synchronously
2017-11-08 01:02:21 +00:00
Josh Soref
94dfa0ed24
spelling: specified
2017-11-08 01:01:51 +00:00
Josh Soref
3196c890f5
spelling: running
2017-11-08 01:01:39 +00:00
Josh Soref
b3be5352c5
spelling: rotates
2017-11-08 00:56:14 +00:00
Josh Soref
4c93b7ac6d
spelling: response
2017-11-08 00:55:09 +00:00
Josh Soref
961e2409f6
spelling: proceeding
2017-11-08 00:50:56 +00:00
Josh Soref
6a8d97c13f
spelling: progress
2017-11-08 00:50:48 +00:00
Josh Soref
867e6a28b3
spelling: merely
2017-11-08 00:49:51 +00:00
Josh Soref
e2df0b80eb
spelling: iterating
2017-11-08 00:49:23 +00:00
Josh Soref
2b430e3cb6
spelling: ignoring
2017-11-08 00:49:06 +00:00
Josh Soref
ed35b75cc9
spelling: identified
2017-11-08 00:48:53 +00:00
Josh Soref
e608c16567
spelling: explicitly
2017-11-08 00:47:54 +00:00
Josh Soref
291c6d3594
spelling: entirely
2017-11-08 00:47:36 +00:00
Josh Soref
9bb6959d22
spelling: empty
2017-11-08 00:47:24 +00:00
Josh Soref
418462a6ca
spelling: dropping
2017-11-08 00:47:15 +00:00
Josh Soref
1ae6489afc
spelling: detached
2017-11-08 00:46:26 +00:00
Josh Soref
8901e6eec2
spelling: decision
2017-11-08 00:46:04 +00:00
Josh Soref
84be0562a0
spelling: coordinate
2017-11-08 00:45:47 +00:00
Josh Soref
e99333e576
spelling: conversion
2017-11-08 00:45:31 +00:00
Josh Soref
d29993ffda
spelling: changelog
2017-11-08 00:45:19 +00:00
Josh Soref
4dda9cf930
spelling: aggressive
2017-11-08 00:44:30 +00:00
Akshay Chhajed
05fc358ed4
send -help output to stdout
2017-11-01 00:24:09 +05:30
Shlomi Noach
087a6cc0cb
Merge branch 'master' into touch-postpone-flag-file
2017-10-17 16:02:11 +03:00
Nikhil Mathew
f06e23282e
Merge branch 'master' into nm-refactor-migration-context
2017-10-02 10:31:57 -07:00
Shlomi Noach
aff7413624
clearer error message when not able to create postpone-cut-over-flag-file
2017-10-02 16:00:58 +03:00
Shlomi Noach
c2186db527
Merge branch 'master' into touch-postpone-flag-file
2017-10-02 15:55:21 +03:00
Shlomi Noach
9890e66755
Merge pull request #491 from zmoazeni/enable-extra-port
...
Allow gh-ost to modify the server using extra port
2017-10-02 14:26:59 +03:00
Nikhil Mathew
84bdfdb1ad
Cache DB connection pools on the migrationContext where applicable
2017-09-22 16:06:06 -07:00
Nikhil Mathew
5788ab5347
CR Revisions: Make concurrent operations safe
2017-09-22 15:17:58 -07:00
dcadevil
351bfe1a24
Correct spelling for initiateStreaming() comment.
2017-09-21 11:36:27 +08:00
Zach Moazeni
df27c5b76f
Allow gh-ost to modify the server using extra port
...
Both Percona and Maria allow MySQL to be configured to listen on an extra port when their thread pool is enable.
* https://www.percona.com/doc/percona-server/5.7/performance/threadpool.html
* https://mariadb.com/kb/en/the-mariadb-library/thread-pool-in-mariadb-51-53/
This is valuable because if the table has a lot of traffic (read or write load), gh-ost can end up starving the thread pool as incomming connections are immediately blocked.
By using gh-ost on the extra port, MySQL locking will still behave the same, but MySQL will keep a dedicated thread for each gh-ost connection.
When doing this, it's important to inspect the extra-max-connections variable. Both Percona and Maria default to 1, so gh-ost may easily exceed with its threads.
An example local run using this
```
$ mysql -S /tmp/mysql_sandbox20393.sock -e "select @@global.port, @@global.extra_port"
+---------------+---------------------+
| @@global.port | @@global.extra_port |
+---------------+---------------------+
| 20393 | 30393 |
+---------------+---------------------+
./bin/gh-ost \
--initially-drop-ghost-table \
--initially-drop-old-table \
--assume-rbr \
--port="20395" \
--assume-master-host="127.0.0.1:30393" \
--max-load=Threads_running=25 \
--critical-load=Threads_running=1000 \
--chunk-size=1000 \
--max-lag-millis=1500 \
--user="gh-ost" \
--password="gh-ost" \
--database="test" \
--table="mytable" \
--verbose \
--alter="ADD mynewcol decimal(11,2) DEFAULT 0.0 NOT NULL" \
--exact-rowcount \
--concurrent-rowcount \
--default-retries=120 \
--panic-flag-file=/tmp/ghost.panic.flag \
--postpone-cut-over-flag-file=/tmp/ghost.postpone.flag \
--execute
```
2017-09-20 16:05:20 -04:00
Shlomi Noach
6362dbe51a
Clearer error message on config file read error
2017-09-10 08:20:01 +03:00
Shlomi Noach
a2847015d6
Merge branch 'master' into nm-refactor-migration-context
2017-09-10 08:08:39 +03:00
Shlomi Noach
8940649a32
Merge branch 'master' into checking-thresholds
2017-09-06 15:14:17 +03:00
Peng Sung
113e115c49
Update the main.go word spelling error
2017-09-06 15:12:09 +08:00
Shlomi Noach
78c1e0432f
Merge branch 'master' into checking-thresholds
2017-09-06 08:07:35 +03:00
Shlomi Noach
e2171e0162
Validating password length
2017-09-03 10:27:04 +03:00
Shlomi Noach
3437cf44c4
Validating shared key column types
2017-09-03 09:57:26 +03:00
Shlomi Noach
f2cd598376
Merge branch 'master' into autocommit
2017-09-03 07:55:13 +03:00
Christian Joudrey
2646a0a997
Remove unused acceptSignals in migrator
2017-09-02 21:59:09 -04:00
Shlomi Noach
a3ee81e116
Merge branch 'master' into autocommit
2017-08-30 11:40:56 +03:00
Shlomi Noach
10cafb71c0
gh-ost uses autocommit=1 for all connections
2017-08-30 11:38:22 +03:00
Nikhil Mathew
0a7e713e9f
Ensure cleanup happens, even on error
2017-08-28 15:53:47 -07:00
Nikhil Mathew
e4bb70df43
Use change from outbrain/golib PR
2017-08-28 15:19:05 -07:00
Nikhil Mathew
3b21f4db37
Close remaining DB connections
2017-08-28 14:05:15 -07:00
Shlomi Noach
c9f2da82f0
Merge branch 'master' into json-57-test
2017-08-27 08:01:02 +03:00
Shlomi Noach
2174588ebf
Supporting JSON types
2017-08-22 11:51:20 +03:00
Shlomi Noach
79e4f1cdbe
detect range end based on OFFSET
2017-08-21 08:12:41 +03:00
Nikhil Mathew
a1473dfb58
Stop infinite goroutines
2017-08-08 15:31:25 -07:00
Nikhil Mathew
7412f42371
Remove no longer needed acceptSignals func
2017-08-08 13:44:53 -07:00
Nikhil Mathew
b399533d4f
Simplify GoMySQLReader
2017-08-08 13:41:46 -07:00
Nikhil Mathew
982b8eede9
Refactor global migration context
2017-08-08 13:36:54 -07:00
Shlomi Noach
d6f1675f1b
Merge branch 'master' into interpolate-params
2017-07-26 10:01:41 +03:00
Shlomi Noach
d9a23a2212
interpolateParams=true: client side prepared statements
2017-07-25 09:21:30 +03:00
Arthur Neves
02359b2a79
Dont share the same context in the table name tests
2017-07-24 06:26:31 -04:00
Arthur Neves
843c74f960
Add a force-table-names command flag
...
usage example:
```
gh-ost ... --force-table-names=migration_tbl
```
The force-table-names flag will receive a table name and use that name
when creating the _del, _gho, _ghc tables.
For instance for that example above, the tables would be:
_migration_tbl_del
_migration_tbl_gho
_migration_tbl_ghc
[fixes #399 ]
2017-07-21 15:37:57 -04:00
Shlomi Noach
e2c14f5aba
Merge branch 'master' into dynamic-dml-batch-size
2017-07-20 16:47:22 +03:00
Shlomi Noach
260374aeb9
Merge branch 'master' into long-table-names
2017-07-20 16:46:41 +03:00
Shlomi Noach
de0ebcbbb0
documentation
2017-07-19 16:48:22 +03:00
Shlomi Noach
6da0b8af3d
Dynamic DML batch size; apadting buffer size to match
2017-07-19 16:44:18 +03:00
Shlomi Noach
39e73e37d0
testing context.TimestampOldTable
2017-07-19 15:22:46 +03:00
Shlomi Noach
168676bd9d
ghost/changelog/del table names support max characters limit
2017-07-19 15:14:03 +03:00
Shlomi Noach
4a3a5a1ce1
Merge branch 'master' into case-insensitive-columns
2017-07-16 06:51:39 +03:00
Shlomi Noach
effae08fbf
supporting change of column name-case
2017-07-12 12:33:32 +03:00
Shlomi Noach
e6dd2c90d2
responding to errors in rowcopy
2017-07-11 13:39:11 +03:00
Shlomi Noach
81403699b7
Merge branch 'master' into throttle-http-freno-aware
2017-07-05 10:16:56 +03:00
Shlomi Noach
a8932ee1b8
freno-aware http-throttler
2017-07-04 10:28:09 +03:00
Shlomi Noach
f9788a61ed
Merge branch 'master' into dml-batch-size-cap
2017-07-04 08:54:49 +03:00
Shlomi Noach
095863b774
dml-batch-size capped at 1000
2017-07-04 08:50:25 +03:00
Shlomi Noach
36bcc031ce
Merge branch 'master' into fix-variables-spelling
2017-06-29 20:53:49 +03:00
Shlomi Noach
e8b384e43c
supporting greater batch size
2017-06-28 19:35:55 +03:00
Shlomi Noach
cb3c7f3e20
Merge branch 'master' into touch-postpone-flag-file
2017-06-11 08:57:45 +03:00
Shlomi Noach
8da0f60582
fixed critical-load check for hibernation
2017-05-24 10:53:00 +03:00
Shlomi Noach
ad47f7c147
throttling just prior to leaving hibernation, so as to allow re-throttle checks to apply
2017-05-24 10:42:47 +03:00
Shlomi Noach
3955a6d67f
hibernate on critical-load
2017-05-24 08:32:13 +03:00
Shlomi Noach
aca288e88d
-postpone-cut-over-flag-file implies touching indicated file
2017-05-07 14:58:18 +03:00
Shlomi Noach
eaa3489fe0
Merge branch 'master' into server-report-coordinates
2017-05-01 07:50:13 +03:00
Shlomi Noach
79cee99f57
support for 'coordinates' command
2017-04-28 15:50:51 -07:00
Jacob Bednarz
0243e76b33
s/applyEnvironmentVairables/applyEnvironmentVariables
2017-04-24 07:45:21 +10:00
Shlomi Noach
e2a6b857d0
Merge branch 'master' into drop-add-same-column
2017-04-23 20:26:31 +03:00
Shlomi Noach
8a0f1413eb
dropped columns are not 'shared' and no data copy attempted for such columns
2017-04-23 08:38:35 +03:00
Shlomi Noach
85c498511e
parser recognizes DROP COLUMN tokens
2017-04-23 08:23:56 +03:00
Shlomi Noach
3092e9c5c0
error and nil checks on socket connection
2017-04-13 08:27:42 +03:00
Shlomi Noach
8405b0c3b8
Merge branch 'master' into close-sync-with-err
2017-03-28 08:01:18 +03:00
Shlomi Noach
7a3912da80
fixing binlog syncer double-close()
2017-03-27 08:33:06 +03:00
Shlomi Noach
c413d508cc
HEAD instead of GET
2017-03-26 13:12:56 +03:00
Shlomi Noach
23ce390d69
supporting throttle-http
2017-03-26 13:10:34 +03:00
Shlomi Noach
7517238bd3
Merge branch 'master' into fix-infinite-cutover-loop
2017-03-06 21:50:26 +02:00
Shlomi Noach
91521f9357
Merge branch 'master' into old-table-name-timestamp
2017-03-05 18:10:14 +02:00
Shlomi Noach
8b2c8fe86a
Merge branch 'master' into old-table-name-timestamp
2017-03-05 14:38:06 +02:00
Shlomi Noach
f11f6f978f
mitigating cut-over/write race condition
2017-02-24 14:48:10 -07:00
Shlomi Noach
06c909bd10
Validating table name length
2017-02-21 17:34:49 -07:00
Shlomi Noach
8db8e127eb
supporting --timestamp-old-table
2017-02-21 10:34:24 -07:00
Shlomi Noach
289ce46a2b
Merge branch 'master' into fix-infinite-cutover-loop
2017-02-16 14:16:32 +02:00
Shlomi Noach
2d5f8398d6
gomysql_reader.go adapted to api changes in go-mysql
2017-02-12 13:15:53 +02:00
Shlomi Noach
1acbca105d
applied gofmt
2017-02-09 09:09:35 +02:00
Shlomi Noach
109b82bb3a
Merge branch 'master' into master
2017-02-09 09:04:22 +02:00
Shlomi Noach
b289041ccb
Merge branch 'master' into fix-reappearing-throttled-reasons
2017-02-08 13:11:34 +02:00
Shlomi Noach
6ecf7cc0ee
Auto-merged master into interactive-command-question on deployment
2017-02-08 13:03:45 +02:00
Shlomi Noach
57409c2198
added mising slaveSQLRunning value
2017-02-08 12:24:44 +02:00
Ivan Groenewold
3511aa35df
improve output without --verbose option
2017-02-07 15:12:34 -03:00
Shlomi Noach
10edf3c063
Migration only starting after first replication lag metric collected
2017-02-07 12:13:19 +02:00
Shlomi Noach
91a3fd4a84
Merge branch 'master' into which-mysql-versions
2017-02-07 11:12:48 +02:00
Shlomi Noach
ed5dd7f049
Collecting and presenting MySQL versions of applier and inspector
2017-02-07 09:41:33 +02:00
Shlomi Noach
6fa32f6d54
cut-over failure on test-on-replica starts replication again
2017-02-07 09:31:52 +02:00
Shlomi Noach
9d47992d7e
testing/running on replica: gets lags via SHOW SLAVE STATUS
2017-02-02 11:18:07 +02:00
