Commit Graph

58 Commits

Author SHA1 Message Date
Shuode Li
13b5f48ca5 Remove useless code 2019-12-17 04:03:51 +00:00
Shuode Li
ae762694fb Support Azure Database for MySQL. 2019-12-17 03:47:14 +00:00
Rafe Kettler
b4da7cfbfb Improve SSL support
Related issue: https://github.com/github/gh-ost/issues/521

 - Add --ssl-cert and --ssl-key options to specify SSL public/private
   key files
 - Allow combining --ssl-allow-insecure with other --ssl* flags.
   `mysql.RegisterTLSConfig` allows combining the corresponding
   parameters in the `tls.Config` it receives, so gh-ost should
   allow this. I found being able to pass --ssl-allow-insecure along
   with --ssl-ca, --ssl-cert, and --ssl-key useful in testing.
 - Use the same TLS config everywhere. Since the CLI only supports
   a single set of --ssl* configuration parameters, this should be
   fine -- `mysql.RegisterTLSConfig` documentation indicates the
   TLS config given will not be modified, so it can safely be used
   in many goroutines provided we also do not modify it. The previous
   implementation did not work when the TLS config was duplicated,
   which happens when gh-ost walks up the replication chain trying
   to find the master. This is because, when the config is duplicated,
   we must call `RegisterTLSConfig` again with the new config. This
   config is exactly the same, so it's easiest to side-step the issue
   by registering the TLS config once and using it everywhere.
2019-02-22 10:33:19 -08:00
Brandon Bodnar
5b0dfb009c Wireup allowing insecure ssl 2019-02-04 16:21:25 -06:00
Matt Belisle
79df0d1c5d Adding --ssl-insecure flag 2019-02-04 14:46:08 -06:00
Brandon Bodnar
c440112d44 Explain default setting for TLS param in DSN 2019-01-31 17:23:19 -06:00
Brandon Bodnar
dc599bb036 Remove unnecessary branching 2019-01-31 17:20:11 -06:00
Brandon Bodnar
23617f287f Add initial support for ssl encryption connections to database servers.
- Adding a command line option for users to enforce tls/ssl connections
  for the applier, inspector, and binlog reader.
- The user can optionally request server certificate verification through
  a command line option to specify the ca cert via a file path.
- Fixes an existing bug appending the timeout option to the singleton
  applier connection.
2019-01-31 15:22:03 -06:00
Shlomi Noach
97c1e72dbe
Merge branch 'master' into minor_changes 2018-09-17 08:58:04 +03:00
MOON_CLJ
14eda7efe0 fix GetReplicationLag not used args 2018-05-23 13:13:25 +08:00
MOON_CLJ
21d455013e minor changes and typo correction 2018-05-23 13:13:25 +08:00
Shlomi Noach
db871b42c4 Support for GENERATED (aka virtual) columns 2018-05-22 12:36:52 +03:00
Shlomi Noach
710abd5a5b Fixed test 2018-03-12 16:20:46 +02:00
Shlomi Noach
2311df412d gbk support, simplified 2018-03-12 16:17:53 +02:00
Shlomi Noach
2209136258 rename parameters 2018-03-07 16:30:02 +02:00
ceshihao
330650baa9 rename includeMultibyteCharset -> includeRiskCharset, multibyteCharset -> riskCharset 2018-01-17 00:15:50 +08:00
ceshihao
346138772c add an option --include-multibyte-charset 2018-01-16 23:43:33 +08:00
Nikhil Mathew
ec6ceffbcb Pass in a migrationContext UUID for a migration specific connections cache 2017-12-14 15:53:29 -08:00
Nikhil Mathew
538833ea84
Merge branch 'master' into nm-refactor-migration-context 2017-11-14 10:29:49 -08:00
Nikhil Mathew
2ad65ee1d7 Refactor usages of GetDB to use mysql 2017-11-10 15:44:22 -08:00
Nikhil Mathew
bb4a0be4c6 Add GetDB to mysql.GetDB 2017-11-10 15:41:37 -08:00
Josh Soref
b3be5352c5 spelling: rotates 2017-11-08 00:56:14 +00:00
Josh Soref
ed35b75cc9 spelling: identified 2017-11-08 00:48:53 +00:00
Josh Soref
1ae6489afc spelling: detached 2017-11-08 00:46:26 +00:00
Josh Soref
84be0562a0 spelling: coordinate 2017-11-08 00:45:47 +00:00
Nikhil Mathew
84bdfdb1ad Cache DB connection pools on the migrationContext where applicable 2017-09-22 16:06:06 -07:00
Shlomi Noach
a2847015d6 Merge branch 'master' into nm-refactor-migration-context 2017-09-10 08:08:39 +03:00
Shlomi Noach
e2171e0162 Validating password length 2017-09-03 10:27:04 +03:00
Shlomi Noach
10cafb71c0 gh-ost uses autocommit=1 for all connections 2017-08-30 11:38:22 +03:00
Nikhil Mathew
0a7e713e9f Ensure cleanup happens, even on error 2017-08-28 15:53:47 -07:00
Nikhil Mathew
3b21f4db37 Close remaining DB connections 2017-08-28 14:05:15 -07:00
Shlomi Noach
d9a23a2212 interpolateParams=true: client side prepared statements 2017-07-25 09:21:30 +03:00
Shlomi Noach
06c909bd10 Validating table name length 2017-02-21 17:34:49 -07:00
Shlomi Noach
b289041ccb Merge branch 'master' into fix-reappearing-throttled-reasons 2017-02-08 13:11:34 +02:00
Shlomi Noach
57409c2198 added mising slaveSQLRunning value 2017-02-08 12:24:44 +02:00
Shlomi Noach
10edf3c063 Migration only starting after first replication lag metric collected 2017-02-07 12:13:19 +02:00
Shlomi Noach
baee4f69f9 fixing phantom throttle-control-replicas lag result 2017-01-29 10:18:39 +02:00
Shlomi Noach
fc831b0548 Reading replication lag via _changelog_ table, also on control replicas 2016-12-26 21:31:35 +02:00
Shlomi Noach
35eeb56032 improved log/error messages 2016-12-05 13:41:49 +01:00
rj03hou
8f02ab0fed check the slave status when find recursive find the master, so support if the dba using reset slave instead of reset slave all. 2016-12-05 19:42:16 +08:00
Shlomi Noach
bf92eec214 validating table structure on applier and migrator
- reading column list on applier
- comparing original table on applier and migrator, expecting exact column list
- or else bailing out
2016-10-20 11:29:30 +02:00
Shlomi Noach
c1a6773c02 better handling of --assume-master-host
separated logic and not even attempting to crawl topology
2016-10-11 16:42:19 +02:00
Shlomi Noach
791d963ea0 Character set recognition and manipulation
- Identifying textual characters sets; converting into specific type when applying dml events
- Refactored `ColumnsList`: introducing `Column` type
- Refactored `unsigned` handling, as part of `Column`
- `Column` type supports `convertArg()`: converting value of argument according to column data type
- DB URI attempts `utf8mb4,utf8,latin1` charsets in that order (first one to be recognized wins)
- Local tests filter by pattern
- Local tests append table schema on failure
- Local tests do not have postpone flag file
- Added character set local tests: `utf8`, `utf8mb4`, `latin1`
2016-09-07 14:24:11 +02:00
James Brown
b2fee9a037 detect IPv6 literals and wrap them in square brackets in GetDBUri 2016-08-05 16:54:26 -07:00
Shlomi Noach
e900dae2e9 More informative information upon control-replicas lagging 2016-07-27 09:59:46 +02:00
Shlomi Noach
690e046c51 adding --allow-master-master 2016-06-22 10:38:13 +02:00
Shlomi Noach
62b8a897e3 Retries, better visibility, documentation
- Rowcopy time is bounded by copy end-time
- Retries are configurable via `--default-retries` (default: `60`)
- `migrator` notes the hostname
- `applier` and `inspector` note `impliedKey` (`@@hostname` and `@@port`)
- Added lots of code comments
- Adding documentation for "triggerless design"
2016-06-19 17:55:37 +02:00
Shlomi Noach
23cb8ea7e9 Throttling & critical load
- Added `--throttle-query` param (when returns > 0, throttling applies)
- Added `--critical-load`, similar to `--max-load` but implies panic and quit
- Recoded *-load as `LoadMap`
- More info on *-load throttle/panic
- `printStatus()` now gets printing heuristic. Always shows up on interactive `"status"`
- Fixed `change column` (aka rename) handling with quotes
- Removed legacy `mysqlbinlog` parser code
- Added tests
2016-06-18 21:12:07 +02:00
Shlomi Noach
9b54d0208f - Handling gomysql.replication connection timeouts: reconnecting on last known position
- `printStatus()` takes ETA into account
- More info around `master_pos_wait()`
2016-05-19 15:11:36 +02:00
Shlomi Noach
9d055dbda7 renaming to gh-ost 2016-05-16 11:09:17 +02:00