Commit Graph

64 Commits

Author SHA1 Message Date
Dirkjan Bussink
40acde0222
Set the ServerName for TLS configuration (#988)
When TLS hostname validation used for the MySQL connection, the
ServerName property needs to be set so that it knows which name to
validate on the certificate.

Without this option and with InsecureSkipVerify set to false, validation
will error here with a fatal error otherwise:

```
FATAL tls: either ServerName or InsecureSkipVerify must be specified in the tls.Config
```
2021-06-17 14:51:43 +02:00
Shlomi Noach
c41823ecc9
All MySQL DBs limited to max 3 concurrent/idle connections #15 (#931)
* v1.1.0

* WIP: copying AUTO_INCREMENT value to ghost table
Initial commit: towards setting up a test suite

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* greping for 'expect_table_structure' content

* Adding simple test for 'expect_table_structure' scenario

* adding tests for AUTO_INCREMENT value after row deletes. Should initially fail

* clear event beforehand

* parsing AUTO_INCREMENT from alter query, reading AUTO_INCREMENT from original table, applying AUTO_INCREMENT value onto ghost table if applicable and user has not specified AUTO_INCREMENT in alter statement

* support GetUint64

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* minor update to test

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* adding test for user defined AUTO_INCREMENT statement

* Generated column as part of UNIQUE (or PRIMARY) KEY

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* skip analysis of generated column data type in unique key

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* All MySQL DBs limited to max 3 concurrent/idle connections

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
2021-05-27 20:00:58 +02:00
Tim Vaillancourt
8c4cd10d79
Merge branch 'master' into ipv6 2020-10-22 11:39:46 +02:00
Shlomi Noach
97f2d71616 fix unit tests 2020-02-16 12:52:26 +02:00
Shlomi Noach
9f5edc923e Support --mysql-timeout flag 2020-02-16 12:48:23 +02:00
Rafe Kettler
b4da7cfbfb Improve SSL support
Related issue: https://github.com/github/gh-ost/issues/521

 - Add --ssl-cert and --ssl-key options to specify SSL public/private
   key files
 - Allow combining --ssl-allow-insecure with other --ssl* flags.
   `mysql.RegisterTLSConfig` allows combining the corresponding
   parameters in the `tls.Config` it receives, so gh-ost should
   allow this. I found being able to pass --ssl-allow-insecure along
   with --ssl-ca, --ssl-cert, and --ssl-key useful in testing.
 - Use the same TLS config everywhere. Since the CLI only supports
   a single set of --ssl* configuration parameters, this should be
   fine -- `mysql.RegisterTLSConfig` documentation indicates the
   TLS config given will not be modified, so it can safely be used
   in many goroutines provided we also do not modify it. The previous
   implementation did not work when the TLS config was duplicated,
   which happens when gh-ost walks up the replication chain trying
   to find the master. This is because, when the config is duplicated,
   we must call `RegisterTLSConfig` again with the new config. This
   config is exactly the same, so it's easiest to side-step the issue
   by registering the TLS config once and using it everywhere.
2019-02-22 10:33:19 -08:00
Brandon Bodnar
5b0dfb009c Wireup allowing insecure ssl 2019-02-04 16:21:25 -06:00
Matt Belisle
79df0d1c5d Adding --ssl-insecure flag 2019-02-04 14:46:08 -06:00
Brandon Bodnar
c440112d44 Explain default setting for TLS param in DSN 2019-01-31 17:23:19 -06:00
Brandon Bodnar
dc599bb036 Remove unnecessary branching 2019-01-31 17:20:11 -06:00
Brandon Bodnar
23617f287f Add initial support for ssl encryption connections to database servers.
- Adding a command line option for users to enforce tls/ssl connections
  for the applier, inspector, and binlog reader.
- The user can optionally request server certificate verification through
  a command line option to specify the ca cert via a file path.
- Fixes an existing bug appending the timeout option to the singleton
  applier connection.
2019-01-31 15:22:03 -06:00
Shlomi Noach
97c1e72dbe
Merge branch 'master' into minor_changes 2018-09-17 08:58:04 +03:00
Shlomi Noach
959d1af211 support ipv6 without port 2018-09-16 11:52:59 +03:00
Shlomi Noach
a7cfaa4d33 added testing 2018-09-16 11:48:15 +03:00
Shlomi Noach
49b80df27b Parsing ipv6 addresses 2018-09-16 11:44:52 +03:00
MOON_CLJ
14eda7efe0 fix GetReplicationLag not used args 2018-05-23 13:13:25 +08:00
MOON_CLJ
21d455013e minor changes and typo correction 2018-05-23 13:13:25 +08:00
Shlomi Noach
db871b42c4 Support for GENERATED (aka virtual) columns 2018-05-22 12:36:52 +03:00
Shlomi Noach
710abd5a5b Fixed test 2018-03-12 16:20:46 +02:00
Shlomi Noach
2311df412d gbk support, simplified 2018-03-12 16:17:53 +02:00
Shlomi Noach
2209136258 rename parameters 2018-03-07 16:30:02 +02:00
ceshihao
330650baa9 rename includeMultibyteCharset -> includeRiskCharset, multibyteCharset -> riskCharset 2018-01-17 00:15:50 +08:00
ceshihao
346138772c add an option --include-multibyte-charset 2018-01-16 23:43:33 +08:00
Nikhil Mathew
ec6ceffbcb Pass in a migrationContext UUID for a migration specific connections cache 2017-12-14 15:53:29 -08:00
Nikhil Mathew
538833ea84
Merge branch 'master' into nm-refactor-migration-context 2017-11-14 10:29:49 -08:00
Nikhil Mathew
2ad65ee1d7 Refactor usages of GetDB to use mysql 2017-11-10 15:44:22 -08:00
Nikhil Mathew
bb4a0be4c6 Add GetDB to mysql.GetDB 2017-11-10 15:41:37 -08:00
Josh Soref
b3be5352c5 spelling: rotates 2017-11-08 00:56:14 +00:00
Josh Soref
ed35b75cc9 spelling: identified 2017-11-08 00:48:53 +00:00
Josh Soref
1ae6489afc spelling: detached 2017-11-08 00:46:26 +00:00
Josh Soref
84be0562a0 spelling: coordinate 2017-11-08 00:45:47 +00:00
Nikhil Mathew
84bdfdb1ad Cache DB connection pools on the migrationContext where applicable 2017-09-22 16:06:06 -07:00
Shlomi Noach
a2847015d6 Merge branch 'master' into nm-refactor-migration-context 2017-09-10 08:08:39 +03:00
Shlomi Noach
e2171e0162 Validating password length 2017-09-03 10:27:04 +03:00
Shlomi Noach
10cafb71c0 gh-ost uses autocommit=1 for all connections 2017-08-30 11:38:22 +03:00
Nikhil Mathew
0a7e713e9f Ensure cleanup happens, even on error 2017-08-28 15:53:47 -07:00
Nikhil Mathew
3b21f4db37 Close remaining DB connections 2017-08-28 14:05:15 -07:00
Shlomi Noach
d9a23a2212 interpolateParams=true: client side prepared statements 2017-07-25 09:21:30 +03:00
Shlomi Noach
06c909bd10 Validating table name length 2017-02-21 17:34:49 -07:00
Shlomi Noach
b289041ccb Merge branch 'master' into fix-reappearing-throttled-reasons 2017-02-08 13:11:34 +02:00
Shlomi Noach
57409c2198 added mising slaveSQLRunning value 2017-02-08 12:24:44 +02:00
Shlomi Noach
10edf3c063 Migration only starting after first replication lag metric collected 2017-02-07 12:13:19 +02:00
Shlomi Noach
baee4f69f9 fixing phantom throttle-control-replicas lag result 2017-01-29 10:18:39 +02:00
Shlomi Noach
fc831b0548 Reading replication lag via _changelog_ table, also on control replicas 2016-12-26 21:31:35 +02:00
Shlomi Noach
35eeb56032 improved log/error messages 2016-12-05 13:41:49 +01:00
rj03hou
8f02ab0fed check the slave status when find recursive find the master, so support if the dba using reset slave instead of reset slave all. 2016-12-05 19:42:16 +08:00
Shlomi Noach
bf92eec214 validating table structure on applier and migrator
- reading column list on applier
- comparing original table on applier and migrator, expecting exact column list
- or else bailing out
2016-10-20 11:29:30 +02:00
Shlomi Noach
c1a6773c02 better handling of --assume-master-host
separated logic and not even attempting to crawl topology
2016-10-11 16:42:19 +02:00
Shlomi Noach
791d963ea0 Character set recognition and manipulation
- Identifying textual characters sets; converting into specific type when applying dml events
- Refactored `ColumnsList`: introducing `Column` type
- Refactored `unsigned` handling, as part of `Column`
- `Column` type supports `convertArg()`: converting value of argument according to column data type
- DB URI attempts `utf8mb4,utf8,latin1` charsets in that order (first one to be recognized wins)
- Local tests filter by pattern
- Local tests append table schema on failure
- Local tests do not have postpone flag file
- Added character set local tests: `utf8`, `utf8mb4`, `latin1`
2016-09-07 14:24:11 +02:00
James Brown
b2fee9a037 detect IPv6 literals and wrap them in square brackets in GetDBUri 2016-08-05 16:54:26 -07:00