Commit Graph

633 Commits

Author SHA1 Message Date
wangzihuacool
c93060e743 restore connection charset to utf8mb4 2022-08-11 13:14:38 +08:00
wangzihuacool
113094bbd2 Convert character to bytes and insert into table using latin1 2022-08-03 11:43:18 +08:00
Tim Vaillancourt
ffe54f48ed
Enable more golang-ci linters (#1149) 2022-07-18 18:37:18 +02:00
Tim Vaillancourt
f527d63c86
Move .Kill() func from inspector to go/mysql (#1148) 2022-07-11 11:02:04 +02:00
dm-2
ed8c05c210
Merge branch 'master' into rm-go-os-process-go 2022-07-08 10:50:05 +01:00
Tim Vaillancourt
582d4cb115
Merge branch 'master' into rm-go-os-process-go 2022-07-08 00:21:25 +02:00
Tim Vaillancourt
30c868816c Remove unused go/os/process.go 2022-07-08 00:19:54 +02:00
dm-2
607ea8d3a9
Merge branch 'master' into changelog-migrated-state 2022-07-07 17:01:16 +01:00
Tim Vaillancourt
54db4174b3 Consolidate no-op states 2022-07-07 17:40:01 +02:00
Tim Vaillancourt
246800e053 Fix lint 2022-07-07 17:38:41 +02:00
dm-2
9c611bda50 switch to golang.org/x/term from deprecated golang.org/x/crypto/ssh/terminal module; go mod vendor 2022-07-07 16:38:22 +01:00
Tim Vaillancourt
d6d1d3b436
Merge branch 'master' into changelog-migrated-state 2022-07-07 17:35:59 +02:00
dm-2
f9fa4231d7
Merge branch 'master' into golang-ci-staticcheck-linter 2022-07-07 15:47:59 +01:00
dm-2
6d5d49616f
Merge branch 'master' into golang-ci-staticcheck-linter 2022-07-07 15:46:18 +01:00
dm-2
8eb9708755
Merge branch 'master' into changlog-table-comment 2022-07-07 15:34:30 +01:00
Tim Vaillancourt
e45357097f cleanup v5 2022-07-07 05:10:22 +02:00
Tim Vaillancourt
eead61cb6d cleanup v4 2022-07-07 05:05:37 +02:00
Tim Vaillancourt
c3593c94ff cleanup v3 2022-07-07 03:27:36 +02:00
Tim Vaillancourt
0303031fc2 cleanup v2 2022-07-07 03:22:53 +02:00
Tim Vaillancourt
0ec555badf Cleanup 2022-07-07 03:19:53 +02:00
Tim Vaillancourt
500fbefdb9 golang-ci: fix staticcheck linter warnings 2022-07-07 03:12:44 +02:00
Tim Vaillancourt
09bb5caf88
Merge branch 'master' into go-mysql-binlog-dead-code 2022-07-07 00:39:24 +02:00
Andrew Mason
b751499091
Cancel any row count queries before attempting to cut over (#846)
* Cancel any row count queries before attempting to cut over

Closes #830. Switches from using `QueryRow` to `QueryRowContext`, and
stores a context.CancelFunc in the migration context, which is called to
halt any running row count query before beginning the cut over.

* Make it threadsafe

* Kill the count query on the database side as well

* Explicitly grab a connection to run the count, store its connection id
* When the query context is canceled, run a `KILL QUERY ?` on that connection id

* Rewrite these to use the threadsafe functions, stop exporting the cancel func

* Update logger

* Update logger

Co-authored-by: Tim Vaillancourt <timvaillancourt@github.com>
Co-authored-by: Tim Vaillancourt <tim@timvaillancourt.com>
Co-authored-by: dm-2 <45519614+dm-2@users.noreply.github.com>
2022-07-07 00:23:23 +02:00
Tim Vaillancourt
0918bab29b
Add context/timeout to HTTP throttle check (#1131)
* Add context/timeout to HTTP throttle check

* Dont run `.GetThrottleHTTPInterval()` on every loop

* Update help message

* Var rename

* 2022

* Add timeout flag

* Add unix/tcp server commands, use ParseInt() for string->int64

* Var rename

* Re-check http timeout on every loop iteration

* Remove stale comment

* Make throttle interval idempotent

* var rename

* Usage grammar

* Make http timeout idempotent too

* Parse time.Duration once

* Move timeout to NewThrottler

* Help update

* Set User-Agent header

* Re-add newline

Co-authored-by: dm-2 <45519614+dm-2@users.noreply.github.com>
2022-07-06 23:56:07 +02:00
Tim Vaillancourt
0b066c16a5
Use switch statements for readability, simplify .NewGoMySQLReader() (#1135)
* Use `switch` statements for readability

* Simplify initBinlogReader()
2022-07-06 23:45:26 +02:00
Tim Vaillancourt
6bf32f2015
Ensure mysql rows responses are closed (#1132)
Co-authored-by: dm-2 <45519614+dm-2@users.noreply.github.com>
2022-07-06 23:32:15 +02:00
dm-2
b7f7e2c0b0
Merge branch 'master' into fix-issue-1117 2022-07-06 16:58:51 +01:00
Tim Vaillancourt
38e86e5791
Merge branch 'master' into fix-two-phase-commit-lost-data_v2 2022-06-25 00:37:12 +02:00
shaohk
87970f6312 fix(lost data in mysql two-phase commit): lost data in mysql two-phase commit 2022-06-24 11:38:05 +08:00
Tim Vaillancourt
de339dd042 Fix integer divide by zero panic in migrator 2022-06-23 03:37:18 +02:00
Tim Vaillancourt
387b32594e Var rename 2022-06-07 01:49:18 +02:00
Tim Vaillancourt
bf0c13b46d Add table comment to changelog 2022-06-07 01:47:51 +02:00
Tim Vaillancourt
eb801441c2 Remove unused Type field 2022-06-06 02:31:47 +02:00
Tim Vaillancourt
261e1f7820 Remove unused code in go/mysql/binlog.go 2022-06-06 02:29:01 +02:00
Tim Vaillancourt
ec199f1185 revert out of scope change 2022-06-05 00:06:41 +02:00
Tim Vaillancourt
af99b247f9 Add Migrated changelog event 2022-06-04 23:59:00 +02:00
Tim Vaillancourt
ed46138c06
Add golangci-lint CI action, fix gosimple, govet + unused lint errors (#1127)
* Add `golangci-lint`, fix `gosimple`, `govet` and `unused` linter complaints

* Go 1.16

* Update copyright dates
2022-05-31 21:23:39 +02:00
Tim Vaillancourt
5ea949f31e Remove end-of-life MySQL 5.5.x 2022-02-25 00:02:27 +01:00
dm-2
260e662197
Merge branch 'master' into inspector-connectionConfig-instanceKey-string 2022-02-24 10:34:19 +00:00
Tim Vaillancourt
1aa74bfe60 Fix needless fmt.Sprintf call in go/logic/inspector.go 2022-02-24 01:30:29 +01:00
Tim Vaillancourt
f6db40e4cd Use .String() for logging connection-config InstanceKey 2022-02-23 23:28:03 +01:00
dm-2
94ffadf101
Merge branch 'master' into customize_status_hook_interval 2022-02-07 11:51:42 +00:00
Arthur Schreiber
997c5b8bff Reduce the minimal chunk size from 100 to 10. 2022-02-07 11:23:39 +00:00
Tyler Knodell
fbb53f4b04 Add flag to customize the interval which the onStatus hook is called 2022-02-01 08:54:55 -07:00
dm-2
619d9d3f5e
Merge branch 'master' into documentation-update 2022-01-18 16:41:18 +00:00
dm-2
83413c339e Security: fix vulnerability where -database parameter accepts arbitrary DSN strings 2022-01-18 11:23:49 +00:00
John Nicholls
47fd51f804 Document critical-load-hibernate-seconds 2021-12-15 16:34:05 +00:00
Tim Vaillancourt
732a064231
Replace deprecated go-mysql library (#994) 2021-07-15 21:49:50 +02:00
Andrew Mason
6e1daf90ee
Check RowsAffected when applying DML events to get more accurate statistics (#844)
* Check RowsAffected when applying DML events to get more accurate statistics

Addresses #600.

When applying a DML event, check the RowsAffected on the `Result`
struct. Since all DML event queries are point queries, this will only
ever be 0 or 1. The applier then takes this value and multiplies by
the `rowsDelta` of the event, resulting in a properly-signed, accurate
row delta to use in the statistics.

If an error occurs here, log it, but do not surface this as an
actual error .. simply assume the DML affected a row and move on. It
will be inaccurate, but this is already the case.

* Fix import

* update wording to warning log message

Co-authored-by: Tim Vaillancourt <timvaillancourt@github.com>

Co-authored-by: Tim Vaillancourt <timvaillancourt@github.com>
2021-07-14 16:48:03 +02:00
Tim Vaillancourt
47d49c6b92
Add go mod (#935)
* Add a go.mod file

* run go mod vendor again

* Move to a well-supported ini file reader

* Remove GO111MODULE=off

* Use go 1.16

* Rename github.com/outbrain/golib -> github.com/openark/golib

* Remove *.go-e files

* Fix for `strconv.ParseInt: parsing "": invalid syntax` error

* Add test for '[osc]' section

Co-authored-by: Nate Wernimont <nate.wernimont@workiva.com>
2021-06-24 20:19:37 +02:00