Commit Graph

Select branches

Hide Pull Requests

abort-missing-backend-tables

adaptive-eta

add-dry-run-env

add-tls-support

arthur/fix-incompatibility

cenkore_iss909

changelog-state

const_custom_type

dependabot/go_modules/golang.org/x/crypto-0.1.0

dependabot/go_modules/golang.org/x/net-0.7.0

dependabot/go_modules/golang.org/x/sys-0.1.0

dependabot/go_modules/golang.org/x/text-0.3.8

dev-get-started

dev_copyrow_hangs

devel-docs-start

enum-to-varchar

eta-ewma

exponential-backoff

fClose-Fix

gbk_encoding

gh-ost/fix-inspector-column-types

go-mod

go-mysql-updates-201709

guineveresaenger-socket-file-doc

hidden-password-cmdline

latin1-utf8mb4

master

merge

minor_changes

mysql8-rename-table

no-lock-in-share-mode

rafe/ssl-crt-key

release-1-1-4

release-1.1.5

resurrect

row-copy-by-app

row-image-minimal

rowcopy-contrib

safe-gtid-inspector-same-applier

safe-socket-file-length

sanity-uptime

skip-strict-mode

socket-file-doc

update-dependabot-103d88a8

update-dependabot-58b4cab8

update-dependabot-b523e386

update_docs_throttle_http_flags

utf8-in-latin1

wagnerm/docs

zmoazeni/bump-localtests-default-retries

zmoazeni/upgrade-to-go-1.12

#10

#100

#1002

#1003

#1003

#1004

#1004

#1005

#1006

#101

#1011

#1018

#1019

#1020

#1020

#1024

#1024

#1027

#103

#1033

#1033

#1040

#105

#1054

#1059

#106

#1060

#1061

#1066

#107

#1076

#1078

#1079

#108

#1083

#1087

#1089

#109

#1090

#1095

#1096

#1097

#1100

#1101

#1108

#111

#112

#1120

#1120

#1124

#1124

#1125

#1125

#1126

#1127

#1129

#1131

#1132

#1134

#1135

#1136

#1137

#1138

#1139

#1140

#1141

#1143

#1144

#1145

#1146

#1147

#1148

#1149

#115

#1151

#1153

#1153

#1154

#1156

#1158

#116

#1160

#1161

#1162

#1163

#1163

#1164

#1165

#1166

#1168

#117

#1170

#1170

#1172

#1173

#1174

#1175

#1177

#1177

#1178

#1179

#1180

#1181

#119

#1190

#1192

#1193

#1194

#120

#1200

#1201

#1202

#1203

#1203

#1206

#121

#1213

#1213

#122

#1224

#1224

#1226

#1226

#1227

#1229

#1229

#123

#1231

#1231

#1232

#1232

#1236

#1236

#1237

#1237

#124

#1245

#1245

#1246

#1246

#125

#1257

#1257

#1258

#1258

#126

#1260

#1260

#1261

#1261

#1269

#1269

#1279

#128

#1280

#1280

#130

#131

#131

#134

#134

#137

#139

#14

#140

#140

#143

#145

#145

#147

#148

#152

#155

#156

#158

#158

#16

#160

#165

#165

#170

#171

#172

#173

#174

#175

#176

#177

#18

#181

#182

#183

#185

#186

#189

#19

#190

#191

#192

#196

#199

#2

#20

#200

#204

#206

#206

#208

#21

#211

#213

#217

#218

#219

#22

#220

#221

#222

#223

#227

#23

#231

#232

#233

#233

#234

#236

#238

#24

#240

#241

#248

#248

#249

#251

#252

#253

#260

#263

#264

#265

#265

#266

#269

#27

#274

#277

#279

#28

#281

#282

#283

#286

#286

#289

#29

#291

#293

#294

#295

#296

#297

#298

#299

#301

#303

#303

#305

#306

#309

#31

#310

#315

#316

#319

#321

#324

#325

#327

#330

#330

#333

#334

#337

#339

#34

#340

#343

#343

#345

#345

#347

#349

#35

#351

#352

#355

#358

#36

#363

#364

#365

#366

#367

#368

#37

#370

#372

#373

#374

#375

#376

#379

#38

#385

#386

#387

#389

#392

#393

#4

#40

#401

#404

#405

#407

#407

#409

#409

#41

#411

#413

#415

#416

#416

#419

#424

#426

#427

#43

#433

#438

#44

#440

#445

#447

#45

#451

#452

#453

#455

#456

#457

#458

#459

#46

#460

#462

#47

#471

#472

#473

#478

#478

#479

#481

#487

#488

#489

#49

#490

#491

#494

#498

#499

#500

#503

#503

#504

#504

#508

#51

#510

#510

#514

#515

#516

#517

#518

#52

#520

#524

#525

#526

#528

#53

#533

#535

#536

#539

#54

#541

#546

#547

#547

#55

#551

#553

#556

#557

#559

#559

#56

#560

#562

#563

#564

#564

#565

#567

#568

#575

#578

#58

#581

#582

#585

#586

#587

#587

#588

#589

#59

#590

#592

#595

#599

#6

#60

#606

#607

#607

#61

#611

#612

#612

#613

#613

#615

#615

#616

#63

#633

#634

#64

#640

#641

#644

#648

#650

#651

#652

#655

#656

#659

#66

#661

#663

#664

#665

#666

#667

#668

#669

#67

#670

#671

#672

#673

#674

#675

#68

#680

#682

#684

#686

#689

#69

#692

#693

#694

#695

#696

#699

#70

#701

#703

#704

#704

#705

#706

#706

#707

#71

#710

#712

#715

#716

#717

#718

#718

#720

#723

#724

#726

#73

#731

#731

#732

#732

#736

#738

#74

#743

#745

#75

#750

#752

#754

#755

#759

#76

#77

#772

#774

#775

#776

#777

#778

#78

#780

#780

#787

#787

#788

#789

#79

#790

#791

#792

#793

#795

#796

#798

#798

#8

#802

#803

#805

#807

#81

#810

#817

#818

#819

#820

#823

#824

#83

#831

#833

#835

#836

#837

#837

#84

#843

#843

#844

#846

#85

#858

#858

#86

#861

#862

#862

#864

#865

#866

#867

#867

#87

#872

#873

#873

#876

#877

#878

#879

#883

#883

#888

#893

#899

#899

#9

#90

#904

#904

#905

#906

#908

#908

#91

#91

#910

#911

#912

#912

#915

#918

#918

#919

#92

#921

#922

#93

#930

#930

#931

#932

#934

#935

#936

#94

#940

#941

#948

#950

#950

#951

#954

#958

#962

#963

#964

#965

#966

#967

#968

#969

#969

#97

#970

#979

#98

#982

#985

#985

#986

#987

#987

#988

#990

#993

#994

#995

#999

#999

revert-pr-915

v0.7

v0.7.1

v0.7.16

v0.7.2

v0.8.2

v0.8.3

v0.8.4

v0.8.5

v0.8.7

v0.8.8

v0.9.2

v0.9.6

v0.9.7

v0.9.8

v0.9.9

v1.0.1

v1.0.10

v1.0.11

v1.0.13

v1.0.14

v1.0.17

v1.0.18-experimental

v1.0.2

v1.0.20

v1.0.21

v1.0.28

v1.0.30

v1.0.32

v1.0.34

v1.0.35

v1.0.36

v1.0.40

v1.0.42

v1.0.44

v1.0.45

v1.0.46

v1.0.47

v1.0.48

v1.0.49

v1.0.50-beta

v1.0.6

v1.0.7

v1.0.8

v1.0.9

v1.1.0

v1.1.1

v1.1.2

v1.1.3

v1.1.4

v1.1.5

Mono Color

• ec34a5ef75 master_pos_wait is now OK to return NULL. We only care if it returns with -1 Shlomi Noach 2016-05-18 15:08:47 +0200
• 079f1c0f98 Merge pull request #40 from github/fix-single-row-copy Shlomi Noach 2016-05-18 14:53:52 +0200
• 9f56a84b57 Fixing single-row table migration - BuildUniqueKeyRangeEndPreparedQuery supports includeRangeStartValues argument - applier sends this.migrationContext.GetIteration() == 0 as argument Shlomi Noach 2016-05-18 14:53:09 +0200
• 45371d9374 Merge pull request #36 from github/master-pos-wait-fix Shlomi Noach 2016-05-18 12:21:22 +0200
• 55e463350b Merge pull request #38 from github/credentials-config-cli Shlomi Noach 2016-05-17 15:37:04 +0200
• df0a7513f5 - user/password provided in CLI override those in config file - user no longer defaults to . - config is now part of Context, and is protected by mutex Shlomi Noach 2016-05-17 15:35:44 +0200
• fbfe0c71ff Merge pull request #37 from github/postpone-swap-tables-flag-file v0.7.2 Shlomi Noach 2016-05-17 14:42:12 +0200
• 879b2b425e - Support for --postpone-swap-tables-flag-file: while this file exists, final table swap does not take place, and the ghost table keeps being synchronized - Fixed version printing - rowCopyCompleteFlag is a hint that allows us to escape the infinite loop of rowcopy once we are sure we have reached the end Shlomi Noach 2016-05-17 14:40:37 +0200
• 065d9c40ec some messagages are now Info instead of Debug Shlomi Noach 2016-05-17 11:57:43 +0200
• 9a3c607665 Merge pull request #35 from github/app-version Shlomi Noach 2016-05-17 11:51:55 +0200
• 41b0a4f317 supporting --version Shlomi Noach 2016-05-17 11:51:21 +0200
• c6a371e6db Merge pull request #34 from github/rename-gh-ost v0.7.1 Shlomi Noach 2016-05-16 11:11:55 +0200
• 21f6ae9dca renaming to gh-ost Shlomi Noach 2016-05-16 11:10:12 +0200
• 9d055dbda7 renaming to gh-ost Shlomi Noach 2016-05-16 11:09:17 +0200
• 92d09dbcf6 Merge pull request #31 from github/fix-out-of-order-dml-apply Shlomi Noach 2016-05-16 11:03:50 +0200
• 1e10f1f29e Solved various race conditions: - Operation would terminate after events lock noticed but before applying all events: race condition where the event would be captured asynchronously. The event is now handled sequentially with the DML events, hence now safe. - Multiple rowcopy operations would still write to rowCopyComplete channel. This is still the case, but now we only wait for the first and then just flush (read and discard) any others, to avoid blocking - Events DML listener is only added after table creation: the problem was that with very busy tables, the events func buffer would fill up, and the "tables-created" event would be blocked. - waitForEventsUpToLock() unifies the waiting on all variants of complete-migration - With --test-on-replica, now stopping replication "nicely", using master_pos_wait() - With --test-on-replica, not throttling on replication after replication is stopped (duh) - More debug output Shlomi Noach 2016-05-16 11:03:15 +0200
• 134bf385fd initial, simple solution to our-of-order applying of DML events Shlomi Noach 2016-05-05 17:14:55 +0300
• 619be65524 Merge pull request #29 from github/status-eta v0.7 Shlomi Noach 2016-05-05 09:19:11 +0300
• 6528010742 Adding ETA starting at 2% progress Shlomi Noach 2016-05-05 09:18:19 +0300
• ee673e2f2e Merge pull request #28 from github/fix-statistics-query Shlomi Noach 2016-05-04 09:50:31 +0300
• 800c1109b0 fixed statistics query: getting the correct column names by unique key Shlomi Noach 2016-05-04 09:50:00 +0300
• dfdac964f1 Merge pull request #27 from github/exact-rowcount Shlomi Noach 2016-05-04 08:26:17 +0300
• 74d8b06db1 exact-rowcount implices updating number of rows as we make progress Shlomi Noach 2016-05-04 08:23:34 +0300
• 567cd32ef0 Merge pull request #24 from github/initially-dropping-tables Shlomi Noach 2016-05-03 12:58:35 +0300
• bd741371d0 Merge pull request #23 from github/cnf-file Shlomi Noach 2016-05-03 12:55:59 +0300
• 36905d82e3 - supporting --initially-drop-old-table - supporting --initially-drop-ghost-table - validating existence of old and ghost before beginning operation Shlomi Noach 2016-05-03 12:55:17 +0300
• 627e412b6b fixed password assignment Shlomi Noach 2016-05-03 11:56:53 +0300
• 86fd2b617a initial support for config file Shlomi Noach 2016-05-03 10:28:48 +0300
• 93ac641455 Merge pull request #22 from github/throttle-control-replicas Shlomi Noach 2016-05-01 21:37:12 +0300
• 07063a4181 - added throttle-control-replicas flag, a list of control replicas - when --test-on-replica, the tested replica is implicitly a control replica - added replication-lag-query, an alternate query to SHOW SLAVE STATUS to get replication lag - throttling takes both the above into consideration Shlomi Noach 2016-05-01 21:36:36 +0300
• 128658579a Merge pull request #21 from github/lossless-swap-tables Shlomi Noach 2016-05-01 21:32:46 +0300
• 421ab0fc83 woohoo, logic complete - Introduced SwapTablesTimeoutSeconds; RENAME is limited by this timeout - If RENAME fails (due to the above), we throttle and retry - SwapTablesAtomic() sets lock_wait_timeout and notifies with connection id - GrabVoluntaryLock() intentionally grabs (and later releases) voluntary lock. It notifies when it is taken and awaits instructions as for when it could be released. - IssueBlockingQueryOnVoluntaryLock() does what it says. It notifies with its connection_id so that it can be easily traced - stopWritesAndCompleteMigrationOnMasterViaLock() does the thang. Oh dear this was agonizing and the code is a pain to look at, though under the limitations I do believe it is as clean as I could hope for. Shlomi Noach 2016-04-22 19:46:34 -0700
• 1ed1b0d156 - quick-and-bumpy-swap-tables uses quicker swap tables, at the expense of a period where the table does not exist (non atomic renames) - refactored lock-and-swap code, in preparation for atomic swap Shlomi Noach 2016-04-22 13:41:20 -0700
• 54c6d059b5 - quick-and-bumpy-swap-tables uses quicker swap tables, at the expense of a period where the table does not exist (non atomic renames) - refactored lock-and-swap code, in preparation for atomic swap Shlomi Noach 2016-04-22 13:18:56 -0700
• fbe226ebb2 Merge pull request #20 from github/noop-replica-and-stuff Shlomi Noach 2016-04-19 04:26:22 -0700
• 3c85298b77 - Better, fewer NOOP checks around the code - Keeping track of TotalDMLEventsApplied Shlomi Noach 2016-04-19 04:25:32 -0700
• 4efbfd6e0f Merge pull request #19 from github/oops-leftovers Shlomi Noach 2016-04-18 10:59:58 -0700
• 9dce88e6c0 oops, leftover file Shlomi Noach 2016-04-18 10:59:34 -0700
• bb264fdc2b Merge pull request #18 from github/ongoing-initial-work-3 Shlomi Noach 2016-04-18 10:58:59 -0700
• eeffa701d6 - Added ok-to-drop-table flag - Added switch-to-rbr flag; applying binlog format change if needed - Using dedicated db instance for locking & renaming on applier (must be used from within same connection) - Heartbeat now uses time.RFC3339Nano - Swap tables works! Caveat: short table outage - --test-on-replica works! - retries: using panicAbort: from any goroutine, regardless of context, it is possible to terminate the operation - Reintroduced changelog events listener on streamer. This is the correct implementation. Shlomi Noach 2016-04-18 10:57:18 -0700
• 75c3fe0bee Merge pull request #16 from github/ongoing-initial-work-2 Shlomi Noach 2016-04-14 13:39:48 +0200
• a4ee80df13 - Building and applying queries from binlog event data! - INSERT, DELETE, UPDATE statements - support for --noop - initial support for --test-on-replica. Verifying against --allow-on-master - Changelog events no longer read from binlog stream, because reading it may be throttled, and we have to be able to keep reading the heartbeat and state events. They are now being read directly from table, mapping already-seen-events to avoid confusion Changlelog listener pools table in 2*frequency of heartbeat injection Shlomi Noach 2016-04-14 13:37:56 +0200
• 0d25d11b40 added types_test Shlomi Noach 2016-04-11 19:06:47 +0200
• 04525887f3 - Throttling-check is now an async routine running once per second - Throttling variables protected by mutex - Added --throttle-additional-flag-file: operation pauses when this file exists; hint: keep default, use for throttling multiple gh-osc operations - ColumnList is not a struct which contains ordinal mapping - More implicit write changelog + audit changelog - builder now builds DELETE and INSERT queries from data it will eventually get from DML event - Sanity check for binlog_row_image - Restarting replication to be sure binlog settings apply - Prepare for accepting SIGHUP (reloading configuration) Shlomi Noach 2016-04-11 17:27:16 +0200
• 80163b35b6 extracted on-throttle functions outside loop Shlomi Noach 2016-04-08 14:44:36 +0200
• a1a34b8150 ongoing development: - accepts --max-load - accepts multiple conditions in --max-load - throttle includes reason - chunk-size sanity check - change log state writes both in appending (history) mode and in replacing (current) mode - more atomic checks - inspecting ghost table columns, unique key - comparing unique keys between tables; sanity - intersecting columns between tables - prettify status - refactored throttle() and retries() Shlomi Noach 2016-04-08 14:35:06 +0200
• 4652bb7728 Merge pull request #14 from github/ongoing-initial-work Shlomi Noach 2016-04-08 10:35:57 +0200
• 75f68c0752 - row copy and row events are now handled by a single routine which prioritizes events over rowcopy - Supporting --throttle-file-flag - Printing status - Supporting transactional table syntax - code cleanup; refactoring - proper use of atomic where required - iterations are in changelog (erm... maybe too much) - LOCK TABLES, UNLOCK TABLES working Shlomi Noach 2016-04-08 10:34:44 +0200
• 0e7b23e6fe - Creating an populating Changelog table - Using heartbeat - Throttling works based on heartbeat - Refactored binlog_reader stuff. Now streaming events (into golang channel, which makes for nice buffering and throttling) - Binlog table listeners work - More Migrator logic; existing logic for waiting on state events (e.g. TablesCreatedState) Shlomi Noach 2016-04-07 15:57:12 +0200
• 4dd5a93ed7 - More work on go-mysql reader: now properly notes binlog coordinates as it makes progress - Properly creating multiple BinlogEntry-ies when reading multi statement binlog event (Still todo: transactions) Shlomi Noach 2016-04-06 18:44:54 +0200
• 5deff2adb6 - working POC of row-copy iteration cycle - initial work on table columns - initial work on events streamer Shlomi Noach 2016-04-06 13:05:58 +0200
• f771016bd5 . Shlomi Noach 2016-04-06 13:05:21 +0200
• d8fefb3d6f exploded args on range query building; iteration works Shlomi Noach 2016-04-05 19:50:49 +0200
• 3583ab5dc5 beginning support for ranges and iteration. Still WIP Shlomi Noach 2016-04-05 09:14:22 +0200
• ea0906f4e5 reading table (range) min/max values, right now according to hardcoded unique key Shlomi Noach 2016-04-04 18:19:46 +0200
• 937491674c adding applier, instance_key, instance_key_map Shlomi Noach 2016-04-04 15:30:49 +0200
• cf87d16044 detecting master (includes sanity checks). Introducing Applier. Creating and altering ghost table Shlomi Noach 2016-04-04 15:29:02 +0200
• bba352922a Merge pull request #10 from github/initial-db-access Shlomi Noach 2016-04-04 12:29:35 +0200
• c75cd998fb a bunch of 'inspector' initial tests on the replica Shlomi Noach 2016-04-04 12:27:51 +0200
• 1b8c5ff286 preparing for db access Shlomi Noach 2016-04-01 16:05:54 +0200
• f5b276415a initial work on context Shlomi Noach 2016-04-01 16:05:44 +0200
• b461c320cc added BuildUniqueKeyRangeEndPreparedQuery() Shlomi Noach 2016-04-01 16:05:17 +0200
• f0e37cde1c added vendor packages Shlomi Noach 2016-04-01 16:04:27 +0200
• 5e5b868c5c Merge pull request #9 from github/build-range-prepared Shlomi Noach 2016-04-01 13:56:33 +0200
• 41c3d15963 building prepared queries (with positional params) Shlomi Noach 2016-04-01 13:55:51 +0200
• 9b368a7720 Merge pull request #8 from github/sql-queries-manipulations Shlomi Noach 2016-04-01 13:54:20 +0200
• 20a74d578d Merge pull request #6 from github/experimenting-go-mysql Shlomi Noach 2016-04-01 13:54:07 +0200
• 39ebc75c43 initial work on sql query building Shlomi Noach 2016-04-01 13:36:56 +0200
• 8f3d13e071 still experimenting, yes? go-mysql binlog parser looks good Shlomi Noach 2016-03-30 15:43:40 +0200
• e3210a9fa2 Merge pull request #4 from github/parsing-mysqlbinlog Shlomi Noach 2016-03-30 10:25:59 +0200
• 30a472f741 begin work on positional columns. Incomplete Shlomi Noach 2016-03-24 22:27:39 +0100
• 96a8fd50c3 minor refactoring; fixed license comments Shlomi Noach 2016-03-24 15:11:56 +0100
• fb04eb232f simplified appendBinlogEntry() Shlomi Noach 2016-03-24 14:52:49 +0100
• 8aa6a9750c adjusted automaton to accept multi-statement entries; added tests Shlomi Noach 2016-03-24 14:25:52 +0100
• 1768b55b3b mysqlbinlog_reader is now a simple state machine Shlomi Noach 2016-03-23 15:25:45 +0100
• e088a3abac Merge pull request #2 from github/initial-code Shlomi Noach 2016-03-23 13:53:30 +0100
• e36bb51b69 initial seems-to-be-working parser for binary logs, which reads log pos, end log pos, statement type, schema and table. Performs some validations along the way. Reads in chunks of up to 32M per chunk Shlomi Noach 2016-03-23 12:40:17 +0100
• 1576119576 adding build file Shlomi Noach 2016-03-23 12:39:24 +0100
• 09046c8f6e initial binlog package Shlomi Noach 2016-03-22 15:12:51 +0100
• bb1c6dae30 adding initial bunch of code Shlomi Noach 2016-03-21 15:57:01 +0100
• 295d98c5b2 Initial commit Shlomi Noach 2016-03-21 11:08:52 +0100