官方online ddl
一、5.6版本online DDL
操作 | In Place | Rebuilds Table | Concurrent DML | Only Modifies Metadata | 說明 |
---|---|---|---|---|---|
add/create secondary index | yes | No | yes* | no | 當表上有FULLTEXT索引除外,需要鎖表,阻塞寫 |
drop index | yes | no | yes | yes | - |
add fulltext index | yes | no | no | no | - |
add primary key | yes* | yes | yes | no | 即使in-place,但需要copy data,不過效率比copy方式高 |
drop primary key | no | yes | no | no | 即使in-place,但需要copy data,不過效率比copy方式高 |
Dropping a primary key and adding another | Yes | Yes | Yes | No | - |
add column | yes | yes | yes* | no | 自增列阻塞寫 |
drop column | yes | yes | yes | no | - |
Rename a column | yes | no | yes* | yes | 只改變列名不改變型別才支援寫 |
Reorder columns | yes | yes | yes | no | - |
Set default value for a column | yes | no | yes | yes | - |
Change data type of column | no | yes | no | no | - |
Dropping the column default value | yes | no | yes | yes | - |
Changing the auto-increment value | yes | no | yes | no* | 修改時記憶體值不是資料檔案 |
Making a column NULL | yes | yes | yes | no | - |
Making a column NOT NULL | yes | yes | yes | no | - |
Modifying the definition of an ENUM or SET column | yes | no | yes | yes | - |
Adding a foreign key constraint | yes* | no | yes | yes | INPLACE只有在foreign_key_checks=off |
Dropping a foreign key constraint | yes | no | yes | yes | - |
Changing the ROW_FORMAT | yes | yes | yes | no | — |
Changing the KEY_BLOCK_SIZE | yes | yes | yes | no | — |
Convert character set | no | yes | no | no | - |
optimize table | yes* | yes | yes | no | 從5.6.17支援in-place,但當帶有fulltext index的表用copy table方式並且阻塞寫 |
alter table...engine=innodb | yes* | yes | yes | no | 從5.6.17支援in-place,當帶有fulltext index的表用copy table方式並且阻塞寫 |
Renaming a table | yes | no | yes | yes | - |
二、5.7版本online DDL(僅突出與5.6不同的地方,列出如下(未列出的同5.6))
操作 | In Place | Rebuilds Table | Concurrent DML | Only Modifies Metadata | 說明 |
---|---|---|---|---|---|
Renaming an index | yes | no | yes | yes | 5.7新增 |
Adding a SPATIAL index | Yes | No | No | No | 5.7新增 |
Extending VARCHAR column size | Yes | No | Yes | Yes | 5.7新增,只能在[0-255],[256-~]位元組區間擴大 |
1、擴充套件varchar長度測試
1.1)varchar從大變小
操作 | In Place | Rebuilds Table | Concurrent DML | Only Modifies Metadata | 說明 |
---|---|---|---|---|---|
varchar從大變小 | no | yes | no | no | 阻塞DML |
1.2) varchar從小變大
對於大小為0到255位元組的VARCHAR列,需要一個長度位元組來編碼該值。對於大小為256或更多的位元組的VARCHAR列,需要兩個長度位元組。
因此,in-place演算法更改表只支援將VARCHAR列的大小從0位元組增加到255位元組,或者從256位元組增加到更大的值。
in-place演算法不支援將VARCHAR列的大小從小於256位元組增加到等於或大於256位元組的大小,因為在這種情況下,所需長度位元組的數量從1變化到2,只能通過copy方式實現。
比如將VARARAR(255位元組)更改為VARCHAR(256位元組),只能copy演算法並且阻塞寫
-------------------測試---------------------
```
說明:採用utf8,如果存中文字元,一個字元需要3個位元組,因此255位元組對應最大字元數是varchar(85),也就是[0-255]對應varchar(0-85),[256-~]對應varchar(86-~)
表結構:
create table varchar_test(
c1 varchar(1) not null default '0'
);
1)採用online方式,擴大到85字元---支援
alter table varchar_test change c1 c1 varchar(85) not null default '0',ALGORITHM=INPLACE, LOCK=NONE;
Query OK, 0 rows affected (0.00 sec)
Records: 0 Duplicates: 0 Warnings: 0
2)採用online方式,擴大到86字元---不支援
alter table varchar_test change c1 c1 varchar(86) not null default '0',ALGORITHM=INPLACE, LOCK=NONE;
ERROR 1846 (0A000): ALGORITHM=INPLACE is not supported. Reason: Cannot change column type INPLACE. Try ALGORITHM=COPY.
3)採用copy方式
alter table varchar_test change c1 c1 varchar(86) not null default '0',ALGORITHM=copy;
Query OK, 1 row affected (0.00 sec)
Records: 1 Duplicates: 0 Warnings: 0
4)採用online方式,從86擴大到259字元---支援
alter table varchar_test change c1 c1 varchar(259) not null default '0',ALGORITHM=INPLACE, LOCK=NONE;
Query OK, 0 rows affected (0.00 sec)
Records: 0 Duplicates: 0 Warnings: 0
## 三、8.0版本online DDL(僅列出與5.7不同的地方)
Operation| Instant |In Place| Rebuilds Table| Permits Concurrent DML| Only Modifies Metadata|desc
---|---|---|---|---|---|---|
add column|yes*|yes|yes|yes*|no|自增列阻塞寫
modify index type|yes|yes|no|yes|yes|-
**3、實現原理**
目標表T1上MDL(SHARED_NO_WRITE),阻塞所有寫操作
判斷是否符合線上加欄位的條件,如符合,執行步驟3,否則按照官方方式來新增欄位。
建立跟目標表T1同構的臨時表frm檔案S1.frm(含新增欄位)
目標表T1的字典鎖升級為排它鎖,所有關於T1表的所有讀寫操作被阻塞
修改T1表InnoDB內部資料字典資訊,增加新欄位。
將S1.frm重新命名為T1.frm
釋放表T1的字典鎖
**2、支援instant的ddl**
Change index option
Rename table (in ALTER way)
SET/DROP DEFAULT
MODIFY COLUMN
Add/drop virtual columns
Add columns– We call this instant ADD COLUMN
**4、使用最新演算法instant條件:**
1)不能合併寫到其他不支援instant演算法的alter 語句後面;
alter table sbtest1 add index idx_2(k),add c5 varchar(10) not null default '0' ,ALGORITHM=instant;
ERROR 1845 (0A000): ALGORITHM=INSTANT is not supported for this operation. Try ALGORITHM=COPY/INPLACE.
2) 不支援before和after關鍵字,只能預設的加到最後一列;
alter table sbtest1 add c2 varchar(10) not null default '0';
Query OK, 0 rows affected (0.09 sec)
Records: 0 Duplicates: 0 Warnings: 0
alter table sbtest1 add c4 varchar(10) not null default '0' after id;
Query OK, 0 rows affected (26.42 sec)
Records: 0 Duplicates: 0 Warnings: 0
alter table sbtest1 add c5 varchar(10) not null default '0' after id,ALGORITHM=instant;
ERROR 1845 (0A000): ALGORITHM=INSTANT is not supported for this operation. Try ALGORITHM=COPY/INPLACE.
3)不支援ROW_FORMAT=COMPRESSED型別的表;
root:sbtest> show table status like 'sbtest1'\G
*************************** 1. row ***************************
Name: sbtest1
Engine: InnoDB
Version: 10
Row_format: Compressed
Rows: 9906340
Avg_row_length: 73
Data_length: 724033536
Max_data_length: 0
Index_length: 101171200
Data_free: 3145728
Auto_increment: 20000020
Create_time: 2018-10-21 15:48:22
Update_time: NULL
Check_time: NULL
Collation: utf8_general_ci
Checksum: NULL
Create_options: max_rows=100000000 row_format=COMPRESSED
Comment:
1 row in set (0.00 sec)
root:sbtest>
root:sbtest> alter table sbtest1 add c5 varchar(10) not null default '0',ALGORITHM=instant;
ERROR 1845 (0A000): ALGORITHM=INSTANT is not supported for this operation. Try ALGORITHM=COPY/INPLACE.
root:sbtest>
4)表上有FULLTEXT index,不支援instant演算法;
5)不支援臨時表加欄位;
6)如果表上存在大事務,instant也會被阻塞
## 六、官方online ddl限制
![](https://img2018.cnblogs.com/blog/818283/201811/818283-20181103182516815-346174374.png)
**1、大事務可能引起MDL鎖(即使是8.0.12 instant方式也是需要獲取MDL鎖的)**
session 1:
alter table sbtest1 ALGORITHM=INPLACE,drop column c2 ;
session2:寫入事務未提交
set autocommit=0;
begin;
insert into sbtest1(c) values("session2"); --當執行後,session 1將被阻塞,狀態有alter table -->Waiting for table metadata lock
session3:
insert into sbtest1(c) values("session2"); --被阻塞
結論發現:
dbadmin:sbtest> show processlist;
+----------+---------+-----------+--------+---------+------+---------------------------------+------------------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+----------+---------+-----------+--------+---------+------+---------------------------------+------------------------------------------------------+
| 78213439 | dbadmin | localhost | sbtest | Query | 0 | starting | show processlist |
| 78213440 | root | localhost | sbtest | Query | 1763 | Waiting for table metadata lock | alter table sbtest1 ALGORITHM=INPLACE,drop column c2 |
| 78213441 | dbadmin | localhost | sbtest | Query | 1373 | Waiting for table metadata lock | insert into sbtest1(c) values("darren") |
```
經過漫長的時間,發現session2 插入語句被回滾了(因為客戶端連線超過30分鐘斷開導致未提交的事務自動回滾),session 1和session 3執行成功。
2、online ddl無法暫停和進度監控
3、online ddl大表可能導致從庫延遲嚴重