005: 存儲引擎
阿新 • • 發佈:2017-11-24
acl 指針 disk mailto ole 內部 update 性能 separate
存儲引擎
1.MySQL上支持的存儲引擎
mysql> show engines;
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine | Support | Comment | Transactions | XA | Savepoints |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| CSV | YES | CSV storage engine | NO | NO | NO |
| MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
| MyISAM | YES | MyISAM storage engine | NO | NO | NO |
| BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO |
| InnoDB | DEFAULT | Supports transactions, row-level locking, and foreign keys | YES | YES | YES |
| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO |
| ARCHIVE | YES | Archive storage engine | NO | NO | NO |
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
| FEDERATED | NO | Federated MySQL storage engine | NULL | NULL | NULL |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec)
2.存儲引擎的概念
用來處理數據庫的相關CRUD操作
每個數據庫都有存儲引擎,只是MySQL比較強調存儲引擎的概念。
3. MySQL存儲引擎
- 官方存儲引擎
- MyISAM
InnoDB
-- 推薦;其他引擎已經體停止維護和開發- Memory
- Federated
- CSV
- Archive
- 第三方存儲引擎
- TokuDB -- 開源,適合插入密集型
- InfoBright -- 商業,開源版本有數據量限制。屬於列存儲,面向OLAP場景
Spider
第三方存儲引擎在特定場合下比較適合,除此之外,都應該使用InnoDB
3.存儲引擎之MyISAM
3.1MyISAM特點
- MySQL5.1版本之前的默認存儲引擎
- 堆表數據結構
- 表鎖設計
- 支持數據靜態壓縮
- 不支持事物
- 數據容易丟失
- 索引容易損壞
- 唯一優點
- 數據文件可以直接拷貝到另一臺服務器使用
- 現在MySQL中還有用MyISAM的表,
-主要是歷史原因。數據庫文件以MY
開頭的基本都是MyISAM的表- 部分如User,DB等系統表(MyISAM引擎),可以直接拷貝,比較方便
- 性能好,或者存儲小
不是
MyISAM的優點,也不是存在的原因
3.2 MyISAM文件組成
frm
表結構文件MYI
索引文件MYD
數據文件- 數據文件是堆表數據結構,堆是無序數據的集合
MYI
中的葉子節點,指向MYD
中的數據頁- 當數據移動到頁外時,需要修改對應指針
3.3 myisamchk
- myisamchk 最好是關閉數據庫,然後修復
- 不停機需要鎖住表
lock table sysbench.test_log_copy read
再修復
mysql> show create table test_log_copy\G;
*************************** 1. row ***************************
Table: test_log_copy
Create Table: CREATE TABLE `test_log_copy` (
`scenario` varchar(30) NOT NULL DEFAULT ‘‘ COMMENT ‘測試場景‘,
`server_name` varchar(15) NOT NULL COMMENT ‘主機名‘,
`test_type` varchar(15) NOT NULL COMMENT ‘read-only,read-write,insert等‘,
`sb_threads` int(11) NOT NULL DEFAULT ‘0‘ COMMENT ‘sysbench 測試線程‘,
`create_time` datetime DEFAULT NULL COMMENT ‘開始時間‘,
`done_time` datetime DEFAULT NULL COMMENT ‘完成時間‘,
`server_load` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘以當前線程測試完後立刻記錄15分鐘負載值‘,
`request_read` int(11) NOT NULL DEFAULT ‘0‘,
`request_write` int(11) NOT NULL DEFAULT ‘0‘,
`transactions_per_second` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘QPS‘,
`request_per_second` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘TPS‘,
`95_pct_time` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘單位毫秒‘,
KEY `createtime` (`create_time`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
1 row in set (0.00 sec)
ERROR:
No query specified
[root@localhost-m(252) /r2/soft/dbtest/mysql-5.7.18/mysqldata]# myisamchk sysbench/test_log_copy.MYI
Checking MyISAM file: sysbench/test_log_copy.MYI
Data records: 60 Deleted blocks: 0
- check file-size
- check record delete-chain
- check key delete-chain
- check index reference
- check data record references index: 1
- check record links
myisamchk
通過掃描MYD文件來重建MYI文件;如果MYD文件中某條記錄有問題,將跳過該記錄
4. Memory存儲引擎
4.1 Memory介紹
- 全內存存儲的引擎
- 數據庫重啟後數據丟失
- 支持哈希索引
- 不支持事物
4.2 Memory特性
千萬不要用Memory存儲引擎去做緩存(Cache)
, 性能上不及Redis和Memcahced- Memory
不能禁用
,當涉及內部排序操作的臨時表時,使用該存儲引擎max_heap_table_size
決定使用內存的大小,默認時16M
- 無論該表使用的什麽引擎,只要使用到臨時表,或者指定Memory,都受參數影響
- 當上面設置的內存放不下數據時,(>=5.6)轉為MyISAM,(>=5.7)轉為InnoDB
- 註意磁盤上臨時路徑空間的大小(
tmpdir
)
- 註意磁盤上臨時路徑空間的大小(
- 內存使用為會話(SESSION)級別,當心內核OOM
- 支持哈希索引,且僅支持等值查詢
mysql> show global status like "%tmp%tables";
+-------------------------+-------+
| Variable_name | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 673 | -- 內存放不下,轉成磁盤存儲的數量,如果過大,考慮增大內存參數
| Created_tmp_tables | 6455 | -- 創建臨時表的數量
+-------------------------+-------+
2 rows in set (0.00 sec)
mysql> show variables like "tmp%";
+----------------+----------------------------------------+
| Variable_name | Value |
+----------------+----------------------------------------+
| tmp_table_size | 33554432 |
| tmpdir | /r2/soft/dbtest/mysql-5.7.18/mysqldata | -- memory轉成磁盤存儲的路徑
+----------------+----------------------------------------+
2 rows in set (0.00 sec)
mysql> show create table User\G
*************************** 1. row ***************************
Table: User
Create Table: CREATE TABLE `User` (
`id` int(11) NOT NULL,
`name` varchar(128) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `name` (`name`) USING HASH -- 對這個字段使用USING HASH,創建hash索引
) ENGINE=MEMORY DEFAULT CHARSET=utf8
1 row in set (0.00 sec)
4.3 Memory的物理特性
- 內存不會一次性分配最大空間,而是隨著使用逐步增到到最大值
- 通過鏈表管理空閑空間
- 使用固定長度存儲數據
- 不支持
BLOB
和TEXT
類型 - 可以創建自增主鍵
5. CSV存儲引擎
5.1 CSV介紹
- CSV - Comma-Separated Values,使用逗號分隔
- 不支持特殊字符
- CSV是一種標準文件格式
- 文件以純文本形式存儲表格數據
- 使用廣泛
5.2 CSV文件組成
frm
表結構CSV
數據文件CSM
元數據信息
5.3 CSV特性
- MySQL CSV存儲引擎運行時,即創建CSV
文件
- 通過MySQL標準接口來查看和修改CSV文件
- 無需將CSV文件導入到數據庫,只需創建相同字段的表結構,拷貝CSV文件即可
- CSV存儲引擎表每個字段
必須是NOT NULL
屬性
6.Federated存儲引擎
6.1. Federated介紹
- 允許本地訪問遠程MySQL數據庫中表的數據
- 本地不存儲任何數據文件
- 類似Oracle中的DBLink
- Federated存儲引擎默認不開啟, 需要在
my.cnf
的[mysqld]
標簽下添加federated
- MySQL的Federated不支持異構數據庫訪問,MariaDB中的
FederatedX
支持
6.2 Federated 語法
scheme://user_name[:password]@host_name[:port_num]/db_name/tbl_name
CONNECTION=‘mysql://username:password@hostname:port/database/tablename‘
- 例句如下:
CREATE TABLE `test001` (
`scenario` varchar(30) NOT NULL DEFAULT ‘‘ COMMENT ‘測試場景‘,
`server_name` varchar(15) NOT NULL COMMENT ‘主機名‘,
`test_type` varchar(15) NOT NULL COMMENT ‘read-only,read-write,insert等‘,
`sb_threads` int(11) NOT NULL DEFAULT ‘0‘ COMMENT ‘sysbench 測試線程‘,
`create_time` datetime DEFAULT NULL COMMENT ‘開始時間‘,
`done_time` datetime DEFAULT NULL COMMENT ‘完成時間‘,
`server_load` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘以當前線程測試完後立刻記錄15分鐘負載值‘,
`request_read` int(11) NOT NULL DEFAULT ‘0‘,
`request_write` int(11) NOT NULL DEFAULT ‘0‘,
`transactions_per_second` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘QPS‘,
`request_per_second` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘TPS‘,
`95_pct_time` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘單位毫秒‘,
KEY `createtime` (`create_time`)
) ENGINE=FEDERATED DEFAULT CHARSET=utf8 CONNECTION=‘mysql://federated:[email protected]:3306/sysbench/test_log‘;
6.3 Federated測試
- 終端A(root@252)-192.168.48.168
mysql> create user federated@‘192.168.24.180‘ identified by ‘123456‘;
Query OK, 0 rows affected (0.00 sec)
mysql> grant select on sysbench.* to federated@‘192.168.24.180‘;
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.01 sec)
mysql> show grants for federated@‘192.168.24.180‘;
+--------------------------------------------------------------+
| Grants for federated@192.168.24.180 |
+--------------------------------------------------------------+
| GRANT USAGE ON *.* TO ‘federated‘@‘192.168.24.180‘ |
| GRANT SELECT ON `sysbench`.* TO ‘federated‘@‘192.168.24.180‘ |
+--------------------------------------------------------------+
2 rows in set (0.00 sec)
--終端A數據表為sysbench.test_log表
mysql> select scenario ,server_name, test_type,sb_threads,create_time,done_time ,server_load from sysbench.test_log limit 1;
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
| scenario | server_name | test_type | sb_threads | create_time | done_time | server_load |
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
| oltp_update_index | localhost | read-write | 2 | 2017-11-15 11:55:07 | 2017-11-15 12:45:18 | 1.56 |
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
1 row in set (0.00 sec)
- 終端B(proxy.gczheng.com)-192.168.24.180
--測試賬號可用
[root@proxy ~]# mysql -h192.168.48.168 -ufederated -p123456 -e "select scenario ,server_name, test_type,sb_threads,create_time,done_time ,server_load from sysbench.test_log limit 1;"
mysql: [Warning] Using a password on the command line interface can be insecure.
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
| scenario | server_name | test_type | sb_threads | create_time | done_time | server_load |
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
| oltp_update_index | localhost | read-write | 2 | 2017-11-15 11:55:07 | 2017-11-15 12:45:18 | 1.56 |
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
(root@localhost) 13:21:28 [(none)]> show engines;
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine | Support | Comment | Transactions | XA | Savepoints |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| MyISAM | YES | MyISAM storage engine | NO | NO | NO |
| CSV | YES | CSV storage engine | NO | NO | NO |
| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO |
| BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO |
| MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
| InnoDB | DEFAULT | Supports transactions, row-level locking, and foreign keys | YES | YES | YES |
| ARCHIVE | YES | Archive storage engine | NO | NO | NO |
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
| FEDERATED | NO | Federated MySQL storage engine | NULL | NULL | NULL | -- FEDERATED引擎沒有啟動
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec)
[root@proxy ~]# cat /etc/my.cnf
# ... 省略 ...
[mysqld]
federated -- 新增的配置項,表示打開Federated引擎
# ... 省略 ...
--重啟MySQL
[root@proxy ~]# /usr/local/mysql/support-files/mysql.server stop
Shutting down MySQL..... SUCCESS!
[root@proxy ~]# /usr/local/mysql/support-files/mysql.server start
Starting MySQL... SUCCESS!
(root@localhost) 13:23:28 [(none)]> show engines;
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine | Support | Comment | Transactions | XA | Savepoints |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| CSV | YES | CSV storage engine | NO | NO | NO |
| MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
| MyISAM | YES | MyISAM storage engine | NO | NO | NO |
| BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO |
| InnoDB | DEFAULT | Supports transactions, row-level locking, and foreign keys | YES | YES | YES |
| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO |
| ARCHIVE | YES | Archive storage engine | NO | NO | NO |
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
| FEDERATED | YES | Federated MySQL storage engine | NO | NO | NO | -- 顯示YES,表示federated引擎已經啟用
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec)
(gcdb@localhost) 13:47:40 [(none)]> create database federated; --創建federated庫
Query OK, 1 row affected (0.00 sec)
(gcdb@localhost) 13:47:53 [(none)]> use federated;
Database changed
--創建test001庫映射(終端A sysbench.test_log表)
(gcdb@localhost) 13:48:02 [federated]> CREATE TABLE `test001` (
-> `scenario` varchar(30) NOT NULL DEFAULT ‘‘ COMMENT ‘測試場景‘,
-> `server_name` varchar(15) NOT NULL COMMENT ‘主機名‘,
-> `test_type` varchar(15) NOT NULL COMMENT ‘read-only,read-write,insert等‘,
-> `sb_threads` int(11) NOT NULL DEFAULT ‘0‘ COMMENT ‘sysbench 測試線程‘,
-> `create_time` datetime DEFAULT NULL COMMENT ‘開始時間‘,
-> `done_time` datetime DEFAULT NULL COMMENT ‘完成時間‘,
-> `server_load` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘以當前線程測試完後立刻記錄15分鐘負載值‘,
-> `request_read` int(11) NOT NULL DEFAULT ‘0‘,
-> `request_write` int(11) NOT NULL DEFAULT ‘0‘,
-> `transactions_per_second` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘QPS‘,
-> `request_per_second` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘TPS‘,
-> `95_pct_time` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘單位毫秒‘,
-> KEY `createtime` (`create_time`)
-> ) ENGINE=federated connection=‘mysql://federated:[email protected]:3306/sysbench/test_log‘;
Query OK, 0 rows affected (0.01 sec)
(gcdb@localhost) 15:26:15 [federated]> select scenario ,server_name, test_type,sb_threads,create_time,done_time ,server_load from test001 limit 1,10; --測試可以查到數據
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
| scenario | server_name | test_type | sb_threads | create_time | done_time | server_load |
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
| oltp_update_index | localhost | read-write | 4 | 2017-11-15 12:45:18 | 2017-11-15 13:35:33 | 1.03 |
| oltp_update_index | localhost | read-write | 8 | 2017-11-15 13:35:33 | 2017-11-15 14:25:40 | 0.93 |
| oltp_update_index | localhost | read-write | 16 | 2017-11-15 14:25:40 | 2017-11-15 15:15:40 | 1.30 |
| oltp_update_index | localhost | read-write | 24 | 2017-11-15 15:15:40 | 2017-11-15 16:04:40 | 1.66 |
| oltp_update_index | localhost | read-write | 32 | 2017-11-15 16:04:40 | 2017-11-15 16:46:26 | 2.09 |
| oltp_update_index | localhost | read-write | 64 | 2017-11-15 16:46:26 | 2017-11-15 17:26:17 | 2.64 |
| oltp_update_index | localhost | read-write | 96 | 2017-11-15 17:26:17 | 2017-11-15 18:06:47 | 1.90 |
| oltp_update_index | localhost | read-write | 128 | 2017-11-15 18:06:48 | 2017-11-15 18:47:24 | 2.52 |
| oltp_update_index | localhost | read-write | 196 | 2017-11-15 18:47:24 | 2017-11-15 19:27:57 | 2.44 |
| oltp_update_index | localhost | read-write | 2 | 2017-11-16 03:48:32 | 2017-11-16 04:38:32 | 5.49 |
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
10 rows in set (0.01 sec)
-- 由於只有select權限,無法對該`federated`.`test001`表進行insert操作
(gcdb@localhost) 15:27:09 [federated]> INSERT INTO `federated`.`test001` (
-> `scenario`,
-> `server_name`,
-> `test_type`,
-> `sb_threads`,
-> `create_time`,
-> `done_time`,
-> `server_load`,
-> `request_read`,
-> `request_write`,
-> `transactions_per_second`,
-> `request_per_second`,
-> `95_pct_time`
-> )
-> VALUES
-> (
-> ‘扯淡計劃‘,
-> ‘拯救小黑羊‘,
-> ‘read-write‘,
-> ‘2‘,
-> ‘2017-11-24 11:55:07‘,
-> ‘2017-11-24 12:45:18‘,
-> ‘1.00‘,
-> ‘0‘,
-> ‘699408‘,
-> ‘290.10‘,
-> ‘290.10‘,
-> ‘0.78‘
-> );
ERROR 1296 (HY000): Got error 10000 ‘Error on remote system: 1142: INSERT command denied to user ‘federated‘@‘192.168.24.180‘ for table ‘‘ from FEDERATED
005: 存儲引擎