1. 程式人生 > >005: 存儲引擎

005: 存儲引擎

acl 指針 disk mailto ole 內部 update 性能 separate

存儲引擎

1.MySQL上支持的存儲引擎

mysql> show engines;
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine             | Support | Comment                                                        | Transactions | XA   | Savepoints |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| CSV | YES | CSV storage engine | NO | NO | NO | | MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO | | MyISAM | YES | MyISAM storage
engine | NO | NO | NO | | BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO | | InnoDB | DEFAULT | Supports transactions, row-level locking, and foreign
keys | YES | YES | YES | | PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO | | ARCHIVE | YES | Archive storage engine | NO | NO | NO | | MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO | | FEDERATED | NO | Federated MySQL storage engine | NULL | NULL | NULL | +--------------------+---------+----------------------------------------------------------------+--------------+------+------------+ 9 rows in set (0.00 sec)

2.存儲引擎的概念

  • 用來處理數據庫的相關CRUD操作

  • 每個數據庫都有存儲引擎,只是MySQL比較強調存儲引擎的概念。

3. MySQL存儲引擎

  • 官方存儲引擎
    • MyISAM
    • InnoDB -- 推薦;其他引擎已經體停止維護和開發
    • Memory
    • Federated
    • CSV
    • Archive
  • 第三方存儲引擎
    • TokuDB -- 開源,適合插入密集型
    • InfoBright -- 商業,開源版本有數據量限制。屬於列存儲,面向OLAP場景
    • Spider

      第三方存儲引擎在特定場合下比較適合,除此之外,都應該使用InnoDB

3.存儲引擎之MyISAM

3.1MyISAM特點

  • MySQL5.1版本之前的默認存儲引擎
  • 堆表數據結構
  • 表鎖設計
  • 支持數據靜態壓縮
  • 不支持事物
  • 數據容易丟失
  • 索引容易損壞
  • 唯一優點
    • 數據文件可以直接拷貝到另一臺服務器使用
  • 現在MySQL中還有用MyISAM的表,
    -主要是歷史原因。數據庫文件以MY開頭的基本都是MyISAM的表
    • 部分如User,DB等系統表(MyISAM引擎),可以直接拷貝,比較方便
    • 性能好,或者存儲小不是MyISAM的優點,也不是存在的原因

3.2 MyISAM文件組成

  • frm 表結構文件
  • MYI 索引文件
  • MYD 數據文件
    • 數據文件是堆表數據結構,堆是無序數據的集合
    • MYI中的葉子節點,指向MYD中的數據頁
    • 當數據移動到頁外時,需要修改對應指針

3.3 myisamchk

  • myisamchk 最好是關閉數據庫,然後修復
  • 不停機需要鎖住表 lock table sysbench.test_log_copy read 再修復
mysql> show create table test_log_copy\G;
*************************** 1. row ***************************
       Table: test_log_copy
Create Table: CREATE TABLE `test_log_copy` (
  `scenario` varchar(30) NOT NULL DEFAULT ‘‘ COMMENT ‘測試場景‘,
  `server_name` varchar(15) NOT NULL COMMENT ‘主機名‘,
  `test_type` varchar(15) NOT NULL COMMENT ‘read-only,read-write,insert等‘,
  `sb_threads` int(11) NOT NULL DEFAULT ‘0‘ COMMENT ‘sysbench 測試線程‘,
  `create_time` datetime DEFAULT NULL COMMENT ‘開始時間‘,
  `done_time` datetime DEFAULT NULL COMMENT ‘完成時間‘,
  `server_load` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘以當前線程測試完後立刻記錄15分鐘負載值‘,
  `request_read` int(11) NOT NULL DEFAULT ‘0‘,
  `request_write` int(11) NOT NULL DEFAULT ‘0‘,
  `transactions_per_second` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘QPS‘,
  `request_per_second` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘TPS‘,
  `95_pct_time` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘單位毫秒‘,
  KEY `createtime` (`create_time`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
1 row in set (0.00 sec)

ERROR:
No query specified
[root@localhost-m(252) /r2/soft/dbtest/mysql-5.7.18/mysqldata]# myisamchk  sysbench/test_log_copy.MYI
Checking MyISAM file: sysbench/test_log_copy.MYI
Data records:      60   Deleted blocks:       0
- check file-size
- check record delete-chain
- check key delete-chain
- check index reference
- check data record references index: 1
- check record links

myisamchk通過掃描MYD文件來重建MYI文件;如果MYD文件中某條記錄有問題,將跳過該記錄

4. Memory存儲引擎

4.1 Memory介紹

  • 全內存存儲的引擎
  • 數據庫重啟後數據丟失
  • 支持哈希索引
  • 不支持事物

4.2 Memory特性

  • 千萬不要用Memory存儲引擎去做緩存(Cache), 性能上不及Redis和Memcahced
  • Memory不能禁用,當涉及內部排序操作的臨時表時,使用該存儲引擎
    • max_heap_table_size決定使用內存的大小,默認時16M
      • 無論該表使用的什麽引擎,只要使用到臨時表,或者指定Memory,都受參數影響
    • 當上面設置的內存放不下數據時,(>=5.6)轉為MyISAM,(>=5.7)轉為InnoDB
      • 註意磁盤上臨時路徑空間的大小(tmpdir)
    • 內存使用為會話(SESSION)級別,當心內核OOM
  • 支持哈希索引,且僅支持等值查詢
mysql> show global status like "%tmp%tables";
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 673   |      -- 內存放不下,轉成磁盤存儲的數量,如果過大,考慮增大內存參數
| Created_tmp_tables      | 6455  |      -- 創建臨時表的數量
+-------------------------+-------+
2 rows in set (0.00 sec)


mysql> show variables like "tmp%";
+----------------+----------------------------------------+
| Variable_name  | Value                                  |
+----------------+----------------------------------------+
| tmp_table_size | 33554432                               |
| tmpdir         | /r2/soft/dbtest/mysql-5.7.18/mysqldata | -- memory轉成磁盤存儲的路徑
+----------------+----------------------------------------+
2 rows in set (0.00 sec)

mysql> show create table User\G
*************************** 1. row ***************************
Table: User
Create Table: CREATE TABLE `User` (
`id` int(11) NOT NULL,
`name` varchar(128) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `name` (`name`) USING HASH          -- 對這個字段使用USING HASH,創建hash索引
) ENGINE=MEMORY DEFAULT CHARSET=utf8
1 row in set (0.00 sec)

4.3 Memory的物理特性

  • 內存不會一次性分配最大空間,而是隨著使用逐步增到到最大值
  • 通過鏈表管理空閑空間
  • 使用固定長度存儲數據
  • 不支持BLOBTEXT類型
  • 可以創建自增主鍵

5. CSV存儲引擎

5.1 CSV介紹

  • CSV - Comma-Separated Values,使用逗號分隔
  • 不支持特殊字符
  • CSV是一種標準文件格式
  • 文件以純文本形式存儲表格數據
  • 使用廣泛

5.2 CSV文件組成

  • frm 表結構
  • CSV 數據文件
  • CSM 元數據信息

5.3 CSV特性

  • MySQL CSV存儲引擎運行時,即創建CSV文件
  • 通過MySQL標準接口來查看和修改CSV文件
  • 無需將CSV文件導入到數據庫,只需創建相同字段的表結構,拷貝CSV文件即可
  • CSV存儲引擎表每個字段必須是NOT NULL屬性

6.Federated存儲引擎

6.1. Federated介紹

  • 允許本地訪問遠程MySQL數據庫中表的數據
  • 本地不存儲任何數據文件
  • 類似Oracle中的DBLink
  • Federated存儲引擎默認不開啟, 需要在my.cnf[mysqld]標簽下添加 federated
  • MySQL的Federated不支持異構數據庫訪問,MariaDB中的FederatedX支持

技術分享圖片

6.2 Federated 語法

scheme://user_name[:password]@host_name[:port_num]/db_name/tbl_name

CONNECTION=‘mysql://username:password@hostname:port/database/tablename‘

  • 例句如下:
CREATE TABLE `test001` (
  `scenario` varchar(30) NOT NULL DEFAULT ‘‘ COMMENT ‘測試場景‘,
  `server_name` varchar(15) NOT NULL COMMENT ‘主機名‘,
  `test_type` varchar(15) NOT NULL COMMENT ‘read-only,read-write,insert等‘,
  `sb_threads` int(11) NOT NULL DEFAULT ‘0‘ COMMENT ‘sysbench 測試線程‘,
  `create_time` datetime DEFAULT NULL COMMENT ‘開始時間‘,
  `done_time` datetime DEFAULT NULL COMMENT ‘完成時間‘,
  `server_load` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘以當前線程測試完後立刻記錄15分鐘負載值‘,
  `request_read` int(11) NOT NULL DEFAULT ‘0‘,
  `request_write` int(11) NOT NULL DEFAULT ‘0‘,
  `transactions_per_second` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘QPS‘,
  `request_per_second` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘TPS‘,
  `95_pct_time` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘單位毫秒‘,
  KEY `createtime` (`create_time`)
) ENGINE=FEDERATED DEFAULT CHARSET=utf8 CONNECTION=‘mysql://federated:[email protected]:3306/sysbench/test_log‘;

6.3 Federated測試

  • 終端A(root@252)-192.168.48.168

mysql> create user federated@‘192.168.24.180‘ identified by ‘123456‘;
Query OK, 0 rows affected (0.00 sec)

mysql> grant select on sysbench.* to federated@‘192.168.24.180‘;
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.01 sec)

mysql> show grants for federated@‘192.168.24.180‘;
+--------------------------------------------------------------+
| Grants for federated@192.168.24.180                          |
+--------------------------------------------------------------+
| GRANT USAGE ON *.* TO ‘federated‘@‘192.168.24.180‘           |
| GRANT SELECT ON `sysbench`.* TO ‘federated‘@‘192.168.24.180‘ |
+--------------------------------------------------------------+
2 rows in set (0.00 sec)

--終端A數據表為sysbench.test_log表

mysql> select scenario ,server_name, test_type,sb_threads,create_time,done_time ,server_load from sysbench.test_log limit 1;
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
| scenario          | server_name | test_type  | sb_threads | create_time         | done_time           | server_load |
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
| oltp_update_index | localhost   | read-write |          2 | 2017-11-15 11:55:07 | 2017-11-15 12:45:18 |        1.56 |
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
1 row in set (0.00 sec)
  • 終端B(proxy.gczheng.com)-192.168.24.180
--測試賬號可用
[root@proxy ~]# mysql -h192.168.48.168 -ufederated -p123456 -e "select scenario ,server_name, test_type,sb_threads,create_time,done_time ,server_load from sysbench.test_log limit 1;"
mysql: [Warning] Using a password on the command line interface can be insecure.
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
| scenario          | server_name | test_type  | sb_threads | create_time         | done_time           | server_load |
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
| oltp_update_index | localhost   | read-write |          2 | 2017-11-15 11:55:07 | 2017-11-15 12:45:18 |        1.56 |
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+


(root@localhost) 13:21:28 [(none)]> show engines;
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine             | Support | Comment                                                        | Transactions | XA   | Savepoints |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| MyISAM             | YES     | MyISAM storage engine                                          | NO           | NO   | NO         |
| CSV                | YES     | CSV storage engine                                             | NO           | NO   | NO         |
| PERFORMANCE_SCHEMA | YES     | Performance Schema                                             | NO           | NO   | NO         |
| BLACKHOLE          | YES     | /dev/null storage engine (anything you write to it disappears) | NO           | NO   | NO         |
| MRG_MYISAM         | YES     | Collection of identical MyISAM tables                          | NO           | NO   | NO         |
| InnoDB             | DEFAULT | Supports transactions, row-level locking, and foreign keys     | YES          | YES  | YES        |
| ARCHIVE            | YES     | Archive storage engine                                         | NO           | NO   | NO         |
| MEMORY             | YES     | Hash based, stored in memory, useful for temporary tables      | NO           | NO   | NO         |
| FEDERATED          | NO      | Federated MySQL storage engine                                 | NULL         | NULL | NULL       |   -- FEDERATED引擎沒有啟動
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec)

[root@proxy ~]# cat /etc/my.cnf
# ... 省略 ...
[mysqld]
federated           -- 新增的配置項,表示打開Federated引擎
# ... 省略 ...

--重啟MySQL
[root@proxy ~]# /usr/local/mysql/support-files/mysql.server stop
Shutting down MySQL..... SUCCESS!
[root@proxy ~]# /usr/local/mysql/support-files/mysql.server start
Starting MySQL... SUCCESS!


(root@localhost) 13:23:28 [(none)]> show engines;
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine             | Support | Comment                                                        | Transactions | XA   | Savepoints |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| CSV                | YES     | CSV storage engine                                             | NO           | NO   | NO         |
| MRG_MYISAM         | YES     | Collection of identical MyISAM tables                          | NO           | NO   | NO         |
| MyISAM             | YES     | MyISAM storage engine                                          | NO           | NO   | NO         |
| BLACKHOLE          | YES     | /dev/null storage engine (anything you write to it disappears) | NO           | NO   | NO         |
| InnoDB             | DEFAULT | Supports transactions, row-level locking, and foreign keys     | YES          | YES  | YES        |
| PERFORMANCE_SCHEMA | YES     | Performance Schema                                             | NO           | NO   | NO         |
| ARCHIVE            | YES     | Archive storage engine                                         | NO           | NO   | NO         |
| MEMORY             | YES     | Hash based, stored in memory, useful for temporary tables      | NO           | NO   | NO         |
| FEDERATED          | YES     | Federated MySQL storage engine                                 | NO           | NO   | NO         |  -- 顯示YES,表示federated引擎已經啟用
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec)


(gcdb@localhost) 13:47:40 [(none)]>  create database federated;   --創建federated庫
Query OK, 1 row affected (0.00 sec)

(gcdb@localhost) 13:47:53 [(none)]> use federated;
Database changed

--創建test001庫映射(終端A sysbench.test_log表)
(gcdb@localhost) 13:48:02 [federated]> CREATE TABLE `test001` (
    ->   `scenario` varchar(30) NOT NULL DEFAULT ‘‘ COMMENT ‘測試場景‘,
    ->   `server_name` varchar(15) NOT NULL COMMENT ‘主機名‘,
    ->   `test_type` varchar(15) NOT NULL COMMENT ‘read-only,read-write,insert等‘,
    ->   `sb_threads` int(11) NOT NULL DEFAULT ‘0‘ COMMENT ‘sysbench 測試線程‘,
    ->   `create_time` datetime DEFAULT NULL COMMENT ‘開始時間‘,
    ->   `done_time` datetime DEFAULT NULL COMMENT ‘完成時間‘,
    ->   `server_load` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘以當前線程測試完後立刻記錄15分鐘負載值‘,
    ->   `request_read` int(11) NOT NULL DEFAULT ‘0‘,
    ->   `request_write` int(11) NOT NULL DEFAULT ‘0‘,
    ->   `transactions_per_second` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘QPS‘,
    ->   `request_per_second` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘TPS‘,
    ->   `95_pct_time` decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘單位毫秒‘,
    ->   KEY `createtime` (`create_time`)
    -> ) ENGINE=federated  connection=‘mysql://federated:[email protected]:3306/sysbench/test_log‘;
Query OK, 0 rows affected (0.01 sec)


(gcdb@localhost) 15:26:15 [federated]> select scenario ,server_name, test_type,sb_threads,create_time,done_time ,server_load from test001 limit 1,10;  --測試可以查到數據
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
| scenario          | server_name | test_type  | sb_threads | create_time         | done_time           | server_load |
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
| oltp_update_index | localhost   | read-write |          4 | 2017-11-15 12:45:18 | 2017-11-15 13:35:33 |        1.03 |
| oltp_update_index | localhost   | read-write |          8 | 2017-11-15 13:35:33 | 2017-11-15 14:25:40 |        0.93 |
| oltp_update_index | localhost   | read-write |         16 | 2017-11-15 14:25:40 | 2017-11-15 15:15:40 |        1.30 |
| oltp_update_index | localhost   | read-write |         24 | 2017-11-15 15:15:40 | 2017-11-15 16:04:40 |        1.66 |
| oltp_update_index | localhost   | read-write |         32 | 2017-11-15 16:04:40 | 2017-11-15 16:46:26 |        2.09 |
| oltp_update_index | localhost   | read-write |         64 | 2017-11-15 16:46:26 | 2017-11-15 17:26:17 |        2.64 |
| oltp_update_index | localhost   | read-write |         96 | 2017-11-15 17:26:17 | 2017-11-15 18:06:47 |        1.90 |
| oltp_update_index | localhost   | read-write |        128 | 2017-11-15 18:06:48 | 2017-11-15 18:47:24 |        2.52 |
| oltp_update_index | localhost   | read-write |        196 | 2017-11-15 18:47:24 | 2017-11-15 19:27:57 |        2.44 |
| oltp_update_index | localhost   | read-write |          2 | 2017-11-16 03:48:32 | 2017-11-16 04:38:32 |        5.49 |
+-------------------+-------------+------------+------------+---------------------+---------------------+-------------+
10 rows in set (0.01 sec)

-- 由於只有select權限,無法對該`federated`.`test001`表進行insert操作
(gcdb@localhost) 15:27:09 [federated]> INSERT INTO `federated`.`test001` (
    -> `scenario`,
    -> `server_name`,
    -> `test_type`,
    -> `sb_threads`,
    -> `create_time`,
    -> `done_time`,
    -> `server_load`,
    -> `request_read`,
    -> `request_write`,
    -> `transactions_per_second`,
    -> `request_per_second`,
    -> `95_pct_time`
    -> )
    -> VALUES
    -> (
    -> ‘扯淡計劃‘,
    -> ‘拯救小黑羊‘,
    -> ‘read-write‘,
    -> ‘2‘,
    -> ‘2017-11-24 11:55:07‘,
    -> ‘2017-11-24 12:45:18‘,
    -> ‘1.00‘,
    -> ‘0‘,
    -> ‘699408‘,
    -> ‘290.10‘,
    -> ‘290.10‘,
    -> ‘0.78‘
    -> );
ERROR 1296 (HY000): Got error 10000 ‘Error on remote system: 1142: INSERT command denied to user ‘federated‘@‘192.168.24.180‘ for table ‘‘ from FEDERATED

005: 存儲引擎