文章目錄

• 一、copy遷移資料：
• 二、錯誤排查

一、copy遷移資料：

copy匯出資料

#!/bin/bash
for kspname in {xn_dolphin_1,xn_im,xn_imstatus,magpie_persistent,eagle_persistent,callserver}
do 
	createKeyspacePath="/root/cassandra/create/$kspname"
	if [ ! -x "$createKeyspacePath" ]; then  
		mkdir -p $createKeyspacePath
	fi 
	cqlsh 192.23.24.13 -ucassandra -pcassandra -e  "desc  $kspname" >$createKeyspacePath/$kspname.cql
	
	dstKeyspacePath="/root/cassandra/$kspname"
	if [ ! -x "$dstKeyspacePath" ]; then  
		mkdir -p $dstKeyspacePath
	fi 
	
	for table in $(cqlsh 192.23.24.13  -ucassandra -pcassandra  -k $kspname  -e  'DESC TABLES')
	do
		cqlsh 192.23.24.13 -ucassandra -pcassandra -k $kspname  -e  "COPY $table TO '$dstKeyspacePath/${table}.CSV'"
	done
done
 

建庫指令碼更改操作：
sed -i ‘s/datacenter1/DC1-K8/g’ xn_dolphin_1.cql

copy匯入資料

#!/bin/bash
currentPath=$PWD
for kspname in `ls cassandra/`;
do
    for table in `ls cassandra/$kspname/`
        do
            backTableFile=`basename $table`
            currentTableName=`basename $table|awk -F'.' '{print $1}'`
            tablePath="$currentPath/cassandra/$kspname/$backTableFile"
            echo $currentTableName
            cqlsh  192.23.24.113 30222 -ucassandra -pcassandra -k $kspname  -e  "COPY $currentTableName from '$tablePath'"
        done
done
 

二、錯誤排查

1、錯誤：

--Starting copy of xn_dolphin_1.dolphin_conversation_message with columns [siteid, converid, messageid, arrivetime, createat, duration, errortype, expiredtime, extension, fromuser, message, msgtype, size, sourceurl, sys, tousers, type, url].
<stdin>:1:Failed to import 92 rows: ParseError - Failed to parse {''} : Empty values are not allowed,  given up without retries
<stdin>:1:Failed to import 24 rows: ParseError - Failed to parse {''} : Empty values are not allowed,  given up without retries
 

解決：

在匯入的csv檔案裡存在"{''}" 無法解析
 sed -i "s#{''}##g" dolphin_conversation_message.csv

2、錯誤：

--Starting copy of xn_dolphin_1.dolphin_visitorinfo_snapshot with columns [siteid, customerid, customerinfo, lasttime].
later, attempt 3 of 5
<stdin>:1:Failed to import 20 rows: InvalidRequest - Error from server: code=2200 [Invalid query] message="Batch too large",  will retry later, attempt 4 of 5
<stdin>:1:Failed to import 20 rows: InvalidRequest - Error from server: code=2200 [Invalid query] message="Batch too large",  given up after 5 attempts

解決：batch_size_fail_threshold_in_kb的預設值只有50,一條DML語句就超過了這個閾值.調整大小。

3、錯誤

<stdin>:1:Failed to import 5000 rows: Error - field larger than field limit (131072),  given up after 1 attempts
<stdin>:1:Exceeded maximum number of insert errors 1000
<stdin>:1:Failed to process 5336 rows; failed rows written to import_xn_dolphin_1_dolphin_visitorinfo_snapshot.err

解決：

[email protected] ~]#vim /etc/cassandra/conf/cqlshrc.sample 
; field_size_limit = 13107200000

注：
網上提供的解決辦法：
https://stackoverflow.com/questions/24168235/cassandra-cqlsh-text-field-limit-on-copy-from-csv-field-larger-than-field-limit
嘗試操作失敗