Lab 005 - jazzwang/hive_labs GitHub Wiki

實作五

Sqoop (5) : 將 MS SQL 資料表匯入 HBase

執行記錄匯入

  1. 請輸入如下指令並將「帳號」處替換成您的帳號：
user@master ~ $ export DBID=帳號
user@master ~ $ sqoop import --connect "jdbc:sqlserver://sql.3du.me;database=$DBID" --table MSSQL_DATA --username $DBID -P --hbase-row-k
ey id --hbase-create-table --column-family data --hbase-table MSSQL_DATA
Enter password: 輸入密碼
  1. 若資料表匯入有正常執行，您將看到以下的類似訊息：
13/12/22 15:51:19 INFO manager.SqlManager: Using default fetchSize of 1000
13/12/22 15:51:19 INFO tool.CodeGenTool: Beginning code generation
13/12/22 15:51:20 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM [MSSQL_DATA] AS t WHERE 1=0
13/12/22 15:51:20 INFO orm.CompilationManager: HADOOP_HOME is /opt/hadoopmr
Note: /tmp/sqoop-user/compile/24981ba074b67292d219153cf6e3b770/MSSQL_DATA.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
13/12/22 15:51:23 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-user/compile/24981ba074b67292d219153cf6e3b770/MSSQL_DATA.jar
13/12/22 15:51:23 INFO mapreduce.ImportJobBase: Beginning import of MSSQL_DATA
... 略 ...
13/12/22 15:52:51 INFO mapreduce.ImportJobBase: Retrieved 2 records.

檢查匯入結果

  1. 轉換之結果，可於 HBase 儲存於 HDFS 的路徑查到。
user@master ~ $ hadoop fs -ls /hbase/MSSQL_DATA
No encryption was performed by peer.
Found 3 items
-rw-------   3 etu supergroup        559 2013-12-22 15:51 /hbase/MSSQL_DATA/.tableinfo.0000000001
drwxr-xr-x   - etu supergroup          0 2013-12-22 15:51 /hbase/MSSQL_DATA/.tmp
drwx------   - etu supergroup          0 2013-12-22 15:51 /hbase/MSSQL_DATA/1f0078cc9341be82f7137b8009f8fb9e
  1. 您也可以透過 Etu Appliance 的管理介面觀察 HBase 資料表的狀態。
    • Etu Appliance HBase Cluster Status

參考資料

  1. "Sqoop: Big data conduit between NoSQL and RDBMS", Surajit Paul, Advisory Consultant, IBM, 23 Jul 2013
Add a custom footer
⚠️ **GitHub.com Fallback** ⚠️