Importing Data From RDBMS to Hive using Sqoop - manojkumar3036/BigData-using-Hadoop GitHub Wiki
Loading Data to MySQL
load data local infile '/tmp/deliveries.csv' into table DELIVERIES fields terminated by ',' lines terminated by '\n' ignore 1 rows;
Error of Secure file priv
mysql --help | more
At the end of the instruction, you will see some thing like this: Default options are read from the following files in the given order:
_/etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf _
If you don't find the my.cnf in /etc and in home then create those files.
secure_file_priv = '' should be inside those files.
SELECT @@GLOBAL.secure_file_priv should come null or empty
Starting and stopping the mysql services
sudo service mysql stop for stopping the mysql service
sudo service mysql start for starting the mysql service
Loading Data to Hive using Sqoop
Hive Configuration for MySQL
Refer to Hive Configuration for MySql
sqoop import command
sqoop import --connect jdbc:mysql://localhost:3306/IPLDATASET -username root -P --table DELIVERIES --hive-import
ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com.fasterxml.jackson.databind.ObjectMapper.readerFor(Ljava/lang/Class;)Lcom/fasterxml/jackson/databind/ObjectReader;
For this please check the jackson-core, jackson-databind and jackson-annotation jar. The jar should be of the latest version. Usually it comes due to the older version. Place these jar inside the hive lib and sqoop lib. Along with please check the libthrift jar, both in hive and hbase it should be same and copy the same in sqoop lib