sqoop将数据从Teradata传输到HDFS(sqoop to transfer data to HDFS from Teradata)

sqoop将数据从Teradata传输到HDFS(sqoop to transfer data to HDFS from Teradata)

sqoop从Teradata传输数据到HDFS：获取错误如下：-bash-4.1 $ sqoop import --connection-manager com.cloudera.sqoop.manager.DefaultManagerFactory --driver com.teradata.jdbc.TeraDriver \

--connect jdbc：teradata：//dwsoat.dws.company.co.uk/DATABASE=TS_72258_BASELDB \ --username userid -P --table ADDRESS --num-mappers 3 \ --target-dir / user / nathalok / ADDRESS警告：/apps/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/bin/../lib/sqoop/../accumulo不存在！ Accumulo进口将失败。 请将$ ACCUMULO_HOME设置为您的Accumulo安装的根目录。 14/10/29 14:00:14信息sqoop.Sqoop：运行Sqoop版本：1.4.4-cdh5.1.3 14/10/29 14:00:14 WARN tool.BaseSqoopTool：在命令行上设置密码是不安全的。 考虑使用-P来代替。 14/10/29 14:00:14错误sqoop.ConnFactory：Sqoop无法正确创建连接管理器。 有些连接器支持显式驱动程序，有些则不支持。 请尝试指定--driver或将其忽略。 14/10/29 14:00:14错误tool.BaseSqoopTool：创建数据库管理器时发生错误：java.io.IOException：java.lang.NoSuchMethodException：com.cloudera.sqoop.manager.DefaultManagerFactory。（java.lang.String， com.cloudera.sqoop.SqoopOptions）在org.apache.sqoop.ConnFactory.getManager（ConnFactory.java:165）位于org.apache.sqoop.tool.BaseSqoopTool.init（BaseSqoopTool.java:243）at org.apache.sqoop .tool.ImportTool.init（ImportTool.java:84）at org.apache.sqoop.tool.ImportTool.run（ImportTool.java:494）at org.apache.sqoop.Sqoop.run（Sqoop.java:147）at在org.apache.sqoop.Sqoop.runSqoop（Sqoop.java:183）上的org.apache.hadoop.util.ToolRunner.run（ToolRunner.java:70）at org.apache.sqoop.Sqoop.runTool（Sqoop.java： 222）at org.apache.sqoop.Sqoop.runTool（Sqoop.java:231）at org.apache.sqoop.Sqoop.main（Sqoop.java:240）引起：java.lang.NoSuchMethodException：com.cloudera.sqoop 。java.lang.Class.getConstructor0（Class.java:2810）在java中的.manager.DefaultManagerFactory。（java.lang.String，com.cloudera.sqoop.SqoopOptions） .lang.Class.getDeclaredConstructor（Class.java:2053）at org.apache.sqoop.ConnFactory.getManager（ConnFactory.java:151）... 9 more

-bash-4.1 $

任何帮助将不胜感激。

sqoop to transfer data to HDFS from Teradata: Getting error as below: -bash-4.1$ sqoop import --connection-manager com.cloudera.sqoop.manager.DefaultManagerFactory --driver com.teradata.jdbc.TeraDriver \

--connect jdbc:teradata://dwsoat.dws.company.co.uk/DATABASE=TS_72258_BASELDB \ --username userid -P --table ADDRESS --num-mappers 3 \ --target-dir /user/nathalok/ADDRESS Warning: /apps/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 14/10/29 14:00:14 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4-cdh5.1.3 14/10/29 14:00:14 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 14/10/29 14:00:14 ERROR sqoop.ConnFactory: Sqoop wasn't able to create connnection manager properly. Some of the connectors supports explicit --driver and some do not. Please try to either specify --driver or leave it out. 14/10/29 14:00:14 ERROR tool.BaseSqoopTool: Got error creating database manager: java.io.IOException: java.lang.NoSuchMethodException: com.cloudera.sqoop.manager.DefaultManagerFactory.(java.lang.String, com.cloudera.sqoop.SqoopOptions) at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:165) at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:243) at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:84) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:494) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) at org.apache.sqoop.Sqoop.main(Sqoop.java:240) Caused by: java.lang.NoSuchMethodException: com.cloudera.sqoop.manager.DefaultManagerFactory.(java.lang.String, com.cloudera.sqoop.SqoopOptions) at java.lang.Class.getConstructor0(Class.java:2810) at java.lang.Class.getDeclaredConstructor(Class.java:2053) at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:151) ... 9 more

-bash-4.1$

Any help will be appreciated.

最满意答案

要使Teradata使用Cloudera发行版正常工作，您需要执行以下操作：

将Teradata JDBC jar安装在/ var / lib / sqoop中。 对我来说，这些是terajdbc4.jar和tdgssconfig.jar。 安装由Teradata提供支持的Cloudera Connector或安装在文件系统某处的Teradata连接器 （我更喜欢/ var / lib / sqoop）。 在/etc/sqoop/conf/managers.d/中，创建一个文件（任何名称）并添加com.cloudera.connector.teradata.TeradataManagerFactory=<location of connector jar> 。 例如，我有/etc/sqoop/conf/managers.d/teradata => com.cloudera.connector.teradata.TeradataManagerFactory=/var/lib/sqoop/sqoop-connector-teradata-1.2c5.jar 。

有多种不同的方式来安装Teradata连接器 。 例如，使用Cloudera Manager可能会更容易。

如果您仍遇到问题，请尝试联系到sqoop邮件列表 。

To get Teradata working properly using a Cloudera distribution, you need to do the following:

Install the Teradata JDBC jars in /var/lib/sqoop. For me these were terajdbc4.jar and tdgssconfig.jar. Install either Cloudera Connector Powered by Teradata or the Cloudera Connector for Teradata installed somewhere on your filesystem (I prefer /var/lib/sqoop). In /etc/sqoop/conf/managers.d/, create a file (of any name) and add com.cloudera.connector.teradata.TeradataManagerFactory=<location of connector jar>. For example, I have /etc/sqoop/conf/managers.d/teradata => com.cloudera.connector.teradata.TeradataManagerFactory=/var/lib/sqoop/sqoop-connector-teradata-1.2c5.jar.

There are different ways to install the Teradata connector as well. For example, it may be easier to use Cloudera Manager.

If you're still having trouble, try reaching out to the sqoop mailing list.