spark怎么读取hbase表

这篇文章主要介绍了spark怎么读取HBase表的相关知识，内容详细易懂，操作简单快捷，具有一定借鉴价值，相信大家阅读完这篇spark怎么读取hbase表文章都会有所收获，下面我们一起来看看吧。

一.场景:
    spark通过phoenix 读取hbase表，其实说白了先要去ZooKeeper建立connection。

二.代码:

1. val zkUrl = "192.168.100.39,192.168.100.40,192.168.100.41:2181"
   

2. val fORMatStr = "org.apache.phoenix.spark"
   

3. val oms_orderinfoDF = spark.read.format(formatStr)
   

4.   .options(Map("table" -> "oms_orderinfo", "zkUrl" -> zkUrl))
   

5.   .load

三.查看SparkJob日志:

1. 17/10/24 03:25:25 INFO zookeeper.ClientCnxn: Opening Socket connection to server hadoop40/192.168.100.40:2181. Will not attempt to authenticate using SASL (unknown error)
   

2. 17/10/24 03:25:25 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /192.168.100.48:35952, server: hadoop40/192.168.100.40:2181
   

3. 17/10/24 03:25:25 WARN zookeeper.ClientCnxn: Session 0x0 for server hadoop40/192.168.100.40:2181, unexpected error, closing socket connection and attempting reconnect
   

4. java.io.IOException: Connection reset by peer

5.     at sun.NIO.ch.FileDispatcherImpl.read0(Native Method)
   

6.     at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
   

7.     at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
   

8.     at sun.nio.ch.IOUtil.read(IOUtil.java:192)
   

9.     at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
   

10.     at org.apache.phoenix.shaded.org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:68)
    

11.     at org.apache.phoenix.shaded.org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:355)
    

12.     at org.apache.phoenix.shaded.org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
    

13. 17/10/24 03:25:25 INFO yarn.ApplicationMaster: Deleting staging directory hdfs://nameservice1/user/hdfs/.sparkStaging/application_1507703377455_4854
    

14. 17/10/24 03:25:25 INFO util.ShutdownHookManager: Shutdown hook called

关于“spark怎么读取hbase表”这篇文章的内容就介绍到这里，感谢各位的阅读！相信大家对“spark怎么读取hbase表”知识都有一定的了解，大家如果还想学习更多知识，欢迎关注编程网精选频道。