使用akka+http 提供spark 服务问题

使用 akka-http to 提供 http 服务 大致代码如下:

object SparkServer extends App with SparkServices {
implicit val system = ActorSystem("spark-akka-http")
implicit val materializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
val bindingFuture = Http().bindAndHandle(route, "192.168.xx.xx", 8001)}

SparkServices

val sparkConf = new SparkConf().setMaster("local[8]")
val sparkSession = SparkSession.builder
.appName("SpanMsgServer")
.config(sparkConf)
.enableHiveSupport()
.getOrCreate()

使用 spark-submit, 提交 出现错误:

com.typesafe.config.ConfigException$Missing: No configuration setting found for key akka.stream

按照网上提示pom 加入


reference.conf

错误没怎么改变,修改 application.conf 添加如下http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M2/stream-configuration.html配置到 application.conf :

出现了com.typesafe.config.ConfigException$Missing: No configuration setting found for key debug-logging  怎么解决啊

http://www.tuicool.com/articles/reqYrqZ

在resources下面添加一个application.conf文件,把akka.stream的配置以key-value的方式添加进去