setup(Scala)
Loading...

Setup

Sets up a Kafka cluster, with single partition topics "input1", "intput2".

%run ./utils
Warning: classes defined within packages cannot be redefined without a cluster restart. Compilation successful.
Warning: classes defined within packages cannot be redefined without a cluster restart. Compilation successful.
import sys.process._
val kafkaCluster = com.databricks.spark.LocalKafka.setup(spark, stopSparkOnKafkaNodes = false, numKafkaNodes = 1)  
import sys.process._ kafkaCluster: com.databricks.spark.LocalKafka = com.databricks.spark.LocalKafka@5c65b51
// Capture the output to avoid cluttering things.
Console.withOut(new java.io.ByteArrayOutputStream) {
  kafkaCluster.deleteTopicIfExists("input1")
  kafkaCluster.deleteTopicIfExists("input2")
  kafkaCluster.createTopic("input1", 1)
  kafkaCluster.createTopic("input2", 1)
}
java.lang.RuntimeException: Command failed