用kafka读取__consumer_offsets

I want to read from the topic __consumer_offsets using this library: https://github.com/segmentio/kafka-go

My problem is that unless I specify a partition nothing seems to happen. There are 100 partitions for this topic by default, it seems unreasonable to query kafka for the list of partitions and then loop over them to read them, and I'm hoping for a pre-existing method in the library that reads messages from all partitions in the topic.

Currently the following works, after I verified with kafkacat that there are messages in partition 15 of __consumer_offsets topic:

  r := kafka.NewReader(kafka.ReaderConfig{
    Brokers:   []string{"kafka:9092"},
    Topic:     "__consumer_offsets",
    Partition: 15
  })
  r.SetOffset(0)

  for {
    m, err := r.ReadMessage(context.Background())
    if err != nil {
      log.Println("Error while trying to read message")
      log.Fatal(err)
      break
    }
    log.Printf("message at offset %d
", m.Offset)
  }

  r.Close()

I would imagine that partition choosing should be transparent on the user level unless needed. Am I wrong?

Is there a way to read from the topic regardless of which partitions the messages are in? or to rephrase, read from all partitions?

Use the consumer group API, and you don't need to give partitions.

https://github.com/segmentio/kafka-go#consumer-groups

// GroupID holds the optional consumer group id.  If GroupID is specified, then
// Partition should NOT be specified e.g. 0
GroupID string


// Partition to read messages from.  Either Partition or GroupID may
// be assigned, but not both
Partition int

https://godoc.org/github.com/segmentio/kafka-go#ReaderConfig