I am using samuel's go-zookeeper and have been encountering "Authentication failed: zk: session has been expired by the server" issue. Once this issue is encountered it does retry to reconnect to zookeeper and also get successfully connected. But the problem is it again gets "Authentication failed" issue or fails to register the consumers. Also, if it does get successfully connected and is Authenticated, consumer never consumes the message as "Currently registered consumers: 0" and this process continues unless I restart my client manually.
What might be the issue and its solution??
Here is the log:
[Scheduler] 2016/05/15 04:49:20 structs.go:21: Recv loop terminated: err=read tcp 10.244.0.9:41810->10.0.30.4:2181: i/o timeout
[Scheduler] 2016/05/15 04:49:24 structs.go:21: Send loop terminated: err=<nil>
[Scheduler] 2016/05/15 04:49:26 structs.go:21: Connected to 10.0.30.4:2181
[Scheduler] 2016/05/15 04:49:53 structs.go:21: Authentication failed: read tcp 10.244.0.9:43854->10.0.30.4:2181: i/o timeout
[Scheduler] 2016/05/15 04:49:57 structs.go:21: Connected to 10.0.30.4:2181
[Scheduler] 2016/05/15 04:50:25 structs.go:21: Authentication failed: read tcp 10.244.0.9:43863->10.0.30.4:2181: i/o timeout
[Scheduler] 2016/05/15 04:50:32 structs.go:21: Connected to 10.0.30.4:2181
[Scheduler] 2016/05/15 04:51:06 structs.go:21: Authentication failed: read tcp 10.244.0.9:43873->10.0.30.4:2181: i/o timeout
[Scheduler] 2016/05/15 04:51:55 structs.go:21: Connected to 10.0.30.4:2181
[Scheduler] 2016/05/15 04:52:08 structs.go:21: Authentication failed: zk: session has been expired by the server
[Sarama] 2016/05/15 04:52:08 [controller/fa956f688354] Triggering rebalance due to consumer list change
[Scheduler] 2016/05/15 04:52:10 structs.go:21: Connected to 10.0.30.4:2181
[Scheduler] 2016/05/15 04:52:10 structs.go:21: Authenticated: id=95829041879007139, timeout=4000
[Sarama] 2016/05/15 04:52:10 [controller/fa956f688354] Currently registered consumers: 0
[Sarama] 2016/05/15 04:52:10 [controller/fa956f688354] topic :: Started topic consumer
[Sarama] 2016/05/15 04:52:10 [controller/fa956f688354] topic :: Stopped topic consumer
[Sarama] 2016/05/15 04:53:22 client/metadata fetching metadata for all topics from broker 10.0.30.4:9092
[Sarama] 2016/05/15 05:03:23 client/metadata fetching metadata for all topics from broker 10.0.30.4:9092
.
.
.
[Sarama] 2016/05/15 04:53:22 client/metadata fetching metadata for all topics from broker 10.0.30.4:9092