Manual offset commit
In this example, we will run the application that continuously reads data from a Kafka topic. When the application reaches the last offset, we will move the offset to the beginning of the topic. To make the experiment a little more complicated, we will simulate a random application crash and check the application's behavior after the crash.
Implementation
- Clone repository https://github.com/pszymczyk/kafka-native-java-playground
- Switch to step4 branch
- Run SetupData
- After running, the following messages are written to the step4 topic:
- Little Miss Muffet
- Sat on a tuffet,
- Eating her curds and whey;
- There came a big spider,
- Who sat down beside her
- And frightened Miss Muffet away.
- Your task is to write a Kafka Consumer in the SubscribeRunner class. Consumer continuously reads messages and prints them on the terminal in exactly the order as above.
- The Consumer, after reading all the messages from the topic, should start reading the message sequence from the beginning
- The application occasionally throws a random error that closes the application. It is important that when the application is restarted, it does not read the entire sequence of messages from the beginning but continues to print from the last printed line, for example:
- Little Miss Muffet
- Sat on a tuffet,
- Error!
- Start an application
- Eating her curds and whey;
- There came a big spider,
- Who sat down beside her
- And frightened Miss Muffet away.
- Little Miss Muffet
- And so on...
- OpenĀ SubscribeRunner class
- Use following Kafka Consumer functionalities
- Disabled auto commit
- Set auto offset reset to earliest
- Pass small value to poll method, when poll returns empty iterator it means we are on the end of the topic
- If you have any problems, the solution to the task is on the master branch