message.commit() For commiting specific message.

[aliowka]

When I want to commit only one message (a specific offset to a partition the message came from), I'm doing this:

from kafka import KafkaConsumer, OffsetAndMetadata

consumer = KafkaConsumer(TOPIC, enable_auto_commit=False)
message = consumer.poll(1000, max_records=1)
if message:
    offsets = {list(message.keys())[0]:
                OffsetAndMetadata(list(message.values())[0][0].offset + 1, '')}
    consumer.commit(offsets)

I wonder if this makes sense and if there is a simpler way of getting the offset {TopicPartition: OffsetAndMetadata} for a message?

Have you considered to add something like message.commit() method?

[jeffwidman]

This library follows Java's consumer / producer / client as much as possible, and I'm not aware of a message.commit() style of interface...

I understand it's confusing to figure out how to achieve this, Java might have added a suitable method in the AdminClient(#935) that might help, but not sure...

[aliowka]

Sounds reasonable.
Regarding the interface, there is message.finish() in NSQ python client, but I think it really does not matter where the method would be placed - it may be outside the message, something like consummer.commit_message(msg). Futhermore consummer.commit(offsets) receives offsets as an optional parameter. But there is no currently a simple way to get those offsets. One should go and build the {TopicPartition: OffsetAndMetadata} object manually..
But once again, the idea to follow the Java interface is totally understood.