🔹 Default Maximum Message Size in Kafka

• By default, Kafka has a maximum message size of 1 MB.
• This is controlled by the broker property:

message.max.bytes=1048576

(= 1 MB = 1,048,576 bytes).

🔹 Can it be Increased? ✅

Yes, Kafka allows larger messages, but you must configure it in multiple places:

1. Broker Side

   • message.max.bytes (max size a broker will accept for a single message).

2. Topic Level (Optional)

   • Can override using:

   kafka-topics.sh --create \
     --topic large-messages \
     --config max.message.bytes=52428800
   

   (e.g., 50 MB for this topic).

3. Producer Side

   • max.request.size (default 1 MB).

   • Increase it to match the largest message you want to send.

   props.put("max.request.size", 52428800); // 50 MB
   

4. Consumer Side

   • fetch.message.max.bytes (old name) or max.partition.fetch.bytes (new name).

   • Controls the maximum size of a message batch the consumer will fetch.

   props.put("max.partition.fetch.bytes", 52428800); // 50 MB
   

👉 All three must align (producer, broker, consumer), or you’ll hit errors.

🔹 Practical Considerations

• Kafka is optimized for many small messages, not giant ones.
• Very large messages can:
  • Cause high memory usage (serialization, deserialization).
  • Lead to longer GC pauses in JVM.
  • Break batching efficiency → reduced throughput.

📌 Best Practice: Keep messages small (KB range). For large payloads:

• Store data in object storage (e.g., S3, HDFS, MinIO).
• Send only a reference (URL, ID, metadata) in Kafka.

🔹 In Short

• Default max message size = 1 MB.
• Can be increased via:
  • Broker → message.max.bytes
  • Producer → max.request.size
  • Consumer → max.partition.fetch.bytes
• ✅ Possible to send 10MB, 50MB, or even more… but ❌ not recommended for performance.