Data processing¶
Turn incoming Lambda events into validated application data, and handle retries without repeating completed work. Choose utilities according to the event source and the operation your handler performs.
Choose a utility¶
| Task | Guide |
|---|---|
| Process SQS, Kinesis or DynamoDB Streams records with partial failures | Batch processing |
| Prevent duplicate operation execution | DynamoDB idempotency or Redis/Valkey persistence |
| Parse event payloads into typed values | Parser |
| Check inputs and responses against JSON Schema | Validation |
| Query JSON and unwrap supported event envelopes | JMESPath |
| Decode Kafka payloads and preserve lazy codec boundaries | Kafka |
Compose a processing flow¶
Start from the Batch example when your handler receives records. Parse each record with an application-owned callback, validate the shape your application expects, and report failed records using the event source's documented response contract. Batch and Parser stay independently installable.
Use Idempotency around the operation whose repetition would duplicate a side effect. Review the persistence and expiration behavior in its guide instead of assuming every retry is safe. For Kafka, select the source or JSON delivery mode before adding Avro or Protobuf codecs.
See Security and configuration for masking sensitive fields and loading application settings.