Skip to main content

Transform

Transform files into Kafka events and manage transformation jobs.

Overview​

The Transform API enables you to:

  • Parse files according to a file format specification
  • Publish records to Kafka topics in real-time
  • Schedule transformations to run at specific times
  • Track status of transformation jobs

Transformation Flow​

1. File Upload
    ↓
2. Parse according to Spec
    ↓
3. Apply validation/correction rules
    ↓
4. Publish to Kafka Topic
    ↓
5. Return status and metrics

Operations​

📄️ Transform file → Kafka events

Upload a file and immediately parse it using the referenced spec. Each record

📄️ Schedule a transformation

Schedule a transformation job from a server-side file path, SFTP server, S3 bucket,

📄️ Get transformation status

Poll the status of a transformation job using the `correlationId` returned when

Example: Transform a CSV File​

# 1. Create a spec (from Spec Management API)
# Copy the spec ID from response

# 2. Transform file to Kafka events
curl -X POST http://localhost:8080/api/v1/transform/file-to-events \
  -F "specId=<spec-id>" \
  -F "kafkaTopic=transform.records" \
  -F "file=@transactions.csv"

# 3. Check transformation status
curl http://localhost:8080/api/v1/transform/status/{correlationId}

Prerequisites​

  • A valid file specification (from Spec Management API)
  • File matching the specification format
  • Kafka topic available to publish to

Schema References​

See the Schemas section in the sidebar for detailed definitions. Key schemas for this API: