TransformRequest

Payload to schedule a transformation job from a server-side source

specIdstringrequired

ID of the spec to use for parsing and validation

Example: acme-payments-v2

sourceTypeSourceType (string)

Where the source file comes from

Possible values: [FILE_UPLOAD, SFTP, S3, LOCAL_PATH]

Example: LOCAL_PATH

sourceFilePathstring

Absolute path to the source file (LOCAL_PATH or S3 URI)

Example: /data/payments/payments_20240315.txt

destinationTypestring

Where to publish transformed records

Default value: KAFKA_TOPIC

Example: KAFKA_TOPIC

kafkaTopicstring

Target Kafka topic name

Example: payments.inbound.raw

outputFilePathstring

Output file path (used when destinationType is a file)

Example: /data/output/payments_transformed.csv

skipInvalidRecordsboolean

Skip invalid records instead of aborting the job

Default value: false

delayMsinteger<int64>

Milliseconds to wait before starting the job (0 = immediate)

Default value: 0

Example: 0

cronExpressionstring

Cron expression for recurring execution (e.g. 0 2 * * * = 2 AM daily)

Example: 0 2 * * *

runOnceboolean

Run the job once then remove it (false = recurring cron job)

Default value: true

TransformRequest

{
  "specId": "acme-payments-v2",
  "sourceType": "LOCAL_PATH",
  "sourceFilePath": "/data/payments/payments_20240315.txt",
  "destinationType": "KAFKA_TOPIC",
  "kafkaTopic": "payments.inbound.raw",
  "outputFilePath": "/data/output/payments_transformed.csv",
  "skipInvalidRecords": false,
  "delayMs": 0,
  "cronExpression": "0 2 * * *",
  "runOnce": true
}