pluginsplugin-datagencore

Generate​Generate

Generate data

This task generates data, using a configured data generator. Generators can support Datafaker

yaml
type: "io.kestra.plugin.datagen.core.Generate"
yaml
id: datagen_person_json
namespace: com.example.datagen

tasks:
  - id: datagen
    type: io.kestra.plugin.datagen.core.Generate
    generator:
      type: io.kestra.plugin.datagen.generators.JsonObjectGenerator
      locale: ["fr", "FR"]
      value:
        name: "#{name.fullName}"
        email: "#{internet.emailAddress}"
        age: 30
        address:
          city: "#{address.city}"
          zip: "#{address.zipCode}"
        skills: [ "#{job.keySkills}", "#{job.position}", "hardcoded" ]
        ts: "{{ now() }}"

  - id: log
    type: io.kestra.plugin.core.log.Log
    message: "Created: {{ outputs.datagen.value.name }} ({{ outputs.datagen.value.email }})!"
yaml
id: datagen_person_json_batch
namespace: com.example.datagen

tasks:
  - id: datagen
    type: io.kestra.plugin.datagen.core.Generate
    batchSize: 100
    store: true
    generator:
      type: io.kestra.plugin.datagen.generators.JsonObjectGenerator
      locale: ["fr", "FR"]
      value:
        name: "#{name.fullName}"
        email: "#{internet.emailAddress}"
        age: 30
        address:
          city: "#{address.city}"
          zip: "#{address.zipCode}"
        skills: [ "#{job.keySkills}", "#{job.position}", "hardcoded" ]
        ts: "{{ now() }}"

  - id: log
    type: io.kestra.plugin.core.log.Log
    message: "Created: {{ outputs.datagen.value.name }} ({{ outputs.datagen.value.email }})!"
yaml
id: datagen_person_csv
namespace: com.example.datagen
inputs:
  - id: rows
    type: INT
    displayName: "Number of Rows"
    description: "Number of data rows to generate (excluding header)."
    defaults: 10

  - id: separator
    type: STRING
    displayName: "Separator"
    description: "Column separator character (e.g., ',', ';')."
    defaults: ","

  - id: header
    type: BOOL
    displayName: "Include header"
    description: "Include header row in the CSV output."
    defaults: true
tasks:
  - id: datagen
    type: io.kestra.plugin.datagen.core.Generate
    generator:
      type: io.kestra.plugin.datagen.generators.StringValueGenerator
      locale: ["en", "US"]
      value: |
        #{csv '{{ inputs.separator }}','"','{{inputs.header}}','{{ inputs.rows}}','name_column','#{Name.first_name}','last_name_column','#{Name.last_name}'}

  - id: log
    type: io.kestra.plugin.core.log.Log
    message: |
        {{ outputs.datagen.value }}
Properties

The Data Generator

The data generator implementation responsible for producing the execution data.

Default 1

Batch size

The number of items to generate when storing data. This is only applicable when 'store' is set to true.

Default false

Store generated data

If true, the generated data will be persisted to Kestra's internal storage. If false, the data is emitted part of the task output.

Total number of the generated items

The total number of items generated by the task.

Size of the generated data

The total size in bytes of the generated data value or file.

Format uri

The generated file URI.

Generated data value

The actual content of the generated data. This can be a string, number, JSON object, or binary payload depending on the generator configuration.

The string value

The string value to generate for each output value

SubType string

Locales

List of locale values in the format language, country, variant (e.g., "en", "US", "fr", "FR"). Controls the language and region of the generated data.

Object to generate

A map of key-value pairs where values can contain Datafaker expressions (e.g., #{name.firstName}) to be evaluated for each output record.

SubType string

Locales

List of locale values in the format language, country, variant (e.g., "en", "US", "fr", "FR"). Controls the language and region of the generated data.

Default 0

Byte array size

The number of bytes to generate for each output value