Data Engineering
Build Pipelines. Transform Anything.
Import large datasets, connect different data sources and transform raw information into structured, usable data.
Turn repeatable operations into reusable ETL workflows, combining SQL, JSPython, databases, files and APIs.
The building blocks ship today
A pipeline is an import, a few transforms and some SQL, run in the right order. Each of those steps exists in Falcon Data Studio now; what is in development is saving the sequence and running it again.
- Streaming CSV import with mapping, typed conversion, validation and batched writes
- JSPython expressions evaluated for every row during an import or a Save to Database
- SQL scripts with GO batches, executed on one connection with every server message captured
- Save to Database from any grid or JSON file: Merge, Append or Bulk insert
- Export of any result as CSV, TSV or JSON for the next tool in the chain
Reusable pipelines
The planned pipeline editor saves what you do by hand today as a definition: a source (a file, a table, a query, later an API), a mapping with transforms, a destination and the SQL to run before and after. Run it on the next file, parameterise it, chain it with other pipelines across connections.
Pipelines will live in the workspace next to your scripts and saved queries, be shared with a team and be callable by authorised AI agents.
One definition for repeated loads instead of a wizard walk-through each time ELT between connections: extract with a query, load into another database Scheduling and triggers on the server host
From raw files to reliable data pipelines
Large-file streaming
Available CSV, gzip and zip sources of any size.
Schema inference
Available Types, sizes and new tables from a sample.
Data mapping
Available Column matching with per-column transform chains.
SQL transformations
Available Scripts with batches, messages and errors by line.
JSPython
Available Row-level expressions and script tabs.
ETL / ELT pipelines
Coming soon Saved, parameterised, repeatable sequences of steps.
Where data engineering in Falcon Data Studio is heading
The order of work: pipeline definitions first, then scheduling on the server host, then REST sources and destinations.
- Streaming import
Available The import wizard with mapping, transforms, validation and rejects.
- JSPython transforms
Available Expressions per column with access to the whole row.
- Save to Database
Available Grids and JSON files into tables with Merge, Append or Bulk insert.
- Pipeline definitions
Coming soon Saved sequences of import, transform and SQL steps.
- Scheduling
Coming soon Timers and triggers for pipelines on the server host.
- REST sources
Coming soon HTTP requests as pipeline steps.
From Raw Data to Working Solutions.
One workspace to connect, query, explore, transform, automate and build.
Whether you are managing databases, engineering pipelines or creating applications, Falcon Data Studio helps you get more from your data.
Start with the modern Data IDE: the Windows desktop app, or open it in your browser.