Data Import & ETL

Big Data. Simple Imports.

Load large CSV files into SQL Server, PostgreSQL or Sybase ASE without the file ever having to fit in memory.

Map columns, convert types, validate records and write in batches. Rows that fail are collected in a file shaped like the source, so you can fix them and import them again.

Stream files of any size

Available

Import Data reads the file as a stream: a plain CSV, a .gz archive or an entry of a .zip, in a Web Worker that never holds the whole file. Rows are parsed, mapped, batched and written to the database, and the reader only moves on when the batch has been saved.

The first 5,000 rows become the sample for type detection, mapping and the preview, so the wizard is instant even for a multi-gigabyte file.

  • CSV, TSV, TXT and pipe-separated files; gzip and zip archives
  • Delimiter, header and file settings with a live preview of the sample
  • Batched writes with progress and a pause on failure: Retry, Skip batch or Stop

Four steps from file to table

The wizard keeps you on a step until it is ready, so nothing is written half-configured.

1. Source

Pick the file and its format, choose an existing table or describe a new one, including a clustered columnstore index on SQL Server.

Available

2. Map & Transform

Match source columns to table columns, accept the detected types and add transforms per column.

Available

3. Preview & Validate

See the mapped rows and every validation problem in the sample before a single row is written.

Available

4. Import

Run the whole file in batches with Merge, Append or Bulk insert, and download the rejected rows and reasons.

Available

Map, transform, validate

Available

Column types are inferred from the sample, and new string columns are sized from the longest value with headroom. Each column can carry a chain of transforms: Trim, Upper case, Lower case, Date with a source format, Number with a decimal separator, Boolean with your true and false values, Replace, Default value, Null if, and a JSPython expression for everything else.

Validation mirrors the server conversion, so a bad value is rejected on the client instead of failing its batch. Rejected rows are written with their original cells, the reasons counted per field.

  • A transform editor with a live source-to-result view over the sample
  • Expressions see the cell, the whole row, the row number and the file name
  • Rejected rows download as a CSV shaped like the source, ready to fix and re-import

Built for the databases you load into

A new table is created with the inferred types: varchar when the sample is plain ASCII on SQL Server, nvarchar otherwise, with a primary key you choose. On SQL Server the Source step can add a clustered columnstore index, created before the first batch, for analytics-sized loads.

Into an existing table you choose Merge (upsert on the key), Append or Bulk insert. The same mapping and transform engine powers Save to Database, so a JSON file opened with Open File, or any grid, can be written to a table with the same care.

From raw files to reliable tables

  • Large-file streaming

    Available

    Any size, plain or compressed, in a background worker.

  • Schema inference

    Available

    Types and string sizes from a 5,000-row sample; new tables created for you.

  • Data mapping

    Available

    Suggested column matches you can override, with per-column transforms.

  • Validation and rejected rows

    Available

    Client-side checks, reasons per field, a re-importable rejects file.

  • JSON import

    Preview

    Open a JSON file and save it to a table; streaming JSON is planned.

  • Export

    Available

    Any grid out as CSV, TSV or JSON.

From Raw Data to Working Solutions.

One workspace to connect, query, explore, transform, automate and build.

Whether you are managing databases, engineering pipelines or creating applications, Falcon Data Studio helps you get more from your data.

Start with the modern Data IDE: the Windows desktop app, or open it in your browser.