CSV

CSV

CSV is a text file format, shortcut for Comma-Separated Values. It is a very simple way to store data, where each table record is on one line and values are listed one after the other with comma in between.

CSV is a de-facto standard in data exchanging and many applications allow data export in a CSV format. Such files can grow very large in size. Our applications can both read and write CSV in streaming mode, so we never have to load whole large file into memory - this also means we can easily handle huge CSV files, files many gigabytes in size. We don’t even impose any limit in file size. Larger files will take longer to process, but we can generally handle any file size.


Parsing CSV is very easy. Parsing all possible variants out there is not trivial at all.

When a value contains commas itself, as is the case in a text, value has to be quoted. Being such a simple format, inevitably many applications write it in a slightly different way. Some quote each value, some quote only values containing commas, some quote all textual values etc. There are differencing encoding quotes - some applications precede nested quotes with backslash character, some use two quote characters one after the other. CSV file values can in fact be delimited with a character different from comma - often TAB character is used (ASCII value 9). Depending if Windows or Linux/Unix/Mac system writes the file, lines can be written by two characters (ASCII 13, then 10) or a single character (ASCII 10). You shouldn’t store multiline text data in CSV files, because you’ll prematurely detect end of record.

We have developed our own CSV parser and employ a rich test suite to make sure it handles common differences in CSV files our user have. Even though CSV format doesn’t contain metadata with field types, we analyze the data and determine optimal data type of each field.

Full Convert supports CSV database directly. CSV is also known as TSV, Flat file, Comma-separated text, TAB-separated text (related files: csv, tsv, txt).

CSV data types we support

Integral

bigint, integer

Decimal

double (decimal)

Text

ntext, nvarchar

Date/Time

date, datetime, time

Large objects

blob, ntext

Other

logical

Export CSV database

It may make sense to migrate your data away from CSV. You may want to do it permanently or just need to share your tables with a collague in a different format.

We will copy all your tables with their data and apply indexing and relationships exactly as they are in your current CSV database. In a nutshell, you get exactly the same database in another database engine. Each time you run the migration, we will copy all the tables again. Of course, we have a built-in scheduler, so you can run this overnight and have a fresh database copy in the morning.

Take a look at the quick tutorials below to see how it's done.