Best Open-Source Data ETL Program?

May 13, 2007 12:32 PM

What is the best open-source ETL (extract, transform, loading) tool for data? Esp. for loading text into SQL Server.

I do a lot of data ETL, loading text (and also DBF, Access, Excel) into and out of Microsoft SQL Server. I find SSIS to be pretty poor at handling text-files, esp. compared to DataJunction, a proprietary tool I also use.

I feel like there must be a good open-source solution, and I'm wondering what others recommend.

posted by jjsonp to computers & internet (8 comments total)

Honestly, I've never heard of "ETL" proper, but that sounds pretty much like what I'd use Perl for.
posted by kcm at 1:08 PM on May 13, 2007

I'm not the only one - here's an article on same.
posted by kcm at 1:11 PM on May 13, 2007

Why must it be open-source? SQL Server has DTS - Data Transformation Services, built-in.
posted by jkaczor at 4:08 PM on May 13, 2007

I use Perl for all my "ETL" (which seems like DBAspeak/DBA Buzzword).
posted by SirStan at 4:11 PM on May 13, 2007

ETL...yeah if you've never heard of it you probably won't know the answer as it's pretty narrowly specific. That's why I used the jargon, so that hopefully data-warehousers who use these tools would answer.

I use DTS and it's pretty good on some things...but SQL Server 2005 switched to SSIS, which frankly sucks compared to DTS for wizard-type stuff; its handling of DBF and text is horrible for simple in-and-out.

The reason I want open source is because I prefer using it when it's available; typically solutions are designed by people who actually use the tools for their intended purpose which seems to not be the case with SSIS. And ETL is becoming so commonplace that I figure there must be some decent open-source solution.
posted by jjsonp at 5:36 PM on May 13, 2007

We use DTS packages where I work, but then, we're using 2000, not 2005.
posted by Civil_Disobedient at 7:45 PM on May 13, 2007

Oh, and to answer your question about various open-source options: the only one's I'm aware of are KETL and the similar-sounding Kettle. Here's a web seminar on MySQL's page that talks about Kettle integration.
posted by Civil_Disobedient at 8:07 PM on May 13, 2007

Kettle is great, use it all the time. Much better than the DTS packages for SQL Server 2000.... not sure about 2005 though.
posted by gus at 9:37 PM on May 13, 2007