Quantcast
Channel: Welcome To TechBrothersIT
Viewing all articles
Browse latest Browse all 2005

How to create CSV file per Excel File and Load All Sheets from Excel File to it in SSIS Package - SSIS Tutorial

$
0
0

Scenario: Script to Download

You are working as ETL Developer / SSIS developer. You got this requirement where you get single or multiple Excel files in Source or Input folder. Each File Can have single Sheet or Multiple Sheets.  If there are multiple Sheets they have same metadata( Same Columns).
You need to create cvs file per Excel file and load all sheets from it to newly created csv file. 

Here are sample files with multiple Sheet that I am going to use for this post. The Sheets on each Excel file has same number or columns.

How to create CSV File for each Excel File in SSIS Package by using Script Task

Customer_TechBrothersIT1.xlsx has two sheets in our case with columns Id, name and dob.
Create CSV files dynamically from Excel Files and Sheets - Script Task C#


Customer_TechBrothersIT2.xlsx has two sheets with columns id and name.
How to Create Single CSV File per Excel Sheet in SSIS Package and load data - Script Task C#

Solution:

We are going to use Script Task with C# as scripting language in our SSIS Package to preform this requirement. 

Step 1:
Create three variables as shown below.
SourceFolderPath: This is the folder where your Excel files should exists. 
DestinationFolderPath: This variable should hold the folder path where you want to create csv files
FileDelimited: You can provide comma(,) or Pipe( | ) or any other character you like to use as delimiter in your csv file. This will give us opportunity to change to any delimiter without changing the package. I have used comma in this post.

How to use variable in Script Task to Create CSV File for Excel Sheets Dynamically in SSIS


Step 2:
Get the Script Task from Toolbox and drag to Control Flow Pane. Open Script Task by double click and then map the variables as shown. 

How to map variables in Script Task for Creating csv file from Excel Sheet dynamically in SSIS


Step 3:
Click on Edit Script Button in above snapshot and it will open Script Task Editor. Under 
#region Namespaces
add using System.IO;

Then go to public void Main()
{
and paste below code


          string SourceFolderPath=Dts.Variables["User::SourceFolderPath"].Value.ToString();          string DestinationFolderPath = Dts.Variables["User::DestinationFolderPath"].Value.ToString();
string FileDelimited = Dts.Variables["User::FileDelimited"].Value.ToString();
var directory = new DirectoryInfo(SourceFolderPath);
FileInfo[] files = directory.GetFiles();

//Declare and initilize variables
string fileFullPath = "";

//Get one Book(Excel file at a time)
foreach (FileInfo file in files)
{
string filename = "";
fileFullPath = SourceFolderPath+"\\"+file.Name;
filename = file.Name.Replace("
.xlsx","");
MessageBox.Show(fileFullPath);

//Create Excel Connection
string ConStr;
string HDR;
HDR="
YES";
ConStr = "
Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + fileFullPath + ";Extended Properties=\"Excel 12.0;HDR=" + HDR + ";IMEX=0\"";
OleDbConnection cnn = new OleDbConnection(ConStr);


//Get Sheet Name
cnn.Open();
DataTable dtSheet = cnn.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
string sheetname;
sheetname="";
Int16 sheetcnt = 0;
foreach (DataRow drSheet in dtSheet.Rows)
{
sheetcnt += 1;
if (drSheet["TABLE_NAME"].ToString().Contains("$"))
{
sheetname=drSheet["TABLE_NAME"].ToString();
//Display Sheet Name , you can comment it out
// MessageBox.Show(sheetname);

//Load the DataTable with Sheet Data
OleDbCommand oconn = new OleDbCommand("select * from [" + sheetname + "]", cnn);
//cnn.Open();
OleDbDataAdapter adp = new OleDbDataAdapter(oconn);
DataTable dt = new DataTable();
adp.Fill(dt);

//drop $from sheet name
sheetname = sheetname.Replace("$", "");

//Create CSV File and load data to it from Sheet
StreamWriter sw = new StreamWriter(DestinationFolderPath+"\\"+filename+".csv", true);
int ColumnCount = dt.Columns.Count;
//we are checking SheetCnt=1, so put header in csv for only one time
if (sheetcnt == 1)
{
// Write the Header Row to File
for (int i = 0; i < ColumnCount; i++)
{
sw.Write(dt.Columns[i]);
if (i < ColumnCount - 1)
{
sw.Write(FileDelimited);
}
}
sw.Write(sw.NewLine);
}


// Write All Rows to the File
foreach (DataRow dr in dt.Rows)
{
for (int i = 0; i < ColumnCount; i++)
{
if (!Convert.IsDBNull(dr[i]))
{
sw.Write(dr[i].ToString());
}
if (i < ColumnCount - 1)
{
sw.Write(FileDelimited);
}
}
sw.Write(sw.NewLine);
}
sw.Close();
}
}


Step 5:
Save the script in Script Task Editor and then close the Editor windows. Now you are all set to run your SSIS Package to create cvs file per excel file that will load multiple sheets data to it.

As I have two sample files with two sheets per file,If I will run the package, It should create two csv files and load data from 4 sheets.

How to create csv file per Excel File and load data to it in SSIS Package - Script Task C#





Things we learnt in this post
How to create variables in SSIS Package and use them in Script Task
How to read data from single/ multiple Excel files in SSIS Package
How to read data from single or multiple excel sheets for a excel file in SSIS Package
How to create csv file and load data to it from datatable by using C# language
How to avoid writing header again to csv file when loading multiple excel sheets in SSIS Package
How to use C# to read data from Excel file and write to csv files




Viewing all articles
Browse latest Browse all 2005

Trending Articles