Quantcast
Channel: Welcome To TechBrothersIT
Viewing all articles
Browse latest Browse all 1976

How to Load Specific Sheet Number from Excel Files to SQL Server Table in SSIS Package - SSIS Tutorial

$
0
0

Scenario: Download Script

You are working as ETL Developer or SSIS Developer for Banking Firm. You get Single/Multiple Excel Files in Source Folder. Each Excel file has multiple sheets. You need to always load first sheet from each file to dbo.Customer table. You went ahead and created SSIS Package.

Few days later you got kind of same requirement but you need to load 2nd Sheet from each Excel file to table dbo.AsiaRegionCustomer Table.

From above statement if you understand, it would be great if you can create an SSIS Package with some variables so you can use the same SSIS Package even you need to load the data to different tables and from Different Sheet Numbers for multiple Excel Files. You take copy of your package and simply changes few values and you are ready to use it. 

How would you create an SSIS Package that should be dynamic to accept Schema Name, Table Name, Sheet Number and Folder path where Excel files and ready to use?


Solution:

With built in Excel Source in SSIS Package, this is not really possible. We need to use Script Task in SSIS Package so by changing values for variables, we can use this for different requirements.

Here are two excel files, I have. Each Excel file has multiple sheet. I would always like to load First Sheet from Excel file to dbo.Customer table. 

CREATETABLE dbo.Customer (
id INT
,name VARCHAR(50)
,dob DATE
)
GO
Excel file TechBrothersIT.com.xlsx has below sheets. I would like to load First sheet
Excel File with Multiple Sheet, Load only First Sheet to SQL Server Table by Script Task


TechBrothersIT.com_1.xlsx has below sheets.
How to Load specific Sheet from Excel File to SQL Server Table by using SSIS Script Task


If you noticed in Sheet Name in First Excel file=Sheet and Sheet Name in second Excel file=MySheet.
I am not considering the name of sheet. Instead of that I would like to use Sheet Number, As this is the first sheet. I am going to always load first sheet from each file, does not matter name is same or different.

Step 1: Create Variable in SSIS to Make your SSIS Package Dynamic
Create below variables in your SSIS Package
FolderPath: In this variable you will save the path from where you would like to read Excel Files
SchemaName: Schema of your Destination Table
TableName: Table in which you want to load the data
SheetNumberToLoad: Provide the Sheet Number you would like to load from each Excel File.
If your Sheet is always on 2nd number on each file. You will change the value to 2.


How to load specific sheet from excel files to SQL Server Table by using C# in Script Task in SSIS

Step 2: Create ADO.NET Connection in SSIS Package to use in Script Task
Create ADO.NET Connection Manager so we can use in Script Task to Load data from Excel Sheets to SQL Server Table.

Create ADO.NET Connection in SSIS Package to use in Script Task for Dynamic Excel Loading


Step3: Add Variables to Script Task to use from SSIS Package
Bring the Script Task on Control Flow Pane in SSIS Package and open by double clicking. Map the variables as shown below.

How to add variables to Script Task to load Specific Sheet Number for Excel to SQL Server Table



You have noticed that I selected Microsoft Visual C# 2012 for script in Script Task. You can use VB, if you like but below code is provided in C#.


Step 4: Add Script to Script task Editor in SSIS To load Specific Excel Sheet to SQL Server Table
Click Edit Button and it will open Script Task Editor.
Under #region Namespaces, I have added below code
using System.IO;
using System.Data.OleDb;
using System.Data.SqlClient;


Under public void Main() { 
I have added below code. 

String FolderPath = Dts.Variables["User::FolderPath"].Value.ToString();
String TableName = Dts.Variables["User::TableName"].Value.ToString();
String SchemaName = Dts.Variables["User::SchemaName"].Value.ToString();
Int16 SheetNumberToLoad = (Int16)(Dts.Variables["User::SheetNumberToLoad"].Value);
SheetNumberToLoad -= SheetNumberToLoad;
var directory = new DirectoryInfo(FolderPath);
FileInfo[] files = directory.GetFiles();

//Declare and initilize variables
string fileFullPath = "";


SqlConnection myADONETConnection = new SqlConnection();
myADONETConnection = (SqlConnection)(Dts.Connections["DBConn"].AcquireConnection(Dts.Transaction) as SqlConnection);

//Get one Book(Excel file at a time)
foreach (FileInfo file in files)
{
fileFullPath = FolderPath + "\\" + file.Name;

//Create Excel Connection
string ConStr;
string HDR;
HDR = "
YES";
ConStr = "
Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + fileFullPath + ";Extended Properties=\"Excel 12.0;HDR=" + HDR + ";IMEX=0\"";
OleDbConnection cnn = new OleDbConnection(ConStr);

//Get Sheet Name
cnn.Open();
DataTable dtSheet = cnn.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
string sheetname;
sheetname = "";
//Only read data from provided SheetNumber
Int16 sheetcnt = 0;
foreach (DataRow drSheet in dtSheet.Rows)
{
if (sheetcnt == SheetNumberToLoad)
{
if (drSheet["TABLE_NAME"].ToString().Contains("$"))
{
sheetname = drSheet["TABLE_NAME"].ToString();

MessageBox.Show(sheetname);

//Load the DataTable with Sheet Data so we can get the column header
OleDbCommand oconn = new OleDbCommand("select top 1 * from [" + sheetname + "]", cnn);
OleDbDataAdapter adp = new OleDbDataAdapter(oconn);
DataTable dt = new DataTable();
adp.Fill(dt);
cnn.Close();

//Prepare Header columns list so we can run against Database to get matching columns for a table.
string ExcelHeaderColumn = "";
string SQLQueryToGetMatchingColumn = "";
for (int i = 0; i < dt.Columns.Count; i++)
{
if (i != dt.Columns.Count - 1)
ExcelHeaderColumn += "'" + dt.Columns[i].ColumnName + "'" + ",";
else
ExcelHeaderColumn += "'" + dt.Columns[i].ColumnName + "'";
}

SQLQueryToGetMatchingColumn = "select STUFF((Select ',['+Column_Name+']' from Information_schema.Columns where Table_Name='" +
TableName + "' and Table_SChema='" + SchemaName + "'" +
"and Column_Name in (" + @ExcelHeaderColumn + ") for xml path('')),1,1,'') AS ColumnList";

// MessageBox.Show(SQLQueryToGetMatchingColumn);
// MessageBox.Show(ExcelHeaderColumn);


//Get Matching Column List from SQL Server
string SQLColumnList = "";
SqlCommand cmd = myADONETConnection.CreateCommand();
cmd.CommandText = SQLQueryToGetMatchingColumn;
SQLColumnList = (string)cmd.ExecuteScalar();

// MessageBox.Show(" Matching Columns: " + SQLColumnList);


//Use Actual Matching Columns to get data from Excel Sheet
OleDbConnection cnn1 = new OleDbConnection(ConStr);
cnn1.Open();
OleDbCommand oconn1 = new OleDbCommand("select " + SQLColumnList + " from [" + sheetname + "]", cnn1);
OleDbDataAdapter adp1 = new OleDbDataAdapter(oconn1);
DataTable dt1 = new DataTable();
adp1.Fill(dt1);
cnn1.Close();


//Load Data from DataTable to SQL Server Table.
using (SqlBulkCopy BC = new SqlBulkCopy(myADONETConnection))
{
BC.DestinationTableName = SchemaName + "." + TableName;
foreach (var column in dt1.Columns)
BC.ColumnMappings.Add(column.ToString(), column.ToString());
BC.WriteToServer(dt1);
}

}
}
sheetcnt += 1;
}

}


Step 5: Run and Test your SSIS Package if It is loading Specific Sheet Number from Excel Files into SQL Server Table

Save all changed in Script Task Editor and then run your SSIS Package. It should load first sheet from each Excel file in my case to dbo.Customer Table as I have set the value of Variable SheetNumberToLoad=1.
If you like to load any other Sheet Number from your Excel files. You can change value in this variable.

Here are my results after SSIS Package completed.

How to Import Specific Sheet Number Data from Multiple Excel Files to SQL Server Table by using SSIS Package Script Task.





Things we learnt in this post
How to load Data from Excel Files to SQL Server Table by using C#
How to read data from Specific Sheet Number from Excel File and load to SQL Table
How to read Excel File Name in SSIS Package
How to Check matching Header Column Names to SQL Server Table by using C sharp
How to save scalar string value from SQL Server Query to Variable in C Sharp
How to use BulkCopy to load data from DataTable to SQL Server Table
How to Map Columns Dynamically in BulkCopy C Sharp

Viewing all articles
Browse latest Browse all 1976

Trending Articles