Simple PS script to copy files from one share to another

XCopy is great but sometimes you need something a little more powerful. Powershell can give you the ability to perform more complex manipulations.

Here we’re finding the previous weekday and using it to find, copy and rename a set of files into an automated regression testing environment.

Continue reading “Simple PS script to copy files from one share to another”

Include HTML in a SSIS Configuration File

This stumped me for a while, but a useful pointer from a colleague reminded me how this can be achieved…

Quite simply as the configuration file is XML we can use CDATA so that the content won’t be parsed. Here’s an example below:

<!-- Package replaces {SQLResults} with an HTML table showing Count by User for the last loaded date -->
<Configuration ConfiguredType="Property" Path="\Package.Variables[User::EmailBody].Properties[Value]" ValueType="String">
<ConfiguredValue>
<![CDATA["
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body>
<div style="background-color:#fff;padding:20px;">
<table cellpadding="0" cellspacing="0">
<tr>{Message}</tr>
<tr>{SQLResults}</tr>
</table>
</body>
</html>
"]]>
</ConfiguredValue>
</Configuration>

Using the CDATA attribute inside the XML nodes will allow you to add HTML or any other XML tags inside your configuration file.

A simple restore script

This is a simple script that finds all (backup) files in a directory that match a pattern specified. It then derives the database name from the filepath and attempts to do a full restore. It should be executed in SQLCMD mode.

This is a script I created purely for a continuous integration setup on my local machine. Continue reading “A simple restore script”

SSIS – The script task junkie!

The script task in SSIS was a fantastic improvement on DTS Active X script. The reasons for this in my view are 2 fold:-

  1. Firstly, by allowing the user to code in Visual Studio it is far easier to develop and debug. Because the developer can make use of intellisense & object explorer, it means even the casual programmer can read & write code.
  2. Secondly, because we can use it to access other external .net assemblies, it provides functionality for logic that can’t be achieved using the standard SSIS components.

These are both reasons in themselves that make it a compelling choice when deciding how to develop your package logic.

However, I have noticed at both client sites and on the forums, that there are those (the script task junkies!) that use the script task in almost any scenario when it is just not necessary. It makes maintenance and support tricky at best and sometimes impossible…

Microsoft SQL Svr Developer Edtn 2008 English DVD (PC)

Continue reading “SSIS – The script task junkie!”

Excel VBA to prevent user generated errors in SSIS

I don’t know about you, but one of the things that drives me to distraction when writing SSIS packages is being forced to use excel as a data source. I’d like to qualify that statement by adding that it’s the use of excel in the face of all reason, to do the job that other software would be better for. Unfortunately though, corporations and people being what they are, they are a fact of life for an ETL developer.

Amazon Link: Excel 2007 Power Programming with VBA (Mr. Spreadsheets Bookshelf)

Continue reading “Excel VBA to prevent user generated errors in SSIS”

SSIS – Writing to a package variable in a dataflow

Here’s the scenario… You have a RAW file which contains data from many files.

In the subsequent dataflow you need to perform a lookup against a large reference table, however you want just a subset that reflects the period contained within your RAW file.

Question 1, how do you find out the earliest date used within your RAW file data? And question 2, how do you write it to a variable so that you can use it in the subsequent data flow?

Programming Microsoft SQL Server 2008 (PRO-Developer)

Continue reading “SSIS – Writing to a package variable in a dataflow”

SSIS – Writing SQL results to a string variable

In an earlier post, I demonstrated how to send HTML formatted mail using the script task. Another frequently requested use of the mail task is to be able to send query results within the email message body. Here’s how…

Introduction to Internet Marketing; Search Engine Optimization, Adword Marketing, Email Promotion, and Affiliate Programs

Continue reading “SSIS – Writing SQL results to a string variable”

Dynamic table loading in SSIS (Part 2)

I talked in a previous post about the possibility of using the format command and bulk insert task inside of a foreach loop in order to load all your tables using a single package… I don’t much like the method as it means you have to run each table load in series, and you’re not taking advantage of SSIS (high speed dataflow task, parallelism and eliminating staging with a single pass transformation).

SQL Server Developer Edition 2005 Win32/X64/IA64 English CD/DVD

Continue reading “Dynamic table loading in SSIS (Part 2)”

SSIS – download a file from a website

Some of the companies I’ve worked for have had reporting contracts with smaller 3rd party providers. Campaign or website analysis provided by them would have to be loaded to the datawarehouse

Security policy dissallowed downloading the files over FTP, because credentials are passed in clear text, and our 3rd party report providers didn’t have a sFTP facility, so in this scenario what other options are available?

Microsoft Office 2007 SharePoint Designer Edition (PC)

Continue reading “SSIS – download a file from a website”

SSIS system variable – ExecutionInstanceGUID

ExecutionInstanceGUID is a system variable of type string. If you are exporting this variable to a package management, custom logging or auditing table ensure that you have the correct mapping in your execute SQL task. It is a string variable and not a GUID (the same applies to other variables whose type may not be the same as the value they represent).

SQL Server 2005 Reporting Services Custom Controls

Continue reading “SSIS system variable – ExecutionInstanceGUID”