Posts Tagged 'SQL Server 2005'

Database Code Deployments

One of the responsibilities of my current position as a SQL Developer is to prepare the scripts for deployment. For quality assurance reasons we store all of the database objects in a source control system, so each object is stored in a separate file. This can make deployments very tedious, depending on the volume of files.

Disclaimer

These processes are my processes for an internally managed application. These scripts and processes are not distributed in any way, and quite frankly are not suited for that purpose.

This does not cover any T-SQL best practices. There are far better articles written by far more knowledgeable people covering these topics. Click here for a list of best practice articles.

The Development

Here are my general guidelines for deployment, in no particular order:

1. Each script must be able to be executed multiple times without error.

There is nothing worse that running a script to test a deployment and then getting an error because an object already exists or a key violation because the data already exists. All scripts must check for the existence of the finals state before doing anything else. For stored procedures, this may me a conditional drop and then create and for tables this may mean checking for column existence before running an alter statement.

2. Object permission must be part of the object script.

Your mileage may vary with this one, but I am a fan of explicit permission granted to a database role for each database object that is being deployed. For newer architectures your roles may have blanket permissions to a schema, but I have not seen many instances of security set up this way.

Having the permission set at the end of the script (just in case you drop an re-create it :-)) will save a lot of headache if you ever need to setup the application in a new environment. NOTE: Please do not forget the batch separator (i.e. “GO”) in the script. I hate seeing permission errors where a stored procedure is trying execute permissions to itself.

3. Group the data scripts by table.

If you follow guideline #1 this shouldn’t be much of an issue, but if you have the same or worse, similar, updates in multiple places you could in a world of hurt.

4. Try to group like objects.

I try to keep like objects together. With some exceptions I will deploy objects in the following order:

  1. Tables – With the exception of using a function in a default value or check constraint, it is a pretty safe bet that you can deploy all table changes first without error. The most tedious part of deploying tables is their relationships. Since they are dependant on one another, order is important. I have not found a perfect way to deal with this dependency.
  2. Views – Again, it is possible to use functions as part of a view, but I wouldn’t recommend it. It is also possible to reference another view as part of a view definition, but I would not recommend that either. Pay attention to the order, just in case.
  3. Functions – In SQL Sever you cannot call a stored procedure as part of a scalar or table valued function, so deploying these before the stored procedures is advisable.
  4. Stored Procedures – These are next to last because they can use any of the previous three objects. They may, however, be dependant on one another, so watch the order here as well.
  5. Triggers – Triggers can be added to tables or view and they can use stored procedure or functions, so I save these for last.

5. Take as much care with the rollback scripts as you do with the future scripts.

What can be worse than beginning a deployment and then realizing that you have to rollback. After the rollback completes, you realize that you just dropped a primary table in your database that you just meant to alter. Oops.

For this reason I will hand write table rollback scripts. I use a script generator like ScriptDB to generate all of the other rollback scripts a week or so prior to deployment to capture any run fixes that were deployed during the development process.

I maintain a 1:1 relationship between deployment and rollback scripts. I execute the rollback scripts in reverse order to the deployments scripts. No muss, no fuss and no additional lists of scripts to maintain.

6. Don’t forget the jobs!

If you update requires job changes, add those to source control for deployment as well. All of the objects that are part of the deployment should be scripted.

7. Store the deployment order some place safe.

The largest deployment that I have worked on to date was over 600 individual scripts, with many exceptions to the steps listed above. Rebuilding that list would have taken a large amount of effort and testing, which I really did not have the time to do.

I prefer to store them in a database table table so that the list is available to other developers to make the necessary changes. I never want to be the one left holding the ball, let alone the one to drop, so let’s keep this in a safe place.

The Deployment

Take the list that you have so tediously maintained throughout the development process and script it into a file. I use sqlcmd to deploy all of the database objects, so this is a relatively simple process. Be sure that error handling is written into the script. I prefer that the batch ends when an error is encountered. If you followed #1 above, you can fix any errors and start the batch from the beginning.

The table that I use to store the scripts has the definition below. I have a template parameter for the database in the script. Use Ctrl+Shift+M or click the e “Specify Values for Template Parameters” button to populate this field.

USE <Database,,> ;
GO
IF EXISTS ( SELECT
                *
            FROM
                sys.objects
            WHERE
                object_id = OBJECT_ID(N'[dbo].[SQLScripts]') AND
                type IN ( N'U' ) )
    DROP TABLE [dbo].[SQLScripts]
GO
SET ANSI_NULLS ON ;
GO
SET QUOTED_IDENTIFIER ON ;
GO
SET ANSI_PADDING ON ;
GO
CREATE TABLE dbo.SQLScripts
    (
      SQLScriptOrder SMALLINT NOT NULL,
      SQLScriptDatabase VARCHAR(255) NOT NULL,
      SQLScriptName VARCHAR(255) NOT NULL,
      SQLScriptRelativePath VARCHAR(255) NOT NULL,
      CreateTimeStamp DATETIME NOT NULL
                               CONSTRAINT DF_SQLScripts_CreatedTimeStamp DEFAULT ( GETDATE() ),
      CreateUserID VARCHAR(50) NOT NULL
                               CONSTRAINT DF_SQLScripts_CreateUserID DEFAULT ( SUSER_SNAME() ),
      UpdateTimeStamp DATETIME NOT NULL
                               CONSTRAINT DF_SQLScripts_UpdatedTimeStamp DEFAULT ( GETDATE() ),
      UpdateUserID VARCHAR(50) NOT NULL
                               CONSTRAINT DF_SQLScripts_UpdateUserID DEFAULT ( SUSER_SNAME() ),
      CONSTRAINT PK_SQLScripts PRIMARY KEY CLUSTERED ( SQLScriptOrder ASC ),
      CONSTRAINT AK_SQLScripts UNIQUE ( SQLScriptDatabase ASC, SQLScriptName ASC )
    ) ;
GO
CREATE TRIGGER dbo.SQLScriptUpdate ON dbo.SQLScripts
    AFTER UPDATE
AS
UPDATE
    ss
SET
    UpdateTimeStamp = GETDATE(),
    UpdateUserID = SUSER_NAME()
FROM
    dbo.SQLScripts ss
WHERE
    EXISTS ( SELECT
                1
             FROM
                INSERTED i
             WHERE
                i.SQLScriptDatabase = ss.SQLScriptDatabase AND
                i.SQLScriptName = ss.SQLScriptName ) ;

To create the batch file used for deployment I use to SQL outlined below. I find this option to be very flexible. Run the result to text. I hate to use cursors like everyone else, but this ensures that I get the proper format out.

SET NOCOUNT ON ;
DECLARE
    @Server VARCHAR(50),
    @LocalPath VARCHAR(255),
    @SQLScriptRelativePath VARCHAR(255),
    @SQLScriptName VARCHAR(255),
    @Command VARCHAR(1000) ;

SET @Server = '(local)' ;
SET @LocalPath = 'C:\Temp' ;

PRINT 'REM Deployment Script Generator'
PRINT 'REM Generated On: ' + CONVERT(VARCHAR(10), GETDATE(), 101)
PRINT 'REM Server Name: ' + @Server ;
PRINT '' ;

DECLARE ScriptCursor CURSOR LOCAL FORWARD_ONLY READ_ONLY
    FOR SELECT
            SQLScriptRelativePath,
            SQLScriptName
        FROM
            dbo.SQLScripts
        ORDER BY
            SQLScriptOrder ;

OPEN ScriptCursor ;

FETCH NEXT FROM ScriptCursor INTO @SQLScriptRelativePath,@SQLScriptName ;
WHILE @@FETCH_STATUS = 0
    BEGIN
        PRINT 'REM ' + @SQLScriptRelativePath + '\' + @SQLScriptName ;
        PRINT 'sqlcmd -S ' + @Server + ' -i "' + @LocalPath + '\' + @SQLScriptRelativePath + '\' + @SQLScriptName + '"' ;
        PRINT 'if not %errorlevel%==0 goto :error' ;

        FETCH NEXT FROM ScriptCursor INTO @SQLScriptRelativePath,@SQLScriptName ;
    END ;

CLOSE ScriptCursor ;
DEALLOCATE ScriptCursor ;

PRINT '' ;
PRINT 'pause' ;
PRINT 'exit' ;
PRINT '' ;
PRINT ':error' ;
PRINT 'echo ERROR ENCOUNTERED' ;
PRINT 'pause' ;

Take the output of this script and save it as a batch file and VIOLA! Change the local path and order the scripts in reverse order to generate the rollback batch file.

Now you have a quick, dependable and free SQL deployment method.

Happy querying,

~Ron

Advertisements

Refeshing Views

Running on a recently updated to SQL Server 2005 cluster, we were noticing some performance issues with one stored procedure in particular. We initially blamed the issue on optimizer changes, but were looking a fix that did not involve and object change.

Someone came up with the idea (not me) to refresh the view. I am aware that a view that contains the dreaded “SELECT *” may not contain new table columns until the view is refreshed, but I have never actually run into this issue in real life. In our situation the view definition had not changed, nor had the underlying objects, but refreshing the view resolved the performance issues.

So on that note i wrote a quick little cursor up refresh all views in a database. I hope someone finds this useful.

/*******************************************************************************
Name:			Refresh Views
Description:	Declare our variables and cursor. The cursor looks at and 
                schema bound views since the are not able to be refreshed.
Dependencies:	
Testing:		
********************************************************************************
Author - Date - Change Description
--------------------------------------------------------------------------------
Ron Carpenter - 07/01/2010 - Initial Version
*******************************************************************************/
DECLARE @View varchar(255) ;

DECLARE ViewCursor CURSOR LOCAL READ_ONLY FORWARD_ONLY
    FOR SELECT
            [Name]
        FROM
            sys.objects o
        WHERE
            Type_Desc = 'VIEW' AND
            NOT EXISTS (SELECT
                            1
                        FROM
                            sys.sql_dependencies d
                        WHERE
                            o.object_id = d.object_id AND
                            d.class = 1)

OPEN ViewCursor ;

FETCH NEXT FROM ViewCursor INTO @View ;
WHILE @@FETCH_STATUS = 0 
    BEGIN
        EXECUTE sp_refreshview @View ;

        FETCH NEXT FROM ViewCursor INTO @View ;
    END ;

CLOSE ViewCursor ;
DEALLOCATE ViewCursor ;

Efficient way to do Paging in SQL Server

I was going through SQLServerCentral.com and found this pretty nice article to do paging in an efficient manner. So thought of sharing with you all and ofcourse it is a note to myself 🙂


WITH Keys
AS
(
 SELECT
 TOP (@PageNumber * @PageSize)
 rn = ROW_NUMBER() OVER (ORDER BY P1.Post_ID ASC)
 P1.Post_ID
 FROM
 dbo.Post P1
 ORDER BY
 P1.Post_ID ASC
),
SelectedKeys AS
(
 SELECT
 Top(@PageSize)
 SK.rn,
 SK.Post_ID
 FROM
 Keys SK
 WHERE
 SK.rn >((@PageNumber -1 ) * @PageSize)
 ORDER BY
 SK.Post_ID ASC
)
SELECT
 SK.rn,
 P2.Post_ID,
 P2.Thread_ID,
 P2.Member_ID,
 P2.Created_Date,
 P2.Title,
 P2.Body
FROM
 SelectedKeys SK
JOIN
 dbo.Post P2
ON
 P2.Post_ID = SK.Post_ID
ORDER BY
 SK.Post_ID ASC

Happy Programming!!!

Cheers,

Raja

Getting column description in SQL Server 2005

We are working on code generation and thought it would be neat to pull information from description (where we intend to put validations for ex. DOB – No future date).  The following query did that trick.


SELECT
 [Table Name] = i_s.TABLE_NAME,
 [Column Name] = i_s.COLUMN_NAME,
 [Description] = s.value
FROM
 INFORMATION_SCHEMA.COLUMNS i_s
LEFT OUTER JOIN
 sys.extended_properties s
ON
 s.major_id = OBJECT_ID(i_s.TABLE_SCHEMA+'.'+i_s.TABLE_NAME)
 AND s.minor_id = i_s.ORDINAL_POSITION
 AND s.name = 'MS_Description'
WHERE
 OBJECTPROPERTY(OBJECT_ID(i_s.TABLE_SCHEMA+'.'+i_s.TABLE_NAME), 'IsMsShipped')=0
 --AND i_s.TABLE_NAME = 'table_name'
ORDER BY
 i_s.TABLE_NAME, i_s.ORDINAL_POSITION

Hope this helps someone.

Happy Programming!!!

Cheers,

Raja

Snippets for SQL Server Management Studio (SSMS)

How many times have you had to open an existing SQL object and copy the header information to add to a new procedure, view or function? How many times have you done this and forgot to change the information in the header? Do you like standards? If you answered “Yes!” to any of these questions then templates are for you! Microsoft SQL Server Management Studio comes with a plethora of built-in templates, but this little gem of a feature also allows you to create your own!

To open template explorer, select View -> Template Explore or press Ctrl+Alt+T. This will open a side bar (by default, but you can change it) with the templates installed by default. To add your own templates, right-click on “SQL Server Templates” and then “New” from the context menu. This will give you the option to create a new folder or template. I prefer to keep my user-created templates in their own folder, since that makes it easier to transfer templates between machines.

Let’s create a folder called “CodeSnippets” as shown in the screen shot below.

Template Context Menu (Folder)

Template Context Menu (Folder)

Now that we have our folder created, let’s create some templates. We will create a basic header template and a stored procedure template. To create our templates, Right click on the newly create folder and select New -> Template from the context menu.

Template Context Menu (Template)

Template Context Menu (Template)

Now that we have the blank templates created, let’s add some content. Right click on the template and select “Edit” from the context menu.

Template Context Meny (Edit)

Template Context Meny (Edit)

From here you can write or paste your content into the template.After creating your templates, usage is as simple as dragging and dropping your template into the proper location in the query window.

I mentioned earlier about transferring templates between machines. To move your templates between machines browse to

C:\Documents and Settings\[User Profile]\Application Data\Microsoft\Microsoft SQL Server\[90 for SQL 2005 or 100 for SQL 2008]\Tools\Shell\Templates\Sql\[Your Snippet Folder Name]

for Windows XP and Windows Server 2003 or

C:\Users\[User Profile]\AppData\Roaming\Microsoft\Microsoft SQL Server\[90 for SQL 2005 or 100 for SQL 2008]\Tools\Shell\Templates\Sql\[Your Snippet Folder Name]

for Windows Vista, Windows 7 and Windows Server 2008. (Please replace the items in [] with your specific value for your environment) and copy the data to the appropriate path on the source machine.

I hope you find this as useful as I do! Happy coding!

~Ron

Search SQL Server Reporting Services Report Definition

A little over a year ago, Raja and I were having a discussion about reporting services. The discussing revolved around writing the T-SQL in the report definition or creating stored procedures for each dataset and executing the stored procedure in the report definition.

However, one of the drawbacks of keeping the T-SQL in the report definition is that you cannot just simply search in syscomments to find objects that may be affected by a schema change. But searching the report definition is still possible! A report definition file is simply XML stored in the ReportServer database in the Catalog table. (It is stored using the image data type, even in SQL 2008. What is up with this, Microsoft? I thought the image data type was deprecated in favor of varbinary(max).)

So the first thing that we need to do is convert the image data to XML. You cannot explicitly convert image to XML, but you can explicitly convert image to varbinary and varbinary to XML, so that is what we will do. Next we will shred the XML using the nodes() method. This is the most effective method since a report definition can contain multiple data sets, and we want to capture all of them.

The SQL below will create a view with the report path, report name, data set name and the T-SQL for easy searching.

NOTE: The XML namespace may be different, depending on the version you are using. If this does not work for you at first, run the T-SQL from the derived table and click on the XML. The top line of the XML will contain the namespace corresponding to the version you are using.


USE [ReportServer]
GO
IF EXISTS (SELECT
 *
 FROM
 sys.views
 WHERE
 object_id = OBJECT_ID(N'[dbo].[ReportDefinition]'))
 DROP VIEW [dbo].[ReportDefinition]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE VIEW [dbo].[ReportDefinition]
AS
WITH XMLNAMESPACES
(
'http://schemas.microsoft.com/sqlserver/reporting/2008/01/reportdefinition' AS RES
)
SELECT
 c.Path,
 c.Name,
 DataSet.value('@Name','varchar(MAX)') DataSourceName,
 DataSet.value('RES:Query[1]/RES:CommandText[1]','varchar(MAX)') CommandText
FROM
 (SELECT
 ItemID,
 CAST(CAST(Content AS varbinary(max)) AS xml) ReportXML
 FROM
 dbo.Catalog
 WHERE
 Type = 2) ReportXML
CROSS APPLY ReportXML.nodes('RES:Report/RES:DataSets/RES:DataSet') DataSetXML (DataSet)
INNER JOIN dbo.Catalog c
 ON ReportXML.ItemID = c.ItemID
GO

Happy coding!

~Ron

SQL Function to get desired characters

We had a requirement to check the data in a table by stripping off special characters since the match in the filter was getting too much (ex. Robin Jr. wont match Robin Jr but we wanted to match it). So searched for a generalized function which would do the job and found this:

CREATE FUNCTION dbo.GetCharacters(@myString varchar(500), @validChars varchar(100))
RETURNS varchar(500) AS
BEGIN

 While @myString like '%[^' + @validChars + ']%'
 Select @myString = replace(@myString,substring(@myString,patindex('%[^' + @validChars + ']%',@myString),1),'')

 Return @myString
END
Go

So while writing the where clause I used this function and it works like a charm.


Select ColumnA, ColumnB From TableA WHERE dbo.GetCharacters(ColumnB,'a-z') = dbo.GetCharacters(@SearchItem,'a-z')

You can use the above function to get just characters (a-z) or just numbers (0-9) or alpha numeric (0-9a-z) or specific characters like (abcd). This function is pretty effective since it uses regular expression to do the job.

Hope this helps.

Happy Programming!!!

Cheers,
Raja