Archive

Archive for the ‘SQLskills’ Category

A SQL Weekend – SQL Friends and SQL Saturday #119

May 22, 2012 4 comments

I had the pleasure of spending the weekend with some of the best and brightest folks in the SQL Server world. Friday I attended the second SQLFriends lunch with Aaron Lowe (b|t), Jes Borland (b|t), Ted Krueger (b|t) and a number of other great people. It was three hours of good food and better conversation. Any opportunity to spend time with Jes is welcome, as her energy is infectious. I had not met Ted Krueger before but it was a privilege to talk to him at the lunch and throughout the weekend. Many thanks to Aaron for organizing the event.

After lunch, I joined Aaron, Ted and others at DeVry University in Addison to help set up for SQL Saturday #119. After several hours of stuffing bags, I was off to the speaker dinner at Dave & Buster’s. It was a wonderful evening of conversation and camaraderie with the likes of Jes, Ted, Aaron, Brent Ozar (b|t), Tim Ford (b|t), Ben DeBow (b|t), Allan Hirt (b|t) and Bill Lescher (t). It was another reminder of the strength of the SQL community. The willingness of these experts to share their time and knowledge is really inspiring.

SQL Saturday 119 First Session 085
David Klee and Andy Galbraith on Virtualizing SQL Server

I was back at DeVry at 8 a.m. Saturday, having lost an epic battle with my snooze bar. By that point, other volunteers had registration under control so I spent some time catching up with people before the first session. It was Virtualizing Business Critical SQL Servers by David Klee (b|t) and Andy Galbraith (b|t) of House of Brick Technologies. Andy and David were in the SQLskills Immersion Event with me in April. Also, I worked with David and House of Brick on a VMWare proof of concept last month, so David asked me to talk about my experience.

For the next session, I planned on seeing Ted Krueger present on Query Tuning for Developers and DBAs. By the time I made it to the room it was standing-room-only. Instead, I went to hear Christina Leo (b|t) present on What’s Buried in the Plan Cache. Christina connected a number of concepts I was aware of, providing some very powerful tools for investigating and troubleshooting issues. After the session, I talked to Christina about the upcoming SQL Saturday she is running in Nashville October 13. You can find information about all of the upcoming SQL Saturdays here.

SQL Saturday 119 Fourth Session 014
Ted Krueger Talking About Installing SQL Server

After lunch, I went to the computer lab. To thank DeVry for donating its campus for SQL Saturday Chicago the past two years, three sessions were scheduled for DeVry students and alumni. Ted Krueger presented on installing SQL Server and the components of SQL Server (Analysis Services, Intergration Services and Reporting Services) and I talked about building your first database. I arrived in time to hear the second session while making sure everything was set for my presentation.

SQL Saturday 119 Fifth Session 146
Me Talking About ACID Properties

I got started around 4:15 p.m. and spoke for about an hour. I think I did alright, but I was so fried by the end of the day I don’t remember much. It was my first time presenting on SQL Server outside of work, and I am glad to have that under my belt. I am already looking at upcoming SQL Saturdays in the area with the idea of submitting abstracts.

I’ve come a long way since attending my first SQL Saturday last year. I am really happy to give back to the community that helped me get here.

SQL Server Log Space Reservation – Part II

April 30, 2012 2 comments

In my last post, I wrote about SQL Server reserving space for open transactions to account for potential rollback. This reserved space is released once the transaction commits. You can track the amount of space reserved using the sys.dm_tran_database_transactions DMO. In this post, I will demonstrate how expensive this space reservation can be for long-running transactions.

I created two databases to allow me to track log space reservation for two processes. The first will update all values in a 100,000-row tables 10 times, running all updates within single transaction. The second will run each update in an explicit transaction. Here is the script to create the database:

CREATE DATABASE [DUDLOOPTRAN]
 CONTAINMENT = NONE
 ON  PRIMARY 
( NAME = N'DUDLOOPTRAN_Data', FILENAME = N'D:\Data\DUDLOOPTRAN_01.mdf' , SIZE = 2097152KB , FILEGROWTH = 1048576KB )
 LOG ON 
( NAME = N'DUDLOOPTRAN_log', FILENAME = N'D:\Data\DUDLOOPTRAN_01.ldf' , SIZE = 8388608KB , FILEGROWTH = 2097152KB )
GO
ALTER DATABASE [DUDLOOPTRAN] SET COMPATIBILITY_LEVEL = 110
GO
ALTER DATABASE [DUDLOOPTRAN] SET ANSI_NULL_DEFAULT OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET ANSI_NULLS OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET ANSI_PADDING OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET ANSI_WARNINGS OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET ARITHABORT OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET AUTO_CLOSE OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET AUTO_CREATE_STATISTICS ON 
GO
ALTER DATABASE [DUDLOOPTRAN] SET AUTO_SHRINK OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET AUTO_UPDATE_STATISTICS ON 
GO
ALTER DATABASE [DUDLOOPTRAN] SET CURSOR_CLOSE_ON_COMMIT OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET CURSOR_DEFAULT  GLOBAL 
GO
ALTER DATABASE [DUDLOOPTRAN] SET CONCAT_NULL_YIELDS_NULL OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET NUMERIC_ROUNDABORT OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET QUOTED_IDENTIFIER OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET RECURSIVE_TRIGGERS OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET  DISABLE_BROKER 
GO
ALTER DATABASE [DUDLOOPTRAN] SET AUTO_UPDATE_STATISTICS_ASYNC OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET DATE_CORRELATION_OPTIMIZATION OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET PARAMETERIZATION SIMPLE 
GO
ALTER DATABASE [DUDLOOPTRAN] SET READ_COMMITTED_SNAPSHOT OFF 
GO
ALTER DATABASE [DUDLOOPTRAN] SET  READ_WRITE 
GO
ALTER DATABASE [DUDLOOPTRAN] SET RECOVERY FULL 
GO
ALTER DATABASE [DUDLOOPTRAN] SET  MULTI_USER 
GO
ALTER DATABASE [DUDLOOPTRAN] SET PAGE_VERIFY CHECKSUM  
GO
ALTER DATABASE [DUDLOOPTRAN] SET TARGET_RECOVERY_TIME = 0 SECONDS 
GO
USE [DUDLOOPTRAN]
GO
IF NOT EXISTS (SELECT name FROM sys.filegroups WHERE is_default=1 AND name = N'PRIMARY') ALTER DATABASE [DUDLOOPTRAN] MODIFY FILEGROUP [PRIMARY] DEFAULT
GO

CREATE DATABASE [DUDONETRAN]
 CONTAINMENT = NONE
 ON  PRIMARY 
( NAME = N'DUDONETRAN_Data', FILENAME = N'D:\Data\DUDONETRAN_01.mdf' , SIZE = 2097152KB , FILEGROWTH = 1048576KB )
 LOG ON 
( NAME = N'DUDONETRAN_log', FILENAME = N'D:\Data\DUDONETRAN_01.ldf' , SIZE = 8388608KB , FILEGROWTH = 2097152KB )
GO
ALTER DATABASE [DUDONETRAN] SET COMPATIBILITY_LEVEL = 110
GO
ALTER DATABASE [DUDONETRAN] SET ANSI_NULL_DEFAULT OFF 
GO
ALTER DATABASE [DUDONETRAN] SET ANSI_NULLS OFF 
GO
ALTER DATABASE [DUDONETRAN] SET ANSI_PADDING OFF 
GO
ALTER DATABASE [DUDONETRAN] SET ANSI_WARNINGS OFF 
GO
ALTER DATABASE [DUDONETRAN] SET ARITHABORT OFF 
GO
ALTER DATABASE [DUDONETRAN] SET AUTO_CLOSE OFF 
GO
ALTER DATABASE [DUDONETRAN] SET AUTO_CREATE_STATISTICS ON 
GO
ALTER DATABASE [DUDONETRAN] SET AUTO_SHRINK OFF 
GO
ALTER DATABASE [DUDONETRAN] SET AUTO_UPDATE_STATISTICS ON 
GO
ALTER DATABASE [DUDONETRAN] SET CURSOR_CLOSE_ON_COMMIT OFF 
GO
ALTER DATABASE [DUDONETRAN] SET CURSOR_DEFAULT  GLOBAL 
GO
ALTER DATABASE [DUDONETRAN] SET CONCAT_NULL_YIELDS_NULL OFF 
GO
ALTER DATABASE [DUDONETRAN] SET NUMERIC_ROUNDABORT OFF 
GO
ALTER DATABASE [DUDONETRAN] SET QUOTED_IDENTIFIER OFF 
GO
ALTER DATABASE [DUDONETRAN] SET RECURSIVE_TRIGGERS OFF 
GO
ALTER DATABASE [DUDONETRAN] SET  DISABLE_BROKER 
GO
ALTER DATABASE [DUDONETRAN] SET AUTO_UPDATE_STATISTICS_ASYNC OFF 
GO
ALTER DATABASE [DUDONETRAN] SET DATE_CORRELATION_OPTIMIZATION OFF 
GO
ALTER DATABASE [DUDONETRAN] SET PARAMETERIZATION SIMPLE 
GO
ALTER DATABASE [DUDONETRAN] SET READ_COMMITTED_SNAPSHOT OFF 
GO
ALTER DATABASE [DUDONETRAN] SET  READ_WRITE 
GO
ALTER DATABASE [DUDONETRAN] SET RECOVERY FULL 
GO
ALTER DATABASE [DUDONETRAN] SET  MULTI_USER 
GO
ALTER DATABASE [DUDONETRAN] SET PAGE_VERIFY CHECKSUM  
GO
ALTER DATABASE [DUDONETRAN] SET TARGET_RECOVERY_TIME = 0 SECONDS 
GO
USE [DUDONETRAN]
GO
IF NOT EXISTS (SELECT name FROM sys.filegroups WHERE is_default=1 AND name = N'PRIMARY') ALTER DATABASE [DUDONETRAN] MODIFY FILEGROUP [PRIMARY] DEFAULT
GO

And this is the code to create the tables:

USE DUDLOOPTRAN
GO

-- Create an populate a table in each database to use in the update loops

CREATE TABLE looptranu
(runnumber INT
,rundate DATETIME
,vartext VARCHAR(10)
,chartext CHAR(10))

USE DUDONETRAN
GO

CREATE TABLE onetranu
(runnumber INT
,rundate DATETIME
,vartext VARCHAR(10)
,chartext CHAR(10))

USE DUDLOOPTRAN
GO

INSERT INTO looptranu
VALUES
(1
,GETDATE()
,REPLICATE('a',10)
,REPLICATE('b',10))
GO 100000

USE DUDONETRAN
GO

INSERT INTO onetranu
VALUES
(1
,GETDATE()
,REPLICATE('a',10)
,REPLICATE('b',10))
GO 100000

After creating the databases and tables, I ran the following scripts. First, this script runs the updates in a single transaction:

USE DUDONETRAN
GO

-- Declare @loopcount variable to run the loop
-- Declare @textlen to vary the length of the text fields

DECLARE @loopcount INT
DECLARE @textlen TINYINT
SET @loopcount = 1

-- Begin an explicit transaction that will remain open for the duration of the loop
BEGIN TRAN

WHILE @loopcount <= 10
BEGIN

	-- Use the modulus operator to set text length to the remainder of @loopcount / 10

	SET @textlen = (@loopcount % 10)
	
	-- Update onetranu using the values described below
	UPDATE onetranu
	-- Set runnumber equal to @loopcount
	SET runnumber = @loopcount, 
	-- Set rundate equal the current datetime
	rundate = GETDATE(), 
	-- Set vartext to a string of a's, with the length determined by the @textlen variable
	vartext = REPLICATE('a',@textlen), 
	-- Set chartext to a string of b's, with the length determined by the @textlen variable
	chartext = REPLICATE('b',@textlen)

	-- Increment @loopcount
	SET @loopcount = @loopcount + 1

END

COMMIT

This code does each update in an explicit transaction:

USE DUDLOOPTRAN
GO

-- Declare @loopcount variable to run the loop
-- Declare @textlen to vary the length of the text fields

DECLARE @loopcount INT
DECLARE @textlen TINYINT
SET @loopcount = 1

-- Begin an explicit transaction that will remain open for the duration of the loop


WHILE @loopcount <= 10
BEGIN
	
	BEGIN TRAN
	-- Use the modulus operator to set text length to the remainder of @loopcount / 10

	SET @textlen = (@loopcount % 10)
	
	-- Update onetranu using the values described below
	UPDATE looptranu
	-- Set runnumber equal to @loopcount
	SET runnumber = @loopcount, 
	-- Set rundate equal the current datetime
	rundate = GETDATE(), 
	-- Set vartext to a string of a's, with the length determined by the @textlen variable
	vartext = REPLICATE('a',@textlen), 
	-- Set chartext to a string of b's, with the length determined by the @textlen variable
	chartext = REPLICATE('b',@textlen)

	-- Increment @loopcount
	SET @loopcount = @loopcount + 1
	
	COMMIT
	
END

Finally, this code will insert the contents of sys.dm_tran_database_transactions into a temp table for analysis. Run this while the update scripts are running:

USE master
GO

-- Create a temp table to hold the result set from sys.dm_tran_database_transactions

CREATE TABLE #transactions
(tranid BIGINT
,databaseid INT
,trantype INT
,transtate INT
,recordcount INT
,bytesused BIGINT
,bytesreserved BIGINT)

-- Run an infinite loop for the duration of each update script
-- stopping it when the script completes

WHILE 1 = 1
BEGIN

	INSERT INTO #transactions
	SELECT transaction_id AS [Tran ID], 
	database_id AS [Database ID], 
	database_transaction_type AS [Tran Type], 
	database_transaction_state AS [Tran State], 
	database_transaction_log_record_count AS [Log Record Count],
	database_transaction_log_bytes_used AS [Log Bytes Used],
	database_transaction_log_bytes_reserved AS [Log Bytes Reserved]
	FROM sys.dm_tran_database_transactions

END

-- Use the select statement below to see the results
-- This query will return a single row per transaction
-- for read/write activity (trantype = 1)
-- that has generated log records (transtate = 4)

SELECT tranid, 
MAX(recordcount) AS [Record Count], 
(MAX(bytesused)/1045876) AS [MB Used], 
(MAX(bytesreserved)/1045876) AS [MB Reserved] 
FROM #transactions
WHERE databaseid = DB_ID('DUDONETRAN')  -- Make sure to use the correct database name
AND trantype = 1
AND transtate = 4
GROUP BY tranid

-- Clean up the temp table after the run

--DROP TABLE #transactions

Here are the results of the single transaction:

DUDONETRAN Results

And here are the results of the explicit transactions in the loop:
DUDLOOPTRAN Results

 
As you can see, the total space used and reserved is approximately the same. The difference is the updates running in explicit transactions release the reserved space when each transaction commits. While the total reserved space for the explicit transaction is 344mb, 36mb is the most space reserved at any one time. However, the 342mb reserved for the single transaction is held for the duration of the transaction.

The examples I’ve shown here update 100,000 narrow rows 10 times. The cost of running updates or inserts to much wider rows millions of times would be exponentially greater. I hope these examples have demonstrated this.

SQL Server Log Space Reservation – Part I

April 27, 2012 1 comment

My biggest challenge in blogging about the SQLskills Internals Immersion Event is picking a topic to start with. I am going to choose log file space consumption because of an issue that occurred this week in our test environment.

An application ran a job that inserted over 400 million in a single transaction. After the job ran for more than 24 hours, the application requested the job be killed because it was “taking too long.” Go figure. The app was advised the rollback could take as long as the initial job run, and they decided to have the process killed. The rollback ended up taking about the same amount of time. When SQL Server rolls back a transaction, it generates and executes an anti-operation for each record in the log. In the case of 400 million inserts, 400 million deletes needed to be generated and executed.

While I was aware of this behavior going into the class, I learned that this scenario is even more costly because SQL Server reserves space in the log for any potential rollback activity. This means all of the log records for the inserts are written into the log and SQL reserves additional space for the rollback, more than doubling the amount of space needed. You can see the amount space reserved by a transaction in the sys.dm_tran_database_transactions dynamic management object in column database_transaction_log_space_reserved. The reserved space is released once the transaction commits. Additionally, because all of the log records for the large transaction are needed for a rollback, SQL Server cannot reuse the log file and it will continue to grow.

Both of these problems can be solved by running the single transaction in smaller batches. Committing more frequently will release the reserved undo space. It will also increase the probability SQL Server can reuse the existing log file. In my next post I will compare the log file use and reservation single long-running transaction to shorter, batched transactions performing the same activity.

SQL Friends Lunch – Friday, May 18th

April 25, 2012 Leave a comment

I have survived the week-long SQLskills Internals Immersion Event, having spent the better part of the weekend recovering. I am still organizing my notes, but expect to have a long series of blog posts up starting this week. It was the best, most challenging training I have taken, and I cannot recommend it highly enough.

I wanted to let people know about the second SQLFriends lunch on May 18th, 2012. The lunch will be hosted by SQL MVP Ted Krueger (blog | twitter). You can register for the lunch here. I attended the inaugural event hosted by Brent Ozar, and it was a great opportunity to talk with a SQL MVP and meet other DBAs.

The lunch is just down the road from SQL Saturday #119. As of today, registration is still open. I attended my first SQL Saturday in Chicago last year and it was great. The sessions were first-rate and I met some great people who are involved in the community. Register today!

Using an UPDATE Statement to Update a Column and Set a Variable

April 17, 2012 1 comment

I am a day and a half into the SQLskills Internals Immersion Event and I expect to have a lot more to post once I’ve recovered. Kimberly Tripp (blog | twitter) covered a feature of the UPDATE statement that I was unaware of. It allows you to update a column and set a variable with the updated value in a single statement. Here is an example:


-- Create a test table
CREATE TABLE UpdateTest
(id INT
,firstname VARCHAR(30)
,lastname VARCHAR(30)
,balance MONEY)

-- Insert a row into the test table
INSERT INTO UpdateTest
VALUES
(1
,'Skreeby'
,'Looby'
,1234.56)

-- Declare variables for use below
DECLARE @balance MONEY
DECLARE @debit MONEY
SET @debit = 500.00

-- Update table, updating the balance column equal to balance - @debit and setting
-- the @balance at the same time
-- NOTE: If more than one row is updated, your variable will contain the column value 
-- from the last row updated

UPDATE UpdateTest
SET @balance = balance = balance - @debit
WHERE id = 1

-- Select the variable and table
SELECT @balance AS [Balance]

SELECT * FROM UpdateTest

Here are the results:

Balance Results

 
I had no idea you could do this with an update statement and wanted to get it out there.