skreebydba

I work in a clustered environment running more than 200 instances of SQL Server 2008. Several years ago, we began having problems with the SQL Server Agent service failing after an instance failed over. We had already developed a process to check the health of the SQL Server service, so I set out to add an Agent check to it.

Doing some web searching, I came across a number of links that suggested running the following query to determine if SQL Server Agent was running:

SELECT COUNT(*) 
FROM master.dbo.sysprocesses 
WHERE program_name = N'SQLAgent - Generic Refresher'

In the tests I ran, this worked just fine. I put it into a procedure, rolled it out to our test environment and let it run for a week, where we had no issues. It caught a number of times that the Agent was down and provided no false positives. At this point I rolled the proc out to production.

Several weeks later, we started receiving messages that the SQL Agent was down on an instance. Management Studio and Windows Failover Cluster Management showed the process was up and running. We were able to access Agent jobs and metadata, and jobs were running successfully. I queried the dbo.sysprocesses DMO on the instance and did not get a hit for 'SQL Agent - Generic Refresher'.

This led me to open a ticket with Microsoft to find out if they could tell me why there was no row in the sys.processes DMO when the process was running. If they couldn't do that, I was hoping they could tell me a better way to check status programmatically. I was told that querying the sys.processes table was not recommended and that they could not provide me with any other suggestions.

At that point, other things took precedence until two weeks ago, when we started receiving messages the Agent was down on an instance. Taking a look at the sys.processes table, there was no row for 'SQLAgent - Generic Refresher'.

The instance that was failing was excluded from the job and I worked on another method of checking the Agent service. Since the last time I looked at this issue, I gained experience using xp_cmdshell in T-SQL code. This led me to use the following command:

EXEC xp_cmdshell 'tasklist'
GO

This code returns a list of all of the processes running on the machine. By inserting the results of this code into a temp table, you can run a select to check for the existence of the SQL Agent process. That code looks like this:

USE master
GO

CREATE TABLE #processes
(processlist VARCHAR(4000))

INSERT INTO #processes
EXEC xp_cmdshell 'tasklist'

SELECT * FROM #processes
WHERE processlist LIKE '%sqlagent%'

DROP TABLE #processes

Here are the query results:

I've put that block of code into a procedure that loops through all of the instances in our environment checking the status of SQL Server Agent. If the check fails, an e-mail is sent out to the DBA team. I will cover this procedure in my next post.