Skip to main content

The Purge SQL Agent Job for MDW takes a long time to complete

I use the dbWarden alerts to inform me if a SQL job is taking longer to complete than normal and I got one this morning:
 
Click to enlarge

I noticed by looking at the history this purge job was gradually taking longer and longer to complete each day since I installed it again (see mylast post on this):

RunDate
RunTime
Duration
ExecutionStatus
JobName
04/10/2013
02:00:01
00:00:45
Succeded
mdw_purge_data_[MDW]
05/10/2013
02:00:00
00:13:27
Succeded
mdw_purge_data_[MDW]
06/10/2013
02:00:00
00:17:03
Succeded
mdw_purge_data_[MDW]
07/10/2013
02:00:01
00:30:58
Succeded
mdw_purge_data_[MDW]
08/10/2013
02:00:01
01:04:25
Succeded
mdw_purge_data_[MDW]
09/10/2013
02:00:01
01:41:39
Succeded
mdw_purge_data_[MDW]
10/10/2013
02:00:01
02:35:17
Succeded
mdw_purge_data_[MDW]
11/10/2013
02:00:00
03:34:21
Succeded
mdw_purge_data_[MDW]
12/10/2013
02:00:01
04:12:51
Succeded
mdw_purge_data_[MDW]
13/10/2013
02:00:01
04:54:04
Succeded
mdw_purge_data_[MDW]
14/10/2013
02:00:00
05:38:58
Succeded
mdw_purge_data_[MDW]
15/10/2013
02:00:01
07:07:13
Succeded
mdw_purge_data_[MDW]
16/10/2013
02:00:00
08:21:11
Cancelled
mdw_purge_data_[MDW]

The last job I cancelled so I could figure out what was going on. A quick Google took me to this page which recommended adding a couple of indexes. Before I did so I took a look at the size of both the tables mentioned in the article:

sp_spaceused  '[snapshots].[query_stats]'

name
rows
reserved
data
index_size
unused
query_stats
5,702,294
599072 KB
587616 KB
11320 KB
136 KB

sp_spaceused '[snapshots].[notable_query_text]'

name
rows
reserved
data
index_size
unused
notable_query_text
26,383
34872 KB
33616 KB
272 KB
984 KB

...And the execution plan of the following query before I made any changes:

SELECT qt.[sql_handle]
INTO #tmp_notable_query_text_test
FROM snapshots.notable_query_text AS qt
WHERE NOT EXISTS (
   SELECT snapshot_id
   FROM snapshots.query_stats AS qs
   WHERE qs.[sql_handle] = qt.[sql_handle])

Execution Plan Before

Click to enlarge

You can clearly see that the scan of CIDX_query_stats costs by far the most of any operation in the query.
So I applied the following indexes as recommended in the article and looked at the execution plan of the query again:

USE [MDW]
GO

SET ANSI_PADDING ON
GO

CREATE NONCLUSTERED INDEX [IX_sql_handle]
    ON [snapshots].[query_stats]
          (
          [sql_handle] ASC
          )
       GO

CREATE NONCLUSTERED INDEX [IX_sql_handle]
    ON [snapshots].[notable_query_text]
          (
          [sql_handle] ASC
          )  
GO




Execution Plan After

Click to enlarge

Now various changes have happened to the query play including a Nested Loop operator replacing the Hash Match and an Index Seek replacing the Index Scan.

I ran the job manually and it completed much faster :

RunDate
RunTime
Duration
ExecutionStatus
JobName
16/10/2013
10:41:56
00:05:32
Succeded
mdw_purge_data_[MDW]

But much of the hard work would have already been done from the previous night’s schedule, so I will need to wait until tonight to see if the time had be reduced.

**Update**

It looks like the queries have done their job. The  duration has greatly improved:

RunDate
RunTime
Duration
ExecutionStatus
JobName
17/10/2013
02:00:00
00:21:00
Succeded
mdw_purge_data_[MDW]


A word of warning

Even though the MDW has been active for only two weeks and I am monitoring only one medium-busy server, nearly 14GB of data has been generated:

Click to enlarge




Comments

  1. Very useful. I have many customers that simply LOVE the Data Collector interface. Unfortunately, DC is full of problems (random errors in collection jobs, database growing too much, report errors).

    ReplyDelete

Post a Comment

Popular posts from this blog

How to move the Microsoft Assessment and Planning Toolkit (MAP) database to a different drive

The Microsoft Assessment and Planning Toolkit (MAP) is a very useful tool for scanning your network to find instances of SQL Server plus all manner of detailed information about the installed product, OS and hardware it sits on.


<Click image to enbiggen>
There is an issue with it the database it uses to store the data it collects, however. Assuming you don't have an instance called MAPS on your server, the product will install using LocalDB (a cut down version of SQL Server Express) and puts the databases on your C: drive. If you then scan a large network you could easily expand the database to 10GB which may cause issues on a server when that drive is often one of the smallest. However, there is a simple solution: connect to LocalDB using Management Studio, detach the databases, move to a different drive, set permissions on the new location if required and reattach the database. How do you connect to LocalDB? Here you go:

Connect to (localdb)\MAPTOOLKIT


The databases I move…

Fun and games with the Management Data Warehouse (MDW and Data Collectors)

The SQL Server Management Data Warehouse (when you first come across it) seems to promise so much if the verbiage from Microsoft and some other websites is to to believed. But when you install it you may find that it is not as useful as it could be. This is a shame but we are currently only on v2 of the product with SQL 2012 so one hopes it will improve in subsequent versions.

However, it probably is worth playing with if you have never used it before - at least you can show your boss some reports on general server health when he asks for it and you have nothing else in place.

There is one big problem with it though if you decide that you don't want to use it any more, uninstalling it is not supported! Mad, I know. But as usual some very helpful people in the community have worked out, what seems to me, a pretty safe way of doing it.

I had a problem with my MDW. The data collector jobs were causing a lot of deadlocking on some production servers and impacting performance. It looks…

SSIS Job fails when it calls Excel via the SQL Agent but succeeds from SSDT

If you have an SSIS package which fails when run on a schedule but succeeds when executed interactively in Visual Studio/BIDS/SSDT, it may produce an error like this:
Executed as user: DOMAIN\user. Microsoft (R) SQL Server Execute Package Utility  Version 12.0.4100.1 for 64-bit  Copyright (C) Microsoft Corporation. All rights reserved.    Started:  10:52:25  Error: 2017-02-06 10:52:26.26     Code: 0x00000001     Source: Open Spreadsheet and Run Macro      Description: Exception has been thrown by the target of an invocation.  End Error  DTExec: The package execution returned DTSER_FAILURE (1).  Started:  10:52:25  Finished: 10:52:26  Elapsed:  0.485 seconds.  The package execution failed.  The step failed.
The issue is a type of permissions error. Excel needs to have its permissions changed via the DCOM applet in Control Panel. By default it is on “The launching user.” This needs to be changed to a user with more permissions, in this case we have used the service account used by SQL A…