Skip to main content

How to delete records in batches while reducing locking

I have used the technique below to delete rows in batches which includes pauses to let other process access the table. USE IT AT YOUR OWN RISK!! I take NO responsibility if you run it on your own system - in fact that goes for any of the code on my blog.


DECLARE @MaxID INT
DECLARE @MinID INT
DECLARE @Date DATETIME
DECLARE @MyTableVar TABLE (ID INT)

SET @Date = GETDATE()

IF EXISTS (SELECT * FROM tempdb..sysobjects WHERE id=OBJECT_ID('tempdb..#DelRows'))
    DROP TABLE #DelRows

CREATE TABLE #DelRows (ID INT)

/*
To try and reduce table scans on large tables we get the minimum and maximum ID (assuming ID is a Clustered Index) of the date range and use those as the minimum and maximum criteria for the filter
*/
-- Find latest ID in range
SET @MaxID = (
              SELECT TOP 1 t1.Table1_ID
              FROM dbo.Table1 t1 WITH (NOLOCK)        
              WHERE t1.TableDate < DATEADD(HOUR, - 24, @date)
              ORDER BY t1.TableDate DESC
              )
-- Find earliest ID in range
SET @MinID = (
              SELECT TOP 1 t1.Table1_ID
              FROM dbo.Table1 t1 WITH (NOLOCK)
              WHERE t1.TableDate >= @Date
              ORDER BY t1.TableDate ASC
              )

-- Insert the records to delete into a temporary table
INSERT #DelRows
SELECT TOP 500000 ID
FROM Table1 t1(NOLOCK)
WHERE t1.Table1_ID BETWEEN @MinID
              AND @MaxID

-- Delete the records in batches of 1000
WHILE 1 = 1
BEGIN
       DELETE TOP (1000) t1
       OUTPUT DELETED.ID
       INTO @MyTableVar
       FROM Table1 t1
       JOIN #DelRows d ON t1.ID = d.ID

       DELETE d2
       FROM #DelRows d2
       JOIN @MyTableVar m ON d2.ID = m.ID

       DELETE @MyTableVar
      
-- We put in a delay to give other queries a chance
       WAITFOR DELAY '000:00:05'

END

Comments

Popular posts from this blog

How to move the Microsoft Assessment and Planning Toolkit (MAP) database to a different drive

The Microsoft Assessment and Planning Toolkit (MAP) is a very useful tool for scanning your network to find instances of SQL Server plus all manner of detailed information about the installed product, OS and hardware it sits on.


<Click image to enbiggen>
There is an issue with it the database it uses to store the data it collects, however. Assuming you don't have an instance called MAPS on your server, the product will install using LocalDB (a cut down version of SQL Server Express) and puts the databases on your C: drive. If you then scan a large network you could easily expand the database to 10GB which may cause issues on a server when that drive is often one of the smallest. However, there is a simple solution: connect to LocalDB using Management Studio, detach the databases, move to a different drive, set permissions on the new location if required and reattach the database. How do you connect to LocalDB? Here you go:

Connect to (localdb)\MAPTOOLKIT


The databases I move…

The Purge SQL Agent Job for MDW takes a long time to complete

I use the dbWarden alerts to inform me if a SQL job is taking longer to complete than normal and I got one this morning:
I noticed by looking at the history this purge job was gradually taking longer and longer to complete each day since I installed it again (see mylast post on this):
RunDate RunTime Duration ExecutionStatus JobName 04/10/2013 02:00:01 00:00:45 Succeded mdw_purge_data_[MDW] 05/10/2013 02:00:00 00:13:27 Succeded mdw_purge_data_[MDW] 06/10/2013 02:00:00 00:17:03 Succeded mdw_purge_data_[MDW] 07/10/2013 02:00:01

Generate scripts to attach multiple databases

There is a handy little "by product", if you like, when running queries which means you can quickly generate scripts to do different things. Below is an example of generating multiple "attach" commands that you can copy from the results pane into the main SSMS window for execution. I have found this very handy in the past:


SELECT 'CREATE DATABASE ['+name+'] ON ( FILENAME = N''F:\MSSQL\Data\'+name+'.mdf'' ), ( FILENAME = N''E:\MSSQL\Log\'+name+'_log.ldf'' )  FOR ATTACH GO ' FROMmaster.dbo.sysdatabases WHEREnamenotin('master','msdb','model','tempdb')
ORDERBYname