Skip to main content

How to delete records in batches while reducing locking

I have used the technique below to delete rows in batches which includes pauses to let other process access the table. USE IT AT YOUR OWN RISK!! I take NO responsibility if you run it on your own system - in fact that goes for any of the code on my blog.


DECLARE @MaxID INT
DECLARE @MinID INT
DECLARE @Date DATETIME
DECLARE @MyTableVar TABLE (ID INT)

SET @Date = GETDATE()

IF EXISTS (SELECT * FROM tempdb..sysobjects WHERE id=OBJECT_ID('tempdb..#DelRows'))
    DROP TABLE #DelRows

CREATE TABLE #DelRows (ID INT)

/*
To try and reduce table scans on large tables we get the minimum and maximum ID (assuming ID is a Clustered Index) of the date range and use those as the minimum and maximum criteria for the filter
*/
-- Find latest ID in range
SET @MaxID = (
              SELECT TOP 1 t1.Table1_ID
              FROM dbo.Table1 t1 WITH (NOLOCK)        
              WHERE t1.TableDate < DATEADD(HOUR, - 24, @date)
              ORDER BY t1.TableDate DESC
              )
-- Find earliest ID in range
SET @MinID = (
              SELECT TOP 1 t1.Table1_ID
              FROM dbo.Table1 t1 WITH (NOLOCK)
              WHERE t1.TableDate >= @Date
              ORDER BY t1.TableDate ASC
              )

-- Insert the records to delete into a temporary table
INSERT #DelRows
SELECT TOP 500000 ID
FROM Table1 t1(NOLOCK)
WHERE t1.Table1_ID BETWEEN @MinID
              AND @MaxID

-- Delete the records in batches of 1000
WHILE 1 = 1
BEGIN
       DELETE TOP (1000) t1
       OUTPUT DELETED.ID
       INTO @MyTableVar
       FROM Table1 t1
       JOIN #DelRows d ON t1.ID = d.ID

       DELETE d2
       FROM #DelRows d2
       JOIN @MyTableVar m ON d2.ID = m.ID

       DELETE @MyTableVar
      
-- We put in a delay to give other queries a chance
       WAITFOR DELAY '000:00:05'

END

Comments

Popular posts from this blog

How to configure the SSAS service to use a Domain Account

NB Updating SPNs in AD is not for the faint hearted plus I got inconsistent results from different servers. Do so at your own risk! If you need the SSAS account on a SQL Server to use a domain account rather than the local “virtual” account “NT Service\MSSQLServerOLAPService”. You may think you just give the account login permissions to the server, perhaps give it sysadmin SQL permissions too. However, if you try and connect to SSAS  remotely  you may get this error: Authentication failed. (Microsoft.AnalysisService.AdomdClient) The target principal name is incorrect (Microsoft.AnalysisService.AdomdClient) From Microsoft: “A Service Principle Name (SPN) uniquely identifies a service instance in an Active Directory domain when Kerberos is used to mutually authenticate client and service identities. An SPN is associated with the logon account under which the service instance runs. For client applications connecting to Analysis Services via Kerberos authentication, th

How to move the Microsoft Assessment and Planning Toolkit (MAP) database to a different drive

The Microsoft Assessment and Planning Toolkit (MAP) is a very useful tool for scanning your network to find instances of SQL Server plus all manner of detailed information about the installed product, OS and hardware it sits on. <Click image to enbiggen> There is an issue with it the database it uses to store the data it collects, however. Assuming you don't have an instance called MAPS on your server, the product will install using LocalDB (a cut down version of SQL Server Express) and puts the databases on your C: drive. If you then scan a large network you could easily expand the database to 10GB which may cause issues on a server when that drive is often one of the smallest. However, there is a simple solution: connect to LocalDB using Management Studio, detach the databases, move to a different drive, set permissions on the new location if required and reattach the database. How do you connect to LocalDB? Here you go: Connect to (localdb)\MAPTOOLKIT The

SAN performance testing using SQLIO

Introduction This document describes how to use Microsoft’s SQLIO to test disk/SAN performance. It is biased towards SQL Server – which uses primarily 64KB and 8KB data pages so I am running the tests using those cluster sizes, however, other sizes can be specified.  Download SQLIO from https://www.microsoft.com/en-gb/download/details.aspx?id=20163   SQLIO is a command line tool with no GUI so you need to open a command prompt at  C:\Program Files (x86)\SQLIO  after you have installed it. Configuration First of all edit param.txt so that you create the test file we will be using. The file needs to be bigger than the combined RAID and on-board disk caches. In this case we are using a 50GB file. The “ 2”  refers to the number of threads to use when testing, you don’t need to change this now. The “ 0x0”  value indicates that all CPUs should be used, which you probably don’t want to change either, “ #”  is a comment. The only part you may want to change is 51200 (50GB) a