Skip to main content

How to delete records in batches while reducing locking

I have used the technique below to delete rows in batches which includes pauses to let other process access the table. USE IT AT YOUR OWN RISK!! I take NO responsibility if you run it on your own system - in fact that goes for any of the code on my blog.


DECLARE @MaxID INT
DECLARE @MinID INT
DECLARE @Date DATETIME
DECLARE @MyTableVar TABLE (ID INT)

SET @Date = GETDATE()

IF EXISTS (SELECT * FROM tempdb..sysobjects WHERE id=OBJECT_ID('tempdb..#DelRows'))
    DROP TABLE #DelRows

CREATE TABLE #DelRows (ID INT)

/*
To try and reduce table scans on large tables we get the minimum and maximum ID (assuming ID is a Clustered Index) of the date range and use those as the minimum and maximum criteria for the filter
*/
-- Find latest ID in range
SET @MaxID = (
              SELECT TOP 1 t1.Table1_ID
              FROM dbo.Table1 t1 WITH (NOLOCK)        
              WHERE t1.TableDate < DATEADD(HOUR, - 24, @date)
              ORDER BY t1.TableDate DESC
              )
-- Find earliest ID in range
SET @MinID = (
              SELECT TOP 1 t1.Table1_ID
              FROM dbo.Table1 t1 WITH (NOLOCK)
              WHERE t1.TableDate >= @Date
              ORDER BY t1.TableDate ASC
              )

-- Insert the records to delete into a temporary table
INSERT #DelRows
SELECT TOP 500000 ID
FROM Table1 t1(NOLOCK)
WHERE t1.Table1_ID BETWEEN @MinID
              AND @MaxID

-- Delete the records in batches of 1000
WHILE 1 = 1
BEGIN
       DELETE TOP (1000) t1
       OUTPUT DELETED.ID
       INTO @MyTableVar
       FROM Table1 t1
       JOIN #DelRows d ON t1.ID = d.ID

       DELETE d2
       FROM #DelRows d2
       JOIN @MyTableVar m ON d2.ID = m.ID

       DELETE @MyTableVar
      
-- We put in a delay to give other queries a chance
       WAITFOR DELAY '000:00:05'

END

Comments

Popular posts from this blog

How to create a custom Windows Event Log view and email trigger

The filtering on Windows event logs can be slow, clunky and although you can do it on fields like event ID, it seems that many event IDs are shared amongst many different errors – the event ID may match but the body of the error (therefore the actual error) may be completely unrelated. Fortunately, it is possible to filter on the contents of the body of the error message but it requires creating a custom XML query. Also, it would be handy to send out a notification email when this event gets logged. Read on to find out how to work this magic…. This example is looking for a  Warning  event  1309  for  ASP.NET 4.0.30319.0  on a web server. If you were to just filter the log on the criteria above today it would return 435 results because it is a fairly general error ID. If I filter it using XML for SqlException (what I’m really interested in) only 5 results are returned. So the first step is go to the Application Log and choose  Create Custom View…  Select the  XML  tab, check  Edit

How to configure the SSAS service to use a Domain Account

NB Updating SPNs in AD is not for the faint hearted plus I got inconsistent results from different servers. Do so at your own risk! If you need the SSAS account on a SQL Server to use a domain account rather than the local “virtual” account “NT Service\MSSQLServerOLAPService”. You may think you just give the account login permissions to the server, perhaps give it sysadmin SQL permissions too. However, if you try and connect to SSAS  remotely  you may get this error: Authentication failed. (Microsoft.AnalysisService.AdomdClient) The target principal name is incorrect (Microsoft.AnalysisService.AdomdClient) From Microsoft: “A Service Principle Name (SPN) uniquely identifies a service instance in an Active Directory domain when Kerberos is used to mutually authenticate client and service identities. An SPN is associated with the logon account under which the service instance runs. For client applications connecting to Analysis Services via Kerberos authentication, th

How to import a large xml file into SQL Server

(Or how to import the StackOverflow database into SQL Server) Introduction NB  This process can be generalised to import any large (>2G) xml file into SQL Server. Some SQL Server training you can find online including that by Brent Ozar uses the StackOverflow database for practice. The tables from it are available online for download in xml format. In the past it was possible to use the scripts found here, https://www.toadworld.com/platforms/sql-server/w/wiki/9466.how-to-import-the-stackoverflow-xml-into-sql-server , to import them but as each xml file is now over 2GB you will get an error like this when you try to execute them: Brent Ozar, has a link to SODDI.exe, https://github.com/BrentOzarULTD/soddi , which can import the files (I haven’t tried it) but it means downloading and importing eight tables: Badges, Comments, PostHistory, PostLinks, Posts, Tags, Users, and Votes tables which amounts to >30GB of compressed xml increasing to ~200GB when decompre