My Azure DevOps Server 2020 Update 1.1 database has grown to over 1TB, and I see no reason for it becoming so big. I have quite aggressive retention policy (all builds are set to be deleted as soon as possible, and test results are to be deleted with builds), but it does not help.

I suspect some cleanup job has stopped working, but I cannot figure out what/where.

Here are some results from the database, scoped to the most problematic build definition.

SELECT COUNT(*) FROM Build.tbl_Build where DefinitionId = 244
-- Result: 79

SELECT BuildDeleted, COUNT(*) FROM tbl_BuildConfiguration 
WHERE BuildDefinitionId = 244
GROUP BY BuildDeleted
-- Result BuildDeleted=0: 1085
-- Result BuildDeleted=1: 18722

Table tbl_Content is at 919GB, which breaks down to this:

SELECT Owner = 
CASE
WHEN OwnerId = 0 THEN 'Generic' 
WHEN OwnerId = 1 THEN 'VersionControl'
WHEN OwnerId = 2 THEN 'WorkItemTracking'
WHEN OwnerId = 3 THEN 'TeamBuild'
WHEN OwnerId = 4 THEN 'TeamTest'
WHEN OwnerId = 5 THEN 'Servicing'
WHEN OwnerId = 6 THEN 'UnitTest'
WHEN OwnerId = 7 THEN 'WebAccess'
WHEN OwnerId = 8 THEN 'ProcessTemplate'
WHEN OwnerId = 9 THEN 'StrongBox'
WHEN OwnerId = 10 THEN 'FileContainer'
WHEN OwnerId = 11 THEN 'CodeSense'
WHEN OwnerId = 12 THEN 'Profile'
WHEN OwnerId = 13 THEN 'Aad'
WHEN OwnerId = 14 THEN 'Gallery'
WHEN OwnerId = 15 THEN 'BlobStore'
WHEN OwnerId = 255 THEN 'PendingDeletion'
END,
SUM(CompressedLength) / 1024.0 / 1024.0 / 1024 AS BlobSizeInGB
FROM tbl_FileReference AS r
JOIN tbl_FileMetadata AS m
ON r.ResourceId = m.ResourceId
AND r.PartitionId = m.PartitionId
WHERE r.PartitionId = 1
GROUP BY OwnerId
ORDER BY 2 DESC

-- Result (rounded):
-- TeamTest         573
-- FileContainer    295
-- VersionControl    11
-- WorkItemTracking   8
-- BlobStore          2
-- CodeSense          0
-- Generic            0
-- ProcessTemplate    0