REPOSITORY HEALTH REPORT
Example Organization
Baseline assessment, four weeks. 4,200,000 items in 4 repositories and 250 collections.
EXECUTIVE SUMMARY
One number, and what is behind it
The AITS Content Tracking System (CTS) assessed every item in the four repositories, 4,200,000 in all, against the sixteen KPIs the executive team chose. The result is a composite Repository Health score of 70.6%, against a target of 85.0%.
Three things are holding the score down
- 1Sensitive items are too widely visible. 89,900 of the 310,000 sensitive items can be opened by people outside the team that owns them.
- 2Many collections have no authoritative version. In 105 of 250 collections, nothing says which copy is the one to rely on. An AI assistant would answer from any of them.
- 3A third of items are missing required fields. 1,386,000 items cannot be reliably found, classified or governed because their fields are empty.
What to do first
Fix access to the sensitive items first. It is a contained piece of work and it removes the largest risk. The full list, with the gain expected from each action, is in section 03.
The official repositories were not touched during this assessment. CTS worked on a replica hosted in-country.
01 / THE SCORE
The eight dimensions behind the score
The composite score is a weighted average of eight dimensions. The executive team chose the KPIs in each and set the weights. Data Quality carries the most weight here because it is what this organization's decisions depend on most.
| Dimension | Score | Weight | Points | |
|---|---|---|---|---|
| Content | 76.5% | 10% | 7.65 | |
| Structure | 80.0% | 10% | 8.00 | |
| Metadata | 63.0% | 15% | 9.45 | |
| Search | 73.0% | 10% | 7.30 | |
| Data Quality | 70.0% | 20% | 14.00 | |
| Security | 74.0% | 15% | 11.10 | |
| Governance | 70.0% | 10% | 7.00 | |
| AI Readiness | 61.0% | 10% | 6.10 | |
| Composite score | 70.6% | 100% | 70.60 |
Points are the score multiplied by the weight. The points add up to the composite score.
The KPIs in each dimension
| Dimension | KPI | Measured |
|---|---|---|
| Content | Items that are not duplicates | 81.0% |
| Items that are current, not superseded or expired | 72.0% | |
| Structure | Items filed in a recognized location | 86.0% |
| Items with a recognized document type | 74.0% | |
| Metadata | Required fields completed | 67.0% |
| Field values valid and consistent | 59.0% | |
| Search | Items with searchable text | 79.0% |
| Items with a meaningful title | 67.0% | |
| Data Quality | Records with no conflicting values | 76.0% |
| Related records correctly linked | 64.0% | |
| Security | Sensitive items with restricted access | 71.0% |
| Access that matches the item's classification | 77.0% | |
| Governance | Items with a retention rule | 64.0% |
| Items within their retention period | 76.0% | |
| AI Readiness | Collections with one authoritative version | 58.0% |
| Items that pass every check an AI assistant relies on | 64.0% |
A dimension's score is the average of its KPIs. Every KPI can be opened in CTS to list the items behind it.
02 / WHERE THE RISK SITS
Health by repository
The same KPIs, scored for each repository. The shared network drives hold about a quarter of the items and most of the problems.
| Repository | Score | Items | |
|---|---|---|---|
| Enterprise content platform | 78.6% | 1,850,000 | |
| Shared network drives | 58.9% | 1,120,000 | |
| Cloud file storage | 70.2% | 940,000 | |
| Legacy records system | 66.4% | 290,000 | |
| All repositories | 70.6% | 4,200,000 |
The figure for all repositories is the average of the four scores, weighted by the number of items in each. It equals the composite score.
03 / FINDINGS AND ACTIONS
What to fix first
Six findings, in order of risk, each with the action CTS recommends and the gain expected from it. Staff decide on every one. CTS corrects repository records such as filing, links and classification. It never alters the content of a document.
| Finding | Affected | Recommended action | Expected gain |
|---|---|---|---|
| 1. Sensitive items open to too many people | 89,900 sensitive items Worst in: Shared network drives |
Restrict access to the owning team. CTS prepares the change for each item; a records officer approves it. Approval required. KPI from 71.0% to 96.0%. |
+1.9 points |
| 2. Collections with no authoritative version | 105 collections Worst in: Cloud file storage |
Mark one version as authoritative in each collection. CTS proposes the version; the collection owner confirms. Approval required. KPI from 58.0% to 84.0%. |
+1.3 points |
| 3. Required fields left empty | 1,386,000 items Worst in: Shared network drives |
Complete the fields from the document's own content. CTS fills them where it is confident and flags the rest for staff. Routine, with sampling. KPI from 67.0% to 86.0%. |
+1.4 points |
| 4. Items with no retention rule | 1,512,000 items Worst in: Legacy records system |
Apply the retention schedule by document type. Items past their period are listed for review, never removed by CTS. Approval required. KPI from 64.0% to 88.0%. |
+1.2 points |
| 5. Scanned items with no searchable text | 882,000 items Worst in: Legacy records system |
Run text recognition on the scanned images and check a sample of the results. Routine, with sampling. KPI from 79.0% to 95.0%. |
+0.8 points |
| 6. Duplicate items | 798,000 items Worst in: Shared network drives |
Link duplicates to one retained copy. Nothing is deleted without approval. Routine, with sampling. KPI from 81.0% to 93.0%. |
+0.6 points |
| All six actions: from 70.6% to 77.8% | +7.2 points | ||
Expected gain is the change in the KPI multiplied by that KPI's share of the composite score. "Routine, with sampling" means CTS may apply the fix once staff have approved that type of fix, and staff check a sample. "Approval required" means a person approves each change.
WHAT THE SIX ACTIONS DO NOT REACH
The six actions bring the score to 77.8%, still 7.2 points short of the target. The remainder sits in Data Quality and Metadata: conflicting values and broken links between related records. Those need decisions from the business units that own the records, and are the subject of the next phase.
04 / AI READINESS
Which collections are ready for AI
AI does not repair a repository. It inherits it. This is where an AI assistant could be used today, and where it should wait.
Ready: 96
38% of collections. Passes every check. An AI assistant can be pointed at these today.
Needs work: 105
42% of collections. Usable once an authoritative version is marked.
Not ready: 49
20% of collections. Fails on security or data quality. Keep AI away until fixed.
250 collections in all. The list of collections in each group is part of the report delivered to the organization.
05 / MONITORING
What continuous monitoring would show
A health check is a baseline. With CTS monitoring, every new item is scored as it arrives and the composite score is kept current. This is an example of the twelve weeks after the baseline, as the six actions are carried out.
EXAMPLE ALERT, WEEK 4
- What happened
- A bulk upload of 60,000 items arrived on the shared network drives with no required fields completed. The composite score fell from 72.6% to 72.1%.
- What CTS did
- It raised an alert the same day, showed which upload caused the fall, and recommended action: complete the fields from the documents' own content.
- What staff did
- They approved the fix and checked a sample. The score recovered the following week.
CTS learns from every finding staff confirm, correct or dismiss, so its alerts sharpen over time.
06 / HOW IT WAS MEASURED
The method, in brief
You define healthy
The executive and managers choose the KPIs and set the weights and the target. AITS recommends a starting set. All of it can be changed as priorities change.
Every item is assessed
Not a sample. Each item is loaded into the replica and scored as it arrives, the same way a digitization project loads records into a repository.
Nothing is touched
CTS runs on a replica of the repository environment, hosted in-country. The official repositories remain as they are, and any result can be compared against the original.
AI trained on your content
The AI learns what a sound record looks like in this organization. It tells you what the BI charts and reports mean, not only what they show.
People stay in charge
CTS recommends action. Staff confirm, correct or dismiss. Standing permission for routine fixes is given one type at a time and can be withdrawn.
Any repository
No particular platform is required. Enterprise content systems, cloud storage, shared drives and legacy systems are measured the same way.
Find out your repository health
This report is a sample. A Repository Health Check gives an executive the same thing for their own content: a scored baseline, the detail behind it, and a prioritized list of what to do next.
Contact AITS to book a Repository Health Check info@appliedits.comON THE AITS WEBSITE
Repository Health Self-Assessment: Sixteen questions and an estimate of your own score.
Repository Health Check: How the health check and monitoring work.
Solution Brief: Is Your Repository Ready for AI?: The Repository Health Check in two pages.
ABOUT AITS
Applied IT Solutions (AITS) is a Toronto-based consultancy founded in 2008 and led by its principal consultant. It advises government and Fortune 500 organizations on artificial intelligence, enterprise data strategy, document and records management, and business process improvement, with one aim: deriving tangible business value from the application of information technology. Most engagements are delivered end to end by the principal, from requirements gathering through solution design and delivery to train-the-trainer and hand-off to the client's own team. For larger projects, AITS brings in experienced consultants it works with regularly. AITS has run projects worldwide and works in any time zone. Out of this work, AITS developed the Content Tracking System (CTS), an AI-powered monitor that scores the health of a document repository against the organization's own KPIs.
Example Organization is fictional. No figure in this report describes a real organization.