Research Diagnostic
Fixed priceCAD 495
- 60-minute video call
- Two-page risks and next-steps memo
- Optional 30-minute debrief
Paid in full at booking.
Memo within 5 business days of the call.
Ask about thisML model validation
An independent check of a research prediction model, looking beyond one accuracy score to how it was built, evaluated and interpreted. Starting at CAD 495.
Typical questions
What teams bring to us.
- Does our evaluation design match the way the model will be used?
- Could data leakage explain an apparently strong result?
- Are discrimination and calibration both good enough?
- Does the model perform equally well across groups?
- What would external validation need to show?
- How should we report the model so reviewers trust it?
What you receive
- Validation review: a structured report on design, data handling, leakage risks and reporting.
- Performance analysis: discrimination and calibration with confidence intervals, on the data you provide.
- Subgroup analysis: performance across the groups that matter for your use, where data allow.
- Risks and next steps: a ranked list of problems and what would fix each.
- Reproducible code: for every metric and figure we report.
Exact deliverables are agreed in writing.
How it works
From first call to handover.
- 01
First call
the model, its intended use and the evidence you already have.
Week 1
- 02
Written scope
what is reviewed (code, predictions, data), outputs, fee and timeline.
Within 2 business days of the call
- 03
Review and analysis
design review, re-analysis of predictions and subgroup checks.
Typically 2 to 4 weeks
- 04
Report and debrief
written review, figures and a call to walk through the findings.
Typically 1 week
Durations are typical; the written scope sets the actual dates.
What we need from you
Enough to start well.
- A description of the model, its intended use and the population it targets.
- Predictions and true labels on a test set, or the code and data to produce them.
- A description of how the data were split, tuned and tested.
- Permission to use the data for the review, and one named contact person.
Please do not send patient information or identifiable data. How we handle data
Methods and quality checks
How results are checked.
- AUC and ROC analysis
- Calibration curves
- Decision-curve analysis
- Nested cross-validation
- External validation
- Leakage audit
- Subgroup performance
- Bootstrap confidence intervals
We recompute every reported metric from the predictions, test whether results survive a stricter evaluation design, and state clearly whether we had any role in developing the model. A review of work we helped build is never described as fully independent.
Fields we apply this in
Areas that use this service.
Boundaries
The review does not certify a model, establish clinical utility, or make it fit for deployment in patient care. It does not replace a regulatory submission or a regulator's assessment.
Price and what changes it
Start at the right scale.
Data-to-Results Sprint
FromCAD 3,000
- Agreed analysis and quality-control log
- Figures, tables and methods text
- R or Python code and recorded environment
50% to start; 50% on delivery.
Final price and delivery date agreed after scoping.
Ask about thisWhat makes a project larger
- Whether we review predictions only or rerun your code.
- The number of models, outcomes and subgroups.
- External validation on a second dataset.
- The size and complexity of the data.
- Deadlines shorter than the usual timeline.
Questions
About ML model validation.
What does an independent model check include?
A review of the design, data splits, tuning and reporting; recomputed discrimination and calibration; subgroup results; and a ranked list of risks with fixes.
Do you need our code, or only predictions and labels?
Predictions and true labels are enough for a performance review. Code lets us check for leakage and rerun the evaluation, which gives a stronger review.
Can you check performance across groups (fairness)?
Yes, for any groups recorded in the data. We report each group with confidence intervals and say where the data are too thin to judge.
What is the difference between internal and external validation?
Internal validation tests the model on held-out parts of the same data. External validation tests it on data from a different site, source or period, which is stronger evidence.
Will you sign an NDA?
Yes, before any code, data or confidential documents are shared.
Can the report support a regulatory submission?
No. The review is for research use. It can help you decide what evidence a submission would need, but it does not replace the regulatory process.
How long does a model review take?
Most reviews take two to four weeks after we receive the predictions, labels and documentation. A review that reruns your code or adds external validation takes longer; the written scope gives the date.
What happens if the review finds a serious problem?
We tell you as soon as we find it, explain why it matters and suggest how to fix it. Finding a problem before a reviewer or a regulator does is the point of the review.
Can you also help develop a model?
Development support can be scoped separately. Its role and ownership are stated openly, and a later review of that model is not described as independent.
Let us define the evidence your study needs.
Start with a short, non-confidential description. We reply with a suggested next step.