How do you calculate AUC by hand?

Sort the samples by predicted score, then sweep a threshold from high to low. At each distinct score, count true positives and false positives to get the TPR (TP/P) and FPR (FP/N), giving a point on the ROC curve. The AUC is the area under that staircase, summed as trapezoids: AUC = Σ (FPRᵢ − FPRᵢ₋₁)·(TPRᵢ + TPRᵢ₋₁)/2. Equivalently, rank all scores and apply the Mann–Whitney formula (R⁺ − P(P+1)/2)/(P·N) — both give the same number.

What is a good AUC score?

0.5 is no better than a coin flip; 1.0 is perfect. The widely-cited Hosmer–Lemeshow bands read 0.7–0.8 as acceptable discrimination, 0.8–0.9 as excellent, and above 0.9 as outstanding. Below 0.7 is weak. An AUC under 0.5 usually means the score is inverted — flip the score direction and it becomes 1 − AUC.

What is the difference between ROC and AUC?

The ROC (Receiver Operating Characteristic) curve is the full plot of true positive rate against false positive rate as you vary the decision threshold. The AUC (Area Under the Curve) is a single number — the area beneath that curve — summarising the classifier's ranking quality across every threshold at once. The ROC shows the trade-offs; the AUC condenses them to one score you can compare between models.

What does an AUC of 0.5 mean?

It means the classifier ranks a random positive above a random negative exactly half the time — no discrimination at all. On the plot the ROC curve sits on the diagonal no-skill line. This happens when scores carry no signal, or when every score is tied. An AUC of 0.5 is the baseline every useful model must beat.

Can you compute AUC from a confusion matrix?

No. A confusion matrix is fixed to a single threshold, so it gives you one (FPR, TPR) point — not the whole curve. AUC is threshold-independent: it needs the predicted scores or probabilities for each sample so the threshold can be swept. If you only have one confusion matrix, you can compute accuracy, precision, recall and F1, but not AUC. That is why this tool asks for scores, not counts.

How does this tool handle tied scores?

Tied scores are resolved with the average-rank convention, the same one scikit-learn uses. A block of equal scores collapses the ROC step into a single diagonal segment, so the trapezoidal area and the Mann–Whitney rank form still agree exactly. The all-tied preset (AUC 0.5) shows this: every score is identical, the curve becomes the diagonal, and ties counted as half give precisely 0.5.

What is Youden's J and the optimal threshold?

Youden's J = TPR − FPR measures how far a threshold sits above the no-skill diagonal. The threshold with the highest J is a common choice of operating point when false positives and false negatives matter equally — it is the point on the ROC curve furthest from the diagonal. This calculator marks it on the chart and in the threshold table, but you should weight it against your own costs of each error type.

Does this calculator send my predictions anywhere?

No. Parsing your labels and scores, sweeping the thresholds, and computing the AUC all run in your browser with plain JavaScript. Nothing is uploaded, logged, or stored, and the page keeps working offline once loaded. You can paste validation-set predictions with no privacy concern.

AI · Machine learning

ROC Curve & AUC Calculator

Paste your binary labels and predicted scores to get the ROC curve, the AUC, the Gini coefficient, and the Youden-optimal threshold — every value shown with its formula and cross-checked two ways. It matches scikit-learn's roc_auc_score, runs entirely in your browser, and needs no signup.

By Induwara Ashinsana— Executive Director, Ryzera TechnologiesUpdated Jun 10, 2026

ROC curve & AUC calculator

Your data — one sample per line: label score

Label is the true class (0 or 1); score is the model's predicted probability or any real-valued output. Separate the two with a space, tab, or comma. Needs at least one positive and one negative row.

Positive class

Which label counts as the positive (event) class.

Score direction

Whether a larger or a smaller score indicates the positive class.

Presets

AUC

0.8056

Area under the ROC curve (0–1)

Gini coefficient

0.6111

2·AUC − 1

Optimal threshold

0.82

max Youden's J = 0.5000

Samples (P / N)

6 / 6

12 total

AUC 0.8056 — Excellent discrimination. An AUC of 0.8056 means a randomly chosen positive sample outranks a randomly chosen negative one about 80.56% of the time (ties counted as half).

Decimals

ROC curve

ModelNo-skill (AUC 0.5) Optimal threshold

Formulas

TPR = TP / P, FPR = FP / N
AUC = Σ (FPRᵢ − FPRᵢ₋₁)·(TPRᵢ + TPRᵢ₋₁)/2
AUC = (R⁺ − P(P+1)/2) / (P·N) (Mann–Whitney)
Gini = 2·AUC − 1
Youden's J = TPR − FPR

Cross-check. The trapezoidal rule gives AUC = 0.8056; the independent Mann–Whitney rank form (R⁺ = 50 over P = 6, N = 6) gives 0.8056. They reconcile, as they must — the result is verified.

Recommended operating point. Threshold 0.82 maximises Youden's J at 0.5000 — TPR 0.5000, FPR 0.0000 (TP 3, FP 0, TN 6, FN 3).

Threshold table

Threshold	TP	FP	TN	FN	TPR	FPR	Youden's J
0.95	1	0	6	5	0.1667	0.0000	0.1667
0.88	2	0	6	4	0.3333	0.0000	0.3333
0.82optimal	3	0	6	3	0.5000	0.0000	0.5000
0.78	3	1	5	3	0.5000	0.1667	0.3333
0.65	4	1	5	2	0.6667	0.1667	0.5000
0.6	4	2	4	2	0.6667	0.3333	0.3333
0.55	5	2	4	1	0.8333	0.3333	0.5000
0.48	5	3	3	1	0.8333	0.5000	0.3333
0.4	5	4	2	1	0.8333	0.6667	0.1667
0.35	6	4	2	0	1.0000	0.6667	0.3333
0.28	6	5	1	0	1.0000	0.8333	0.1667
0.22	6	6	0	0	1.0000	1.0000	0.0000

Method: TPR = TP/P, FPR = FP/N swept across every distinct score; AUC = Σ (ΔFPR)(TPR + TPRₚᵣₑᵥ)/2 (trapezoidal, scikit-learn roc_auc_score), cross-checked against the Mann–Whitney rank form (Hanley & McNeil 1982). Sources cited below the calculator. No data leaves this page.

How it works

The ROC curveplots a classifier's true positive rate against its false positive rate as the decision threshold moves, and the AUCis the single number underneath it. The method here follows scikit-learn's roc_curve / roc_auc_score and the definitions in Fawcett's 2006 ROC primer.

With P positive and N negative samples, each threshold yields a confusion matrix and a point:

TPR = TP / P FPR = FP / N

Sortthe samples by predicted score, descending. If you pick “lower score = positive”, the scores are negated first so the sweep can always treat a higher value as more positive.
Sweep the threshold through every distinct score. Classify a sample positive when its score meets the threshold, and record the running TPR and FPR — one ROC point per distinct score, with (0,0) prepended and (1,1) appended.
Integrate the area under those points with the trapezoidal rule:
AUC = Σ (FPRᵢ − FPRᵢ₋₁)·(TPRᵢ + TPRᵢ₋₁)/2
Cross-checkwith the Mann–Whitney / Wilcoxon rank form (Hanley & McNeil 1982). Rank every score ascending — averaging ranks for ties — and with R⁺ the sum of the positive ranks:
AUC = (R⁺ − P(P+1)/2) / (P·N)
This equals the probability that a random positive outranks a random negative (ties counted as half) and must match the trapezoidal value — the tool asserts the two agree to floating-point precision.

From the AUC the tool reports the Gini coefficient = 2·AUC − 1 and a plain-English discrimination band. For each threshold it also computes Youden's J = TPR − FPR; the threshold with the highest J — the point on the curve furthest from the no-skill diagonal — is flagged as a common operating-point choice. Inputs with only one class are rejected, because the AUC is then undefined.

Worked examples

Four samples, no ties — AUC 0.75 (the Textbook preset)

Data (label, score): (1, 0.9) (0, 0.6) (1, 0.4) (0, 0.2). P = 2, N = 2
t = 0.9: TP 1, FP 0 → (FPR 0.0, TPR 0.5)
t = 0.6: TP 1, FP 1 → (0.5, 0.5)
t = 0.4: TP 2, FP 1 → (0.5, 1.0)
t = 0.2: TP 2, FP 2 → (1.0, 1.0)
Trapezoid: 0.25 + 0.50 = 0.75; Gini = 2(0.75) − 1 = 0.5
Mann–Whitney: ranks 0.2→1, 0.4→2, 0.6→3, 0.9→4; R⁺ = 4 + 2 = 6
AUC = (6 − 2·3/2) / (2·2) = 3/4 = 0.75 ✓ (both forms agree)

Tied scores at a threshold — AUC 0.875 (average-rank handling)

Data: (1, 0.8) (1, 0.5) (0, 0.5) (0, 0.2). P = 2, N = 2
t = 0.8: TP 1, FP 0 → (0.0, 0.5)
t = 0.5: a tied 1 and 0 enter together → TP 2, FP 1 → (0.5, 1.0)
t = 0.2: TP 2, FP 2 → (1.0, 1.0)
Trapezoid: 0.375 + 0.50 = 0.875
Mann–Whitney: the two 0.5 share ranks 2 and 3 → average 2.5
R⁺ = 4 (for 0.8) + 2.5 (for the positive 0.5) = 6.5
AUC = (6.5 − 3) / 4 = 0.875 ✓ — ties keep both forms identical

No skill — AUC 0.5 (all scores tied)

Data: (1, 0.5) (0, 0.5) (1, 0.5) (0, 0.5). P = 2, N = 2
One distinct threshold (0.5): every sample is predicted positive
TP 2, FP 2 → the curve runs straight from (0,0) to (1,1)
Trapezoid area of the diagonal = 0.5; Gini = 0
Mann–Whitney: all four ranks average to 2.5; R⁺ = 5
AUC = (5 − 3) / 4 = 0.5 ✓ — exactly the random-guess baseline

Frequently asked questions

Sources & references

The formulas on this page were last cross-checked against these sources on 2026-06-10. ROC and AUC are stable mathematical definitions, so this tool needs no rate or schedule updates — only the worked examples are periodically re-reconciled against scikit-learn.

Related tools

LiveAI

ROUGE Score Calculator

Calculate ROUGE-1, ROUGE-2, ROUGE-L and ROUGE-Lsum precision, recall and F1 between a generated summary and one or more references, entirely in your browser. Shows matched n-grams and the longest common subsequence. Matches Google rouge-score, no signup.

Open tool

LiveAI

TF-IDF Calculator

Compute the full TF-IDF weighting for a small corpus — the term-frequency table, the IDF per term, and the final TF-IDF matrix, with the complete per-term working. Supports the textbook and scikit-learn smoothed variants, runs entirely in your browser.

Open tool

LiveAI

Gini Impurity Calculator

Compute the Gini impurity of a decision-tree node from class counts or proportions, with the full 1 − Σ pₖ² working, a Shannon-entropy comparison, and the Gini gain of a candidate split. Matches scikit-learn, runs in the browser.

Open tool

Rate this tool

Be the first to rate

Comments & feedback

Spotted a bug or want an improvement? Tell us — our team reviews every comment, and good ideas get built. Comments are public and anonymous.

Found a bug, edge case, or want to suggest an improvement?

Email me at [email protected] — most fixes ship within 24 hours.