Shannon Entropy Calculator
Enter the probability of each category (summing to ≈ 1) or paste raw frequencies (they will be normalised automatically). The tool applies Shannon's 1948 formula H = −Σ pᵢ log₂ pᵢ to return the information entropy in bits, the maximum possible entropy log₂ k, the normalised entropy (Pielou's evenness), the redundancy and the per-category contribution −pᵢ log₂ pᵢ. Information theory, decision-tree splits (ID3, C4.5, CART), ecological diversity indices and password-strength analysis are all direct applications of the same formula.
Common examples (click to load)
Entropy metrics
Shannon entropy H
—
bits
Maximum entropy log₂ k
—
bits
Normalised entropy H / Hₘₐₓ
—
Ranges 0–1; 1 is perfectly uniform (maximum uncertainty), 0 is a certain event. Also known as Pielou's evenness.
Redundancy 1 − H/Hₘₐₓ
—
Shannon's 1948 "redundancy" — how far the distribution is from uniform; larger values leave more room for compression.
Total weight
—
Categories (non-zero / total)
—
Per-category contribution
| Category | Weight | Probability pᵢ | −pᵢ log₂ pᵢ |
|---|---|---|---|
| Sum (should equal H) | — | — | — |
Units are bits (log base 2). For nats (natural log), divide by ln 2 ≈ 0.6931; for dits / Hartleys (log₁₀), divide by log₂ 10 ≈ 3.3219. References: Shannon (1948); Cover & Thomas, "Elements of Information Theory", Ch. 2.
Formula
H(X) = − Σᵢ pᵢ · log₂ pᵢ (units: bits) Hₘₐₓ = log₂ k (k = non-zero category count) Normalised H = H / Hₘₐₓ ∈ [0, 1] Redundancy = 1 − H / Hₘₐₓ Convention: pᵢ · log pᵢ ≡ 0 whenever pᵢ = 0
- · Units depend on the logarithm base: log₂ → bits, log_e → nats, log₁₀ → dits / Hartleys. 1 bit ≈ 0.693 nat ≈ 0.301 dit.
- · Upper bound: H ≤ log₂ k, with equality iff the distribution is uniform. A fair 6-sided die has H = log₂ 6 ≈ 2.585 bits — the maximum any 6-category distribution can reach.
- · Lower bound: H ≥ 0, with equality iff one pᵢ = 1 (a fully deterministic event).
- · Cryptography: a perfectly random password drawn from an alphabet of k symbols carries log₂ k bits per character — lowercase 26 ≈ 4.70, lowercase+digits 36 ≈ 5.17, mixed case + digits 62 ≈ 5.95, plus symbols (~94) ≈ 6.55. The tool reports the entropy of the symbol distribution; multiply by the password length for a string-entropy estimate.
- · Machine learning: a decision-tree's information gain is IG(parent, split) = H(parent) − Σ (|child|/|parent|) · H(child), with H the very formula above. CART uses Gini impurity instead — different formula but a similar-magnitude substitute.
- · Ecology: the Shannon–Weaver diversity index H′ is the same H, reported together with Pielou's evenness J = H / Hₘₐₓ; standard practice in biodiversity studies since the 1960s.
- · References: Shannon, "A Mathematical Theory of Communication", Bell System Technical Journal 27 (1948) 379; Cover & Thomas, "Elements of Information Theory", 2nd ed., Ch. 2; Pielou (1966); Beker & Piper, "Cipher Systems" (1982) for English letter frequencies.
Frequently asked
Why is English-letter entropy only ≈ 4.18 bits, not log₂ 26 ≈ 4.70?
Because English letters are far from uniformly distributed: E ≈ 12.7%, T ≈ 9.1%, A ≈ 8.2%, but Q ≈ 0.10% and Z ≈ 0.07%. Plugging the 26 frequencies into H = −Σ pᵢ log₂ pᵢ gives ≈ 4.18 bits (Shannon, 1951). If all 26 letters were equally likely (1/26 each) the entropy would be log₂ 26 ≈ 4.70 bits — the tool's "maximum entropy". The gap of ~0.52 bits (≈ 11%) is precisely the "letter-level redundancy" Shannon described. Add bigram statistics (th, er, in …) and the conditional entropy drops to ~3.3 bits per letter; add full word-level statistics and it falls to ~1.0–1.5 bits — the theoretical floor that modern language models are chasing.
My password-strength tool says my password has 80 bits — is that Shannon entropy?
Yes, but two assumptions hide behind it: (1) the password is i.i.d. across characters; (2) each character is drawn from a uniform distribution, so the strength becomes H = log₂ k × length. For genuinely random passwords (Bitwarden, 1Password generators) the estimate is sound. For human-chosen passwords — where people gravitate to "password123", "qwerty" and the like — the real entropy is far below the nominal log₂ k × length. NIST SP 800-63B (2017) officially dropped the "force complexity + count characters" guidance in favour of "blocklist common passwords + encourage long passphrases", precisely because the Shannon-entropy assumptions (i.i.d. + uniform) do not hold for human-chosen passwords. zxcvbn (Dropbox's open-source library) gives a much more realistic strength estimate on top of this tool.
How does Shannon's H differ from Simpson's 1/D in biodiversity studies?
Both measure the "evenness" of species distribution within a community, but they weight rare species very differently. Shannon's H = −Σ pᵢ log pᵢ treats every species symmetrically, so adding a rare species visibly nudges H upward. Simpson's D = Σ pᵢ² is dominated by common species, so a rare species with pᵢ ≈ 0.01 barely moves the index. In practice: Shannon is appropriate for thoroughly sampled communities where you want to capture full community structure; Simpson is the safer choice when sampling is incomplete and rare species are likely missing (because the missing rare ones hardly affect it). Most biodiversity papers report both, usually as a triple: H, Pielou's J = H / log S, and Simpson's 1/D.
Related tools
Percentage Calculator
Percent of, percent change, and percent add/subtract in one.
GCD & LCM Calculator
Enter 2–6 positive integers to get the greatest common divisor (HCF / GCD) and least common multiple (LCM), with the Euclidean step chain shown.
Average Calculator (Mean / Median / Mode)
Enter a list of numbers to get the mean, median, mode, range plus standard deviation, variance and total.
Quadratic Equation Solver
Enter the coefficients of ax² + bx + c = 0 to find the real or complex roots, discriminant and vertex.
Password Strength (Entropy) Calculator
Estimate a password's bit entropy, brute-force time and strength tier. All computation happens in your browser.
Scientific Notation Converter
Convert between standard decimal numbers and scientific notation, with significant figures and order of magnitude.
Permutations & Combinations (nPr / nCr) Calculator
Compute permutations P(n,r), combinations C(n,r) and factorial n! — useful for probability problems, lottery odds and combinatorics homework.
Standard Deviation Calculator
Paste a list of numbers to compute mean, median, sample and population variance and standard deviation — with the working shown.
Triangle Calculator (SSS / SAS / ASA)
Solve a triangle from 3 sides, 2 sides + 1 angle, or 2 angles + 1 side — area, perimeter and remaining parts via the law of sines / cosines.
Pythagorean Theorem Calculator
Given any two sides of a right triangle (two legs, or one leg plus the hypotenuse), instantly find the third side, area, perimeter and the two non-right angles.
Circle Calculator (radius / diameter / circumference / area)
Enter any one of radius, diameter, circumference or area to get the other three — useful for design, engineering and DIY.
Roman Numeral Converter
Two-way conversion between Arabic numbers (1–3999) and Roman numerals (I, V, X, L, C, D, M) — handy for typesetting, chapter titles and homework.
Slope & Line Equation Calculator (y = mx + b from Two Points)
Enter two points (x₁, y₁) and (x₂, y₂) to instantly get the slope, y-intercept, line equation y = mx + b, distance, and midpoint — a classroom staple for algebra and coordinate geometry.
Birthday Paradox Calculator
Enter group size n to see the probability that at least two people share a birthday — the classic birthday problem.
Logarithm Calculator (log / ln / log₂ / any base)
Compute logₐ(x) for any base — natural log (ln), common log (log₁₀), binary log (log₂) and a custom base, with the change-of-base steps shown.
Z-Score (Standard Score) Calculator
Enter a value, the mean and the standard deviation to compute the z-score and the corresponding normal-distribution percentile and probabilities.
Screen Pixel Density (PPI) Calculator
Enter the screen resolution and diagonal size to get pixel density (PPI), real width/height, dot pitch and total pixel count.
Hong Kong Mark Six Odds Calculator
Enter the number of tickets / selections and see the actual probability of hitting first, second … prizes in a Mark Six (6-of-49) draw.
Decimal to Fraction Converter
Convert any decimal (including repeating decimals) to a simplified fraction and a mixed number.
Sphere Volume & Surface Area Calculator
Give a sphere any one of radius, diameter, surface area or volume and instantly get the other three — plus the great-circle circumference and area.
Cylinder Volume & Surface Area Calculator
Enter the radius and height of a cylinder to get volume (π r²h), lateral surface, base area and total surface area.
Permutations (nPr) Calculator
Compute nPr — the number of ordered arrangements of r items chosen from n — alongside n!, r! and the related combination nCr.
Prime Factorization Calculator
Factor any integer from 2 up to 10¹² into primes, see the canonical exponent form, count and list all divisors.
Geometric Mean Calculator
Compute the n-th root of the product of n positive numbers — the right average for growth rates, returns and ratios — alongside the arithmetic mean for comparison.
Fibonacci Sequence Calculator
Enter any integer n from 0 to 1500 to instantly compute F(n) and F(n−1) with BigInt precision, the consecutive-term ratio (converging to the golden ratio φ) and the first 30 terms of the sequence.
Dice Roll Probability Calculator
Pick the number of dice, sides (d4 / d6 / d8 / d10 / d12 / d20) and a target sum to compute the probability of rolling exactly that total, at least, or at most.
Arithmetic Series Calculator
Enter first term a, common difference d and number of terms n to compute the nth term aₙ and the partial sum Sₙ = n/2·(2a + (n − 1)d).
Survey Sample Size Calculator
Enter confidence level, margin of error, expected proportion (and optional population size) to compute the survey sample size you need.
Geometric Series Sum Calculator
Enter the first term a, common ratio r and number of terms n to find the sum of the first n terms of a geometric series — plus the infinite-series sum when |r| < 1.
Cone Volume & Surface Area Calculator
Enter base radius and height to get the cone volume, slant height, lateral, base and total surface area.
Music Note Frequency Calculator
Pick a note (C, C♯, D, …), octave and tuning reference A4 (440 Hz by default) and the tool returns the frequency in Hz, the wavelength in air and the MIDI note number via f = A4 × 2^((n − 69)/12).
Linear Interpolation Calculator (Lerp)
Enter two known points (x₁, y₁) and (x₂, y₂) and a target x to instantly read off y via y = y₁ + (x − x₁)(y₂ − y₁)/(x₂ − x₁) — the tool flags whether your x sits inside the two points (interpolation) or outside (extrapolation).
Trapezoid Area Calculator
Enter the two parallel sides and height of a trapezoid to get its area, midsegment length and perimeter when the legs are known.
Binomial Probability Calculator
Enter the number of trials n, the per-trial success probability p and a target number of successes k to get P(X = k), P(X ≤ k), P(X ≥ k) along with the distribution mean and standard deviation.
Pearson Correlation Coefficient Calculator
Paste two parallel data series (X and Y) and get the Pearson correlation coefficient r, r², the best-fit line slope and intercept, and the sample means and standard deviations.
Confidence Interval for the Mean Calculator
Enter the sample mean, sample standard deviation, sample size and confidence level to compute a confidence interval for the population mean using the t (or z) distribution, with margin of error and standard error.
Cohen's d Effect Size Calculator
Enter the means, standard deviations and sample sizes of two groups to compute Cohen's d and Hedges' g effect sizes, classified per Cohen 1988 as trivial / small / medium / large.
Modulo (Remainder) Calculator
Enter dividend a and divisor n to compute the quotient and remainder a mod n. The truncated (C / JavaScript %), floored (Python / mathematical) and Euclidean conventions are shown side by side so the negative-input differences are obvious.
chmod Permission Converter (Octal ↔ rwx)
Toggle read/write/execute for user, group and other to instantly get both octal (e.g. 755) and symbolic (e.g. rwxr-xr-x) notations.
Vector Magnitude & Direction Calculator (2D / 3D)
Enter 2D or 3D vector components (x, y, z) to compute the magnitude, unit vector and direction angles.
Percent Error Calculator (Percent Error & Percent Difference)
Enter experimental and theoretical (accepted) values to get absolute percent error, signed relative error, and absolute error. Switch to percent-difference mode to compare two measurements where neither is the accepted value.
Poisson Distribution Probability Calculator
Enter the mean event rate λ and an event count k to get P(X = k), P(X ≤ k), P(X ≥ k), the mean, variance and standard deviation — useful for queueing, call-centre staffing and rare-event modelling.
Bayes’ Theorem Probability Calculator
Enter the prior P(A), the sensitivity P(B|A) and the false-positive rate P(B|¬A); the tool applies Bayes’ theorem to give the posterior P(A|B) — handy for medical tests, spam filters and any classification decision.
Arc Length & Sector Area Calculator
Enter a circle’s radius and central angle (degrees or radians) to compute the arc length s = r·θ, sector area A = ½·r²·θ and chord length — useful for geometry, machining and architectural layout.
Matrix Determinant Calculator (2×2 and 3×3)
Enter the entries of a 2×2 or 3×3 matrix and instantly get the determinant det(A) via ad − bc or cofactor expansion along the first row, with an invertibility check and the step-by-step 2×2 minors shown for linear-algebra study.
Vector Dot Product Calculator (2D / 3D)
Enter two 2D or 3D vectors and instantly compute the dot product, included angle, scalar projection and vector projection — with hints for orthogonal, parallel and anti-parallel cases.
Cosine Similarity Calculator
Enter two numeric vectors of any dimension and compute cos θ = (a·b) / (|a|·|b|), plus the angular distance — the staple similarity metric in NLP, recommender systems and document-similarity scoring.
Percent Difference Calculator (Symmetric)
Enter two values and compute the symmetric percent difference |a − b| / ((|a| + |b|) / 2) alongside the directional percent change (b − a)/a — a common source of confusion in lab reports and news headlines.
Vector Cross Product Calculator (3D)
Enter two 3D vectors a and b, compute the cross product a × b together with its magnitude (the area of the parallelogram they span) and sin θ — the workhorse of physics torque, computational geometry and 3D graphics.
Distance Between Two Points Calculator (2D / 3D)
Enter two points in the plane or in 3D space and the tool returns the straight-line Euclidean distance √Σ(Δᵢ)², each axis delta and the midpoint — handy for geometry homework, CAD measurements and planar GIS work.
Coefficient of Variation Calculator (CV)
Enter a dataset and the tool returns the coefficient of variation CV = σ / μ × 100% (standard deviation divided by mean) — the standard yardstick for comparing dispersion across datasets with different units or magnitudes (lab repeatability, portfolio risk).
Median, Quartiles & IQR Calculator
Enter a dataset and the tool sorts it and returns the median, first and third quartiles, the inter-quartile range (IQR) and Tukey 1.5×IQR outlier fences — the five-number summary behind every box plot.
Harmonic Mean Calculator
Enter a set of positive numbers and compute the harmonic mean HM = n / Σ(1/xᵢ) alongside the arithmetic and geometric means — the right average when the data behaves like rates or denominators (speeds, P/E ratios, flow rates).
Regular Polygon Calculator (Area, Angles, Radii)
Enter the side count n and side length s of a regular polygon and the tool returns the interior and exterior angles, the apothem and circumradius, the area and the perimeter in one shot — works for triangles, squares, pentagons, hexagons and beyond.
Catalan Number Calculator (Cₙ)
Enter n and the tool returns the n-th Catalan number Cₙ = (2n)! / ((n+1)! · n!) — the textbook count for balanced parentheses, binary tree shapes, Dyck paths and many other combinatorics problems.
RMSE / MAE Forecast Error Calculator
Enter a column of actual values and a column of predicted values; the tool computes MAE, MSE, RMSE, R² and MAPE side-by-side — the staple set of regression-error metrics in machine learning, forecasting and lab-vs-model comparisons — and explains how each one weights outliers differently.
Linear System Solver — 2 Equations, 2 Unknowns
Enter the 6 coefficients of ax + by = e and cx + dy = f. The tool applies Cramer's rule to return x, y, the determinant D and the system's classification (unique solution / no solution / infinitely many), showing the working out so it doubles as a homework check.
At-Least-One Probability Calculator
Enter a per-trial probability p and a trial count n; the tool returns the probability of at least one success 1 − (1 − p)ⁿ, the expected count np, and the trials needed to reach a target probability — the maths behind raffles, gacha pulls, A/B tests and redundancy design.