The catalog itself: every identifier as a PiiCatalog.Pattern.
This module is for maintainers and for readers who want to audit a regex.
Use PiiCatalog.patterns/0 and the other functions in PiiCatalog to
read the catalog.
Adding a pattern
- An entry below. Its
nameis final once released, as callers store it. Use<identifier>_<country>in lowercase ASCII, the identifier in its local name and the country as its ISO 3166 code (personnummer_se,nino_gb); no country for an identifier used across borders (iban). - A validator in
PiiCatalog.Validatorswhere the identifier has a check digit or a structure the regex cannot express, with unit tests. - A vector file in
test/vectors/<name>.exswith valid values, values with the right shape that fail the check, and look-alikes. The catalog test fails without one. - A benchmark with every pattern on, on prose and on number-heavy text, noted in the pull request. See "Scan cost" below.
- A line in
CHANGELOG.mdnaming the identifier, and thepii_catalog.readmeMix task run to regenerate the README table.
Boundaries
Patterns are compiled without the u flag so that they match bytewise and
hold on text that is not valid UTF-8. \b then reads each byte as
Latin-1, so the bytes of a non-ASCII character next to a value would count
as word characters and the boundary would not be seen. The boundary
fragments below ask only about ASCII word characters (A-Z, a-z, 0-9
and _): a value directly after or before any other byte, including the
bytes of a non-ASCII letter, is found.
Where the pattern itself begins or ends with a word character, half of the boundary is already given and only the other half is asserted. The one lookaround keeps the scan several times cheaper than the two-sided form, which is needed only where the neighbouring character may be punctuation: the start of an email address.
Scan cost
PiiCatalog.Scanner scans every pattern that starts with
(?<![A-Za-z0-9_]) and then a digit or a capital letter in one shared
pass, so such a pattern adds little to the scan. A pattern that cannot
start that way should start with a literal or a keyword with a single
first letter (cpr, national), which the regex engine finds without
trying every position; a keyword with several first letters, or a leading
character class, costs a full pass of its own. A literal every match
contains goes in requires, and the pattern is skipped on text without
it. A keyword-led pattern whose keywords start with different letters
lists one fast regex per letter in requires instead (svnr_at,
nif_pt).
Summary
Functions
@spec all() :: [PiiCatalog.Pattern.t()]
Every pattern, in catalog order, with its regex compiled.
Builds the list on each call. Use PiiCatalog.compile/1 once and keep
the scanner rather than calling this on every scan.
@spec get(atom()) :: PiiCatalog.Pattern.t() | nil
The pattern with this name, with its regex compiled, or nil. Builds
only that pattern.
@spec names() :: [atom(), ...]
Every pattern name, in catalog order.