GenId

CryptographicRandomField{T} <: AbstractGeneratedField

Cryptographically secure random field using RandomDevice() for unpredictable random generation.

This field type provides cryptographic-quality randomness suitable for security-sensitive applications, following RFC 9562 Section 6.9 recommendation to use CSPRNG (Cryptographically Secure Pseudo-Random Number Generator).

Characteristics:

• Uses /dev/urandom on Unix or CryptGenRandom on Windows
• Unpredictable even with knowledge of previous values
• Suitable for security tokens, session IDs, or security-critical UUIDs
• Slightly slower than RandomField due to system entropy source

When to use:

• UUIDs used as security tokens or session identifiers
• Applications requiring unpredictable IDs for security
• Compliance with strict security standards
• When RFC 9562 "SHOULD use CSPRNG" applies

When NOT to use:

• High-performance scenarios where security is not critical
• Database primary keys (use monotonic counter instead)
• Simple distributed IDs without security requirements

Examples

# Secure UUIDv7 configuration
TSIDGenericContainer(
    Int128,
    :SecureUUIDv7,
    [
        TimestampField(Int128, 80, 48, UNIX_EPOCH_START),
        ConstantField(Int128, :version, 76, 4, 7),
        CryptographicRandomField(UInt128, 64, 12),  # Secure 12-bit random
        ConstantField(Int128, :variant, 62, 2, 2),
        CryptographicRandomField(UInt128, 0, 62)    # Secure 62-bit random
    ],
    text_coder
)

TSIDGenericContainer(type, name, fields, text_coder)

Generic container for time-sorted unique identifiers (TSIDs/UUIDs).

127-Bit Limitation for Int128 Containers

IMPORTANT: When using Int128 as the container type, only bits 0-126 (127 bits total) are available for ID fields. Bit 127 is reserved as the sign bit and must remain zero to ensure positive integer values.

Rationale:

• Database compatibility: Some systems don't handle signed 128-bit integers correctly if bit 127 is set
• Avoids negative number issues: Setting bit 127 makes the Int128 negative
• Consistent behavior: All IDs remain positive for reliable comparisons and sorting

Maximum representable value: typemax(Int128) = 2^127 - 1 (all bits 0-126 set)

Field Layout: Ensure the sum of all field bit lengths ≤ 127 and the highest field bit position ≤ 126

Fields

• type::DataType: Container integer type (Int64, Int128, UInt64, or UInt128)
• name::Symbol: Identifier name for this UUID scheme
• fields::Vector{AbstractField}: Ordered list of bit fields composing the ID
• text_coder::TextCoder: Encoding/decoding configuration for text representation

Examples

# 64-bit Snowflake ID (uses all 63 bits available in Int64)
container = TSIDGenericContainer(
    Int64,
    :SnowflakeId,
    [
        TimestampField(Int64, 22, 41, epoch_start),  # bits 22-62
        ConstantField(Int64, :machine, 12, 10, machine_id),  # bits 12-21
        MachineSequenceField(Int64, 0, 12)  # bits 0-11
    ],
    make_crockford_base_32_coder()
)

# 128-bit ULID (uses 127 bits: bits 0-126)
container = TSIDGenericContainer(
    Int128,
    :ULID,
    [
        TimestampField(UInt64, 80, 47, UNIX_EPOCH_START),  # bits 80-126
        RandomField(UInt128, 0, 80)  # bits 0-79
    ],
    make_crockford_base_32_coder()
)

UUIDv7DistributedCounterField{T} <: AbstractGeneratedField

Custom field type for UUIDv7 distributed counter that increments within each millisecond per machine and resets when timestamp advances.

Typically uses 6 bits (0-63), providing 64 IDs per millisecond per machine. When combined with 6-bit machine ID, provides 4096 IDs/ms across cluster.

Thread-safe via lock-protected atomic operations.

UUIDv7MonotonicCounterField{T} <: AbstractGeneratedField

Custom field type for UUIDv7 monotonic counter that increments within each millisecond and resets to a random value when the timestamp advances.

Provides guaranteed monotonic ordering within each millisecond with capacity of 4096 IDs per millisecond (12-bit counter).

Thread-safe via lock-protected atomic operations.

UUIDv7SubMillisecondField{T} <: AbstractGeneratedField

Custom field type for UUIDv7 sub-millisecond precision that captures high-resolution time within the current millisecond.

With 12 bits, provides theoretical resolution of 1ms / 4096 ≈ 244 nanoseconds. Actual precision is limited by system clock (typically 100ns-1μs).

Does not guarantee uniqueness - must be combined with random bits.

UUIDv7DistributedDefinition(machine_id::Int)

Creates a distributed UUIDv7 definition for multi-datacenter deployments.

This implementation uses:

• 48-bit Unix millisecond timestamp (bits 80-127)
• 4-bit version field = 7 (bits 76-79)
• 6-bit machine ID (bits 70-75) - identifies the node/datacenter/pod (0-63)
• 6-bit monotonic counter (bits 64-69) - per-machine counter (0-63 per ms)
• 2-bit variant field = 0b10 (bits 62-63)
• 62-bit random field (bits 0-61)
• Total: 62 bits of cryptographic randomness

The machine ID must be assigned uniquely to each node (0-63). The counter increments independently on each machine, ensuring no runtime coordination needed.

Bit allocation:

• Machine ID: 6 bits = 64 unique machines/datacenters/pods
• Counter: 6 bits = 64 IDs per millisecond per machine
• Total cluster capacity: 64 × 64 = 4096 IDs/ms

Machine ID assignment strategies:

• Environment variable: parse(Int, get(ENV, "MACHINE_ID", "0"))
• MAC address hash: hash(mac_address) % 64
• Kubernetes pod ordinal: parse(Int, ENV["POD_ORDINAL"]) % 64
• Consul/etcd registration: auto-assigned from service registry

Characteristics:

• Uniqueness: Guaranteed (64 IDs/ms per machine)
• Distributed: 64 machines, no runtime coordination
• Per-node capacity: 64 IDs/millisecond = 64K IDs/second
• Cluster capacity: 4096 IDs/millisecond total
• State management: Counter + timestamp + machine ID
• Thread safety: Lock-protected atomic operations
• Configuration: One-time machine ID assignment

Important: Use only ONE machine ID per process. Multiple UUIDv7DistributedDefinition instances with different machine IDs in the same process will share counter state (edge case limitation).

Use cases:

• Multi-datacenter deployments
• Kubernetes clusters (use pod ID as machine ID)
• Distributed microservices
• Systems without central coordination

Advantages:

• ✅ No runtime coordination needed
• ✅ Fault-tolerant (node failures don't affect others)
• ✅ Clear capacity per node
• ✅ Similar to Twitter Snowflake pattern

Text encoding: Compact hex format without hyphens (32 characters)

Security Note: Uses non-cryptographic random for performance. Do not use UUIDs as security tokens. For cryptographic security, consider Julia's stdlib UUIDs.uuid7().

Arguments

• machine_id::Int: Unique machine identifier (0-63)

Examples

julia> machine_id = parse(Int, get(ENV, "MACHINE_ID", "0"))
julia> defn = UUIDv7DistributedDefinition(machine_id)
julia> id = tsid_generate(defn)
julia> uuid_str = tsid_to_string(defn, id)
"0192e209-8e14-7b4a-9f71-8c5d3e2a1b0f"

# Extract machine ID from generated UUID
julia> tsid_getfield_value(defn, :machine_id, id)
0

UUIDv7MonotonicDefinition()

Creates a monotonic UUIDv7 definition with guaranteed ordering within milliseconds.

This is the RECOMMENDED default implementation using:

• 48-bit Unix millisecond timestamp (bits 80-127)
• 4-bit version field = 7 (bits 76-79)
• 12-bit monotonic counter (bits 64-75) - increments within each millisecond
• 2-bit variant field = 0b10 (bits 62-63)
• 62-bit random field (bits 0-61)
• Total: 62 bits of cryptographic randomness

The counter increments within each millisecond and resets to a random value when the timestamp advances, ensuring:

• Strict monotonic ordering within same millisecond
• No collisions (up to 4096 IDs/ms capacity)
• Unpredictable counter initialization across restarts

Characteristics:

• Uniqueness: Guaranteed (4096 IDs per millisecond)
• Collision probability: Zero (within capacity limits)
• Monotonicity: Strict within millisecond
• State management: Global counter + timestamp (24 bytes)
• Thread safety: Lock-protected atomic operations
• Performance: ~4M IDs/second (4096/ms capacity)

Use cases:

• Database primary keys (PRIMARY RECOMMENDATION)
• High-throughput event logging
• Distributed tracing systems
• Any system requiring strict ordering

Advantages:

• ✅ Best B-tree index locality for databases
• ✅ Guaranteed monotonic ordering
• ✅ Clear capacity limits (4096/ms)
• ✅ RFC 9562 Section 6.2 Method 1 (officially recommended)

Text encoding: Compact hex format without hyphens (32 characters)

Security Note: Uses non-cryptographic random for performance. Do not use UUIDs as security tokens. For cryptographic security, consider Julia's stdlib UUIDs.uuid7().

Examples

julia> defn = UUIDv7MonotonicDefinition()
julia> id1 = tsid_generate(defn)
julia> id2 = tsid_generate(defn)
julia> id1 < id2  # Guaranteed monotonic
true
julia> uuid_str = tsid_to_string(defn, id1)
"0192e209-8e14-7b4a-9f71-8c5d3e2a1b0f"

UUIDv7SecureDistributedDefinition(machine_id::Int)

Cryptographically secure version of UUIDv7DistributedDefinition().

Identical to UUIDv7DistributedDefinition except:

• Uses CryptographicRandomField for 62 random bits
• Machine ID and counter are still predictable (structural data, not random)
• ~2x slower than standard distributed configuration

When to use this instead of UUIDv7DistributedDefinition():

• ✅ Multi-datacenter authentication systems
• ✅ Distributed security-sensitive applications
• ✅ Kubernetes deployments requiring unpredictable IDs
• ❌ High-throughput microservices (>50K IDs/second per node)

Security Note: Machine ID and counter values are predictable by design (needed for distributed coordination). Only the 62-bit random payload is cryptographically secure.

RFC 9562 Compliance: Fully compliant with Section 6.9 CSPRNG recommendation for random bits.

Arguments

• machine_id::Int: Unique machine identifier (0-63)

Examples

julia> machine_id = parse(Int, get(ENV, "MACHINE_ID", "0"))
julia> defn = UUIDv7SecureDistributedDefinition(machine_id)
julia> id = tsid_generate(defn)

UUIDv7SecureMonotonicDefinition()

Cryptographically secure version of UUIDv7MonotonicDefinition() with secure random bits.

Identical to UUIDv7MonotonicDefinition except:

• Uses CryptographicRandomField for 62 random bits
• Counter initialization still uses fast rand() (counter behavior, not security-critical)
• Main random payload is cryptographically secure
• ~1.5-2x slower than standard monotonic configuration

When to use this instead of UUIDv7MonotonicDefinition():

• ✅ Database primary keys for security-sensitive tables (user auth, payments)
• ✅ Event logging where UUIDs must be unpredictable
• ✅ Systems requiring both ordering AND security
• ❌ High-frequency event streams (>100K IDs/second)
• ❌ Non-sensitive database records

Security Note: Counter values are sequential and predictable (by design for monotonicity). Only the 62-bit random payload is cryptographically secure. If counter values must also be unpredictable, use UUIDv7SecureSimpleDefinition() instead.

RFC 9562 Compliance: Fully compliant with Section 6.9 CSPRNG recommendation for random bits.

Examples

julia> defn = UUIDv7SecureMonotonicDefinition()
julia> id1 = tsid_generate(defn)
julia> id2 = tsid_generate(defn)
julia> id1 < id2  # Still monotonic
true

UUIDv7SecureSimpleDefinition()

Cryptographically secure version of UUIDv7SimpleDefinition() using RandomDevice().

Identical to UUIDv7SimpleDefinition except:

• Uses CryptographicRandomField for all 74 random bits
• Cryptographically unpredictable random values from /dev/urandom (Unix) or CryptGenRandom (Windows)
• Suitable for security-sensitive applications
• ~2-3x slower than standard simple configuration

When to use this instead of UUIDv7SimpleDefinition():

• ✅ UUIDs used as security tokens or session identifiers
• ✅ Authentication systems requiring unpredictable IDs
• ✅ Compliance with strict security standards (HIPAA, PCI-DSS, etc.)
• ✅ Applications where security > performance
• ❌ High-throughput database primary keys (use monotonic instead)
• ❌ Non-security-critical distributed IDs

RFC 9562 Compliance: Fully compliant with Section 6.9 CSPRNG recommendation.

Examples

julia> defn = UUIDv7SecureSimpleDefinition()
julia> id = tsid_generate(defn)  # Cryptographically secure
julia> uuid_str = tsid_to_string(defn, id)
"0192e209ae3f7c8b8f714d9e2b1a0c3f"

UUIDv7SecureSubMillisecondDefinition()

Cryptographically secure version of UUIDv7SubMillisecondDefinition().

Identical to UUIDv7SubMillisecondDefinition except:

• Uses CryptographicRandomField for 62 random bits
• Sub-millisecond fraction still uses time_ns() (temporal data, not random)
• ~2x slower than standard sub-millisecond configuration

When to use this instead of UUIDv7SubMillisecondDefinition():

• ✅ High-precision event logging with security requirements
• ✅ Scientific instrumentation requiring unpredictable IDs
• ✅ Forensics and audit trails
• ❌ High-frequency trading (latency-critical)

RFC 9562 Compliance: Fully compliant with Section 6.9 CSPRNG recommendation.

Examples

julia> defn = UUIDv7SecureSubMillisecondDefinition()
julia> id = tsid_generate(defn)

UUIDv7SimpleDefinition()

Creates a simple RFC 9562 compliant UUIDv7 definition with pure random bits.

This is the baseline implementation using:

• 48-bit Unix millisecond timestamp (bits 80-127)
• 4-bit version field = 7 (bits 76-79)
• 12-bit random field (bits 64-75)
• 2-bit variant field = 0b10 (bits 62-63)
• 62-bit random field (bits 0-61)
• Total: 74 bits of cryptographic randomness

Characteristics:

• Uniqueness: 2^74 combinations (~1.9×10²²)
• Collision probability: Negligible for <10M IDs per millisecond
• Monotonicity: None within same millisecond
• State management: None required (stateless)
• Thread safety: Native (no shared state)
• Performance: ~10M IDs/second

Use cases:

• General-purpose UUID generation
• Applications without high-frequency bursts
• Strict RFC compliance requirements
• Systems where simplicity is prioritized

Text encoding: Compact hex format without hyphens (32 characters)

Security Note: Uses non-cryptographic random for performance. Do not use UUIDs as security tokens. For cryptographic security, consider Julia's stdlib UUIDs.uuid7().

Examples

julia> defn = UUIDv7SimpleDefinition()
julia> id = tsid_generate(defn)
julia> uuid_str = tsid_to_string(defn, id)
"0192e209-8e14-7b4a-9f71-8c5d3e2a1b0f"

UUIDv7SubMillisecondDefinition()

Creates a UUIDv7 definition with sub-millisecond precision for precise event ordering.

This implementation uses:

• 48-bit Unix millisecond timestamp (bits 80-127)
• 4-bit version field = 7 (bits 76-79)
• 12-bit sub-millisecond fraction (bits 64-75) - ~244ns theoretical resolution
• 2-bit variant field = 0b10 (bits 62-63)
• 62-bit random field (bits 0-61)
• Total: 62 bits of cryptographic randomness

The sub-millisecond field captures high-precision time within the current millisecond using time_ns(). With 12 bits, this provides 4096 divisions per millisecond (theoretical resolution ~244 nanoseconds).

Note: Actual precision is limited by system clock resolution (typically 100ns-1μs). This does NOT guarantee uniqueness - collisions are still possible.

Characteristics:

• Uniqueness: Probabilistic (62 bits random)
• Collision probability: Negligible (62 bits entropy)
• Temporal resolution: ~244 nanoseconds (theoretical)
• Actual precision: 100ns-1μs (system dependent)
• State management: None required
• Thread safety: Native
• Performance: ~5M IDs/second

Use cases:

• High-frequency trading systems
• Precision event logging (microsecond resolution)
• Scientific instrumentation
• Systems with high-precision clock requirements
• Debugging and forensics

Text encoding: Compact hex format without hyphens (32 characters)

Security Note: Uses non-cryptographic random for performance. Do not use UUIDs as security tokens. For cryptographic security, consider Julia's stdlib UUIDs.uuid7().

Examples

julia> defn = UUIDv7SubMillisecondDefinition()
julia> id = tsid_generate(defn)
julia> uuid_str = tsid_to_string(defn, id)
"0192e209-8e14-7b4a-9f71-8c5d3e2a1b0f"

bit_mask_int(type, v, from, to)

Applies a 64 bit signed integer mask to a value v, with ones between bits in postitions between from and to and zeroes for other bits.

Counting of bits starts at 0.

v can't be a negative integer.

Examples

julia> bit_mask_int(Int64, typemax(Int64), 0,0)
1
julia> bit_mask_int(Int64, typemax(Int64), 0,1)
3

bit_mask_uint(type, v, from, to)

Masks a 64 bit unsigned integer v with ones between bits in postitions between from and to and zeroes for other bits.

Counting of bits starts at 0.

Examples

julia> bit_mask_uint(UInt64, typemax(UInt64), 0,0)
0x0000000000000001
julia> bit_mask_uint(UInt64, typemax(UInt64), 12, 21)
0x00000000003ff000

bit_mask_uint(from, to)

Creates a 64 bit unsigned integer mask, with ones between bits in postitions between from and to and zeroes for other bits.

Counting of bits starts at 0.

Examples

julia> bit_mask_uint(0,0)
0x0000000000000000000000000000000000000000000000000000000000000001
julia> bit_mask_uint(63,63)
0x1000000000000000000000000000000000000000000000000000000000000000
julia> bit_mask_uint(12,21)
0x0000000000000000000000000000000000000000001111111111000000000000

crockford32_decode_int64(s_input::String; skip_fn=skip_dashes_13_1, with_checksum=false)

Decodes a Crockford Base 32 encoded text into an Int64.

If with_checksum is set to true, parses the last character from the text as a modulo 37 checksum.

Examples

julia> crockford32_decode_int64("Z")
31

julia> crockford32_decode_int64("ZZ"; with_checksum=true)
31

julia> crockford32_decode_int64("Z-Z"; with_checksum=true)
31

crockford32_decode_uint64(s_input::String; skip_fn=skip_dashes_13_1, with_checksum=false)

Decodes a Crockford Base 32 encoded text into an UInt64.

If with_checksum is set to true, parses the last character from the text as a modulo 37 checksum.

Examples

julia> crockford32_decode_uint64("Z")
0x000000000000001f

julia> crockford32_decode_uint64("ZZ"; with_checksum=true)
0x000000000000001f

julia> crockford32_decode_uint64("Z-Z"; with_checksum=true)
0x000000000000001f

crockford32_encode_int64(n::UInt64; started_init::Bool=false, with_checksum::Bool=false)

Encodes an Int64 n using Crockford Base 32.

If with_checksum is set to true, adds a modulo 37 checksum character at the end.

If started_init is set to true, pads the string to the left to 13(14 when a checksum is generated) characters.

Examples

julia> crockford32_encode_int64(31)
"Z"

julia> crockford32_encode_int64(31; with_checksum=true)
"ZZ"

julia> crockford32_encode_int64(31; with_checksum=true, started_init=true)
"000000000000ZZ"

crockford32_encode_uint64(n::UInt64; started_init::Bool=false, with_checksum::Bool=false)

Encodes an UInt64 n using Crockford Base 32.

If with_checksum is set to true, adds a modulo 37 checksum character at the end.

If started_init is set to true, pads the string to the left to 13(14 when a checksum is generated) characters.

Examples

julia> crockford32_encode_uint64(0x000000000000001f)
"Z"

julia> crockford32_encode_uint64(0x000000000000001f; with_checksum=true)
"ZZ"

julia> crockford32_encode_uint64(0x000000000000001f; with_checksum=true, started_init=true)
"000000000000ZZ"

Reset the UUIDv7 distributed counter to a random or specified value.

Examples

julia> reset_uuidv7_distributed_counter()  # Random value
julia> reset_uuidv7_distributed_counter(10)  # Specific value

Reset the UUIDv7 monotonic counter to a random or specified value.

Examples

julia> reset_uuidv7_monotonic_counter()  # Random value
julia> reset_uuidv7_monotonic_counter(100)  # Specific value

tsid_int_from_string(def::TsIdDefinition, s::AbstractString)

Creates a new UUID from a textual representation in s based on text_* flags in def`.

Examples

julia> tsid_from_string(iddef, "DJR0RGDG0401")
489485826766409729

tsid_to_string(def::TsIdDefinition, tsid::T) where T <: Integer

Creates a new UUID from a textual representation in s based on text_* flags in def`.

Examples

julia> tsid_to_string(iddef, 489485826766409729)
"DJR0RGDG0401"