schism devlog - 06/26

Last cycle ended with a note to myself: stop simulating individuals at scale entirely. The binning approach was more complex than the original, produced a different history, and was not clearly faster at equivalent population sizes. The obvious next step was a histogram. This cycle I actually built it, and the speed difference is not subtle.

The histogram idea

The problem with the individual model was that the per-tick cost scaled with headcount. At 75 million adherents, iterating over every person every tick is just slow, no matter how you slice it. The insight is that schism doesn't actually need individual people. The only things that matter for each adherent are heterodoxy, age, and religion. Everything else is noise.

So instead of a list of structs, the population is now a 3D grid: age band by heterodoxy bin by religion. Each cell holds a single u64 count. The entry

count[age_band_4][het_bin_6][Orthodox] = 118_000

means "118,000 Orthodox people have heterodoxy around bin 6 and age around band 4." Not 118,000 structs. One number.

The axis order matters. I originally had heterodoxy as the outer axis, then flipped it to age-band-first. The reason is that the hot operations all want to walk by age. Births add to age band 0 across all heterodoxy bins. Aging shifts the entire het distribution one band forward. With age on the outside, those are both cheap. With het on the outside, you're jumping around for every birth.

The types ended up like this, with newtypes for the bin indices so I can't accidentally swap them:

pub struct HeterodoxyBin(usize);
pub struct AgeBand(usize);

Each Histogram is instantiated with a configurable number of bins per axis, so I can tune the resolution. The .bin() method on the histogram takes a raw float and maps it to the nearest bin index.

The new tick loop

The core loop is now three steps per tick: deaths, births, aging. No dead-adherent filtering. No soft deletes. Just numbers.

Deaths use a logistic mortality curve I added this cycle. The rate scales with population relative to carrying capacity K, so the sim converges to an equilibrium instead of growing forever. Per cell, it's one binomial draw: how many of the N people in this cell died? I draw per cell rather than per age band total, because drawing from the band total and then distributing proportionally would cut disproportionately into the smallest bins.

Births loop through every heterodoxy bin in the fertile age bands, draw a binomial for how many children are born from that cell, and deposit them at age band 0 with a heterodoxy pulled from the child heterodoxy distribution. That last part reuses the Beta distribution logic from before, but I pre-build one Beta per het bin at the start of the tick instead of constructing one per birth. That was a meaningful speedup once religions started multiplying.

Aging is the cleanest part of the whole thing. Pop the oldest band (they died), prepend an empty band at age 0 (fills in from births), shift everything else up by one index. That's it. No individual traversal at all.

Speed

The first version with just the birth/death/aging cycle and no schism ran 500 generations in 0.14 seconds. That was too fast to even measure meaningfully, and the population exploded to 10 trillion, which meant the birth rate was way too high relative to deaths. Adding the logistic mortality curve settled things down. After that:

50,000 generations: ~9 seconds

Population stabilizes around the carrying capacity and stays there. The interesting part is that 50k generations at this speed means you can actually start thinking about long-run evolutionary dynamics, not just a couple hundred ticks before the whole thing grinds to a halt.

Getting schisms working again

Fast ticks are useless without splits. Re-wiring the schism logic to the histogram took most of the week.

The trigger is the same as before: if the fraction of a religion's population above a heterodoxy threshold exceeds a max fraction, a schism fires. I print the actual fraction per religion per tick while tuning, and found that with the starting distribution the fraction was hovering around 0.015. Setting the max fraction above that meant no schisms ever. I dropped it to 0.01 (1%) which is realistic anyway, you don't need a majority to leave to start a new sect.

When a schism fires, a new religion is created with the current one as parent. Converts are drawn per het bin using a binomial: how many people in this bin defect? The probability is bin_heterodoxy * conversion_base_rate. High-heterodoxy bins lose more people, which is the right behavior.

The first version immediately produced 149,650 religions over a run, with 10,801 still active and 138,849 extinct. About 1,000 religions born and dying per generation. That's broken.

The bug was in how converts were deposited into the new religion. I was moving people into the daughter religion at their original heterodoxy bin. But the whole reason they converted is that they were the most heterodox people in the parent religion. So the new religion started with a heterodox population, immediately hit the schism threshold itself, fired again, and the cascade never stopped.

The fix: when people convert to a new religion, rescale their heterodoxy relative to the new religion's mean. The converts become the orthodoxy of the new sect. They're zealots relative to the parent, but within their new community they're mainstream. That one change collapsed the religion explosion completely.

There was also a structural issue with extinct religions. I was keeping them in the active slotmap and skipping them during iteration, which meant the map kept growing. Now extinct religions get moved to a separate Vec as they die. The active map only ever contains living religions. The dead vec gets consumed at the end for the readout.

Results

After all of that:

cargo run --release -- run -n 500  110.69s user 0.16s system 99% cpu 1:51.37 total

{
  "totals": {
    "people": "16,987,082 (million)",
    "religions": 14,
    "active": 7,
    "extinct": 7,
    "new_this_generation": 14,
    "mean_heterodoxy": 0.19593369184890003
  },
  "religions": [
    {
      "name": "Scarabine Creed",
      "adherents": "15,601,231 (million)",
      "status": "active",
      "founding_date": 0,
      "extinction_date": null,
      "age": "10,000 (thousand)",
      "parent": "none",
      "new": true
    },
    {
      "name": "Quietist Scarabine Creed",
      "adherents": "448,577 (thousand)",
      "status": "active",
      "founding_date": 80,
      "extinction_date": null,
      "age": "9,920 (thousand)",
      "parent": "Scarabine Creed",
      "new": true
    },
    ...
    {
      "name": "Esoteric Indigenous Scarabine Creed",
      "adherents": "0",
      "status": "extinct",
      "founding_date": 920,
      "extinction_date": 1120,
      "age": "200",
      "parent": "Indigenous Scarabine Creed",
      "new": true
    }
  ]
}

14 religions in 500 generations, 7 active and 7 extinct. You can read the tree from the names: Indigenous split off Scarabine Creed early and then itself spawned six children before going extinct. Quietist and Monist are direct children of the root and are still alive. The pattern of early rapid splitting followed by a long steady state makes sense. Once heterodoxy settles out within each daughter religion, the schism rate drops off.

At 1000 generations the run produces around 60 active religions and takes about 270 seconds. That's slower than I want, and the binomial drawing per histogram cell is probably where most of that time goes. Each cell needs its own draw because the count is different per cell, so there's no obvious way to batch them the way I batched the Beta samples before. That's the next thing to investigate.

What's actually slow now

The speed problem has shifted. It's no longer iteration over millions of adherents. The bottleneck now is the number of active religions multiplied by the number of histogram cells per religion. With 60 active religions each having hundreds of non-empty cells, and each cell requiring at least one binomial draw per tick, the work adds up. The binomial constructor is cheap but not free, and doing it tens of thousands of times per tick across 1000 generations is noticeable.

What's next

The sim is producing believable output for the first time at meaningful timescales. The next thing I want to build is a frontend: a visual timeline of when religions split, when they die, and how large they were. The readout JSON already has everything needed for it. Something that shows the branching structure as a tree with thickness representing adherent count would make the output much easier to read than a wall of JSON. That's the plan for next cycle.

Interreligion conversion (people switching between existing religions, not just in schism events) is also on the list. Right now religions only change size through births, deaths, and schism events. Adding drift between existing sects would make the long-run dynamics a lot more interesting.