Building, checking and diagramming a theory

theoryforge treats a scientific theory as a versioned, machine-checkable object. This vignette shows how to build a theory, check its rigour and diagram it. The package overview describes the R/Python parity guarantee.

library(theoryforge)

Building a theory

A theory is built incrementally with the BUILDING-mode verbs. Each verb appends to the theory, records a provenance entry and returns the theory, so the steps compose with the native pipe.

theory <- tf_theory("panic-network", "A network theory of panic") |>
  tf_add_construct("c_arousal", "Physiological arousal",
                   "Bodily activation in response to a stressor.",
                   measurement = "heart rate variability",
                   boundary_conditions = "awake adults") |>
  tf_add_construct("c_threat", "Perceived threat",
                   "Appraised danger of bodily sensations.",
                   measurement = "self-report appraisal scale",
                   boundary_conditions = "awake adults") |>
  tf_add_proposition(
    "p1", "c_arousal", "c_threat", "causes",
    mechanism = "Activation raises the salience of threat cues."
  ) |>
  tf_add_prediction("h1", "Arousal raises threat appraisal by a fixed amount.",
                    "point", derives_from = "p1")

isTRUE(tf_validate(theory)) # structural checks: required fields and enums
[1] TRUE
# also ids, cross-references and the rest of the schema
isTRUE(tf_validate(theory, full = TRUE))
[1] TRUE

tf_validate() returns TRUE invisibly on success and otherwise stops with a message listing every problem. The calls above wrap it in isTRUE() to show the result. With full = TRUE, it additionally checks referential integrity: that ids are unique and that every cross-reference points to a declared id. It then checks the rest of the schema, such as the evidence directions, a logical passed on each test outcome and the type of every other field. It does not require any optional dependency.

A failed validation shows what the full pass adds. A prediction pointing at a proposition that was never declared still passes the structural checks, since every required field is present and well formed. Only full = TRUE reports the problem.

broken <- theory
broken$predictions[[1]]$derives_from <- "p_missing"
tf_validate(broken, full = TRUE)
Error:
! invalid theory object: prediction[0] derives_from 'p_missing' is not a known proposition

BUILDING mode has three further verbs. tf_add_alternative() registers a rival account, which a prediction can name in its diagnostic_vs to declare that it discriminates between the two. tf_add_assumption() records an auxiliary assumption, and tf_set_formal_model() attaches a formal-model reference. Because each verb returns a modified copy, assigning the result to a new object leaves the original theory untouched.

extended <- theory |>
  tf_add_alternative("alt_cognitive", "Cognitive model of panic",
                     key_constructs = "catastrophic misinterpretation") |>
  tf_add_assumption("a1", "Arousal is measured at rest.", protects = "h1") |>
  tf_set_formal_model("sem", spec_ref = "panic-sem.lavaan")

The provenance entries are stored in the theory object and make up the record of how it was built. Each entry records the step, the action and its detail.

do.call(rbind, lapply(extended$provenance, as.data.frame))
  step              action        detail
1    1           tf_theory panic-network
2    2    tf_add_construct     c_arousal
3    3    tf_add_construct      c_threat
4    4  tf_add_proposition            p1
5    5   tf_add_prediction            h1
6    6  tf_add_alternative alt_cognitive
7    7   tf_add_assumption            a1
8    8 tf_set_formal_model           sem

A theory can be written to and read back from disk. The format follows the file extension, .json for JSON and otherwise YAML.

path <- tempfile(fileext = ".yaml")
tf_write(theory, path)
roundtrip <- tf_read(path)
identical(roundtrip$id, theory$id)
[1] TRUE

Checking rigour

tf_check() runs the 12-item rigour checklist defined in the copy of rigor_checklist.yaml included in the package.

report <- tf_check(theory)

report$aggregate_score   # weighted score, 0-100, rounded to 1 dp
[1] 87
report$gate              # "pass", "blocked", or "advisory" (draft maturity)
[1] "pass"
report$n_blockers_failed # count of failed blocker items
[1] 0
# Per-item detail (checklist order preserved):
report$items[[1]]$id      # "falsifiability"
[1] "falsifiability"
report$items[[1]]$status  # "pass" / "warn" / "fail"
[1] "pass"
report$items[[1]]$score   # numeric in [0, 1]
[1] 1

An item with nothing to assess, such as the redundancy screen of a theory with a single construct, has the status "n/a" and a NULL score. It is left out of the aggregate score, which is the weighted mean of the applicable items. report$coverage gives the applicable items’ share of the checklist’s weight.

tf_report() renders the report as valid, pretty-printed JSON or as an HTML fragment. The JSON string is shown here verbatim.

cat(tf_report(theory, format = "json"))
{
  "theory_id": "panic-network",
  "schema_version": "1.0",
  "checklist_version": "2.0",
  "maturity": "building",
  "aggregate_score": 87,
  "coverage": 0.92,
  "gate": "pass",
  "n_blockers_failed": 0,
  "items": [
    {
      "id": "falsifiability",
      "status": "pass",
      "score": 1,
      "weight": 0.15,
      "severity_if_fail": "blocker",
      "citation": "Popper (1959); Bacharach (1989)"
    },
    {
      "id": "precision",
      "status": "pass",
      "score": 1,
      "weight": 0.1,
      "severity_if_fail": "warning",
      "citation": "Meehl (1967, 1990a)"
    },
    {
      "id": "risk_severity",
      "status": "pass",
      "score": 0.9,
      "weight": 0.1,
      "severity_if_fail": "warning",
      "citation": "Popper (1959); Meehl (1967, 1990b)"
    },
    {
      "id": "parsimony",
      "status": "n/a",
      "score": null,
      "weight": 0.08,
      "severity_if_fail": "warning",
      "citation": "Lakatos (1970); Meehl (1990a)"
    },
    {
      "id": "non_redundancy",
      "status": "pass",
      "score": 1,
      "weight": 0.1,
      "severity_if_fail": "warning",
      "citation": "Kelley (1927); Le et al. (2010); Lawson & Robins (2021)"
    },
    {
      "id": "construct_clarity",
      "status": "pass",
      "score": 1,
      "weight": 0.08,
      "severity_if_fail": "warning",
      "citation": "Suddaby (2010); Cronbach & Meehl (1955); Flake & Fried (2020)"
    },
    {
      "id": "scope",
      "status": "pass",
      "score": 1,
      "weight": 0.06,
      "severity_if_fail": "warning",
      "citation": "Whetten (1989); Bacharach (1989)"
    },
    {
      "id": "logical_why",
      "status": "pass",
      "score": 1,
      "weight": 0.08,
      "severity_if_fail": "warning",
      "citation": "Sutton & Staw (1995); Whetten (1989)"
    },
    {
      "id": "causal_testability",
      "status": "pass",
      "score": 1,
      "weight": 0.06,
      "severity_if_fail": "warning",
      "citation": "Textor et al. (2016); Eronen & Bringmann (2021)"
    },
    {
      "id": "diagnosticity",
      "status": "warn",
      "score": 0,
      "weight": 0.06,
      "severity_if_fail": "warning",
      "citation": "Platt (1964); Fiedler (2017)"
    },
    {
      "id": "formalisation",
      "status": "warn",
      "score": 0,
      "weight": 0.05,
      "severity_if_fail": "warning",
      "citation": "Robinaugh et al. (2021); Guest & Martin (2021)"
    },
    {
      "id": "derivation_chain",
      "status": "pass",
      "score": 1,
      "weight": 0.08,
      "severity_if_fail": "blocker",
      "citation": "Scheel et al. (2021); Szollosi et al. (2020)"
    }
  ]
}

The HTML fragment can be inserted directly into a page, where it renders as a table.

cat(tf_report(theory, format = "html"))

Rigour report: panic-network

Aggregate score: 87.0 · gate: pass

item status score grounding
falsifiability pass 1.0 Popper (1959); Bacharach (1989)
precision pass 1.0 Meehl (1967, 1990a)
risk_severity pass 0.9 Popper (1959); Meehl (1967, 1990b)
parsimony n/a n/a Lakatos (1970); Meehl (1990a)
non_redundancy pass 1.0 Kelley (1927); Le et al. (2010); Lawson & Robins (2021)
construct_clarity pass 1.0 Suddaby (2010); Cronbach & Meehl (1955); Flake & Fried (2020)
scope pass 1.0 Whetten (1989); Bacharach (1989)
logical_why pass 1.0 Sutton & Staw (1995); Whetten (1989)
causal_testability pass 1.0 Textor et al. (2016); Eronen & Bringmann (2021)
diagnosticity warn 0.0 Platt (1964); Fiedler (2017)
formalisation warn 0.0 Robinaugh et al. (2021); Guest & Martin (2021)
derivation_chain pass 1.0 Scheel et al. (2021); Szollosi et al. (2020)

Screening for redundant constructs

The lexical redundancy screen reports the Jaccard similarity and the overlap coefficient of every pair of construct definitions, sorted by descending similarity. A pair is flagged for review when the definitions are near duplicates or one is contained in the other.

tf_redundancy_check(theory)
          a        b similarity overlap flag
1 c_arousal c_threat      0.143    0.25   ok

The lexical screen is deterministic but shallow. When a semantic comparison is wanted, tf_embedding_redundancy() takes any embedder that maps a definition to a numeric vector and reports the cosine similarity of each construct pair. It needs no optional dependency, since the embedder is supplied by the caller. Here, a toy bag-of-words embedder stands in for a language model.

vocab <- c("bodily", "activation", "appraised", "danger", "salience")
embedder <- function(def) {
  words <- strsplit(tolower(def), "[^a-z]+")[[1]]
  vapply(vocab, function(w) sum(words == w), numeric(1))
}
tf_embedding_redundancy(theory, embedder)
          a        b   cosine flag
1 c_arousal c_threat 0.408248   ok

Diagramming

tf_diagram() emits deterministic intermediate representations for several diagram types. The digraphs are Graphviz DOT, and the causal DAG uses dagitty syntax.

cat(tf_diagram(theory, type = "nomological_net"))
digraph nomological_net {
  graph [rankdir=LR, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
  node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
  edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
  "c_arousal" [label="Physiological\narousal", fillcolor="#E4F1F1", color="#1E7B7B"];
  "c_threat" [label="Perceived threat", fillcolor="#E4F1F1", color="#1E7B7B"];
  "c_arousal" -> "c_threat" [label="causes"];
}
cat(tf_diagram(theory, type = "causal_dag"))
dag {
  c_arousal -> c_threat
}

The DOT strings render with any Graphviz tool. In R, tf_render_diagram() returns them as a DiagrammeR widget for the viewer and for R Markdown, or as a standalone SVG string with as = "svg". The packages it uses are optional and listed in Suggests, so the rest of the package works without them. The figure below is drawn from the nomological-net DOT printed above.

cat(
  '<div class="tf-figure tf-diagram">',
  tf_render_diagram(theory, "nomological_net", as = "svg"),
  '</div>',
  sep = ""
)
nomological_net c_arousal Physiological arousal c_threat Perceived threat c_arousal->c_threat causes

The causal DAG is the one view that tf_render_diagram() does not render, since it is written in dagitty syntax. Paste it into a dagitty tool such as dagitty.net instead. Within R, tf_implications() checks the causal DAG for acyclicity and returns the conditional independencies it entails, which the article ‘Developing and testing a theory’ works through.

Simulation

tf_simulate() treats each construct as a state variable and integrates the signed proposition network as a linear dynamical system. The trajectory is deterministic.

sim <- tf_simulate(theory, steps = 5)
unlist(sim$states)
[1] "c_arousal" "c_threat" 
unlist(sim$trajectory[[1]]) # the common initial state
[1] 1 1
unlist(sim$trajectory[[6]]) # after five Euler steps
[1] 0.773781 1.181034

The two states separate immediately. Arousal, which receives no incoming coupling, decays under the damping term. Perceived threat is pushed up by the positive coupling from arousal before the decay eventually takes over.

The default method = "euler" takes fixed explicit steps, which stay close to the solution only while dt is small against the network’s rates. The function warns when the Euler trajectory departs from the exact solution by more than 5 per cent. method = "exact" propagates the system with its matrix exponential, so each row is the solution at its time for any dt. It will become the default in the next minor release.

exact <- tf_simulate(theory, steps = 5, method = "exact")
unlist(exact$trajectory[[6]]) # the exact solution at time 0.5
[1] 0.778801 1.168201