Changes in version 0.1.0 (2026-08-30)                  

First release.

aiEvalR provides a statistical and psychometric framework for evaluating
large language models and other AI systems, organized around seven
modules.

Modules

  - Psychometric quality (methodological core): multi-facet
    generalizability theory for AI evaluation. ai_generalizability()
    fits a crossed G-study (case x prompt x model x run) via lme4;
    ai_dstudy() projects the generalizability coefficient and
    dependability index (Phi) for any measurement design without
    refitting. Also ai_conditional_sem() and ai_decision_consistency().
  - Reliability: ai_test_retest() (ICC), ai_internal_consistency()
    (Cronbach's alpha), ai_bootstrap_reliability(), and the
    ai_reliability() wrapper.
  - Robustness: stress_test(), prompt_sensitivity(),
    adversarial_stability(), and the ai_robustness() wrapper.
  - Fairness: ai_group_disparity() (demographic parity, equalized odds),
    ai_dif_mh() (Mantel-Haenszel screening), and ai_fairness(), which
    integrates with the companion aiDIF package for item-level IRT-based
    AI-scoring bias when item MLEs are supplied.
  - Calibration: ece(), brier_score(), calibration_curve(),
    empirical_output_interval(), and the ai_calibration() wrapper, with
    input validation throughout.
  - Hallucination: hallucination_rate() (aggregation of adjudicated
    verdicts), lexical_overlap() (a lexical baseline, deliberately named
    to avoid implying factual-consistency detection), and
    citation_accuracy().
  - Dashboard: ai_dashboard() combines rescaled module scores into a
    descriptive overall profile.

Testing

Comprehensive test suite (~93% line coverage), verifying statistical
behavior against known-value cases rather than merely checking for
non-null output.

Vignettes

  - getting-started: a module-by-module tour with runnable examples.
  - psychometric-core: a deeper treatment of the generalizability-theory
    workflow (G-studies and D-studies).