jeb

jeb is a small Jev-inspired decision model built around a shared text state and independent question branches. It currently trains two typed outputs:

• Choice returns probabilities over only the supplied answer options.
• Noul returns the probability that a yes/no proposition is true.

Noul is not a fallback Choice option. Missing or unrecognized labels are masked from the loss rather than being treated as a yes or no answer.

Prepare data

Install with uv sync, then build the default dataset:

jev-prepare --output data/jev --max-per-dataset 10000 --max-questions 10

The defaults include BoolQ, PIQA, and WinoGrande. BoolQ rows become native Noul questions. Training uses one deterministic yes/no judgment per PIQA and WinoGrande item; validation uses both candidate judgments. The source item is split first, so both validation judgments stay together. Other multiple-choice rows remain Choice tasks. QASC's gold supporting-fact fields are omitted from model input because they can reveal the correct option.

The JSONL files contain structured questions as well as their rendered form. Choice branches look like this:

<|context|>...
<|question|>...
<|choice1|>...
<|choice2|>...
<|answer|>

A Noul branch has a proposition and a type marker, but no answer options:

<|context|>...
<|question|>Question: ...
Candidate answer: ...
Is this candidate a correct answer?
<|noul|><|answer|>

The target is stored separately from input tokens. Questions in a pack can read the shared context and their own branch, but cannot read other branches. With --no-group-by-context, each question's own context is placed inside its isolated branch, so it is preserved.

Train

jev-train \
  --data data/jev \
  --model answerdotai/ModernBERT-base \
  --output runs/modernbert-jev \
  --epochs 2 --batch-size 4 --grad-accumulation 4

To resume, pass the checkpoint and a new output directory. Model dimensions, tokenizer, and attention settings default to the values saved with the checkpoint; explicit CLI arguments override them.

jev-train --data data/jev \
  --resume runs/modernbert-jev/checkpoint-1 \
  --output runs/modernbert-jev-resumed

Inference

Evaluate a JSONL file:

jev-infer --checkpoint runs/modernbert-jev/best \
  --data data/jev/validation.jsonl --limit 8

Run a one-off Choice question:

jev-infer --checkpoint runs/modernbert-jev/best \
  --question "What is 2 + 2?" --choice "3" --choice "4"

Run a one-off Noul question:

jev-infer --checkpoint runs/modernbert-jev/best \
  --question-type noul \
  --question "Does the message ask for a refund?" \
  --context "I was charged twice. Please return the extra payment."