Model Card for Llama3-MAGneT

Llama-3-8B-Instruct fine-tuned on the MAGneT dataset for mental health counseling dialogue generation. MAGneT sessions are generated using a coordinated multi-agent framework incorporating specialized response agents (reflection, questioning, solutions, normalizing, psycho-education), a technique agent, a CBT agent, and a response generation agent — resulting in more nuanced and clinically grounded counseling dialogues than single-agent approaches.

Model Details

Base model: meta-llama/Meta-Llama-3-8B-Instruct

Input: A system prompt with a fixed counselor role instruction, followed by the client's demographic information, reason for seeking counseling, and the dialogue history.

Output: The next counselor turn in the therapeutic dialogue.

Training data: MAGneT — 442 synthetic multi-turn counseling sessions grounded in client profiles from the CACTUS dataset, generated using a coordinated multi-agent framework.

Fine-tuning method: QLoRA

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
 
model_id = "UKPLab/Llama3-MAGneT"
 
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
 
system_prompt = (
    "You are playing the role of a counselor in a psychological counseling session. "
    "Your task is to use the provided client information to generate the next counselor "
    "response in the dialogue by combining psychological techniques like reflections, "
    "questioning, providing solutions, normalizing and psychoeducation. The goal is to "
    "create a natural and engaging response that builds on the previous conversation. "
    "Please ensure that the response is empathetic and understanding of the client's issues "
    "and builds trust between the counselor and the client. Please be mindful to only generate "
    "the counselor response for a single turn, and do not include extra text like \"here is the "
    "next counselor utterance\" or \"Here is a possible next utterance\" or anything mentioning "
    "the used technique."
)
 
client_information = (
    "Name:\nLaura Saunders\nAge:\n45\nGender:\nfemale\nOccupation: Office Job\n"
    "Education: College Graduate\nMarital Status: Single\nFamily Details: Lives alone"
)
 
reason_counseling = (
    "I decided to seek counseling because this negative belief is hindering my enjoyment "
    "of running and affecting my overall mood."
)
 
history = (
    "Counselor: Hello, Laura. It's nice to meet you. What brings you in today?\n"
    "Client: Hi, thank you. I've been struggling with the thought that I can't run far, "
    "despite enjoying running as a hobby. It's really getting me down."
)
 
user_content = (
    f"Client Information:\n{client_information}\n"
    f"Reason for seeking counseling:\n{reason_counseling}\n"
    f"Counseling Dialogue:\n{history}"
)
 
messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user",   "content": user_content},
]
 
input_ids = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)
 
with torch.no_grad():
    output = model.generate(
        input_ids,
        max_new_tokens=256,
        do_sample=True,
        temperature=0.7,
        top_p=0.9,
    )
 
response = tokenizer.decode(output[0][input_ids.shape[-1]:], skip_special_tokens=True)
print(response)

Citation

Please cite this model using:

@inproceedings{mandal-etal-2026-magnet,
    title = "{MAG}ne{T}: Coordinated Multi-Agent Generation of Synthetic Multi-Turn Mental Health Counseling Sessions",
    author = "Mandal, Aishik  and
      Chakraborty, Tanmoy  and
      Gurevych, Iryna",
    editor = "Lee, Hung-Yi  and
      Dev, Sunipa  and
      Li, Jessy  and
      Elliott, Desmond",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {EMNLP} 2026",
    month = oct,
    year = "2026",
    address = "Budapest, Hungary",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.findings-emnlp.282/",
    doi = "10.18653/2026.findings-emnlp.282",
    pages = "5934--5983",
    ISBN = "979-8-89176-462-0",
    abstract = "The growing demand for scalable psychological counseling highlights the need for high-quality, privacy-compliant data, yet such data remains scarce. Here we introduce MAGneT, a novel multi-agent framework for synthetic psychological counseling session generation that decomposes counselor response generation into coordinated sub-tasks handled by specialized LLM agents, each modeling a key psychological technique. Unlike prior single-agent approaches, MAGneT better captures the structure and nuance of real counseling. We further propose a unified evaluation framework that consolidates diverse automatic metrics and expands expert assessment from four to nine counseling aspects, thus addressing inconsistencies in prior evaluation protocols. Empirically, MAGneT substantially outperforms existing methods: experts prefer MAGneT-generated sessions in 77.2{\%} of cases on average across the nine aspects over the strongest baseline, and sessions generated by MAGneT using Llama3-8B-Instruct backbone yield 3.2{\%} higher general counseling skills and 4.3{\%} higher CBT-specific skills on cognitive therapy rating scale (CTRS). An open source Llama3-8B-Instruct model fine-tuned on MAGneT-generated data also outperforms models fine-tuned using baseline synthetic datasets by 6.9{\%} on average on CTRS. We make our code, data and fine-tuned model public."
}