קולוקוויום וסמינרים

Factual reliability remains a critical bottleneck for LLMs, whose tendency to hallucinate undermines user trust. To mitigate this problem, we need methods to scientifically measure what a model knows and why it makes factual mistakes. However, many fundamental research questions in this space are not trivial to test, demanding the design of dedicated, controlled experiments precisely tailored to isolate the relevant phenomenon. Using three recent papers as case studies, we will explore the full research cycle: from defining the research question and formulating a hypothesis, to designing a controlled study to test it. 

The papers we will discuss are:
[1] Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? (EMNLP 2024)
[2] Inside-Out: Hidden Factual Knowledge in LLMs (COLM 2025)
[3] Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs (COLM 2026)

Modern software engineering agents solve tasks by interleaving reasoning with API calls that observe and modify their environment. On long-horizon tasks this degrades performance: the context fills with API documentation, authentication flows, and parameter details, forcing the agent to interleave reasoning about the task with discovery of the interface used to solve it, a conflict that compounds into failure.

We introduce SCOPE, an approach that decouples task analysis from implementation by a method of API transpiling. A planner agent first reasons about the task in isolation, defines a focused set of environment interactions it requires, and transpiles the raw APIs into a compact, task-specific toolset, verifying each tool before handoff. A solver agent then operates over this narrowed
interface, free of extraneous API documentation and complicated workflows.

On the AppWorld Benchmark with Minimax-M2, SCOPE reaches 81% accuracy on test_normal, a 5-point gain over a vanilla ReAct baseline under identical conditions, and a 71% accuracy on test_challenge, a 6-point gain over the baseline, while shortening solver trajectories by 48% and collapsing the exposed API surface from 74 endpoints to 4 task-specific tools on average.

My research focuses on mechanistic interpretability-understanding and improving multi-modal models by analyzing their internal computations. I will present recent studies, including Diffusion Lens (ACL 2024), Padding Tone (NAACL 2025), and Follow the Flow (ACL 2026), and DeLeaker (ICL 2026). These works propose methods to reveal what text-to-image models learn and how computations are performed internally. These insights are then applied to solve concrete problems, such as semantic leakage, paving the way for more reliable and transparent generative AI.