Xingyuan Zhao

Methodology

Research on the annotation, evaluation, comparison, and causal-inference methods required to make computational research outputs auditable and interpretable.

R&R at JCSSLLM AnnotationHPCNSF FundedText-as-Data

Using Messy Text in Future LLM Annotations

Researchers often inherit annotations built over years of costly human coding, but noise and ambiguity in older corpora make them unusable for modern text-as-data pipelines. This research is supported by an NSF grant.

  • Deployed an LLM pipeline on HPC that uses codebook-guided attention to realign annotations with noisy text, producing clean, evidence-traceable summaries for downstream tasks.
  • Enables researchers to reuse annotations acquired over years of costly human coding, reducing weeks of manual labor to hours of parallelized computation without requiring re-annotation.
LLM EvaluationBenchmarkingHPC

The Scope Lines for Social Science Research Tasks Using LLMs

Existing LLM benchmarks use QA and multiple-choice formats on low-risk texts. This project measures how much LLM bias degrades research results across annotation, extraction, classification, and simulation tasks with varying risk levels.

  • Benchmarks LLM bias magnitude across research tasks and corpora with varying risk levels, not the QA and multiple-choice formats that dominate existing benchmarks.
  • Produces scope lines identifying which task-text pairs tolerate unmitigated LLM bias for research use, replacing trial-and-error with reproducible, task-specific thresholds.
Hyperspectral ImagingArcGISMultimodal LLMBenchmarking

UAV Hyperspectral Landmine and UXO Detection: ArcGIS vs Multimodal LLM

A direct head-to-head comparison of classical geospatial methods and AI-assisted detection on a humanitarian mine and UXO field survey.

  • Benchmarks ArcGIS hyperspectral anomaly detection against a locally served multimodal LLM on UAV aerial imagery from a humanitarian mine and UXO field survey.
  • Establishes an empirical comparison between classical geospatial ML and AI-assisted detection on a safety-critical task where false negatives have lethal consequences.
Information ExtractionNamed Entity ExtractionComparative PoliticsBenchmarking

LLM-Based Rhetoric Extraction from Censorship-Related Legal Texts

This paper extracts rights- and security-related rhetoric from censorship legislation and measures it at cross-national scale.

  • Benchmarks two extraction pipelines against each other, establishing a reproducible, auditable method for measuring censorship rhetoric across national legal corpora.
  • Span-verifiable annotations make every measurement checkable and the cross-national comparison reproducible at scale.
Synthetic ControlCausal Inference

Contextualized Synthetic Control: The NVIDIA Chip Ban and the AI Arms Race

Standard synthetic control requires the treated unit to be comparable to others in the donor pool. This paper extends the estimator to handle cases where no comparable donors exist.

  • Augments synthetic control with multi-head attention to produce valid counterfactuals when the treated unit has no comparable donors.
  • Relaxes the donor-pool comparability requirement, extending causal inference to cases previously excluded by scale disparity.