Victor Agostinelli

Victor Agostinelli is a PhD candidate in the School of Electrical Engineering and Computer Science at Oregon State University, advised by Lizhong Chen. His research interests cover the intersection between efficient deep learning architectures and the design of custom hardware to accelerate those architectures in the form of software and hardware (SW/HW) co-design.

Previously, Victor earned his Bachelor of Science degree from Oregon State University in 2020 and his Master of Science degree from Oregon State University in 2023. Currently, he actively collaborates with Antonino Tumeo at Pacific Northwest National Laboratory as part of the Distinguished Graduate Research Program to improve upon the optimization of specialized hardware for artificial intelligence.

news

Jun 26, 2024	New preprint on simultaneous translation with LLMs! SimulMask represents a significant departure from typical causal masking to unify fine-tuning and inference context management for simultaneous tasks.
Mar 16, 2024	Two papers accepted in short order! LeaPformers and Simul-LLM at ICML ‘24 and ACL ‘24 respectively. To be presented as posters at both conferences.
Jan 20, 2024	Started working with researchers at Pacific Northwest National Laboratory in a more formal fashion! Playing around with HLS and Torch-to-Verilog flow optimization with MLIR for AI acceleration.

selected publications

arXiv

Simultaneous Masking, Not Prompting Optimization: A Paradigm Shift in Fine-tuning LLMs for Simultaneous Translation

Matthew Raffel, Victor Agostinelli, and Lizhong Chen

2024

Abs arXiv Code

Large language models (LLMs) have achieved state-of-the-art performance in various language processing tasks, motivating their adoption in simultaneous translation. Current fine-tuning methods to adapt LLMs for simultaneous translation focus on prompting optimization strategies using either data augmentation or prompt structure modifications. However, these methods suffer from several issues, such as unnecessarily expanded training sets, computational inefficiency from dumping the key and value cache, increased prompt sizes, or restriction to a single decision policy. To eliminate these issues, in this work, we propose SimulMask, a new paradigm for fine-tuning LLMs for simultaneous translation. It utilizes a novel attention mask approach that models simultaneous translation during fine-tuning by masking attention for a desired decision policy. Applying the proposed SimulMask on a Falcon LLM for the IWSLT 2017 dataset, we have observed a significant translation quality improvement compared to state-of-the-art prompting optimization strategies on five language pairs while reducing the computational cost.
ACL ’24

Simul-LLM: A Framework for Exploring High-Quality Simultaneous Translation with Large Language Models

Victor Agostinelli, Max Wild, Matthew Raffel, and 2 more authors

In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Aug 2024

Abs arXiv Code

Large language models (LLMs) with billions of parameters and pretrained on massive amounts of data are now capable of near or better than state-of-the-art performance in a variety of downstream natural language processing tasks. Neural machine translation (NMT) is one such task that LLMs have been applied to with great success. However, little research has focused on applying LLMs to the more difficult subset of NMT called simultaneous translation (SimulMT), where translation begins before the entire source context is available to the model. In this paper, we address key challenges facing LLMs fine-tuned for SimulMT, validate classical SimulMT concepts and practices in the context of LLMs, explore adapting LLMs that are fine-tuned for NMT to the task of SimulMT, and introduce Simul-LLM, the first open-source fine-tuning and evaluation pipeline development framework for LLMs focused on SimulMT.
ICML ’24

LeaPformer: Enabling Linear Transformers for Autoregressive and Simultaneous Tasks via Learned Proportions

Victor Agostinelli, Sanghyun Hong, and Lizhong Chen

In Forty-first International Conference on Machine Learning, Aug 2024

Abs arXiv Code

A promising approach to preserving model performance in linearized transformers is to employ position-based re-weighting functions. However, state-of-the-art re-weighting functions rely heavily on target sequence lengths, making it difficult or impossible to apply them to autoregressive and simultaneous tasks, where the target and sometimes even the input sequence length are unknown. To address this issue, we propose Learned Proportions (LeaP) and LeaPformers2. Our contribution is built on two major components. First, we generalize the dependence on explicit positional representations and sequence lengths into dependence on sequence proportions for re-weighting. Second, we replace static positional representations with dynamic proportions derived via a compact module, enabling more flexible attention concentration patterns. We evaluate LeaPformer against eight representative efficient transformers on the Long-Range Arena benchmark, showing that LeaPformer achieves the best quality-throughput trade-off, as well as LeaPformer to Wikitext-103 autoregressive language modeling and simultaneous speech-to-text translation for two language pairs, achieving competitive results.