Shreya Shankar

schoolUniversity of Californiabar_chart#2639 Global Rank

Total Papers

HCI Rank

#2639

Research Areas

Years Active

Recent Publications

RAG Without the Lag: Enabling "What-If" Analysis for Retrieval-Augmented Generation Pipelines

Retrieval-augmented generation (RAG) pipelines have become the de-facto approach for building AI assistants with external knowledge. Given a user query, RAG pipelines retrieve (R) information from external sources, before invoking a Large Language Model (LLM), augmented (A) with this information, to generate (G) respo…

UIST '251st AuthorHonorable MentionHuman-LLM Collaboration +2

Steering Semantic Data Processing With DocWrangler

Unstructured text has long been difficult to analyze automatically at scale. Large language models (LLMs) now make it possible to perform “semantic data processing,” where common data operators like map, reduce, and filter are powered by LLMs instead of traditional code. However, building these pipelines is challengin…

UIST '254th AuthorHuman-LLM Collaboration +1

Rethinking Dataset Discovery with DataScout

Dataset Search---the process of finding appropriate datasets for a given task---remains a critical yet under-explored challenge in data science workflows. Assessing dataset suitability for a task (e.g., training a classification model) is a multi-pronged affair that involves understanding: data characteristics (e.g. g…

CSCW '241st AuthorSession 3e: Trusting the Machine: Fake News. AI Decision-Making, and Auditing

"We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning

Organizations rely on machine learning engineers (MLEs) to deploy models and maintain ML pipelines in production. Due to models' extensive reliance on fresh data, the operationalization of machine learning, or MLOps, requires MLEs to have proficiency in data science and engineering. When considered holistically, the j…

UIST '241st AuthorHuman-LLM Collaboration

Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences

Due to the cumbersome nature of human evaluation and limitations of code-based evaluation, Large Language Models (LLMs) are increasingly being used to assist humans in evaluating LLM outputs. Yet LLM-generated evaluators simply inherit all the problems of the LLMs they evaluate, requiring further human validation. We…

Quick Actions

Share this page

ios_share

X in mail

https://hci.top/en/authors/27431?source=all