AIware 2024
Mon 15 - Tue 16 July 2024 Porto de Galinhas, Brazil, Brazil
co-located with FSE 2024
Mon 15 Jul 2024 14:50 - 14:55 at Mandacaru - Industry Talk2 + Human AI Conversation Chair(s): Qinghua Lu

The integration of Large Language Models (LLMs) and chatbots introduces new challenges and opportunities for decision-making in software testing. Decision-making relies on a variety of information, including code, requirements specifications, and other software artifacts that are often unclear or exist solely in the developer's mind. To fill in the gaps left by unclear information, we often rely on assumptions, intuition, or previous experiences to make decisions. This paper explores the potential of LLM-based chatbots like Bard, Copilot, and ChatGPT, to support software testers in test decisions such as prioritizing test cases effectively. We investigate whether LLM-based chatbots and human testers share similar ``assumptions'' or intuition in prohibitive testing scenarios where exhaustive execution of test cases is often impractical. Preliminary results from a survey of 127 testers indicate a preference for diverse test scenarios, with a significant majority (96%) favoring dissimilar test sets. Interestingly, two out of four chatbots mirrored this preference, aligning with human intuition, while the others opted for similar test scenarios, chosen by only 3.9% of testers. Our initial insights suggest a promising avenue within the context of enhancing the collaborative dynamics between testers and chatbots.

Mon 15 Jul

Displayed time zone: Brasilia, Distrito Federal, Brazil change

14:00 - 15:30
Industry Talk2 + Human AI ConversationMain Track / Industry Statements and Demo Track at Mandacaru
Chair(s): Qinghua Lu Data61, CSIRO
14:00
20m
Industry talk
AI Assistant in JetBrains IDE: Insights and Challenges
Industry Statements and Demo Track
Andrey Sokolov JetBrains Research
14:20
10m
Paper
Unveiling the Potential of a Conversational Agent in Developer Support: Insights from Mozilla’s PDF.js Project
Main Track
João Correia PUC-Rio, Morgan C. Nicholson University of São Paulo, Daniel Coutinho Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Caio Barbosa Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Marco Castelluccio Mozilla, Marco Gerosa Northern Arizona University, Alessandro Garcia Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Igor Steinmacher Northern Arizona University
DOI Pre-print
14:30
10m
Paper
From Human-to-Human to Human-to-Bot Conversations in Software Engineering
Main Track
Ranim Khojah Chalmers | University of Gothenburg, Francisco Gomes de Oliveira Neto Chalmers | University of Gothenburg, Philipp Leitner Chalmers | University of Gothenburg
DOI Pre-print
14:40
10m
Paper
RUBICON: Rubric-Based Evaluation of Domain-Specific Human AI Conversations
Main Track
Param Biyani Microsoft, Yasharth Bajpai Microsoft, Arjun Radhakrishna Microsoft, Gustavo Soares Microsoft, Sumit Gulwani Microsoft
DOI
14:50
5m
Paper
Unveiling Assumptions: Exploring the Decisions of AI Chatbots and Human Testers
Main Track
Francisco Gomes de Oliveira Neto Chalmers | University of Gothenburg
DOI
14:55
35m
Live Q&A
Session Q&A and topic discussions
Main Track