An Interdisciplinary Perspective on Evaluation and Experimental Design for Visual Text Analytics : Position Paper
(2022) 9th IEEE Workshop on Evaluation and Beyond - Methodological Approaches to Visualization, BELIV 2022 p.28-37- Abstract
Appropriate evaluation and experimental design are fundamental for empirical sciences, particularly in data-driven fields. Due to the successes in computational modeling of languages, for instance, research outcomes are having an increasingly immediate impact on end users. As the gap in adoption by end users decreases, the need increases to ensure that tools and models developed by the research communities and practitioners are reliable, trustworthy, and supportive of the users in their goals. In this position paper, we focus on the issues of evaluating visual text analytics approaches. We take an interdisciplinary perspective from the visualization and natural language processing communities, as we argue that the design and validation... (More)
Appropriate evaluation and experimental design are fundamental for empirical sciences, particularly in data-driven fields. Due to the successes in computational modeling of languages, for instance, research outcomes are having an increasingly immediate impact on end users. As the gap in adoption by end users decreases, the need increases to ensure that tools and models developed by the research communities and practitioners are reliable, trustworthy, and supportive of the users in their goals. In this position paper, we focus on the issues of evaluating visual text analytics approaches. We take an interdisciplinary perspective from the visualization and natural language processing communities, as we argue that the design and validation of visual text analytics include concerns beyond computational or visual/interactive methods on their own. We identify four key groups of challenges for evaluating visual text analytics approaches (data ambiguity, experimental design, user trust, and "big picture"concerns) and provide suggestions for research opportunities from an interdisciplinary perspective.
(Less)
- author
- Kucher, Kostiantyn ; Sultanum, Nicole ; Daza, Angel ; Simaki, Vasiliki LU ; Skeppstedt, Maria ; Plank, Barbara ; Fekete, Jean Daniel and Mahyar, Narges
- organization
- publishing date
- 2022
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- keywords
- Human-centered computing -, Visualization, Visualization design and evaluation methods
- host publication
- Proceedings - 2022 IEEE 9th Workshop on Evaluation and Beyond - Methodological Approaches to Visualization, BELIV 2022
- pages
- 10 pages
- publisher
- IEEE - Institute of Electrical and Electronics Engineers Inc.
- conference name
- 9th IEEE Workshop on Evaluation and Beyond - Methodological Approaches to Visualization, BELIV 2022
- conference location
- Virtual, Online, United States
- conference dates
- 2022-10-17
- external identifiers
-
- scopus:85145772994
- ISBN
- 9798350396294
- DOI
- 10.1109/BELIV57783.2022.00008
- language
- English
- LU publication?
- yes
- additional info
- Publisher Copyright: © 2022 IEEE.
- id
- cc9d3309-aad2-4ff1-93ed-ac0d0142de3f
- date added to LUP
- 2023-02-01 12:56:44
- date last changed
- 2023-02-01 13:26:56
@inproceedings{cc9d3309-aad2-4ff1-93ed-ac0d0142de3f,
abstract = {{<p>Appropriate evaluation and experimental design are fundamental for empirical sciences, particularly in data-driven fields. Due to the successes in computational modeling of languages, for instance, research outcomes are having an increasingly immediate impact on end users. As the gap in adoption by end users decreases, the need increases to ensure that tools and models developed by the research communities and practitioners are reliable, trustworthy, and supportive of the users in their goals. In this position paper, we focus on the issues of evaluating visual text analytics approaches. We take an interdisciplinary perspective from the visualization and natural language processing communities, as we argue that the design and validation of visual text analytics include concerns beyond computational or visual/interactive methods on their own. We identify four key groups of challenges for evaluating visual text analytics approaches (data ambiguity, experimental design, user trust, and "big picture"concerns) and provide suggestions for research opportunities from an interdisciplinary perspective.</p>}},
author = {{Kucher, Kostiantyn and Sultanum, Nicole and Daza, Angel and Simaki, Vasiliki and Skeppstedt, Maria and Plank, Barbara and Fekete, Jean Daniel and Mahyar, Narges}},
booktitle = {{Proceedings - 2022 IEEE 9th Workshop on Evaluation and Beyond - Methodological Approaches to Visualization, BELIV 2022}},
isbn = {{9798350396294}},
keywords = {{Human-centered computing -; Visualization; Visualization design and evaluation methods}},
language = {{eng}},
pages = {{28--37}},
publisher = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
title = {{An Interdisciplinary Perspective on Evaluation and Experimental Design for Visual Text Analytics : Position Paper}},
url = {{http://dx.doi.org/10.1109/BELIV57783.2022.00008}},
doi = {{10.1109/BELIV57783.2022.00008}},
year = {{2022}},
}