Advanced

Observer reliability of industrial activity analysis based on video recordings

Kazmierczak, Karolina LU ; Mathiassen, SE; Neumann, Patrick LU and Winkel, J (2006) In International Journal of Industrial Ergonomics 36(3). p.275-282
Abstract
The aim of this study was to assess the agreement between observers analyzing activity patterns during truck engine assembly work based on video recordings. Two observers observed the recordings of nine workers, on the average 2.2 h long, assigning activities to four activity categories. For each activity category data were obtained on the mean duration of uninterrupted sequences of activities and their relative time proportion in the job. This data was analyzed with 2-way crossed ANOVA algorithms to derive the components of variance attributed to disagreement between observers, to differences between filmed subjects, and to residual "unexplained" variance. The latter was interpreted as an estimate of within-observer variability and... (More)
The aim of this study was to assess the agreement between observers analyzing activity patterns during truck engine assembly work based on video recordings. Two observers observed the recordings of nine workers, on the average 2.2 h long, assigning activities to four activity categories. For each activity category data were obtained on the mean duration of uninterrupted sequences of activities and their relative time proportion in the job. This data was analyzed with 2-way crossed ANOVA algorithms to derive the components of variance attributed to disagreement between observers, to differences between filmed subjects, and to residual "unexplained" variance. The latter was interpreted as an estimate of within-observer variability and possible interactions between subject and observer. While the observers disagreed about the overall time proportions for the four activity categories by no more than 3.7% of time, their second-to-second classification disagreed for 13% of the total analysis time. The between-observer variance was small as compared to within-observer variance and the variance between Subjects performing the same job. Simulations based oil the variance components showed that a group mean of the proportion of direct work could be determined with a standard deviation within 5% of the mean by having two observers analyzing one 2-h video recording once, each. Relevance to industry The results of this study may support decision making when designing a reliable video-based analysis of industrial work. Thus, the study helps production engineers, ergonomics practitioners and researchers allocate resources between data collection and data analysis, based on their preferences for precision and power of a particular study. (c) 2006 Elsevier B.V. All rights reserved. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
assembly, video recordings, observer reliability, activity analysis, ergonomics, work
in
International Journal of Industrial Ergonomics
volume
36
issue
3
pages
275 - 282
publisher
Elsevier
external identifiers
  • wos:000236506700010
  • scopus:33644655044
ISSN
0169-8141
DOI
10.1016/j.ergon.2005.12.006
language
English
LU publication?
yes
id
7727b4ea-a162-44af-a1d2-a9c5fcb829fb (old id 414454)
date added to LUP
2007-10-07 15:06:33
date last changed
2019-09-11 02:38:38
@article{7727b4ea-a162-44af-a1d2-a9c5fcb829fb,
  abstract     = {The aim of this study was to assess the agreement between observers analyzing activity patterns during truck engine assembly work based on video recordings. Two observers observed the recordings of nine workers, on the average 2.2 h long, assigning activities to four activity categories. For each activity category data were obtained on the mean duration of uninterrupted sequences of activities and their relative time proportion in the job. This data was analyzed with 2-way crossed ANOVA algorithms to derive the components of variance attributed to disagreement between observers, to differences between filmed subjects, and to residual "unexplained" variance. The latter was interpreted as an estimate of within-observer variability and possible interactions between subject and observer. While the observers disagreed about the overall time proportions for the four activity categories by no more than 3.7% of time, their second-to-second classification disagreed for 13% of the total analysis time. The between-observer variance was small as compared to within-observer variance and the variance between Subjects performing the same job. Simulations based oil the variance components showed that a group mean of the proportion of direct work could be determined with a standard deviation within 5% of the mean by having two observers analyzing one 2-h video recording once, each. Relevance to industry The results of this study may support decision making when designing a reliable video-based analysis of industrial work. Thus, the study helps production engineers, ergonomics practitioners and researchers allocate resources between data collection and data analysis, based on their preferences for precision and power of a particular study. (c) 2006 Elsevier B.V. All rights reserved.},
  author       = {Kazmierczak, Karolina and Mathiassen, SE and Neumann, Patrick and Winkel, J},
  issn         = {0169-8141},
  keyword      = {assembly,video recordings,observer reliability,activity analysis,ergonomics,work},
  language     = {eng},
  number       = {3},
  pages        = {275--282},
  publisher    = {Elsevier},
  series       = {International Journal of Industrial Ergonomics},
  title        = {Observer reliability of industrial activity analysis based on video recordings},
  url          = {http://dx.doi.org/10.1016/j.ergon.2005.12.006},
  volume       = {36},
  year         = {2006},
}