ShaReD

Sharing and Reusing Data
(Video)

LLiB Colloquium

Jürgen Schneider

06 December 2023

What is video data in education?

  • audiovisual recordings
  • captured within educational settings (primarily classrooms)
  • for the purpose of conducting scientific research (Andersson & Sørvik, 2013; Corti, 2000)


shiksastudio on youtube

  • What are we talking about when it comes to “video data” in education?
  • …
  • …
  • This could include videos of counseling sessions with school psychologists or parent-teacher talks
  • for us, and for the purpose of this presentation, this mainly refers to classroom videos

What is video data in education?


(Kristen et al., 2018)


(Troll et al., 2023)

  • Now the camera setup for these studies can vary considerably between studies, resulting in different numbers of videos and image content
  • on the left side you’ll see a very classical setup with two cameras one focusing the teacher (filming from the back) and one focusing the students (filming from front)
  • but there are also other setups with a multitude of cameras and microphones resulting in a very different video data collection and therefore different needs when sharing data

What is video data in education?


Diverse research designs

Depending on the research interest, e.g.

  • longitudinally observe one group/teacher (Syring et al., 2023)
  • cross-section over specific subject & grade (Grünkorn et al., 2020)


Connected with other data sources

Student assessment

Student survey

Teacher survey/interview

  • As you have probably suspected, these setups are designed to answer different research interests
  • And the research interest determines not only the camera setup, but also who is filmed and how often
  • So you could for example …
  • What is more: Video data is usually not used as a standalone source. Typically you would integrate the camera perspective with, …

Why is video data powerful?

1. direct observation
grants insights into processes and practices of learning and instruction

2. high degree of richness/complexity

  • spoken word, intonation, modulation
  • facial expressions, gestures, postures
  • positioning and orientation in the room
  • temporal sequence and simultaneity of events
  • …

3. allows for nuanced evaluations
repeating or manipulating (vs. in situ observation)

(Derry et al., 2010; Lahn & Klette, 2023)

  • So why am I advocating for video data in general? The answer from research on teaching and teacher quality would be: Because it is a powerful data source to investigate learning.
  • it complements product oriented measures (such as assessments) and allows us to actually observe processes and practices of learning while it happens
  • it transports a high degree of richness, by making it able to observe what is said and what is done in which way, but also what is NOT said
  • and compared to “in situ” observation, video obviously is at an advantage, because it allows us to pay detailed attention to a multitude of these aspects within the complex interactions in the classroom

Why is video data powerful?

1. direct observation
grants insights into processes and practices of learning and instruction

2. high degree of richness/complexity

  • spoken word, intonation, modulation
  • facial expressions, gestures, postures
  • positioning and orientation in the room
  • temporal sequence and simultaneity of events
  • …

3. allows for nuanced evaluations
repeating or manipulating (vs. in situ observation)

(Derry et al., 2010; Lahn & Klette, 2023)

Major contributions

  • theory of teaching and learning
  • new perspectives in teaching research

TIMSS-Video (Stigler et al., 2000)
IPN-Video Study (Seidel et al., 2006)
Pythagoras-Study (Klieme et al., 2009)
PERLE-Study (Lipowsky et al., 2014)
DESI-Study (DESI-Konsortium, 2008)
IGEL-Study (Fauth et al., 2014)
TALIS-Video Study (Grünkorn et al., 2020)

  • For this reason, video studies have made essential contributions to …
  • One of the main focus areas of video studies so far was for example: What makes successful teaching?
  • And on this slide I just listed some of the most impactful studies on this topic using video data, that involve German-speaking classrooms

Why is video data powerful?

1. direct observation
grants insights into processes and practices of learning and instruction

2. high degree of richness/complexity

  • spoken word, intonation, modulation
  • facial expressions, gestures, postures
  • positioning and orientation in the room
  • temporal sequence and simultaneity of events
  • …

3. allows for nuanced evaluations
repeating or manipulating (vs. in situ observation)

(Derry et al., 2010; Lahn & Klette, 2023)


video predestined for multiple analyses from different perspectives/ research paradigms



high cost: producing quality video data heavily taxes financial, personal and time resources

  • So to sum up: …
  • On the flip side, producing quality video data will heavily tax resources within a project
  • But once a research group has achieved high-quality video data, this makes it all the more valuable for the research community

ShaReD

  • Against this background, investigating how to best share quality video data for reuse is particularly desirable
  • As we saw before: There are standards for data sharing: If we want to optimize data sharing for reuse, making data FAIR is a central framework to follow

ShaReD

Findable

  • How do we make video data FAIR?
  • Compared to log data, we already have a pretty good idea of which recommendations can be made and which standards and templates can be used.
  • These are not equally detailed for all areas, but there are usually at least initial approaches
  • For example: If you want to make your classroom videos findable for reuse, the ideal option would be to provide it to a RDC.
  • You would provide your videos in a package together with several other documents, such as a description of the camera setup and the context of the study.
  • The RDC will also ask you to provide metadata about your videos. Metadata are descriptions about what others would be able to find in your data. For example “grade level”, “school subject”, “angle of the camera”, …

ShaReD

Findable

  • RDCs then offer a user interface where people interested in reusing video data can search for suitable videos
  • Looking through every video would be way too time consuming
  • So RDC need the metadata, descriptions of the videos, to make the data searchable and therefore findable

ShaReD

Accessible

  • Are videos usually directly accessible for the public?
  • Usually no. But what makes video data fair is to clearly state under which conditions interested people may reuse the data.
  • For many classroom videos, this condition is the use of the videos for research purposes paired with a certain qualification level (PhD student)

ShaReD

Interoperable

  • Making video data interoperable, meaning others can open and manipulate the file, can be achieved by adhering to certain file format conventions
  • these are usually defined by the RDC

ShaReD

Reusable

  • So researchers found your data, they were able to access and open it.
  • But they can only meaningfully reuse the video data if they understand what the context of the data collection is
  • The interpretation of video data may depend, for example, on whether the usual teacher delivered the lesson or whether a teacher was randomly assigned for the study
  • So if researchers provide a methods report, for example simply by linking it to a journal paper
  • and if they additionally provide a short field report that, for example specifies the camera setup
  • there are good conditions for reuse.

Project phases & focus areas

  • But, we wouldn’t be doing a project on sharing and reusing research data, if there are no challenges
  • The takeup of sharing as well as reusing classroom videos is somewhat slow, I will show you some numbers in a second

Project phases & focus areas

SCREENING

  • Which videos are available?

  • Which resources are available?

  • Facilitators & barriers in the research ecosystem

  • Cooperation with similar projects

DEVELOPMENT

  • Measures for further development of the ecosystem

  • Expanding the availability of videos

  • Creation of guidelines, handouts, training courses

TESTING

  • User studies & evaluation

  • Implementation of first training courses with evaluation

  • Presentation of the concept at conferences

  • Development of dissemination strategies

SUSTAINABILITY

  • Materials for the implementation of training courses

  • Materials for the production of videos
  • So over 4 project phases we investigate facilitators and barriers to sharing and reusing research data
  • and develop suitable measures to overcome certain challenges
  • I will not go through all project phases now, just maybe for a first idea:
  • in the first screening phase
    • we check “what is available to people who want to reuse video data?”
    • we check which resources are available to support researchers in sharing or reusing data
    • And more general: Where are the barriers and what are the requirements for sharing and reusing data?

Project phases & focus areas

SCREENING

  • Which videos are available?

  • Which resources are available?

  • Facilitators & barriers in the research ecosystem

  • Cooperation with similar projects

DEVELOPMENT

  • Measures for further development of the ecosystem

  • Expanding the availability of videos

  • Creation of guidelines, handouts, training courses

TESTING

  • User studies & evaluation

  • Implementation of first training courses with evaluation

  • Presentation of the concept at conferences

  • Development of dissemination strategies

SUSTAINABILITY

  • Materials for the implementation of training courses

  • Materials for the production of videos

Project phases & focus areas

SCREENING

  • Which videos are available?

  • Which resources are available?

  • Facilitators & barriers in the research ecosystem

  • Cooperation with similar projects

DEVELOPMENT

  • Measures for further development of the ecosystem

  • Expanding the availability of videos

  • Creation of guidelines, handouts, training courses

TESTING

  • User studies & evaluation

  • Implementation of first training courses with evaluation

  • Presentation of the concept at conferences

  • Development of dissemination strategies

SUSTAINABILITY

  • Materials for the implementation of training courses

  • Materials for the production of videos

Project phases & focus areas

SCREENING

  • Which videos are available?

  • Which resources are available?

  • Facilitators & barriers in the research ecosystem

  • Cooperation with similar projects

DEVELOPMENT

  • Measures for further development of the ecosystem

  • Expanding the availability of videos

  • Creation of guidelines, handouts, training courses

TESTING

  • User studies & evaluation

  • Implementation of first training courses with evaluation

  • Presentation of the concept at conferences

  • Development of dissemination strategies

SUSTAINABILITY

  • Materials for the implementation of training courses

  • Materials for the production of videos

Current projects

So in a current project we are exactly investigating this last question, on …

Barriers and facilitators to sharing and reusing



Which factors contribute to classroom videos being shared or reused?

… which factors …

Barriers and facilitators to sharing and reusing



Which factors contribute to classroom videos being shared or reused?



Sharing:

  • 36 publications on the analysis of classroom videos in 2022

  • in 13 there is a “data statement” (all in journals with a corresponding commitment)


  • in 0 there is a “data statement” if this is not mandatory by the journal

  • classroom videos from 1 publication were shared (own website, not RDC)
  • and I promised you some numbers
  • So I looked at publications in the English language where researchers collected video data
  • and I checked for one aspect of FAIR data, that is: is it stated if the data are available for reuse and under what condition
  • out of the 36 publications I found, 13 specified this in a “data statement”, all of which in journals where these were mandatory
  • there are 0 data statements in journals where this was not mandatory
  • and 1 of 36 studies made video data available
  • (whereas this is actually not a metric I would evaluate, there could be very good reasons not to share data)

Barriers and facilitators to sharing and reusing



Which factors contribute to classroom videos being shared or reused?



Sharing:

  • 36 publications on the analysis of classroom videos in 2022

  • in 13 there is a “data statement” (all in journals with a corresponding commitment)


  • in 0 there is a “data statement” if this is not mandatory by the journal

  • classroom videos from 1 publication were shared (own website, not RDC)

Reuse (FDZ Bildung):

  • 296 total applications (video data)
    (annual report 2022)
  • Coverage via reporting and google alerts underestimates coverage (covers approx. 75%) (Daniel et al., 2023)
  • Between 2016-2022: 8 publications and reports recorded (excluding students & teaching)
  • when it comes to reuse, we can look at some numbers of the RDC here at DIPF
  • so far there were 296 applications for reuse of video data
  • whereas so far I detected 8 (practice oriented and scientific) publications based on these data
  • again, I can’t really judge if this is high or low, there are no goals of quantity set and merely looking at a number will ignore the impact and quality of publications
  • but I found it still interesting to get a first impression

Barriers and facilitators to sharing and reusing



Which factors contribute to classroom videos being shared or reused?



Systematic Review
(top-down)

  • Search for
    • scientific studies
    • Position papers, guidelines, checklists, surveys
  • Focus on video data or qualitative data
  • So to answer the question …
  • we adopt a top-down and bottom-up approach by
  • first conducting a systematic review searching for studies, guidelines, and so on
  • that focus on video or qualitative data concenrning this question

Barriers and facilitators to sharing and reusing



Which factors contribute to classroom videos being shared or reused?



Systematic Review
(top-down)

  • Search for
    • scientific studies
    • Position papers, guidelines, checklists, surveys
  • Focus on video data or qualitative data

Delphi-Study
(bottom-up)

  • Experts:
    • FDZ employees
    • (Non-) reusers, (non-) providers
    • …
  • Complement and evaluate factors
  • and we complement this with a Delphi-Study to mutually uncover blind spots

Barriers and facilitators to sharing and reusing



Preregistration
doi.org/10.17605/OSF.IO/7FP5S


Open Data
So far on the github repo
(will be zenodo)

  • so far I have identified 1214 records from databases for the systematic review
  • I have screened these and I am currently retrieving 144 of those for full text scan
  • of course the procedure is preregistered and the current status of the data base can be viewed at github

Barriers and facilitators to sharing and reusing

SCREENING

  • Which videos are available?

  • Which resources are available?

  • Facilitators & barriers in the research ecosystem

  • Cooperation with similar projects

  • So with review and delphi-study we take a meta-perspective on the entire research ecosystem, where barriers and facilitators could be located

Guideline and resources for researchers



What are the steps involved in sharing/ reuse?
Which resources facilitate these steps?

  • A second current project deals with a challenge that has already been expressed to us a number of times
  • That is: What are necessary steps when sharing or reusing FAIR data?
  • And what are resources that might make it much more easy for researchers to realize these steps?

Guideline and resources for researchers



What are the steps involved in sharing/ reuse?
Which resources facilitate these steps?


Conceptual manuscript
(submitted)

  • Shows typical steps of sharing and reuse
  • Deliberately skips unspecific steps (e.g. data analysis)
  • Co-authors: Schreyer, Kunter, Porzelt

  • So together with colleagues from qualitative research and the infrastructure side
  • We formulated a guideline for researchers that want to share or reuse data on necessary steps and where to find resources for these

Guideline and resources for researchers



What are the steps involved in sharing/ reuse?
Which resources facilitate these steps?


Example: Informed Consent

  • classroom videos are necessarily personal data
  • need to specify the purpose, also those of future reuse
  • anticipation of reuse typically not possible
  • for example: an informed consent is mandatory when collecting video data, as these would always be personal data
  • in the consent you would have to state the purpose of the data use
  • but if you want to share the videos, you typically can’t anticipate the purpose others will reuse the data for
  • so does that mean that you can’t share the data?

Guideline and resources for researchers



What are the steps involved in sharing/ reuse?
Which resources facilitate these steps?


Example: Informed Consent

  • classroom videos are necessarily personal data
  • need to specify the purpose, also those of future reuse
  • anticipation of reuse typically not possible

Resources

  • broad consent:
    “The videos collected in this study are used by other researchers for research into teaching quality.”

  • Checklist of legally sound consents (VerbundFDB, 2019)
  • Template informed consent, standard language (VerbundFDB, 2019)
  • Template informed consent, easy language (VerbundFDB, 2019)
  • Explanations & template (DGPs)
  • a solution to this is a broad consent, where you’ll explicitly state that the videos will be reused, e.g. for a certrain field of research
  • To make it easy for you, there are several templates and checklists on informed consents using this broad consent, making sure you are actually allowed to share the data

Guideline and resources for researchers

SCREENING

  • Which videos are available?

  • Which resources are available?

  • Facilitators & barriers in the research ecosystem

  • Cooperation with similar projects

Further aspects


Podcast episode


Episode on classroom videos


GEBF 2024

  • Contribution to symposium (Schreyer)
  • “Hilfreiche Ressourcen und Arbeitsschritte für die Bereitstellung oder Nachnutzung von Unterrichtsvideos”


Data acquisition

  • Extension of the offer to FDZ Bildung
  • Contact with several projects is slow



EARLI SIG 4/17 2024

  • Contribution “Collaborative space”
  • “Promoting Open Science in Learning Research: Collaborative Exploration of Future Directions”

Here you will find some information about what else we have been working on, but I won’t go into that in detail.

Next steps

Next steps



  • Guideline on “Repository or research data center?”


  • Expanding the availability of videos (strategy needed)


  • pilot projects: Automated evaluation of classroom videos
  • Concerning the next steps we are complementing the available resources at the RDC
  • One of the questions that is still challenging to judge for researchers is: When is my data suitable for reuse at a research data center and when does is make more sense to “only” archive data? Turns out, a guideline on this question is more challenging than I initially anticipated. For example it is hard to pin down the analysis potential of videos in order to answer this question.
  • We are also currently trying to expand the availability of classroom videos on the RDC. We do that by establishing contact with authors from video studies. This, however, is going quite slow. We need to establish a strategy, otherwise I don’t see this move forward.
  • We are also considering pilot projects in the automatic analysis of video data. Like in Daniels project, we are trying to explore possibilities and examples that may inspire further research.

Let’s back up a bit

Overview

Log Data

Current challenge
No guidelines/standards on how to document Log Data to make it understandable (reusable)


Current & next steps
Pilot projects to explore possibilities of documentation and derive recommendations


Video Data

Current challenge
Unclear what hinders and enables video data sharing



Current & next steps
Meta-view and analysis of the research ecosystem, deriving appropriate measures

  • In both projects we are working on “how to make research data FAIR”
  • What is different, is that in our respective fields there are currently different challenges laying ahead
  • For log data it is sill unclear on how we can document data so that others understand and are able to reuse
  • So Daniel for example works on pilot projects that explore possibilities for FAIR documentation and the question if we can derive more general recommendations from these
  • Whereas with video data, the current challenge is to investigate where are the barriers and facilitators to video data sharing
  • we apply rather a meta-view on the research ecosystem and try to locate barriers and prerequisites

Thank you



Jürgen Schneider
ju.schneider@dipf.de

References

Andersson, E., & Sørvik, G. O. (2013). Reality Lost? Re-Use of Qualitative Data in Classroom Video Studies. Forum Qualitative Sozialforschung, 14(3). https://doi.org/10.17169/fqs-14.3.1941
Corti, L. (2000). Progress and Problems of Preserving and Providing Access to Qualitative Data for Social ResearchThe International Picture of an Emerging Culture. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, Vol 1, No 3 (2000): Text . Archive . ReAnalysis. https://doi.org/10.17169/FQS-1.3.1019
Daniel, A., Jakowatz, S., Jung, N., Kleine, L., Kocaj, A., Meyermann, A., Mozygemba, K., & Schuster, A. (2023). Die Erfassung von Publikationen aus der Datennutzung: Verfahren, Herausforderungen und Nutzen.: Ein Erfahrungsbericht von Forschungsdatenzentren. RatSWD Working Paper. https://doi.org/10.17620/02671.80
Derry, S. J., Pea, R. D., Barron, B., Engle, R. A., Erickson, F., Goldman, R., Hall, R., Koschmann, T., Lemke, J. L., Sherin, M. G., & Sherin, B. L. (2010). Conducting Video Research in the Learning Sciences: Guidance on Selection, Analysis, Technology, and Ethics. Journal of the Learning Sciences, 19(1), 3–53. https://doi.org/10.1080/10508400903452884
DESI-Konsortium (Ed.). (2008). Unterricht und Kompetenzerwerb in Deutsch und Englisch: Ergebnisse der DESI-Studie. Beltz.
Fauth, B., Decristan, J., Rieser, S., Klieme, E., & Büttner, G. (2014). Student ratings of teaching quality in primary school: Dimensions and prediction of student outcomes. Learning and Instruction, 29, 1–9. https://doi.org/10.1016/j.learninstruc.2013.07.001
Grünkorn, J., Klieme, E., Praetorius, A.-K., & Schreyer, P. (2020). Mathematikunterricht im internationalen Vergleich. Ergebnisse aus der TALIS-Videostudie Deutschland. pedocs.
Klieme, E., Pauli, C., & Reusser, K. (2009). The Pythagoras Study. Investigating effects of teaching and learning in Swiss and German mathematics classrooms. [Online; Gedruckt]. In T. Janik & T. Seidel (Eds.), The power of video studies in investigating teaching and learning in the classroom (pp. 137–160). Waxmann.
Kristen, C., Stanat, P., Kogan, I., Lorenz, G., Gentrup, S., & Rahmann, S. (2018). Unterrichtsbeobachtung - KuL. Kompetenzerwerb und Lernvoraussetzungen (KuL). https://doi.org/10.7477/287:1:0
Lahn, L. C., & Klette, K. (2023). Reactivity beyond contamination. An integrative literature review of video studies in educational research. International Journal of Research & Method in Education, 46(1), 1–18. https://doi.org/10.1080/1743727X.2022.2094356
Lipowsky, F., Stubbe, T. C., Faust, G., Künsting, J., Hadeler, S., & Bos, W. (2014). Was leisten Schülerinnen und Schüler der privaten BIP-Kreativitätsgrundschulen im nationalen Vergleich? https://doi.org/10.25656/01:9678
Seidel, T., Prenzel, M., Rimmele, R., Dalehefte, I. M., Herweg, C., Kobarg, M., & Schwindt, K. (2006). Blicke auf den Physikunterricht. Ergebnisse der IPN Videostudie. https://doi.org/10.25656/01:4489
Stigler, J. W., Gallimore, R., & Hiebert, J. (2000). Using Video Surveys to Compare Classrooms and Teaching Across Cultures: Examples and Lessons From the TIMSS Video Studies. Educational Psychologist, 35(2), 87–100. https://doi.org/10.1207/S15326985EP3502_3
Syring, M., Beck, N., Bohl, T., & Tesch, B. (2023). Klasse 6b. Eine Unterrichtswoche. Bildungswissenschaftliche und fachdidaktische Analysen. https://doi.org/10.15496/PUBLIKATION-75461
Troll, B., Pietsch, M., & Besser, M. (2023). Verhaltensbezogenes Engagement im Unterricht: Eine Analyse der Generalisierbarkeit und Zuverlässigkeit von Videobeobachtungen. Zeitschrift für Pädagogische Psychologie, 37(3), 157–171. https://doi.org/10.1024/1010-0652/a000286

Slides: bit.ly/shared-slides (view)

ShaReD Sharing and Reusing Data (Video) LLiB Colloquium Jürgen Schneider 06 December 2023

  1. Slides

  2. Tools

  3. Close
  • ShaReD
  • What is video data in education?
  • What is video data in education?
  • What is video data in education?
  • Why is video data powerful?
  • Why is video data powerful?
  • Why is video data powerful?
  • ShaReD
  • ShaReD
  • ShaReD
  • ShaReD
  • ShaReD
  • ShaReD
  • Project phases & focus areas
  • Project phases & focus areas
  • Project phases & focus areas
  • Project phases & focus areas
  • Project phases & focus areas
  • Current projects
  • Barriers and facilitators to sharing and reusing
  • Barriers and facilitators to sharing and reusing
  • Barriers and facilitators to sharing and reusing
  • Barriers and facilitators to sharing and reusing
  • Barriers and facilitators to sharing and reusing
  • Barriers and facilitators to sharing and reusing
  • Barriers and facilitators to sharing and reusing
  • Guideline and resources for researchers
  • Guideline and resources for researchers
  • Guideline and resources for researchers
  • Guideline and resources for researchers
  • Guideline and resources for researchers
  • Further aspects
  • Next steps
  • Next steps
  • Let’s back up a bit
  • Overview
  • Thank you
  • References
  • f Fullscreen
  • s Speaker View
  • o Slide Overview
  • e PDF Export Mode
  • ? Keyboard Help