Computational Literacies Lab

Pilot study

Guided by a design-based research conjecture (Sandoval, 2014), conduct a small pilot study of your tool with users (ideally from your target population, otherwise other users, minimum three, are fine), collecting quantitative and qualitative data. Think of this assignment as a brief introduction to working with learning analytics, and as a sketch of possible future research you might conduct after this course is over.

Format

Prepare a brief report summarizing your methods, results, and analysis. A full report would have a substantial background section grounding your research questions, but you may omit that here. The pilot study will be more of a white paper for an internal audience of this class. The entire pilot study should be no more than three pages, and two pages may be quite sufficient.

Your report should have the following structure:

  • Introduction: A short paragraph positioning the study in broader context. The introduction should explain the project you're working on
  • Background: In bullet points, list the big ideas your project is exploring. End by listing your research questions, one design question and one theory question.
  • Methods: Explain the context of the study, including where you conducted it and with whom. No need to inflate this; if you met with three people on Zoom for 10 minutes each, say so. Explain how you collected your data, and your plans for analyzing it. You are required to have one qualitative and one qualitative data source.
  • Results: Present a summary of the data you collected, connecting it to your research questions.
  • Discussion: In one or two paragraphs, reflect on what you learned about your reserach questions, your project, and through conducting this research.

Note: Your final paper for the course should not be about this pilot study: there isn't time after the pilot study to include a thoughtful analysis and write-up. Instead, let the pilot study (and particularly your planning for it) help clarify your theoretical and design ideas, and inspire plans for the future.

Steps

  1. Define your research questions using Sandoval's {{< pr sandoval_2014_conjecture >}} conjecture mapping framework. You should have at least one design question and one theory question.
  2. Describe your methodology, planning backwards from your research questions. What kind of data could help you answer your research questions? What kind of environment, data collection, and analysis could help produce such data?
  3. Conduct a small-scale pilot study and report the results.