Computational Literacies Lab

Pilot Study

Guided by a design-based research conjecture (Sandoval, 2014), conduct a small pilot study of your tool with users (ideally from your target population, otherwise other users, minimum three, are fine), collecting quantitative and qualitative data. Think of this assignment as a brief introduction to working with learning analytics, and as a sketch of possible future research you might conduct after this course is over.


Prepare a brief report summarizing your methods, results, and analysis. A full report would have a substantial background section grounding your research questions, but you may omit that here. The pilot study will be more of a white paper for an internal audience of this class. The entire pilot study should be no more than three pages, and two pages may be quite sufficient.

Your report should have the following structure:

Note: Your final paper for the course should not be about this pilot study: there isn't time after the pilot study to include a thoughtful analysis and write-up. Instead, let the pilot study (and particularly your planning for it) help clarify your theoretical and design ideas, and inspire plans for the future.


  1. Define your research questions using Sandoval's {{< pr sandoval_2014_conjecture >}} conjecture mapping framework. You should have at least one design question and one theory question.
  2. Describe your methodology, planning backwards from your research questions. What kind of data could help you answer your research questions? What kind of environment, data collection, and analysis could help produce such data?
  3. Conduct a small-scale pilot study and report the results.