10. Field study design
Week 10 (April 6)Study design for design questions and theoretical questions
LLMs in the news
- Project Glasswing: Anthropic's new model, Mythos, is apparently very good at finding vulnerabilities in software--so good that they are not releasing it for now. Instead, they are sharing it with other tech companies so they have a chance to find and fix vulnerabilities before they can be exploited.
- An extensive New Yorker profile on Sam Altman
Locally-hosted models
One technical topic we have not yet discussed is how (and whether and why) to run locally-hosted models. I have posted a lab on this topic which you can work through if you're interested. The video below is a speedrun through the lab, which should be of interest even if you're not planning to dabble in the code.
GSE's AI Server
If you are going to work through the lab, you may also be interested in trying out GSE's AI server--see the AI server page of the Computational Literacies Lab handbook. You're welcome to use the server for your own projects--including potentially using AI inference on data protected by IRB, since the data remains local.
Our AI server is running LMStudio instead of ollama--they are both fundamentally similar,
but have slightly different APIs (that is, you have to interact with them in sligntly different
ways). So you would need to modify the code in goodmorning.py to use the gseai package.
Study design for design questions and theoretical questions
By the end of the week, please send Chris an email describing your field study design. We recognize that these field studies will be limited and imperfect due to time constraints and logistical barriers--but at least the field study you conduct here can help you develop the research design for a later, more robust study.
- What is the main research question? (It should be part of your conjecture map, either a design question or a theoretical question.)
- Brieefly, what is your research design? A few possibilities:
- A quantitative observational study, where you study the association between two factors.
- A quantitative experimental study, where you study a causal relationship between two factors.
- A qualitative case study, where you interpret interaction bewtween two factors.
- What data will you use to answer the question?
- For a design question, you need data about embodiment (what affordances users used) and mediating processes.
- For a theoretical question, you need data about mediating processes and about outcomes.
- What is the current status of your field study?