The technologies surveyed in 0001.a have effects at three levels of scale. An individual meets them as a feed or a page of search results. Institutions, such as companies, hospitals, and employers, reorganize their work around them. Societies adopt them, regulate them, and change their habits in response. This section moves up through those levels, and business incentives connect them, because what a company earns money from shapes what an individual sees and how an institution operates. Influence also runs in both directions, since society shapes technology as well as the reverse, which is where the section ends.

What individuals see#

Most people experience computing's influence through ranked lists, where software has decided what comes first. Recommendation systems are algorithms that rank content using signals such as clicks and watch time, which show what holds a user's attention and help predict what to show next. Optimizing for engagement can favor provocative content over accurate content, because provocation tends to hold attention whatever its accuracy, and sensational stories can crowd out better reporting. Ranking by past behavior can also narrow what users see. Since popularity isn't a measure of accuracy, a video's view count says little about whether it's true. The same mechanism does offer a person some control: changing what you follow and engage with changes what is ranked highly.

Search looks like a more neutral version of the same thing, since you asked the question, but it's ranked too. Search engines order results with proprietary algorithms and may include paid placements, marked "sponsored," which appear at least partly because a company paid. Search engine optimization (SEO) is changing a website to improve its rank, and companies keep ranking details secret partly to limit that kind of manipulation, changing their methods as well. Results differ across users because of location and history, so two students with the same query may not see the same page. For all of these reasons, a high rank doesn't prove reliability, and it makes sense to compare top results with what independent sources say.

What pays for it#

These rankings make more sense once you know who is paying. Services that are free to users are mostly funded by advertising, often targeted using data about those users. The advertisers are the customers, and users' attention and data are what's sold, a situation often summarized by saying that the user is the product. This is why a free app may request data it doesn't need for its features, since additional data makes its advertising more valuable, and why a service may prefer whatever keeps people on it longest. A paid, ad-free tier replaces that revenue with direct payment, so the user pays with money rather than with attention and data.

Institutions and businesses often pay for software in a second common way. Software as a service (SaaS) delivers applications over the Internet, usually by subscription, with the provider handling hosting and updates. The older alternative is on-premise software, installed and run on the customer's own computers and servers, so the customer handles hosting and updates and keeps working even if the vendor changes its terms. Providers gain recurring revenue and data about use. Customers gain easy access, but they depend on the provider's continued service and terms and hold no permanent copy, so they can lose access if payments stop or terms change, and a vendor's closure makes exporting data and finding a replacement urgent. Here the same pattern appears at the level of an organization: a school district that moves its records to a hosted tool has gained convenience and given up some control.

How institutions change#

Dependence of this kind is sharpest where the stakes are highest, and health care shows both sides. Electronic health records let clinicians share information quickly across sites, so an emergency room can see allergies and medications recorded elsewhere. They also concentrate risk, because one breach or outage can affect many patients at once: a single system holds many identifiable, valuable records, and care itself comes to depend on that system staying up. Some of the risk comes from insiders, and role-based access (permissions limited to what a person's job requires) with reviewed logs deters staff from snooping. One change in how an institution stores information has thus brought new benefits, new vulnerabilities, and new kinds of oversight.

Institutions also change the work people do, and automation is the clearest case. It tends to replace specific tasks rather than whole occupations, shifting the mix of work within a job. It eliminates some jobs, changes others, and creates roles that need different skills, such as maintaining and monitoring robots. Its costs fall hardest on workers who can't get retraining for those new skills. Even when a tool produces the output, evaluating that output still takes skill, so people keep the work of judging and directing it.

Software can also take over the supervising. Algorithmic management is software that assigns work, sets pay, and evaluates workers, as on gig platforms, making decisions that workers may find hard to see, understand, or challenge. What sets it apart from a human manager is this opacity: a driver may not know why a shift went to someone else. Metrics also miss work they don't measure, such as training new staff, so human review and appeal matter. And because software control over pay and access can be extensive, it raises questions about how independent "independent" workers really are.

How societies respond#

Health records concentrate trust in the institution that keeps them. A blockchain approaches record-keeping differently, spreading it among many participants, and so it raises questions at the level of a whole economy. A blockchain is a shared, append-only ledger, meaning that entries are added and not overwritten. Each block includes a hash of the previous block (hashes are described under 0004.b), so altering an old entry changes its hash, breaks the links to later blocks, and can be detected. That property makes past transactions hard to alter, and it brings costs along with it. Many cryptocurrencies use substantial energy, especially under proof of work, a method of validating transactions in which computers compete at deliberately costly calculations. Payments generally can't be reversed, and consumer protections are limited, so someone who pays a scammer has little recourse. A ledger protects what was recorded, not whether it was true when recorded, which means incorrect data entered at the start is simply hard to correct.

A blockchain's trade-offs came from choices about how it should work, which is true of technology in general. Technologies aren't neutral or inevitable. The values, incentives, and regulations of the societies that build them shape design. Designers' priorities affect how well a system works for different groups, default settings reflect choices (autoplay turned on reflects an interest in engagement), and a data-storage law can force a redesign. Adoption then changes culture, often in ways designers never intended, as smartphones changed expectations about memory and response time.¹ Once you see technology as shaped by human choices, it becomes fair to ask whose interests those choices serve, and 0001.c takes up that question.

• recommendation system: an algorithm that ranks content using signals such as clicks and watch time
• search engine optimization (SEO): changing a website to improve its rank in search results
• on-premise software: software installed and run on the customer's own computers and servers
• software as a service (SaaS): applications delivered over the Internet, usually by subscription, with the provider handling hosting and updates
• electronic health record: a digital patient record that clinicians can share across sites
• algorithmic management: software that assigns work, sets pay, and evaluates workers
• blockchain: a shared, append-only ledger in which each block includes a hash of the previous block
• proof of work: a method of validating transactions through costly computation, which uses substantial energy

Practice: practice questions for 0001.b

1. LAI 516, Design of a UB-BPS high school: how values and institutions shape computing education. ↩