Spring 2021: Law and Algorithms
This cross-cutting and interdisciplinary course investigates several aspects of algorithms and their impact on society and law. Specifically, the course connects concepts of proof, verifiability, privacy, security, trust, and randomness in computer science with legal concepts of autonomy, consent, governance, and liability, and examines interests at the evolving intersection of technology and the law.
Syllabus [link]
Readings
Class 1: Intro to Law and Algorithms (January 21)
Required
- Catherine D’Ignazio and Lauren F. Klein, “The Power Chapter,” from Data Feminism (2020) – read from “Power and the Matrix of Domination” through the end
- How Computers Work: https://www.explainthatstuff.com/howcomputerswork.html
- Doug Lloyd, Algorithms, Data Structures https://youtu.be/FnPn6RH-75M?t=969 — watch 16:00-26:00
- Alan Turing, Computing Machines and Intelligence, https://academic.oup.com/mind/article/LIX/236/433/986238 — read Sections 1-4, and 6(4)
- Kevin Lacker, Giving GPT-3 a Turing Test (July 6, 2020)
Optional
- David Aurebach, The Stupidity of Computers, n+1 (2012)
- Mireille Hildebrandt, “Law, Democracy, and the Rule of Law,” from Law for Computer Scientists and Other Folk (2020)
- Algorithmic Accountability
- Maranke Wieringa, What to Account for When Accounting for Algorithms, Proceedings on the 2020 Conference of Fairness, Accountability, and Transparency (2020)
- Joshua Kroll, Joanna Huey, Solon Barocas, Edward Felten, Joel Reidenberg, David Robinson, Harlan Yu, Accountable Algorithms, 165 U. Penn. L. Rev. 633 (2017)
- danah boyd & Kate Crawford, Critical Questions for Big Data (2012)
- Regulation of Algorithms
- Rebecca Crootof & BJ Ard, Structuring Techlaw, 82 Harv. J.L. & Tech. (forthcoming 2021)
- Danielle Keats Citron, Technological Due Process, 85 Wash. U.L. Rev. 1249 (2008)
- Tal Zarsky, The Trouble With Algorithmic Decisions, 41 Science, Technology, & Human Values 118 (2016)
Resources
- CS 50 for Lawyers: https://cs50.harvard.edu/law/2019/
Class 2: Tensions in Approach, Tensions in Terminology (January 28)
Required
- James Grimmelman’s CPU, Esq. — watch 19:30 through 27:15 on the “taxonomy of meaning.”
- Intro to Probability
- Seeing Theory: a visual introduction to probability — read, understand, and play with the demos in “Chance Events” (chapter 1, section 1), “Conditional Probability” (chapter 2, section 3), and “Bayes’ Theorem” (chapter 5, section 1).
- How One System Can Fail the Other
- Edward K. Cheng, Fighting Legal Innumeracy, 17 Green Bag 2d 271 (2014) – read sections II and III only.
- Hany Farid, Assessing the Reliability of Clothing Based Forensic Identification — watch the first 25 min
- Rebecca Kelly Slaughter, Remarks of Commissioner: Algorithms and Social Justice, UCLA Law (2020) – read Sections I and II only
- Alan R. Wagner, Jason Borenstein, Ayanna Howard, Overtrust in the Robotic Age
- The Hidden Influence of Designers
- Langdon Winner, Do Artifacts Have Politics?, 109 Daedalus 121 (1980) – read “Technical Arrangements and Forms of Order,” pp. 123–128.
- Ruha Benjamin, Introduction: Discriminatory Design, Liberating Imagination, in Captivating Technology (2019) – read pp. 4–7 (“Discriminatory Design”) only
Optional
- Human Systems and Computational Systems
- Denise Kersten, Bytes vs. Brains, Gov’t Executve (Sept. 1, 2005)
- James Grimmelman’s CPU, Esq: Should Law Be More Like Software?, DIMACS Workshop on Co-development of CS and Law (2020) — watch 19:30 through 27:15
- Pew Report on the Algorithm Age, Humanity and human judgment are lost when data and predictive modeling become paramount
- How One System Can Fail the Other
- Declaration of Charles J. Cicchetti in Texas 2020 election fraud lawsuit — from page 20
- Washington Post Trump’s effort to steal the election comes down to some utterly ridiculous statistical claims — tip: incognito mode
- Artifacts Having Politics
- Ruha Benjamin, From Park Bench to Lab Bench – What Kind of Future Are We Designing?, TEDx Baltimore (2015)
- Kate Crawford, The Hidden Biases in Big Data
- Williams, Brooks, and Shmargad, How Algorithms Discriminate Based on Data They Lack: Challenges, Solutions, and Policy Implications
- Statistical Evidence
- Michael S Pardo, The paradoxes of legal proof: a critical guide, Boston University Law Review, Vol. 99, 2019
- Edward K Cheng, Reconceptualizing the burden of proof, Yale Law Journal, Vol 122(5), 2013
- Allen, et al, Probability and Proof in State v. Skipper: An Internet Exchange, Jurimetrics 35, no. 3 (1995)
Class 3: The COMPAS Algorithm (February 4)
Required
- Algorithms in Criminal Systems
- Andrew G. Ferguson, Policing Predictive Policing, 94 Wash. U. L. Rev. 1109 (2017) – read Section I only
- Angèle Christin, Alex Rosenblat, and danah boyd, Courts and Predictive Algorithms, Data & Society (Oct. 27, 2015) – read pp. 6–9 (“Using algorithms in the criminal justice system: shifting discretion” and “‘Overrides,’ incarceration, and the role of punishment”) only
- The COMPAS Algorithm
- Julia Angwin & Jeff Larson, Machine Bias, ProPublica (May 23, 2016)
- Challenging COMPAS
- State v. Loomis, 881 N.W.2d 749 (Wisc. 2016) – read excerpt
- Henderson v. Stensberg (W.D. Wisc. March 20, 2020) – read excerpt
- Software Transparency and Intellectual Property
- Jeff Larson, Surya Mattu, Lauren Kirchner and Julia Angwin, How We Analyzed the COMPAS Recidivism Algorithm, ProPublica (May 23, 2016)
- Rebecca Wexler, Code of Silence: How Private Companies Hide Flaws in the Software that Governments Use to Decide Who Goes to Prison and Who Gets Out, Wash. Monthly (2017)
Optional
- If this is your first experience reading a legal opinion, we recommend you read Orin Kerr, How to Read a Legal Opinion, 11 Green Bag 2d 51 (2007)
- Rashida Richardson, Jason M. Schultz, and Kate Crawford, Dirty Data, Bad Predictions: How Civil Rights Violations Impact Police Data, Predictive Policing Systems, and Justice, 94 N.Y.U. L. Rev. 192 (2019)
- Rebecca Wexler, Life, Liberty, and Trade Secrets, 70 Stan. L. Rev. 1343 (2018)
- Rashida Richardson, Jason M. Schultz, and Vincent M. Sutherland, Litigating Algorithms: 2019 Report, AI Now (2019)
- Anne Washington, How to Argue with an Algorithm: Lessons from the COMPAS ProPublica Debate, 17 Colo. Tech. L.J. 131 (2018)
Class 4: The Optimization Paradox (February 11)
Required
- Deborah Hellman, What is Discrimination, when is it wrong, and why? Plenary talk at FAT* 2019
- Kleinberg, Mullainathan, & Raghavan, Inherent Trade-Offs in the Fair Determination of Risk Scores, — read Section 1
- Julila Angwin and Jeff Larson, Bias in Criminal Risk Scores Is Mathematically Inevitable, Researchers Say, ProPublica, 2016
- Karen Hao and Jonathan Stray, Can you make AI fairer than a judge? Play our courtroom algorithm game, MIT Technology Review (2019)
- New Jersey v Pickett (2021) — Read pages 3–8
- Justice in Forensic Algorithms Act of 2019
Optional
- , , and A computer program used for bail and sentencing decisions was labeled biased against blacks. It’s actually not that clear, Washington Post, 2016
- Canetti, et al. From Soft Classifiers to Hard Decisions: How fair can we be?, FAT* 2019
- Nicholas Diakopoulos, Algorithmic Accountability Reporting: On the Investigation of Black Boxes
- Christian Sandvig, et al. Auditing Algorithms:Research Methods for Detecting Discrimination on Internet Platforms, 2014
- Added Feb 8: Adam Pearce, Measuring Fairness, Google
Class 5: Is There a “Right” Way to Use Algorithms in Criminal Sentencing? (February 18)
Required
- Andrea Roth, Machine Testimony, 126 Yale L.J. 1972 (2017) – read Section III(A) only
- Aziz Huq, Racial Equity in Algorithmic Criminal Justice, 68 Duke L.J. 1043 (2019) – read Section III(D) only
- Deborah Hellman, Measuring Algorithmic Fairness, 106 Va. L. Rev. 811 (2020) — read Section I(B) and Section II only
- Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig & Sendhil Mullainathan, Human Decisions and Machine Predictions, The Quarterly Journal of Economics (2018) — read Sections I and VII only
- Sandra G. Mayson, Bias In, Bias Out, 128 Yale L.J. 2219 (2019) – Read sections IV(C) and IV(D) only
Optional
- Robyn Caplan, Joan Donovan, Lauren Hanson, & Jenna Matthews, Algorithmic Accountability: A Primer, Data & Society (2019)
- Sonia Katyal, The Paradox of Source Code Secrecy, 104 Cornell L. Rev. 1183 (2019)
- Laura M. Moy, A Taxonomy of Police Technology’s Racial Inequity Problems, 2021 U. Ill. L. Rev. 139 (2021)
- Anupam Chander, The Racist Algorithm?, 115 Mich. L. Rev. 1023 (2017)
- Virginia Foggo, John Villasenor, & Pratyush Garg, Algorithms and Fairness, 17 Ohio St. Tech. L.J. 123 (2021)
- Ben Green, The False Promise of Risk Assessments: Epistemic Reform and the Limits of Fairness, Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (2020)
- Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, Moritz Hardt, Delayed Impact of Fair Machine Learning, International Conference on Machine Learning (2018)
Class 6: Artificial Intelligence and Anti-Discrimination Laws (February 25)
Required
Introduction to Machine Learning and Data Mining:
- Yufeng Guo, The 7 Steps of Machine Learning, Google Cloud – watch all
- Solon Barocas & Andrew Selbst, Big Data’s Disparate Impact, 104 Cal. L. Rev. 671 (2016) – read Section I only
- Ruha Benjamin, Race After Technology – read excerpt on “Raising Robots,” circulated separately
Algorithms and the Fair Housing Act
- Valerie Schneider, Locked Out By Big Data: How Big Data, Algorithms and Machine Learning May Undermine Housing Justice, 52.1 Colum. Hum. Rights L. Rev. 251 (2020) – read section II(A) only
- Lauren Kirchner, Can Algorithms Violate Fair Housing Laws?, The Markup (Sept. 24, 2020) – read all
- Proposed Rule, HUD’s Implementation of the Fair Housing Act’s Disparate Impact Standard, 84 Fed. Reg. 42854 (July 29, 2019) – read excerpt
- Reconsideration of HUD’s Implementation of the Fair Housing Act’s Disparate Impact Standard, Center for Democracy & Technology (with 22 other organizations and individuals) (Oct. 18, 2019) – read all
- Lauren Sarkesian & Spandana Singh, HUD’s New Rule Paves the Way for Rampant Algorithmic Discrimination in Housing Decisions, New America (Oct. 1, 2020) – read all
- Mass. Fair Housing Ctr. v. HUD (D. Mass. Oct. 25, 2020) – read excerpt
Optional
Introduction to ML
- Noah Yonack, A Non-Technical Introduction to Machine Learning, SafeGraph (March 3, 2017)
- Reinforcement learning examples:
- Moritz Hardt and Benjamin Recht, Patterns, Predictions, and Actions: A story about machine learning
- A Neural Network Playground: http://playground.tensorflow.org/
Other Machine Learning Readings
- Jenna Burrell, How the Machine “Thinks”: Understanding Opacity in Machine Learning, Big Data & Society (2016)
- Deborah Raji, How our data encodes systematic racism, MIT Tech. Review (Dec. 10, 2020)
Fair Housing Act, Disparate Impact, and HUD:
- If you are new to administrative law, we recommend you read An Overview of the Federal Regulations and Rulemaking Process, Congressional Research Service (January 7, 2019).
- Note: the “one-in, two-out” rule mentioned in this reading was rescinded by President Biden on his first day in office.
- Niara Savage, ‘It Was a Slap In the Face’: Black Couple’s Home Valuation Increased by 50 Percent After White Friend Posed as Homeowner During the Inspection, Atl. Black Star (Feb. 16, 2021)
Other legal responses:
- Tim Simonite, New York City Proposes Regulating Algorithms Used in Hiring, Ars Technica (Jan. 10, 2021)
- Gabriel Geiger, Court Rules Deliveroo Used “Discriminatory” Algorithm, Vice (Jan. 5, 2021)
- Alice Xiang, Reconciling Legal and Technical Approaches to Algorithmic Bias, 88 Tenn. L. Rev. (2020)
Class 7: Using and Regulating Artificial Intelligence (March 4)
Required
Advanced techniques, advanced concerns
- Ram Shankar Siva Kumar, Jeffrey Snover, David O’Brien, Kendra Albert, & Salomé Viljoen, Failure Modes in Machine Learning, Microsoft (Nov. 11, 2019) – read all
- Kendra Albert, Jonathon Penney, Bruce Schneier & Ram Shankar Siva Kumar, Politics of Adversarial Machine Learning, Eighth International Conference on Learning Representations (2020)
The limits of AI
- Aravind Narayanan, How to Recognize AI Snake Oil (2019) – read all
Responses to bias concerns
- Manish Raghavan, Solon Barocas, Jon Kleinberg, and Karen Levy, Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices, FAT* (2020) – read section 5 only
- Alfred Ng, Can Auditing Eliminate Bias from Algorithms?, The Markup (Feb. 23, 2021)
- Ifeoma Ajunwa, The Paradox of Automation As Anti-Bias Intervention, 41 Cardozo, L. Rev. 1671 (2020) – read section IV(B) only
Effective AI Regulation
- Adriano Koshiyama et al., Toward Algorithm Auditing (2021) – read Section 4 only
- Emily Bender, Timnit Gebru, Angelina McMillan-Major, & Margaret Mitchell, On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? FAccT (2021) – read sections 7 and 8 only
- Rebecca Crootof, Artificial Intelligence Research Needs Responsible Publication Norms, Lawfare (Oct. 24, 2019) – read all
Other
Adversarial apps
- https://adversarial.io/
- https://github.com/BillDietrich/fake_contacts
Other readings
- Ryan Calo, Artificial Intelligence Policy: A Primer and Roadmap (2017)
- Timnit Gebru, Jamie Morgenstern, Brianna Vecchione, Jennifer Wortman Vaughn, Hanna Wallach, Hal Daumé III, and Kate Crawford, Datasheets for Datasets, FAT-ML (2018)
- Ram Shankar Siva Kumar, David R. O’Brien, Kendra Albert, Salomé Viljoen, Law and Adversarial Machine Learning (2018)
- Preparing for the Future of Artificial Intelligence, Executive Office of the President, National Science and Technology Council (Oct. 2016)
- Javier Sánchez-Monedero, Lina Denick, & Lilian Edwards, What Does It Mean to “Solve” the Problem of Discrimination in Hiring? (2019)
- Bo Cowgill, Fabrizio Dell’Acqua, Samuel Deng, Daniel Hsu, Nakul Verma, and Augustin Chaintreau, Bised Programmers? Or Biased Data? A Field Experiment in Operationalizing AI Ethics, 34th Conf. on Neural Info. Processing Sys. (2020)
- Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, Cass R. Sunstein, Discrminiation in the Age of Algorithms, 10 J. Legal Analysis 113 (2019)
- Shakir Mohamed, Marie-Therese Png & William Isaac, Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence, 33 Philosophy & Technology 659 (2020)
Class 8: Privacy vs. Security vs. Encryption (March 18)
Required
Legal Concepts of Privacy
- Daniel Solove, Understanding Privacy (2008) – read chapter entitled “Privacy: A Concept in Disarray”
Encryption
- Mike Rosulek, The Joy of Cryptography, Ch 1 (2021) — Everybody should read Section 1.1. Non-law students should also read Section 1.2.
Limiting Encryption
- D. Victoria Baranetsky, Encryption and the Press Clause, 6 N.Y.U. J. I.P. & Ent. 179 (2017) – read pp. 208–218 (start at beginning of part III(A), stop at “In the wake of Bernstein, however….”)
- Rianna Pfefferkorn, DOJ Plans to Strike Against Encryption While the Techlash Iron Is Hot, Stanford CIS (Feb. 25, 2020) – read all
Empowering Encryption
- Complaint, In re Henry Schein Practice Solutions (FTC May 23, 2016) – read excerpt
- If you’re curious you can view the case’s settlement order and other materials here.
- FTC v. Wyndham Worldwide Corp., 799 F.3d 236 (3d Cir. 2015) – read excerpt
- William McGeveran, The Duty of Data Security, 103 Minn. L. Rev. 1135 (2019) – read introduction and section II(C)
Optional
Concepts of Privacy
- Danielle Keats Citron & Daniel Solove, Privacy Harms, B.U. L. Rev. (forthcoming 2021)
Non-technical intro to encryption:
- Kendra Albert, Computer Security Tools and Concepts for Lawyers, 20 Green Bag 2d 127 (2017)
- Orin S. Kerr & Bruce Schneier, Encryption Workarounds, 106 Geo. L. J. 989 (2018) – Section 1
- Mike Rosulek, The Joy of Cryptography (2021)
- Preface — Why is Cryptography a Difficult Subject
- Chapter 1.1 (and 1.2 if you can manage)
Technical introduction to cryptography
- Mike Rosulek, The Joy of Cryptography (2021)
- Rafael Pass & abhi shelat, A Course in Cryptography (2010)
Encryption and surveillance:
- Orin S. Kerr & Bruce Schneier, Encryption Workarounds, 106 Geo. L. J. 989 (2018)
- Harold Abelson et al., Keys Under Doormats: Mandating Insecurity by Requiring Government Access to All Data and Communications (2015)
- Aloni Cohen & Sunoo Park, Compelled Decryption and the Fifth Amendment: Exploring the Technical Boundaries, 32 Harv. J. L. & Tech. 169 (2018)
The policy dimensions of cryptography
- Seny Kamara, Crypto For the People (2020) – watch 4:40 to 21:14
- Scott Skinner-Thompson, Privacy at the Margins (2020)
- James Mickens, There Are No Secrets (2017)
- Philip Rogaway, The Moral Character of Cryptographic Work (2015)
- Bruce Shneier, The Value of Encryption (2016)
Class 9: Differential Privacy and the Census (March 25)
Required
Intro to Differential Privacy
- Wood et al. “Differential Privacy: A Primer for a Non-Technical Audience.” Read Executive Summary and Section II (pp 211-214, 221-225). [link]
- Dwork and Roth, “Algorithmic Foundations of Differential Privacy.” Read Chapter 1 (pp 5-10). [link]
Intro to the Census
- 13 U.S.C. § 9 – read all
- Baldrige v. Shapiro, 455 U.S. 345 (1982) – read excerpt
- 13 U.S.C. § 141 – read subsections (b)–(c)
Differential Privacy and the Census
- John M. Abowd, “The U.S. Census Bureau Tries to be a Good Data Steward in the 21st Century.” 8:16-22:00. [link]
- danah boyd, Differential Privacy in the 2020 Decennial Census and Implications for Available Data Products (July 8, 2019) – read from pg. 6–9 (“The Production of Census Data Products”)
- Differential Privacy for Census Data Explained, Nat’l Conference of State Legislatures – read “Current Status” and “Questions Differential Privacy Has Prompted for Redistricters”
- “Implications of Differential Privacy for Census Bureau DAta and Research” [link] Read from “Differences between Differential Privacy and Census Law” (pp 8) through “Virtually all Census Bureau…” (pp 10)
A Legal Challenge to Differential Privacy
- Complaint, Alabama v. Dep’t of Commerce, No. 21-cv-211 (M.D. Ala. filed March 10, 2021) – read the following:
- ¶¶ 1–11 (Introduction);
- ¶¶ 50–63 (Section I(C));
- ¶¶ 79–91 (intro to Section III);
- ¶¶ 103–121 (Section III(B));
- ¶¶ 122–132 (Section III(C));
- ¶¶ 133–158 (Section IV(A))
Class 10: Conducting Analysis over Secret Data (April 1)
Required
Intro to MPC
- What Is Secure Multiparty Computation?
- Yehuda Lindell, A Primer in Secure Multiparty Computation, – read pp 2–6 (stopping at “Two-Party Secure Computation of RSA”). For the purposes of this class, you can safely think of ‘⊕‘ as meaning ‘+‘
Case Studies
- Boston Women’s Workforce Council: Qin et al, From Usability to Secure Computing and Back Again
- Read Sections 1–4 of the PDF
- Watch 3:35–5:45 of the video
- Google and Mastercard
- Benny Pinkas, Private Set Intersection, — watch 0:30–4:30
- Google and Mastercard Cut a Secret Ad Deal to Track Retail Sales, Bloomberg News (PDF uploaded to Microsoft Teams)
“Use” vs. “Disclosure” in privacy regulation
- Neil Richards, Reconciling Data Privacy and the First Amendment, 52 U.C.L.A. L. Rev. 1149 (2005) – read pp. 1190–97 (all of Section III(B) and the start of section III(C), stopping at “The contemporary First Amendment critique…”)
Optional
- A Pragmatic Introduction to Secure Multi-Party Computation
- Lawmakers Move to Push Forward Privacy-Enhancing Tech
- Secure Multiparty Computation Goes Live
- Student Right to Know Before You Go Act of 2019 and It’s time to tell students what they need to know
Class 11: Vote by Paper, Vote by Mail, Vote by Smartphone (April 8)
Required
The Evolution of Voter Security
- Douglas Jones, A Brief Illustrated History of Voting (2003) – read all
- Svetlana Lowry & Poorvi Vora, Desirable Properties in Voting Systems, NIST (Sept. 25, 2009) – read sections 1 and 2
- Sunoo Park, Michael Specter, Neha Narula, & Ronald Rivest, Going from Bad to Worse: From Internet Voting to Blockchain Voting (2020) – read sections 1 and 2
- Voting Laws Roundup: March 2021, Brennan Ctr. for Justice (April 1, 2021) – read introduction and “Alarming Trends,” skim remaining sections
Ken Thomson, Reflections on Trusting Trust — read to understand the overall ideas even if you don’t understand the code snippets
Software in elections
- Ron Rivest, John Wack On the notion of “software independence” in voting systems — Read Sections 1-3
- J. Alex Halderman, Hacking Democracy — Watch the whole thing
US Vote Foundation, The Future of Voting — Read Section 4.2
Optional
Michael A. Specter, James Koppel, Daniel Weitzner, The Ballot is Busted Before the Blockchain: A Security Analysis of Voatz, the First Internet Voting Application Used in U.S. Federal Elections, USENIX Security (2020)
Jill Lapore, Rock, Paper, Scissors, New Yorker (Oct. 6, 2008)
Class 12: How Do We Trust the Vote (April 15)
Required
Trusting Elections
- Ethan Zuckerman, Mistrust (2021) – read section on “Why Do We Bother to Vote?,” circulated separately (4 pages)
- Confidence in Elections and the Acceptance of Results, Kofi Annan Foundation (2016) – read section III only (8 pages)
- Eli Sherman, As Many States Grapple with Election Security, Massachusetts Remains Confident, Government Technology (Aug. 20, 2018) (5 pages)
- Edgar B. Herwick III, What Poll Watchers Can and Cannot do in Massachusetts, GBH (Oct. 15, 2020) (6 pages)
Risk Limiting Audits
- Mark Lindeman and Philip B. Stark, A Gentle Introduction to Risk-limiting Audits — Read all (7 pages)
- El Dorado County 2020 Risk Limiting Audit Summary — Read all (2 pages)
- El Dorado County 2020 Risk Limiting Audit Data — Download spreadsheet and answer for yourself the following questions:
- What risk limit was used?
- How many ballots were inspected in total?
Zero-Knowledge Proofs
- Kenneth A. Bamberger, Ran Canetti, Shafi Goldwasser, Rebecca Wexler, & Evan Zimmerman, Verification Dilemmas, Law, and The Promise of Zero-Knowledge Proofs, Berkeley Tech. L. J. (forthcoming 2022) — Read Introduction and Section 2 (15 pages)
Optional
Trust and Media
- Yochai Benkler, Casey Tilton, Bruce Etling, Hal Roberts, Justin Clark, Robert Faris, Jonas Kalser, and Carolyn Schmitt, Mail-In Voter Fraud: Anatomy of a Disinformation Campaign (2020)
Risk limiting audits
- Philip B. Stark, Super-Simple Simultaneous Single-Ballot Risk-Limiting Audits
- Ron Rivest, Bayesian Tabulation Audits Explained and Extended
- CA Risk Limiting Audits page
- MA 2020 Post-Election Audit Report Narrative — Not a risk-limiting audit
Legal Challenges to Elections
- Donald J. Trump for President Inc. v. Boockvar, (M.D. Pa. 2020)
Class 13: Law and Algorithms (April 22)
Required
Actors, Actions, and Understanding Behavior:
- Maranke Wieringa, What to Account for When Accounting for Algorithms, FAT* (2020) – read Section 2 and Subsections 3.2 and 3.4
- Margaret Mitchell, Simone Wu, Andrew Saldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, & Timnit Gebru, Model Cards for Model Reporting, FAT* (2019) – read Section 2, Section 3, and the introduction to Section 4 through the end of Section 4.2, as well as the example model cards
Impact Assessment Frameworks:
- Margot E. Kaminski & Gianclaudio Malgieri, Algorithmic Impact Assessments Under the GDPR: Producing Multi-Layered Explanations, Int’l Data Privacy Law (2020) – read Introduction (pp. 1–3); A Model AIA: Towards Multi-Layered Explanations (pp. 10–14)
Changes in Governance:
- Andrew Tutt, An FDA for Algorithms, 69 Admin. L. Rev. 83 (2017) – read Section II only
Changes in Design:
- Sasha Costanza-Chock, Design Justice (2020) – read the following sections from Chapter 2 (“Design Practices: Nothing About Us Without Us”)
Optional
- Dillion Reisman, Jason Schultz, Kate Crawford, & Meredith Whittaker, Algorithmic Impact Assessments: A Practical Framework for Public Agency Accountability, AI Now (2018)
- Joshua Kroll, Joanna Huey, Solon Barocas, Edward W. Felten, Joel R. Reidenberg, David G. Robinson, and Harlan Yu, Accountable Algorithms, 165 U. Penn. L. Rev. 633 (2017)
- Sasha Costanza-Chock, Maya Wagoner, Berhan Taye, Caroline Rivas, Chris Schweidler, Georgia Bullen, & the T4SJ Project, #MoreThanCode: Practitioners Reimagine the Landscape of Technology for Justice and Equity (2018)
- Jessica Fjeld, Nele Achten, Hannah Hilligoss, Adam Christopher Nagy, and Madhulika Srikumar, Principled Artificial Intelligence: Mapping Consensus in Ethical and Rights-Based Approaches in AI (2020)
- Jessie Finocchiaro, Roland Maio, Faidra Monachou, Gourab K. Patro, Manis Raghaven, Ana-Andreea Stoica, Stratis Tsirtis, Bridging Machine Learning and Mechanism Design towards Algorithmic Fairness, FAccT (2021)
- Margo Kaminski, The Right to Explanation, Explained, 34 Berkeley Tech. L.J. 189 (2018)
- Alicia Solow-Niederman, Administering Artificial Intelligence, 93 S. Cal. L. Rev. 633 (2020)