Data Society ~ Algorithmic Accountability Primer.pdf

WHAT IS AN ALGORITHM

• “An algorithm is a set of instructions for how a computer should accomplish a par-ticular task.”
  
    ◦ “often compared to recipes, which take a specific set of ingredients and transform them [...]into a 
      predictable output.”
    ◦ algorithms can be exceptionally complex

• Algorithmic accountability

    ◦ “Critically, there are few consumer or civil rights protections that limit the types of data used to build 
       data profiles or that require the auditing of algorithmic decision-making.”
    ◦ → Standards and enforcement for fairness, accountability, and transparency
	needed as algorithmic decision-making can result in discriminatory and inequitable outcomes

HOW ARE ALGORITHMS USED TO MAKE DECISIONS?

Apart from the entertainment and consumer sector they are used to:

• Sort résumés for job applications; • Allocate social services; • Decide who sees advertisements for open positions, housing, and products; • Decide who should be promoted or fired; • Estimate a person’s risk of committing crimes or the length of a prison term; • Assess and allocate insurance and benefits; • Obtain and determine credit; and • Rank and curate news and information in search engines

Problems:

• While they appear unbiased they are not
• Algorithms = black boxes and not neutral

**“An algorithm is an “'opinion embedded in mathematics'”** (O’Neil, Cathy. 2016. Weapons of Math Destruction. 
     Crown.

• They can “replicate known inequalities”, [reenforcing] systematical exclusion of minorities and impede civil 
  rights, lead to discrimination
    ◦ tied to the distribution of goods and services in society (education, housing, and other human and civil 
      rights)
• This redlining is actually illegal [USA], but elusive because in its digital form
EXAMPLE: RACIAL BIAS IN ALGORITHMS OF INCARCERATION

• “COMPAS”: algorithmic system for justice system recommendation
    ◦ meant to “help balance protecting public safety while also eliminating the possible bias of human judges”
    ◦ inspite of internal and external validation that yielded comparable results for defendants of different 
      races, an investigation of ProPublica found “clear evidence of algorithmic bias”
    ◦ problematic and still not solved: “there are no standard definitions for
    ◦ algorithmic bias, and there is no mechanism for holding stakeholders accountable”: Because of difficulties 
      to define what racial fairness means nobody is accountable (see text for better understanding) + standard 
      definitions for algorithmic bias don't exist

COMPLICATIONS WITH ALGORITHMIC SYSTEMS

• algorithms “can create surprising new forms of risk, bias, and harm”
• “complications in assessing fairness” result from stakeholders intentionally keeping them opaque
• these conditions have to be changed to enable auditing algorithms or enforcing regulation

_FAIRNESS AND BIAS

• algorithms can codify existing bias or introduce new bias, although they're often intended to correct human bias
    ◦ “bias can exist in multiple places within one algorithm.”
    ◦ algorithms “can take on unintended values that compete with designed values”
    ◦ training data can result in the algorithm feeding back what “it has learned” into the real life decision 
      process (e. g.: black people having been arrested more often historically, leads to a prediction of 
      recidivism mostly by black people) 
• algorithms can be morally rigid and become outdated quickly (values are transferred from one context to another, 
  reinforcement of values they were created with)

_OPACITY AND TRANSPARENCY

• auditing can sometimes be impossible as alg. Are kept closed off and because of
    ◦ intentional secrecy
    ◦ inadequate education of auditors
    ◦ overwhelming complexity
• → transparency is needed to judge whether an alg. Is reliable, fair and “does
  what it says it does”
• in contexts prone to hacking different layers of transparency are needed (e. g. twitter)

_REPURPOSING DATA AND REPURPOSING ALGORITHMS

• alg. Are expensive and difficult to build
• using alg. In a “foreign” context is problematic as 
  “standards that were set and ethical issues that were dealt with in an algorithm’s original context may be problems in a new application.” (e. g. assing police  to hotspots using earthquake alg.)
• crime data is racially biased, therefore bad as traininig data for an alg.
• Datasets can be misinterpreted or mismatch e. g. credit history and hiring decsions: “connections between credit 
  history and work capability are dubious at best.”

_LACK OF STANDARDS FOR AUDITING

• like in the financial sector since the 70s, independent auditing could be used to detect bias in algorithms
    ◦ underutilized because because of lack of industry standards and guidelines for assessing social impact
• two examples for standards
    ◦ 1) Automated decision-making has to be held to the same standards as equivalent human decision-making. 
         Applied at every stage of creation → responsibility clearly in industry's hands (Association for 
         Computing Machinery)
    ◦ 2) social impact statements to accompany the sale and deployment of algorithmic products (coalition of 
         industry and university researchers)
• other proposals
    ◦ independent audits of platforms and internet companies
    ◦ data public trust e. g. ask Facebook to share anonymized datasets for public good”
    ◦ + data code of ethics for data protections and limiting digital profiling (both “Data for Black lives”)
• “negative example”
    ◦ FB's reaction to Cambridge Analytia: deleting pages and limiting access to data, thus shutting off outside 
      review

POWER AND CONTROL

• a primary decision an alg. Takes is which dataset is relevant to other data points
    ◦ this not neutral but shaped by political agenda whether implicitly or explicitly
• exrtemely important when alg. Have a “gatekeeping role”:
    ◦ alg. replicate social values & embed them into systems
      (...“creating new standards and expectations for what is important in a given context.”)
• existing laws are circumvented by missing protection against data brokering
    ◦ thus “data points act as proxies” for these usually protected categories
    ◦ → assemblage into profiles → profiles sold to 3rd parties → technological redlining

TRUST AND EXPERTISE

• regarding trustworthiness algorithms are held to higher standards than humans
    ◦ as mentioned above alg. “are as capable of bias as humans, as they are embedded with subjective values.
• Usually who is endowed with trust should also be liable for errors
    ◦ as mentioned above where liability lies is often difficult in relation to algorithms

WHAT IS ALGORITHMIC ACCOUNTABILITY? (Whole paragraph is worth reading)

• assignment of responsibility
    ◦ how is an alg created?
    ◦ what is its impact on society?
• Alg. Are products of human and machine learning
    ◦ alg. do the processing
    ◦ but humans deliver the input, design and thus outcome

• **“Critically, algorithms do not make mistakes, humans do.”**
• creating the possibilities of “assigning responsibility is critical for quickly remediating discrimination”
    ◦ PLUS restoring the public's faith “that oversight is in place”
• accountability must be grounded in enforcable policies and standardized assesments

_AUDITUNG BY JOURNALISTS

• data journalism, reverse engineering, pairing inputs with outputs, individual or peer-sourced research

_ENFORCEMENT AND REGULATION

• governance of algorithms happens on an ad hoc basis across sectors
• sometimes existing regulations are reinterpreted to apply to technological systems and guide behavior
• algorithmic systems bring up new issues not before properly covered by the logic of existing precedents
• ultimately appropriate governance structures are needed e. g. 
    ◦ market and design solutions
    ◦ industry self regulation in favor of
    ◦  public interest
    ◦ state intervention (taxes, subsidies for certain kinds of algorithmic behavior)

• Much of the processes for obtaining data, aggregating it, making it into digital profiles, and applying it to 
  individuals are corporate trade secrets
    ◦ no control of citizens and regulators
    ◦ no body in place for proper oversight

“While law has always lagged behind technology, in this instance technology has become de facto law affecting the lives of millions— [...]”