Big Data, Surveillance and Race | Wisconsin Public Television

Big Data, Surveillance and Race

Big Data, Surveillance and Race

Record date: Feb 22, 2018

Cecelia Klingele, Associate Professor at the University of Wisconsin Law School, Simon A. Cole, Professor of Criminology, Law and Society at Cornell University, and Margaret Hu, Associate Professor at Washington and Lee School of Law, discuss ways gathered information is used.

University Place Campus: 

University Place Subjects: 

Episode Transcript

- Thank you so much everybody,

for joining us tonight

to this Science and the

Public, for the Holtz Center.

First, my name is Samer Alatout,

and I'm the Director

of the Holtz Center

for Science and

Technology Studies.

Now I will introduce--

I'm not going to

introduce the speakers,

but I will introduce the

introducer of the speakers.


My colleague,

Cecelia Klingele, and

who is a professor in

the law school at UW.

After receiving her JD from the

University of Wisconsin

Law School, in 2005,

Cecelia Klingele

has a great kind of

profile, right?

Served as a law clerk

to Chief Judge Barbara Crabb

of the United States

District Court,

for the Western

District of Wisconsin,

Judge Susan Black, of the

United States Court of Appeals

for the Eleventh Circuit,

and Associate Justice,

and this was impressive,

really, to me, although

every, all the judges are,

you know, with respect

to all of them,

but Associate Justice

John Paul Stevens of

the US Supreme Court.

She returned to the University

of Wisconsin in 2009

as a visiting

assistant professor,

and has been on the

permanent faculty,

a permanent faculty

member in 2011.

Professor Klingele's

academic research focuses on

criminal justice administration,

with an emphasis on

community supervision of

those on conditional release.

She served as

Associate Reporter for

the American Law

Institute's Model Panel Code,

sentencing revision,

sentencing revision,

External Co-Director

of the University of Minnesota

Robina Institute's Sentencing

Law and Policy Program.

And past co-chair of

the Academic Committee

of the American

Bar Association's

Criminal Justice Section.

So with that, please welcome

Cecelia and the rest

of the panel for,

for this, and enjoy the talks.


- Thank you, Samer,

and you can only guess

with that nice of an intro

for me, how fantastic

our actual panelists

are going to be tonight.

It is my pleasure to introduce

them, and I will actually

do that first,

and then make a few opening

remarks about the theme

of tonight's presentation.

So first, Simon Cole

joins us from the

University of

California at Irvine,

and where he's professor of

Criminology Law and Society,

and serves as Director

of the Newkirk Center

for Science and Society.

Professor Cole

specializes in historical

and sociological study

of the interaction

between science, technology,

law, and criminal justice.

He is the author of

Suspect Identities:

A History of Fingerprinting

and Criminal Identification,

which was awarded the

2003 Rachel Carson Prize

for the Society, by the

Society for Social Science,

Social Studies of Science.

He is co-author

of Truth Machine:

The Contentious History

of DNA Fingerprinting,

as well, and has spoken

widely on the subject of


scientific evidence,

and science in the law.

He's also consulted as

an expert in the field.

He's written for many general

interest publications,

including the New York Times,

and the Wall Street Journal,

and currently focuses on the

sociology of forensic science

and the development of criminal

identification databases

and biometric technologies.

His teaching interests focus on

forensic science and society,

surveillance and society,

miscarriages of justice,

and the death penalty.

Our second panelist

is Margaret Hu,

who is Associate

Professor of Law

at Washington and Lee

University School of Law.

Her research interests

include the intersection

of immigration policy,

national security,

cyber surveillance,

and civil rights.

Previously, she served

as senior policy advisor

for the White House

Initiative on Asian Americans and Pacific Islanders,

and also served as

special policy counsel in the Office of Special Counsel

for Immigration-Related

Unfair Employment Practices,

Civil Rights Division, in

the US Department of Justice.

As Special Policy Counsel, she

managed a team of attorneys

and investigators in

the enforcement of


provisions of the

Immigration and Nationality Act,

and was responsible for federal

immigration policy review

and for coordination.

She is also the author

of a forthcoming book,

The Big Data Constitution:

Constitutional Reform in

the Cybersurveillance State.

So we have an action-packed

evening in front of us.

I am honored to kick it off,

though I will keep my remarks

brief, so that we get more time

from our esteemed guests.

It was a little

ironic, I thought,

that I was asked tonight,

to speak on the subject of

Big Data in criminal justice,

and the ways in which

it might affect

racial disparities.

What I find ironic about it,

is that most of the time,

when we talk about data

in criminal justice, we're

not lamenting that there's

too much of it, but

that there's not enough.

For those who work in the field, you already know what I mean,

and for those outside of it,

let me give you some sense.

The collection of data

in criminal justice

in the United States is

complicated by many factors.

Among them are the fragmentation

of criminal justice agencies.

Every police agency,

every county clerk

of courts office,

every district

attorney's office,

every iteration of

public defenders,

and they exist in many

different iterations

around the country,

every correctional

agency and jail

maintains separate

databases in this country.

There's no regularized

norm around what data

is collected, how it's reported,

whether it's audited,

how it's reviewed or stored,

backed up, maintained,

made accessible.

Any researcher who has

filed freedom of information

request acts has often been

greeted with the response

from criminal justice officials,

"Well, I don't think

we can get you that.

"We'd have to go through

every single file

"and pull it out, cause it's written by hand, somewhere!

"And probably sorted in archive if it's been maintained."

That fragmentation

of data, and the lack

of adequate controls,

usually leads us, again

to lament the inability

to gather aggregate data

about the functioning

of our criminal

justice agencies,

and the way they are either

positively or negatively

affecting our communities,

as measured by

any number of different metrics.

While we have some

ways of gathering some

aggregate statistics,

mostly through the

Department of Justice,

and the FBI's Crime Statistics,

there are many flaws

to those, as well,

and good reason to

question the reliability

of at least some of that data.

So, again it's funny

that we're here tonight,

to talk about the opposite

problem, in many ways.

And that's the

problem created by

drawing on Big Data, the

compilation of large amounts

of information, gathered

about individuals

not necessarily aggregated

across police systems.

More commonly

aggregated by private

data collection

agencies, and then sold

to law enforcement or

other interested parties,

or generated based on

criminal records information

that we might have in

state court databases.

So if criminal justice

system actors are so reticent

to collect data, and be

held accountable for it,

what explains the draw,

and I would say,

the increasing trend

to rely on it, when it

comes to things like

risk prediction algorithms,

or hot spot, or predictive

policing algorithms?

I think that's best explained

by the configuration

of the criminal

justice system, itself,

and the laws that govern it.

Again, for those of you who

are familiar with the system,

you're aware that although

there is of course law,


statutory, administrative,

that governs the behavior

of system actors,

from police, through

prosecutors, through judges

and correctional agents.

In fact, many of the

day-to-day decisions

made by those actors are

not dictated by any statute,

but are rather governed by a principle we call discretion.

That is the

legally-authorized authority

of system actors

to select between

multiple, equally-permissible

legal options.

For the beat officer, that

means that when you see

the child spray

painting the wall,

you can either tell

him to go home,

or you can give him a ticket.

Or you could arrest him,

you could refer him

for charges, or not.

You could take him to

his parents, right?

Any number of

choices, all of them

equally legally permissible.

The same is true, not

only for arrest decisions,

but for the decisions

about whom to surveil.

About what charges to leverage

against an individual defendant.

About whether to

set bail, and if so,

what the amount to be, or

the conditions of release.

What sentence to impose

for an individual

who's been convicted of a crime,

and what kind of

supervision or custody

to give to those who are

already being punished

for a criminal sentence.

Those are hard decisions.

They don't have clear,

right or wrong answers.

They require the balancing

of many difficult and

complicated moral,

sociological, and other factors,

outside of the law,

that often leaves

police officers and sentencing

judges awake at night,

wondering if they've

done the right thing.

The idea that we

could outsource those

really hard decisions to a math

problem is super appealing.

Math sounds so objective,

and fair, and clear-cut,

and the desire to assuage

our guilty consciences,

or at least our anxiety about getting the right answer

in a difficult and complex

situation, is such that I think

data feels concrete,

and safe, and reassuring

to many in the criminal

justice system.

Now I'm oversimplifying.

There are also those who

are inherently skeptical.

Particularly in

criminal justice,

and particularly when

we talk about the law,

and the legal process.

Because, as all of us know,

if lawyers and judges were

good at math, we'd be doctors.


That's for you Jerry.

[audience chuckling]

Of course, there are some in law

with mathematical talents

that far exceed my own,

but the reality is

there is something about

feeling safe that

math is certain,

and not quite

understanding the magic mix

that goes into

the data-crunching

behind many of these numbers

that makes it

particularly appealing

in a complex and

difficult enterprise

like the administration

of criminal justice.

But there are, of

course, dangers

of over-relying on data.

And I do mean over-relying,

because, certainly,

there are many positive

things to be found

in numbers, that check

our intuitive gut sense

of what's happening

on the ground,

or who is being

affected in what ways

by the decisions that we make.

Data plays an important

role and in many ways,

we need to get better

about collecting it.

But the ways we use it matter,

and there are several dangers

that I will throw out, and then

turn the stage over to those

who can unpack it for us

much better than I can.

The first is a failure

to often recognize

that the data themselves

are often flawed.

When we rely on

information in systems,

whether that information is used

as it is first presented,

or whether it's first

put through a complex

equation to generate

a new number or

prediction for us,

the quality of

that data matters.

And the reality is that

turning something into a number

doesn't change inequities

in the collection of that data.

Or problems in the

quality that exists.

Take for example, predictions

about criminal recidivism.

Those are ones with which

I'm particularly familiar,

because they often affect

sentencing decisions

and correctional supervision.

In those cases, we

strive, with algorithms,

to generate

aggregate predictions

of about what individuals

with similar backgrounds

to a particular defendant

we believe are likely

to do in the future

when it comes to re-offense.

But, there are a lot

of problems with that.

First of all, and

most simplistically,

those data don't tell

us anything about

actual human behavior.

All they tell us about

is human behavior

that's gone awry, and been

detected by law enforcement.

The reality is that the

universe of criminal behavior

is much, much,

much, much broader,

than that of detected

criminal behavior.

And as a result, depending

on where you live, and

how old you are, and whether

you're a guy or a girl,

you are more, or less likely,

to be detected

committing crimes.

And if we're relying

on information about

crime detection,

that'll tell us about

the likelihood perhaps,

of someone like you being

detected again in the future,

but it doesn't tell

us anything about

the actual behavior of you

or the general population.

Or very little.

Second, I think there's a

danger of misunderstanding

the limitations of

the data themselves.

Again, in the risk

prediction context,

often, risk of future offense

in the sentencing arena,

is predicted out as

whether a person has a low,

a medium, or a high

risk of recidivism.

And usually when I poll

lawyers and judges,

they tell me they

can't tell me what

the exact percentage is, that

means low, medium, or high,

but they're sure there is one.

Not true.

In fact, most of these

data are not absolute.

They're comparative.

They're comparing populations

against each other

for the frequency of

predicted re-offense.

In other words, if you live in a really, really dangerous place,

it may be [laughs] low-risk

people are actually

higher risk than they might

be in a different population.

In so far then, as what

we are trying to do

in the criminal justice

system, is not only,

protect people from

future risk of harm,

but maybe more importantly,

hold people accountable

for actual decisions that

they have made in the past

to harm others.

These data may not tell us

all that we think is

important about either

moral culpability, propensity

to offend, or to change,

character, the possibility

of future growth.

And if those

are the things that matter

to us at sentencing,

then, bad news, guys.

We can't outsource it.

And so I hope

today's conversation

will help us, first of all,

better understand

all those numbers,

cause I don't understand them,

and maybe some of you

are with me on that.

But also better understand

what the limitations

and the possibilities

of this information,

ways in which it may be

having an inequitable effect

on some members

of our community,

and ways that

hopefully, we can use it

to make our system

better, and more fair.

So with that I'm

happy to turn it over

to Simon and Margaret.


- Thank you so much

for the wonderful

privilege to join

you this evening,

for this very important

conversation, and

very grateful to Lynn and

Samer at the Holtz Center,

and to Cecelia for that

very generous introduction,

and for contextualizing

these issues

in such a brilliant

way, thank you so much.

So what I wanted to do today

is focus on Big Data

in discrimination.

And, really, feel very

fortunate to be here

because I just

published an article

with the Wisconsin Law Review



where I talk about

the conflation

of crime, immigration,

and counter-terrorism,

or national security rationales,

through programs such

as the Muslim Ban,

and extreme vetting.

I also have another article

that I just published on

the Muslim Ban and

extreme vetting

called Algorithmic Jim Crow,

that was published in

Fordham Law Review.

So I wanted to start today,

by talking about extreme vetting

as a way to help

us wrap our minds

around modern governance

that involves Big Data,

and the rationales

that support it.

Then I wanna go into a

little bit more detail

about mass surveillance,


Big Data intelligence

gathering methods,

Small Data surveillance

that we used to have

in the Small Data world, versus

the Big Data,

cyber-surveillance schools

that we now have

at our disposal.

And then, if there's time, get

to the Snowden Disclosures.

So, extreme vetting, what is it?

Extreme vetting is a way

to understand the modern

landscape that we have, with

the ubiquity of social media,

and online information.

So back in December 2015,

then-presidential candidate

Donald Trump published a

Statement on Preventing

Muslim Immigration on

his campaign website.

The statement called for a

total and complete shutdown

of Muslims entering

the United States

until our country's


can figure out what is going on.

Then, shortly before the

election, he announced

a proposal for what he

called extreme vetting

of immigrants and refugees.

He later explained, the

Muslim ban is something that

in some form has morphed

into an extreme vetting

from certain areas in the world.

So the Muslim ban or

what is referred to as

the Travel Ban, and

extreme vetting,

should be understood as one

of the same, so what is it?

So you have under the

former administration,

the former Director

of the United States

Citizenship and

Immigration Services Office

of the Department of

Homeland Security,

explained that they

already started

some form of extreme vetting

during the Obama

administration, that

prospective refugees from

Syria and Iraq, since 2015,

had their Facebook, Twitter,

and Instagram accounts checked,

and then, the then Department of Homeland Security Secretary,

now Chief of Staff, John Kelly,

explained, in

Congressional testimony,

that extreme vetting would be

an accounting of what websites

that those refugee applicants

would be visiting,

what telephone contact

information they had

in their phones, to see

who they were talking to,

social media information,

including passwords.

So, further information about

extreme vetting was revealed

in a media report that the

Department of

Homeland Security held

what they called an

Industry Day in 2017,

in Arlington, Virginia.

It circulated a

document, at that time,

called the Extreme

Vetting Initiatives, and

a host of industry

representatives were there.

The document stated

that, right now,

it is difficult for the

government to assess threat

because the data is collected,

that is fragmented,

and it's very time-consuming

and labor-intensive

to make sense of it.

And so they were inviting

solicitations of proposals

to help do this

type of vetting tool

that would automate,

centralize, and streamline

vetting procedures,

while simultaneously

making determinations

of who would be

considered a security risk.

The system was purportedly

supposed to then,

predict the probability of

an individual becoming a

positive member of society,

as well as whether

or not they had

criminalistic or

terroristic tendencies.

The attendees

included for example,

IBM, Booz Allen Hamilton,

LexisNexis, and other companies,

and in the solicitation, it stated that the contractor

shall analyze and

apply techniques

to exploit publicly

available information

such as media, blogs, public

hearings, conferences,

academic websites,

social media websites

such as Twitter, Facebook,

LinkedIn, radio, television,

press, geospatial

sources, Internet sites,

specialized publications

with intent to extract

pertinent information

regarding targets,

including criminals, refugees,

non-immigrant violators,

and targeted national security

threats and their location.

So, basically,

what they said was

all publicly available

information of all

persons to be turned into

some type of automated tool,

some type of algorithm,

to assess risk and

predict terrorism,

so during a follow-up

Q and A session,

one of the attendees asked

whether or not this

was basically legal.

And it was an

anonymous question,

and the Department of

Homeland Security responded

that there's a prediction

that in the future

Congressional legislation

might address this,

but, basically, they'll

continue to do it

until they're told

that it's not legal.

And that's basically

where we are.

In the legal regime, in

our regulatory structure

for this type of

online collection,

or using social media,

or using sort of internet

digital footprints

that we leave behind,

that's publicly available, all

of it's considered fair game.

So, this leads us to what is

Big Data cyber-surveillance,

and what are some of the

justifications for it?

So in a Big Data

cyber-surveillance world

that we find ourselves

in, right now, the

analytical method is

you start with the data.

And what that means,

for the government,

is that the data

becomes suspicious.

It's not necessarily that

people are suspicious anymore.

And this also has led

to pre-crime ambitions.

So just as the Department

of Homeland Security

invited these corporations

to come up with tools

to assess risk and predict

terrorism and crime

before it occurs, that

has led to basically,

a minority report type of world.

And the way that this

was explained by the

CIA's Chief Technology

Officer was that,

since you cannot connect

dots that you don't have,

it drives us into mode

of fundamentally trying

to collect everything,

and hang onto it forever.

Forever being in

quotes, of course.

It is nearly within our

grasp, he explained,

to compute on all

human-generated information.

So, he said, "It's all

about the data, stupid."

Revolutionize Big

Data exploitation,

acquire, federate,

secure, and exploit.

Grow the haystack,

and magnify the

needle, so oftentimes

you hear in the

intelligence community,

that you need to have the

biggest haystack possible

in order to find the needle.

But what some experts

have explained

is this is a put the haystack

before the needle

approach to governance

and intelligence gathering.

And what this does

is it pre-supposes

that there is a

needle in the haystack

before you even know

that there's a needle.

And so, in a Small Data world,

you started with the needle.

You started with a person

who was a criminal,

a suspect, a crime,

and then you went vertical

into the information gathering.

Because, the

resource limitations

and the technological


required a vertical,

downward drilling of data

to support that

original question.

But in the Big Data

world, you actually

flip everything around, and

you start with the data.

You answer the question

after the fact.

You look at the data,

and then you say,

"From this data, can

I find a suspect?

"Can I find a crime?

Can I find a terroristic risk?"

And so everything has

been flipped upside down,

and this has led to

basically virtual

suspects or we can have a

digital avatar of ourselves

that becomes the representation

of the targeting of

the government action.

So Edward Snowden,

when he came forward

with the Snowden

Disclosure said,

"Does this method work, or

are we just drowning in hay?"

Are we building

more and more hay,

and do we have larger

and larger haystacks,

that doesn't necessarily

lead us to any

conclusive resolution.

And also another risk of this,

as explained to another

intelligence source,

is that everyone

now is a target.

So even though extreme

vetting, for example,

is presented on

just trying to find

a target who is potentially

a suspect, a criminal,

or a terrorist

from these refugee

vetting procedures,

you saw from that

extreme vetting day

that you Collect it All.

It is a Collect it All

method, it doesn't matter

what your citizenship

is, it doesn't matter

your immigration status.

The method requires

collecting it all,

and everyone who has any

digital communications

becomes a target.

And what this also means,

is that our digital devices

are the ancillary


of us, ourselves,

and so, Snowden explained

that part of the

key inquiry of the

intelligence community

is whether the

phone is suspicious.

Not necessarily whether or

not the person is suspicious.

So, we now have for example

in the Obama administration,

it was revealed that we

have signature strikes,

where the identity

of the individual

of the drone strike

is not known.

And you had one drone

strike operator explain

that please people get hung up

that there's a targeted list

of people on the kill list.

It's really like we're

targeting a cell phone.

We're not going after people,

we're going after the phones

in the hopes

that the person on the other end

of the missile is the bad guy.

So not knowing the identity,

but because you

have suspicious data

being generated by

a suspicious phone,

you're killing the phone,

you're killing the person

who's holding the phone.

And you also had

this, shortly after

the Snowden Disclosures,

the former Director of

the CIA and the NSA,

General Mike Hayden,

saying we kill people

based on metadata.

Metadata is data about data.

Time of a call, place of

a call, length of a call.

In a Small Data world,

it would be hard to

imagine killing people

based on that type of data,

but in a Big Data world,

everything has switched around,

and now, you have the

former Director of the CIA

and the NSA, saying

that metadata can form

the basis for actual,

lethal consequence.

This also has led to

pre-crime ambitions.

So, for example, the

Department of Homeland Security

has a test pilot

program that they call

the Future Attributes

Screening Technology Program.

In the news, and the

media, it's referred to

as a pre-crime program.

Under this test pilot program,

you have the Department of

Homeland Security saying

that they're going to

collect physiological cues

such as body and eye

movements, eye blink rate,

pupil dilation, body heat

changes, breathing patterns,

as well as linguistic cues,

such as voice pitch changes,

alterations of the

rhythm, and changes

in the intonations of

speech, to detect malintent.

Or, in order to detect this

threat risk assessment.

And the volunteers

of this program

were informed that the

consequences could range

from none to being

temporarily detained,

deportation, prison, and death.

And this is based on

things like linguistic cues

and physiological cues.

And so, something

else that we have seen

from the Snowden

Disclosures is that

these types of fragments

of data are leading to

more and more aggregated

information that

allows for the government

to believe that they are

engaging in the most efficacious

types of decision making that

they think is now possible

through these

technological advances.

And, you have some scholars,

such as Jack Balkin

at Yale Law School,

calling this the National

Surveillance State.

So he says, "It's one of the

most important developments

"in American Constitutionalism,

"that is the gradual


"of the United States into this

National Surveillance State

"that is the logical

successor of basically

"our administrative state

and our welfare state."

He said that it's just going

to become a way of governance,

and we're going to

integrate surveillance

into the way that

we just carry out

our day-to-day business.

You have other experts

such as Benjamin Wittes,

saying that the problem of the

justification of mass

surveillance is that,

the problem is not so much

a rule of law problem,

it's really a

technological problem.

That, in the United

States we might have

one of the most constrained

intelligence communities

in the world

but that the US intelligence

ambitions scale up to

our geopolitical ambitions.

And that we have the

most awesomely powerful


capacities of anyone,

on earth, so part

of the question now

is how is it possible

that we have, potentially,

the most constrained legal

apparatus imposed on us

for intelligence gathering,

and yet, still have

potentially the most

pervasive and invasive system.

So, I wanted to move to

now some of the legal

apparatus and justifications

that help support this.

So for example we have

the National Security

Presidential Directive

59 Homeland Security

Presidential Directive 24.

It was signed by

President Bush in 2008,

and it's called The

Biometrics for Identification

and Screening to Enhance

National Security.

So it directs the military

and the federal government

to collect, store, use,

analyze, and share biometrics,

and contextual information.

It doesn't define

contextual information,

but Microsoft

defined it this way.

All locations that you go,

all the purchases you ever make,

all your relationships, all

activity, all your health,

governmental, employer,

academic and financial records,

your web search history, your

calendars and appointments,

all your phone calls,

data, texts, email,

all peoples connected

to your social circle,

all your personal interests,

and all other personal data.

So this helps to

explain, for example,

this slide from the

Snowden Disclosures.

So here you had Sniff it All,

Know it All, Collect it All,

Process it All, Exploit it All, Partner it All.

That basically summarizes

the philosophy,

or the ethos, of the new

Big Data cyber-surveillance

systems that we

find ourselves in.

Here's a slide from one

program, Social Radar,

from the US Air Force,

and you see that it is

Collect it All.

It is any possible

potential communication,

any type of information,

that can be gathered, you

see military, religious,

political, economic health,

geography, demography,

econometrics, using

public sources, polls,

and surveillance,

and then, using that

to see into the future.

You also have this slide.

Total Information Awareness,

which was technically defunded

after 9/11, but this is also

another Collect it All program.

So you see that starts

with biometric data,

and then it goes to automated

virtual data repositories,

intelligence data,

transactional data,

including financial,

education, travel, medical,

veterinary, country

entry, place, event,

transportation, housing,

resources, government


So what I wanted

to point out was

that here is this

Hollerith machine.

This was found in

the Holocaust Museum.

And you had Edwin

Black, a journalist,

go into the Holocaust

Museum in Washington, DC,

and asking the archivist,

"What is this IBM machine

"doing here in the

Holocaust Museum?"

And after a multi-year study,

he produced his book,

The IBM and the Holocaust,

and what he found was that

through the punch card system,

the Third Reich was able

to very efficiently,

through this data collection,

and processing system,

execute the Final Solution.

And, I wanted to put two

posters side-by-side.

The one on this side, is the

Third Reich's Hollerith

machine poster,

and then the other one is

the United States poster

for the Total Information

Awareness Program.

And there's some uncomfortable

parallels between

the two posters.

So I wanna now talk about

Big Data intelligence,

and datafication.

And this brings us to

a little bit more

information just on

Big Data and how does it work?

So part of what's happening is

we're going through a

transformational moment

in our history, where,

as in 2000, 25% of all

stored information

was digitized.

By 2012, 98% of all

information was digitized.

So as far as the scalability

of it for example, imagine

a gigabyte as a full-length

feature film in digital form,

and a petabyte is one

million gigabytes,

an exabyte is one

billion gigabytes,

and a zettabyte is one

trillion gigabytes.

And that's the type

of stored data world

that we find ourselves in now.

So, the prediction is that

the data, the digital

data that we create

is expected to double every

two years, through 2020.

And by the year 2020,

we're going to have

5,200 gigabytes

of data for every

man, woman, and child on earth.

And, what some experts

such as Mayer-Schonenberger

and Cukier in their book,

Big Data: A Revolution

That Will Transform How

We Live, Work, and Think,

explained, is that this converts

all social and physical reality

into a digital format, and

that the transformation

of that digital data is

converting data into

new forms of value.

And so, I just

wanted to give a few

statistics on datafication.

So for example

Google's more than

100 petabytes in size,

experiences more than 7.2

billion page views per day.

Processes more than 24

petabytes of data per day,

a volume that's thousands

times more the quantity

of all printed material in

the US Library of Congress.

By 2012, Facebook had already

reached one billion users,

35% of all the digital

photos are stored on Facebook,

with 10 million

new photos uploaded per hour.

YouTube is more than

1000 petabytes in size,

over 72 hours of video uploaded

to YouTube every

minute, more than

four billion views per day.

800 million users upload over

an hour of video every second.

Twitter, more than 124

billion tweets per year,

grows at 200% per year, and

at 4500 tweets per second.

So we're also seeing

a datafication of us

human beings, through

the government.

The datafication

through biometrics,

for example, 70

million fingerprints

in the criminal master file

at the FBI, 34

million civil prints,

over 10 million DNA files,

and approximately 70

million digital photos

for the facial recognition

technology for the FBI.

The Department of

Homeland Security

is similarly moving towards

datafication through biometrics.

Approximately 300,000

fingerprint scans

collected every day,

130 million fingerprint

files on record,

and the Department of Homeland

Security has started to

gather the DNA of refugees, at at least two refugee sites.

You have the Department

of State also gathering

digital photos that can serve

facial recognition technology,

such as 200 million digital

photos through the passports,

and 75 million digital

photos through visas.

So bringing this back to

the Snowden Disclosures,

we can see that biometrics and

facial recognition technology

is forming the basis of new

forms of intelligence and

the way in which the

government is using

biometric identity

in order to anchor

decision making

through a formulation

of targeting, that is the

combination of biometric

and biographic information.

So, through one of the Snowden

Disclosures it was revealed

that the NSA intercepts

millions of images per day

off of the Internet,

including 55 thousand


recognition-quality images.

And it was explained

that, it's not just

the traditional

communications we're after,

but it is the

compilation of biographic

and biometric information to

implement precision targeting.

So it's basically the fusion,

24/7, of biometric body

tracking, with the 360-degree

biographical tracking of us

as individuals, through,

for example, our

digital footprints,

and our online personas,

in order to facilitate

data-driven decision-making.

So, I want to just wrap

up by explaining...

Okay, Big Data is

technically defined.

So you do have these

definitions such as

volume, velocity, variety,

veracity, value, variability,

I don't know why they

focused on the Vs,

but they did, when

they defined Big Data.

But Big Data is

much more than that.

Big Data really is

better understood

as a philosophy of governance.

Big Data as a

philosophy is a theory

of knowledge, it's a

theory of decision-making,

it transforms knowledge,

and it transforms in the

eyes of government,

what it can do, and

what it should do.

And this affects all of us, so going back to extreme vetting,

extreme vetting is misunderstood as a national security program.

Extreme vetting is the

new normal of the world

that we find ourselves in today.

Extreme vetting, and the

forms of Big Data Collection

and Big Data cyber-surveillance

is going to,

eventually, influence

every right, privilege, freedom,

and liberty that we have.

So we're already at the

earliest stages of it,

in the way that you have,

for example, voting rights,

having database

screening through the

Department of Homeland

Security databases before

there's an assessment of

whether or not you are the

appropriate individual of a

certain identity that can vote.

So there's digital

mediation of voting,

you have digital mediation

of your right to work,

so, employers are

screening employee's data

through the Department of

Homeland Security databases.

You have determinations of

who can enter the border,

so freedom of movement,

but this is just

the earliest stages.

Right now, through things

like the No Fly List,

the No Work List,

the No Vote List,

these are the earliest

stages of the Big Data

type of capacities

that the government has

to mediate our rights

and privileges.

And so, we cannot

continue to see

the world through

these Small Data eyes.

The world that we understand

it, as it is, right now,

we see through the

lens of Small Data

because that's what's

humanly knowable.

We are about to

transform into a world

where the government

and those that

formulate policies for us,

look at the world

through Big Data glasses.

So what they can

understand algorithmically,

who they can classify as a risk,

and what decisions they can make

based on those threat-risk


And so that's the future

of discrimination.

So unlike the world

that we once had,

for example, under Jim Crow,

so this is my thesis in

algorithmic Jim Crow,

when you had race, you had

the classification based

on something like skin

color, for example,

and then you had the

delegated screening

that was conducted by

humans, bus drivers,

people who owned theaters,

people who owned restaurants.

They were supposed

to do the screening,

they were supposed to

isolate individuals

and then reject them

based on their skin color.

In a modern world of Big

Data, discrimination is

going to be operated


So the classification won't

necessarily be based on

race, national origin, religion,

though it might

correlate with it,

it's going to be

based on, for example,

statistical data assessing risk,

and then you're gonna

have classifications

that will not necessarily

be human judgment,

but will be through

an algorithm.

The algorithm is going

to do the screening,

and then that is how you're

going to have deprivations

of privileges and freedoms

through some type of

technological method.

And we simply do not yet have

the legal tools, or

constitutional tools,

to mediate that type of

new form of discrimination

that we're now

starting to witness

at the very earliest stages.

So why don't I

conclude, here, and

thank you so much for

this opportunity to speak

with you today.


- We're talking

about Big Data, race,

and the criminal justice system.

And I wanna begin just by

briefly mentioning that

these issues have kind of been

endemic in biometric


since the very beginning.

Issues of race,

issues of colonialism.

You'll find that biometric identification was

first sort of pioneered in

the laboratory of the

colony, and only then,

brought back to the metropole,

which is something

that Professor Hu

in her other work talks

about, is still going on, now.

And the issue of predicting

individual behavior.

So, I'll begin by saying

a little bit about race.

The first ways of identifying

people were through

looking at their kind of

gross, physical aspects

through photographs, or through

the anthropometric

system, where they took

measurements, did meticulous

facial descriptions,

and looked at peculiar marks

and scars on their bodies.

And this, this system

looked, in great detail

at the kind of continuum

of human variety.

The anthropometric system had

more than 20 different shades

of brown eyes that it

identified people with.

But this system was considered

by Europeans to be unworkable

in the colonies, and

here's a quote, quotation,

from Francis Galton,

saying, well, we can't

use this system in India,

and they said the same

about China, because

they all look the same

to us, so we can't, and,

and so, it's for that reason

that fingerprinting

arises in India

and not back home, in Europe.

It's the same in

the United States,

where fingerprinting was

considered extremely appealing

for identifying Chinese

immigrants in the

late 19th century.

But it was considered

not useful for

identifying Europeans,

and here's this great quotation

from the San Francisco

Police Chief, saying, well,

you can use fingerprint, you

can use fingerprints for

indifferent Hindus

and wandering Arabs,

but when you're

identifying white people,

we need something else.

We need to look at their faces.

And we had the same

thing in Argentina,

the other cradle of

biometric identification,

where race played out in kind

of a different way between

northern and southern Europeans.

We have, in the United States,

in 1903, the famous

Will West case in which,

which supposedly demonstrated

that fingerprinting

could distinguish between two

supposedly indistinguishable

African American men,

who were then distinguished

by their fingerprints.

This didn't actually happen,

but it, the important

thing is it became

the sort of origin myth of

why we use fingerprints.

Then we fast forward a

couple decades to 1925,

and we have, in the New York

City Police Department,

the head of the

identification bureau saying,

who's talking about

his fingerprint file.

And here he's got

supposedly this extremely


biometric technology

that can identify people down to

the individual level,

and distinguish between

identical twins, and so on,

and he's saying,

"Yes, we have that,

"but we also sort our

files into three groups.

"Black, white, and yellow.

"And we do that just by

looking at them, cause we know

"those races when we see them."

So we have sort of

the coexistence of

this individualizing technology

and this very crude division

of people into three groups.

And I'd argue that

we're gonna see this

kind of throughout this history.

This kind of progression,

this effort towards


that ends up with grouping.

And the whole push

to use biometrics in

criminal identification was about being able to treat

criminals as individuals.

The idea was that

if people were,

if we didn't, before we had

biometric identification,

people would use aliases,

and then we wouldn't know how

long their criminal records

were, and then we

wouldn't actually be able

to measure recidivism.

Once we had biometric


then the people who ran prisons,

and police departments,

could think

that we have a pretty good

way of detecting recidivism.

And then we can do what

we've always wanted to do,

which is punish

first-time offenders

and recidivists differently.

Now this was sort of promoted

as individualized punishment,

that we would tailor the punishment to the individual.

What actually

happened was grouping.

There were two sets of prisons.

One for the first-timers,

and one for the recidivists.

So the whole population was

divided into two groups,

but you did at least

segregate off the first-timers

from the recidivists,

who supposedly would

contaminate them

with their criminal

ways and you could maybe

keep them away from that.

It's important to note, for

what we're going to get to later

in our discussion,

that even though biometrics

was useful for detecting

some recidivism, it prevented

people from using an alias

every single time

they were arrested.

It didn't of course

it wasn't, of course,

a perfect tool for

detecting recidivism,

for precisely the reasons

mentioned by Professor Klingele,

that we don't, that law

enforcement doesn't detect

all criminal behavior.

In addition, there are

other issues with

defining recidivism.

What do we mean by recidivism?

Do we mean committing

the same crime,

or do we mean a different crime?

Do we mean if you are

crime-free for 20 years

and then you commit another

crime, are you a recidivist?

Do parole violations

count as recidivism?

And does that have something

to do with how stringent

your parole conditions are?

And so on.

Fast forward a

couple more decades,

and people start

talking about this

fingerprint system is great,

and shouldn't everybody

be in a database?

Shouldn't we put the

whole population in

the database, and this

discussion, in this country,

took place in many countries, and this country sort of peaked

in the late 20s, and early 30s.

And Americans rejected

it, between 1935 and '43,

efforts to create a

full citizen fingerprint

database were defeated.

And this in a sense, created

what I call the

arrestee compromise.

It created two groups of people.

Criminals, which meant,

not just people convicted,

but anyone who'd been arrested.

'Cause if you get arrested,

your fingerprints go

in the database.

Plus, some other people.

Civil servants, military people,

people who work with children.

And then, the rest of us

who get the privilege of

not being in the database.

And so this arrestee

compromise is

going to come back

today, if we fast forward

a few more decades,

we get another

biometric technology,

seemingly more powerful

in many ways, called DNA.

And that brings about a kind

of public policy problem.

How big should the

DNA database be,

and who should be in it?

So it's important

to realize, that

the smaller your

DNA database gets,

the less useful it is.

And if your DNA database is

anything short of everybody,

there are gonna be some rapes that are going to go undetected.

And, those people are

going to rape other people.

So there are certainly costs

to reducing the size

of the database,

but those have to be

balanced against what we,

as a society, think is, is fair.

And in this country, and

in most other countries,

the DNA database

public policy problem

kind of ended up

in the same place

the fingerprint one

ended up with it,

which is on the

arrestee compromise.

In 30 states,

including Wisconsin, and

my state of California,

the US and the UK, we

have arrestee databases.

And so it is, it is sort

of the same compromise

that we struck with,

with fingerprints.

Now the problem with

arrestee databases,

of course, is that arrest

practices in policing

is not race-neutral, and

it's not class-neutral,

and it's not geography-neutral.

So the fact of having

an arrest seems to have

something to do with

your race, and it

can become a sort

of backdoor to race,

or as Professor Hu has written,

race at the backend

of the process.

And in this little,

sort of quick and dirty

kind of attempt to estimate

the racial composition

of arrestee databases,

population-wide databases,

and convict databases,

I found that the arrestee

database is the one

that gives you the smallest

number of white people

and the largest number

of black people,

as sort of not surprisingly.

And this is kind of exacerbated

by the fact that

familial searching

can now be done,

which means that if

somebody is in the database,

their close blood relatives

are effectively in the database,

even though they're

not in the database.

And so, the racial

implications sort of

balloon out from there.

So the arrestee compromise

is in some sense

the least fair possible

system we could come up with,

which is what we've

decided as a society to do.

There are two fairer solutions.

One is to have a universal

database, and put all of us

in the database,

and then we can all

bear equally the burdens

of the privacy violations,

and the risk of being

wrongly convicted, and the

burdens of familial

searching, and so on,

and a number of people have

advocated this, who

are list, listed here.

Kind of on the principle of


including Alec Jeffreys,

who developed DNA typing, and

the law professor Michael Smith, who teaches here, I believe.

So go see him, he

wants all of your DNA

to be in a database.

But for good reason.

As an anti-discrimination


Or, you could maybe

have a convict database,

which is possibly justifiable

on the grounds that you

should have reduced privacy

because you've been

convicted of a crime,

not just merely

arrested for one.

I wanna move now to

a slight, slightly

different topic that is,

I'm gonna try to bring

together with DNA in a minute.

And Professor Hu writes about

this as well, which is the

development of a

kind of philosophy

called forensic

intelligence, which we could,

perhaps distinguish between from

treating forensic

information as evidence.

So forensic intelligence

is kind of a new,

a hypothetical, mostly,

new approach,

towards forensics,

that distinguishes

from the old approach

in the following ways.

Whereas evidence

was oriented towards

law and the trial,

forensic intelligence is

oriented toward policing,

and security.

Forensic evidence is

reactive, it comes in

after a crime has occurred.

It tries to solve the crime,

and it tries to

prove who did it.

Forensic intelligence

isn't interested in that.

It's interested in

preventing crimes,

and being proactive in

linking crimes together.

Forensic intelligence is very

appealing in a lot of ways.

There's a lot of

problems with bias,

and unscientific reasoning

in forensic science

that I talk about

on other occasions.

And forensic intelligence

has a lot in common with

people like me who are trying to

reduce those issues

in forensic science.

So, it will be less biased.

It uses more, kind of

probabilistic and

scientific reasoning.

So it has a certain appeal.

But, let me get to the

unappealing aspects of it.

So the sociologist Sarah

Brayne has written, recently,

in her study of Big

Data policing, how

this kind of approach widens

the criminal justice dragnet,

but in unequally

distributed ways,

while appearing to be objective,

as Professor Klingele mentioned,

having the appearance

of being simply math.

That this bias can

kind of self-perpetuate

the existing biases in the data

that is fed into

these algorithms.

And again the simplest

example of this

is having an arrest, or

not having an arrest.

Your chances of

acquiring an arrest,

if you're a drug

user, are not the same

depending on your

race, your class,

and your geographic location.

And so, we have criminal

justice algorithms,

and we have the Loomis

case, here, in Wisconsin,

and complaints about the use

of criminal justice algorithms

to predict human behavior,

and as Professor Klingele

mentioned these algorithms

are in some sense, appealing.

And the main reason

is concerns about bias

in the non-algorithmic ways

about in the use of discretion

by human beings,

which is disturbing.

And, as Professor

Hu has written, the

criminal justice algorithms

sort of appear, on their face,

to be, actually,

equality enhancing.

Now, two main critiques

of these algorithms

are lack of discretion, that

same discretion that we're

sort of getting rid of,

and lack of transparency.

So I'm gonna get to

those two in second,

but now I'm just

gonna add a third

kind of tech modality

to forensic intelligence

and DNA databases,

and that's probabilistic


So I don't have too

much time to go into

probabilistic genotyping.

If you wanna know more about it, you can talk to Professor Keith Finley in the law school,

and he'll maybe tell

you a little more.

But, forensic scientists

are pretty good

at handling very clean,

single-source DNA samples.

But forensic DNA

profiling starts to get

very complicated when

you have mixed samples

with multiple

contributors to them

and small amounts of DNA,

and they become very

difficult to interpret

and there have been

problems with biased

interpretation of these samples.

And so, along

comes probabilistic

genotyping, which says,

let's have a computer

interpret these things,

according to a set of rules that we've pre-programmed before

we got the case, so

we're not gonna be biased

towards some particular

outcome in this case,

and plus, the statistics

for these complex mixtures

is very difficult anyway,

you need a computer

to do them anyway.

So that's the kind

of appeal of them.

On, at first glance, and again,

that's a legitimate appeal.

You have a couple

of problems, though,

so in the bottom left

you have the legal battles

over the source code.

'Cause defendants say,

"Fine, but I'd like

"to see the source code that

led to this interpretation,"

and the makers of

the software say,

"That's the entire

intellectual property

"of our company, and

so you can't see it."

And the courts

have to resolve this,

which they usually do

in favor of the company.

On the upper right, you

have the Oral Hillary case,

which is a murder

case in upstate New York,

allegedly committed by a

black man who was dating

a white woman, in a very

white part of the country.

And, with a very small

DNA sample that required

very complex interpretation,

and the two, two different

pieces of probabilistic

software reached

two different

results in this case.

This got very

complicated, and then,

in the bottom right,

I have a press release from

the company, Cybergenetics,

one of the main vendors of

this probabilistic genotyping

software saying, that NIST,

the National Institute of

Standards and Technology,

which announced a study

where they would test

these algorithms, and measure

how accurate they were

on known samples,

Cybergenetics released this

news release saying this was

a waste of taxpayer money

and anti-science,

and that they are

sort of trying to create a,


for over a decade,

NIST manufactured crises in

DNA mixture interpretation

to amass money and power.

So, the two critiques,

lack of discretion,

and lack of transparency.

Here's an op-ed from

the New York Times

that focuses on the

lack of discretion,

and her antidote to

these criminal

justice algorithms

is go back to discretion.

Have a good old human judge

work these things out.

Well the problem

with that was some,

there were some problems

with discretion, and the

good old-fashioned

human, human judge.

So it's not clear how

appealing that is.

And here's the lack of

transparency argument.

This was published by

a computer scientist, Duke,

who objects to

the proprietary nature

of the compass algorithm that

was used in the Loomis case,

and says, "I have a

non-proprietary, transparent,

"open algorithm,

which is better."

And that is true.

Transparency is

better than secrecy,

but notice that she

still has an algorithm

that she says predicts

criminal behavior really well.

And so I wanna emphasize is

what we're doing is

the same old thing.

We're predicting

criminal behavior, which

we're measuring by

arrest, or contacts

with the police,

and we're predicting it from

arrest, and contacts

with the police.

And we are bracketing

the problem,

the detection problem,

of whether we're actually

detecting criminal behavior

or we're just detecting

that criminal behavior

which led to contact

with the police.

And so we're protecting,

we're bracketing the problem,

the problems with the data

that are both input and

output from this algorithm,

even if it is transparent,

which is better.

And so, in conclusion,

I want to suggest that with

criminal justice algorithms,

we're sort of ending

up in the same place,

in something kind of like

the arrestee compromise,

where we use these algorithms

to kind of predict dangerousness

about people kind of

like arrestees, right?

They're people with enough

contacts with the state

that they start

to look dangerous

in these predictive algorithms.

And so, and that seems

like it's going to be

kind of discriminatory

through the back door

or at the back end.

And it kind of brings us back to

if we don't like this,

what are our other choices?

And again with the DNA database,

we had two choices.

We had every, we had

everybody [laughs]

or, just convicts.

Now what would that

look like in terms of

criminal justice algorithms?

Well the everybody

would be everybody.

But as Professor

Hu just told us,

it wouldn't just be our

DNA samples anymore,

it would be everything about us.

So it would be

everything about everybody.

So that would be fair.

We'd all bear the

privacy violations equally.

We'd all bear them together,

but we end up with sort of

a total surveillance society,

or as Professor Hu said,

a National Surveillance State.

Our other choice is to sort

of go back to something like

convicts, and to

focus, and to focus on

people who we can

prove have done crimes.

And that we're

reasonably sure did them,

and not on people who

we predict will do them.

And kind of the most,

the most extreme

advancement of this view,

is Bernard Harcourt,

who's written a book

called Against Prediction,

which, as the title indicates,

for kind of the

reasons I've explained,

is against prediction altogether,

as an activity,

not making it better,

doesn't, believes we

shouldn't do it at all.

So thank you.


Share this page