Let’s start with a theoretical scenario: Al Capone and Mother Theresa are both arrested and charged with committing murder. Based on that charge, bond is set at $5 million for each of them. Given what we know about them, who do you think would be seen as a lower risk to the community if released? Probably Mother Theresa. But who do you think is going to be able to pay bail and be released? Definitely Al Capone.
It’s a conundrum that has plagued the criminal justice system in the U.S. for decades, as high incarceration rates have been driven by pretrial detention, i.e., holding people awaiting trial.
But what if there was a way to review someone’s personal history — previous interactions with law enforcement and courts, involvement with drugs and alcohol, living situation, etc. — in order to assess their risk if released?
That’s the idea behind pretrial risk assessments, which are used within the criminal justice system to determine if and under what conditions someone can be released into the community while they wait for the judicial process to play out. In Colorado, such a tool has been used by many jurisdictions since 2013 in an attempt to lower pretrial jail populations, largely supported by district attorneys, pretrial services departments, prominent legislators and others. However, there’s been increasing concern among criminal and racial justice advocates across the country that these assessments reproduce certain biases — especially toward people of color and those experiencing homelessness — that exist within the current criminal justice system.
In response to a recent study of the Colorado Pretrial Assessment Tool (CPAT) that analyses this bias and recommends a revised tool, ACLU Colorado argues the use of such a tool leads statewide pretrial reform efforts in the wrong direction. What has been more effective, they say, is some of the simple policies local jails and prisons have used in response to COVID-19, essentially halving the state’s incarcerated population in six months.
“Risk assessment tools have not decreased the pretrial population,” Rebecca Wallace, senior staff attorney and senior policy counsel of ACLU Colorado, says. “They haven’t worked. It is a crutch. It has the aura of objective decision-making that is actually a recreation of the biases that are baked into the current system, but we take those biases and we make them look objective, and that is actually more harmful.”
Pretrial detention is considered a major contributor to the nation’s high rates of incarceration, growing 433% between 1970 and 2015 according to the Vera Institute of Justice. People detained while awaiting trial make up two-thirds of local jail populations across the country, the vast majority of whom are charged with nonviolent crimes.
“Taking away somebody’s liberty by the government is the most serious control mechanism that we have,” Wallace says. “And it should be reserved for the most serious potential risks.”
Seen as a reform leader, Colorado adopted a law in 2013 that provided guidelines for the way in which judges are supposed to set bail, and encouraged, though didn’t require, jurisdictions to use pretrial services and risk assessments in order to prevent unnecessary pretrial detention. This led to the development of the CPAT, which uses an algorithm based on a list of 12 factors to determine a risk score of one through four. It scores individuals based on whether or not they are currently housed or own a cell phone, whether they rent or own their home or contribute to residential payments, whether they have past or present problems with alcohol or receive mental health treatment, previous jail or prison sentences or pending cases and the age of first arrest.
That risk score is then presented to a judge, sometimes with additional information, to determine conditions of release, which can range from personal recognizance (PR) bonds to a variety of community supervision methods like drug and alcohol monitoring, mental health treatment and even employment help.
“The goal here is to ensure that people are not fleeing the jurisdiction and they’re going to come back to court and also that they’re not posing an immediate risk to the community,” says Boulder County District Attorney Michael Dougherty, who generally supports the use of CPAT, especially when compared to leaving these determinations up to an individual judge’s discretion, as was the practice previously. “But when one is trying to predict human behavior in any walk of life, I think that’s extremely difficult to do. And there’s no perfect instrument for that.”
In Boulder County, the average length in the pretrial system is about six months, although it can vary widely from three months to two years, according to Monica Rotner, division manager for Boulder County Community Justice Services, which includes the pretrial program.
Of the 10,000 or so people that are booked at Boulder County Jail each year, Rotner says about half are interviewed and assessed, the rest are either on hold from another jurisdiction or not “bond eligible” for a variety of reasons.
“There’s no doubt that assessment tools of all sorts maintain the bias that exists in our society,” Rotner says, but that’s why Boulder County bond commissioners — those tasked with assessing pretrial defendants — have an entire matrix of factors to use, and don’t rely solely on the CPAT. “We believe strongly in combining it with an in-person interview and also being able to override those scores, so we never make a recommendation to the court that’s based alone on the CPAT,” she says.
In an effort to more fully understand the impact of the CPAT, the state, with some additional resources from the University of Northern Colorado (UNC), funded a recent study from UNC researchers Victoria Terranova and Kyle Ward.
To assess for bias in CPAT, they relied on self-reported data about race, ethnicity, sex and residential status — whether a person was housed or homeless when arrested.
“We didn’t find differences where being black versus being white led you to a higher score,” Ward says. “Where the differences lie was when we dove into those false positives and false negatives.”
The false positive rate being the rate at which those given a high-risk score don’t actually have a high-risk outcome, such as failing to appear in court or being rearrested when released on bond.
“Our bias analysis focuses on the false positive rate, because from that kind of perspective, if you think about it, those who are assigned as a high risk but may not result in a high-risk outcome are the ones who are receiving the detrimental consequences of being justice-involved,” Terranova says.
The researchers found that the CPAT in its current form had a higher false positive rate for black defendants versus white, and unhoused defendants versus housed. Of the 12 factors in the original CPAT, Terranova and Ward found prior violent arrest and questions about residential status to be the most problematic. Their study includes a recommendation for an updated risk assessment tool based on their research, the CPAT-R, that only considers nine factors.
“To say that a tool is actually biased I would say is a bit of a stretch just solely relying on the false positive rate,” Terranova says. “Being a descriptive statistic, there really isn’t a threshold for what would be biased.” In essence, there isn’t statistical certainty that biases exist within the tool.
But in a letter to the researchers, Wallace and Aaron Horowitz, deputy chief analytics officer for ACLU National, argue that the discrepancy in false positive rates (FPR) show that bias does exist. “FPR inequality is a well-established way to determine if predictive bias exists in predictive models,” Horowitz writes in an email to Boulder Weekly. “Given the magnitude of the differences between false positive rates among homeless versus housed people and black people versus white people, we have no reason to doubt these disparities are real and meaningful.”
And that makes the use of the CPAT and even the CPAT-R problematic in assigning risk to pretrial defendants, according to its detractors.
“The new study indicates that CPAT has racial bias,” Maureen Cain, legislative and policy director for the Office of the Colorado State Public Defender, says. “That’s a given. I think every risk assessment tool by the nature of the factors that correlate to risk, you’re going to find over-representative numbers for over-policed communities.”
According to the Pretrial Justice Institute (PJI), a Baltimore-based think tank that long supported the use of pretrial risk assessments, these tools should no longer be considered a solution for equitable pretrial justice systems. In a February 2020 position brief, PJI argues that regardless of scientific methodologies, every pretrial risk assessment is inherently flawed because every risk assessment uses data that reflects the “structural racism and institutional inequity” in our current court and law enforcement policies and practices. “Use of that data then deepens the inequity,” PJI argues.
Risk assessment tools are built upon historic policing and criminal justice data, which is problematic, Wallace says, because these systems tend to be inherently biased against communities of color and those living in poverty. For example, the age of first arrest (not conviction) is the most heavily weighted factor in most risk assessments, but that correlates with over-policed communities where young people can be arrested for low-level offenses like smoking in the park.
“The analytics community has a term for it: garbage in, garbage out,” Wallace says. “Your outputs for an algorithm are only as good as the data that’s going into it.”
This also comes into play when looking at pretrial defendants experiencing homelessness, those who are often incarcerated for low-level offenses directly tied to their lack of housing, such as sleeping on a park bench, violating camping and smoking bans or urinating in public.
“We know that these individuals are being viewed as extremely high risk under the risk assessment tool. If they go back outside, they get released, they’re going to sleep on a park bench again,” Wallace says. “This is not a useful tool and leads to thinking these are risky populations when they are not risky at all.”
In general, the pretrial population is very low risk, as less than 2% will be charged with a violent offense if released, according to the UNC study. And yet, the risk assessment tool still divides people into low-risk and high-risk categories, which is misleading, Wallace says.
The tool “forces a certain number of people into these high-risk categories that by and large are not high risk,” she says.
“We believe it over-defines risks — the people in the highest risk category in the instrument really are people who will substantially more likely than not, not re-offend, not fail to appear,” Cain adds. “And if they re-offend, the most common re-offense is a misdemeanor traffic offense.”
Which raises another area of concern the ACLU and others have with risk assessment tools. Measuring success based on failure-to-appear rates also skews success rates against impoverished communities, those that don’t have cars and rely on public transportation, those who can’t easily get time off work without losing employment or may have trouble with childcare.
“There is constantly going to be a poverty factor, there is going to be a policing factor,” Cain says. “The tool should help for release; it shouldn’t drive detention.”
Overall, Wallace argues, why rely on such a tool if it hasn’t moved the needle on decreasing our jail populations?
“There are jurisdictions that have managed to decrease their jail population while using tools, and New Jersey is a great example of that,” Wallace says. “But there are plenty of jurisdictions including Kentucky that have increased their jail population while using these tools.”
New Jersey adopted a statewide pretrial assessment method in 2017, and the jail population has decreased significantly since then. However, racial disparities persist, and more and more advocates are arguing against the method.
Kentucky’s incarceration rate, on the other hand, has remained high, ranking seventh in the country for per-capita incarceration, despite implementing risk assessments more than a decade ago. Plus, judges in that state who were provided with risk assessment scores have shown increased bias in their decision-making, according to a 2019 Harvard study. It found that not only did the use of a risk assessment tool in Kentucky increase racial disparities in initial bond settings, but judges were more likely to override risk assessment recommendations, setting harsher bond conditions for black defendants compared to similar white defendants.
Earlier this year, New York implemented a law that all but eliminates pretrial detention and cash bail without the use of a risk assessment tool in most misdemeanor and nonviolent felony cases. It has already received some pushback, however, over public safety concerns. In California, a measure to fully eliminate cash bail in favor of risk assessment was on the ballot this year, but with pushback from racial justice advocates like Human Rights Watch, PJI and the ACLU, it appears to have failed.
A key factor in moving away from pretrial risk assessments are increased community resources to address people’s needs as opposed to their risk, according to PJI. Allocating funding toward policies that improve transportation options, court reminders, access to childcare and behavioral health treatment could significantly contribute to addressing failure to appear rates and even likelihood of re-offending. Wallace adds an increased use of summons in lieu of arrests could also help.
“What we want to do is dramatically decrease the pool of folks who are arrested, period,” Wallace says. “If we dramatically decrease the pool of people who are [incarcerated] pretrial, then the folks who are there, we can have more individualized, meaningful decision-making about them, in part because they’re well-represented by counsel, who have time to prepare and meet with them and make bond arguments before the judge. … Due process hearings like that are really lacking from our current system, and that has a lot to do with volume.”
For now, both DA Dougherty and Rotner from Boulder County say having a tool like CPAT is better than nothing, even if it’s clear the tool needs to be improved.
“The CPAT is something that grounds the judges in having to look at data and try to be objective about it,” Dougherty says. “And obviously that needs to be improved upon, I can’t stress that enough, but I’d rather have that then the judge on Monday being completely different than the judge on Tuesday when it comes to bond. And you can only make the right decision on individual cases if you have accurate information besides just one’s gut reaction.”
Dougherty is also concerned about public safety, especially considering that Colorado falls into the bottom 10 states in the nation when it comes to the re-offense rate: Almost 50% of people who leave state prison will be rearrested in three years, he says. But some of this does have to do with lack of other social services, like substance abuse and mental health support, which face significant pandemic budget cuts, further hampering solutions that don’t rely on risk assessment. Nevertheless, it appears COVID-19 has moved the needle in ways decades of criminal justice reform have failed to do.
“The COVID experience is actually going to be more informative and helpful towards bail reform than perhaps the results of this study,” Cain says.
According to an Oct. 13 ACLU Colorado report, jail populations decreased by 46% statewide at the height of the pandemic, meaning there was an average of 6,000 fewer people in jail every day. Whereas the average jail occupancy was 81% pre-pandemic, it’s now at 47%. (Since July, when jury trials and court sentencing resumed, jail populations have increased 17%.)
The report attributes the jail population to four policies in particular: Active management of county jail populations to reserve beds only for those who posed a threat to others; increased use of summons and heightened arrest standards, which led to a 48% drop in people being held pretrial on misdemeanor charges; increased use of personal recognizance bonds, releasing folks without having to pay; and changes in probation, parole and pretrial services, which meant non-criminal infractions and technical violations didn’t send people back to jail.
“I don’t think that we need to wait for some concept of comprehensive bail reform that has eluded Colorado for so long in order to take substantial steps to keep our jail population as low as it is right now,” Wallace says.
In Boulder County, COVID policies have resulted in a dramatic increase in PR bonds for people who scored high on the CPAT, according to Dougherty, and at the height of the pandemic, those with low scores were released 100% of the time.
“In Boulder County, we’re going to take the lessons that we learned through the pandemic and make long-term changes based on that,” Dougherty says. “We’re living through a forced experiment.”
