Several states conduct periodic independent reviews of the performance of state judges.  While the details vary from state to state, the intent of the program is consistent: to educate voters about judges’ job performance and to provide objective feedback so that a judge can address any shortcomings.

In Colorado, every new judge is required to stand for retention election after being appointed to a two-year provisional term. Once retained, experienced judges then need to be re-elected every four to ten years depending on the level of the judge.  The Colorado Office of Judicial Performance Evaluation (OJPE) conducts its evaluations in cycles so that every judge gets evaluated for professional development purposes one year before they are up for retention and then evaluated again in the year of the retention election. In the first evaluation, information is shared only with the judge so that they can improve their performance. During the retention year, the Colorado Commission on Judicial Performance uses the data to inform a recommendation on retention for the judge and a summary of the data for each judge is also shared with the public.

Colorado’s questionnaire was developed by Colorado Commission on Judicial Performance, a group that includes legislators, judges, attorneys and non-attorneys. The survey includes ratings of a judge’s overall performance, such aspects as administrative skills, application and knowledge of law, communications, diligence and demeanor.  The survey also provides the opportunity for respondents to explain their reasons for the ratings.

The objective for sampling of respondents is to include a cross section of those who have personal and professional interactions with judges in the courtroom. This includes: prosecutors, public defenders, private attorneys, jurors, litigants, law enforcement personnel, court employees, court interpreters, probation office employees, social services employees, crime victims, and other judges.

While conducting and reporting on a survey seems simple enough, this effort is very complex and exacting.

• Multiple groups that have contact with a specific judge are surveyed.

• If a member of a group has an experience with more than one judge they are asked to rate each judge.

• The number of potential respondents is very large, with there being hundreds of thousands of people who interact with judges across the state in a given quarter. The contact files are obtained from individual courts and formatting differs so the preparation of contact lists is very time consuming. Files must be obtained, reformatted and checked.

• Matching each respondent and their responses with the specific judge being evaluated is critical. With more than four hundred active judges and tens of thousands of respondents this is a daunting responsibility.

• Many respondents provide in-depth comments. Each comment must be carefully read by an analyst and any information that may reveal the identity of the respondent redacted.

• Each judge is provided a customized report with their ratings, verbatim comments, and comparisons to other peer-level judges. In addition, summary information is provided for posting on the commissions website.

Market Decisions Research collects the data using a combination of online surveys, and paper (mailed) surveys, depending on whether the contact file has an email address or only a physical address.  MDR staff conduct all the work on this assignment, controlling quality, anonymity of respondents and confidentiality of the data.  They program or layout the survey, email or mail invitations, enter paper survey data, analyze the data and prepare all the reports.

While the Commission provides a recommendation on retention ultimately it is up the voters to decide who serves them as a judge.  For the period from 1990 through 2016, The Commission made 1392 recommendations for retention of judges and 14 recommendations not to retain. Of the recommendations to retain, the voters reelected 99.7% and of the 22 recommendations not to retain voted in agreement with the commission on 8 or 36% of the time.

Market Decisions Research also conducts an online evaluation survey of attorneys, court staff, jurors, juvenile court professionals, and others for the Utah Judicial Performance Evaluation Commission and conducts survey data analysis for the Massachusetts Judicial Performance Evaluation Committee and the Supreme Judicial Court.

Tableau Data Visualization