Title
Consider Review of the Alameda Police Department's Implementation of Truleo, including Providing Direction on Termination of the Contract between City of Alameda and Truleo and the Removal of Alameda "Case Study" from Truleo's Website; and Consider Reviewing Implementation of the City's Body Worn Camera (BWC) Recordings Retention Policy, including Providing Truleo Audio Files from BWC Recordings from January 1, 2021 to Date (i.e., Files from Over One Year Ago). (Councilmember Herrera Spencer) [Not heard on May 2, 2023]
Body
COUNCIL REFERRAL FORM
The Council can take any of the following actions:
1) Take no action.
2) Refer the matter to staff to schedule as a future City Council agenda item.
3) Take dispositive action if sufficiently noticed such that the public and Council have been provided sufficient information by the published agenda, and no formal published notice of a public hearing is required.
Name of Councilmember requesting referral: Councilmember Herrera Spencer
Date of submission to City Clerk (must be submitted before 5:00 p.m. on the Monday two weeks before the Council meeting requested): 4/17/2023
Council Meeting date: 5/2/2023
Brief description of the subject to be printed on the agenda, sufficient to inform the City Council and public of the nature of the referral:
On January 4, 2022, City Council agreed to allow the City to enter into a contract with Truleo.
Public speakers expressed concerns about the use of Truleo:
"Stated that she does not have confidence in the use of Truleo; the resource is unproven; expressed support for taking steps towards UBI; stated there should be a continued increase in Police accountability; she supports oversight: Laura Cutrona, Alameda.
Stated the Citizen Oversight Committee must go forward; expressed concern over the use of Truleo; stated human contact is needed on Police matters: Marilyn Rothman, Alameda." Minutes, Jan. 4, 2022 City Council meeting.
The Police Chief stated: "Truleo started as Greenkey and is primarily used in customer service for major banks; the system analyzes and assesses speech and is able to decipher intent and identify specific language; the system produces a daily report and identifies at-risk encounters with staff; the system can identify whether staff are engaging or exposed to at-risk behaviors; the system is able to decipher commands associated with force and tone and is able to identify at-risk patterns of behavior; real-time identification of Officers’ body worn camera footage is possible within 24 hours; encounters can be flagged for review as opportunities for re-training, holding people accountable and acknowledging unacceptable behavior and negative encounters; negative encounters are approached from a wellness perspective; discussed employee wellness; stated the Department is leveraging technology to accomplish both aspects; Truleo technology provides a real-time opportunity to address acceptable behaviors and praise behavior that goes above and beyond; policies and procedures can be adjusted to provide toplevel services; expressed concern about missing little things, which can avalanche in bad behavior; unchecked, bad behavior can amount to larger issues; the technology is not a solve all; however, it helps people make decisions in the correct direction; the technology is efficient; outlined review of body worn camera footage; stated the Department is not solely relying on the technology; the technology is to help point people in the right direction to ensure appropriate action is taken, at-risk patterns of behavior are curbed, wellness concerns are addressed and good behavior is applauded."
And, continued: "Alameda would be the first City in the State to use the technology; stated several cities and local agencies in the Bay Area are subscribing as well; the City of Seattle is conducting reform and Truleo is being used to point people in the right direction."
Now, Truleo has prepared a "case study" of Alameda which is presented on their website. <https://www.truleo.co/alameda-case-study> Their website does not have "case studies" from any other City. The "Case Study" includes "statistics," which they use to sell their product and fundraise. <https://www.globenewswire.com/en/news-release/2023/04/17/2647931/0/en/Body-Camera-Analytics-Company-Truleo-Reaches-500K-in-Investments.html>The Case Study has limited information of how they arrived at their "analytics."
Truleo's "Case Study" is based on a comparison of audio files from Alameda BWC recordings from January 1 - June 30, 2021 to audio files from BWC recordings from January 1 - June 30, 2022. I do not believe that comparison to old BWC recordings was presented by staff of what Truleo would be doing and thus I do not believe that was approved by City Council. I believe the approval was based upon looking forward, from the date of the contract, reviewing audio files from BWC recordings during the period of the Truleo contract, not from a year earlier. The "Case Study" provides that Truleo was given access to all of the old audio files from BWC recordings from January 1 - June 30, 2021, and then used that to compare against new audio files from BWC recordings from January 1 - June 30, 2022. This raises questions in regards to whether or not the City is following it's BWC recordings retention policies.
APD Policy 450 provides for limited retention of Body Worn Cameras recordings and a deletion schedule.
450.8: "...Videos that have no evidentiary value need to be changed from "uncategorized" to "No Evidentiary Value" and purged in accordance to the deletion schedule."
APD Policy 450.8.1 ."...Files should be securely stored in accordance with state records retention laws and no longer than useful for purposes of training or for use in an investigation or prosecution. All recordings shall be retained for a period consistent with the requirements of the organization’s records retention schedule but in no event for a period less than 90 days. While it is critical to retain recordings of evidentiary value, it is important that files of insignificant incidents are deleted in a timely fashion according to schedule. The following retention schedule should serve as a guide:
1. Homicide - permanent
2. Inspectional Services - 6 year retention
3. Felony - 3 year retention
4. All Use of Force - 3 year retention
5. Liability to the City - 3 year retention
6. Misdemeanor (including traffic related misdemeanors) - 1 year retention
7. 5150 W&I - 1 year retention
8. Citations/Warnings - 1 year retention
9. Field Interview Card/Incident Card/ Public Contact- Minimum 180 day retention
10. 901a/Fatal - 3 year retention
11. No Evidentiary Value - 90 days
12. Public Relations - 2 years
13. Admin Interviews - 3 years"
As the City's retention policy provides BWC recordings should be retained no longer than "useful for purposes of training or for use in an investigation or prosecution" in no event would all BWC recordings be retained for over a year. Files with no evidentiary value should be deleted after 90 days. Thus, it appears that in order for Truleo to have been able to conduct their Case Study, that they either were not provided all audio files from BWC recordings January 1 - June 30, 2021 as suggested in their Case Study or the City's retention policy is not being followed. If Truleo was not provided all audio files from the first six months of 2021 and the retention policy is being followed, then that conflicts with their Case Study and would impact their "analytics."
Recent research has shown that Truleo's program has potential issues (e.g., accuracy and bias) and raises significant privacy and civil liberties concerns.
• Seattle has recently (approximately February, 2023), terminated their contract with Truleo:
"Seattle Police Department has stopped using an artificial intelligence platform designed to analyze police body-cam footage in search of potentially inappropriate officer behavior. SPD had been using the software since 2021, when it became an “anchor customer” for Chicago-based startup Truleo <https://www.truleo.co/>, which states on its website that its mission is to “improve trust in the police with body camera analytics.”
“SPD entered a technology demonstration project and decided it had sufficient promise to attempt a limited pilot, to validate functionality,” Det. Judinna Gulpan, a public information officer with SPD, told GeekWire on Friday. “SPD was in the process of that validation but has since discontinued the project, in light of reactions to the recent Axios <https://www.axios.com/2023/01/30/police-tyre-nichols-bodycam-footage> and GeekWire <https://www.geekwire.com/2023/seattle-police-department-using-ai-software-to-analyze-body-cam-footage-and-officer-behavior/> articles.”<https://www.geekwire.com/2023/seattle-police-dept-discontinues-use-of-ai-platform-to-analyze-body-cam-footage-and-office-behavior/>
• "[T]here are potential issues with this kind of software, said Os Keyes <https://www.washington.edu/news/people/os-keyes/>, a doctoral student in Human Centered Design and Engineering at the University of Washington who has spoken out against the use of AI with body-cams <https://venturebeat.com/business/ai-weekly-the-perils-of-ai-analytics-for-police-body-cameras/>.
At a basic level, body-cams record every interaction an officer has with other people, and can record images and audio of nearby conversations or acts the officer isn’t involved with. A body-cam on a police officer becomes a sort of roving digital surveillance tool, Keyes said. There’s reason to fear that the AI will misinterpret what it hears in body-cam audio recordings, Keyes said. Even the best AI struggles to correctly identify sarcasm, and a number of natural-language processing models have exhibited bias along racial <https://academic.oup.com/applij/advance-article/doi/10.1093/applin/amac066/6901317>, ethnic and gender <https://hbr.org/2019/05/voice-recognition-still-has-significant-race-and-gender-biases> lines. A Washington Post-commissioned study <https://www.washingtonpost.com/graphics/2018/business/alexa-does-not-understand-your-accent/> determined that Amazon and Google smart speakers were 30% less likely to understand humans who didn’t speak English with American accents. The accuracy rate for speakers with Chinese, Indian or Spanish accents was about 80%. Combine the roving surveillance camera issue with AI’s struggle to understand the full spectrum of human speech, and you’ve got a problem, Keyes said. Analyzing body-cam recordings “can be a really useful practice, not only for training but for detecting things like unreported brutality or harassment,” Keyes said. But because it’s so important, “it’s completely inappropriate” to use a system that hasn’t been independently reviewed, Keyes noted. <https://www.geekwire.com/2023/seattle-police-department-using-ai-software-to-analyze-body-cam-footage-and-officer-behavior/>
• Given how much information body cameras collect about civilians - even those not suspected of crimes - the ACLU of Washington told Axios Seattle <https://www.axios.com/newsletters/axios-seattle-3e44ebe6-4452-4766-b94d-6cecff762ca1.html?utm_source=newsletter&utm_medium=email&utm_campaign=newsletter_axioslocal_seattle&stream=top> earlier this month that the technology raises significant privacy and civil liberties concerns. <https://www.axios.com/local/seattle/2023/02/14/seattle-police-truleo-artificial-intelligence>
Thus, I request that City Council review implementation of Truleo, consider terminating the contract between City and Truleo and request removal of Alameda "case study" from Truleo's website; and review implementation of the City's Body Worn Camera (BWC) recordings retention policy, including providing Truleo audio files from BWC recordings from January 1, 2021 to date (i.e., files from over one year ago).
Thank you for your consideration.
The City Council, at its July 25, 2020 Priority Setting Workshop, established the following 5 priorities for 2021:
1. Preparing Alameda for the future
2. Encouraging economic development across the Island
3. Supporting enhanced livability and quality of life, including addressing the housing crisis and homelessness
4. Protecting core services
5. Ensuring effective and efficient operations
Briefly describe which Council priority the subject falls under and how it relates:
Preparing Alameda for the future; supporting enhanced livability and quality of life; and, protecting core services.
Attachments:
Minutes of January 4, 2022 City Council meeting
Alameda Police Department Policy 450 "Portable audio/video recorders"