Philly Truce App &
Chat Bot
Internal report management app
Automated SMS response to consumers' submitted reports
My Role
UX Researcher
Task
In Phase 5 of the app's design
Act as ambassador between UX research and design teams
Objectives
To increase user engagement with the chatbot
To improve usability of the app
Constraints
11 weeks, 4 sprints
iOS (app)
SMS (chatbot)
Impact
Produced specific recommendations addressing insights
100^% of chatbot users voiced the need for Philly Truce to build community trust
100% of app users voiced demand for the integration of additional features to replace their current system.
Established Tech Literacy of internal users
The majority of app users- employees of Philly Truce- have recently served custodial sentences. Gaining an understanding of their technical limitations allowed us to recommend several significant adjustments to the app.
Uncovered conflict of interest between B2B and B2C users
Philly Truce works with the Philadephia Police Department (PPD). Outlining the specifics of that relationship has negatively correlating impacts on the safety of chatbot and app users.
Context
Philly Truce
Philly Truce is a not for profit organisation based in Philadelphia, which aims to combat crime and give black and brown boys the best start in life through community programmes in areas of Philadelphia with high violent crime rates.
Based on district attoourney crime stats and resident information, Philly Truce employees called Peace Patrol Officers (PPOs) visit high-crime blocks and engage with the community.
Chat Bot (already launched)
Chatbot users are community members who want to report escalating conflicts. The Chatbot is an automated SMS service which requests basic details about the conflict and assures the user that their report will be addressed.
The aim is to avoid violence through Philly Truce mediation.
Philly Truce App (not yet launched)
Once a report has been made (through the chatbot or face-to-face with a Philly Truce member), it will be stored in the Philly Truce App.
Peace Patrol Officers can log into the app to organise the reports as they work to address and resolve them.
Sprint 1: Understanding the Project
Previous Interviews
An interview betwen the Phase 4 team and Mazzie- one of Philly Truce's founders- revealed that the intention of the Chatbot was to provide reassurance to anyone reporting a conflict, encouraging them to be confident that help would be coming.
Product Evaluation
We looked at the App's prototype and shared our thoughts on it during a FigJam meeting. Initial thoughts were that it was simple and user friendly, with room for features like notes to be added to each case. I noted that some important real estate was taken up by low-priority information.
The Chatbot felt a little long-winded but was no worse than the average automated message service.
Literature Review
The team discussed options and decided to first conduct a literature review, gaining context for the community both in Philadelphia as a whole and in the neighbourhoods selected and partrolled by Philly Truce to prevent crime. The document would be provided to all teams on the project.
My focus was on census data including crime, education, income and family structure stats. We analysed the data as a team.
Client Interview: Takeaways
Users using the chatbot to report petty conflicts, rather than those likely to escalate to violence was the client's biggest frustration.
The intention of the Chatbot was to provide reassurance to anyone reporting a conflict, encouraging them to be confident that help would be coming before a member of the team could contact them.
Evaluation: Initial Thoughts
Most of us on the team were concerned that people may be uncomfortable reporting more serious conflicts to the Chatbot for a variety of reasons, including fear of worsening the situation and being viewed as a 'snitch'.
I highlighted areas where the app had unclear and slightly inconsistent labelling which I initially considered minor problems (but more on that later).
The explanation of the language used to categorise each stage of a report's progress felt too wordy to several of us.
Literature Review: Key Findings
Many neighbourhoods in Philadelphia are being gentrified- a well-documented issue which was mentioned early on by a Philadelphia-based team member. Articles on the subject have indicated that these changes have displaced organised criminal turfs, resulting in turf wars over the peripheral patches.
Whilst high-crime neighbourhoods had lower property prices than average and other expected signs of low-income, there was significant variation in levels of education and profession.
Kensington- an area with some of the highest crime rates in Philly- is central to a long-standing history with opioid addiction in the city.
Tensions between ethnic minorities and the Philadelphia Police Department (PPD) were highlighted by a 2019 incident where 15 PPD officers were fired and dozens more were disciplined following the discovery of racist and violent posts on their social media accounts.
After consolidating our data, we wanted to gain qualitative insight into the community response to Philly Truce and the Chatbot. We planned to do this in the next phase.
Sprint 2: Exploratory Interviews
Recruitment
To gain a fuller picture of the local community, we began recruitment for user interviews. Tech Fleet is not for profit, so without a recruitment budget we used two primary methods:
Reaching out to the founders, who are valuable members of the local community, both to interview them and for introductions to community members
One of our team members lived on a block local to Philly Truce. She invited some of her neighbours to participate and they kindly accepted
Assumptions about the Chatbot
The local community were distrustful of police because of historical racial tension
Concern over reports being passed along to police are making users hesitant to report incidents to Philly Truce
Interview Chatbot Findings
There is a huge amount of distrust between the local community and the police, however it is multi-faceted. One participant recalled their reserved brother being arrested from a peaceful protest and beaten up by police; another recounted a threatening note they received, after police were seen requesting their ring doorbell footage following a shooting. An intrinsic part of the culture in Philly is about staying out of trouble for self-protection. Part of that means avoiding police contact.
Users of all races are absolutely worried about involving police in conflicts and generally prefer not to unless it is absolutely essential, feeling that the police discriminate or worsen problems.
Users like the idea of seeking mediation from community members, but their trust needs to be earned through community contact. Philly Truce must engage face-to-face as much as possible.
Assumptions about the App
The app will be used expplicitly to manage reports of conflicts which could escalate into violence
The app will be the sole tool used to manage reports
Peace Patrol Officers (PPOs) are long-standing members of the Philadelphia community, who are volunteering to help
Interview App Findings
Philly Truce Peace Patrol Officers (PPOs) report a number of environmental issues to the relevant organisations. Overgrown lots are a common example.
PPOs use iPads during patrols and frequently plan patrol routes using Google Maps and photograph environmental problems- features currently missing from the app or in development.
PPOs are employees of Philly Truce. Most of them have been incarcerated and recruited through halfway houses. They are often not from Philadelphia and it is common knowledge that they prefer not to patrol near where they live, to avoid trouble. This tends to be an initial job post-incarceration and turnover is high, as many PPOs find work closer to where they are from.
PPOs are instructed not to communicate with minors. This needs to be addressed, as a protocol for handling reports made by minors is not fully established.
The member of the team living in Philadelphia was able to join a patrol and noted a lack of technical literacy but great community engagement on patrols.
Recommendations
To increase Chatbot use, anonimity has to be the absolute priority. Philly Truce should be clear about needing to report issues to the police in emergencies, as community members value transparency. They should continue to spend time engaging with the community and need to increase awareness of their presence, potentially handing out fliers and spending time in local parks at outdoor events.
The app needs to accommodate a wider range of report types, including for environmental reports and the ability to upload photos. There needs to be clearer labelling and a move away from reliance on icon recognition.
The strategy team should work with Philly Truce to create a protocol for dealing with reports from minors.
Given the high user turnover, the app must be extremely easy to navigate and should include an initial simple tutorial.
Sprint 3: App Usability Testing ^ & Personas
Test Recruitment
Our handover from the previous phase had mentioned a poor turnout for app usability testing, so we had been recruiting since Sprint 2. My colleague in Philadelphia fostered relationships with the Peace Patrol Officers (PPOs) she met on patrol and asked them at the time to participate in future testing, allowing us to be on the front foot for recruitment.
We consolidated our ideas for usability tests alongside the ambassador from the design team and planned the tests through FigJam and Google Docs.
The Usability Test
The test was set up and carried out on Maze, with a Figma backup link. We designed the test shown below:
Reviewing a report
Goals:
To test the hierarchy by establishing the thought process users use to decide how to address reports.
To see how easily users can navigate through the app.
Creating a new report
Goals:
To test how easily users can create reports.
Reassign a report to a different PPO
Goals:
To test how easily users can reassign reports.
Edit a closed report
Goals:
To understand how users locate closed reports.
To test how easily users can edit reports.
View reports assigned to self
Goals:
To test the language of the reports filters, which had raised concerns over ambiguity.
App Insights
The hierarchy needed to be changed to better prioritise information.
The user flow was generally very simple and easily understandable.
Participants felt there needed to be more scope to add detail to reports.
Abbreviations like "PPD" and text-free icons needed to be replaced with full text, as several users misunderstood meanings. More tech-literate participants noted that these were areas which may cause their colleagues confusion.
General Insights
The minimum viable product must include a photo upload feature and GPS tracking, as users would otherwise have to juggle a number of apps on patrols. Having a feature to allow phonecalls and fast contact should be considered down the line to replace the current walkie-talkie system.
If Philly Truce is viewed as an extension of the police (which they absolutely are not), it poses a significant risk to PPOs. It should be strongly communicated that they are mediators, aiming to prevent conflict escalation. It should also be clear that they cannot enter into live gun violence and would have to defer at that point. This remains a difficult line to tow.
Community members who speak directly to PPOs show an overwhelmingly positive response to having others in the community trying to diffuse conflicts.
PPOs should be able to access the Philly Truce App through their phones, which vary in make and release date. The development team will need to factor this in. Further research should be done to investigate whether Android or iOS should be considered first.
Personas
Given the nuances of the local community and PPO profiles, we wanted to create personas to enable the other teams to understand their core users. There was disagreement on the team about whether there should be photos included.
Many on the team felt that details like race and age could unintentionally provoke assumptions about users.
I and one other colleague wanted to include photos. Philly Truce primarily targets black and ethnic minority young men, which they do not shy away from. Most PPOs are also black or latino and all are men.
Ultimately, we agreed not to include photos. The most important shared features were added, which we were satisfied reflected both groups accurately.
Sprint 4: User Journey Mapping and Test Report
Test Report
The report was divided into sections. As I suggested the first test, I covered the findings in the final report with recommendations for the design team.
We all left notes and others on the team polished the report. We then presented our findings in the final sprint retrospective of the phase.
User Journey Mapping
I led the user journey map workshop covering Chatbot users.
The goal was to gain a better understanding of areas of focus for the next phase and beyond. Community member reservations were the biggest barrier to Chatbot success, so improving the steps leading to conversion would be a focal point in the future.
Handover
The handover for the Phase 6 team documented all unimplemented recommendations and a summary of the key findings from the Phase.
Reflections
Iteration
In the future I will seek to improve the flow between the research and design teams, to produce more iterations per Phase (optimally one per sprint).
Differentiation from the Police
Striking the balance between communicating the need for police involvement in case of emergency and framing Philly Truce as a positive tool to resolve escalating conflicts continues to be difficult. It will likely be an ongoing process of trial and error to build trust within the community.
