14  Sorting and filtering to find stories

14.1 A sorting miracle

After Ferguson, Mo., police killed Michael Brown in 2014, advocates and journalists began examining the racial and ethnic gap between police departments and the communities they served. The New York Times found a 7-year-old survey conducted by the Justice Department that allowed it to compare the data for major cities in a standalone graphic that it published later that year.

When newer data reflecting departments’ makeup in 2012 was released a year later, Matt Apuzzo and I hoped it would show some differences. It didn’t. So we were left trying to find news in the data that was clearly of public interest.

After matching up the demographics of police departments with their cities, I started sorting, filtering and Googling. Could there be news in the outliers on the list? Which departments most closely represented their communities? Which ones had unusually large gaps?

Chief William T. Riley III. Credit: Laura McDermott for The New York Times

I quickly stumbled on telling anecdote to frame the story: Inkster, Mich. had one of the least representative departments in the country, and had recently hired a new police chief to help mend the department’s fraught relationship with its largely African-American community. Where had he come from? Selma, Ala., one of the most representative police departments in the nation. Interviews with the chief, William T. Riley III, suggested one reason for some cities’ disparities: there was no state or federal money to pay for training new police officers.

The story, “Police Chiefs, Looking to Diversity Forces, Face Structural Hurdles” helped explain the persistent gap between the makeup of police in some areas and the communities they served.

14.2 Sorting and filtering as a reporting tool

“Sorting” is the process of re-arranging the rows of your data from low to high, or high to low. “Filtering” is the process of picking out items from your data using specific criteria.

Sorting and filtering can:

• Narrow your focus to specific items that you want to examine in your story.
• Show you rows containing the highest and lowest values of any column. That can be news or it can be errors or other problems with the data.
• Let you answer quick “how many?” questions, with a count of the rows that match your criteria. (In the next lesson, you’ll see that counting this way isn’t always the most efficient way to answer your “how many?” questions.)

Don’t underestimate the power of a simple filter and sort to help you hone in on anecdotes that will make your story come alive, by finding the oldest or youngest, the nearest or farthest, or the smallest or largest examples from your data.