A Skeptic’s Guide to ROC’s Half-Baked “Diners’ Guide”

This week marks the Los Angeles release of ROC’s annual “Diners’ Guide,” which ranks restaurants on their labor practices instead of the food and service they offer. (ROC, for those unfamiliar with the group, is an organized labor front group masquerading as a restaurant employment center.)

Given ROC’s own history of poor labor practices and the demonstrated failure of its restaurant business model, it’s the last group that should be lecturing others on their approach to human resources. But in this case, it’s not just ROC’s credibility that’s in question—it’s the credibility of their entire guide.

For their Diners’ Guide, ROC follows a four-step research process:

1. Recruit Unpaid Workers to Help Collect the Data 

It’s true – while ROC’s guide puts a premium on the wages and benefits that restaurants pay their employees, this concern for compensation doesn’t extend to the people who are putting together the guide itself.  As evidenced by job posts the organization posted this fall, the telephone calls and data collection for the project were done by unpaid interns.

2. Use a Tiny Sample of Restaurants to Make Broad Claims about All of Them

ROC claims that they asked for wage and benefit information from the country’s “top 150 most popular restaurants.”  But this isn’t quite accurate: In the back of their guide, ROC reveals that data for any chain or franchise restaurants came from “individual restaurant locations in Texas, Louisiana, South Carolina, and Alabama.” (ROC’s guide doesn’t indicate whether their researchers spoke with an owner, a store manager, or an employee who happened to be on shift when they called.)

It’s a telling admission: Restaurants identified as “national” in ROC’s guide, like IHOP and Denny’s and Subway, are owned primarily (or even entirely) by franchisees. Interviewing one restaurant in one region of the country and acting as though the pay, benefits, and potential for promotion there are representative of the entire country is not just disingenuous—it’s flat-out dishonest.

ROC tries to partially cover itself, adding a new line in their 2013 guide to clarify that its guide reflects the “lowest wages and benefits offered to any worker in a particular company.” But even this rings false: For Houlihan’s restaurant, ROC chooses to single out one franchised location in Chicago for praise–a location that signed on as a ROC-favored employer–while criticizing the rest of the chain.

3. Give Yourself and Your Allies the Best Ratings

The best ratings in ROC’s guide are reserved for restaurants that ROC itself has a stake in, or those small operations that have signed on to be “high road” partners in ROC’s ideological agenda. No national chain received a favorable rating from ROC.

This biased rating system creates some confusing results. For instance, one “cake boutique” in Chicago that offers neither the pay nor benefits that ROC declares sufficient is nonetheless awarded a silver prize in the guide because the restaurant signed on to be a ROC partner. Ditto for a vegetarian restaurant in Detroit. Yet, a national chain restaurant with a similar rating profile received no such distinction because they failed to sign on to ROC’s partner program.

Similarly, restaurants that ROC was unable to collect any information about are still included in the guide, to indicate to readers that the restaurant failed to sign on as a part of ROC’s industry roundtable. And failing to renew your “membership” brings consequences: Five Guys Burgers and Fries was a gold medal winner in 2012, receiving praise for their benefits package, promotion potential, and participation in ROC’s restaurant roundtable. This year, Five Guys was no longer participating in ROC’s programs–and their favorable ratings disappeared.

4. Present Your Findings to the Media, But Don’t Release Interview Data or Questions

Keeping with tradition, ROC’s Diners’ Guide hit the media with maximum hype and minimum detail. ROC omitted the survey questions from their report, as well as any information about who was interviewed at the restaurants and what sort of data they provided. Consumers are, presumably, supposed to take ROC’s word that it isn’t cooking the books.

Loose standards like this would be unacceptable in any serious research project, and it goes double in ROC’s case. On prior research projects, the group has instructed researchers to “know the agenda” of the project, and to “prompt people” to get the best quotes. Given ROC’s well-known agenda, as well as their history of playing fast and loose with the facts, ROC’s reports aren’t worthy of receiving the benefit of the doubt.

You can find a visual representation of ROC’s research methods here.