ColeslawNearMe Maybe Dead..

So I have made some considerable progress on my Coleslaw Maps App over the past few weeks.

I’ve designed it, created a script to populate some initial coleslaw restaurants and ratings, and successfully deployed it to a real website that you can actually visit. It’s pretty cool and I’m pretty proud of myself.

Now that I’ve gotten it deployed with a test set of 100 restaurants in Chicago, the next (and biggest) step is to figure out how to get coverage for ALL restaurants that have coleslaw throughout the entire USA. A big task.

This is where I’m realizing we may have a serious problem, and that the whole website may be dead on arrival. Here’s the issue:

I’ve been using the Google Places API (which is the database behind Google Maps) to find restaurants and detect if they have coleslaw.

Today I finally used up all of my free-trial Google Places API credits, with only about 400 restaurants in coverage. Looking at my coleslaw restaurant-finding script, I’ve realized:

1. Google doesn’t directly include restaurant menus in their database

2. The fields it does include (name, description, reviews) don’t reliably mention coleslaw even for known coleslaw restaurants (ex: KFC)

3. Either way, the cost to go through every restaurant in the US would be insane.

Doing the math here, if I used the Google API to go through every US restaurant’s description and reviews to see if they mention coleslaw, it would cost me ~$16,175

Unfortunately, I don’t have that kind of coleslaw cash these days.

And even if I did, Google Places just isn’t a reliable way to identify if a restaurant has coleslaw on the menu. We need to use a different strategy to go over menus directly and identify if coleslaw is listed. There are a few ways to do this:

1. Manual Chains: Start with just chain restaurants (~40% of all restaurants), build a list manually, and go through all their menus

2. User Submitted: Go fully user-submitted and just let people add the restaurants for me when they visit my website (ie. the Wikipedia model) (Coleslawpedia does have a good ring to it…)

3. First-Party Scraping: Write a script to just scrape every restaurant’s website, find the menu, and look for coleslaw

4. Third-Party Menu Aggregators: Use a third-party that has already done the work of aggregating menus and search over their APIs (ex: delivery apps like Grubhub and Doordash, or review apps like Yelp and OpenTable)

There are some serious pro’s and con’s here. Manually finding chains or doing User submissions would be the most cost-effective, but would be the slowest and would miss the majority of restaurants. Scraping and/or Aggregators would be much more reliable and quicker, but would still cost money.

So… I think the best place to start would be testing out a 3rd party API and seeing how cost-effective it would be. The next question then is which API is best? We have about 6 options: UberEats, SinglePlatform, Doordash, Grubhub, Yelp, and MenuAPI.

Some key considerations: can I search menu items directly (ex: search=coleslaw) or do I have to parse the entire menu? How much does it cost per search? What % of restaurants do they cover? How often do they update their data?

Here’s how they compare:

Overall there’s no obvious one-size-fits-all approach here. And the results vary especially by metro vs. suburban vs. rural. If I wanted to get every restaurant, I’d likely have to use a hybrid approach across multiple APIs and methods.

Right now I’d imagine it could look like this:

1. Start in one state or region and create a manual list of chains with known coleslaw on the menu to knock out the big ones (KFC, Popeyes, Canes, etc.)

2. From there, use the cheaper third-party API’s to get any restaurants in urban and suburban areas

3. Rely on user-submissions to get restaurants that don’t show up in those two (ie. independent, small-town restaurants)

Long story short… getting reliable coverage of every restaurant in the US with coleslaw on the menu is not as trivial as I thought, and a lot more expensive.

When I started this project I didn’t really have the foresight to understand this, and now that I’m here, I’m wondering how much effort is worthwhile to fill in the blanks.

So my friends, unless I can find an affordable way to do this for enough restaurants, this could mean the end of our coleslaw journey.

Stay tuned,

Ramsey