Partner Places Ingestion

Partner Places Ingestion is an Offline Job that ingests partner-supplied place records into the Foursquare places dataset. This offline job is considered complete after the harmonization step in our ingestion pipeline, which will match your records to existing places in our dataset. The output of this job will be a JSON storing stats about the ingestion. See Output for more details.

In order to see if an entry in your input dataset has been matched to a new place in our Foursquare Places dataset, you can use the Resolve Endpoint. This will allow you to pass in a foreign_id to a place from your input dataset, and will return the matching Foursquare Place if one exists.

Before delivering data, review the Data Contribution Guidelines for detailed expectations on formatting, chain contributions, and data quality.

Input

Expectations

Deliveries for a single day are expected to be in parquet format with consistent schemas across all files in accordance with the list of fields enumerated below.
Strings should be utf-8 encoded, all other columns should adhere to their specific schema requirements. This will ensure that data processing is not unnecessarily delayed by data corruption and schema incompatibility.
It is expected that the foreign_id will be a unique and stable identifier for each record across all files.
Any records within the delivery containing a duplicate foreign_id will result in a failed job, and you will see this as an error message in the Status endpoint.

Updates

For each delivery, you are expected to send a full snapshot of your dataset.
Each time you provide an updated delivery, we will replace the data from the previous delivery with the data from the updated delivery, based on the provided foreign_id. Therefore, each delivery you make must include the full information about the place, not just the diff or update. Files must be refreshed at least once every 90 days.

Supported File Types

parquet

For best results, please keep missing values as null over empty strings.

Field Reference

Field Name	Type	Description	Required
`address`	`string`	Street address of the place. See Best Practices. Ex: `50 W 23rd St Fl 8`	true
`category_labels`	`array<string>`	Category labels for the place. If IDs are provided, they take precedence. Labels should only be provided if you don't know the appropriate ID. Ex: `["coffee shop"]`	false
`chain_store_id`	`string`	The official store number of a multi-store chain according to that chain. This should not be the data provider's own internal id unless that is also the official store number. Ex: `1234`	false
`closed`	`boolean`	Whether the place is still in business or exists. If blank, the place will be assumed to still be open. This should not be used to indicate temporary closures, only permanent closures. Only set this to `true` if the place is truly closed, and not if you no longer manage the listing.	false
`country`	`string`	ISO 3166-1 alpha-2 country code. Ex: `US`	true
`foreign_chain_id`	`string`	A stable identifier for a chain that the place belongs to. See Chain Contributions for more details. Ex: `11112`	false
`fsq_category_ids`	`array<string>`	IDs of appropriate categories for this place according to our taxonomy, if known. Ex:`["4bf58dd8d48988d110951735"]`	false
`foreign_id`	`string`	Your unique identifier for the place. Must be unique across every row in all deliveries — duplicates fail the entire file (see Input Validation). Do not include spaces or slashes; this field should be consistent from delivery to delivery and will be used to create a link to a place. Ex: `chain-x-1234`	true
`hours`	`map<string,array<array`	Hours of operation. Ex: `{"monday":[["9:00","17:00"]],"tuesday":[["9:00","17:00"]]}`	false
`latitude`	`double`	Latitude in decimal degrees. Roof top or front door geocodes preferred. Ex: `40.742241903`	true
`locality`	`string`	City, town, or equivalent of the place. Ex: `New York`	true
`longitude`	`double`	Longitude in decimal degrees. Roof top or front door geocodes preferred. Ex: `-73.991716687`	true
`name`	`string`	Name of the place. See Best Practices. Ex: `Foursquare HQ`	true
`neighborhoods`	`array<string>`	An array of the neighborhood(s) or other informal geography in which this entity is found. Ex: `["East Side", "Midtown", "Murray Hill"]`	false
`postcode`	`string`	Postcode, zipcode, or equivalent. Ex `10010`	false
`region`	`string`	State, province, territory, or equivalent. For US states, expects standard 2-letter abbreviated state code. Ex: `NY`	false
`tel`	`string`	The primary phone number. Prefer local numbers over national or corporate numbers. Ex: `(310) 286-9400`	false
`website`	`string`	The official website. See Best Practices. Ex: https://foursquare.com	false

Validation Rules

You must make sure your input follows the below validation rules for a successful run:

You cannot have duplicate foreign_ids in your input data.
You must make sure there is at least one valid row.
You must follow the above schema.

Output

Currently, the job will only output metrics after the harmonization step in the partner places ingestion pipeline. The output file is named metrics.json and will live under the output prefix provided to you in the initialize endpoint.

Note if you retry this job, your output metrics will get overwritten.

Output File Format

The metrics will be of type JSON.

Fields Reference

Field Name	Description
`received`	Total number of rows in the input data.
`accepted`	Total number of rows that passed validation.
`rejected`	Total number of rows that failed the validation step.
`unchanged`	Total number of rows whose `foreign_id` and payload match one of your previous deliveries. Not further processed. On your first ingestion, this should equal 0.
`updated`	Total number of rows whose `foreign_id` was seen in one of your previous delivers, but whose payload differs. These rows are re-validated and re-matched. On your first ingestion, this should equal 0.
`new`	Total number of rows whose `foreign_id` was not seen in any of your previous deliveries. On your first ingestion, this should be equal to `received`.
`previously_matched`	Total number of `new` and `updated` rows that were already matched to a Foursquare place from a prior ingestion. No new matching decision needed.
`newly_matched`	Total number of `new` and `updated` rows that were newly matched to a Foursquare place during this ingestion.
`unmatched`	Total number of `new` and `updated` rows that still have no match to a Foursquare place. The number of newly created places during this ingestion will be at most `unmatched`.