Data Normalization Best Practices
Thresholds and Aggregation
Normalized visits may show some volatility due to the nature of our normalization process. Foursquare recommends analyzing visits by venue and month, rather than daily or weekly metrics, for reasonably stable patterns. For example, we may see a visit from a user with a large weighting (SAG score) because that user is a part of a highly underrepresented demographic in our panel, and this can lead to occasional anomalous spikes in visits, sometimes even for venues with a large number of raw visits. This volatility is particularly evident at granular reporting levels (e.g. venue-level, daily) where visit volume may be sparser.
Minimum Raw Visit Thresholds
Minimum raw visit thresholds should be used to determine feasibility. At low raw visit volumes, normalized visit trends can become noisier and less reliable. By assessing variation in venue level patterns against overall chain and category level patterns, we can further expand the number of feasible venues with a lower raw minimum, while excluding venues with potentially volatile, anomalous monthly trending over time.
The following thresholds are derived by assessing deviations of foot traffic indices at the venue-level against broader chain-level indices, flagging venues that have unreasonably high average deviations from chain patterns over the last 12 months.
|Monthly Visit Feeds*||Minimum Raw Visits||Maximum Variation|
|High Confidence||> 75 average raw visits/ month||Raw Visits: Monthly Standard deviation < 100% of mean SAG Score: Standard Deviation <350|
|Medium Confidence||> 30 and <=75 average raw visits/month||Raw Visits: Monthly Standard Deviation <70% of Mean SAG Score: Mean <300 & Std <300|
|Low Confidence||Venues that do not meet above criteria|
These thresholds are provided as a general guideline for a broad set of large chains and categories. When analyzing a specific set of venues, we may recommend applying different thresholds, depending on the vertical and geographic skews of the specific venues. As a final filter, Foursquare also recommends omitting any individual SAG scores that exceed 1,000 prior to aggregating to a venue, month level.
Updated 8 months ago