This page contains the full documentation for each function of the Studio Data SDK. Use the sidebar to navigate to any function.

Map Functions

This section contains detailed reference documentation covering the Data SDK's map functions.

To view each function with a brief description and an example, visit the map functions documentation.

create_map

Create a map record from JSON, including the map configuration and list of associated datasets.

data_sdk.create_map(
      name: Optional[str] = None,
      description:  Optional[str] = None,
      map_state: Optional[MapState] = None,
      datasets: Optional[Iterable[Union[Dataset, UUID, str0]]])

uf-data-sdk create-map \
      --name <name> \
      --description <description> \
      --map-state <path> \
      --dataset-ids <uuid1>,<uuid2>

POST https://data-api.foursquare.com/v1/maps/ HTTP/1.1

Body

The body of the request should be the JSON data for the map record you want to create. All properties are optional, and unknown properties or those which cannot be updated will be ignored. In order to refer to datasets in the map state, they must be included in the datasets list, which can be either a list of dataset UUIDs or a list of objects in the form {"id": "string"}.

The map state is fully documented in the Studio Map Configuration format specification.

Response

Updated map record

{
  "id": "string",
  "name": "string",
  "createdAt": "2020-11-03T21:27:14.000Z",
  "updatedAt": "2020-11-13T01:44:07.000Z",
  "description": "string",
  "privacy": "private",
  "permission": "editor",
  "latestState": {
    "id": "string",
    "data": MapConfig
  },
  "datasets": [
    {
      "id": "string",
      "name": "string",
      "createdAt": "2020-11-10T18:09:39.000Z",
      "updatedAt": "2020-11-10T18:09:39.000Z",
      "privacy": "private",
      "permission": "editor",
      "isValid": true
    }
  ]
}

Example

unfolded_map = data_sdk.create_map(
      name="map name",
      description="map description",
      map_state={"id": "<uuid>", "data": {...}},
      datasets=['<uuid1>', '<uuid2>'])

uf-data-sdk create-map \
      --name "map name" \
      --description "map description" \
      --map-state /path/to/map-state.json \
      --dataset-ids <uuid1>,<uuid2>

curl -X POST https://data-api.foursquare.com/v1/maps/ \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  --data-binary '@/path/to/my_map.json'

copy_map

Creates a copy of an existing map, duplicating its layers and map state.

The user must choose whether to copy the target map's datasets or point to them as a data source.

Method

data_sdk.copy_map(
      map: Union[Map, str, UUID],
      copy_datasets: bool,
      name: Optional[str] = None
      ) -> Map:

uf-data-sdk clone-map \
      --map-id <map-id> \
      --clone-datasets/--no--clone--datasets \
      --name <name>

POST https://data-api.foursquare.com/v1/maps/<uuid>/clone HTTP/1.1

Positional Parameters

Parameter	Type	Description
`map`	`Map`, `string`, or `UUID`	The map record to copy. Can be a `Map` object representing a created map or a `string`/`UUID` id pointing to an existing map.

Keyword Parameters

Parameter	Type	Description
`copy_datasets`	`bool`	Required. If `true`, copy all underlying datasets of the target map.
`name`	`string`	The name to give the copied map. Default: "Copy of {source_map_name}"

Response

{
  "id": "string",
  "name": "string",
  "createdAt": "2020-11-03T21:27:14.000Z",
  "updatedAt": "2020-11-13T01:44:07.000Z",
  "description": "string",
  "privacy": "private",
  "permission": "editor",
  "latestState": {
    "id": "string",
    "data": MapConfig
  },
  "datasets": [
    {
      "id": "string",
      "name": "string",
      "createdAt": "2020-11-10T18:09:39.000Z",
      "updatedAt": "2020-11-10T18:09:39.000Z",
      "privacy": "private",
      "permission": "editor",
      "isValid": true
    }
  ]
}

Examples

unfolded_map = data_sdk.copy_map(
      map = "<uuid>",
      copy_datasets = True,
      name = "My Copied Map Example")

uf-data-sdk clone-map \
      --map-id "<uuid>" \
      --clone-datasets/--no--clone--datasets \
      --name "My Cloned Map Example"

curl -X POST https://data-api.foursquare.com/v1/maps/<uuid>/clone \
    -H 'Authorization: Bearer <token>

replace_dataset

Replace a dataset on a map, updating the visualization with the data from the new dataset.

By default, this function expects a dataset with an identical schema and will error if the new dataset is not compatible with the old one. To override the error, set force = True.

Method

data_sdk.replace_dataset(
        map: Union[Map, str, uuid.UUID],
        dataset_to_replace: Union[Dataset, str, uuid.UUID],
        dataset_to_use: Union[Dataset, str, uuid.UUID],
        force: bool = False
        ) ‑> Map

uf-data-sdk replace-dataset \
      --map-id, \
      --dataset-to-replace-id \
      --dataset-to-use-id
      --force

POST https://data-api.foursquare.com/v1/maps/<uuid>/datasets/replace HTTP/1.1

Positional Parameters

Parameter	Type	Description
`map`	`Map`, `string`, or `UUID`	Required. The map record containing the dataset to replace. Can be a `Map` object representing a created map or a `string`/`UUID` id pointing to an existing map.
`dataset_to_replace`	`Dataset`, `string`, or `UUID`	Required. The dataset to replace. Can be a `Dataset` object representing a dataset or a `string`/`UUID` id pointing to an existing dataset.
`dataset_to_use`	`Dataset`, `string`, or `UUID`	Required. The new dataset to use in the replace operation. Can be a `Dataset` object representing a dataset or a `string`/`UUID` id pointing to an existing dataset.

Keyword Parameters

Parameter	Type	Description
`force`	`bool`	If `True`, force the dataset replacement operation, overriding any errors caused by mismatched schemas. Default: `False`

Response

The Map object that was operated on.

{
  "id": "string",
  "name": "string",
  "createdAt": "2020-11-03T21:27:14.000Z",
  "updatedAt": "2020-11-13T01:44:07.000Z",
  "description": "string",
  "privacy": "private",
  "permission": "editor",
  "latestState": {
    "id": "string",
    "data": MapConfig
  },
  "datasets": [
    {
      "id": "string",
      "name": "string",
      "createdAt": "2020-11-10T18:09:39.000Z",
      "updatedAt": "2020-11-10T18:09:39.000Z",
      "privacy": "private",
      "permission": "editor",
      "isValid": true
    }
  ]
}

Examples

data_sdk.replace_dataset(
      map_id = "38bbed5-eb0e-4c65-8bcc-cc173dc497qb",
      dataset_to_replace = "750dfn07-f8b9-4d37-b698-bacd1d8e6156",
      dataset_to_use =  "c9ff8f3e-8821-4k68-b7fc-94cb95fe65e2"

curl -X POST https://data-api.foursquare.com/v1/maps/<uuid>/datasets/replace \
    -H 'Authorization: Bearer <token>' \

curl -X POST https://data-api.foursquare.com/v1/maps/<uuid>/clone \
    -H 'Authorization: Bearer <token>

get_map_by_id

Get a map record by id.

unfolded_map = data_sdk.get_map_by_id(uuid: str) -> Map

uf-data-sdk get-map <uuid>

GET https://data-api.foursquare.com/v1/maps/<uuid> HTTP/1.1

Parameters

Parameter	Type	Description
`id`	`string`	The UUID of the map record to get.

Response

Map record, including the full map state and a list of associated datasets. The map state (the configuration of the map styles, layers, etc) is omitted in the sample record below due to its complexity.

{
  "id": "string",
  "name": "string",
  "createdAt": "2020-11-03T21:27:14.000Z",
  "updatedAt": "2020-11-13T01:44:07.000Z",
  "description": "string",
  "privacy": "private",
  "permission": "editor",
  "latestState": {
    ...
  },
  "datasets": [
    {
      "id": "string",
      "name": "string",
      "createdAt": "2020-11-10T18:09:39.000Z",
      "updatedAt": "2020-11-10T18:09:39.000Z",
      "privacy": "private",
      "permission": "editor",
      "isValid": true
    }
  ]
}

Example

unfolded_map = data_sdk.get_map_by_id("<uuid>")

uf-data-sdk get-map <uuid>

curl -X GET https://data-api.foursquare.com/v1/maps/<uuid> \
  -H 'Authorization: Bearer <token>'

update_map

Update a map record, including the latest state and list of associated datasets.

update_map(
      map_id: Union[Map, UUID, str],
      name: Optional[str] = None,
      description: Optional[str] = None,
      map_state: Optional[MapState] = None,
      datasets: Optional[Iterable[Union[Dataset, UUID, str]]] = None) -> Map:

uf-data-sdk update-map \
      --map-id <uuid> \
      --name <name> \
      --description <description> \
      --map-state <path> \
      --dataset-ids <uuid1>,<uuid2>

PUT https://data-api.foursquare.com/v1/maps/{id} HTTP/1.1

Parameters

Parameter	Type	Description
`id`	`string`	The UUID of the map record to update.
`name`	`string`	A new name for the map.
`description`	`string`	A new description for the map.
`map_state`	`MapState`	The latest `MapState` of the Studio map object.
`datasets`	`Dataset` list	A list of `Dataset` objects associated with the map.

Body

The body of the request should be the JSON data for the map record you want to update. All properties are optional, and unknown properties or those which cannot be manually updated will be ignored. In order to refer to datasets in the map state, they must be included in the datasets list, which can be either a list of dataset UUIDs or a list of objects in the form {"id": <uuid>}.

The map state is fully documented in the Studio Map Configuration format specification.

Response

Updated map record

{
  "id": "string",
  "name": "string",
  "createdAt": "2020-11-03T21:27:14.000Z",
  "updatedAt": "2020-11-13T01:44:07.000Z",
  "description": "string",
  "privacy": "private",
  "permission": "editor",
  "latestState": {
    "id": "string",
    "data": MapConfig
  },
  "datasets": [
    {
      "id": "string",
      "name": "string",
      "createdAt": "2020-11-10T18:09:39.000Z",
      "updatedAt": "2020-11-10T18:09:39.000Z",
      "privacy": "private",
      "permission": "editor",
      "isValid": true
    }
  ]
}

Example

data_sdk.update_map(
    map_id = map.id,
    map_state = {
        "id": map.latest_state.id,
        "data": map.latest_state.data
    }
)

uf-data-sdk update-map \
      --map-id "map-uuid" \
      --name "new name" \
      --description "new description" \
      --map-state map-state.json \
      --dataset-ids <uuid1>,<uuid2>

curl -X PUT https://data-api.foursquare.com/v1/maps/<uuid> \
    -H 'Authorization: Bearer <token>' \
    -H 'Content-Type: application/json' \
    --data-binary '@/path/to/my_map.json'

delete_map

Delete a map record by id. This will not delete datasets associated with the map.

data_sdk.delete_map(uuid: str) -> None

uf-data-sdk delete-map <uuid>

DELETE https://data-api.foursquare.com/v1/maps/<uuid> HTTP/1.1

Parameters

Parameter	Type	Description
`id`	`string`	The UUID of the map record to delete.

Response

A message indicating if deletion was successful.

{
  "message": "string"
}

Example

# List maps on account
maps = data_sdk.list_maps()
# Select map, then delete
map_to_delete = maps[0]
data_sdk.delete_map(map)

uf-data-sdk delete-map <uuid>

curl -X GET https://data-api.foursquare.com/v1/maps/<uuid> \
  -H 'Authorization: Bearer <token>'

list_maps

Get all map records for the authenticated user.

maps = data_sdk.list_maps()

uf-data-sdk list-maps

GET https://data-api.foursquare.com/v1/maps HTTP/1.1

Get all map records for the organization of the authenticated user.

org_maps = data_sdk.list_maps(organization=True)

uf-data-sdk list-maps --organization

GET https://data-api.foursquare.com/v1/maps/for-organization HTTP/1.1

Named Parameters

Parameter	Type	Description
`organization`	`boolean`	If `True`, list map records for organization of authenticated user.

Response

List of map records.

{
  "items": [
    {
      "id": "string",
      "name": "string",
      "createdAt": "2020-11-03T21:27:14.000Z",
      "updatedAt": "2020-11-13T01:44:07.000Z",
      "description": "",
      "privacy": "private",
      "permission": "editor"
    }
  ]
}

For non-enterprise users, organization=True will cause the request to fail with:

403: Insufficient permission level to perform this action

Example

maps = data_sdk.list_maps()

uf-data-sdk list-maps

curl https://data-api.foursquare.com/v1/maps -H 'Authorization: Bearer <token>'

Data Functions

This section contains detailed reference documentation covering the Data SDK's dataset functions.

To view each function with a brief description and an example, visit the data functions documentation.

upload_file

Create a dataset from a data upload.

# upload a file
data_sdk.upload_file(
  self,
  file: Union[BinaryIO, str, Path],
  name: Optional[str] = None,
  *,
  dataset: Optional[Union[Dataset, str, UUID]] = None,
  media_type: Optional[Union[str, MediaType]] = None,
  description: Optional[str] = None,
  chunk_size: int = 256 * 1024,
  progress: bool = False,
) -> Dataset:
# upload pandas DataFrame or geopandas GeoDataFrame
data_sdk.upload_dataframe(
  self,
  df: Union["pd.DataFrame", "gpd.GeoDataFrame"],
  name: Optional[str] = None,
  index: bool = True,
  **kwargs: Any,
) -> Dataset:

uf-data-sdk upload-file --name <name> --desc <description> --media-type <media_type> <path>

POST https://data-api.foursquare.com/v1/datasets/data?name={name}&description={description}
HTTP/1.1

Parameters

HTTP API

Parameter	Type	Description
`name`	`string`	Name of the dataset to create.
`description`	`string`	Optional. Description of the dataset to create.

Headers

Header	Description
`Content-Type`	Required. MIME type of data you are uploading, e.g. `text/csv` or `application/json`

Body

The body of the request should be the binary data you want to upload, in a format matching the supplied Content-Type.

Python

Use upload_file for uploading data files. Use upload_dataframe for uploading a Pandas DataFrame or GeoPandas GeoDataFrame.

`upload_file`

Positional Arguments

Argument	Type	Description
`file`	`string` of a path, or `file` object	Path or file object to use for uploading data.
`name`	`string`	Optional. Name of the dataset to create.

Keyword Arguments

Argument	Type	Description
`dataset`	`Dataset`, `str`, `UUID`	Optional. If provided, dataset whose data should be updated. Otherwise, creates a new dataset.
`media_type`	`string` or `MediaType`	Optional. File type (e.g. `MediaType.CSV` or `text/csv`). By default, tries to infer media type from file name.
`description`	`string`	Optional. Description of the dataset to create.
`chunk_size`	`int`	Optional. Number of bytes to upload at a time. Used for `progressbar`. Default: 256 * 1024
`progress`	`bool`	Optional. When true, display a progress bar.

`upload_dataframe`

Positional Arguments

Argument	Type	Description
`df`	`pandas.DataFrame` or `geopandas.GeoDataFrame`	Either a Pandas `DataFrame` or a GeoPandas `GeoDataFrame` to upload to Studio.
`name`	`string`	Name of dataset record. Required if creating a new dataset record (instead of updating an existing dataset record).
`index`	`boolean`	Optional. If `True`, include row names in output. Default: `True`
**kwargs	Any	Optional. Keyword arguments to pass to upload_file

The map state is fully documented in the Studio Map Configuration format specification.

Response

Created dataset record

{
  "id": "string",
  "name": "string",
  "createdAt": "2021-02-04T00:17:38.652Z",
  "updatedAt": "2021-02-04T00:17:38.652Z",
  "description": "string",
  "isValid": true
}

Example

# upload a file
data_sdk.upload_file(
    file='new_file.csv',
    name='Dataset name',
    media_type=MediaType.CSV,
    description='Dataset description')
# upload pandas or geopandas dataframe
data_sdk.upload_dataframe(
    dataframe,
    name='Dataset name',
    description='Dataset description')

uf-data-sdk upload-file \
  --name "Dataset name" \
  --desc "Dataset description" \
  --media-type text/csv \
  new_file.csv

curl -X POST https://data-api.foursquare.com/v1/datasets/data?name=My+Dataset \
-H 'Authorization: Bearer <token>' \
-H 'Content-Type: text/csv' \
--data-binary '@/path/to/my_dataset.csv'

get_dataset_by_id

Retrieve a dataset metadata record in JSON format.

data_sdk.get_dataset_by_id(dataset: Union[Dataset, str, UUID]) -> None

uf-data-sdk get-dataset --dataset-id <uuid>

GET https://data-api.foursquare.com/v1/datasets/<uuid> HTTP/1.1

Parameters

Parameter	Type	Description
`dataset`	`dataset` object or `string`	`dataset` object or UUID of the dataset record to retrieve.

Response

Dataset record

{
  "id": "string",
  "name": "string",
  "createdAt": "2021-02-04T00:17:38.652Z",
  "updatedAt": "2021-02-04T00:17:38.652Z",
  "description": "string",
  "isValid": true
}

Example

# List and select dataset
datasets = data_sdk.list_datasets()
dataset = datasets[0]
# Get dataset record
data_sdk.get_dataset_by_id(dataset)

uf-data-sdk get-map <uuid>

curl -X GET https://data-api.foursquare.com/v1/datasets/<uuid> \
  -H 'Authorization: Bearer <token>'

download_dataset

Download data from a dataset record by id.

data_sdk.download_dataset(
    dataset: Union[Dataset, str, UUID],
    output_file: Optional[Union[BinaryIO, str, Path]] = None) -> Optional[bytes]
data_sdk.download_dataframe(
    dataset: Union[Dataset, str, UUID]) -> Union[pandas.DataFrame, geopandas.GeoDataFrame]

uf-data-sdk download-dataset --dataset-id <uuid> --output-file <path>

GET https://data-api.foursquare.com/v1/datasets/<uuid>/data HTTP/1.1

Parameters

`download_dataset`

Parameter	Type	Description
`dataset`	`dataset` object or `string`	`dataset` object or UUID of the dataset record to download.
`output_file`	`string` of a path, or `file` object	If provided, a path or file object to write dataset's data to. Otherwise will return a bytes object with the dataset's data.

`download_dataframe`

Parameter	Type	Description
`dataset`	`dataset` object or `string`	`dataset` object or UUID of the dataset record to download.

Response

`download_dataset`

If output_file was provided, returns None and writes data to the provided file. Otherwise, returns a bytes object with the dataset's data.

`download_dataframe`

Returns either a pandas DataFrame or a geopandas GeoDataFrame. If the original dataset was a CSV file, a pandas DataFrame will be returned. If it was a GeoJSON file, a geopandas GeoDataFrame will be returned.

Example

datasets = data_sdk.list_datasets()
dataset = datasets[0]
# download to local file
data_sdk.download_dataset(dataset, output_file='output.csv')
# download to buffer
buffer = data_sdk.download_dataset(dataset)
# download to a dataframe
df = data_sdk.download_dataframe(dataset)

uf-data-sdk download-dataset --dataset-id <uuid> --output-file output.csv

curl -X GET https://data-api.foursquare.com/v1/datasets/<uuid> \
  -H 'Authorization: Bearer <token>'

delete_dataset

Delete a dataset record by id. This will also delete any data associated with the dataset.

❗️
Warning
This operation cannot be undone. If you delete a dataset currently used in one or more maps, the dataset will be removed from those maps, possibly causing them to render incorrectly.

data_sdk.delete_dataset(dataset: Union[Dataset, str, UUID]) -> None

uf-data-sdk delete-dataset --dataset-id <uuid>

DELETE https://data-api.foursquare.com/v1/datasets/<uuid> HTTP/1.1

Parameters

Parameter	Type	Description
`dataset`	`string` or `dataset` object	The UUID or `dataset` object of the dataset to delete.

Response

A message indicating if deletion was successful.

{
  "message": "string"
}

Example

datasets = data_sdk.list_datasets()
dataset = datasets[0]
data_sdk.delete_dataset(dataset)

uf-data-sdk delete-dataset --dataset-id <uuid>

curl -X DELETE https://data-api.foursquare.com/v1/datasets/<uuid> \
  -H 'Authorization: Bearer <token>'

list_datasets

Get all dataset records for the authenticated user.

datasets = data_sdk.list_datasets()

uf-data-sdk list-datasets

GET https://data-api.foursquare.com/v1/datasets HTTP/1.1

Get all map records for the organization of the authenticated user.

datasets = data_sdk.list_datasets(organization=True)

uf-data-sdk list-datasets --organization

GET https://data-api.foursquare.com/v1/datasets/for-organization HTTP/1.1

Named Parameters

Parameter	Type	Description
`organization`	`boolean`	If `True`, list dataset records for organization of authenticated user.

Response

List of dataset records

{
  "items": [
    {
      "id": "string",
      "name": "string",
      "createdAt": "2021-02-03T23:51:14.527Z",
      "updatedAt": "2021-02-03T23:51:14.527Z",
      "description": "string",
      "isValid": true
    }
  ]
}

For non-enterprise users, organization=True will cause the request to fail with:

403: Insufficient permission level to perform this action

Example

datasets = data_sdk.list_datasets()

uf-data-sdk list-datasets

curl https://data-api.foursquare.com/v1/datasets -H 'Authorization: Bearer <token>'

Hex Tile Functions

This section contains detailed reference documentation covering the Data SDK's Hex Tile functions.

To view each function with a brief description and an example, visit the Hex Tile functions documentation.

create_hextile

Data can be processed into Hex Tiles using the Studio Data SDK.

# hextile a file
  data_sdk.create_hextile(
      source: Union[Dataset, str, UUID],
      *,
      target: Optional[Union[Dataset, str, UUID]] = None,
      source_hex_column: Optional[str] = None,
      source_time_column: Optional[str] = None,
      time_intervals: Optional[Sequence[Union[TimeInterval, str]]] = None,
      target_res_offset: Optional[int] = None,
      _tile_mode: Optional[Union[TileMode, str]] = None,
      output_columns: Optional[
          Sequence[Union[dict, HexTileOutputColumnConfig]]
      ] = None,
      _positional_indexes: Optional[bool] = None,
  ) -> Dataset

POST https://data-api.foursquare.com/internal/v1/datasets/hextile HTTP/1.1

Python

You can access the Studio Data SDK with Python to process your dataset into Hex Tiles.

Imports for Optional Parameters

To specify time_intervals, you may import a custom TimeInterval enum from the Studio Data SDK.

Import	Description
`from unfolded.data_sdk.enums import TimeInterval`	Time granularities for Hex Tiles. Accepted intervals: `YEAR`, `MONTH`, `DAY`, `HOUR`, `MINUTE`, `SECOND`. Example usage: `TimeInterval.HOUR`

create_hextile can create new aggregated data columns during Hex Tile generation. To format output_column, then specify AggregationMethod and Dtype, you may import custom classes defined in the Studio Data SDK.

Import	Description
`from unfolded.data_sdk.models import HexTileOutputColumnConfig`	Model used to format `output_column` paramaters. See Python example below.
`from unfolded.data_sdk.enums import AggregationMethod`	Method to aggregate the data column with when generating coarser tile resolutions. Accepted methods: `SUM`, `COUNT`, `MIN`, `MAX`, `MEAN`, `MEDIAN`, `MODE`. Example usage: `AggregationMethod.SUM`
`from unfolded.data_sdk.enums import Dtype`	Data type to encode the aggregated column in the Hex Tile dataset. Accepted data types:`BOOL`, `FLOAT16`, `FLOAT32`, `FLOAT64`, `INT16`, `INT32`, `INT64`, `INT8`, `UINT16`, `UINT32`, `UINT64`, `UINT8`. Example usage: `Dtype.INT32`

`create_hextile` function

Use the create_hextilefunction to create Hex Tiles from a dataset.

Positional Arguments

Argument	Type	Description
`source`	`string`	Required. `Dataset` record or UUID of the dataset to convert to Hex Tile.

Keyword Arguments

Argument	Type	Description
`target`	`string`	`Dataset` record or UUID of an existing Hex Tile dataset to overwrite.
`source_hex_column`	`string`	Name of the hex (h3) column in the source dataset. Hex column must contain hex indexes as `string`.
`source_lat_column`	`string`	Name of the latitude column in the source dataset.
`source_lng_column`	`string`	Name of the longitude column in the source dataset.
`finest_resolution`	`int`	Finest resolution for the data hexes within a tile (when creating a tileset from lat/lng columns).
`source_time_column`	`string`	Name of the time column in the source dataset, or null if non-temporal data.
`time_intervals`	`string array`	Array of time intervals to generate for temporal datasets. Accepted intervals: `["YEAR"]`, `["MONTH"]`, `["DAY"]`, `["HOUR"]`, `["MINUTE"]`, `["SECOND"]`. Import `TimeInterval` to use time intervals enums. Example usage: `TimeInterval.DAY` or ["DAY"]
`output_columns`	`object array`	Object array used to aggregate a new data column during Hex Tile generation. Import `HexTileOutputColumnConfig` to format `output_column` parameters. See Python example below.
`output_columns.source_column`	`string`	Column name in the source dataset.
`output_columns.target_column`	`string`	Column name in the target hex tile dataset.
`output_columns.agg_method`	`string`	Method to aggregate the data column with when generating coarser tile resolutions. Accepted methods: `"sum"`, `"count"`, `"min"`, `"max"`, `"mean"`, `"median"`, `"mode"`. Defaults to `"sum"` for numeric columns. Import `AggregationMethod` to use aggregation method enums. Example usage: `AggregationMethod.SUM` or `"sum"`
`output_columns.dtype`	`string`	Data type to encode the column in the Hex Tile dataset. Example values include `"float32"`, `"uint8"`, `"int64"`. Import `Dtype` to use data type enums. Example usage: `Dtype.INT64` or `"int64"`
`target_res_offset`	`int`	Optional integer controlling the depth of the tile hierarchy.
`tile_mode`	`string`	Experimental. `"dense"`, `"sparse"`, or `"auto"`. Defaults to `"auto"`.
`positional_indexes`	`boolean`	Experimental. Enables the positional indexes encoding feature.

HTTP API

You can access the Studio Data SDK through the HTTP REST API to process your dataset into Hex Tiles.

Headers

Header	Description
`Content-Type`	Must be `application/json`. This header is required.

Body

The body of the request should be the parameters encoded as a JSON blob.

Parameter	Type	Description
`source`	`string`	Required. `Dataset` record or UUID of the dataset to convert to Hex Tile.
`target`	`string`	`Dataset` record or UUID of an existing Hex Tile dataset to overwrite.
`sourceHexColumn`	`string`	Name of the hex (h3) column in the source dataset. Hex column must contain hex indexes as `string`.
`sourceLatColumn`	`string`	Name of the latitude column in the source dataset.
`sourceLngColumn`	`string`	Name of the longitude column in the source dataset.
`finestResolution`	`int`	Finest resolution for the data hexes within a tile (when creating a tileset from lat/lng columns)
`sourceTimeColumn`	`string`	Name of the time column in the source dataset, or null if non-temporal data.
`timeIntervals`	`string array`	Array of time intervals to generate for temporal datasets. Accepted intervals: `["YEAR"]`, `["MONTH"]`, `["DAY"]`, `["HOUR"]`, `["MINUTE"]`, `["SECOND"]`.
`outputColumns`	`object array`	Object array used to aggregate a new data column during Hex Tile generation.
`outputColumns.sourceColumn`	`string`	Column name in the source dataset.
`outputColumns.targetColumn`	`string`	Column name in the target hex tile dataset.
`outputColumns.aggMethod`	`string`	Method to aggregate the data column with when generating coarser tile resolutions. Accepted methods: `"sum"`, `"count"`, `"min"`, `"max"`, `"mean"`, `"median"`, `"mode"`. Defaults to `"sum"` for numeric columns.
`outputColumns.dtype`	`string`	Data type to encode the column in the Hex Tile dataset. Example values include `"float32"`, `"uint8"`, `"int64"`.
`targetResOffset`	`int`	Optional integer controlling the depth of the tile hierarchy.
`tileMode`	`string`	Experimental. `"dense"`, `"sparse"`, or `"auto"`. Defaults to `"auto"`.
`positionalIndexes`	`boolean`	Experimental. Enables the positional indexes encoding feature.

Response

Upon completion, you will receive a response containing the metadata of your dataset.

{
  "id": "string",
  "name": "string",
  "createdAt": "2021-02-04T00:17:38.652Z",
  "updatedAt": "2021-02-04T00:17:38.652Z",
  "description": "string",
  "isValid": true
}

Dataset Location

Once processed, your dataset will be stored on your Studio Cloud account. You may download it using the download dataset function in the Data SDK.

Check Hex Tiling Status via API

You can find the status of the Hex Tile dataset in the API.

Retrieve the dataset's metadata, then find one of three codes in the dataStatus field:

Status code	Description
`pending`	The tiling process is still running.
`ready`	The tiling process is complete and the Hex Tiles can be used.
`error`	The tiling process has failed.

Examples

# Import the HexTileOutputColumn Model from Studio Data SDK
from unfolded.data_sdk.models import HexTileOutputColumnConfig
# Import enum members from Studio Data SDK
from unfolded.data_sdk.enums import (
    AggregationMethod,
    Dtype
)
# Create Hex Tiles
hextile_dataset = data_sdk.create_hextile(
    source="tree_data_set_hex10",
    source_hex_column="hex10",
    output_columns=(
      # Use HexTileOutputColumnConfig
      # to format source and target columns
      # aggregation method, and data type
      HexTileOutputColumnConfig(
          source_column="TreeID",
          target_column="TreeID_sum",
          agg_method=AggregationMethod.SUM,
          dtype=Dtype.UINT16
      ),
    )
)

curl -X POST https://data-api.foursquare.com/internal/v1/datasets/hextile \
-H 'Authorization: Bearer <token>' \
-H 'Content-Type: application/json' \
--data-raw '{
    "source": "<source_dataset_id>",
    "sourceHexColumn": "hex",
    "sourceTimeColumn": "datestr",
    "timeIntervals": ["DAY"],
    "targetResOffset": 4,
    "outputColumns": [
        {
        "sourceColumn": "metric",
        "targetColumn": "metric_sum",
        "aggMethod": "sum",
        "dtype": "uint16"
        }
    ]
}

enrich

Datasets can be enriched with Hex Tiles using the Studio Data SDK.

# enrich a dataframe or existing dataset
data_sdk.enrich(
    dataset: Union[pd.DataFrame, Dataset, UUID, str],
    source_id: UUID,
    source_column: str,
    *,
    h3_column: Optional[str] = None,
    lat_column: Optional[str] = None,
    lng_column: Optional[str] = None,
    time_column: Optional[str] = None,
) -> pd.DataFrame

POST https://data-api.foursquare.com/internal/v1/query HTTP/1.1

HTTP API

Enrichment is provided through the Query API, which can support a range of flexible queries. The following parameters describe a simple enrichment query.

Headers

Header	Description
`Content-Type`	Must be `application/json`. This header is required.
`Accept`	May be `_/_`, `application/json`, `text/csv`, or `application/vnd.apache.arrow.file`. The response dataset will have the corresponding data format (by default, `text/csv`).

Body

The body of the request should be the parameters, encoded as a JSON blob.

Parameters

Parameter	Type	Required	Description
`type`	`string`	Yes	Use `enrich` to select the `enrich` process.
`sourceId`	`string`	Yes	The UUID of the Hex Tile dataset for enrichment.
`sourceColumn`	`string` or `string array`	Yes	The label of the Hex Tile column for enrichment, or an array of labels for multiple columns.
`targetType`	`string`	Yes	Must be either `"H3"` or `"LATLNG"`.
`column`	`string`	Yes for type `H3`	Column in target dataset containing H3 addresses.
`latColumn`	`string`	Yes for type `LATLNG`	Column in target dataset containing latitude values.
`lngColumn`	`string`	Yes for type `LATLNG`	Column in target dataset containing longitude values.
`timeColumn`	`string`	Yes for temporal datasets.	Column in target dataset containing time values in epoch timestamp or ISO-8601 format.
`timeInterval`	`string`	No	Time interval to use for enrichment. The target time interval must be available in the Hex Tile dataset. Accepted methods: `YEAR`, `MONTH`, `DAY`, and `HOUR`. Defaults to the finest available interval.
`input`	`string array`	Yes	Array containing a single object describing the target dataset, in the form `{"type": "dataset", "uuid": <uuid>}`

Python

`enrich` function

Positional Arguments

Argument	Type	Required	Description
`dataset`	`string`	Yes	Pandas `DataFrame`, `Dataset` record, or UUID of the dataset to hextile.
`source_id`	`string`	Yes	UUID of the Hex Tile dataset to use for enrichment.
`source_column`	`string`	Yes	Label of the Hex Tile column to use for enrichment.

Keyword Arguments

Argument	Type	Required	Description
`h3_column`	`string`	Yes for H3 data.	Column in target dataset with H3 addresses.
`lat_column`	`string`	Yes for lat/lng data.	Column in target dataset with latitude values.
`lng_column`	`string`	Yes for lat/lng data.	Column in target dataset with longitude values.
`time_column`	`string`	Yes for temporal data.	Column in target dataset with time values in epoch timestamp or ISO-8601 format.

Response

Upon completion, you will receive the enriched dataset in CSV, JSON, or Arrow format depending on the Accept header.

Examples

data_sdk.enrich(
    dataset="my-dataset-uuid",
    source_id="my-hex-tile-uuid",
    source_column="some_value",
    lat_column="lat",
    lng_column="lng",
    time_column="date",
)

curl -X POST https://data-api.foursquare.com/internal/v1/datasets/hextile \
-H 'Authorization: Bearer <token>' \
-H 'Content-Type: application/json' \
-H 'Accept: text/csv' \
--data-raw '{
    "type": "enrich",
    "input": [
        {
            "type": "dataset",
            "uuid": "my-target-uuid"
        }
    ],
    "sourceId": "my-hex-tile-uuid",
    "sourceColumn": "some_value",
    "timeColumn": "date",
    "targetType": "LATLNG",
    "latColumn": "lat",
    "lngColumn": "lng"
}'

extract

You may specify an area of Hex Tiles (represented by a GeoJSON geometry) to extract. Returns a geopandas H3 dataframe.

# extract hex tiles
  data_sdk.extract(
   self,
    source_id: Union[str, UUID],
    geojson: Dict,
    *,
    source_column: Optional[Union[str, List[str]]] = None,
    res: Optional[int] = None,
    h3_column: Optional[str] = None,
    time_column: Optional[str] = None,
    time_interval: Optional[Union[Dict, TimeInterval]] = None,
) -> pd.DataFrame:

POST https://data-api.foursquare.com/internal/v1/query HTTP/1.1
    Note: Extracting Hextiles makes use of Studio's Query API.
    Please contact us if you wish to use the Query API to extract Hex Tiles.

Python

Use the tile_extractfunction to extract a region of Hex Tiles from a Hex Tile dataset.

Positional Arguments

Argument	Type	Description
`source_id`	`string`	Required. `Dataset` UUID of the dataset to convert to Hex Tile.
`geojson`	`dict`	Required. A `geojson` geometry of the area to extract.

Keyword Arguments

Argument	Type	Description
`source_column`	`string`	Column in Hex Tile dataset.
`res`	`int`	H3 resolution of data to extract.
`h3_column`	`string`	Name of the output column containing H3 indexes. Default: `h3_<res>`
`time_column`	`string`	Name of the output column containing time indexes. Default: `date`
`time_interval`	`string array`	Time interval to extract. Accepted intervals: `"YEAR"`, `"MONTH"`, `"DAY"`, `"HOUR"`, `"MINUTE"`, `"SECOND"`.

Response

Upon completion, you will receive a response containing a Pandas dataframe with the extracted dataset.

# Extract Hex Tiles within a specified GeoJSON geometry
extracted_dataset = data_sdk.extract(
    source_id="<UUID>",
    geojson="
    {
      "type": "Feature",
      "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [0, 0],
            [0, 10],
            [10, 0],
            [0, 0]
          ]
        ]
      }
    };,
    source_column="hextile_column",
    res=8,
    h3_column = "h3_8",
    time_column = "year",
    time_interval = "YEAR"
)

Map Functions

create_map

Body

Response

Example

copy_map

Method

Positional Parameters

Keyword Parameters

Response

Examples

replace_dataset

Method

Positional Parameters

Keyword Parameters

Response

Examples

get_map_by_id

Parameters

Response

Example

update_map

Parameters

Body

Response

Example

delete_map

Parameters

Response

Example

list_maps

Named Parameters

Response

Example

Data Functions

upload_file

Parameters

HTTP API

Headers

Body

Python

upload_file

Positional Arguments

Keyword Arguments

upload_dataframe

Positional Arguments

Response

Example

get_dataset_by_id

Parameters

Response

Example

download_dataset

Parameters

download_dataset

download_dataframe

Response

download_dataset

download_dataframe

Example

delete_dataset

❗️Warning

Parameters

Response

Example

list_datasets

Named Parameters

Response

Example

Hex Tile Functions

create_hextile

Python

Imports for Optional Parameters

create_hextile function

Positional Arguments

Keyword Arguments

HTTP API

Headers

Body

Response

`upload_file`

`upload_dataframe`

`download_dataset`

`download_dataframe`

`download_dataset`

`download_dataframe`

❗️
Warning

`create_hextile` function

`enrich` function