Syncing Data

Most list endpoints support modified_after and created_after filters, making it easy to build incremental sync pipelines that only fetch what's changed.

Incremental Sync Pattern

1. On the first sync, fetch all records (paginate through the full dataset)
2. Store the timestamp of when you started the sync
3. On subsequent syncs, pass ?modified_after=LAST_SYNC_TIMESTAMP to only fetch records updated since your last run
4. Paginate through results until next_cursor is null

Code
 
# Initial sync
curl "https://api.example.com/v1/items?limit=100" -H "x-api-key: KEY"

# Subsequent syncs — only records changed since last run
curl "https://api.example.com/v1/items?limit=100&modified_after=2025-01-15T00:00:00Z" \
  -H "x-api-key: KEY"

Supported Filters

Recommendations

• Sync frequency: Poll every 5–15 minutes for most use cases. Use webhooks (coming soon) for real-time needs.
• Overlap your timestamps: Use a small overlap (e.g., subtract 1 minute from your last sync time) to avoid missing records changed during the previous sync.
• Handle pagination fully: Always paginate through all pages before considering a sync complete.
• Use expand sparingly during sync: Expansions increase response size. Fetch expanded data only for records you need to process in detail.