Realtime Data Export API

Owned by Former user (Deleted)
Last updated: Mar 05, 2018 by Former user (Deleted)
4 min read

Through the Realtime Data Export API, it is possible to get your own data stream in Realtime, outputted to either Google's PubSub and BigQuery, Amazon's Kinesis or over HTTPS. There is a maximum of 3 outputs set as default.

Contents on this page

GET /realtime-data-export

Provides a list of all realtime data exports associated with the authenticated user's account.

Method

GET

Path

/api/realtime-data-export

Parameters

NameTypeRequiredDescription
idnumberNID of an existing realtime data export to retrieve

Example cURL

curl -X GET \
'https://platform.flxone.com/api/realtime-data-export?id=5' \
-H 'Cache-Control: no-cache' \
-H 'X-Auth: es5a88dmlpktb5t6k9eifm6Pqjd0l70otkm1p2dv6g13h3h2thkptomb3k8dqe3ms3Xvorgr72h99uec0n92mf9hki9d23aprtsmds3' \
-H 'X-CSRF: 14f61670139eb434adf7f148bbea1a30'

Output

{
    "response": {
        "status": "OK",
        "realtime_data_export": {
            "id": "5",
            "customer": "11",
            "user": "29",
            "title": "Export pixel 20310",
            "settings": {
                "rules": [
                    {
                        "id": "pixel_id",
                        "comparator": "equals",
                        "value": "20310",
                        "type": "dimension"
                    }
                ],
                "outputs": [
                    {
                        "type": "bigquery",
                        "settings": {
                            "projectId": "123456",
                            "datasetName": "DMP-Realtime",
                            "tablePrefix": "DMP-RT",
                            "jsonKey": "M79zxxas122uy5PE9JR82SGKGvKGuYnHuOy+DP/NtznzKgy3t763Z0ClgmAtKpyqKnS2myTs5Z8XQljC7mWr/8oISKbofqO4wIjezzfQi9c3jnMNxecjFlNq0APYCP5RRPdmn7ObuBWVOyl0v21R5impQjAlxpovjZLFbWfQM9a3lWOZklotp6EuuVCFbVf/Hqra4xI7wd5VUUbltmQ09sQVX2KAe7tOngBF1YX7Hn4GkE1tKY3kQbOaMoHx7bTc52Jba1aNvYtzNArAd15I72JjrXsaSfFnUdzMKnOtUHbX9SJrHl+JlyTX5ikTkyNTAUwZSV3gp/9aUdV9JPALkZLLHXKh4LOKymqgBHI3s8E8rOnAa+RSzIcYQ8M311hXZfqVP87d1dh0Ye10VeCut9rThLwV+rXsKfSd67KgSe1EtcubSgPxiJ4jZL1Jrz1DmeZM9FQL2YkLBZX2niNsdyayC0BP3zOPIMDGLNSwGHXF0ix0/i/Ba0pkLKUmF4v8YYaPpmRagqgDM08N1H+41kP4hccx7SaTtv3SlftGqQYWXBm5hTBfxJB5xkZ+LPwbkOclISQSOatsD5/PqEmNuGnmjwPywDuYoWqvc0tI127p3/Zi8B/TFBaBAiPkK77cdTJM9icW1eZgZ+AcEMABb7jzLo/FtFQEAxMyKqb/64esq3Csdsa3ZrlPcg7GHNlgKGqryu2UT0+RPWj8mcA2uWZOi9fZ1xXKggt2clfk1+RDLfaeMH+hQDby9nzBX0gutgCnuuPtyN+/UgZH2gaZ8ma/y2g0Bco/4R38kv6sdsdXmYT8hwj4WcfDP0rf86C91tvpnAjjzIBFsFcy/VlWap0e26TPxPB05KIJcLpBXXgVP/0913PeiL2aBhctzwWQ/fpCCYwdy4rxyC22CGSKCgahywDHgtCAuVZmeUEgfK7kop4YsefZbc7yNcg63VVxh9NgquYicqj3THGKxrphEW5PQmFq7FfiE2RoatYldBjkjbPoVt7LxMkMUVb4hh2Q9+lUaMrasdsd1yjF9y3vxncVcRjnMgbr6Iasdz2iAxM2KyvqY0CvCtRr6OzhGeK3+XA3BlbOBYeb542qOi4TMTZTpGf6WCJXckGxhQRN3OaNRGEX+9aj5kLWHaHOJGfLO1MsAty3Ra5QgRBq1USYf1tvSsuvQDf7Ie7znnW2wRwYf051parKJmMCA3PLjdLvc4XqXFZHDda2+Z1TWwuj/8nSFvniv5ICZ2JwrgGnqNOfthOTHCcpVseyz+qfjOT+PzrTz0Nir4R0IYu8GyW2qg7Pa3BA5OyBZjM5iBMgxtnn2z820XVUbRCK5tZXx/JlNtHVo1/pvIeuk50OBg+qV1S/gBELRCGpxXUtgIE2fzo5PUmO4t2Lam0LiTo+aDqhyi1WwyAPGth0PW1UzCi4f0psv7qJbDasdfF2dq78gZvqg9VBg2RkH47TZlpiRwokQOfKhv+mLDyXA9wAV16tUw/Um8Io1XpcQIAth2x5HeRUQZlcLOLp2LxWPcq4BbXkuWRzftmCuB9VFUIGYxpdP9TOOome7TWXK2tG9J9YHpNjRTeNvlZ+MUgHlTfrrHqqLNuO7a0rI/CUkiZokq/VViNHWnyU7o1g8645BLluBxosdGTWpUU0gcQuM3Y3zokdSg879ZvBfORSRBjhMPueGeq8+1O/6Rt4MQkk73fl6PCAXw8H0GAfGMgF0gRAsdsd9ixvk1M51rIlOf5dqdJ0yELzSrZHxHxr3SjGLth5MX5r+VfgQlkxMlHuLWMsge3FfjjGwv/kvYKMu8d6A3e2EwXtnfn5QpsaassdsqT7ziYFaog9Kjx4NfzDq5Yd8bjyjMPPWuaTL9sA/kWRELZxvRtNj/SRD7Txb+Xf3EIi91JWAqrolg98p85KfzJxaNCR1FhUzMjf7DG/b5UIKxU00ZnQXTsqJ8rXDy77N/FAhyasL5kL1oRmKH0YVV++gn+tqB9bYhr9mX1nF9p/sWQAX3gKF0+j0WaCHVWuJ/wYcvoInPas0MmBtbpKREuJnP411TfbviVVXf8pFOemAIIpTrV+rI3a4K/DOVeap5g3x9Nt/EztLUKKWmYspSLXJeO6ZXx5HTc5DGOUvObu7PDeGZZQkRbEKYD+fzzvxSKiHoYevAdUMFuPwTCzZcgQ/GnmhSZBFRRRc4XyuVicP4HP/HF1yH3j4kfiBR5B9MJr1emVrkTHr1VwOIXydEhTnBwYBoh9nPscQ1+z9Df40SG5cDcrpqxLmzp9XNebK1XhcFFdeiv/pV0FEAWtiovcXvYoUFl40xOC2/ScRnCkrjYQeOT4iSFD86QZ2bxEE0bURtlH/18AnD+iMiXUxcvmSozEhMPcsgUm7ynQE9n3RyL3RUikLGJmKy71xdmv/f+lioHgxrs6AIx6agwvaAafpZcltYe+QOvYa9KM4x3X2MxXVFGR4jzBuHwU21BH4ohjE58yVgdGGXDgsSWaSRqAJcAERcC8dsdasdms+88D0mVK9Rz9i+/nteIi4lgti0QYYKXYe5wtUy+sdsG0YHSW6Be9a8kD+mcIf4kCW2abmByoxDBJS6yQ0tdEXsajdXv8EEp+SQy0ILC3RP8gbHoYYh/uqBoIoX9gih0rXu0KK8JWoYL9kiVgfIifoivnyHdLBDdkUtwrnCCV9/aMVN7HLr7z47vUn6z0aoaWZJ/2JZC/FhiCfmqIXvI+MhnVlTwN12i8gUqDsfdduWTQUBNOL11/Oa0ErqzdForzepY19RhFLzdJhLqnlUbbyN4vjWbjTD0mjoz6DAykEOGdGKvOCIJH3SAXOhh4M8X6kaOCeljuMEEnzMxIb2EeLSVl/UCEvuSf6L4SFbCFqqmZt00haaaIDfu9zd3op/edw1zywhIbL7yI0kqhDUvMpahs0/2qAuHqjYBagJeD/aJxPjr3yCMkjJ2y7boeyAdxd1D2EsIdtcRXC8gCjKnSL3iELasd3dp/tiWZ2Or49FR12Y+7xUxez=="
                        }
                    }
                ],
                "fields": []
            },
            "deleted": "0",
            "modified": "2017-07-19 07:33:36",
            "created": "2017-07-19 07:33:36"
        }
    }
}

GET /realtime-data-export?id=[id]

Provides information about a specific realtime data export.

DELETE /realtime-data-export?id=[id]

Deletes a specific realtime data export.

POST /realtime-data-export

Description

Create a Realtime Data Export stream

Endpoint

/realtime-data-export

Method

POST 

Parameters

NameTypeRequiredDescription
titlestringY 
settingsstringYJSON string. Possible options for this are explained below. This must be url encoded.

Settings

NameTypeRequiredDescription
outputsarrayYSee details below
rulesarrayYSee details below
fieldsarrayYFor a possible option, check the Column Names in this link: Log Format

Outputs

The output is where the data will be exported to. The possible options per output can be found below.

NameTypeRequiredDescription
typestringY

The type of output. Can be either:

  • https
  • kinesis
  • pubsub
  • bigquery
settingsstringYJSON representation of this output's settings. The content of this JSON can be found below.

Output Settings

Each output has different settings. These can be found below

Kinesis

NameTypeRequiredDescription
accessKeystringYThe AWS Access Key
accessSecretstringY

The AWS Secret

regionstringYThe AWS region
streamNamestringYTo which Kinesis stream the data should be sent.

PubSub

NameTypeRequiredDescription
projectIdstringY 
topicIdstringYThe PubSub topic
jsonKeystringYGoogle JSON key (string) that has permissions to produce to PubSub

HTTPS

NameTypeRequiredDescription
endpointstringYTo which endpoint the data should be outputted.
secretstringYThe secret of this endpoint.

BigQuery

NameTypeRequiredDescription
projectIdstringY 
datasetNamestringY 
tablePrefixstringY 
jsonKeystringYGoogle JSON key (string) that has permissions to produce to BigQuery, create datasets, etc.

 

Please refer to the section authentication for the auth and csrf tokens.

Rules

NameTypeRequiredDescription
idstring 

Can be either:

  • pixel_id
  • geo_country
  • event_type
  • interaction_type
comparatorstring 

Can be either:

  • equals
  • =
  • notequals
  • !=
  • contains
  • doesnotcontains
  • after
  • >
  • half
  • end
  • >=
  • before
  • <
  • <=
  • between
valuestring Comma separated list of acceptable values
typestring 

Can be either:

  • dimension
  • measure

PubSub

NameTypeRequiredDescription
projectIdstringY 
topicIdstringYThe PubSub topic
jsonKeystringY 

HTTPS

NameTypeRequiredDescription
endpointstringYTo which endpoint the data should be outputted.
secretstringYThe secret of this endpoint.

BigQuery

NameTypeRequiredDescription
projectIdstringY 
datasetNamestringY 
tablePrefixstringY 
jsonKeystringY 

Example JSON

{
	"outputs": {
		"type": "bigquery",
		"settings": {
			"projectid": "test-1234",
			"datasetName": "test",
			"tablePrefix": "test",
			"jsonKey": "{\"a\":\"b\"}"
		}
	},
	"rules": [{
		"id": "pixel_id",
		"type": "dimension",
		"comparator": "=",
		"value": "12345"
	}],
	"fields": ["pixel_id", "uuid"]
}

 

Please refer to the section authentication for the auth and csrf tokens.