Google Cloud Pub/Sub¶
Google Cloud Pub/Sub allows you to reliably deliver streaming data from multiple sources within GCP. Observe supports ingesting data through a variant of the HTTP endpoint.
Navigate to the GCP Cloud Pub/Sub console to create a Pub/Sub push subscription. Note that some options are required and others are optional.
Figure 1 - The Google Cloud Pub/Sub page
Go to the Pub/Sub page in the GCP console.
Select Subscriptions in the left pane, and then click CREATE SUBSCRIPTION.
Configure required subscription details
At minimim, you must configure these required options:
Subscription ID and Pub/Sub topic: Enter a Subscription ID and a topic. You may choose an existing topic from the menu, or enter a new one.
The Delivery type Endpoint URL requires a Basic Authentication token as part of the URL. Construct this token using your Observe Customer ID and ingest token.
Create a Basic Authorization token by Base64 encoding a string containing the customer ID and ingest token separated by a colon
Bash on MacOS and Linux:
$ echo -n "12345:my_ingest_key" | base64 MTIzNDU6bXlfaW5nZXN0X2tleQ==
PowerShell on Windows:
> [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("12345:my_ingest_key")) MTIzNDU6bXlfaW5nZXN0X2tleQ==
Enter the Endpoint URL
https://pubsub.collect.observeinc.com/, appending the Base64 token you created in the previous step.
Leave the Enable authentication box unchecked. This option is for authentication between GCP services, not external ones.
Message retention duration: Select your desired retention period. To reduce storage cost, we recommend no more than 24 hours. This also determines how long an outage your subscription can recover from.
Acknowledgement deadline: Choose a deadline time of at least 60 seconds.
Retry policy: Select Retry after exponential backoff delay, with the default minimum of 10 seconds and maximum of 600 seconds.
Configure optional subscription details
You may also configure these optional settings:
Message ordering is not required.
When you are finished, click Create subscription.
Retry on failure¶
Google Cloud Pub/Sub supports exponential backoff for retries. See the Pub/Sub Handling message failures documentation for more information.