documentationAbout Queries and On-demand Acceleration¶
What is a query?¶
Queries send requests for data stored in Observe and allow you to view that data. Essentially any time you view a dataset in Observe, you perform a query. Under the hood, a query consists of multiple stages, with each stage using a separate OPAL script. An OPAL script consists of a sequence of statements where the output of one statement is the input for the next. This could be a single line with one statement or many lines of complex shaping and filtering.
A script contains the following types of elements:
Query - your query contains a single time window shared across all stages, and each stage consists of a set of input elements, such as a block of OPAL script.
Inputs - can be datasets or other stages in the same query. Each stage contains a single output with an OPAL script consisting of OPAL verbs and functions.
Verbs - define the processing to perform with the data in the stage.
A complete OPAL script consists of inputs, verbs, functions, and outputs defining the desired result. When you create a query in a worksheet using the console and click Run, Observe executes all stages in a query on the datasets. In a Dashboard, Observe runs the query in the background.
Understanding On-demand Acceleration¶
Datasets in Observe have an acceleration window. An acceleration window contains the time range for the queries on each dataset to be fast and efficient.
Observe automatically accelerates the dataset for the prior seven(7) days when you edit or create a dataset. Therefore, Observe performs queries in recent time ranges quickly and efficiently.
Observe also automatically extends this acceleration window forward as new data arrives, so queries on any time range after creating a dataset also perform quickly and efficiently.
Observe does not automatically accelerate access to historical data from before dataset creation. As a result, queries on newly created or edited datasets, such as those created a few days ago, using a time range from far in the past, such as this day last year, cannot be satisfied with accelerated data.
When you select a time range with the Select Time Range option, some options may have a yellow icon 
next to them. These are time ranges extending past the datasets’ acceleration window in your query.
If you select a time range with next to it, two things can occur in Observe:
Because you selected a time range beyond the acceleration window of the datasets you want to query, older data from before the beginning of the acceleration window may be missing from the result.
For example, if a queried Dataset was only accelerated 14 days ago, and you select Past 30 days in the Select Time Range, your query results only reflects the past 14 days, even though you requested the past 30 days.
You see a message indicating that your results may only reflect a partial range of the requested time range.
If you can use partial results, you can dismiss the message.
You can accelerate the dataset by selecting Accelerate full range if you need results for the whole time range.
If you perform this action, Observe initiates a background process to accelerate the datasets in your query for the entire time range you selected from the Select Time Range menu.
You can monitor the acceleration progress on your dataset from the Query Overview window. It can take anywhere from a few seconds to a few minutes for the dataset to update.
Note
Acceleration of historical time ranges for datasets consumes Observe Compute Credits (OCC). You can monitor your credit usage by reviewing the Usage Dashboard for credits attributed to backfill activity.
After completing the background process, your next query returns results for the full time range that you requested. Because it used an accelerated dataset, the query becomes much faster and more cost-efficient compared to querying your raw data. Also, because you accelerated the dataset, all other users in your account enjoy the same performance and efficiency benefit when querying this dataset.
Accelerating Datasets¶
When you open a worksheet, dashboard, or dataset, Observe automatically updates the query, and you can see the Query in progress icon 
. Click the Stop icon to end the query process.
After completing the background process, your following query returns results for the requested time range. Because it uses an accelerated dataset, the query becomes much faster and more cost-efficient than querying your raw data. Also, because you accelerated the dataset, all other users in your account enjoy the same performance and efficiency benefit when querying this dataset.
If the query returns results successfully, the icon changes to the Query Overview icon 
. You can refresh the query using the Refresh icon.
Figure 1 - Query results for a dataset
You can change the time range for the query by using the Select Time Range icon. After you select it, you may see some time ranges with a yellow icon next to them.
Figure 2 - Query range for a dataset
These time ranges may include a window of time where the underlying datasets for your query are not accelerated. If you query these ranges, your query may return partial results compared to a query on raw data ingested into Observe.
You may elect to accelerate the unaccelerated time range to obtain complete results.
Figure 3 - Dataset duration beyond the accelerated range
Clicking on the yellow icon allows you to choose to accelerate the range of data.
Figure 4 - Unaccelerated results for a dataset
To accelerate the full range of data, click Accelerate full range.
Figure 5 - Unable to accelerate data for the selected time range
Acceleration may be unavailable for a selected time range because the dataset on demand materialization has reached the upper limit. Contact Observe support if you encounter this error and need assistance to accelerate a more extended time range.