Understanding Processing Order

 

image\ebd_ebd478.gif

 

The order that fields are processed in is arranged to provide you with the results that matter the most as quickly as possible. This section outlines the various rules that are used to determine what order fields are processed in.

Priority Processing

Tasks are assigned a priority based on how quickly the results are required. The following rules are used when establishing the priority of task.

1.    Alerts

2.    Signals used as Display Fields

3.    Other Display Fields

4.    Fields Displayed in Charts and Spreadsheets

 

Any fields required as inputs to these fields are given a slightly higher priority than the fields themselves.

:    Example: Inputs to display fields have higher priority than display fields, but lower priority than alerts.

 

Periodicity is also used to determine priority. Data with a faster periodicity is given higher priority than data with a slower periodicity. However, fields included in the rules listed above are always given priority over fields that are not included in those rules.

:    Example: Display fields for 1-minute bar data have higher priority than alerts for 5-minute bars.

:    Example: Display fields for end-of-day data have higher priority than any fields not listed in the rules above, regardless of periodicity.

 

Regardless of priority, a field can only be calculated once its inputs are calculated.

Δ    Note: One exception to this is the desired output values of predictions. After they are trained, predictions can be recalculated before their desired output values are recalculated since the desired output is not used when generating a value.

Intra-field Processing Order

Multiple steps may be required each time a field is recalculated. Each time a field is calculated, the following steps must be taken.

1.    Calculate new value.

2.    If field is a prediction and values in the training range are calculated (or recalculated), update prediction analysis.

Δ    Note: This step requires the desired output to be calculated before it can be completed.

3.    If field is a signal, update the signal analysis.

 

In addition, each time a new prediction is created, the following steps must be taken before it can be calculated.

1.    Train Prediction

2.    Pre-calculate Prediction

3.    Pre-analyze Prediction

4.    Optimize Postprocessing

Processing States

In order to increase the speed with which updates can be performed, the internal "state" of each calculation is kept for the last value calculated for each field. This allows TradingSolutions to resume each calculation where it left off when new values are added to the end of each data series.

 

The following are examples of values stored in states:

·      Values stored in neural network memories

·      Previous values of calculations, such as Exponential Moving Averages

 

Since these saved states are associated only with the last value in each data series, overwriting values currently in the data series will cause many calculations to start over from the beginning. Because of this, if you select to overwrite previous values during an import operation or modify values directly in a spreadsheet, calculations can take significantly longer than when values are only added to the end.

Tasks Appearing To Calculate Out of Order

Due to the intra-field processing order, it is possible that tasks will sometimes appear to calculate in the wrong order, especially when calculating from the beginning of the data series. This is because fields are not removed from the Current Task Window until all of the processing steps listed in the Intra-field Processing Order list are completed.

 

Here is a detailed example of how processing can appear to be out of order.

 

Let’s say you have the following fields:

1.    Optimal Signal

2.    Model #1 of Optimal Signal

3.    Model #2 of Optimal Signal

4.    Committee of Model #1 and Model #2

 

When watching the Current Task Window or the Signal Analysis page, these fields may appear to calculate in the following order after an import with overwrite occurs.

1.    Committee of Model #1 and Model #2

2.    Optimal Signal

3.    Model #1 of Optimal Signal

4.    Model #2 of Optimal Signal

 

The actual order of the processing is as follows:

1.    Model #1 of Optimal Signal – Calculate Value

2.    Model #2 of Optimal Signal – Calculate Value

3.    Committee of Model #1 and Model #2 – Calculate Value

4.    Committee of Model #1 and Model #2 – Analyze Signal

5.    Optimal Signal – Calculate Value

6.    Optimal Signal – Analyze Signal

7.    Model #1 of Optimal Signal – Analyze Prediction

8.    Model #1 of Optimal Signal – Analyze Signal

9.    Model #2 of Optimal Signal – Analyze Prediction

10.  Model #2 of Optimal Signal – Analyze Signal

 

This ordering occurs because the Analyze Prediction step for each model cannot be performed until the Optimal Signal has been updated. Therefore, the tasks for the Model fields will stay in the Current Task Window until all of the steps have been performed. This order also causes the Committee field to report its results in the Signal Analysis page before the other fields have completed.