In this tutorial, you will learn how increasing the number of cycles can be beneficial during a scenario execution.
Being acquainted with the What Are Cycles On Integromat? is a prerequisite for this tutorial.
Okay, in the above tutorial, we were left with the question: What are the benefits of increasing the number of cycles?
Let me explain..
Benefits of Cycles
If the maximum number of results is set to a high number and the scenario is executed in a single cycle, there is a risk of encountering a timeout error, in which case the scenario execution is paused and some items may be left unprocessed. This happens due to the rate limits on third party APIs, which is completely out of Integromat’s control.
By increasing the max number of cycles and reducing the maximum number of results, the risk of encountering an error is reduced significantly, allowing for smoother scenario executions.
Another benefit of doing this is that when a module encounters an error, the number of unprocessed bundles is reduced when the maximum number of results is reduced. If we process 1 result per cycle, there will be zero unprocessed bundles when an error takes place.
Let’s take a look at the following scenario..
The scenario consists of the following modules:
- Google Sheets > Watch rows
- HTTP > Get a file
- Dropbox > Upload a file
Here’s the scenario:
This demonstration shows what happens when an error occurs with a scenario that is executed in 1 cycle and wherein the maximum number of results is set to 5.
The scenario begins with the Google Sheets > Watch rows module that watches for new rows of data, followed by the HTTP > Get a file module which retrieves a file from a specific URL and the Dropbox > Upload a file module which uploads the file from the preceding module to a folder in Dropbox.
As mentioned, the maximum number of results is set to 5 and the max number of cycles is set to 1:
Let’s click Run Once to execute the scenario.
Scenario execution 1
Let’s look at the output of the Google Sheets module. Here we can see that the module outputted 5 bundles corresponding to the 5 rows in the Google Sheet:
If we look at the output of the HTTP module, we can see that an error occurred while processing bundle 3 due to an invalid URL, this means that bundles 3,4 and 5 were not processed even though all 5 bundles were fetched by the trigger module:
If there are no proper Error Handlers in place, the error will need to be manually resolved. To ensure no bundles are left unprocessed, before activating the scenario again, we need to choose where to start processing from and manually select the row where the error occurred.
To learn how to automatically resolve errors, check out the course on Error Handlers.
The second demonstration shows what happens when the same error occurs but this time the max number of cycles is set to 5 and the maximum number of results is set to 1:
As we already know, only scheduled scenarios account for the number of cycles specified. So let’s change the schedule to run the scenario every minute:
Choose where to start and select All:
Here under the Scenario settings, we’ll select the option to Allow storing of Incomplete Executions. This is the folder that stores the unprocessed executions when a module encounters an error:
Let’s save the scenario and activate:
Wait for 1 minute then check the scenario History.
Scenario execution 2
As we can see here, there is a warning status for the last execution which means that an error occurred within a cycle during the scenario execution:
Now let’s look at the details of this execution by clicking here:
On the top-right-hand side, we can see that the execution contains all 5 cycles as intended:
Looking at the output of the trigger module, we can see that as per the maximum number of results specified, 1 row of data was outputted by the trigger module which represents 1 bundle:
Similarly, the HTTP and Dropbox modules outputted 1 bundle each and the total number of operations consumed in this cycle is 3:
Now let’s take a look at the third cycle. Here the trigger module outputted 1 bundle but an error occurred in the HTTP module:
The data was also saved in the queue of Incomplete Executions. You can resolve this error manually by clicking this Resolve button:
Although the error occurred in cycle 3, the 4th and 5th cycles were still executed and the rows or bundles were processed accordingly.
- By processing 1 item at a time and increasing the number of cycles, it was ensured that when a particular bundle of data resulted in an error, the subsequent bundles of data were not left unprocessed.From these 2 demonstrations, it is evident that:
- When the max number of cycles was set to 1, the maximum number of results set to 5 and the error occurred while processing bundle 3, the subsequent bundles 4 and 5 were left unprocessed.
- However, when the max number of cycles was increased to 5 and the maximum number of results was decreased to 1, and once again the error occurred while processing bundle 3, the subsequent bundles 4 and 5 were unaffected and were processed successfully.
- Therefore, if we are working with large amounts of critical data, it is highly recommended to increase the number of cycles and decrease the maximum number of results being fetched by the trigger module.
- It’s important to keep in mind that while this setup consumes more operations, it also reduces the risk of encountering errors or leaving data unprocessed.
Hope you learnt something new!