Backfilling

Backfilling is the process of scheduling an FXLoader run for a range of dates in the past. It is useful if you add a new currency to your currency set and need historical rates for that currency to be loaded into your instance.

The safest way to conduct a backfill is to change the ‘Next Start Date’ of the run to the date on which you wish to commence your backfill. FXLoader will process the run as if it was that date; and then the following day; and so on until the present day is reached. The process runs every 5 minutes, so can take a long time if the backfill range is more than a few days (and could delay the next scheduled run).

If you don’t want to backfill right up to the present day then the ‘End Date’ field can be completed. (Note: it is important to delete the ‘End Date’ after the backfill is complete, or else the run will not update to run again at the present time.)

It should be noted that whilst backfilling will load rates for new currencies it will also load rates for all the other currencies within the set, and overwrite rates previously obtained. If this is not desired, an alternative method of backfilling is to create a separate run in order to obtain historical rates for the new currency only. This method is described in Section (B) below.

If a more speedy backfill is required, then it can be done using Immediate Runs (see our article on ‘Run Now’). The disadvantages of Immediate Runs are as below:

• The runs do not verify (if the verify function is enabled).

• You need to select and run each date individually.

• For Oracle ERP Cloud, if an Immediate Run is done too soon after a previous one their ESS requests can overlap.

Backfilling using an existing run

1. Make any required changes to the run (e.g. add a currency to the currency set)

2. On the ‘Run Detail’ page change the ‘Next Start Date’ to the date you want to start your backfill

   

3. After selecting the date from the calendar click the ‘Close’ button.

4. If you don’t want the backfill to run right up to the present day, enter a date in the ‘End Date’ field. The backfill will stop once it reaches this date.

   

5. After the backfill has completed, make sure to delete the ‘End Date’.

6. Click Apply Changes.

Backfilling using a bespoke backfill run

1. Choose the run which has the correct stages for your backfill run

2. Create a copy of that run.

3. Rename the copy. For example, PROD Backfill

4. Manipulate the Currency Set or create a new one.

5. If you only need to backfill one currency pair then you can use the existing Currency Set and simply obtain the rate for that pair by selecting them in the From/To Currency fields.

6. If the required currencies are more complicated it is probably easier to create a new Currency Set.

7. Care should be taken if averages are required for backfilled currencies. Runs which calculate averages usually get rates from a specific run, e.g. PROD Daily, so the Average Run will not get rates from a run called ‘PROD Backfill’ for example.

8. Disable any stages which are not required

9. There may be stages within the run you copied which are not required for the backfill job, e.g. triangulate stages.

10. Follow steps 2-4 from section Backfill to start the backfill.

REST Services

If you use REST services to pull rates from FXLoader you should take care when backfilling. One of the parameters in the REST Service URL is a Run Key which is specific to an FXLoader run. So if you create a bespoke backfill run this will have a different Run Key to the run normally used. Thus, using your usual REST service will not include any rates loaded by the backfill run.