How Data Feeds Work

Guardian’s data feed service allows organizations to submit changes to employees, users, locations, groups, and custom fields in bulk asynchronously. The service is used to submit data related to the creation or update of records. Data files are compiled and sent to Guardian, with each file containing rows of data related to a specific data feed type. The first row in the file lists the field names for the data feed file type to be processed while each subsequent row represents a record. The processing of these rows results in an add (i.e. create) or an edit (i.e. update), or sometimes simply verifies that matching data exists in the Guardian system. Files may be compiled manually, but development of an automated process is recommended.

The Data Feed Service uses the HTTP POST method to receive the file content. Once the server receives the batch of data (i.e. job) it is queued for processing in the order that it is received. Accordingly, files should be submitted to Guardian in the order that they are expected to be processed.

Any errors in file formatting are not reported upon receipt of the file. These errors are reported in the result log that is available after the file is processed. File processing information, including success or error, is accessible within the Guardian application’s administration area and includes detailed information for each record/file row. Results are listed in numbered order. To compare results and/or troubleshoot errors, maintain the original data file and reference the file row number.

Data feed files are processed in the background. A file may take time to complete depending on the data set size and/or number of queued jobs. Once Guardian processes the file an email is sent from the system to the data feed alert e-mail address on record for the organization, notifying the recipient of the processed file.

Processing data feed files typically consists of the following steps:

  1. Create the csv, tsv, or xlsx file or files that represent data associated to specific feed file types
  2. Send the data to Guardian’s file service
  3. When the file has been processed (confirmed via email receipt), access the results for the file in Guardian’s administration interface
  4. Compare the results with the original file to determine which records failed, and take appropriate action.