Hotfolder

Configuration

In addition to starting workflows manually, it is also possible to process files (usually SIPs) automatically. This is done in conjunction with a so-called "hotfolder". A hotfolder is a folder that is under observation: if a SIP (or generally a file) is stored in it, the observer automatically creates an event in feeder for this object. These events can then be processed by respective event handlers, defining which workflow should be started.

The configuration of such hotfolders is done in the file <feeder installation directory>observer\observer\config\doc_observer.yml.

label_of__hotfolder:
  directory_watcher:
    directory: /tmp              # directory to watch
    glob: */**                   # pattern for files to watch
    interval: 5.0                # interval between runs (in seconds)
    stable: 2                    # number of runs until a file is stable (only trigger when copying is finished)
    persist: /tmp/dw_state.yml   # path for persistence file

  webservice:
    uri: localhost:3000          # base uri for feeder
    api_token: abc123            # api token for API

  path:
    parent: false                # submit the parent path of the object { true | false (default) }
    absolute: false              # submit the absolute path of the object { true | false (default) }
    extension: false             # submit the filename with the extension { true | false (default) }

  temporary_folder_path: C:/temp # once the file is stable, move it here and send this path to the workflow execution

Example

The following example sets up a hotfolder at C:/example:

example_hotfolder:                     # this hotfolder is called "example_hotfolder"
  directory_watcher:
    directory: C:/example               # the directory that is being watched
    glob: '*.zip'                       # only ZIP files should be processed
    interval: 5.0                       # the duration between two checks is 5 seconds
    stable: 2                           # a file has to be unchanged for 2 intervals, here 10 seconds
    persist: C:/temp/example_state.yml  # persistence file (does not need to be created manually)

  webservice:
    uri: localhost:3000                 # base uri for the feeder webservice
    api_token: abc123                   # api token for API

  path:
    parent: false                       # these three specifications determine
    absolute: false                     # how the path to the SIP in the hotfolder
    extension: false                    # is to be transmitted to the workflow

  temporary_folder_path: C:/temp/sips   # path for temporary files

If a ZIP file remains unchanged in this folder for 10 seconds, an event is created in feeder with the following payload that can be used in the configuration of an event handler:

{
  "event": {
    "payload": {
      "absolute": false, // whether the value in file_or_folder is an absolute path or not
      "event_type": "File Or Folder Found", // static string of the event type
      "extension": false, // whether the value in file_or_folder should include an eventual file format extension
      "file_or_folder": "example.zip", // the actual name/path of the object found in the hotfolder, extended/truncated based on the parent/absolute/extension paramenters
      "parent": false, // whether the value in file_or_folder should be the parent folder of the actual object found
      "source": "example_hotfolder" // the label of the observer configuration
    }
  }
}