As a user, you can create your own Integration connection for customizable third-party integrations. You will make this new configuration in /usr/local/lumeta/integrations.
Notes regarding extended web services:
- Can only ingest the first page of third-party data; flexible pagination will be supported in Asset Manager 5.0.
- Only key authentication via HTTP header is supported.
- The authentication server must be in the same domain as the data server.
- Use your own YAML file and place it in /usr/local/lumeta/integrations
- A tile will be added to the UI for each new YAML file. The tile will be added to the end of the list, and all new tiles will be sorted alphabetically.
- Add an optional logo for each connection, but if no logo is specified, the Integration Label (name) will be used. An SVG file with a horizontal aspect ratio is recommended.
- Specify that the managed table should be automatically created. To use this option,
The following are not supported:
- Cookies
- POST/PUT payload data for authentication
- A separate authentication server and data server (not in the same domain)
To add an external web service, complete the following steps:
You can provide your own YAML file or request help to build one from Support.
An example of data needed for Access/Secret Key authentication:
An example of data needed for Customer Key authentication:
An example of data needed for Token authentication:
The "config" section is the vendor/instance of the third-party that you are connecting to. This will contain the authentication data needed for connection.
A "flows" section is a single pathway of data that starts with a request to the third-party server, comes into Asset Manager, and is inserted into the Asset Manager database.
- Copy the YAML file using SCP/SSH to /usr/local/lumeta/integrations.
- After you've added the file, in Asset Manager go to Settings > Integrations.
The new connection is displayed as a new tile. - Click Configure.
- Adjust whether the feed is enabled (active), how often polling occurs, and more.
README file to help understand YAML file creation:
This directory contains YAML configuration files for integrations, of the form:
label: a label to use for this feed type (optional, defaults to initcap of the feedTypeName)
toggleLabel: a label for the "Enabled" toggle (optional, defaults to "Enabled")
logo: a logo image to use for this feed type (optional, defaults to the label, should be placed in a directory named "images" under this directory)
defaultPollIntervalMinutes: default feed polling interval in minutes (optional, 1440 minutes--one day--if not specified)
defaultServername: default server name (optional, blank if not specified, applies only if "requiredInput" includes "servername")
config: defaultPort: a default value for the port configuration (optional, applies only if "requiredInput" includes "port") defaultAuthType: a default value for the authType configuration (optional, applies only if "supportedAuthTypes" contains multiple values) -- NONE, USERNAME_PASSWORD, ACCESSKEY_SECRETKEY, CUSTOMER_KEY supportedAuthTypes: a list of supported auth types (see defaultAuthType above for possible values) authenticationConfig: key/value pairs to configure authentication type: NONE, BASIC, BEARER method: GET (default), POST, etc headerName: the name of the HTTP header to use headerValueTemplate: a template for the header value, including ${username} and ${password} or ${key} authUrl: a URL that will be used to authenticate (optional) httpHeaders: a list of header names and values to add to the request requiredInput: a list of required input items, e.g. servername, port, etc zoneMappingPull: a boolean to indicate if "pull" (devicification) is supported per zone zoneMappingPush: a boolean or object to indicate if "push" (remote asset creation) supports per-zone configuration externalNetworkConfig: an optional string; its presence indicates that remote mapping is supported for "push" and its value is the label for the remote mapping allowList: a list of URLs that must be allowed through the firewall for this feed type to work (informational only) disablePurge: boolean to disable "Purge" globalConfig1: config value that will apply to all flows in this file ...
flows:
feedFlowName: flow name (required) url: a URL that will be used to pull assets (optional) pullMethod: An HTTP method that will be used to pull assets (defaults to GET) pullMediaType: The Accept header used to pull assets (defaults to application/json) fileOrPath: a suffix that will be added to the URL (optional) table: the name of the table that will receive the downloaded data: lumeta.public.tablename (optional) pipeline: a pipeline name that matches the parser spec for the table (optional) ignoreOutermostArray: Ignore the outermost array in a JSON file: for use with the built-in JSON parser pipeline (optional) topLevelFieldName: The name of the top-level field in a JSON file: for use with the built-in JSON parser pipeline (optional) repeatingLevel: The repeating level of an XML file: for use with the built-in XML parser pipeline (optional) createTable: a boolean that specifies if the table should be created if it doesn't exist (optional, defaults to false, only supported if a built-in pipeline is used--see below) overwrite: a boolean that specifies if the contents of the table will be overwritten on each ingestion (optional, defaults to true) netflow: a boolean that specifies if the downloaded data should be used for nfdump filtering in NetFlow (optional, defaults to false) config: configKey: config value configKey2: another value ...
feedFlowName: another flow name ...
Supported built-in pipeline names that must be used with createTable above:
- com.lumeta.parser.pipeline.csv
- com.lumeta.parser.pipeline.json
- com.lumeta.parser.pipeline.xjson
- com.lumeta.parser.pipeline.xml