The primary function of the iNSight application is to process data files. File processing involves creating, storing and/or retrieving the contents of a file from a recognisable source.
Data Streams are used to manage these files. Data Streams have processes that must be configured to collect and distribute files. For example, a Data Stream can collect a file from a remote system or locate a received file on a local system, before distributing that file.
In some cases, files are not received or collected from a 3rd party using Data Streams as described above. Instead, data can be extracted directly from a 3rd party’s database, which means they don’t have to create the data file themselves. This is known as Jobs Processing. The extracted data is put into a file which is then injected into a Data Stream to be processed as usual.
You can use the Data Unprocessed Screen to monitor each Data Stream and its processes. The Data History Screen allows you to track the history of a Data Stream as it is processed throughout the iNSight system.
Examples of typical file processing scenarios include:
- Connecting to a 3rd party system to collect or distribute files. Backups can also be generated during this process.
- Locating files on the system and loading their contents into a database so the system can discard the physical file (if required) to save space. Once the content of the file has been loaded into a database, reporting, analytics, etc can be performed on the data.
- Extracting data from a database (locally or remotely) and creating a physical file that can be sent to another system down the line/3rd party system, or even be loaded into another database if required.
Click here for more information on File Processing.
Click here for more information on Jobs Processing.