I. B. Morozov, 07/2009
UNIX Pipe filter for files
This tool describes a pipe filter command that can be used for filtering text file inputs.
A name is assigned to the pipe, by which it can be located by other modules. The pipe name can also be used as a file or dataset name.
For example, assume that we have defined a pipe named "extract_two," designed to extract two fields from each line of its input (for example, by using awk). Then, a file name "file.txt | extract_two" will be interpreted as an instruction to read "file.txt" and filter it through this process. Multiple processes can be combined, for example: "file.txt | extract_two | extract_one".
Another approach to using FPIPE consists in giving its name instead of the file name. For example, if a pipe named "ls" produces a list of files in certain directories, then using "ls" as the file name in READTAB can feed this list into, for example, database tables. As above, multiple filters can be combined, for example: "ls | count_lines".
FPIPE names can also be used directly in some modules for input filtering (such as list FILTER in tool DATASET.
FPIPE is set-up in the Edit phase and takes effect only when another process requests its outputs.
SHELL,UNIX
| *call | FPIPE | NAME | |
NAME |
Name assigned to the pipe |
|
COMMAND |
One line of the filtering command. All lines are concatenated and submitted as a single shell instruction. A single token "%s" may be present in the command, for which the name of the input file is substituted. If no "%s" tokens are present, the pipe process may be used without a file. |
|