Wednesday, April 22, 2015

How to use tFilterRow component in Talend


tFilterRow component is used to filter input rows by setting conditions on the selected columns.

This component uses simple condition and a regular expression to filter a list of records.This scenario will output two tables: the first output will list all records where Order_ID is equals to 11248 or Shipper_ID equals to "1" and the second output will list all rejected records. An error message for each rejected record will display in the same table to explain why such records has been rejected.

This is Orders Input

Order_ID,Customer_ID,Employee_ID,Order_Date,Shipper_ID
11248,1,1,41655,3
11248,2,1,41656,1
11248,4,1,41657,4
11249,9,3,41658,1
11249,10,8,41659,1
11250,11,2,41660,1
11250,13,6,41661,3
11250,6,7,41662,2
11251,7,6,41663,2
11251,3,4,41664,4
11251,2,4,41665,2
11252,11,7,41666,1
11252,7,3,41667,2
11252,5,4,41668,1
11253,8,4,41669,4

Create a new job then create metadata of Orders.CSV file drag it from Metadata > FileDelimited > Orders to design work sapce and select tFileIinputDelimited from pop upwindow.

Drop tFilterRow and tLogRow from the Palette on the design workspace.
Connect the Orders File Delimited to the tFilterRow, using a Row > Main link. 
Then, connect the tFilterRow to the tLogRow, using a Row > Filter link and rename it as Filter. 
Then again drop tLogRow from the Palette into the design workspace and rename it as Reject and then again connect the tFilterRow to the reject, using a Row > Reject link.

Open the tFilterRow component and define its properties.



Sync Columns to propogate the metadata.

  • In the Conditions table, fill the filtering parameters.
  • In InputColumn, select Order_ID, Function as Empty,Operator as Equals.
  • Or you can select other options from dropdown list such as greater than,lower based on your requirement .

  • In the Value column, type 11248 to filter .
  • Now if you want to use Use advanced mode then click check box and type in the following regular expression that includes ID to be searched---input_row.Shipper_ID.equals(1)
  • To combine both conditions (simple and advanced), select Or as logical operator for this example.You can also use And operator based on your requirement.
  • In the Basic settings of tLogRow components, select Table (print values in cells of a table) to show result in table format.

Atlast Run the job.


Here you can see that 8 rows are filtered and 7 rows are rejected.
The records which fullfill the criteria of Order_ID =11248 Or Shipper_ID = 1 they are in filtered records and rest of them are in rejected records.Each rejected record has a corresponding error message that explains the reason of rejection.

No comments:

Post a Comment