Using tContextLoad and Implicit Context Loading to Load Contexts in Talend
Recommended by 0 users
Using contexts and variables
Variables represent values which change throughout the execution of a program. A global variable is a system variable which can be accessed by any module or function. It retains its value after the function or program using it, has completed execution. A context variable is a variable which is defined by the user for a particular context.
Depending on the circumstances the Job is being used in, you might want to manage it differently for various execution types, known as contexts (Prod and Test in the example given below). For instance, there might be various testing stages you want to perform and validate before a Job is ready to go live for production use.
A context is characterized by parameters. These parameters are mostly context-sensitive variables which will be added to the list of variables for reuse in the component-specific properties on the Component view through the Ctrl+Space keystrokes.
Talend Studio offers you the possibility to create multiple context data sets. Furthermore, you can either create context data sets on a one-shot basis from the context tab of a Job, or you can centralize the context data sets in the Contexts node of the Repository tree view in order to reuse them in different Jobs.
You can define the values of your context variables when creating them, or load your context parameters dynamically, either explicitly using the tContextLoad component or implicitly using the Implicit Context Load feature, when your Jobs are executed.
This section describes how to create contexts and variables and define context parameter values.
Using tContextLoad to load contexts
tContextLoad modifies dynamically the values of the active context.
tContextLoad can be used to load a context from a flow.
This component performs also two controls. It warns when the parameters defined in the incoming flow are not defined in the context, and the other way around, it also warns when a context value is not initialized in the incoming flow.
But note that this does not block the processing.
A second method for defining context variables in a different environment is to store them in a file or database external to Talend, and then load them as part of a job.
Open the jo_cook_ch06_0020_loadContextData job.
How to achieve it…
The steps for using tContextLoad to load contexts are as follows:
- Open the tFileInputDelimited component and change the delimiter to, so that it matches the format in the file.
- Drag a tContextLoad component onto the canvas and link it to tFileInputDelimited.
- Open tContextLoad, and click on the option Print operations.
- Unclick Disable warnings.
- Run the job.
How it works…
The context file contains name/value pairs in the form of key=value, so setting the delimiter to = means that we can identify each column in the Talend schema.
When we click on the print operations, it causes Talend to print out what is happening with the context.
The tContextLoad does have some additional features that you may wish to use. They are described in the following sections.
It is usually only necessary to set print operations when debugging code, since it isn’t usually good practice to print the values of say passwords in a log file. It is better, if you wish to log the values, to use the tContextDump component.
The tContextLoad component allows different variations of warnings and error conditions to be specified, depending upon your requirements. It is therefore possible to check for context variables being correctly set, prior to a job running, and to fail the job if they are not.
However, you have to take care with this approach, because job-specific context variables can cause errors and warnings if you do not set them in the file, despite the fact that they may not need a value to be set at the start of the job.
Context file location
The downside of this method as shown is that the context file must reside in the same location on every machine.
Using the implicit context loading to load contexts
This method is very similar to the one discussed previously, that the context data is external to Talend, but differs in the fact that the context data is loaded automatically for every job in a project.
Open the jo_cook_ch06_0030_implicitContextLoad job.
How to achieve it…
The steps for using implicit context load to load contexts are as follows:
- Click on File, then Edit project properties.
- From the dialogue box, expand Job Settings and select the option for Implicit context load.
- Check the option for Implicit tContextLoad.
- Check the option From File.
- Set the From File name to
- Set the Field Separator to =.
- Click on the Print operations.
- Click on Ok to finish.
- Run the job, and you will see the context data being loaded and the warnings being produced.
How it works…
The implicit context load functionality reproduces the code in the previous exercise for all jobs in the project.
As with the previous exercise, the context file, the delimiter, and the actions to be taken while loading the contexts are all specified, just that with this method, it is global for every job.
This method suffers from the same shortcomings as the method in the previous exercise, but is easier to maintain and implement, because the values are set centrally.
Remember to set the project properties back to normal, once you have completed this exercise, unless you plan to perform the next one.