Default Null in DataStage

In DataStage, the way to default null in transformer stage for Server job and Parallel job are different.

In transformer stage in Server job, @NULL is used.
In transformer stage in Parallel job, SetNull() is used.

DataStage job aborted due to ulimits - Need to be able to open at least 16 files; please check your ulimits settings for number of file descriptors

I am running a job on AIX 64-bit with DataStage V8.1 installed. The DataStage job failed with following error messages (refer to screen attached):

Fatal Error: Need to be able to open at least 16 files; please check your ulimits settings for number of file descriptors

Solutions: 

1. Check the value for ulimits in dsenv is set to 102400 or higher. If not, change the value of ulimits and do a restart on DataStage engine. Command to restart DataStage engine as below:
          • cd $DSHOME/
          • . ./dsenv
          • bin/uv admin –stop
          • bin/uv admin -start
2. Login using root and check /etc/security/limits. Look for value of nofiles and nofiles_hard in that file. If the value is lower than 102400 then modify it to 102400 for both of the entries. Add the entries if there are not in the limits file.

To resolve the problem, both of the steps must be applied in dsenv and root level.

References
http://www-01.ibm.com/support/docview.wss?uid=swg21322302
http://www-01.ibm.com/support/docview.wss?uid=swg21385849

Enable BASIC transformer in DataStage

When you are developing a Parallel job in DataStage, we are able to call a server routine in our Parallel job provided that we make use of BASIC transformer.

BASIC transformer is not visible in the Proccesing palette in Parallel job. Hence we would need to bring in the BASIC transformer to our design canvas following the steps stated below:
Go to Repository tree -> Stages Types -> Parallel ->  Processing -> BASIC Transformer.
Drag and drop the BASIC transformer stage to our design canvas.

With BASIC transformer, we are able to perform various server type data transformation in parallel job. For example, calling a server routine, link variables, etcs.

Check Services Running in AIX

In AIX, we can run the following command to verify what are the services that are currently running on the system.
$lssrc -a

Non-Wire protocol drivers VS Wire protocol drivers

Basically there are two type of ODBC drivers being used in DataStage:

1. non-wire protocol drivers - require  database client software to be installed on the DataStage server (the drivers use the API supplied by the database client)
2. wire protocol drivers - do not require database client software (they communicate with the database directly)
   

Technorati Tags: Non-Wire VS Wire protocol drivers