The primary objective of writing this blog is to cover how these operations can be reused in other operation like Data Cleansing , Preprocessing etc.
This is a nice overview of the flexibility of Data Intelligence. While there is nothing wrong with the approach, I would recommend simplifying the graph by using the Read File operator and passing the data to your custom logic. This gives you even more control and makes the graph easier to read/maintain in the future without digging into the custom operator logic. Especially if you have large files on the DI_DATA_LAKE or other file storage, you can split the work into batches or parameterize the reads to make the operator more generic for more use-cases.
Thank you for this great article.
Hello, I can not see the article, it's displaying just an introduction statement. thanks.