Using Dataiku
- I have a date column; some of the fields are empty in this column. Is there a way to "count" if there is a value in the field for that row and place into a new column? Something like below: Operating …
Last answer by TurribeachYes, you can use the Prepare recipe with a formula using the isNonBlank() function.
PS: You don't have a date column, you have a string column. Convert it to a proper date column following these steps:
- Hi! I'm currently testing out the possibilities for leveraging Spark in our ETL pipelines. My usecase is that I have 90% of cases where I start with just raw text files in Azure Blob Storage (usually …Last answer by Turribeach
So is it possible to integrate DSS and Databricks, pushing down compute to Databricks but still reading and writing datasets using DSS' connections? ⇒ Absolutely!
There are some caveats of course mostly around Prepare recipe processors and special coding required to use Python recipes with pushdown into Databricks with DBConnect. But works very well I would say. So all you need is to create the Databricks JDBC connection in Dataiku and you can start tapping into those huge datasets. You don't even need to install the Databricks JDBC driver as Dataiku already privides that.