Skip to content Skip to sidebar Skip to footer
Home / Azure Data Factory / Azure Data Lake / Azure Hdinsight / Pyspark / Python

Access Datalake From Azure Datafactory V2 Using On Demand Hd Insight Cluster

Post a Comment
I am trying to execute spark job from on demand HD Insight cluster using Azure datafactory. Documentation indicates clearly that ADF(v2) does not support datalake linked service fo

Solution 1:

Currently, we don't have support for ADLS data store with HDI Spark cluster in ADF v2. We plan to add that in the coming months. Till then, you will have to contiue using the workaround as you mentioned in your post above. Sorry for the inconvenience.

Solution 2:

The Blob storage is used for the scripts and config files that the On Demand cluster will use. In the scripts you write and store in the attached Blob storage they can write from ADLS to SQLDB for example.

Post a Comment for "Access Datalake From Azure Datafactory V2 Using On Demand Hd Insight Cluster"