Example of Azure Data Factory Connection

 Example of Azure Data Factory Connection                   

Scenario: 

Client A plans to send day-to-day order history through SFTP to the internal Reporting Team which will store in the Azure storage account.

Features: 

Azure Storage account, Azure Data Factory

Key Concept:

Azure Data Factory Pipelines and activities: pipeline is a logical grouping of activities that together perform a task. For example, a pipeline could contain a set of activities that ingest and clean log data, and then kick off a mapping data flow to analyze the log data. The pipeline allows you to manage the activities as a set instead of each one individually. You deploy and schedule the pipeline instead of the activities independently.

A picture containing timeline Description automatically generated

Azure Blob storage: Storage solution for the cloud. Blob storage is optimized for storing massive amounts of unstructured data. Unstructured data is data that doesn't adhere to a particular data model or definition, such as text or binary data.


● Connecting Azure Data factory requires whitelisting/enabling all related ports on the firewall.

Workaround & Configuration

Creating Container and Brob storage

Graphical user interface, text, application Description automatically generated

Setting up Data factory 

Graphical user interface, text, application, email Description automatically generated


Graphical user interface, application, Teams Description automatically generated

Graphical user interface, application Description automatically generated

Graphical user interface, text, application, email Description automatically generated

By manually triggering the pipeline, you will be able to verify the data transaction. 


Comments

Post a Comment

Popular posts from this blog

To Be A CyberMan: Installing PfSense on a WatchGuard Firebox

Image
OX01 Preface My company recently upgraded its firewall to FortiGate Firewall.  They eliminated 3 of the licenses expired WatchGuard Firebox Firewalls. The WatchGuard FireBox models are XTM5 Series, M400, and M470. I was wondering if I can install the opensource Firewall application such as PfSense on FireBox to use as Testing Lab Environment. 0X02 Information      XTM5 Series comes with a 1GB CF card, free space is not enough for the lastest Pfsense,  need to add additional Hard Disk as PfSense system booting drive.     M400 comes with a 4 GB CF card, can use CF card as a PfSense system booting drive.     M470 comes with mSTAT SSD.  can SSD as a PfSense system booting drive. 0X03 Installing PfSense on M400 1. Preparation:           WatchGuard Firebox M400 Appliance          A USB Flash Drive          A CF Card USB reader          A console cable (not necessary) 2. download Pfsense       https://www.pfsense.org/download 3. Download Rufus https://rufus.ie/ 4. Use Rufus to create a Pfsens
Read more

How I passed the CSX Fundamentals within one month

Image
  How I passed the CSX Fundamentals within one month Hi Guys, My name is Trevor Shi. I just passed the CSX within one month. I would like to share some experience maybe it will help you guys to prepare the exam. One of the funny things was, I used one month to learn it. But I spent the other one month to book the exam. I re-booked 5 times of exam as ISACA PSI online exam application has lots of issues.  I met every situation you can imagine. The first-time ISACA website down, the second-time they canceled my booked without excuse, the third-time application couldn't open, and the Fourth time was my fault, I just missed the exam. Finally, I got this certificate. We had a small group named CyberRookie, It just like an unorganized cyber knowledge self-studying group, we are trying to be more organized. So t he purpose of this article is as a guideline of the CyberRookie Project CSX certification learning method . A. The Cybersecurity Fundamentals Certificate
Read more

To be A CyberMan: The Exchange Mobile Access Rules Troubleshooting - not able to see device on Quarantined Devices list

Image
  To be A CyberMan: The Exchange Mobile Access Rules Troubleshooting 0X00: Preview Knowlege The company deceived only allows certain employees can have access to their email through mobile.  we created the Device Access Rules on the Exchange admin center. We put all Andriod and iPhone Device into Quarantined that we can decide to block or allow.  So when mobile devices try to connect to email, the user account will go to the Quarantined Devices list on the Exchange admin center, the IT department will grant access to certain employees. 0X01: Issue Some employees were not able to log into Mobile email, got the notification from exchange says: ------------------- Your device is temporarily blocked from synchronizing using Exchange ActiveSync until your administrator grants it access. Your request is currently pending. Thank You,  IT Department Your device is temporarily blocked from accessing content via Exchange ActiveSync because the device has been quarantined. You don't need to
Read more