Jump to content

Recommended Posts

Udemy – Data Engineering using Databricks on AWS and Azure 2024-8

Video Tutorial
, , , , ,

Description

Data Engineering course using Databricks on AWS and Azure. Data engineering means processing data according to our needs. We have to build various pipelines like batch pipelines and data flow pipelines as part of data engineering. All roles related to data processing are grouped under the heading of data engineering. Traditionally, they are known as ETL development, data warehouse development, etc.

Databricks is a very popular cloud platform independent data engineering technology stack. They are committed to the Apache Spark project. The Databricks runtime provides Spark with cloud flexibility. With Databricks, you pay for what you use. Over time, they came up with the idea of ​​Lakehouse, offering all the features needed for traditional BI as well as AI and ML. Here are some of the key features of Databricks.

What you will learn in the Data Engineering using Databricks on AWS and Azure course

  • Data engineering using Databricks features
  • Use the Databricks CLI to manage files, data engineering jobs, and clusters for data engineering pipelines
  • Deploying data engineering applications developed using PySpark on work clusters
  • Deploying data engineering applications developed using PySpark using notebooks on work clusters
  • Perform CRUD operations using Delta Lake using Spark SQL for applications or data engineering pipelines
  • Perform CRUD operations using Delta Lake using Pyspark for applications or data engineering pipelines
  • Setting up a development environment to develop data engineering applications using Databricks
  • Building data engineering pipelines using Spark Structured Streaming on Databricks clusters
  • Incremental file processing using Spark Structured Streaming using Databricks Auto Loader cloudFiles
  • Auto Loader cloudFiles file discovery modes overview – directory listing and file notifications
  • Differences between Auto Loader cloudFiles file discovery modes – directory listing and file notifications
  • Differences between traditional Spark Structured Streaming and using Databricks Auto Loader cloudFiles for incremental file processing

This course is suitable for people who

  • Beginner or intermediate data engineers who want to learn Databricks for data engineering
  • Intermediate software engineers who want to explore data engineering using Databricks
  • Data and analytics engineers who want to learn data engineering using Databricks
  • Testers who want to learn Databricks to test data engineering applications built using Databricks

Details of the course Data Engineering using Databricks on AWS and Azure

Course headings

Data Engineering using Databricks on AWS and Azure

Prerequisites of Data Engineering course using Databricks on AWS and Azure

  • Programming experience using Python
  • Data Engineering experience using Spark
  • Ability to write and interpret SQL Queries
  • This course is ideal for experienced data engineers to add Databricks as one of the key skill as part of the profile

Course images

Data Engineering using Databricks on AWS and Azure

Sample video of the course

Installation guide

After Extract, view with your favorite Player.

Subtitle: English

Quality: 1080p

download link

Download part 1 – 2 GB

Download part 2 – 2 GB

Download part 3 – 2 GB

Download part 4 – 2 GB

Download part 5 – 2 GB

Download part 6 – 580 MB

File(s) password: www.downloadly.ir

File size

10.6 GB

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...
IPS Community Footer