Installing Spark on Windows (pyspark)
Prerequisite: Follow these steps to install Apache Spark on windows machine. Now-a-days Python is used by many applications. So it is...
Kafka Consumer Advance (Java example)
Kafka Commits, Kafka Retention, Consumer Configurations & Offsets - Prerequisite Kafka Overview Kafka Producer & Consumer Commits & Offset..
Apache Spark Interview Questions
This post include Big Data Spark Interview Questions and Answers for experienced and beginners. If you are a beginner don't worry, answers..
Apache Avro Schema Example (in Java)
Introduction Avro provides data serialization based on JSON Schema. It is language neutral data serialization system, means a language A...
Kafka Producer and Consumer example (in Java)
In this Kafka pub sub example you will learn, Kafka producer components (producer api, serializer and partition strategy) Kafka producer...
Apache Kafka Overview (Windows)
Apache Kafka is middleware solution for enterprise application. It was initiated by LinkedIn lead by Neha Narkhede and Jun Rao. Initially...
Kafka Producer Example
In this Apache Kafka tutorial you will learn - How to Install Apache Kafka on Mac using homebrew. To install Kafka on linux machine refer ..
Spark Transformation example in Scala (Part 2)
Main menu: Spark Scala Tutorial In this post I will walk you through groupByKey, reduceByKey, aggregateByKey, sortByKey, join, cartesian,...
Apache Spark Tutorial Scala: A Beginners Guide to Apache Spark Programming
Learn Apache Spark: Tutorial for Beginners - This Apache Spark tutorial documentation will introduce you to Apache Spark programming..
How to clear Google cloud professional data engineer certification exam?
In this blog you will learn - How to get Google cloud certification? How much it cost to get Google certified? Best Google certification...
How to write single CSV file using spark?
Apache Spark by default writes CSV file output in multiple parts-*.CSV, inside a directory. Reason is simple it creates multiple files...
How to convert RDD to Dataframe?
Main menu: Spark Scala Tutorial There are basically three methods by which we can convert a RDD into Dataframe. I am using spark shell to...
What's Artificial Intelligence, Machine Learning, Deep Learning, Predictive Analytics, Data Science?
Never thought I will spend so much time to understand these high profile terms. I was very confident that I knew theoretically everything...
Spark read Text file into Dataframe
Main menu: Spark Scala Tutorial In this Spark Scala tutorial you will learn how to read data from a text file & CSV to dataframe. This...
StreamingContext: Spark streaming word count example Scala
Main menu: Spark Scala Tutorial In this tutorial you will learn, How to stream data in real time using Spark streaming? Spark streaming...
Loading JSON file using Spark (Scala)
Main menu: Spark Scala Tutorial In this Apache Spark Tutorial - We will be loading a simple JSON file. Now-a-days most of the time you...
Hello World Program Example in Scala with Eclipse IDE
Main menu: Spark Scala Tutorial In this Apache Spark Scala tutorial, you will learn how to create, "Hello World" Scala application with...
Installing Apache Spark and Scala (Mac)
Main menu: Spark Scala Tutorial In this Spark Scala tutorial you will learn, How to install Apache Spark on Mac OS. By the end of this...
Installing Apache Spark and Scala (Windows)
Main menu: Spark Scala Tutorial In this Spark Scala tutorial you will learn how to download and install, Apache Spark (on Windows) Java...
Apache Kafka and Zookeeper Installation & Sample Pub-Sub Model
There are many technologies available today which provides real time data ingestion (refer my previous blog). Apache Kafka is one of my...