Installing Spark on Windows (pyspark)
Prerequisite: Follow these steps to install Apache Spark on windows machine. Now-a-days Python is used by many applications. So it is...
Apache Spark Interview Questions
This post include Big Data Spark Interview Questions and Answers for experienced and beginners. If you are a beginner don't worry, answers..
Write CSV/JSON data to Elasticsearch using Spark dataframes
Elasticsearch-hadoop connector allows Spark-elasticsearch integration in Scala and Java language. Elasticsearch-hadoop library helps...
Just enough Scala for Spark
In this tutorial you will learn just enough Scala for Spark, it's like a quick guide for Scala basics needed for Spark programming, Scala...
Spark Transformation example in Scala (Part 2)
Main menu: Spark Scala Tutorial In this post I will walk you through groupByKey, reduceByKey, aggregateByKey, sortByKey, join, cartesian,...
Spark Transformations example (Part 1)
Apache Spark transformations like Spark reduceByKey, groupByKey, mapPartitions, map etc are very widely used. Apart from these...
Understanding SparkContext textFile & parallelize method
Main menu: Spark Scala Tutorial In this blog you will learn, How Spark reads text file or any other external dataset. Referencing a...
What is SparkContext (Scala)?
Main menu: Spark Scala Tutorial In this blog you will learn, How to start spark-shell? Understanding Spark-shell. Creating Spark context...
Apache Spark Tutorial Scala: A Beginners Guide to Apache Spark Programming
Learn Apache Spark: Tutorial for Beginners - This Apache Spark tutorial documentation will introduce you to Apache Spark programming..
How to write single CSV file using spark?
Apache Spark by default writes CSV file output in multiple parts-*.CSV, inside a directory. Reason is simple it creates multiple files...
How to convert RDD to Dataframe?
Main menu: Spark Scala Tutorial There are basically three methods by which we can convert a RDD into Dataframe. I am using spark shell to...
Spark RDD, Transformations and Actions example
Main menu: Spark Scala Tutorial In this Apache Spark RDD tutorial you will learn about, Spark RDD with example What is RDD in Spark?...
What's Artificial Intelligence, Machine Learning, Deep Learning, Predictive Analytics, Data Science?
Never thought I will spend so much time to understand these high profile terms. I was very confident that I knew theoretically everything...
Spark read Text file into Dataframe
Main menu: Spark Scala Tutorial In this Spark Scala tutorial you will learn how to read data from a text file & CSV to dataframe. This...
Analyzing Twitter Data - Twitter sentiment analysis using Spark streaming
Analyzing Twitter Data - Twitter sentiment analysis using Spark streaming. Twitter Spark Streaming - We will be analyzing Twitter data...
StreamingContext: Spark streaming word count example Scala
Main menu: Spark Scala Tutorial In this tutorial you will learn, How to stream data in real time using Spark streaming? Spark streaming...
Loading JSON file using Spark (Scala)
Main menu: Spark Scala Tutorial In this Apache Spark Tutorial - We will be loading a simple JSON file. Now-a-days most of the time you...
Hello World Program Example in Scala with Eclipse IDE
Main menu: Spark Scala Tutorial In this Apache Spark Scala tutorial, you will learn how to create, "Hello World" Scala application with...
Scala IDE \.metadata\.log error fix (Mac)
Scala IDE is not compatible with Java SE 9 and higher versions. You might need to downgrade or install Java SE 8 in order to fix the...
Installing Apache Spark and Scala (Mac)
Main menu: Spark Scala Tutorial In this Spark Scala tutorial you will learn, How to install Apache Spark on Mac OS. By the end of this...