View By

Categories

 

How to make calls to Twitter APIs using Postman client?

Updated: Nov 9, 2019


In this blog, I am going to invoke Twitter custom APIs with Postman client in order to pull live feeds or you can say tweets from Twitter. Output will be basically JSON text which you can format or change based on your requirement.


Soon I will be writing another post to showcase how you can ingest this data in real time with Kafka and process it using Spark. Or you can directly stream & process the data in real time with Spark streaming. As of now, let's try to connect Twitter API's.


Prerequisites

  • Postman client

  • Twitter developer account



Postman Client Installation


There are basically two ways to install Postman, either you can download the Postman extension for your browser (chrome in my case) or you can simply install native Postman application. I have installed Postman application to write this blog.


Step 1. Google Install Postman and go to the Postman official site to download the application.



Step 2. After opening Postman download link, select your operating system to start Postman download.​​​​​ It's available for all the types of platform - Mac, Linux and Windows. The download link keeps on changing so if the download link doesn't work just Google it as shown above.




Step 3. Once installer is downloaded, run the installer to complete the installation process. It's approximately 250 MB application (for Mac).


Step 4. Sign up.​​



After signing in, you can save your preferences or do it later as shown below.




Step 5. Your workspace will look like below.



Twitter Developer Account


I hope you all have Twitter developers account, if not please create it.


Now, go to Developer Twitter and sign in with your Twitter account. Click on Apps > Create an app at the top right corner of your screen as shown below.



Note: Earlier, developer.twitter.com was known as apps.twitter.com.


Now fill out the form to create an application > specify Name, Description and Website details as shown below. This screen has slightly changed with new Twitter developer interface but overall process is still similar.


If you have any question, please don't forget to write in comment section at the end of this post.


Please provide a proper website name like https://example.com otherwise you will get error while creating the application. Sample has been shown above.


Once you successfully create the app, you will get the below page.




  • ​Make sure access level is set to Read and Write as shown above.

  • Now go to Keys and Access Token tab > click on Create Access Token.


At this point, you will be able to see 4 keys which will used in Postman client.

  1. Consumer Key (API Key)

  2. Consumer Secret (API Secret)

  3. Access Token

  4. Access Token Secret.



New Interface looks like this.



Calling Twitter API with Postman Client

  1. Open Postman application and click on authorization tab.

  2. Select authorization type as OAuth 1.0.

  3. Add authorization data to Request Headers. This is very important step else you will get error.




After setting up authorization type and request header. Fill out the form carefully with 4 keys (just copy-paste) which we generated in Twitter App - Consumer Key (API Key), Consumer Secret (API Secret), Access Token & Access Token Secret.​





Execute it!


Now let's search for tweeter statuses which says snap.

GET some tweets, hit Send button. You will get response as shown below.




GET Examples


Twitter has very nice API documentation on accounts, users, tweets, media, trend, messages, geo, ads etc and there is huge variety of data which you can pull. I am invoking few APIs just for demonstration purpose.


Accounts and users


Lets say you want to search for user name "Elon". You can do it like this,


GET https://api.twitter.com/1.1/users/search.json?q=elon





Now suppose you want to get friend list of Elon Musk, you can do it like this,


GET https://api.twitter.com/1.1/friends/list.json?user_id=44196397


Input user_id is same as id in previous output. You can also change the display => pretty, raw and preview.




Trending Topics


You can pull top 50 trending global topics with id = 1, for example,

GET https://api.twitter.com/1.1/trends/place.json?id=1




POST Examples


You can also POST something like you Tweet in your Twitter web account. For example if you want to Tweet Hello you can do it like this,


POST https://api.twitter.com/1.1/statuses/update.json?status=Hello



You can verify same with your Twitter account, yeah that's me! I rarely use Twitter.




Cursoring


Cursoring is used for pagination when you have large result set. Lets say you want to pull all statuses which says "Elon", it's obvious that there will be good number of tweets and that response can't fit in one page.


To navigate through each page cursoring is needed. For example, lets say you want to pull 5 result per page you can do it like this,


GET https://api.twitter.com/1.1/search/tweets.json?q=Elon&count=5



Now, to navigate to next 5 records you have to use next_results shown in search_metadata section above like this,


GET https://api.twitter.com/1.1/search/tweets.json?max_id=1160404261450244095&q=Elon&count=5&include_entities=1





To get next set of results again use next_results from search_metadata of this result set and so on..


Now, obviously you can't do this manually each time. You need to write loop to get the result set programmatically, for example,


cursor = -1

api_path = "https://api.twitter.com/1.1/endpoint.json?screen_name=targetUser"


do {

url_with_cursor = api_path + "&cursor=" + cursor

response_dictionary = perform_http_get_request_for_url( url_with_cursor )

cursor = response_dictionary[ 'next_cursor' ]

}

while ( cursor != 0 )


In our case next_results is like next_cursor, like a pointer to next page. This might be different for different endpoints like tweets, users and accounts, ads etc. But logic will be same to loop through each result set.


Refer this for complete details.


That's it you have successfully pulled data from Twitter.


#TwitterAPI #Postmaninstallation #Oauth #API #CustomerKey #CustomerSecret #accesstoken #Postman



Next: Analyze Twitter Tweets using Apache Spark



Learn Apache Spark in 7 days, start today!

1. Apache Spark and Scala Installation

1.1 Spark installation on Windows​

1.2 Spark installation on Mac

2. Getting Familiar with Scala IDE

2.1 Hello World with Scala IDE​

3. Spark data structure basics

3.1 Spark RDD Transformations and Actions example

4. Spark Shell

4.1 Starting Spark shell with SparkContext example​

5. Reading data files in Spark

5.1 SparkContext Parallelize and read textFile method

5.2 Loading JSON file using Spark Scala

5.3 Loading TEXT file using Spark Scala

5.4 How to convert RDD to dataframe?

6. Writing data files in Spark

​6.1 How to write single CSV file in Spark

7. Spark streaming

7.1 Word count example Scala

7.2 Analyzing Twitter texts

8. Sample Big Data Architecture with Apache Spark

9. What's Artificial Intelligence, Machine Learning, Deep Learning, Predictive Analytics, Data Science?

10. Spark Interview Questions and Answers


Write your first blog & earn!!

Home   |   Contact Us

©2019 by Data Nebulae