top of page
BlogPageTop

How to pull data from Oracle IDCS (Identity Cloud Services) rest API

Updated: Nov 3, 2018



Oracle IDCS has various rest apis which can be used to pull data and you can utilize it further for data analytics. Lets see how we can pull data using simple shell scripts.

Step1: Create a parameter file "param.txt" which will contain Customer ID, Customer secret and organization url (all in new line). You can leave the environment name as it is. Please note below values are just dummy values to showcase how you need to create param file. Validate using postman if your keys are working properly before running the script.

scripts> pwd

/home/hadoop/scripts

scripts> more param.txt

CID=

61rgrjk5869bjrvrb9999rbre20

CSEC=

01rgt-atbt-4956-9e77-15rjb74756nr64

ORG=

https://idcs-9bbrtj756bjer8gbk753gbvj8f7eh3.identity.oraclecloud.com

ENV=

QUAL


Step2: At the same path create the script for pulling data in JSON format, brief description is given before each step.


#Check if parameter file exists and its parameters

#!/bin/bash

[ -f ./param.txt ] && echo "Parameter file is present" || echo "Parameter file not found!! Create param.txt with CID,CSEC,ORG,ENV details."

ENV=`head -8 ./param.txt | tail -1`

[ -z "$ENV" ] && echo "Environment variable is empty" || echo "Environment variable looks good"

case $ENV in

DEV) echo "Welcome to DEV environment!" ;;

QUAL) echo "Welcome to QUAL environment!"

CID=`head -2 ./param.txt | tail -1`

CSEC=`head -4 ./param.txt | tail -1`

ORG=`head -6 ./param.txt | tail -1`

sleep 1;;

PL) echo "Welcome to ProdLike environment!" ;;

PT) echo "Welcome to ProdTest environment!" ;;

PROD) echo "Welcome to PROD environment!" ;;

*) echo "Invalid environment selection!" ;;

esac

# Create basic_token base64, basic_token generated at https://www.base64encode.org/

basic_token=`echo -n $CID:$CSEC | base64 -w 0`

# Function to regenerate token if token test fails or expires

regenToken()

{

curl -X POST \

"$ORG/oauth2/v1/token" \

-H "Authorization: Basic $basic_token" \

-H "Cache-Control: no-cache" \

-H "Content-Type: application/x-www-form-urlencoded" \

-d "grant_type=client_credentials&scope=urn%3Aopc%3Aidm%3A__myscopes__" | awk -F[":"] '{print$2}' | awk -F[","] '{print$1}' | awk '{print substr($0, 2, length($0) - 2)}' > access_token.tmp

echo "New token is generated.. access_token refreshed!!"

}

# Testing if token is valid, if invalid generate the new token

access_token=`more access_token.tmp`

tokenTest=`curl -X POST "$ORG/oauth2/v1/introspect" -H "Authorization: Basic $basic_token" -H "Cache-Control: no-cache" -H "Content-Type: application/x-www-form-urlencoded" -d token=$access_token | awk -F"," '{print$1}' | awk -F":" '{print$2}' | sed 's/[{}]//g'`

if [ "$tokenTest" = "true" ]; then echo "Token is valid..";

else

echo "Invalid token! Invoking func to pull new token.."

regenToken

access_token=`more access_token.tmp`

fi

# Remove all the previous files. Script can be modified later to pull delta records only

rm -f auditevents.idcs*

# Pull totalResults count

totalResults=`curl -X GET "$ORG/admin/v1/AuditEvents?&count=0" -H "Authorization: Bearer $access_token" -H "Cache-Control: no-cache" | awk -F"\"totalResults\"\:" '{print$2}'

| awk -F"," '{print$1}'`

echo "Total number of qualified records: $totalResults"

sleep 5

# Loop to pull the records with maximum limitation of 1000 items per page

itemsPerPage=1000

startIndex=1

while [ $startIndex -le $totalResults ]

do

echo "startIndex: $startIndex"

curl -X GET \

"$ORG/admin/v1/AuditEvents?&startIndex=$startIndex&count=$itemsPerPage" \

-H "Authorization: Bearer $access_token" \

-H "Cache-Control: no-cache" | awk -F"Resources" '{print$2}' | awk -F"startIndex" '{print$1}' | cut -c 4- | rev | cut -c 4- | rev > auditevents.idcs.json

# Formatting the JSON output and writing into a file with date time

PAT=]},{\"idcsCreatedBy

REP_PAT=]}'\n'{\"idcsCreatedBy

sed "s/$PAT/$REP_PAT/g" auditevents.idcs.json > auditevents.idcs.json_`date +"%Y%m%d_%H%M%S%N"`

startIndex=`expr $startIndex + 1000`

done

# Remove access token temp file

rm -f access_token.tmp

#END

See also -

Thank you!! If you enjoyed this post, I’d be very grateful if you’d help it spread by emailing it to a friend, or sharing it on Google or Facebook. Refer the links below.

Also click on "Subscribe" button on top right corner to stay updated with latest posts. Your opinion matters a lot please comment if you have any suggestion for me.

#howto #PULL #DATA #oracle #idcs #howtopulldatafromoracleidcs #identitycloudservices

Interested in sharing your story?

Disclaimer: Please note that the information provided on this website is for general informational purposes only and should not be taken as legal advice. Dataneb is a platform for individuals to share their personal experiences with visa and immigration processes, and their views and opinions may not necessarily reflect those of the website owners or administrators. While we strive to keep the information up-to-date and accurate, we make no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability, or availability with respect to the website or the information, products, services, or related graphics contained on the website for any purpose. Any reliance you place on such information is therefore strictly at your own risk. We strongly advise that you consult with a qualified immigration attorney or official government agencies for any specific questions or concerns related to your individual situation. We are not responsible for any losses, damages, or legal disputes arising from the use of information provided on this website. By using this website, you acknowledge and agree to the above disclaimer and Google's Terms of Use (https://policies.google.com/terms) and Privacy Policy (https://policies.google.com/privacy).

RECOMMENDED FROM DATANEB

bottom of page