Configuring OKV automation using REST APIs

 This post will go through the process of creating a few simple scripts to automate OKV installation using the REST API capability of OKV.

NOTE: This information was provided by the product manager Peter Wahl who has a great blog on the features of OKV and some great "Ask Tom" sessions on OKV (and other security pieces),you can watch and learn more.

Step #1 Configure RESTful Services and download client tool

First you need to configure the OKV server for RESTful Services. The instructions can be found here. This is done by navigating to the System tab and clicking on RESTful Services.

It is recommended that you only enable the RESTful services when you are actively onboarding new endpoints.

This bring up the window below.

 There are three things you want to do from this window.

1. Click on the "Enable" box to enable RESTful services
2. Download the okvrestcliepackage.zip which are the client utilities.
3. Save this setting to enable RESTful services.

Now that we have this file, we need to download it our client and start creating the scripts to automate this process.

I downloaded the zip file to my DB host to configure it. I unzipped it in /home/oracle/okv/rest

NOTE: you can also download it directly from the OKV hosts by using any of the commands below.

wget --no-check-certificate https://ip_address:5695/okvrestclipackage.zip curl -k https://ip_address:5695/okvrestclipackage.zip -o okvrestclipackage.zip curl -O -k https://ip_address:5695/okvrestclipackage.zip

Step #2 Create a user for the restapi steps

I create a new user in OKV called "restapi" and you can see the permissions below.  Before using the new user, you must login and change the password because the password is expired by default when creating a new user.

Step #3 unzip and configure the client tool 

I unzipped the client tool into my home directory on a DB server so I can put together the automation scripts. In my case I unzipped it into /home/oracle/okv/rest. This creates 3 sub directories. I am going to format the output using this command.

find . | sed -e "s/[^-][^\/]*\// |/g" -e "s/|\([^ ]\)/|-\1/"

Below is what the output looks like

.
 |-lib
 | |-okvrestcli.jar
 |-bin
 | |-okv.bat
 | |-okv
 |-conf
 | |-okvrestcli.ini
 | |-okvrestcli_logging.properties

Step #4 - Set the environment for the CLI

In order to configure OKV, I am going to need some variables set in my environment. I can do this manually, but in my case I decided to create a "setenv.sh" script that will set the variables and add the OKV script to my path to be executed. I also included the ability to pass a parameter (ORACLE_SID) so that you can use the script in a loop across multiple instances on the same host.

 The 3 main variables I will be using are

OKV_RESTCLI_HOME - Location of the scripts that I am going to be installing. If I source the setenv.sh script, it will set the home to this location.

OKV_RESTCLI_CONFIG - Name of the configuration file that contains the rest CLI configuration.

OKV_HOME - Location to install OKV for the current instance.  This location is $ORACLE_BASE/admin/${DB_UNIQUE_NAME}/okv_home.  This follows the standard for ExaCC.

NOTE: If this is a NEW database

If you want to use these steps to configure OKV on a new database, you need to perform the following steps prior to executing "/ ./setenv.sh".

1. Add the new $ORACLE_SIDE for the host in the "/etc/oratab" file.
2. Create the directory "mkdir $ORACLE_BASE/admin/{DB Unique Name}"
3. Create the directory "mkdir -p $ORACLE_BASE/diag/rdbms/${DB Unique Name}/${ORACLE_SID}
4. Use ". oraenv" to set the environment to this $ORACLE_SID

###Ensure that the setenv file is sourced from the current directory to properly set variables if [ "${BASH_SOURCE[0]}" != "${0}" ] then echo " " else echo " " echo "the setenv.sh file must be sourced with \". ./setenv.sh \"" return fi FILE=`pwd`/setenv.sh if [ -f "$FILE" ] then echo " " else echo " " echo "source this from RESTful install directory" return fi ############################################################################################# # # Check to see if a parameter is passed Or ORACLE_SID is set. Otherwise exit # ############################################################################################# if [ $# -eq 0 ] then echo " " if [ -z ${ORACLE_SID} ] then echo "No \$ORACLE_SID set and nothing passed. Exiting " echo " " return fi echo "No \$ORACLE_SID provided. Trying current \$ORACLE_SID value of : $ORACLE_SID" echo " " else echo "\$ORACLE_SID provided. Setting \$ORACLE_SID to : $1" export ORACLE_SID=$1 echo " " fi ############################################################################################# # # make sure SID is in the /etc/oratab so we can determine the # ############################################################################################# sid_found='false' for sids in `cat /etc/oratab | cut -d ":" -f 1` do if [ "$sids" = "${ORACLE_SID}" ] then sid_found='true' fi done if [ ${sid_found} = "false" ] then echo "\$ORACLE_SID : ${ORACLE_SID} not found in /etc/oratab file. Exiting" return fi ############################################################################################# # # make sure SID has exists under $ORACLE_BASE/admin to install software # ############################################################################################# db_dir=`find ${ORACLE_BASE}/diag/rdbms -maxdepth 2 -name ${ORACLE_SID}` delimeter_cut=`echo $db_dir | sed -e 's/\(.\)/\1\n/g' | grep \/ | wc -l` DB_UNIQUE_NAME=`echo $db_dir | cut -d "/" -f $delimeter_cut ` if [ -z ${DB_UNIQUE_NAME} ] then echo $db_dir echo "Invalid \$ORACLE_SID. Exiting script" echo " " echo " " return fi ############################################################################################# # # set the environment # ############################################################################################# export ORAENV_ASK=NO . oraenv export ORAENV_ASK=YES ############################################################################################# # # set the variables # ############################################################################################# echo "create environment variables OKV_HOME, OKV_RESTCLI_HOME and OKC_RESTCLI_CONFIG " ## set variables export OKV_HOME=$ORACLE_BASE/admin/$DB_UNIQUE_NAME/okv_home export OKV_RESTCLI_HOME=`pwd` export OKV_RESTCLI_BIN=$OKV_RESTCLI_HOME/bin export OKV_RESTCLI_CONFIG=$OKV_RESTCLI_HOME/conf/okvrestcli.ini ##add the OKV script to the path echo " " echo "\$OKV_HOME : $OKV_HOME " echo "\$OKV_RESTCLI_HOME : $OKV_RESTCLI_HOME " echo "\$OKV_RESTCLI_CONFIG : $OKV_RESTCLI_CONFIG " echo " " echo "Adding \$OKV_RESTCLI_BIN to the \$PATH " echo " " if [[ "$PATH" =~ (^|:)"${OKV_RESTCLI_BIN}"(:|$) ]] then echo " " else PATH=${OKV_RESTCLI_BIN}:$PATH fi string=`which dcli` echo $string if [[ $string == *"no dcli in"* ]] then export RAC_DATABASE="N" else export RAC_DATABASE="Y" fi ## Set JAVA_HOME to the location of Java if it isn't set already if [ -z ${JAVA_HOME+x} ] then export JAVA_HOME=$ORACLE_HOME/jdk echo "Setting \$JAVA_HOME to \$ORACLE_HOME/jdk : $JAVA_HOME" else echo "\$JAVA_HOME already set to : $JAVA_HOME" fi echo " "

Step #5 - Set initialization parameters in okvrestcli.ini file

Next, I am going to configure the initialization parameters. These are found in the okvrestcli.ini file.

You can see that the file contains a "[Default]" profile and a few other example profiles. We will start with the default profile. In this we are going to set a few of the properties.

LOG_PROPERTY - Location of the logging properties. Default location is ./conf directory.

SERVER - IP address (or DNS) of one or more OKV hosts 

 OKV_CLIENT_CONFIG - location of the config file. Default location is ./conf directory

USER - OKV user that has authority to administer endpoints an wallets. In this case it will be the restapi user that I created.

PASSWORD - Password for the user, or location of wallet containing the password. I am NOT going to use this as I am going to use a wallet file.

 CLIENT_WALLET - I am going to use a wallet to store the password, and this is the location of the wallet file. I will be creating the autologin wallet later.

 

Below is what my "[Default]" configuration file looks like after my changes which is located at $OKV_RESTCLI_HOME/conf/okvrestcli.ini . I am going to use the environmental variables I set in the setenv.sh script. 

NOTE: I am choosing to store my password in wallet rather than clear text in the .ini file.

          You need to change the server to either the server name or the IP address.

[Default] log_property=$OKV_RESTCLI_HOME/conf/okvrestcli_logging.properties server=10.0.0.150 okv_client_config=$OKV_RESTCLI_HOME/conf/okvclient.ora user=restapi client_wallet=$OKV_RESTCLI_HOME/conf

Step #6 Create the wallet to save the password encrypted

Since I chose to put my password in a wallet, I now need to create that wallet. Using the instructions in the document (linked to at the beginning of this blog), I execute the command from the directory I installed into (/home/oracle/okv/rest)

cd /home/oracle/okv/rest . ./setenv.sh ##### Output from script is below ######### # create environment variables OKV_RESTCLI_HOME and OKC_RESTCLI_CONFIG # # $OKV_RESTCLI_HOME : /home/oracle/okv/rest # $OKV_RESTCLI_CONFIG : /home/oracle/okv/rest/conf/okvrestcli.ini # # Adding $OKV_RESTCLI_BIN to the $PATH</i># okv admin client-wallet add --client-wallet $OKV_RESTCLI_HOME/conf --wallet-user restapi #Password: {my password} #{ # "result" : "Success" #}

Step #7 Create and execute the run-me.sh script

The last step is to create the script that will be executed  on the host to create the provision script.  In my script, I took the default and did some checking. This script will

• Ensure the variable OKV_RESTCLI_HOME is set before it can be executed.
• Determine the DB_UNIQUE_NAME from the $ORACLE_BASE/diag/rdbms/*/$ORACLE_SID directory. Solving for the  * should give us the DB_UNIQUE_NAME
• While executing, it tells you what it believes the DB_UNIQUE_NAME is, and gives you a chance to change it if incorrect.
• It will validate if the endpoint group exists by accessing OKV. If the endpoint group already exists, it does not try to create it again. If it doesn't exist it will create the endpoint group.
• It will validate if the wallet exists by accessing OKV. If the wallet already exists, it does not try to create it again. If it doesn't exist it will create the wallet .
• t will add the endpoint, add this endpoint to the endpoint group, and assign the wallet to this endpoint.
• It will install the client software in $ORACLE_BASE/admin/$DB_UNIQUE_NAME/okv_home/okv

Below is the script I am using.

#!/bin/bash ########################################################################################### # # Make sure the setenv script was executed to set the OKV_RESTCLI_HOME variable # ########################################################################################### if [ -v OKV_RESTCLI_HOME ] then echo " " echo "executing script with \$OKV_RESTCLI_HOME=$OKV_RESTCLI_HOME" echo " " echo " " else echo " " echo "\$OKV_RESTCLI_HOME not set" echo " " echo "source the setenv.sh script in the RESTful Services install directory and try again." echo " " exit 0 fi ########################################################################################### # # Determine the DBNAME Unique Name for this SID and have user validate # ########################################################################################### pushd $ORACLE_BASE/diag/rdbms/*/$ORACLE_SID > /dev/null cd .. export DB_UNIQUE_NAME=`printf '%s\n' "${PWD##*/}"` popd > /dev/null echo "DB Unique Name is identified as $DB_UNIQUE_NAME and ORACLE_SID is set to $ORACLE_SID setting" echo " " echo "****************************************************************************************** " echo " " echo "DB Name is identified as $DB_UNIQUE_NAME --- NOTE THIS MAY NOT be the correct DB_NAME --- " echo " --- NOTE THIS MAY NOT be the correct DB_NAME if there is a primary and standby --- " echo " " echo "****************************************************************************************** " echo " " echo " " echo "Press enter to keep this default [$DB_UNIQUE_NAME], or enter the DB Name" read -p 'DB Name [enter for Default] : ' newdbname if [ -z "$newdbname" ] then export DBNAME=`echo $DB_UNIQUE_NAME` else export DBNAME=`echo $newdbname` fi echo " " echo "Using DB Name : $DBNAME" echo "#!/bin/bash" > $OKV_RESTCLI_HOME/okv-install-${ORACLE_SID}.sh ########################################################################################### # # check to see if wallet already exists # create it if it doesn't exist # ########################################################################################### export wallet_exists=`okv manage-access wallet check-status --wallet ${DBNAME^^} | grep Success | wc -l` if [ $wallet_exists -ne 0 ] then echo " Wallet ${DBNAME^^} already exists skipping wallet creation. assuming this is NOT the first node for the database" else cat >> $OKV_RESTCLI_HOME/okv-install-${ORACLE_SID}.sh << EOF2 okv manage-access wallet create --wallet ${DBNAME^^} --description "wallet for database ${DBNAME^^}" EOF2 fi ########################################################################################### # # check to see if endpoint Group already exists # create endpoint group if it doesn't exist # ########################################################################################### export group_exists=`okv manage-access endpoint-group check-status --endpoint-group ${DBNAME^^} | grep Success | wc -l` if [ $group_exists -ne 0 ] then echo " endpoint group ${DBNAME^^} already exists skipping group creation. assuming this is NOT the first node for the Database" else cat >> $OKV_RESTCLI_HOME/okv-install-$ORACLE_SID.sh << EOF2 okv manage-access endpoint-group create --endpoint-group ${DBNAME^^} --description "endpoint group for database ${DBNAME^^}" EOF2 fi export EP_NAME=`echo ${ORACLE_SID^^}_on_${HOSTNAME/.*} | sed 's/\-/\_/g'` ########################################################################################### # # add commands to create/remove directories # ########################################################################################### cat >> $OKV_RESTCLI_HOME/okv-install-$ORACLE_SID.sh << EOF1 mkdir -pv ${OKV_HOME} mkdir -pv ${OKV_HOME}/okv mkdir -pv ${OKV_HOME}/tde mkdir -pv ${OKV_HOME}/tde_seps EOF1 cat > $OKV_RESTCLI_HOME/okv-remove-$ORACLE_SID.sh << EOF1 #!/bin/bash rm -r ${OKV_HOME} rm -r ${ORACLE_BASE}/okv/$ORACLE_SID EOF1 ########################################################################################### # # commands to provision instance on Node # ########################################################################################### cat >> $OKV_RESTCLI_HOME/okv-install-${ORACLE_SID}.sh << EOF3 okv admin endpoint create --endpoint ${EP_NAME} --description "$HOSTNAME, $(hostname -i)" --type ORACLE_DB --platform LINUX64 okv manage-access endpoint-group add-endpoint --endpoint-group ${DBNAME^^} --endpoint ${EP_NAME} okv manage-access wallet set-default --wallet ${DBNAME^^} --endpoint ${EP_NAME} okv manage-access wallet add-access --wallet ${DBNAME^^} --endpoint-group ${DBNAME^^} --access "RM_MW" expect << _EOF set timeout 120 spawn okv admin endpoint provision --endpoint ${EP_NAME} --location $OKV_HOME/okv --auto-login FALSE expect "Enter Oracle Key Vault endpoint password: " send "change-on-install\r" expect eof _EOF EOF3 chmod +x $OKV_RESTCLI_HOME/okv-install-$ORACLE_SID.sh more $OKV_RESTCLI_HOME/okv-install-$ORACLE_SID.sh

Step #8 Execute okv-install-${ORACLE_SID}.sh to validate that enrollment works

[oracle@pmdexa1dbadm01vm01 okv]$ ./okv-ep.sh
{
  "result" : "Success"
}
{
  "result" : "Success"
}
{
  "result" : "Success"
}
spawn okv admin endpoint provision --endpoint DEMO31_on_pmdexa1dbadm01vm01 --location /u01/app/oracle/admin/demo31/wallet/okv --auto-login FALSE
Enter Oracle Key Vault endpoint password:
{
  "result" : "Success"
}

Once I ran it I removed the files that were created for the install.

Step #9 Zip it all up and place it in a location to be downloaded

Below is the scripts that will be part of the zip file.

.
 |-wallet
 | |-ewallet.p12.lck
 | |-ewallet.p12
 | |-cwallet.sso.lck
 | |-cwallet.sso
 |-conf
 | |-okvrestcli_logging.properties
 | |-okvrestcli.ini
 |-setenv.sh
 |-runme.sh
 |-lib
 | |-okvrestcli.jar
 |-bin
 | |-okv
 | |-okv.bat

Now I am ready to download this zip file to my Database Host and enroll a database.

NOTE: To change the script to work on another OKV all host I only had to make 3 changes.

• Update the okvrestcli.ini file with OKV host IP
• Update the okvrestcli.ini file with the the user
• recreate the wallet file that contains the password for the OKV user

Step #10 Run root.sh if this is the first on a host

When a new endpoint is added, a script is downloaded into the $OKV_HOME/okv/bin/ directory called root.sh.. This script will copy the PKCS library (liborapkcs.so) in central location on the host as root. This file is needed for the database to access OKV. The location of this file is different on different OS systems.

NOTE: If you are already using a key management software on this host for Oracle databases, running root.sh could overwrite the library used.