This post will continue the process of automating the enrollment of my RAC database using the OKV REST API, and some automation scripts. the steps to create the scripts are in my previous post.
NOTE: These steps are for ExaCC specific. If you want to learn about configuring OKV with Autonomous Database (ADB) when using ExaCC, the product manager, Peter Wahl has a great blog post on this topic. He also has videos as part of the "Ask Tom" series if you want to learn more about OKV 21c, or just OKV in general.
The first step is to download the zip file I created in the previous post. I downloaded it onto the first DB host in my RAC cluster. I unzipped it into /home/oracle/okv.
Below is what I am starting with.
.
|-lib
| |-okvrestcli.jar
|-bin
|-conf
| |-okvrestcli_logging.properties
| |-okvrestcli.ini
| |-ewallet.p12.lck
| |-ewallet.p12
| |-cwallet.sso.lck
| |-cwallet.sso
| |-okvclient.ora
|-setenv.sh
|-run-me.sh
STEP #1 - Set the environment
First I am going to set my environment to the database instance I want to configure (jckey1), and then I am going to source the environment for my OKV install.
[oracle@exacc1]$ cd /home/oracle/okv
[oracle@exacc1]$ . oraenv
ORACLE_SID = [jckey1] ? jckey1
The Oracle base remains unchanged with value /u02/app/oracle
[oracle@exacc1]$ . ./setenv.sh
create environment variables OKV_RESTCLI_HOME and OKC_RESTCLI_CONFIG
$OKV_RESTCLI_HOME : /home/oracle/okv
$OKV_RESTCLI_CONFIG : /home/oracle/okv/conf/okvrestcli.ini
Adding $OKV_RESTCLI_BIN to the $PATH
STEP #2 - Execute the enrollment creation script
The next step is to execute the run-me.sh that I created in the previous post. This will create the enrollment script. At the end of the output you will see the script it creates (okv-ep.sh).
NOTE: It will default to my DBNAME for the wallet name.
[oracle@exacc1]$ ./run-me.sh
executing script with $OKV_RESTCLI_HOME=/home/oracle/okv
DB Name is identified as jckey and ORACLE_SID is set to jckey1 setting
Press enter to keep this default [jckey], or enter the DB Name
DB Name [enter for Default] :
Using DB Name : jckey
#!/bin/bash
mkdir -pv /u02/app/oracle/admin/jckey/wallet
mkdir -pv /u02/app/oracle/admin/jckey/wallet/okv
okv manage-access wallet create --wallet JCKEY --description "wallet for database JCKEY" --unique FALSE
okv admin endpoint create --endpoint JCKEY1_on_exacc1 --description "exacc11, 10.136.106.36" --type ORACLE_DB --platform L
INUX64 --unique FALSE
okv manage-access wallet set-default --wallet JCKEY --endpoint JCKEY1_on_exacc1
expect << _EOF
set timeout 120
spawn okv admin endpoint provision --endpoint JCKEY1_on_exacc1 --location /u02/app/oracle/admin/jckey/wallet/okv --auto
-login FALSE
expect "Enter Oracle Key Vault endpoint password: "
send "change-on-install\r"
expect eof
_EOF
STEP #2 - Execute the enrollment script
[oracle@exacc1]$ ./okv-ep.sh
{
"result" : "Success"
}
{
"result" : "Success"
}
{
"result" : "Success"
}
spawn okv admin endpoint provision --endpoint JCKEY1_on_exacc1 --location /u02/app/oracle/admin/jckey/wallet/okv --auto-login FALSE
Enter Oracle Key Vault endpoint password:
{
"result" : "Success",
"value" : {
"javaHome" : "/u02/app/oracle/product/19.0.0.0/dbhome_8/jdk"
}
}
STEP #3 - We can verify what the enrollment script did
I am first going to look under $ORACLE_BASE/admin/$DBNAME/wallet where it placed the okv client.
[oracle@exacc1]$ pwd
/u02/app/oracle/admin/jckey/wallet
[oracle@exacc1]$ find . | sed -e "s/[^-][^\/]*\// |/g" -e "s/|\([^ ]\)/|-\1/"
.
|-okv
| |-bin
| | |-okveps.x64
| | |-okvutil
| | |-root.sh
| |-ssl
| | |-ewallet.p12
| |-csdk
| | |-lib
| | | |-liborasdk.so
| |-jlib
| | |-okvutil.jar
| |-conf
| | |-okvclient.ora
| | |-logging.properties
| | |-okvclient.lck
| |-lib
| | |-liborapkcs.so
| |-log
| | |-okvutil.deploy.log
Now I am going to verify in OKV and I can see the wallet got created for my database.
STEP #4 Execute root.sh (only if this is the first install on this host).
I execute the root.sh script in the /bin directory as root.
[root@exacc1]# ./root.sh
Creating directory: /opt/oracle/extapi/64/hsm/oracle/1.0.0/
Copying PKCS library to /opt/oracle/extapi/64/hsm/oracle/1.0.0/
Setting PKCS library file permissions
Installation successful.
STEP #5 - Verify we can contact the OKV server
The next step is to execute the okvutil list command to verify we can contact the OKV host, and that the default wallet is configured.
[oracle@exacc1]$ ./okvutil list
Enter Oracle Key Vault endpoint password:
Unique ID Type Identifier
9E8BD892-D799-44B7-8289-94447E7ACC54 Template Default template for JCKEY1_ON_ECC5C2N1
STEP #6 - change the OKV endpoint password
[oracle@exacc1]$ /u02/app/oracle/admin/jckey/wallet/okv/bin/okvutil changepwd -t wallet -l /u02/app/oracle/admin/jckey/wallet/okv/ssl/
Enter wallet password: change-on-install
Enter new wallet password: {my new password}
Confirm new wallet password: {my new password}
Wallet password changed successfully
STEP #7 Install the client and change the password on all nodes.
I followed the steps above on the other 3 nodes to install the client and change the password.
STEP #8 Upload the keys from the wallet file.
I uploaded the keys from the shared wallet files on ACFS.
[oracle@exacc1]$ /u02/app/oracle/admin/jckey/wallet/okv/bin/okvutil upload -t wallet -l /var/opt/oracle/dbaas_acfs/jckey/wallet_root/tde -v 2 -g JCKEY
okvutil version 21.1.0.0.0
Endpoint type: Oracle Database
Configuration file: /u02/app/oracle/admin/jckey/wallet/okv/conf/okvclient.ora
Server: 10.136.102.243:5696
Standby Servers:
Uploading from /acfs01/dbaas_acfs/jckey/wallet_root/tde
Enter source wallet password:
Enter Oracle Key Vault endpoint password:
ORACLE.SECURITY.DB.ENCRYPTION.Ab8Sv6Ezs08fv9Sy7/zZB8oAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
ORACLE.SECURITY.KM.ENCRYPTION.Ab8Sv6Ezs08fv9Sy7/zZB8oAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
ORACLE.SECURITY.KB.ENCRYPTION.
ORACLE.SECURITY.ID.ENCRYPTION.
ORACLE.SECURITY.KM.ENCRYPTION.ATQdCFHhVk9Yv7er6uZtDf8AAAAAAAAAAAAAAAAAAAAAAAAAAAAA
ORACLE.SECURITY.DB.ENCRYPTION.ATQdCFHhVk9Yv7er6uZtDf8AAAAAAAAAAAAAAAAAAAAAAAAAAAAA
ORACLE.SECURITY.DB.ENCRYPTION.MASTERKEY.BFF45EC14E46013BE053246A880A5564
ORACLE.SECURITY.DB.ENCRYPTION.MASTERKEY
Uploaded 2 TDE keys
Uploaded 0 SEPS entries
Uploaded 0 other secrets
Uploaded 4 opaque objects
Uploading private persona
Uploading certificate request
Uploading trust points
Uploaded 1 private keys
Uploaded 1 certificate requests
Uploaded 0 user certificates
Uploaded 0 trust points
Upload succeeded
STEP #9 Copy current wallet, and add OKV credentials.
Now you copy the current wallet file (from the ACFS location) to the tde directory (new OKV install) next to the OKV install.
In my case since my OKV client is installed in $ORACLE_BASE/admin/jckey/wallet (which will be the WALLET_ROOT), the tde directory will be the file location for wallets.
I am also adding my password credentials to the local wallet.
NOTE: "OKV_PASSWORD" is used to open the wallet. "HSM_PASSWORD" is used to access the OKV server(s).
mkdir /u02/app/oracle/admin/jckey/wallet/tde_seps
mkdir /u02/app/oracle/admin/jckey/wallet/tde
cp /var/opt/oracle/dbaas_acfs/jckey/wallet_root/tde/* /u02/app/oracle/admin/jckey/wallet/tde/.
ADMINISTER KEY MANAGEMENT ADD SECRET 'Welcome1+' FOR CLIENT 'OKV_PASSWORD' TO LOCAL AUTO_LOGIN KEYSTORE '/u02/app/oracle/admin/jckey/wallet/tde_seps';
ADMINISTER KEY MANAGEMENT ADD SECRET 'Welcome1+' FOR CLIENT 'HSM_PASSWORD' TO AUTO_LOGIN KEYSTORE '/u02/app/oracle/admin/jckey/wallet/tde';
STEP # 10 Change the WALLET_ROOT
Since WALLET_ROOT can only be changed with a restart, I am going to shut down all instances in the cluster and perform the next few steps on the first node only.
SQL> alter system set WALLET_ROOT='/u02/app/oracle/admin/jckey/wallet' scope=spfile;
System altered.
SQL> shutdown immediate
startup mount;
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.
SQL>
alter system set tde_configuration='KEYSTORE_CONFIGURATION=OKV|FILE' scope=both;
select b.name pdb_name,wrl_type,
wrl_parameter,
status,wallet_type,
keystore_mode,
fully_backed_up
from v$encryption_wallet a,v$containers b
where a.con_id = b.con_id(+);SQL> SQL> SQL> SQL> SQL> SQL> SQL> 2 3 4 5 6 7
PDB Name Type WRL_PARAMETER Status WALLET_TYPE KEYSTORE Backed Up
---------- ---------- -------------------------------------------------- ------------------------------ -------------------- -------- ----------
CDB$ROOT FILE /u02/app/oracle/admin/jckey/wallet/tde/ OPEN AUTOLOGIN NONE YES
CDB$ROOT OKV OPEN_NO_MASTER_KEY OKV NONE UNDEFINED
PDB$SEED FILE OPEN AUTOLOGIN UNITED YES
PDB$SEED OKV OPEN_NO_MASTER_KEY OKV UNITED UNDEFINED
JCKPDB FILE OPEN AUTOLOGIN UNITED YES
JCKPDB OKV OPEN_NO_MASTER_KEY OKV UNITED UNDEFINED
SQL> shutdown immediate
startup ;
STEP # 11 Combine the local wallet File and OKV.
Next I need to migrate the keys using the local wallet. Note this will rekey the database.
ADMINISTER KEY MANAGEMENT SET ENCRYPTION KEY IDENTIFIED BY "-okv key" MIGRATE USING "-local wallet key-" WITH BACKUP;
STEP # 12 restart the instance and make sure the wallet open.
PDB Name Type WRL_PARAMETER Status WALLET_TYPE KEYSTORE Backed Up
---------- ---------- ------------------------------- ------------------- --------------- --------- ----------
CDB$ROOT FILE /u02/app/oracle/admin/jckey/wallet/tde/ OPEN AUTOLOGIN NONE YES
CDB$ROOT OKV OPEN OKV NONE UNDEFINED
PDB$SEED FILE OPEN AUTOLOGIN UNITED YES
PDB$SEED OKV OPEN OKV UNITED UNDEFINED
JCKPDB FILE OPEN AUTOLOGIN UNITED YES
JCKPDB OKV OPEN OKV UNITED UNDEFINED
STEP # 13 rebuild the local wallet with the password
I deleted the original wallet files from the "tde" and "tde_seps" directories and recreated them using the exact same steps from step #9. The only addition is that I needed to create the wallet first
ADMINISTER KEY MANAGEMENT ADD SECRET 'Welcome1+' FOR CLIENT 'OKV_PASSWORD' TO LOCAL AUTO_LOGIN KEYSTORE '/u02/app/oracle/admin/jckey/wallet/tde_seps';
ADMINISTER KEY MANAGEMENT ADD SECRET 'Welcome1+' FOR CLIENT 'HSM_PASSWORD' TO AUTO_LOGIN KEYSTORE '/u02/app/oracle/admin/jckey/wallet/tde';
I then pushed executed the same commands to create the wallets on all the nodes in the clusters in the same location .
STEP # 14 - Bounce the database.
I bounced the database and made sure the wallet was open on all 4 nodes. Done.
INST_ID PDB Name Type WRL_PARAMETER Status WALLET_TYPE KEYSTORE Backed Up
-------- ---------- ----- ---------------------------------------- ------------------ -------------- -------- ---------
1 CDB$ROOT OKV OPEN OKV NONE UNDEFINED
2 CDB$ROOT OKV OPEN OKV NONE UNDEFINED
3 CDB$ROOT OKV OPEN OKV NONE UNDEFINED
4 CDB$ROOT OKV OPEN OKV NONE UNDEFINED
1 PDB$SEED OKV OPEN OKV UNITED UNDEFINED
2 PDB$SEED OKV OPEN OKV UNITED UNDEFINED
3 PDB$SEED OKV OPEN OKV UNITED UNDEFINED
4 PDB$SEED OKV OPEN OKV UNITED UNDEFINED
1 JCKPDB OKV OPEN OKV UNITED UNDEFINED
2 JCKPDB OKV OPEN OKV UNITED UNDEFINED
3 JCKPDB OKV OPEN OKV UNITED UNDEFINED
4 JCKPDB OKV OPEN OKV UNITED UNDEFINED
1 PDB$SEED FILE OPEN_NO_MASTER_KEY AUTOLOGIN UNITED UNDEFINED
1 CDB$ROOT FILE /u02/app/oracle/admin/jckey/wallet/tde/ OPEN_NO_MASTER_KEY AUTOLOGIN NONE UNDEFINED
2 CDB$ROOT FILE /u02/app/oracle/admin/jckey/wallet/tde/ OPEN_NO_MASTER_KEY AUTOLOGIN NONE UNDEFINED
3 CDB$ROOT FILE /u02/app/oracle/admin/jckey/wallet/tde/ OPEN_NO_MASTER_KEY AUTOLOGIN NONE UNDEFINED
4 CDB$ROOT FILE /u02/app/oracle/admin/jckey/wallet/tde/ OPEN_NO_MASTER_KEY AUTOLOGIN NONE UNDEFINED
1 PDB$SEED FILE OPEN_NO_MASTER_KEY AUTOLOGIN UNITED UNDEFINED
2 PDB$SEED FILE OPEN_NO_MASTER_KEY AUTOLOGIN UNITED UNDEFINED
3 PDB$SEED FILE OPEN_NO_MASTER_KEY AUTOLOGIN UNITED UNDEFINED
4 PDB$SEED FILE OPEN_NO_MASTER_KEY AUTOLOGIN UNITED UNDEFINED
1 JCKPDB FILE OPEN_NO_MASTER_KEY AUTOLOGIN UNITED UNDEFINED
2 JCKPDB FILE OPEN_NO_MASTER_KEY AUTOLOGIN UNITED UNDEFINED
3 JCKPDB FILE OPEN_NO_MASTER_KEY AUTOLOGIN UNITED UNDEFINED
4 JCKPDB FILE OPEN_NO_MASTER_KEY AUTOLOGIN UNITED UNDEFINED
That's all there is to it. I now have my ExaCC database configuring to use OKV as the key store, and autologin into the wallet on all instances !
Hello,
ReplyDeletethanks for valuable content, will this work for oracle exacs db 12.1 ?
Regards
Greg